Purpose
First order reverse mode uses the
operation sequence
,
and zero order forward sweep values,
to compute the first order derivative
of one dependent variable with respect to all the independent variables.
The computations are done in reverse
of the order of the computations in the original algorithm.
Mathematical Form
Suppose that we use the algorithm exp_eps.hpp
to compute
exp_eps(x, epsilon)
with
x
is equal to .5
and
epsilon
is equal to .2.
For this case, the mathematical function for the operation sequence
corresponding to exp_eps is
The corresponding partial derivatives,
and the value of the derivatives, are
epsilon
Since
is an independent variable,
it could included as an argument to all of the
functions below.
The result would be that all the partials with respect to
would be zero and hence we drop it to simplify
the presentation.
f_7
In reverse mode we choose one dependent variable and
compute its derivative with respect to all the independent variables.
For our example, we chose the value returned by exp_eps.hpp
which is
.
We begin with the function
where
is both an argument and the value of the function; i.e.,
All the other partial derivatives of
are zero.
Index 7: f_6
The last operation has index 7,
We define the function
as equal to
except that
is eliminated using
this operation; i.e.
It follows that
All the other partial derivatives of
are zero.
Index 6: f_5
The previous operation has index 6,
We define the function
as equal to
except that
is eliminated using this operation; i.e.,
It follows that
All the other partial derivatives of
are zero.
Index 5: f_4
The previous operation has index 5,
We define the function
as equal to
except that
is eliminated using this operation; i.e.,
Given the information from the forward sweep, we have
and
.
It follows that
Index 4: f_3
The previous operation has index 4,
We define the function
as equal to
except that
is eliminated using this operation; i.e.,
It follows that
Index 3: f_2
The previous operation has index 3,
We define the function
as equal to
except that
is eliminated using this operation; i.e.,
It follows that
Index 2: f_1
The previous operation has index 1,
We define the function
as equal to
except that
is eliminated using this operation; i.e.,
It follows that
Note that
is equal to
,
so the derivative of
exp_eps(x, epsilon)
at
x
equal to .5 and
epsilon
equal .2 is
1.5 in the
x
direction and zero in the
epsilon
direction.
We also note that
forward
forward mode gave the
same result for the partial in the
x
direction.
Verification
The file exp_eps_rev1.cpp
contains a routine
that verifies the values computed above.
It returns true for success and false for failure.
It only tests the partial derivatives of
that might not be equal to the corresponding
partials of
; i.e., the
other partials of
must be equal to the corresponding
partials of
.
Consider the case where
and we first preform a zero order forward mode sweep
for the operation sequence used above (in reverse order).
What are the results of a
first order reverse mode sweep; i.e.,
what are the corresponding values for
for all
such that
.
Create a modified version of
exp_eps_rev1.cpp
that verifies the values you obtained for the previous exercise.
Also create and run a main program that reports the result
of calling the modified version of
exp_eps_rev1.cpp
.