Prev Next exp_reverse Headings

@(@\newcommand{\W}[1]{ \; #1 \; } \newcommand{\R}[1]{ {\rm #1} } \newcommand{\B}[1]{ {\bf #1} } \newcommand{\D}[2]{ \frac{\partial #1}{\partial #2} } \newcommand{\DD}[3]{ \frac{\partial^2 #1}{\partial #2 \partial #3} } \newcommand{\Dpow}[2]{ \frac{\partial^{#1}}{\partial {#2}^{#1}} } \newcommand{\dpow}[2]{ \frac{ {\rm d}^{#1}}{{\rm d}\, {#2}^{#1}} }@)@
Exponential Function Reverse Mode Theory
We use the reverse theory standard math function definition for the functions @(@ H @)@ and @(@ G @)@. The zero order forward mode formula for the exponential is @[@ z^{(0)} = F ( x^{(0)} ) @]@ and for @(@ j > 0 @)@, @[@ z^{(j)} = x^{(j)} d^{(0)} + \frac{1}{j} \sum_{k=1}^{j} k x^{(k)} z^{(j-k)} @]@ where @[@ d^{(0)} = \left\{ \begin{array}{ll} 0 & \R{if} \; F(x) = \R{exp}(x) \\ 1 & \R{if} \; F(x) = \R{expm1}(x) \end{array} \right. @]@ For order @(@ j = 0, 1, \ldots @)@ we note that @[@ \begin{array}{rcl} \D{H}{ x^{(j)} } & = & \D{G}{ x^{(j)} } + \D{G}{ z^{(j)} } \D{ z^{(j)} }{ x^{(j)} } \\ & = & \D{G}{ x^{(j)} } + \D{G}{ z^{(j)} } ( d^{(0)} + z^{(0)} ) \end{array} @]@ If @(@ j > 0 @)@, then for @(@ k = 1 , \ldots , j @)@ @[@ \begin{array}{rcl} \D{H}{ x^{(k)} } & = & \D{G}{ x^{(k)} } + \D{G}{ z^{(j)} } \frac{1}{j} k z^{(j-k)} \\ \D{H}{ z^{(j-k)} } & = & \D{G}{ z^{(j-k)} } + \D{G}{ z^{(j)} } \frac{1}{j} k x^{(k)} \end{array} @]@
Input File: omh/appendix/theory/exp_reverse.omh