Prev Next Index-> contents reference index search external Up-> CppAD Appendix Theory ForwardTheory CppAD-> Install Introduction AD ADFun preprocessor multi_thread library ipopt_solve Example speed Appendix Appendix-> Faq Theory glossary Bib Bugs WishList whats_new deprecated compare_c License Theory-> ForwardTheory ReverseTheory reverse_identity ForwardTheory-> ExpForward LogForward SqrtForward SinCosForward AtanForward AsinForward AcosForward tan_forward Headings-> Taylor Notation Binary Operators ---..Addition ---..Subtraction ---..Multiplication ---..Division Standard Math Functions ---..Differential Equation ---..Taylor Coefficients Recursion Formula ---..Cases that Apply Recursion Above ---..Special Cases

The Theory of Forward Mode

Taylor Notation
In Taylor notation, each variable corresponds to a function of a single argument which we denote by t (see Section 10.2 of Evaluating Derivatives ). Here and below   X(t) ,   Y(t) , and Z(t) are scalar valued functions and the corresponding p-th order Taylor coefficients row vectors are   x ,   y and   z ; i.e.,   $\begin{array}{lcr} X(t) & = & x^{(0)} + x^{(1)} * t + \cdots + x^{(p)} * t^p + o( t^p ) \\ Y(t) & = & y^{(0)} + y^{(1)} * t + \cdots + y^{(p)} * t^p + o( t^p ) \\ Z(t) & = & z^{(0)} + z^{(1)} * t + \cdots + z^{(p)} * t^p + o( t^p ) \end{array}$  For the purposes of this section, we are given   x and   y and need to determine   z .

Binary Operators

  $\begin{array}{rcl} Z(t) & = & X(t) + Y(t) \\ \sum_{j=0}^p z^{(j)} * t^j & = & \sum_{j=0}^p x^{(j)} * t^j + \sum_{j=0}^p y^{(j)} * t^j + o( t^p ) \\ z^{(j)} & = & x^{(j)} + y^{(j)} \end{array}$ 

Subtraction
  $\begin{array}{rcl} Z(t) & = & X(t) - Y(t) \\ \sum_{j=0}^p z^{(j)} * t^j & = & \sum_{j=0}^p x^{(j)} * t^j - \sum_{j=0}^p y^{(j)} * t^j + o( t^p ) \\ z^{(j)} & = & x^{(j)} - y^{(j)} \end{array}$ 

Multiplication
  $\begin{array}{rcl} Z(t) & = & X(t) * Y(t) \\ \sum_{j=0}^p z^{(j)} * t^j & = & \left( \sum_{j=0}^p x^{(j)} * t^j \right) * \left( \sum_{j=0}^p y^{(j)} * t^j \right) + o( t^p ) \\ z^{(j)} & = & \sum_{k=0}^j x^{(j-k)} * y^{(k)} \end{array}$ 

Division
  $\begin{array}{rcl} Z(t) & = & X(t) / Y(t) \\ x & = & z * y \\ \sum_{j=0}^p x^{(j)} * t^j & = & \left( \sum_{j=0}^p z^{(j)} * t^j \right) * \left( \sum_{j=0}^p y^{(j)} * t^j \right) + o( t^p ) \\ x^{(j)} & = & \sum_{k=0}^j z^{(j-k)} y^{(k)} \\ z^{(j)} & = & \frac{1}{y^{(0)}} \left( x^{(j)} - \sum_{k=1}^j z^{(j-k)} y^{(k)} \right) \end{array}$ 

Standard Math Functions
Suppose that   F  is a standard math function and   $Z(t) = F[ X(t) ]$

Differential Equation
All of the standard math functions satisfy a differential equation of the form   $B(u) * F^{(1)} (u) - A(u) * F (u) = D(u)$  We use   a ,   b and   d to denote the p-th order Taylor coefficient row vectors for   A [ X (t) ]  ,   B [ X (t) ] and   D [ X (t) ]  respectively. We assume that these coefficients are known functions of   x , the p-th order Taylor coefficients for   X(t) .

Taylor Coefficients Recursion Formula
Our problem here is to express   z , the p-th order Taylor coefficient row vector for   Z(t) , in terms of these other known coefficients. It follows from the formulas above that   $\begin{array}{rcl} Z^{(1)} (t) & = & F^{(1)} [ X(t) ] * X^{(1)} (t) \\ B[ X(t) ] * Z^{(1)} (t) & = & \{ D[ X(t) ] + A[ X(t) ] * Z(t) \} * X^{(1)} (t) \\ B[ X(t) ] * Z^{(1)} (t) & = & E(t) * X^{(1)} (t) \end{array}$  where we define   $E(t) = D[X(t)] + A[X(t)] * Z(t)$  We can compute the value of   z^{(0)} using the formula   $z^{(0)} = F ( x^{(0)} )$ Suppose by induction (on   j ) that we are given the Taylor coefficients of   E(t) up to order   j-1 ; i.e.,   e^{(k)} for   k = 0 , \ldots , j-1 and the coefficients   z^{(k)} for   k = 0 , \ldots , j . We can compute   e^{(j)} using the formula   $e^{(j)} = d^{(j)} + \sum_{k=0}^j a^{(j-k)} * z^{(k)}$  We need to complete the induction by finding formulas for   z^{(j+1)} . It follows for the formula for the multiplication operator that    $\begin{array}{rcl} \left( \sum_{k=0}^j b^{(k)} t^k \right) * \left( \sum_{k=1}^{j+1} k z^{(k)} * t^{k-1} \right) & = & \left( \sum_{k=0}^j e^{(k)} * t^k \right) * \left( \sum_{k=1}^{j+1} k x^{(k)} * t^{k-1} \right) + o( t^p ) \\ z^{(j+1)} & = & \frac{1}{j+1} \frac{1}{ b^{(0)} } \left( \sum_{k=0}^j e^{(k)} (j+1-k) x^{(j+1-k)} - \sum_{k=1}^j b^{(k)} (j+1-k) z^{(j+1-k)} \right) \\ z^{(j+1)} & = & \frac{1}{j+1} \frac{1}{ b^{(0)} } \left( \sum_{k=1}^{j+1} k x^{(k)} e^{(j+1-k)} - \sum_{k=1}^j k z^{(k)} b^{(j+1-k)} \right) \end{array}$  This completes the induction that computes   e^{(j)} and   z^{(j+1)} .

Cases that Apply Recursion Above
 ExpForward Exponential Function Forward Taylor Polynomial Theory LogForward Logarithm Function Forward Taylor Polynomial Theory SqrtForward Square Root Function Forward Taylor Polynomial Theory SinCosForward Trigonometric and Hyperbolic Sine and Cosine Forward Theory AtanForward Arctangent Function Forward Taylor Polynomial Theory AsinForward Arcsine Function Forward Taylor Polynomial Theory AcosForward Arccosine Function Forward Taylor Polynomial Theory

Special Cases
 tan_forward Tangent and Hyperbolic Tangent Forward Taylor Polynomial Theory

Input File: omh/forward_theory.omh