Prev Next Index-> contents reference index search external Up-> CppAD Appendix Theory ForwardTheory CppAD-> Install Introduction AD ADFun preprocessor multi_thread library ipopt_solve Example speed Appendix Appendix-> Faq Theory glossary Bib WishList whats_new deprecated compare_c License Theory-> ForwardTheory ReverseTheory reverse_identity ForwardTheory-> exp_forward log_forward sqrt_forward sin_cos_forward atan_forward asin_forward acos_forward tan_forward erf_forward Headings-> Taylor Notation Binary Operators ---..Addition ---..Subtraction ---..Multiplication ---..Division Standard Math Functions ---..Differential Equation ---..Taylor Coefficients Recursion Formula ---..Cases that Apply Recursion Above ---..Special Cases

The Theory of Forward Mode

Taylor Notation
In Taylor notation, each variable corresponds to a function of a single argument which we denote by t (see Section 10.2 of Evaluating Derivatives ). Here and below $X(t)$, $Y(t)$, and Z(t) are scalar valued functions and the corresponding p-th order Taylor coefficients row vectors are $x$, $y$ and $z$; i.e., $$\begin{array}{lcr} X(t) & = & x^{(0)} + x^{(1)} * t + \cdots + x^{(p)} * t^p + o( t^p ) \\ Y(t) & = & y^{(0)} + y^{(1)} * t + \cdots + y^{(p)} * t^p + o( t^p ) \\ Z(t) & = & z^{(0)} + z^{(1)} * t + \cdots + z^{(p)} * t^p + o( t^p ) \end{array}$$ For the purposes of this section, we are given $x$ and $y$ and need to determine $z$.

Binary Operators

$$\begin{array}{rcl} Z(t) & = & X(t) + Y(t) \\ \sum_{j=0}^p z^{(j)} * t^j & = & \sum_{j=0}^p x^{(j)} * t^j + \sum_{j=0}^p y^{(j)} * t^j + o( t^p ) \\ z^{(j)} & = & x^{(j)} + y^{(j)} \end{array}$$
Subtraction
$$\begin{array}{rcl} Z(t) & = & X(t) - Y(t) \\ \sum_{j=0}^p z^{(j)} * t^j & = & \sum_{j=0}^p x^{(j)} * t^j - \sum_{j=0}^p y^{(j)} * t^j + o( t^p ) \\ z^{(j)} & = & x^{(j)} - y^{(j)} \end{array}$$
Multiplication
$$\begin{array}{rcl} Z(t) & = & X(t) * Y(t) \\ \sum_{j=0}^p z^{(j)} * t^j & = & \left( \sum_{j=0}^p x^{(j)} * t^j \right) * \left( \sum_{j=0}^p y^{(j)} * t^j \right) + o( t^p ) \\ z^{(j)} & = & \sum_{k=0}^j x^{(j-k)} * y^{(k)} \end{array}$$
Division
$$\begin{array}{rcl} Z(t) & = & X(t) / Y(t) \\ x & = & z * y \\ \sum_{j=0}^p x^{(j)} * t^j & = & \left( \sum_{j=0}^p z^{(j)} * t^j \right) * \left( \sum_{j=0}^p y^{(j)} * t^j \right) + o( t^p ) \\ x^{(j)} & = & \sum_{k=0}^j z^{(j-k)} y^{(k)} \\ z^{(j)} & = & \frac{1}{y^{(0)}} \left( x^{(j)} - \sum_{k=1}^j z^{(j-k)} y^{(k)} \right) \end{array}$$
Standard Math Functions
Suppose that $F$ is a standard math function and $$Z(t) = F[ X(t) ]$$

Differential Equation
All of the standard math functions satisfy a differential equation of the form $$B(u) * F^{(1)} (u) - A(u) * F (u) = D(u)$$ We use $a$, $b$ and $d$ to denote the p-th order Taylor coefficient row vectors for $A [ X (t) ]$, $B [ X (t) ]$ and $D [ X (t) ]$ respectively. We assume that these coefficients are known functions of $x$, the p-th order Taylor coefficients for $X(t)$.

Taylor Coefficients Recursion Formula
Our problem here is to express $z$, the p-th order Taylor coefficient row vector for $Z(t)$, in terms of these other known coefficients. It follows from the formulas above that $$\begin{array}{rcl} Z^{(1)} (t) & = & F^{(1)} [ X(t) ] * X^{(1)} (t) \\ B[ X(t) ] * Z^{(1)} (t) & = & \{ D[ X(t) ] + A[ X(t) ] * Z(t) \} * X^{(1)} (t) \\ B[ X(t) ] * Z^{(1)} (t) & = & E(t) * X^{(1)} (t) \end{array}$$ where we define $$E(t) = D[X(t)] + A[X(t)] * Z(t)$$ We can compute the value of $z^{(0)}$ using the formula $$z^{(0)} = F ( x^{(0)} )$$ Suppose by induction (on $j$) that we are given the Taylor coefficients of $E(t)$ up to order $j-1$; i.e., $e^{(k)}$ for $k = 0 , \ldots , j-1$ and the coefficients $z^{(k)}$ for $k = 0 , \ldots , j$. We can compute $e^{(j)}$ using the formula $$e^{(j)} = d^{(j)} + \sum_{k=0}^j a^{(j-k)} * z^{(k)}$$ We need to complete the induction by finding formulas for $z^{(j+1)}$. It follows for the formula for the multiplication operator that $$\begin{array}{rcl} \left( \sum_{k=0}^j b^{(k)} t^k \right) * \left( \sum_{k=1}^{j+1} k z^{(k)} * t^{k-1} \right) & = & \left( \sum_{k=0}^j e^{(k)} * t^k \right) * \left( \sum_{k=1}^{j+1} k x^{(k)} * t^{k-1} \right) + o( t^p ) \\ z^{(j+1)} & = & \frac{1}{j+1} \frac{1}{ b^{(0)} } \left( \sum_{k=0}^j e^{(k)} (j+1-k) x^{(j+1-k)} - \sum_{k=1}^j b^{(k)} (j+1-k) z^{(j+1-k)} \right) \\ z^{(j+1)} & = & \frac{1}{j+1} \frac{1}{ b^{(0)} } \left( \sum_{k=1}^{j+1} k x^{(k)} e^{(j+1-k)} - \sum_{k=1}^j k z^{(k)} b^{(j+1-k)} \right) \end{array}$$ This completes the induction that computes $e^{(j)}$ and $z^{(j+1)}$.

Cases that Apply Recursion Above
 exp_forward Exponential Function Forward Mode Theory log_forward Logarithm Function Forward Mode Theory sqrt_forward Square Root Function Forward Mode Theory sin_cos_forward Trigonometric and Hyperbolic Sine and Cosine Forward Theory atan_forward Inverse Tangent and Hyperbolic Tangent Forward Mode Theory asin_forward Inverse Sine and Hyperbolic Sine Forward Mode Theory acos_forward Inverse Cosine and Hyperbolic Cosine Forward Mode Theory

Special Cases
 tan_forward Tangent and Hyperbolic Tangent Forward Taylor Polynomial Theory

Input File: omh/theory/forward_theory.omh