Prev Next

The Theory of Forward Mode

Taylor Notation
In Taylor notation, each variable corresponds to a function of a single argument which we denote by t (see Section 10.2 of Evaluating Derivatives ). Here and below  X(t) ,  Y(t) , and Z(t) are scalar valued functions and the corresponding p-th order Taylor coefficients row vectors are  x ,  y and  z ; i.e.,  \[
\begin{array}{lcr}
X(t) & = & x^{(0)} + x^{(1)} * t + \cdots + x^{(p)} * t^p + o( t^p ) \\
Y(t) & = & y^{(0)} + y^{(1)} * t + \cdots + y^{(p)} * t^p + o( t^p ) \\
Z(t) & = & z^{(0)} + z^{(1)} * t + \cdots + z^{(p)} * t^p + o( t^p )
\end{array}
\] 
For the purposes of this section, we are given  x and  y and need to determine  z .

Binary Operators

Addition
 \[
\begin{array}{rcl}
Z(t)
& = & X(t)   + Y(t)
\\
\sum_{j=0}^p z^{(j)} * t^j
& = & \sum_{j=0}^p x^{(j)} * t^j + \sum_{j=0}^p y^{(j)} * t^j  + o( t^p )
\\
z^{(j)} & = & x^{(j)} + y^{(j)}
\end{array}
\] 


Subtraction
 \[
\begin{array}{rcl}
Z(t)
& = & X(t) - Y(t)
\\
\sum_{j=0}^p z^{(j)} * t^j
& = & \sum_{j=0}^p x^{(j)} * t^j - \sum_{j=0}^p y^{(j)} * t^j  + o( t^p )
\\
z^{(j)} & = & x^{(j)} - y^{(j)}
\end{array}
\] 


Multiplication
 \[
\begin{array}{rcl}
Z(t)
& = & X(t) * Y(t)
\\
\sum_{j=0}^p z^{(j)} * t^j
& = & \left( \sum_{j=0}^p x^{(j)} * t^j \right)
*
\left( \sum_{j=0}^p y^{(j)} * t^j \right) + o( t^p )
\\
z^{(j)} & = & \sum_{k=0}^j x^{(j-k)} * y^{(k)}
\end{array}
\] 


Division
 \[
\begin{array}{rcl}
Z(t)
& = & X(t) / Y(t)
\\
x
& = & z * y
\\
\sum_{j=0}^p x^{(j)} * t^j
& = &
\left( \sum_{j=0}^p z^{(j)} * t^j \right)
*
\left( \sum_{j=0}^p y^{(j)} * t^j \right)
+
o( t^p )
\\
x^{(j)} & = & \sum_{k=0}^j z^{(j-k)} y^{(k)}
\\
z^{(j)} & = & \frac{1}{y^{(0)}} \left( x^{(j)} - \sum_{k=1}^j z^{(j-k)} y^{(k)} \right)
\end{array}

\] 


Standard Math Functions
Suppose that  F  is a standard math function and  \[
     Z(t) = F[ X(t) ]
\]


Differential Equation
All of the standard math functions satisfy a differential equation of the form  \[
     B(u) * F^{(1)} (u) - A(u) * F (u)  = D(u)
\] 
We use  a ,  b and  d to denote the p-th order Taylor coefficient row vectors for  A [ X (t) ]  ,  B [ X (t) ] and  D [ X (t) ]  respectively. We assume that these coefficients are known functions of  x , the p-th order Taylor coefficients for  X(t) .

Taylor Coefficients Recursion Formula
Our problem here is to express  z , the p-th order Taylor coefficient row vector for  Z(t) , in terms of these other known coefficients. It follows from the formulas above that  \[
\begin{array}{rcl}
Z^{(1)} (t)
& = & F^{(1)} [ X(t) ] * X^{(1)} (t)
\\
B[ X(t) ] * Z^{(1)} (t)
& = & \{ D[ X(t) ] + A[ X(t) ] * Z(t) \} * X^{(1)} (t)
\\
B[ X(t) ] * Z^{(1)} (t) & = & E(t) * X^{(1)} (t)
\end{array}
\] 
where we define  \[
E(t) =  D[X(t)] + A[X(t)] * Z(t)
\] 
We can compute the value of  z^{(0)} using the formula  \[
     z^{(0)} = F ( x^{(0)} )
\]
Suppose by induction (on  j ) that we are given the Taylor coefficients of  E(t) up to order  j-1 ; i.e.,  e^{(k)} for  k = 0 , \ldots , j-1 and the coefficients  z^{(k)} for  k = 0 , \ldots , j . We can compute  e^{(j)} using the formula  \[
     e^{(j)} = d^{(j)} + \sum_{k=0}^j a^{(j-k)} * z^{(k)}
\] 
We need to complete the induction by finding formulas for  z^{(j+1)} . It follows for the formula for the multiplication operator that   \[
\begin{array}{rcl}
\left( \sum_{k=0}^j b^{(k)} t^k \right)
*
\left( \sum_{k=1}^{j+1} k z^{(k)} * t^{k-1} \right)
& = &
\left( \sum_{k=0}^j e^{(k)} * t^k \right)
*
\left( \sum_{k=1}^{j+1} k x^{(k)} * t^{k-1} \right)
+
o( t^p )
\\
z^{(j+1)} & = & \frac{1}{j+1} \frac{1}{ b^{(0)} }
\left(
     \sum_{k=0}^j e^{(k)} (j+1-k) x^{(j+1-k)}
     - \sum_{k=1}^j b^{(k)} (j+1-k) z^{(j+1-k)}
\right)
\\
z^{(j+1)} & = & \frac{1}{j+1} \frac{1}{ b^{(0)} }
\left(
     \sum_{k=1}^{j+1} k x^{(k)} e^{(j+1-k)}
     - \sum_{k=1}^j k z^{(k)}  b^{(j+1-k)}
\right)
\end{array}
\] 
This completes the induction that computes  e^{(j)} and  z^{(j+1)} .

Cases that Apply Recursion Above
exp_forward Exponential Function Forward Mode Theory
log_forward Logarithm Function Forward Mode Theory
sqrt_forward Square Root Function Forward Mode Theory
sin_cos_forward Trigonometric and Hyperbolic Sine and Cosine Forward Theory
atan_forward Inverse Tangent and Hyperbolic Tangent Forward Mode Theory
asin_forward Inverse Sine and Hyperbolic Sine Forward Mode Theory
acos_forward Inverse Cosine and Hyperbolic Cosine Forward Mode Theory

Special Cases
tan_forward Tangent and Hyperbolic Tangent Forward Taylor Polynomial Theory

Input File: omh/theory/forward_theory.omh