Prev Next

The Theory of Forward Mode

Taylor Notation
In Taylor notation, each variable corresponds to a function of a single argument which we denote by t (see Section 10.2 of Evaluating Derivatives ). Here and below  X(t) ,  Y(t) , and Z(t) are scalar valued functions and the corresponding p-th order Taylor coefficients row vectors are  x ,  y and  z ; i.e.,  \[
\begin{array}{lcr}
X(t) & = & x^{(0)} + x^{(1)} * t + \cdots + x^{(p)} * t^p + o( t^p ) \\
Y(t) & = & y^{(0)} + y^{(1)} * t + \cdots + y^{(p)} * t^p + o( t^p ) \\
Z(t) & = & z^{(0)} + z^{(1)} * t + \cdots + z^{(p)} * t^p + o( t^p ) 
\end{array}
\] 
For the purposes of this section, we are given  x and  y and need to determine  z .

Binary Operators

Addition
 \[
\begin{array}{rcl}
Z(t)   
& = & X(t)   + Y(t)  
\\
\sum_{j=0}^p z^{(j)} * t^j   
& = & \sum_{j=0}^p x^{(j)} * t^j + \sum_{j=0}^p y^{(j)} * t^j  + o( t^p )
\\
z^{(j)} & = & x^{(j)} + y^{(j)}
\end{array} 
\] 


Subtraction
 \[
\begin{array}{rcl}
Z(t)   
& = & X(t) - Y(t)  
\\
\sum_{j=0}^p z^{(j)} * t^j   
& = & \sum_{j=0}^p x^{(j)} * t^j - \sum_{j=0}^p y^{(j)} * t^j  + o( t^p )
\\
z^{(j)} & = & x^{(j)} - y^{(j)}
\end{array} 
\] 


Multiplication
 \[
\begin{array}{rcl}
Z(t)   
& = & X(t) * Y(t)  
\\
\sum_{j=0}^p z^{(j)} * t^j   
& = & \left( \sum_{j=0}^p x^{(j)} * t^j \right)

\left( \sum_{j=0}^p y^{(j)} * t^j \right) + o( t^p )
\\
z^{(j)} & = & \sum_{k=0}^j x^{(j-k)} * y^{(k)}
\end{array} 
\] 


Division
 \[
\begin{array}{rcl}
Z(t)   
& = & X(t) / Y(t)  
\\

& = & z * y
\\
\sum_{j=0}^p x^{(j)} * t^j   
& = & 
\left( \sum_{j=0}^p z^{(j)} * t^j \right)

\left( \sum_{j=0}^p y^{(j)} * t^j \right) 

o( t^p )
\\
x^{(j)} & = & \sum_{k=0}^j z^{(j-k)} y^{(k)}
\\
z^{(j)} & = & \frac{1}{y^{(0)}} \left( x^{(j)} - \sum_{k=1}^j z^{(j-k)} y^{(k)} \right)
\end{array}

\] 


Standard Math Functions
Suppose that  F  is a standard math function and  \[
     Z(t) = F[ X(t) ]
\]


Differential Equation
All of the standard math functions satisfy a differential equation of the form  \[
     B(u) * F^{(1)} (u) - A(u) * F (u)  = D(u)
\] 
We use  a ,  b and  d to denote the p-th order Taylor coefficient row vectors for  A [ X (t) ]  ,  B [ X (t) ] and  D [ X (t) ]  respectively. We assume that these coefficients are known functions of  x , the p-th order Taylor coefficients for  X(t) .

Taylor Coefficients Recursion Formula
Our problem here is to express  z , the p-th order Taylor coefficient row vector for  Z(t) , in terms of these other known coefficients. It follows from the formulas above that  \[
\begin{array}{rcl}
Z^{(1)} (t) 
& = & F^{(1)} [ X(t) ] * X^{(1)} (t) 
\\
B[ X(t) ] * Z^{(1)} (t) 
& = & \{ D[ X(t) ] + A[ X(t) ] * Z(t) \} * X^{(1)} (t)
\\
B[ X(t) ] * Z^{(1)} (t) & = & E(t) * X^{(1)} (t)
\end{array}
\] 
where we define  \[
E(t) =  D[X(t)] + A[X(t)] * Z(t) 
\] 
We can compute the value of  z^{(0)} using the formula  \[
     z^{(0)} = F ( x^{(0)} )
\]
Suppose by induction (on  j ) that we are given the Taylor coefficients of  E(t) up to order  j-1 ; i.e.,  e^{(k)} for  k = 0 , \ldots , j-1 and the coefficients  z^{(k)} for  k = 0 , \ldots , j . We can compute  e^{(j)} using the formula  \[
     e^{(j)} = d^{(j)} + \sum_{k=0}^j a^{(j-k)} * z^{(k)}
\] 
We need to complete the induction by finding formulas for  z^{(j+1)} . It follows for the formula for the multiplication operator that   \[
\begin{array}{rcl}
\left( \sum_{k=0}^j b^{(k)} t^k \right)
*
\left( \sum_{k=1}^{j+1} k z^{(k)} * t^{k-1} \right)
& = & 
\left( \sum_{k=0}^j e^{(k)} * t^k \right) 
*
\left( \sum_{k=1}^{j+1} k x^{(k)} * t^{k-1} \right)
+
o( t^p )
\\
z^{(j+1)} & = & \frac{1}{j+1} \frac{1}{ b^{(0)} } 
\left(
     \sum_{k=0}^j e^{(k)} (j+1-k) x^{(j+1-k)} 
     - \sum_{k=1}^j b^{(k)} (j+1-k) z^{(j+1-k)}  
\right)
\\
z^{(j+1)} & = & \frac{1}{j+1} \frac{1}{ b^{(0)} } 
\left(
     \sum_{k=1}^{j+1} k x^{(k)} e^{(j+1-k)} 
     - \sum_{k=1}^j k z^{(k)}  b^{(j+1-k)} 
\right)
\end{array}
\] 
This completes the induction that computes  e^{(j)} and  z^{(j+1)} .

Cases that Apply Recursion Above
ExpForward Exponential Function Forward Taylor Polynomial Theory
LogForward Logarithm Function Forward Taylor Polynomial Theory
SqrtForward Square Root Function Forward Taylor Polynomial Theory
SinCosForward Trigonometric and Hyperbolic Sine and Cosine Forward Theory
AtanForward Arctangent Function Forward Taylor Polynomial Theory
AsinForward Arcsine Function Forward Taylor Polynomial Theory
AcosForward Arccosine Function Forward Taylor Polynomial Theory

Special Cases
tan_forward Tangent and Hyperbolic Tangent Forward Taylor Polynomial Theory

Input File: omh/theory/forward_theory.omh