Prev Next ipopt_cppad_ode_inverse

An ODE Inverse Problem Example

Problem
We define  y : \R \times \R^3 \rightarrow \R^2 as the solution of our ode forward problem . Suppose we are also given measurement values  z_k \in \R for  k = 1, 2, 3, 4 that are modeled by  \[
     z_k = y_1 ( s_k , a) + e_k
\] 
where  s_k \in \R ,  e_k \sim {\bf N} (0 , \sigma^2 )  , and  \sigma \in \R_+ . The maximum likelihood estimate for  a given  ( z_1 , z_2 , z_3 , z_4 ) solves the following inverse problem  \[
\begin{array}{rcl}
{\rm minimize} \; 
     & \sum_{k=1}^4 H_k [ y( s_k , a ) , a ] 
     & \;{\rm w.r.t} \; a \in \R^3
\end{array}
\] 
where the function  H_k : \R^2 \times \R^3 \rightarrow \R is defined by  \[
     H_k (y, a) = ( z_k - y_1 )^2 
\] 


Black Box Method
A common approach to an inverse problem is to treat the forward problem as a black box (that we do not look inside of or try to understand). In this approach, for each value of the parameter vector  a one uses a numerical procedure to solve for  y_1 ( s_k , a ) for  k = 1 , 2 , 3, 4 .

Two levels of Iteration
As noted above, this numerical procedure often involves iterative procedures for solving a set of equations. Thus, in this approach, there are two levels of iterations, one with respect to the parameter values and the other for solving the forward problem.

Derivatives
In addition, in the black box approach, differentiating the forward problem often involves differentiating an iterative procedure. Since the iterative procedure only returns an approximate solution, finite differences often lead to very inaccurate approximations for the derivatives (which in turn create problems for the optimization process). On the other hand, direct application of AD to compute the derivatives requires a huge amount of memory and calculations to differentiate the forward iterative procedure. (There are special techniques for applying AD to the solutions of iterative procedures, but that is outside the scope of this presentation).

Simultaneous Method
The simultaneous forward and inverse method uses constraints to include the solution of the forward problem in the inverse problem. To be specific for our example,  \[
\begin{array}{rcl}
{\rm minimize} 
     & \sum_{k=1}^4 H_k( y^{k * ns} , a )
     & \; {\rm w.r.t} \; y^1 \in \R^2 , \ldots , y^{4 * ns} \in \R^2 ,
     \; a \in \R^3 
\\
{\rm subject \; to}
     & y^{M+1}  =  y^M + 
\left[ G( y^M , a ) + G( y^{M+1} , a ) \right] * \frac{ s_{M+1} - s_M }{ 2 }
     & \; {\rm for} \; M = 0 , \ldots , 4 * ns - 1
\\
     & y^0 = F(a)
\end{array}
\] 
where  ns is the number of time intervals (used by the trapezoidal approximation) between each of the measurement times. Note that, in this form, the iterations of the optimization procedure also solve the forward problem equations. In addition, the functions that need to be differentiated do not involve an iterative procedure.
Input File: omh/ipopt_cppad_ode2.omh