$Id: introduction.omh 1370 2009-05-31 05:31:50Z bradbell $
/* --------------------------------------------------------------------------
CppAD: C++ Algorithmic Differentiation: Copyright (C) 2003-07 Bradley M. Bell

CppAD is distributed under multiple licenses. This distribution is under
the terms of the 
                    Common Public License Version 1.0.

A copy of this license is included in the COPYING file of this distribution.
Please visit http://www.coin-or.org/CppAD/ for information on other licenses.
-------------------------------------------------------------------------- */
$begin Introduction$$
$spell
	exp_eps
	Griewank
	Andreas
	CppAD
$$


$index introduction, AD$$
$index AD, introduction$$
$index Algorithmic Differentiation, introduction$$
$index Automatic Differentiation, introduction$$

$section An Introduction by Example to Algorithmic Differentiation$$


$head Preface$$

$subhead Algorithmic Differentiation$$
Algorithmic Differentiation 
(often referred to as Automatic Differentiation or just AD)
uses the software representation 
of a function to obtain an efficient method for calculating its derivatives.
These derivatives can be of arbitrary order and are analytic in nature 
(do not have any truncation error).

$subhead Forward Mode$$
A forward mode sweep computes 
the partial derivative of all the dependent variables with respect
to one independent variable (or independent variable direction). 

$subhead Reverse Mode$$
A reverse mode sweep computes 
the derivative of one dependent variable
(or one dependent variable direction) 
with respect to all the independent variables.

$subhead Operation Count$$
The number of floating point operations for either a 
forward or reverse mode sweep
is a small multiple of the number required to evaluate the original function. 
Thus, using reverse mode,
you can evaluate the derivative of a scalar valued function 
with respect to thousands of variables in a small multiple of the
work to evaluate the original function.

$subhead Efficiency$$
AD automatically takes advantage of the
speed of your algorithmic representation of a function.
For example,
if you calculate a determinant using LU factorization,
AD will use the LU representation for
the derivative of the determinant
(which is faster than using the definition of the determinant).

$head Purpose$$
This is an introduction by example
to Algorithmic Differentiation.
Its purpose is to aid in understand what AD calculates,
how the calculations are preformed,
and the amount of computation and memory required 
for a forward or reverse sweep.

$head Outline$$
$list Alpha$$
Demonstrate the use of CppAD to calculate derivatives of a 
polynomial: $cref/get_started.cpp/$$.

$lnext
Present two algorithms that approximate the exponential function.
The first algorithm $cref/exp_2.hpp/$$ is simpler and does not 
include any logical variables or loops.
The second algorithm $cref/exp_eps.hpp/$$ includes
logical operations and a $code while$$ loop.
For each of these algorithms, do the following:

$list number$$
Define the mathematical function corresponding to the algorithm
($cref/exp_2/$$ and $cref/exp_eps/$$).
$lnext
Write out the floating point operation sequence,
and corresponding values,
that correspond to executing the algorithm for a specific input
($cref/exp_2_for0/$$ and $cref/exp_eps_for0/$$).
$lnext
Compute a forward sweep derivative of the operation sequence
($cref/exp_2_for1/$$ and $cref/exp_eps_for1/$$).
$lnext
Compute a reverse sweep derivative of the operation sequence
($cref/exp_2_rev1/$$ and $cref/exp_eps_rev1/$$).
$lnext
Use CppAD to compute both a forward and reverse sweep 
of the operation sequence
($cref/exp_2_cppad/$$ and $cref/exp_eps_cppad/$$).
$lend

$lnext
The program $cref/exp_apx_main.cpp/$$ runs all of the test
routines that validate the calculations in the $cref/exp_2/$$
and $cref/exp_eps/$$ presentation.
$lend

$head Reference$$
An in-depth review of AD theory and methods can be found in 
the book
$italic 
Evaluating Derivatives:
Principles and Techniques of Algorithmic Differentiation
$$,
Andreas Griewank,
SIAM Frontiers in Applied Mathematics, 
2000.

$childtable%
	introduction/get_started/get_started.cpp%
	introduction/exp_apx/exp_2.hpp%
	introduction/exp_apx/exp_eps.hpp%
	introduction/exp_apx/main.cpp
%$$

$end
