When using simulation codes, one often has the task of minimizing a scalar objective function with respect to numerous parameters. This situation occurs when trying to fit (assimilate) data or trying to optimize an engineering design. For simulations in which the objective function is reasonably well behaved, that is, is differentiable and does not contain too many multiple minima, gradient-based optimization methods can reduce the number of function evaluations required to determine the minimizing parameters. However, gradient-based methods are only advantageous if one can efficiently evaluate the gradients of the objective function. Adjoint differentiation efficiently provides these sensitivities, but is often viewed as difficult to implement in simulation codes.
In a modular approach to developing simulations, adjoint differentiation can be achieved relatively easily. The only requirement is that each module not only be able to calculate its forward transformation, but also be able to calculate the derivative of its outputs with respect to its inputs. In a modular approach, the forward simulation process is achieved by linking together the necessary modules. The output of the simulation is the objective function to be minimized, for example, the measure of mismatch to some given measurements. The data-flow diagram (network or graph) that describes the forward calculation automatically provides the path for the reverse or adjoint calculation needed to accumulate the sensitivities of the objective function to any parameters in the simulation model. In this framework, the sensitivities of the output objective function with respect to all the simulation parameters can be automatically calculated in a time that is comparable to the forward simulation calculation.
Two examples of the use of this modular approach to sensitivity calculation are presented. The Bayes Inference Engine (BIE) is a computer application for analyzing radiographs and making inferences about an object being radiographed. The BIE is a graphical programming tool that automatically implements adjoint differentiation, which facilitates advanced model building and allows hundreds or thousands of parameters to be determined in a reasonable time by matching a radiograph. A second application is aimed at determining the optical properties of a volume of tissue from measurements of its effect on infrared light pulses. The process is simulated by solving the partial-differential equations for diffusion. The goal is to minimize the sum of squared differences between the measured and predicted light outputs. To give an idea of the efficacy of using adjoint sensitivities, thousands of unknown diffusion coefficients can be determined in about ten minutes, or in several dozen evaluations of the simulation.
Keywords: Bayesian analysis, MAP estimator, uncertainty estimation, radiographic modeling, Markov chain Monte Carlo, adjoint differentiation, object oriented design
Get presentation (pdf, 196 KB)
Return to publication list
Send e-mail to author at kmh@hansonhub.com