This talk considers goodness of fit diagnostics for time-series data from processes approximately modeled by systems of nonlinear ordinary differential equations. In particular, we seek to determine three nested causes of lack of fit: (i) unmodeled stochastic forcing, (ii) mis-specified functional forms and (iii) mis-specified state variables. Testing lack of fit in differential equations is challenging since the model is expressed in terms of rates of change of the measured variables. Here, lack of fit is represented on the model scale via time-varying parameters. We develop tests for each of the three cases above through bootstrap and permutation methods.
A motivating example is presented from laboratory-based ecology in which algae are grown on nitrogen-rich medium and rotifers are introduced as a predator. The resulting data exhibit dynamics that do not correspond to those generated by classical ecological models. A hypothesized explanation is that more than one algal species are present in the chemostat. We assess the statistical evidence for this claim and show that while models incorporating multiple algal species provide better agreement with the data, their existence cannot be demonstrated without strong model assumptions. We conclude with an examination of the use of control theory to design inputs into dynamic systems to improve parameter estimation and power to detect missing components.