Group for Research in Decision Analysis

Iterated Conditional Expectations

Jamie Stafford

The use of local likelihood methods (Loader 1999) in the presence of data that is either interval censored, or has been aggregated into bins, leads naturally to the consideration of EM-type strategies. Numerical integration permits a local EM algorithm to be implemented as a global Newton iteration where the latter?s excellent convergence properties can be exploited. The method requires an explicit solution of the local likelihood equations at the M-step and this can always be found through the use of symbolic Newton-Raphson (Andrews and Stafford 2000). Iteration is thus rendered on the E-step only where a conditional expectation operator is applied, hence ICE. We primarily focus on a class of local likelihood density estimates where one member of this class retains the simplicity and interpretive appeal of the usual kernel density estimate for completely observed data. It is computed using a fixed point algorithm that generalizes the self-consistency algorithms of Efron (1967), Turnbull (1976), and Li et al. (1997) by introducing kernel smoothing at each iteration. Other local likelihood classes include those for intensity estimation, local regression in the context of a generalized linear model, additive models and so on.

References [1] Andrews, D. F. and Stafford, J. E. (2000). Symbolic Computation for Statistical Inference. OUP: Oxford. [2] Efron, B. (1967). The two sample problem with censored data. Fourth Berkeley Symposium on Mathematical Statistics, University of California Press, 831-853. Author A. [3] Li, L., Watkins, T. and Yu, Q. (1997). An EM algorithm for smoothing the self-consistent estimator of survival functions with interval-censored data. Scandinavian Journal of Statistics 24, 531-542. [4] Loader, C. R. (1999) Local Regression and Likelihood. Springer-Verlag: New York [5] Turnbull, B. W. (1976). The empirical distribution function with arbitrarily grouped, censored and truncated data. Journal of the Royal Statistical Society, Series B 38, 290-295.