Group for Research in Decision Analysis

Statistical Tools: Is there any merit in calibration?

Don Fraser

Many statistical tools are framed in terms of just likelihood \(L (\theta)=f(y^0;\theta)\) and involve integrating the likelihood with a weight function $\pi(\theta)$. These have been incredibly successful in exploring widely and in seeking to eclipse the less organized confidence approach. Some cautions have appeared: for example, Dawid, Stone and Zidek (1973) and Stainforth, Allen, Tredger and Smith (2007). But the claim is pervasive that probability so calculated is something more rather than less than confidence. And of course confidence has its own disarray.

Is it just a free-for-all? Are there risks with an approach that uses only likelihood? Can likelihood be calibrated? Can the approach be calibrated?

We examine aspects of the argument, its history and some of the risks. And argue that calibration is needed, that it is possible but wont be as simple as using just likelihood.