Gordon Smyth Home: Research: Publications

Smyth, G. K., and Verbyla, A. P. (1996). A conditional approach to residual maximum
likelihood estimation in generalized linear models. *Journal of the Royal Statistical Society* B, **58**,
565-572.

Gordon K. Smyth

Department of Mathematics, University of Queensland, Q 4072, Australia.

Ari P. Verbyla

Department of Statistics, University of Adelaide, SA 5005, Australia

Residual maximum likelihood estimation (REML) is often preferred to maximum likelihood estimation as a method of estimating covariance parameters in linear models because it takes account of the loss of degrees of freedom in estimating the mean and produces unbiased estimating equations for the variance parameters. In this note it is shown that REML has an exact conditional likelihood interpretation, where the conditioning is on an appropriate sufficient statistic to remove dependence on the nuisance parameters. This interpretation clarifies the motivation for REML and generalizes directly to non-normal models in which there exists a low dimensional sufficient statistic for the fitted values. The conditional likelihood is shown to be well defined and to satisfy the properties of a likelihood function, even though this is not generally true when conditioning on statistics which depend on parameters of interest. Using the conditional likelihood representation, the concept of REML is extended to generalized linear models with varying dispersion and canonical link. Explicit calculation of the conditional likelihood is given for the oneway layout. A saddle-point approximation for the conditional likelihood is also derived.

* Keywords:* residual maximum likelihood, restricted maximum
likelihood, conditional likelihood, exponential dispersion model, modified profile
likelihood, saddle-point approximation, oneway layout.

Patterson and Thompson (1971) introduced residual maximum likelihood estimation (REML) as a method of estimating variance components in the context of unbalanced incomplete block designs. REML is often preferred to maximum likelihood estimation because it takes account of the loss of degrees of freedom in estimating the mean and produces unbiased estimating equations for the variance parameters. Alternative and more general derivations of REML are given by Harville (1974), Cooper & Thompson (1977) and Verbyla (1990). In all of these the residual likelihood is presented as the marginal likelihood of the error constrasts. This makes generalization of the residual likelihood principle to nonlinear models or non-normal distributions difficult since zero-mean error contrasts do not generally exist. Cox and Reid (1987) give an approximate conditional likelihood which reduces to REML when used to estimate covariance parameters in normal linear models. Although Cox and Reid's conditional likelihood is approximate, and is based on a simplification of Barndorff-Nielsen's (1983, 1985) modified profile likelihood which reduces to REML only in special cases, it does suggest a conditional interpretation for REML. In this paper we show that REML has an exact conditional likelihood interpretation in which the conditioning is on an appropriate sufficient statistic to remove dependence on the nuisance parameters. This interpretation clarifies the motivation for REML and generalizes directly to non-normal models in which there exists a low dimensional sufficient statistic for the fitted values.

Consider the linear model * y = X b + e* where

Using the conditional likelihood representation, the concept of REML is extended to
generalized linear models with varying dispersion. We assume that *y _{1},...,y_{n}*
follow a generalized linear model with canonical link, design matrix

The idea of conditioning to remove nuisance parameters goes back at least to Bartlett (1936, 1937), and is discussed extensively by Kalbfleisch and Sprott (1970). Our conditional likelihood is direct and differs from that suggested by Kalbfleisch and Sprott and motivated by their ``Euclidean assumption''. The difficulties that the Euclidean assumption was intended to overcome do not occur when the conditioning statistic is of the form given above.

McCullagh and Tibshirani (1990) give an estimating equation method of adjusting profile likelihoods for nuisance parameters, which reduces to REML when estimating covariances in normal linear models. Again this is not generally equivalent to our conditional likelihood but, because it produces unbiased estimating equations, may approximate our approach in large samples.

Preprint 30 May 1995 (pdf)