Nonlinear mixed-effects model


Nonlinear mixed-effects models constitute a class of statistical models generalizing linear mixed-effects models. Like linear mixed-effects models, they are particularly useful in settings where there are multiple measurements within the same statistical units or when there are dependencies between measurements on related statistical units. Nonlinear mixed-effects models are applied in many fields including medicine, public health, pharmacology, and ecology.

Definition

While any statistical model containing both fixed effects and random effects is an example of a nonlinear mixed-effects model, the most commonly used models are members of the class of nonlinear mixed-effects models for repeated measures
where
  • is the number of groups/subjects,
  • is the number of observations for the th group/subject,
  • is a real-valued differentiable function of a group-specific parameter vector and a covariate vector,
  • is modeled as a linear mixed-effects model where is a vector of fixed effects and is a vector of random effects associated with group, and
  • is a random variable describing additive noise.

    Estimation

When the model is only nonlinear in fixed effects and the random effects are Gaussian, maximum-likelihood estimation can be done using nonlinear least squares methods, although asymptotic properties of estimators and test statistics may differ from the conventional general linear model. In the more general setting, there exist several methods for doing maximum-likelihood estimation or maximum a posteriori estimation in certain classes of nonlinear mixed-effects models – typically under the assumption of normally distributed random variables. A popular approach is the Lindstrom-Bates algorithm which relies on iteratively optimizing a nonlinear problem, locally linearizing the model around this optimum and then employing conventional methods from linear mixed-effects models to do maximum likelihood estimation. Stochastic approximation of the expectation-maximization algorithm gives an alternative approach for doing maximum-likelihood estimation.

Applications

Example: Disease progression modeling

Nonlinear mixed-effects models have been used for modeling progression of disease. In progressive disease, the temporal patterns of progression on outcome variables may follow a nonlinear temporal shape that is similar between patients. However, the stage of disease of an individual may not be known or only partially known from what can be measured. Therefore, a latent time variable that describe individual disease stage can be included in the model.

Example: Modeling cognitive decline in Alzheimer's disease

is characterized by a progressive cognitive deterioration. However, patients may differ widely in cognitive ability and reserve, so cognitive testing at a single time point can often only be used to coarsely group individuals in different stages of disease. Now suppose we have a set of longitudinal cognitive data from individuals that are each categorized as having either normal cognition, mild cognitive impairment or dementia at the baseline visit. These longitudinal trajectories can be modeled using a nonlinear mixed effects model that allows differences in disease state based on baseline categorization:
where
  • is a function that models the mean time-profile of cognitive decline whose shape is determined by the parameters,
  • represents observation time,
  • and are dummy variables that are 1 if individual has MCI or dementia at baseline and 0 otherwise,
  • and are parameters that model the difference in disease progression of the MCI and dementia groups relative to the cognitively normal,
  • is the difference in disease stage of individual relative to his/her baseline category, and
  • is a random variable describing additive noise.
An example of such a model with an exponential mean function fitted to longitudinal measurements of the Alzheimer's Disease Assessment Scale-Cognitive Subscale is shown in the box. As shown, the inclusion of fixed effects of baseline categorization and the random effect of individual continuous disease stage aligns the trajectories of cognitive deterioration to reveal a common pattern of cognitive decline.

Example: Growth analysis

Growth phenomena often follow nonlinear patters. Factors such as nutrient deficiency may both directly affect the measured outcome, but possibly also timing. If a model fails to account for the differences in timing, the estimated population-level curves may smooth out finer details due to lack of synchronization between organisms. Nonlinear mixed-effects models enable simultaneous modeling of individual differences in growth outcomes and timing.

Example: Modeling human height

Models for estimating the mean curves of human height and weight as a function of age and the natural variation around the mean are used to create growth charts. The growth of children can however become desynchronized due to both genetic and environmental factors. For example, age at onset of puberty and its associated height spurt can vary several years between adolescents. Therefore, cross-sectional studies may underestimate the magnitude of the pubertal height spurt because age is not synchronized with biological development. The differences in biological development can be modeled using random effects that describe a mapping of observed age to a latent biological age using a so-called warping function. A simple nonlinear mixed-effects model with this structure is given by
where
  • is a function that represents the height development of a typical child as a function of age. Its shape is determined by the parameters,
  • is the age of child corresponding to the height measurement,
  • is a warping function that maps age to biological development to synchronize. Its shape is determined by the random effects,
  • is a random variable describing additive variation.
There exists several methods and software packages for fitting such models. The so-called SITAR model can fit such models using warping functions that are affine transformations of time. SITAR can be employed as a sophisticated data reduction tool, reducing growth trajectories to three patient level effects summarising the variability in 'size', 'timing' and 'intensity' of growth across a sample. This can facilitate the assessment of the impact of predictor variables on overall growth trajectory, and has been used to quantify the impact of antenatal steroids on child growth. An alternative approach is the so-called pavpop model that can fit models with smoothly-varying warping functions, an example of which is shown in the box.

Example: Population Pharmacokinetic/pharmacodynamic modeling

for describing exposure-response relationships such as the Emax model can be formulated as nonlinear mixed-effects models. The mixed-model approach allows modeling of both population level and individual differences in effects that have a nonlinear effect on the observed outcomes, for example the rate at which a compound is being metabolized or distributed in the body.

Example: COVID-19 epidemiological modeling

The platform of the nonlinear mixed effect models can be used to describe infection trajectories of subjects and understand some common features shared across the subjects. In epidemiological problems, subjects can be countries, states, or counties, etc. This can be particularly useful in estimating a future trend of the epidemic in an early stage of pendemic where nearly little information is known regarding the disease.

Example: Prediction of oil production curve of shale oil wells at a new location with latent kriging

The eventual success of petroleum development projects relies on a large degree of well construction costs. As for unconventional oil and gas reservoirs, because of very low permeability, and a flow mechanism very different from that of conventional reservoirs, estimates for the well construction cost often contain high levels of uncertainty, and oil companies need to make heavy investment in the drilling and completion phase of the wells. The overall recent commercial success rate of horizontal wells in the United States is known to be 65%, which implies that only 2 out of 3 drilled wells will be commercially successful. For this reason, one of the crucial tasks of petroleum engineers is to quantify the uncertainty associated with oil or gas production from shale reservoirs, and further, to predict an approximated production behavior of a new well at a new location given specific completion data before actual drilling takes place to save a large degree of well construction costs.
The platform of the nonlinear mixed effect models can be extended to consider the spatial association by incorporating the geostatistical processes such as Gaussian process on the second stage of the model as follows:
where
  • is a function that models the mean time-profile of log-scaled oil production rate whose shape is determined by the parameters. The function is obtained from taking logarithm to the rate decline curve used in decline curve analysis,
  • represents covariates obtained from the completion process of the hydraulic fracturing and horizontal directional drilling for the -th well,
  • represents the spatial location of the -th well,
  • represents the Gaussian white noise with error variance ,
  • represents the Gaussian process with Gaussian covariance function,
  • represents the horseshoe shrinkage prior.
The Gaussian process regressions used on the latent level eventually produce kriging predictors for the curve parameters that dictate the shape of the mean curve on the date level. As the kriging techniques have been employed in the latent level, this technique is called latent kriging. The right panels show the prediction results of the latent kriging method applied to the two test wells in the Eagle Ford Shale Reservoir of South Texas.