The Wild Animal Modeling Wiki

**Testing significance of random effects**

## Summary

This page provides an overview of how to test the significance of random effects using log-likelihood ratio tests (in ASReml, ASReml-R, WOMBAT). To do this, you compare the log-likelihoods of models with and without the appropriate random effect - if removing the random effect causes a large enough drop in log-likelihood then one can say the effect is statistically significant. Commonly we assume that a test statistic equal to twice the difference in log-lkelihoods is distributed as chisquare with DF equal to the number of additional parameters in the more complex model (1 if testng a single random effect). There is a sensible argument that this is overly conservative and so variations on this are also used.

It is also possible to get an idea of if a random effect will be significant by examining the parameter estimate and standard error of the variance component. As a rule of thumb, if the estimate is more than twice the standard error then it is likely that the estimate will be significant, although this should always be checked analytically.

MCMCglmm is not likelihood based, instead DIC can be used for model selection.

### Table of contents

## Definitions

This code shows how to test the significance of the random effect 'RANDOMEFFECT' using log-likelihood ratio tests.

Response variable: Size

Fixed effects: Intercept (mu) & SEX

Random effects: Additive genetic variance & RANDOMEFFECT

Data containing phenotypic information: phenotypicdata

Data containing pedigree data:pedigreedata

## Sample Code

### ASReml

Fit a model with and without a particular random effect. Then use log likelihoods reported in the primary results file to perform a ratio test.ASReml analysis of size ANIMAL !P SIZE SEX !A #sex as a factor pedigreedata.ped !skip 1 phenotypicdata.dat !skip 1 !dopart 1 #change part to be run required !part 1 SIZE ~ mu SEX !r ANIMAL RANDOMEFFECT !part 2 SIZE ~ mu SEX !r ANIMAL

### ASReml-R

model1<-asreml(fixed=SIZE~1+SEX ,random=~ped(ANIMAL,var=T,init=1)+RANDOMEFFECT ,data=phenotypicdata ,ginverse=list(ANIMAL=ainv), na.method.X="omit', na.method.Y="omit') model2<-asreml(fixed=SIZE~1+SEX ,random=~ped(ANIMAL,var=T,init=1) ,data=phenotypicdata ,ginverse=list(ANIMAL=ainv), na.method.X="omit', na.method.Y="omit') #calculate the chi-squared stat for the log-likelihood ratio test 2*(model1$loglik-model2$loglik) #calculate the associated significance 1-pchisq(2*(model1$loglik-model2$loglik),1)

### MCMCglmm

We cannot use the Highest Posterior Density intervals (i.e. the HPDinterval() R function) to assess the statistical support for random effects. This is because random effects are constrained to positive values. However, by default, MCMCglmm calculates the Deviance Information Criterion (DIC), and this can be used to assess whether increases in model fit justify the inclusion of any particular effect:

model1.3$DIC model1.4$DIC

### WOMBAT

To test the significance of a random effect (here called`RANDOMEFFECT`

), we compare models with and without the random effect of interest, and compare the final log-likelihoods as reported in `SumEstimates.out`

.Fitting a model with

`RANDOMEFFECT`

usingMODEL FIX SEX RAN ANIMAL NRM RAN RANDOMEFFECT TRAIT SIZE END

And compare the log-likelihood of this model to a model without

`RANDOMEFFECT`

usingMODEL FIX SEX RAN ANIMAL NRM TRAIT SIZE END