6.4 Comparing models with AIC and model weights

To get the AIC or AICc values for a model fit from a MARSS fit, use fit$AIC or fit$AICc. The log-likelihood is in fit$logLik and the number of estimated parameters in fit$num.params. For fits from other functions, try AIC(fit) or look at the function documentation.

Let’s put the AICc values 3 Nile models together:

nile.aic <- c(kem.0$AICc, kem.1$AICc, kem.2$AICc, kem.3$AICc)

Then we calculate the AICc minus the minus AICc in our model set and compute the model weights. \(\Delta\text{AIC}\) is the AIC values minus the minimum AIC value in your model set.

delAIC <- nile.aic - min(nile.aic)
relLik <- exp(-0.5 * delAIC)
aicweight <- relLik/sum(relLik)

And this leads to our model weights table:

aic.table <- data.frame(AICc = nile.aic, delAIC = delAIC, relLik = relLik, 
    weight = aicweight)
rownames(aic.table) <- c("flat level", "linear trend", "stoc level", 
    "stoc level w drift")

Here the table is printed using round() to limit the number of digits shown.

round(aic.table, digits = 3)

                       AICc delAIC relLik weight
flat level         1313.155 31.379  0.000  0.000
linear trend       1290.882  9.106  0.011  0.007
stoc level         1281.776  0.000  1.000  0.647
stoc level w drift 1283.026  1.250  0.535  0.346

One thing to keep in mind when comparing models within a set of models is that the model set needs to include at least one model that can fit the data reasonably well. Reasonably well' means the model can put a fitted line through the data. Can't all models do that? Definitely, not. For example, the flat-level model cannot put a fitted line through the Nile River data. It is simply impossible. The straight trend model also cannot put a fitted line through the flow data. So if our model set only included flat-level and straight trend, then we might have said that the straight trend model isbest’ even though it is just the better of two bad models.