Observed Fisher Information Matrix at the MLE

Returns the observed Fisher Information matrix for a marssMLE object (a fitted MARSS model) via either the analytical algorithm of Harvey (1989) or a numerical estimate.

The observed Fisher Information is the negative of the second-order partial derivatives of the log-likelihood function evaluated at the MLE. The derivatives being with respect to the parameters. The Hessian matrix is the second-order partial derivatives of a scalar-valued function. Thus the observed Fisher Information matrix is the Hessian of the negative log-likelihood function evaluated at the MLE (or equivalently the negative of the Hessian of the log-likelihood function). The inverse of the observed Fisher Information matrix is an estimate of the asymptotic variance-covariance matrix for the estimated parameters. Use MARSShessian() (which calls MARSSFisherI()) to return the parameter variance-covariance matrix computed from the observed Fisher Information matrix.

Note for the numerically estimated Hessian, we pass in the negative log-likelihood function to a minimization function. As a result, the numerical functions return the Hessian of the negative log-likelihood function (which is the observed Fisher Information matrix).

Usage

MARSSFisherI(MLEobj, method = c("Harvey1989", "fdHess", "optim"))

Arguments

MLEobj: An object of class marssMLE. This object must have a $par element containing MLE parameter estimates from e.g. MARSSkem().
method: The method to use for computing the observed Fisher Information matrix. Options are "Harvey1989" to use the Harvey (1989) recursion, which is an analytical solution, "fdHess" or "optim" which are two numerical methods. Although 'optim' can be passed to the function, 'fdHess' is used for all numerical estimates used in the MARSS package.

Details

Method 'fdHess' uses fdHess() from package nlme to numerically estimate the Hessian of the negative log-likelihood function at the MLEs. Method 'optim' uses optim() with hessian=TRUE and list(maxit=0) to ensure that the Hessian is computed at the values in the par element of the MLE object. The par element of the marssMLE object is the MLE.

Method 'Harvey1989' (the default) uses the recursion in Harvey (1989) to compute the observed Fisher Information of a MARSS model analytically. See Holmes (2016c) for a discussion of the Harvey (1989) algorithm and see Holmes (2017) on how to implement the algorithm for MARSS models with linear constraints (the type of MARSS models that the MARSS R package addresses).

There has been research on computing the observed Fisher Information matrix from the derivatives used by EM algorithms (discussed in Holmes (2016a, 2016b)), for example Louis (1982). Unfortunately, the EM algorithm used in the MARSS package is for time series data and the temporal correlation must be dealt with, e.g. Duan & Fulop (2011). Oakes (1999) has an approach that only involves derivatives of $\textrm{E}[LL(\Theta)|\mathbf{y},\Theta']$ but one of the derivatives will be the derivative of the $\textrm{E}[\mathbf{X}|\mathbf{y},\Theta']$ with respect to $\Theta'$. It is not clear how to do that derivative. Moon-Ho, Shumway and Ombao (2006) suggest (page 157) that this derivative is hard to compute.

Value

Returns the observed Fisher Information matrix.

Author

Eli Holmes, NOAA, Seattle, USA.

References

Harvey, A. C. (1989) Section 3.4.5 (Information matrix) in Forecasting, structural time series models and the Kalman filter. Cambridge University Press, Cambridge, UK.

See also J. E. Cavanaugh and R. H. Shumway (1996) On computing the expected Fisher information matrix for state-space model parameters. Statistics & Probability Letters 26: 347-355. This paper discusses the Harvey (1989) recursion (and proposes an alternative).

Holmes, E. E. 2016a. Notes on computing the Fisher Information matrix for MARSS models. Part I Background. Technical Report. https://doi.org/10.13140/RG.2.2.27306.11204/1 Notes

Holmes, E. E. 2016b. Notes on computing the Fisher Information matrix for MARSS models. Part II Louis 1982. Technical Report. https://doi.org/10.13140/RG.2.2.35694.72000 Notes

Holmes, E. E. 2016c. Notes on computing the Fisher Information matrix for MARSS models. Part III Overview of Harvey 1989. https://eeholmes.github.io/posts/2016-6-16-FI-recursion-3/

Holmes, E. E. 2017. Notes on computing the Fisher Information matrix for MARSS models. Part IV Implementing the Recursion in Harvey 1989. https://eeholmes.github.io/posts/2017-5-31-FI-recursion-4/

Duan, J. C. and A. Fulop. (2011) A stable estimator of the information matrix under EM for dependent data. Statistics and Computing 21: 83-91

Louis, T. A. 1982. Finding the observed information matrix when using the EM algorithm. Journal of the Royal Statistical Society. Series B (Methodological). 44: 226-233.

Oakes, D. 1999. Direct calculation of the information matrix via the EM algorithm. Journal of the Royal Statistical Society. Series B (Methodological). 61: 479-482.

Moon-Ho, R. H., R. H. Shumway, and Ombao 2006. The state-space approach to modeling dynamic processes. Chapter 7 in Models for Intensive Longitudinal Data. Oxford University Press.

Examples

dat <- t(harborSeal)
dat <- dat[2:4, ]
MLEobj <- MARSS(dat, model=list(Z=matrix(1,3,1), R="diagonal and equal"))
#> Success! abstol and log-log tests passed at 29 iterations.
#> Alert: conv.test.slope.tol is 0.5.
#> Test with smaller values (<0.1) to ensure convergence.
#> 
#> MARSS fit is
#> Estimation method: kem 
#> Convergence test: conv.test.slope.tol = 0.5, abstol = 0.001
#> Estimation converged in 29 iterations. 
#> Log-likelihood: 11.03248 
#> AIC: -10.06496   AICc: -8.277722   
#>  
#>                    Estimate
#> A.OlympicPeninsula -0.71799
#> A.StraitJuanDeFuca -1.54057
#> R.diag              0.02644
#> U.U                 0.06287
#> Q.Q                 0.00972
#> x0.x0               7.40711
#> Initial states (x0) defined at t=0
#> 
#> Standard errors have not been calculated. 
#> Use MARSSparamCIs to compute CIs and bias estimates.
#> 
MARSSFisherI(MLEobj)
#>                    A.OlympicPeninsula A.StraitJuanDeFuca      R.diag        U.U
#> A.OlympicPeninsula         369.565861        -173.563692    15.59539   30.35953
#> A.StraitJuanDeFuca        -173.563692         421.146898   -60.01054   42.14383
#> R.diag                      15.595391         -60.010538 28688.85792 -181.36337
#> U.U                         30.359534          42.143835  -181.36337 2507.45187
#> Q.Q                        -42.415845         163.214745  7766.17926  493.26631
#> x0.x0                        1.226226           2.962416     5.07240  102.86384
#>                            Q.Q      x0.x0
#> A.OlympicPeninsula   -42.41585   1.226226
#> A.StraitJuanDeFuca   163.21474   2.962416
#> R.diag              7766.17926   5.072400
#> U.U                  493.26631 102.863835
#> Q.Q                31226.28772 -13.795750
#> x0.x0                -13.79575  46.463576
MARSSFisherI(MLEobj, method="fdHess")
#>                    A.OlympicPeninsula A.StraitJuanDeFuca      R.diag       U.U
#> A.OlympicPeninsula         369.567481        -173.564194   -63.59872   30.3338
#> A.StraitJuanDeFuca        -173.564194         421.147436    -3.98411   42.1302
#> R.diag                     -63.598723          -3.984110 28441.35002 -188.4002
#> U.U                         30.333805          42.130202  -188.40022 2508.4098
#> Q.Q                        105.680363         -47.988282  8474.13195  512.7219
#> x0.x0                        1.226384           2.962429   -75.12438  102.8606
#>                            Q.Q      x0.x0
#> A.OlympicPeninsula   105.68036   1.226384
#> A.StraitJuanDeFuca   -47.98828   2.962429
#> R.diag              8474.13195 -75.124382
#> U.U                  512.72193 102.860563
#> Q.Q                28908.51992 205.212282
#> x0.x0                205.21228  46.463580