Loading [MathJax]/jax/output/HTML-CSS/jax.js

Stationarity & introductory functions

FISH 507 – Applied Time Series Analysis

Mark Scheuerell

7 Jan 2021

Topics for today

Characteristics of time series

  • Expectation, mean & variance
  • Covariance & correlation
  • Stationarity
  • Autocovariance & autocorrelation
  • Correlograms

White noise

Random walks

Backshift & difference operators

Expectation & the mean

The expectation (E) of a variable is its mean value in the population

E(x) mean of x=μ

We can estimate μ from a sample as

m=1NNi=1xi

Variance

E([xμ]2) expected deviations of x about μ

E([xμ]2) variance of x=σ2

We can estimate σ2 from a sample as

s2=1N1Ni=1(xim)2

Covariance

If we have two variables, x and y, we can generalize variance

σ2=E([xiμ][xiμ])

into covariance

γx,y=E([xiμx][yiμy])

Covariance

If we have two variables, x and y, we can generalize variance

σ2=E([xiμ][xiμ])

into covariance

γx,y=E([xiμx][yiμy])

We can estimate γx,y from a sample as

Cov(x,y)=1N1Ni=1(ximx)(yimy)

Graphical example of covariance

Graphical example of covariance

Graphical example of covariance

Correlation

Correlation is a dimensionless measure of the linear association between 2 variables, x & y

It is simply the covariance standardized by the standard deviations

ρx,y=γx,yσxσy

1<ρx,y<1

Correlation

Correlation is a dimensionless measure of the linear association between 2 variables x & y

It is simply the covariance standardized by the standard deviations

ρx,y=γx,yσxσy

We can estimate ρx,y from a sample as

Cor(x,y)=Cov(x,y)sxsy

Stationarity & the mean

Consider a single value, xt

Stationarity & the mean

Consider a single value, xt

E(xt) is taken across an ensemble of all possible time series

Stationarity & the mean

Stationarity & the mean

Our single realization is our estimate!

Our single realization is our estimate!

Stationarity & the mean

If E(xt) is constant across time, we say the time series is stationary in the mean

Stationarity of time series

Stationarity is a convenient assumption that allows us to describe the statistical properties of a time series.

In general, a time series is said to be stationary if there is

  1. no systematic change in the mean or variance
  2. no systematic trend
  3. no periodic variations or seasonality

Identifying stationarity

Identifying stationarity

Our eyes are really bad at identifying stationarity, so we will learn some tools to help us

Autocovariance function (ACVF)

For stationary ts, we define the autocovariance function (γk) as

γk=E([xtμ][xt+kμ])

which means that

γ0=E([xtμ][xtμ])=σ2

Autocovariance function (ACVF)

For stationary ts, we define the autocovariance function (γk) as

γk=E([xtμ][xt+kμ])

“Smooth” time series have large ACVF for large k

“Choppy” time series have ACVF near 0 for small k

Autocovariance function (ACVF)

For stationary ts, we define the autocovariance function (γk) as

γk=E([xtμ][xt+kμ])

We can estimate γk from a sample as

ck=1NNkt=1(xtm)(xt+km)

Autocorrelation function (ACF)

The autocorrelation function (ACF) is simply the ACVF normalized by the variance

ρk=γkσ2=γkγ0

The ACF measures the correlation of a time series against a time-shifted version of itself

Autocorrelation function (ACF)

The autocorrelation function (ACF) is simply the ACVF normalized by the variance

ρk=γkσ2=γkγ0

The ACF measures the correlation of a time series against a time-shifted version of itself

We can estimate ACF from a sample as

rk=ckc0

Properties of the ACF

The ACF has several important properties:

  • 1rk1
  • rk=rk
  • rk of a periodic function is itself periodic
  • rk for the sum of 2 independent variables is the sum of rk for each of them

The correlogram

Graphical output for the ACF

Graphical output for the ACF

The correlogram

The ACF at lag = 0 is always 1

The ACF at lag = 0 is always 1

The correlogram

Approximate confidence intervals

Approximate confidence intervals

Estimating the ACF in R

acf(ts_object)

ACF for deterministic forms

ACF for deterministic forms

ACF for deterministic forms

ACF for deterministic forms

Induced autocorrelation

Recall the transitive property, whereby

If A=B and B=C, then A=C

Induced autocorrelation

Recall the transitive property, whereby

If A=B and B=C, then A=C

which suggests that

If xy and yz, then xz

Induced autocorrelation

Recall the transitive property, whereby

If A=B and B=C, then A=C

which suggests that

If xy and yz, then xz

and thus

If xtxt+1 and xt+1xt+2, then xtxt+2

Partial autocorrelation funcion (PACF)

The partial autocorrelation function (ϕk) measures the correlation between a series xt and xt+k with the linear dependence of {xt1,xt2,,xtk1} removed

Partial autocorrelation funcion (PACF)

The partial autocorrelation function (ϕk) measures the correlation between a series xt and xt+k with the linear dependence of {xt1,xt2,,xtk1} removed

We can estimate ϕk from a sample as

ϕk={Cor(x1,x0)=ρ1if k=1Cor(xkxk1k,x0xk10)if k2

xk1k=β1xk1+β2xk2++βk1x1

xk10=β1x1+β2x2++βk1xk1

Lake Washington phytoplankton

Lake Washington phytoplankton

Autocorrelation

Autocorrelation

Lake Washington phytoplankton

Partial autocorrelation

Partial autocorrelation

ACF & PACF in model selection

We will see that the ACF & PACF are very useful for identifying the orders of ARMA models

Cross-covariance function (CCVF)

Often we want to look for relationships between 2 different time series

We can extend the notion of covariance to cross-covariance

Cross-covariance function (CCVF)

Often we want to look for relationships between 2 different time series

We can extend the notion of covariance to cross-covariance

We can estimate the CCVF (gx,yk) from a sample as

gx,yk=1NNkt=1(xtmx)(yt+kmy)

Cross-correlation function (CCF)

The cross-correlation function is the CCVF normalized by the standard deviations of x & y

rx,yk=gx,yksxsy

Just as with other measures of correlation

1rx,yk1

Estimating the CCF in R

ccf(x, y)

Note: the lag k value returned by ccf(x, y) is the correlation between x[t+k] and y[t]

In an explanatory context, we often think of y=f(x), so it’s helpful to use ccf(y, x) and only consider positive lags

Example of cross-correlation

SOME SIMPLE MODELS

White noise (WN)

A time series {wt} is discrete white noise if its values are

  1. independent

  2. identically distributed with a mean of zero

White noise (WN)

A time series {wt} is discrete white noise if its values are

  1. independent

  2. identically distributed with a mean of zero

Note that distributional form for {wt} is flexible

White noise (WN)

$w_t = 2e_t - 1; e_t \sim \text{Bernoulli}(0.5)$

wt=2et1;etBernoulli(0.5)

Gaussian white noise

We often assume so-called Gaussian white noise, whereby

wtN(0,σ2)

Gaussian white noise

We often assume so-called Gaussian white noise, whereby

wtN(0,σ2)

and the following apply as well

    autocovariance:  γk={σ2if k=00if k1

    autocorrelation:   ρk={1if k=00if k1

Gaussian white noise

$w_t \sim \text{N}(0,1)$

wtN(0,1)

Random walk (RW)

A time series {xt} is a random walk if

  1. xt=xt1+wt

  2. wt is white noise

Random walk (RW)

The following apply to random walks

    mean:   μx=0

    autocovariance:   γk(t)=tσ2

    autocorrelation:   ρk(t)=tσ2tσ2(t+k)σ2

Random walk (RW)

The following apply to random walks

    mean:   μx=0

    autocovariance:   γk(t)=tσ2

    autocorrelation:   ρk(t)=tσ2tσ2(t+k)σ2

Note: Random walks are not stationary

Random walk (RW)

$x_t = x_{t-1} + w_t; w_t \sim \text{N}(0,1)$

xt=xt1+wt;wtN(0,1)

SOME IMPORTANT OPERATORS

The backshift shift operator

The backshift shift operator (B) is an important function in time series analysis, which we define as

Bxt=xt1

or more generally as

Bkxt=xtk

The backshift shift operator

For example, a random walk with

xt=xt1+wt

can be written as

xt=Bxt+wtxtBxt=wt(1B)xt=wtxt=(1B)1wt

The difference operator

The difference operator () is another important function in time series analysis, which we define as

xt=xtxt1

The difference operator

The difference operator () is another important function in time series analysis, which we define as

xt=xtxt1

For example, first-differencing a random walk yields white noise

xt=xt1+wtxtxt1=xt1+wtxt1xtxt1=wt

The difference operator

The difference operator and the backshift operator are related

k=(1B)k

The difference operator

The difference operator and the backshift operator are related

k=(1B)k

For example

xt=(1B)xtxtxt1=xtBxtxtxt1=xtxt1

Differencing to remove a trend

Differencing is a simple means for removing a trend

The 1st-difference removes a linear trend

A 2nd-difference will remove a quadratic trend

Differencing to remove a trend

Differencing to remove seasonality

Differencing is a simple means for removing a seasonal effect

Using a 1st-difference with k=period removes both trend & seasonal effects

Differencing to remove seasonality

Differencing to remove a trend in R

We can use diff() to easily compute differences

diff(x,
     lag,
     differences
     )

Differencing to remove a trend in R

diff(x,
     lag,
     differences
     )

lag (h) specifies th

lag = 1 (default) is for non-seasonal data

lag = 4 would work for quarterly data or

lag = 12 for monthly data

Differencing to remove a trend in R

diff(x,
     lag,
     differences
     )

differences is the number of differencing operations

differences = 1 (default) is for a linear trend

differences = 2 is for a quadratic trend

Topics for today

Characteristics of time series

  • Expectation, mean & variance
  • Covariance & correlation
  • Stationarity
  • Autocovariance & autocorrelation
  • Correlograms

White noise

Random walks

Backshift & difference operators