Characteristics of time series
- Expectation, mean & variance
- Covariance & correlation
- Stationarity
- Autocovariance & autocorrelation
- Correlograms
White noise
Random walks
Backshift & difference operators
Mark Scheuerell
7 Jan 2021
Characteristics of time series
White noise
Random walks
Backshift & difference operators
The expectation (E) of a variable is its mean value in the population
E(x)≡ mean of x=μ
We can estimate μ from a sample as
m=1NN∑i=1xi
E([x−μ]2)≡ expected deviations of x about μ
E([x−μ]2)≡ variance of x=σ2
We can estimate σ2 from a sample as
s2=1N−1N∑i=1(xi−m)2
If we have two variables, x and y, we can generalize variance
σ2=E([xi−μ][xi−μ])
into covariance
γx,y=E([xi−μx][yi−μy])
If we have two variables, x and y, we can generalize variance
σ2=E([xi−μ][xi−μ])
into covariance
γx,y=E([xi−μx][yi−μy])
We can estimate γx,y from a sample as
Cov(x,y)=1N−1N∑i=1(xi−mx)(yi−my)
Correlation is a dimensionless measure of the linear association between 2 variables, x & y
It is simply the covariance standardized by the standard deviations
ρx,y=γx,yσxσy
−1<ρx,y<1
Correlation is a dimensionless measure of the linear association between 2 variables x & y
It is simply the covariance standardized by the standard deviations
ρx,y=γx,yσxσy
We can estimate ρx,y from a sample as
Cor(x,y)=Cov(x,y)sxsy
Consider a single value, xt
Consider a single value, xt
E(xt) is taken across an ensemble of all possible time series
Our single realization is our estimate!
If E(xt) is constant across time, we say the time series is stationary in the mean
Stationarity is a convenient assumption that allows us to describe the statistical properties of a time series.
In general, a time series is said to be stationary if there is
Our eyes are really bad at identifying stationarity, so we will learn some tools to help us
For stationary ts, we define the autocovariance function (γk) as
γk=E([xt−μ][xt+k−μ])
which means that
γ0=E([xt−μ][xt−μ])=σ2
For stationary ts, we define the autocovariance function (γk) as
γk=E([xt−μ][xt+k−μ])
“Smooth” time series have large ACVF for large k
“Choppy” time series have ACVF near 0 for small k
For stationary ts, we define the autocovariance function (γk) as
γk=E([xt−μ][xt+k−μ])
We can estimate γk from a sample as
ck=1NN−k∑t=1(xt−m)(xt+k−m)
The autocorrelation function (ACF) is simply the ACVF normalized by the variance
ρk=γkσ2=γkγ0
The ACF measures the correlation of a time series against a time-shifted version of itself
The autocorrelation function (ACF) is simply the ACVF normalized by the variance
ρk=γkσ2=γkγ0
The ACF measures the correlation of a time series against a time-shifted version of itself
We can estimate ACF from a sample as
rk=ckc0
The ACF has several important properties:
Graphical output for the ACF
The ACF at lag = 0 is always 1
Approximate confidence intervals
acf(ts_object)
Recall the transitive property, whereby
If A=B and B=C, then A=C
Recall the transitive property, whereby
If A=B and B=C, then A=C
which suggests that
If x∝y and y∝z, then x∝z
Recall the transitive property, whereby
If A=B and B=C, then A=C
which suggests that
If x∝y and y∝z, then x∝z
and thus
If xt∝xt+1 and xt+1∝xt+2, then xt∝xt+2
The partial autocorrelation function (ϕk) measures the correlation between a series xt and xt+k with the linear dependence of {xt−1,xt−2,…,xt−k−1} removed
The partial autocorrelation function (ϕk) measures the correlation between a series xt and xt+k with the linear dependence of {xt−1,xt−2,…,xt−k−1} removed
We can estimate ϕk from a sample as
ϕk={Cor(x1,x0)=ρ1if k=1Cor(xk−xk−1k,x0−xk−10)if k≥2
xk−1k=β1xk−1+β2xk−2+⋯+βk−1x1
xk−10=β1x1+β2x2+⋯+βk−1xk−1
Autocorrelation
Partial autocorrelation
We will see that the ACF & PACF are very useful for identifying the orders of ARMA models
Often we want to look for relationships between 2 different time series
We can extend the notion of covariance to cross-covariance
Often we want to look for relationships between 2 different time series
We can extend the notion of covariance to cross-covariance
We can estimate the CCVF (gx,yk) from a sample as
gx,yk=1NN−k∑t=1(xt−mx)(yt+k−my)
The cross-correlation function is the CCVF normalized by the standard deviations of x & y
rx,yk=gx,yksxsy
Just as with other measures of correlation
−1≤rx,yk≤1
ccf(x, y)
Note: the lag k
value returned by ccf(x, y)
is the correlation between x[t+k]
and y[t]
In an explanatory context, we often think of y=f(x), so it’s helpful to use ccf(y, x)
and only consider positive lags
SOME SIMPLE MODELS
A time series {wt} is discrete white noise if its values are
independent
identically distributed with a mean of zero
A time series {wt} is discrete white noise if its values are
independent
identically distributed with a mean of zero
Note that distributional form for {wt} is flexible
wt=2et−1;et∼Bernoulli(0.5)
We often assume so-called Gaussian white noise, whereby
wt∼N(0,σ2)
We often assume so-called Gaussian white noise, whereby
wt∼N(0,σ2)
and the following apply as well
autocovariance: γk={σ2if k=00if k≥1
autocorrelation: ρk={1if k=00if k≥1
wt∼N(0,1)
A time series {xt} is a random walk if
xt=xt−1+wt
wt is white noise
The following apply to random walks
mean: μx=0
autocovariance: γk(t)=tσ2
autocorrelation: ρk(t)=tσ2√tσ2(t+k)σ2
The following apply to random walks
mean: μx=0
autocovariance: γk(t)=tσ2
autocorrelation: ρk(t)=tσ2√tσ2(t+k)σ2
Note: Random walks are not stationary
xt=xt−1+wt;wt∼N(0,1)
SOME IMPORTANT OPERATORS
The backshift shift operator (B) is an important function in time series analysis, which we define as
Bxt=xt−1
or more generally as
Bkxt=xt−k
For example, a random walk with
xt=xt−1+wt
can be written as
xt=Bxt+wtxt−Bxt=wt(1−B)xt=wtxt=(1−B)−1wt
The difference operator (∇) is another important function in time series analysis, which we define as
∇xt=xt−xt−1
The difference operator (∇) is another important function in time series analysis, which we define as
∇xt=xt−xt−1
For example, first-differencing a random walk yields white noise
∇xt=xt−1+wtxt−xt−1=xt−1+wt−xt−1xt−xt−1=wt
The difference operator and the backshift operator are related
∇k=(1−B)k
The difference operator and the backshift operator are related
∇k=(1−B)k
For example
∇xt=(1−B)xtxt−xt−1=xt−Bxtxt−xt−1=xt−xt−1
Differencing is a simple means for removing a trend
The 1st-difference removes a linear trend
A 2nd-difference will remove a quadratic trend
Differencing is a simple means for removing a seasonal effect
Using a 1st-difference with k=period removes both trend & seasonal effects
We can use diff()
to easily compute differences
diff(x, lag, differences )
diff(x, lag, differences )
lag
(h) specifies t−h
lag = 1
(default) is for non-seasonal data
lag = 4
would work for quarterly data or
lag = 12
for monthly data
diff(x, lag, differences )
differences
is the number of differencing operations
differences = 1
(default) is for a linear trend
differences = 2
is for a quadratic trend
Characteristics of time series
White noise
Random walks
Backshift & difference operators