Autoregressive Moving Average (ARMA): Sunspots data¶
[1]:
%matplotlib inline
[2]:
import numpy as np
from scipy import stats
import pandas as pd
import matplotlib.pyplot as plt
import statsmodels.api as sm
[3]:
from statsmodels.graphics.api import qqplot
Sunspots Data¶
[4]:
print(sm.datasets.sunspots.NOTE)
::
Number of Observations - 309 (Annual 1700 - 2008)
Number of Variables - 1
Variable name definitions::
SUNACTIVITY - Number of sunspots for each year
The data file contains a 'YEAR' variable that is not returned by load.
[5]:
dta = sm.datasets.sunspots.load_pandas().data
[6]:
dta.index = pd.Index(sm.tsa.datetools.dates_from_range('1700', '2008'))
del dta["YEAR"]
[7]:
dta.plot(figsize=(12,8));

[8]:
fig = plt.figure(figsize=(12,8))
ax1 = fig.add_subplot(211)
fig = sm.graphics.tsa.plot_acf(dta.values.squeeze(), lags=40, ax=ax1)
ax2 = fig.add_subplot(212)
fig = sm.graphics.tsa.plot_pacf(dta, lags=40, ax=ax2)

[9]:
arma_mod20 = sm.tsa.ARMA(dta, (2,0)).fit(disp=False)
print(arma_mod20.params)
/usr/lib/python3/dist-packages/statsmodels/tsa/base/tsa_model.py:159: ValueWarning: No frequency information was provided, so inferred frequency A-DEC will be used.
warnings.warn('No frequency information was'
const 49.659523
ar.L1.SUNACTIVITY 1.390656
ar.L2.SUNACTIVITY -0.688571
dtype: float64
[10]:
arma_mod30 = sm.tsa.ARMA(dta, (3,0)).fit(disp=False)
/usr/lib/python3/dist-packages/statsmodels/tsa/base/tsa_model.py:159: ValueWarning: No frequency information was provided, so inferred frequency A-DEC will be used.
warnings.warn('No frequency information was'
[11]:
print(arma_mod20.aic, arma_mod20.bic, arma_mod20.hqic)
2622.636338065289 2637.5697031728796 2628.606725910535
[12]:
print(arma_mod30.params)
const 49.749898
ar.L1.SUNACTIVITY 1.300810
ar.L2.SUNACTIVITY -0.508093
ar.L3.SUNACTIVITY -0.129650
dtype: float64
[13]:
print(arma_mod30.aic, arma_mod30.bic, arma_mod30.hqic)
2619.403628696713 2638.070335081202 2626.8666135032704
- Does our model obey the theory?
[14]:
sm.stats.durbin_watson(arma_mod30.resid.values)
[14]:
1.956480768899294
[15]:
fig = plt.figure(figsize=(12,8))
ax = fig.add_subplot(111)
ax = arma_mod30.resid.plot(ax=ax);

[16]:
resid = arma_mod30.resid
[17]:
stats.normaltest(resid)
[17]:
NormaltestResult(statistic=49.84503952862609, pvalue=1.5006768784454912e-11)
[18]:
fig = plt.figure(figsize=(12,8))
ax = fig.add_subplot(111)
fig = qqplot(resid, line='q', ax=ax, fit=True)

[19]:
fig = plt.figure(figsize=(12,8))
ax1 = fig.add_subplot(211)
fig = sm.graphics.tsa.plot_acf(resid.values.squeeze(), lags=40, ax=ax1)
ax2 = fig.add_subplot(212)
fig = sm.graphics.tsa.plot_pacf(resid, lags=40, ax=ax2)

[20]:
r,q,p = sm.tsa.acf(resid.values.squeeze(), fft=True, qstat=True)
data = np.c_[range(1,41), r[1:], q, p]
table = pd.DataFrame(data, columns=['lag', "AC", "Q", "Prob(>Q)"])
print(table.set_index('lag'))
AC Q Prob(>Q)
lag
1.0 0.009179 0.026287 8.712021e-01
2.0 0.041793 0.573041 7.508716e-01
3.0 -0.001335 0.573601 9.024484e-01
4.0 0.136089 6.408916 1.706207e-01
5.0 0.092468 9.111821 1.046863e-01
6.0 0.091948 11.793233 6.674372e-02
7.0 0.068748 13.297188 6.519011e-02
8.0 -0.015020 13.369216 9.976172e-02
9.0 0.187592 24.641898 3.393924e-03
10.0 0.213718 39.321988 2.229480e-05
11.0 0.201082 52.361135 2.344952e-07
12.0 0.117182 56.804187 8.574263e-08
13.0 -0.014055 56.868324 1.893903e-07
14.0 0.015398 56.945563 3.997661e-07
15.0 -0.024967 57.149318 7.741472e-07
16.0 0.080916 59.296767 6.872169e-07
17.0 0.041138 59.853736 1.110945e-06
18.0 -0.052021 60.747426 1.548433e-06
19.0 0.062496 62.041689 1.831645e-06
20.0 -0.010301 62.076976 3.381245e-06
21.0 0.074453 63.926652 3.193588e-06
22.0 0.124955 69.154771 8.978353e-07
23.0 0.093162 72.071035 5.799780e-07
24.0 -0.082152 74.346688 4.713014e-07
25.0 0.015695 74.430043 8.289039e-07
26.0 -0.025037 74.642902 1.367283e-06
27.0 -0.125861 80.041149 3.722564e-07
28.0 0.053225 81.009981 4.716277e-07
29.0 -0.038693 81.523807 6.916630e-07
30.0 -0.016904 81.622226 1.151661e-06
31.0 -0.019296 81.750938 1.868765e-06
32.0 0.104990 85.575066 8.927952e-07
33.0 0.040086 86.134568 1.247508e-06
34.0 0.008829 86.161811 2.047823e-06
35.0 0.014588 86.236448 3.263805e-06
36.0 -0.119329 91.248900 1.084453e-06
37.0 -0.036666 91.723869 1.521921e-06
38.0 -0.046193 92.480518 1.938732e-06
39.0 -0.017768 92.592887 2.990674e-06
40.0 -0.006220 92.606710 4.696977e-06
- This indicates a lack of fit.
- In-sample dynamic prediction. How good does our model do?
[21]:
predict_sunspots = arma_mod30.predict('1990', '2012', dynamic=True)
print(predict_sunspots)
1990-12-31 167.047401
1991-12-31 140.992972
1992-12-31 94.859076
1993-12-31 46.860863
1994-12-31 11.242552
1995-12-31 -4.721323
1996-12-31 -1.166941
1997-12-31 16.185657
1998-12-31 39.021841
1999-12-31 59.449824
2000-12-31 72.170091
2001-12-31 75.376732
2002-12-31 70.436411
2003-12-31 60.731545
2004-12-31 50.201761
2005-12-31 42.075996
2006-12-31 38.114257
2007-12-31 38.454611
2008-12-31 41.963779
2009-12-31 46.869245
2010-12-31 51.423214
2011-12-31 54.399670
2012-12-31 55.321643
Freq: A-DEC, dtype: float64
[22]:
fig, ax = plt.subplots(figsize=(12, 8))
ax = dta.loc['1950':].plot(ax=ax)
fig = arma_mod30.plot_predict('1990', '2012', dynamic=True, ax=ax, plot_insample=False)

[23]:
def mean_forecast_err(y, yhat):
return y.sub(yhat).mean()
[24]:
mean_forecast_err(dta.SUNACTIVITY, predict_sunspots)
[24]:
5.636994490394947
Exercise: Can you obtain a better fit for the Sunspots model? (Hint: sm.tsa.AR has a method select_order)¶
Simulated ARMA(4,1): Model Identification is Difficult¶
[25]:
from statsmodels.tsa.arima_process import ArmaProcess
[26]:
np.random.seed(1234)
# include zero-th lag
arparams = np.array([1, .75, -.65, -.55, .9])
maparams = np.array([1, .65])
Let’s make sure this model is estimable.
[27]:
arma_t = ArmaProcess(arparams, maparams)
[28]:
arma_t.isinvertible
[28]:
True
[29]:
arma_t.isstationary
[29]:
False
- What does this mean?
[30]:
fig = plt.figure(figsize=(12,8))
ax = fig.add_subplot(111)
ax.plot(arma_t.generate_sample(nsample=50));

[31]:
arparams = np.array([1, .35, -.15, .55, .1])
maparams = np.array([1, .65])
arma_t = ArmaProcess(arparams, maparams)
arma_t.isstationary
[31]:
True
[32]:
arma_rvs = arma_t.generate_sample(nsample=500, burnin=250, scale=2.5)
[33]:
fig = plt.figure(figsize=(12,8))
ax1 = fig.add_subplot(211)
fig = sm.graphics.tsa.plot_acf(arma_rvs, lags=40, ax=ax1)
ax2 = fig.add_subplot(212)
fig = sm.graphics.tsa.plot_pacf(arma_rvs, lags=40, ax=ax2)

- For mixed ARMA processes the Autocorrelation function is a mixture of exponentials and damped sine waves after (q-p) lags.
- The partial autocorrelation function is a mixture of exponentials and dampened sine waves after (p-q) lags.
[34]:
arma11 = sm.tsa.ARMA(arma_rvs, (1,1)).fit(disp=False)
resid = arma11.resid
r,q,p = sm.tsa.acf(resid, fft=True, qstat=True)
data = np.c_[range(1,41), r[1:], q, p]
table = pd.DataFrame(data, columns=['lag', "AC", "Q", "Prob(>Q)"])
print(table.set_index('lag'))
AC Q Prob(>Q)
lag
1.0 0.254921 32.687677 1.082212e-08
2.0 -0.172416 47.670746 4.450707e-11
3.0 -0.420945 137.159392 1.548466e-29
4.0 -0.046875 138.271301 6.617704e-29
5.0 0.103240 143.675907 2.958722e-29
6.0 0.214864 167.132998 1.823719e-33
7.0 -0.000889 167.133400 1.009206e-32
8.0 -0.045418 168.185752 3.094836e-32
9.0 -0.061445 170.115802 5.837217e-32
10.0 0.034623 170.729854 1.958737e-31
11.0 0.006351 170.750556 8.267056e-31
12.0 -0.012882 170.835909 3.220234e-30
13.0 -0.053959 172.336547 6.181197e-30
14.0 -0.016606 172.478964 2.160215e-29
15.0 0.051742 173.864487 4.089547e-29
16.0 0.078917 177.094280 3.217936e-29
17.0 -0.001834 177.096028 1.093168e-28
18.0 -0.101604 182.471937 3.103824e-29
19.0 -0.057342 184.187771 4.624067e-29
20.0 0.026975 184.568285 1.235671e-28
21.0 0.062359 186.605962 1.530259e-28
22.0 -0.009400 186.652364 4.548196e-28
23.0 -0.068037 189.088184 4.562011e-28
24.0 -0.035566 189.755201 9.901096e-28
25.0 0.095679 194.592622 3.354290e-28
26.0 0.065650 196.874876 3.487623e-28
27.0 -0.018404 197.054612 9.008749e-28
28.0 -0.079244 200.394008 5.773714e-28
29.0 0.008499 200.432501 1.541386e-27
30.0 0.053372 201.953775 2.133192e-27
31.0 0.074816 204.949394 1.550162e-27
32.0 -0.071187 207.667241 1.262289e-27
33.0 -0.088145 211.843155 5.480817e-28
34.0 -0.025283 212.187449 1.215228e-27
35.0 0.125690 220.714897 8.231613e-29
36.0 0.142724 231.734118 1.923082e-30
37.0 0.095768 236.706160 5.937782e-31
38.0 -0.084744 240.607802 2.890887e-31
39.0 -0.150126 252.878983 3.963000e-33
40.0 -0.083767 256.707740 1.996172e-33
[35]:
arma41 = sm.tsa.ARMA(arma_rvs, (4,1)).fit(disp=False)
resid = arma41.resid
r,q,p = sm.tsa.acf(resid, fft=True, qstat=True)
data = np.c_[range(1,41), r[1:], q, p]
table = pd.DataFrame(data, columns=['lag', "AC", "Q", "Prob(>Q)"])
print(table.set_index('lag'))
AC Q Prob(>Q)
lag
1.0 -0.007889 0.031302 0.859567
2.0 0.004132 0.039907 0.980244
3.0 0.018103 0.205418 0.976710
4.0 -0.006760 0.228543 0.993948
5.0 0.018120 0.395028 0.995465
6.0 0.050688 1.700454 0.945086
7.0 0.010252 1.753962 0.972196
8.0 -0.011206 1.818023 0.986091
9.0 0.020292 2.028522 0.991008
10.0 0.001029 2.029065 0.996113
11.0 -0.014035 2.130174 0.997984
12.0 -0.023858 2.422929 0.998427
13.0 -0.002108 2.425220 0.999339
14.0 -0.018783 2.607432 0.999590
15.0 0.011316 2.673700 0.999805
16.0 0.042159 3.595423 0.999443
17.0 0.007943 3.628208 0.999734
18.0 -0.074311 6.503855 0.993686
19.0 -0.023379 6.789068 0.995256
20.0 0.002398 6.792074 0.997313
21.0 0.000487 6.792198 0.998516
22.0 0.017953 6.961436 0.999024
23.0 -0.038576 7.744467 0.998744
24.0 -0.029816 8.213250 0.998859
25.0 0.077850 11.415822 0.990675
26.0 0.040408 12.280446 0.989479
27.0 -0.018612 12.464274 0.992262
28.0 -0.014764 12.580184 0.994586
29.0 0.017649 12.746188 0.996111
30.0 -0.005486 12.762261 0.997504
31.0 0.058256 14.578540 0.994614
32.0 -0.040840 15.473079 0.993887
33.0 -0.019493 15.677305 0.995393
34.0 0.037269 16.425462 0.995214
35.0 0.086212 20.437443 0.976296
36.0 0.041271 21.358841 0.974774
37.0 0.078704 24.716872 0.938949
38.0 -0.029729 25.197050 0.944895
39.0 -0.078397 28.543385 0.891179
40.0 -0.014466 28.657575 0.909268
Exercise: How good of in-sample prediction can you do for another series, say, CPI¶
[36]:
macrodta = sm.datasets.macrodata.load_pandas().data
macrodta.index = pd.Index(sm.tsa.datetools.dates_from_range('1959Q1', '2009Q3'))
cpi = macrodta["cpi"]
Hint:¶
[37]:
fig = plt.figure(figsize=(12,8))
ax = fig.add_subplot(111)
ax = cpi.plot(ax=ax);
ax.legend();

P-value of the unit-root test, resoundingly rejects the null of a unit-root.
[38]:
print(sm.tsa.adfuller(cpi)[1])
0.990432818833742