Time Series Analysis

Time Series Analysis
Univariate and Multivariate Methods

SECOND EDITION
William W. S. Wei
Department of Statistics
The Fox School of Business and Management
Temple University
Boston San Francisco New York

London Toronto Sydney Tokyo Singapore Madrid
Mexico City Munich Patis Cape Town Hong Kong Montreal
Publisher
Executive Editor
Assistant Editor
Managing Editor
Production Supervisor
Marketing Manager
Marketing Assistant
Senior Designer
Prepress Supervisor
Manufacturing Buyer
Composition, Production
Coordination, and Text Design
IlLustrations
Cover Designer
Greg Tobin
Deirdre Lynch
Sara Oliver
Ron Hampton
Kathleen A. Manley
Phyllis Hubbard
Celena Carr
Barbara Atkinson
CaroIine Fell
Evelyn Beaton
Progressive Information
Technologies
Jim McLaughlin
Leslie Haimes
The cover image appears as Figure 17.1 within the text and embodies all the concepts of time series analysis. For instance, the thin solid line represents stationary series as discussed in Chapters 2 and 3; the heavy
solid line is nonstationary series as discussed in Chapter 4. As you learn more, you may also see the concepts of a second nonstationary series, forecasting, backcast, seasonal series, heterocedasticity and GARCH
models, input and output series, noise series, vector time series, nonlinear time series, and cointegration in
the cover image.
Library of Congress Cataloging-in-PublicaHonData
Wei, W i a m W. S.
Time series analysis: univariate and multivariate methods I W i a m W. S.
Wei.- 2nd ed.
p, cm.
lncludes index.
ISBN 0-321-32216-9
1. Time-series analysis. I. Title.
Copyright @ 2006 Pearson Mucation, Inc. All rights reserved. No part of this publication may be reproduced,
stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise; without the prior written permission of the publisher. Printed in the United States
of America.
About the Author

William W. S. Wei is Professor of Statistics at Temple University in Philadelphia, PA.
He earned his B.A. in Economics from the National Taiwan University (1966), B.A. in
Mathematics from the University of Oregon (1969), and M.S. (1972) and Ph.D. (1974) in
Statistics from the University of Wisconsin-Madison. From 1982-87, he was the Chair of
the Department of Statistics at Temple University. His research interest includes time series
analysis, forecasting methods, statistical modeling, and applications of statistics in business
and economics. He has developed new methodology in seasonal adjustment, aggregation and
disaggregation, outlier detection, robust estimation, and vector time series analysis. Some of
his most significant contributions include extensive research on the effects of aggregation,
methods of measuring information loss due to aggregation, new stochastic procedures of
performing data disaggregation, model-free outlier detection techniques, robust methods of
estimating autocorrelations, and statistics for analyzing multivariate time series. He is a
Fellow of the American Statistical Association, a Fellow of the Royal Statistical Society, and
an Elected Member of the International Statistical Institute. He is an Associate Editor of the
Joltma1 of Forecasting and the Journal of Applied Statistical Science.
Contents
Preface
CHAPTER I
Overview
1.1 Introduction
1.2 Examples and Scope of This Book
CHAEYTER 2
Fundamental Concepts
2.1 Stochastic Processes
2.2 The Autocovariance and Autocorrelation Functions
2.3 The Partial Autocorrelation Function
2.4 White Noise Processes
2.5 Estimation of the Mean, Autocovariances, and Autocorrelations
2.5.1 Sample Mean
2.5.2 Sample Autocovariance Function
2.5.3 Sample Autocorrelation Function
2.5.4 Sample Partial Autocorrelation Function
2.6 Moving Average and Autoregressive Representations of
T i e Series Processes
2.7 Linear Difference Equations
Exercises
CHAPTER 3
Stationary Time Series Models

3.1 Autoregressive Processes
3.1.1 The First-Order AutoregressiveAR(1) Process
3.1.2 The Second-Order Autoregressive AR(2) Process
3.1.3 The Genera1 pth-Order Autoregressive AR(p) Process
3.2 Moving Average Processes
3.2.1 The First-Order Moving Average MA(1) Process
3.2.2 The Second-Order Moving Average MA(2) Process
3.2.3 The General 4th-Order Moving Average MA(q) Process
Contents
The Dual Relationship Between AR(p) and

MA(q) Processes
3.4 Autoregressive Moving Average ARMA(p, q) Processes
3.4.1 The General Mixed ARMA(p, q) Process
3.4.2 The ARMA(1, 1) Process
Exercises
3.3
CHAPTER 4
Nonstationary Time Series Models

4.1 Nonstationarity in the Mean
4.1.1 Deterministic Trend Models
4.1.2 Stochastic Trend Models and Differencing
4.2 Autoregressive Integrated Moving Average (ARIMA) Models
4.2.1 The General ARTMA Model
4.2.2 The Random Walk Model
4.2.3 The ARIMA(0, 1, 1) or IMA(1, 1) Model
4.3 Nonstationarity in the Variance and the Autocovariance
4.3.1 Variance and Autocovariance of the ARIMA Models
4.3.2 Variance Stabilizing Transformations
Exercises
CHAPTER 5
Forecasting
5.1 Introduction
5.2 Minimum Mean Square Error Forecasts
5.2.1 Minimum Mean Square Error Forecasts for
ARMA Models
ARTMA Models
5.3 Computation of Forecasts
5.4 The ARIMA Forecast as a Weighted Average of
Previous Observations
5.5 Updating Forecasts
5.6 Eventual Forecast Functions
5.7 A Numerical Example
Exercises
CHAPTER 6
Model Identification
6 , l Steps for Model Identification
6.2 Empirical Examples
6,3 The Inverse Autocorrelation Function (TACF)
Contents
xi
6.4 Extended Sample Autocorrelation Function and Other

Identification Procedures
6.4.1 The Extended Sample Autocorrelation Function (ESACF)

6.4.2 Other Identification Procedures
Exercises
CKAPTER 7
Parameter Estimation, Diagnostic Checking,

and Model Selection
7.1
7.2
The Method of Moments

Maximum Likelihood Method
7.2.1 Conditional Maximum Likelihood Estimation
7.2.2 Unconditional Maximum Likelihood Estimation and
Backcasting Method
7.2.3 Exact Likelihood Functions
7.3 Nonlinear Estimation
7.4 Ordinary Least Squares (OLS) Estimation in Time
Series Analysis
7.5 Diagnostic Checking
7.6 Empirical Examples for Series W1-W7
7.7 Model Selection Criteria
Exercises
CHAPTER 8
Seasonal Time Series Models

General Concepts
Traditional Methods
8.2,l Regression Method
8.2.2 Moving Average Method
8.3 Seasonal ARWlA Models
Exercises
8.1
8.2
CHAPTER 9 Testing for a Unit Root

9.1
9.2
9.3
9.4
Introduction
Some Useful Limiting Distributions
Testing for a Unit Root in theAR(1) Model
9.3.1 Testing the AR(1) Model without a Constant Term
9,3,2 Testing the AR(1) Model with a Constant Term
9.3.3 Testing the AR(1) Model with a Linear Time Trend
Testing for a Unit Root in a More General Model
128
128
133
134
xii
Contents
Testing for a Unit Root in Seasonal Time Series Models

9.5.1 Testing the Simjle Zero Mean Seasonal Model
9.5.2 Testing the General Multiplicative Zero Mean
Seasonal Model
Exercises
9.5
206
207
207
211
CHAPTER 10 Intervention Analysis and Outlier Detection

10.1 Intervention Models
10.2 Examples of Intervention Analysis
10.3 Time Series Outliers
10.3.1 Additive and Innovational Outliers
10.3.2 Estimation of the Outlier Effect When the Timing
of the Outlier Is Known
10.3.3 Detection of Outliers Using an Iterative Procedure
10.4 Examples of Outlier Analysis
10.5 Model Identification in the Presence of Outliers
Exercises
CHAPTER 1 I Fourier Analysis

11.1
11.2
11.3
11.4
11.5
General Concepts
Orthogonal Functions
Fourier Representation of Finite Sequences
Fourier Representation of Periodic Sequences
Fourier Representation of Nonperiodic Sequences:
The Discrete-Time Fourier Transform
11.6 Fourier Representation of Continuous-Time Functions
11.6.1 Fourier Representation of Periodic Functions
11.6.2 Fourier Representation of Nonperiodic Functions:
The Continuous-Time Fourier Transform
11.7 The Fast Fourier Transform
Exercises
CHAPTER 12 Spectral Theory of Stationary Processes

12.1 The Spectrum
12.1.1 The Spectrum and Its Properties
12.1.2 The Spectral Representation of Autocovariance Functions:
The Spectral Distribution Function
12.1.3 Wold's Decomposition of a Stationary Process
12.1.4 The Spectral Representation of Stationary Processes
264
254
264
267
27 1
272
Contents
12,2 The Spectrum of Some Common Processes

12.2.1 The Spectrum and the Autocovariance Generating Function
12.2.2 The Spectrum of ARMA Models
12.2.3 The spectrum of the Sum of ' h o Independent Processes
12.2.4 The Spectrum bf Seasonal Models
12.3 The Spectrum of Linear Filters
12.3.1 The Filter Function
12.3.2 Effect of Moving Average
12.3.3 Effect of Differencing
12.4 Aliasing
Exercises
CHAPTER 13 Estimation of the Spectrum

13.1 Periodogram Analysis
13.1.1 The Periodogram
13.1.2 Sampling Properties of the Periodogram
13.1.3 Tests for Hidden Periodic Components
13.2 The Sample Spectrum
13.3 The Smoothed Spectrum
13.3.1 Smoothing in the Frequency Domain:
The Spectral Window

13.3.2 Smoothing in the Time Domain: The Lag Window
13.3.3 Some Commonly Used Windows
13.3.4 Approximate Confidence Intervals for Spectral Ordinates
13.4 ARMA Spectral Estimation
Exercises
CHAPTER 14 Transfer Function Models

14.1 Single-Input Transfer Function Models
14.2.1 General Concepts
14.1.2 Some Qpical Impulse Response Functions
14.2 The Cross-Correlation Function and Transfer Function Models
14.2.1 The Cross-Correlation Function (CCF)
14.2.2 The Relationship between the Cross-Correlation Function
and the-TransferFunction
14.3 Construction of Transfer Function Models
14.3.1 Sample Cross-Correlation Function
14.3.2 Identification of Transfer Function Models
14.3.3 Estimation of Transfer Function Models
xIii
274
274
274
278
279
281
281
283
284
285
286
X~Y
Contents
14.3.4 Diagnostic Checking of Transfer Function Models

14.3.5 An Empirical ~ x a r n ~ l e
14.4 Forecasting Using Transfer Function Models
14.4.1 Minimum Mean Square Error Forecasts for Stationary
Input and Output Series
Nonstatjonary Input and Output Series
14.4.3 An Example
14.5 Bivariate Frequency-DomainAnalysis
14.5.1 Cross-CovarianceGenerating Functions and the
Cross-Spectrum
14.5.2 Interpretation of the Cross-Spectral Functions
14.5.3 Examples
14.5.4 Estimation of tbe Cross-Spectrum
14.6 The Cross-Spectrum and Transfer Function Models
14.6-1 Construction of Transfer Function Models through
Cross-SpectrumAnalysis
14.6.2 Cross-Spectral Functions of Transfer Function Models
14.7 Multiple-lnput Transfer Function Models
Exercises
CHAPTER 15 Time Series Regression and GARCI-I Models

15.1 Regression with Autocorrelated Errors
15.2 ARCH and GARCH Models
15.3 Estimation of GARCH Models
15.3.1 Maximum Likelihood Estimation
15.32 Iterative Estimation
15.4 Computation of Forecast Error Variance
15.5 Illustrative Examples
Exercises
CHAPTER 16 Vector Time Series Models

16.1 Covariance and Correlation Matrix Functions
16.2 Moving Average and Autoregressive Representations
of Vector Processes
1 6 3 The Vector Autoregressive Moving Average Process
16.3.1 Covariance Matrix Function for the Vector AR(1) Mode1
- 16.3.2 Vector AR(p) Models
16.3.3 Vector MA(1) Models
16.3.4 Vector MA(q) ModeIs
16.3.5 Vector ARMA(1, 1) Models
Contents
16.4 Nonstationary Vector Autoregressive Moving Average Models

1 6 5 Identification of Vector Time Series Models
16.5-1 Sample Correlation Matrix Function
16.5.2 partial Autoregression Matrices
16.5.3 Partial Lag cokelation Matrix Function
16.6 Model Fitting and Forecasting
16.7 An Empirical Example
16.7.1 Model Identification
16.7.2 Parameter Estimation
16.7.3 Diagnostic Checking
16.7.4 Forecasting
16.7.5 Further Remarks
16.8 Spectral Properties of Vector Processes
Supplement 16.A Multivariate Linear Regression Models
Exercises
CHAPTER 17 More on Vector Time Series

17,l Unit Roots and Cointegration in Vector Processes
17.1.1 Representations of Nonstationary Cointegrated Processes
17.1.2 Decomposition of 2,
17.1.3 Testing and Estimating Cointegration
17.2 Partial Process and Partial Process Correlation Matrices
17.2.1 Covariance Matrix Generating Function
17.2.2 Partial Covariance Matrix Generating Function
17,2.3 Partial Process Sample Correlation Matrix Functions
17.2.4 An Empirical Example: The U.S. Hog Data
17.3 Equivalent Representations of a Vector ARMA Model
17.3.1 Finite-Order Representations of a Vector Time
Series Process
17.3.2 Some Implications
Exercises
CHAPTER 18 State Space Models and the Kalman Filter

18.1. State Space Representation
18.2 The Relationship between State Space and ARMA Models
18.3 State Space Model Fitting and Canonical Correlation Analysis
18.5 The Kalman Filter and Its Applications
Supplement 18.A Canonical Correlations
Exercises
mi
Contents
CHAPTER 19 Long Memory and Nonlinear Processes

19.1 Long Memory ~rocessesand Fractional Differencing
19.1.1 Fractionally Integrated ARMA Models and Their ACF
19.1.2 Practical Implications of the GRFIMA Processes
19.1.3 Estimation of the Fractional Difference
19.2 Nonlinear Processes
19.2.1 Cumulants, Polyspectrurn, and Tests for Linearity
and Normality
19.2.2 Some Nonlinear Time Series Models
19.3 Threshold Autoregressive Models
19.3.1 Tests for TAR Models
19.3.2 Modeling TAR Models
Exercises
CHAPTER 20 Aggregation and Systematic Sampling in Time Series

20.1 TemporalAggregation of the ARlMA Process
20.1.1 The Relationship of Autocovariances between the
Nonaggregate and Aggregate Series
20.1.2 Temporal Aggregation of the MA(d, q) Process
20.1.3 Temporal Aggregation of the AR(p) Process
20.1.4 Temporal Aggregation of the ARIMA(p, d, q) Process
20.1.5 The Limiting Behavior of Time Series Aggregates
20.2 The Effects of Aggregation on Forecasting and
Parameter Estimation
20.2.1 Hilbert space
20.2.2 The Application of Hilbert Space in Forecasting
20.2.3 The Effect of Temporal Aggregation on Forecasting
.
20.2.4 Information Loss ~ u toeAggregation in
20.3 Systematic Sampling of the ARIMA Process
20.4 The Effects of Systematic Sampling and Temporal Aggregation
on Causality
20.4.1 Decomposition of Linear Relationship between
Two Time Series
20.4.2 An lltustrative Underlying Model
20.4.3 The Effects of Systematic Sampling and
Temporal Aggregation on Causality
20,5 The Effects of Aggregation on Testing for Linearity and Normality
20.5.1 Testing for Linearity and Normality
20.5,2 The Effects of Temporal Aggregation on Testing
for Linearity and Normality
507
507
508
511
512
5 13
515
520
520
52 1
522
524
526
528
528
53 1
532
534
534
537
Contents
20.6 The Effects of Aggregation on Testing for a Unit Root

20.6.1 The Model of Aggregate Series
20.6.2 The Effects of Aggregation on the Distribution of the
Test Statistics
20.6.3 The Effects ofPAggregationon the Significance Level
and the Power of the Test
20.6.4 Examples
20.6.5 General Cases and Concluding Remarks
20.7 Further Comments
Exercises
References
Appendix
Time Series Data Used for Illustrations
Statistical Tables
Author Index
Subject Index
Preface to the
Second Edition
Since the publication of the first edition, this book has been used by many researchers and
universities worldwide. I am very grateful for the numerous encouraging letters and comments received fiom researchers, instructors, and students. Although tbe original chapters in
the book still form the necessary foundation for time series analysis, many new theories and
methods have been developed during the past decade, and the time has come to incorporate
these new developments into a more comprehensive view of the field. In the process of updating this book, I took the opportunity to clarify certain concepts and correct previous errors.
In rime series analysis, we often encounter nonstationary time series, and a formal testing procedure for a unit root has now become the standard routine in time series modeling.
To address this procedure, Chapter 9 on unit root tests for both nonseasonal and seasonal
models has been added.
Regression analysis is the most commonly used statistical method, and time series data
are widely used in regression modeling, particularly in business and economic research, The.
standard assumptions of uncorrelated errors and constant variance are often violated when
time series variables are used in the model. In a separate new Chapter 15, the use of time
series variables in regression analysis is discussed. In particular, this chapter introduces
models with autocorrelated errors and ARCWGARCH models for heteroscedasticity that are
useful in many economic and financial studies,
Although the basic procedures of model building for univariate time series and vector
time series are the same, there are some important phenomena unique to vector time series.
After an introduction to various vector time series models in Chapter 16, cointegration,
partial processes, and equivalent representations of a vector time series model are covered
in the new Chapter 17. They are useful in understanding and analyzing relationships of time
series variables.
Many time series exhibit characteristics that cannot be described by linear models.
Therefore Chapter 19 on long memory processes and nonlinear time series models that are
useful in describing these long memory and nonlinear phenomena was included.
To aid understanding, two supplements have been added: Supplement 16A on multivariate linear regression models, and Supplement IBA on canonical correlations. In the chapter
on aggregation, some new results are included on the effects of aggregation on testing for
linearity, normality, and unit roots.
In this revision, I follow the fundamental theme of the first edition and continue to balance
the emphasis between both theory and applications. Methodologies are introduced with proper
theoretical justifications and illustrated with empirical data sets that may be downloaded
xix
xx
Preface to the Second Edition
from the Web site: http://www.sbm,temple.edu/-wwei/.

As with the first edition, exercise
problems are included at the end of each chapter to enhance the reader's understanding of the
subject. The book should be useful for graduate and advanced undergraduate students who
have proper backgrounds and are interested in learning the subject. It should also be helpful as
a reference for researchers who encounter time series data in their studies.
As indicated in the first edition, the book was developed from a one-year course given in
the Department of Statistics at Temple University. Topics of univariate time series analysis
from Chapters 1 through 13 were covered during the first semester, and the remaining chapters related to multivariate time series plus supplemental journal articles were discussed in
the second semester. With the proper selection of topics, the book can be used for a variety
of one- or two-semester courses in time series analysis, model building, and forecasting.
I wish to thank Dr. Olcay Akrnan of the College of Charleston, Dr. Mukhtar ALi of the
University of Kentucky, Dr. H. K,Hsieh of the University of Massachusetts, Dr. Robert Miller
of the University of Wisconsin, Dr. Mohsen Pourahamadi of Northern nlinois University,
Dr. David Quigg of Bradley University, and Dr. Tom Short of Indiana University of Pennsylvania for their numerous suggestions and comments that have improved this revision. I am grateful
to Ceylan Yozgatligil for her help in preparing some of the updated examples and tables. Finally,
I would iike to thank Ms. Deirdre Lynch, Executive Editor of Statistics, Addison-Wesley for her
continuing interest and assistance with this project as well as Ms. Kathleen Manley, Ms. Barbara
Atkinson, Mr. Jim Mchughlin, and the staff at Progressive Publishing Alternatives who provided wonderful assistance in the production of the book.
William W. S. Wei
Department of Statistics
The Fox School of Business and Management
Temple University
Philadelphia, Pennsylvania, USA
April 2005
Preface t o the
First Edition
Time series andysis has been an area of considerable activity in recent years. Several books
on the subject have been published, most of them concentrating either on time-domain
analysis, or solely on frequency-domain analysis. Within each of these categories some
books provide inadequate theoretical background material, while others offer very few applications. The majority of these books concentrate only on univariate time series, and those
few books which discuss multivariate time series focus only on theoretical aspects of the
subject. This book, however, attempts to provide a comprehensive introduction to both timedomain and frequency-domain analyses of univariate and multivariate time series.
Since my own student days at the University of Wisconsin-Madison, my attitude toward
time series analysis has been influenced greatly by Professor George E. P. Box's motto that
statistics will not flourish without a happy marriage between theory and practice. I hope,
therefore, that a proper balance between theory and applications has been achieved within
the pages of this volume.
Included in the book are many recent advances in univariate and multivariate time series
methods such as inverse autocorrelation functions, extended sample autoconelation functions, Akaike's information criterion (AIC), intervention analysis and outlier detection, vector ARMA rnodeIs, partial lag correlation matrix functions, partial processes, the state-space
model and Kalman filtering, aggregation problems, and many others.
This book was developed from a one-year course given in the Department of Statistics at Temple University. The course was offered to those graduate students with backgrounds in statistical theory and regression analysis. The first nine chapters were covered
in the first semester; the remaining chapters were covered in the second semester. The
book, however, provides sufficient flexibility regarding the order in which topics can be
covered. With an appropriate selection of topics, the book can be used for a variety of
one- or two-semester courses in time series analysis and forecasting. For those interested
in frequency-domain analysis, Chapters 10, 11, and 12 can be covered earlier. However,
this author believes students have a better appreciation of the frequency-domain approach
after studying time-domain analysis.
Exercise problems are included at the end of each chapter. The purpose of these problems
is to enhance the student's understanding of the subject and not solely to test the student's
mathematical skills. Students are encouraged to attempt to solve as many of these exercises as
possible.
The book should be useful for graduate and advanced undergraduate students in statistics, economics, business, engineering, meteorology, various areas of social science, and any
xxii
Preface to the First Edition
other field where analysis and research of time series are prevalent. The book should also be
useful as a reference for research workers interested in learning the subject.
I am greatly indebted to Professor George E. P. Box, who introduced me to time series
analysis, and Professor George C. Tiao, who supentised my research in the field while I was
a graduate student at the University of Wisconsin - Madison. This book could not have been
written without their inspiration as great teachers and researchers.
I wish to thank all those who assisted in the preparation of this book, which evolved
from a series of lecture notes. I am indebted to Ruth Jackson for typing, and Dror Rom and
Hewa Saladasa for proofreading several chapters of the notes. The preparation of the final
manuscript began in the summer of 1987, when my administrative responsibility as Department Chairman at Temple was completed,
I am very grateful to my then thirteen-year-old son, Stephen, for typing most of the final
manuscript, and to Wai-Sum Chan, Leonard Cupingood, and Jong-Hyup Lee for their careful
reading and checking of the entire manuscript, as well as their assistance with numerous computations and the initial draft of figures using SAS (SAS Institute, 1985) and SCA (Scientific
Computing Associates, 1986) software. Many of their valuable suggestions were incorporated
into the book.
I would also like to thank David Reilly for his excellent time-domain software AUTOBOX and MTS, which were used to analyze many univariate and muItivariate time series
data sets in the book. Further acknowledgments to John Schlater and Alan Izenman for their
valuable comments, and to the various manuscript reviewers for their helpful suggestions
and criticisms.
Special thanks goes to my wife, Susanna, for helping to make this project a family effort.
She carefully checked the h a 1 manuscript, provided many helpful suggestions, developed a
sorting system for indices, and supervised our children in a variety of sorting tasks. Of course,
any errors and omissions which remain are entirefy my responsibility, and I would appreciate
having them brought to my attention.
I wish to extend my gratitude to Professor Donald B. Owen and Addison-Wesley
Publishing Company for permission to adopt the t and Chi-Square tables from their book,
Handbook of Statistical Tables, and to the late Professor E. S. Pearson and the Biometrika trustees for permission to adopt the F table from their book, Biornetrika Tables
for Statisticiarts, Volume I .
Finally, I would like to thank Allan Wylde, Karen Garrison, and the rest of the AddisonWesley staff, and Grace Sheldrick and Frederick Bartlett, for their interest and assistance
with this project.
William Wu-Shyong Wei
March 1989
Overview
1.1 Introduction
A time series is an ordered sequence of observations. Although the ordering is usually
through time, particularly in terms of some equally spaced time intervals, the ordering may
also be taken through other dimensions, such as space, Time series occur in a variety of
fields. In agriculture, we observe annual crop production and prices. In business and economics, we observe daily closing stock prices, weekly interest rates, monthly price indices,
quarterly sales, and yearly earnings. In engineering, we observe sound, electric signals, and
voltage. In geophysics, we record turbulence, such as ocean waves and earth noise in an
area. In medical studies, we measure electroencephalogram PEG) and electrocardiogram
@KG) tracings. In meteorology, we observe hourly wind speeds, daily temperature, and
annual rainfall. In quality control, we monitor a process according to a certain target value.
In the social sciences, we study annual birth rates, mortality rates, accident rates, and various
crime rates. The list of areas in which time series is observed and studied is endless.
A time series, such as electric signals and voltage, that can be recorded continuously in
time, is said to be continuous, A time series, such as interest rates, yields, and volume of
sales, which are taken only at specific time intervals, is said to be discrete, In this book, we
deal exclusively with discrete time series observed at equal intervals. We do so because even
continuous time series provide only digitized values at discrete intervals for computations.
There are various objectives for studying time series, They include the understanding
and description of the generating mechanism, the forecasting of future values, and optimal
control of a system. The intrinsic nature of a time series is that its observations are dependent or correlated, and the order of the observations is therefore important, Hence, statistical
procedures and techniques that rely on independence assumption are no longer applicable,
and different methods are needed. The body of statistical methodology available for analyzing time series is referred to as time series analysis.

Figure 1.1 shows four time series that are studied later in the book. Although they each show
the salient feature of dependence among observations, they also exhibit other very distinctive
characteristics.
CHAPTER 1 Overview
(a) Daily average number of truck
manufacturing defects
@) Yearly U.S. tobacco
production
2500
0
1860 1880 1900 1920 1940 1960 1980
Year
(c) Quarterly U.S. beer productian
Year
[dl Contaminatedblo1~4Iydata
Day
FIGURE 1.1 Examples of some time series.
The daily average number of defects per truck found at the end of the assembly line of a
truck manufacturing plant, shown in Figure I .l(a), appears to vary about a fixed level, Time
series that exhibit this phenomenon are said to be stationary in the mean and are special cases
of stationary time series. The yearly U.S.tobacco production, shown in Figure l.l(b), does not
vary about a fixed level and exhibits an overall upward trend instead. Moreover, the variance
of this tobacco series increases as the level of the series increases. Time series that exhibit
these phenomena are said to be nonstationary in mean and variance and are examples of nonstationary time series. The U,S, quarterly beer production in Figure l.l(c) presents another
characteristic pattern that is repetitive in nature due to seasonal variation. Time series containing seasonal variation are called seasonal time series. Nonstationary time series such as those
shown in Figures l,l(b) and (c) can be reduced to stationary series by proper transformations.
After introducing in Chapter 2 fundamental concepts needed to characterize time series,
we study in chapters 3 to 8 a very general class of parametric models: the autoregressive
integrated moving average (ARIMA) models that are useful for describing stationary,
nonstationary, seasonal, and nonseasonal time series such as those shown in Figures l.l(a),
1.2
Examples and Scope of This Book
(b), and {c). Specifically, in Chapter 2, we introduce the basic ideas of stationarity and
nonstationarity, autocorrelation function, partial autocorrelation function, representations
of time series processes, and linear difference equations needed for parametric modeling.
Chapter 3 introduces stationary- autoregressive moving average (ARMA) models, and
Chapter 4 discusses nonstationary ARIMA models. These three chapters give the necessary
background to identify a tentative time series model. Chapter 5 develops theory and methods
of minimum mean square error (MMSE) forecasting, which are needed for later discussion
of parameter estimation. Chapters 6 and 7 illustrate the iterative procedure of time series
model building, Additional identification tools, such as the inverse autocorrelation function
(IACP) and the extended sample autocorrelation function {ESACF), are introduced. Because
several different models may adequately represent a given series, we also discuss useful
model selection criteria such as Akaike's information criterion (AIC). Chapter 8 extends
these ideas to seasonal time series models. Many time series are nonstationary, and a formal
testing procedure for nonstationarity has become a standard routine in time series modeling.
In Chapter 9, we discuss formal unit root tests for both nonseasonal and seasonal models.
The fourth time series shown in Figure I.l(d) is a contaminated laboratory series of
blowfly data. It reflects another phenomenon of nonstationarity due to a change in structure
in the series from some external disturbance. This type of nonstationarity cannot be removed
by a standard transformation. T i e series are often interrupted by external events. Additional
examples include production series affected by strikes and recorded series marred by a
machine failure. Such external disturbances are termed interventions or outliers. To model
external interruptions, intervention and outlier models are studied in Chapter 10.
The time series approach presented in Chapters 2 through 10, which uses autoco~elatian
and partial autocorrelation functions to study the evolution of a time series through parametric
models, is known as time domain analysis, An alternative approach, which uses spectral functions to study the nonparametric decomposition of a time series into its different frequency
components, is known as frequency domain analysis; it is presented in Chapters 11 to 13.
Although these two approaches are mathematically equivalent in the sense that the autocorrelation function and the spectrum function form a Fourier transform pair, on occasion one
approach is more advantageous than the other. Basic knowledge of both approaches is a necessary tool for time series data analysis and research.
Chapters 2 through 13 deal with the analysis of a single (univariate) time series. Time
series data, however, often involve simultaneous observations on several variables. For
example, Figure 1.2 shows the Lydia Pinkham annual data of advertising and sales. In business, one often tries to promote sales by advertising. It seems natural to build a dynamic
model to relate current sales to present and past values of advertising to study the effect of
advertising and to improve sales forecasting. Thus, in Chapter 14 we present transfer function models useful for relating an output series to one or more input series. Using material
from earlier chapters, both time domain and frequency domain approaches to the problem
are discussed. In analyzing multivariate time series, the fundamental tool used in the time
domain approach is the cross-corNation function (CCF), and the fundamental tool used in
the frequency domain approach is the cross-spectral function. They also form a Fourier
transform pair in time series analysis. The other useful method to analyze the relationship
between variables is regression analysis. The regression model is without a doubt the most
commonly used statistical model, but the standard regression assumptions are often violated
when time series variables are used in the model. Therefore, in Chapter 15 we discuss the
CHAPTER I Overview
\\
01
1900
1910
1920
1930
Year
Advertising
1940
1950
1960
FIGURE 1.2 The Lydia Pinkham advertising and sales data,

use of time series variables in regression analysis including models with autocorrelated
errors and various autoregressive conditional heteroscedasticity [ARCH) models for heterogeneous variances.
In many fieids of applications, the transfer function and regression models may not be
appropriate because complicated feedforward and feedback relationships may exist among
variables. For example, in the Lydia Pinkham data, although advertising is expected to promote sales, sales may also affect advertising because advertising budgets are often set as a
percentage of sales. In Chapter 16, we extend results to multivariate vector time series and
study the joint relationship of several time series variables. Vector autoregressive moving
average models are introduced. Detailed discussions on identification tools such as correlation matrix functions, partial autoregression matrices, and partial lag correlation matrix functions are given. Spectral properties of vector processes are also introduced. In Chapter 17,
we discuss useful notions such as cointegration, partial processes, and equivalent representations of a vector time series model.
In Chapter 18, we discuss the state space model and Kalman filtering. This alternative
approach in time series modeling can be used for both univariate and multivariate time
series. Like other methods introduced in the book, the approach is illustrated with empirical
data sets. The relationship between the state space and ARTMA models is also established.
After discussing various univariate and multivariate linear time series models and their
applications, we present, in Chapter 19, long memory processes and nonlinear time series
models. These methods have been found useful in modeling many time series,
Finally, we address a nontrivial, important problem in time series analysis. In working

with time series data, one must first decide on the time unit to be used i n the analysis. In
some cases, such as the annual yield of grain per acre, there exists a natural time unit for the
series; there are other cases, however, in which several different time units may be available
for adaptation. For example, in analyzing sales, one may consider monthly sales, quarterly
safes, or yearly sales. Does the choice of time interval affect the type of conclusions that are
drawn? In Chapter 20, we try to supply some answers to this question when we discuss
aggregation and sampling in time series.
Fundamental Concepts
In this chapter, we introduce some fundamental concepts that are necessary for proper understanding of time series models discussed in this book. We begin with a simple introduction to
stochastic processes, the autocorrelation and partial autocorrelation functions, and the notion
of white noise processes. We then discuss estimation of the mean, autocovariances, and the
autocorrelation and partial autocorrelation functions. Thus, we can illustrate the sampling
phenomena of time series models starting in Chapter 3, which, we believe, will enhance the
appreciation of model identification discussed in later chapters. In addition to its own theoretical merits, the notion of the moving average and autoregressive representations of processes
is useful for understanding the logic underlying parsimonious linear processes used in time
series analysis. We also give a simple introduction to linear difference equations, with a special emphasis on the solution of homogeneous difference equations, which play a fundamental role in the parsimonious linear time series processes.

A stochastic process is a family of time indexed random variables Z(w, t), where o belongs
to a sample space and t belongs to an index set. For a fixed t, Z(w, t) is a random variable.
For a given o,Z(o, t), as a function of t, is called a sample function or realization. Tbe population that consists of all possible realizations is called the ensemble in stochastic processes
and time series analysis. Thus, a time series is a realization or sample function from a certain
stochastic process. To have a proper appreciation for time series analysis, we introduce some
essential concepts of stochastic processes in this section.
In our discussion, we assume that the index set is the set of all integers unless
mentioned otherwise. Consider a finite set of random variables {Z,,, Z,,, . . . ,Zfn)from a stochastic process {Z(o, t): t = 0, f1, f2, . . . J . The n-dimensional distribution function is
defined by
..
where xi,i = 1, , , n are any real numbers. A process is said to be first-order stationary
in distribution if its one-dimensional distribution function is time invariant, i.e., if
F&,(xl) = F&l+,(xl)for any integers tl,k, and tl k; second-order stationary in distribution if
2.1
FG,,Gz(xl,x2) = F & , + , , Z ~ + ~x2)

( Xfor
~ , any integers tl, t2, k, tl
stationary in distribution if
F&,,...,&L;
. ,~ n =) F&,
*
Stochastic Processes
+ k, and tz + k; and nth-order
..,%+,(XI, .- ~ n )
9
(2.1 -2)
for any n-tuple (tl,. . . , tn) and k of integers. A process is said to be strictly stationary if
(2.1.2) is true for any n, i.e., n = 1,2, , , , The terms strorrgly stationary and completely stationary are also used to denote a strictly stationary process. Clearly, if (2.1.2) is true for n =
m, it is also true for n 5 m because the ntth-order distribution function determines all distribution functions of lower order. Hence, a higher order of stationarity always implies a lower
order of stationarity.
With proper understanding that a stochastic process, Z(o, t), is a set of time indexed random variables defined on a sample space, we usually suppress the variable o and simply
write Z(o,t ) as Z(t) or Z,, just as we often denote a random variable by X rather than by X(w).
The process is called a real-valued process if it assumes only real values. Unless mentioned
otherwise, processes discussed in this book are referred to as the real-valued processes. For a
given real-valued process (2,: t = 0,f1, f2, . . .), we define the mean function of the
process
the variance function of the process
the covariance function between Z, and Z,
and the correlation function between Z,, and Z,
For a strictly stationary process, since the distribution function is the same for all t, the
mean function pt = p is a constant, provided E([ztf)
< oo, Likewise, if ~ ( 2 :<) oo,
then u : = u 2 for all t and hence is also a constant. Moreover, since
Fq,,&2(xl,x2) = FZl,+bq2+k(~1,
x 2 ) for any integers ti, t2, and k, we have
and
CHAPTER 2 Fundamental Concepts
Letting ti = t - k and t2 = t, we get
and
Thus, for a strictly stationary process with the first two moments finite, the covariance and
the correlation between Z, and ZISkdepend only on the time difference k.
So far, we have discussed a strong sense of stationarity of a process in terms of its
distribution function. A trivial example of a strictly stationary process is a sequence of
independent identically distributed (i.i.d.) random variables. This sequence of independent random variables usually does not exist or renders no interest in time series. Other
than this simple i,i.d. case, however, it is very difficult or impossible actually to verify a
distribution function, particularly a joint distribution function from an observed time
series. Thus, in time series analysis, we often use a weaker sense of stationarity in terms
of the moments of the process.
A process is said to be nth-order weakly stationary if all its joint moments up to order n
exist and are time invariant, i.e., independent of time origin. Therefore, a second-order
weakly stationary process will have constant mean and variance, with the covariance and the
correlation being functions of the time difference alone. Sometimes, the terms statio~lalyin
the wide sense or covariance stationaly are also used to describe a second-order weakly stationary process. It follows from the definitions that a strictly stationay process with the first
two moments finite is also a second-order weakly or covariance stationary process. Yet, a
strictly stationary process may not have finite moments and hence may not be covariance
stationary. A trivial example is the process consisting of a sequence of i.i.d. Cauchy random
variables. The process is clearly strictly stationary, but it is not weakly stationary of any
order because no joint moments exist.
EXAMPLE 2.1 Let 2, be twice the value of a true die shown on the tth toss. If the die is
tossed three times independently, we have a stochastic process Z,
to the index set = (1, 2,3} and w belongs to the sample space
Z(w, t) where t belongs
For a particular w, say (1, 3, 2), the realization or sample function will be (2, 6, 4). The
ensemble contains a total of 216 possible realizations. If a true die is tossed independently
and repeatedly, then we have a stochastic process 2, = Z(w, t) with the index set being all
positive integers and the sample space being
A particular w bf (1,4, 2 , 3 , . . .) would have the realization (2, 8,4,6, . . .). In this case, the
total number of possible realizations in the ensemble is infinite. Clearly, the process in either
case is strictly stationary as it relates to a sequence of i,i.d, randomvariables.
EXAMPLE 2.2 Consider the time sequence

2, = A sin(ut
+ 0),
(2.1.9)
where A is a random variable with a zero hean and a unit variance and 8 is a random variable with a uniform distribution on the interval [ - r,T] independent ofA. Then
which depends only on the time difference k. Hence, the process is covariance stationary.
EXAMPLE 2.3 Let Z, be a sequence of independent random variables alternately following a standard normal distribution N(0, 1) and a two-valued discrete uniform distribution
= 1
)for all t.
with equal probability 112 of being 1 or - 1. Clearly, E(Z,f = 0 and ~ ( 2
Now
and
Hence, the process is covariance stationary. The process, however, is not strictly stationary. It
is, in fact, not stationary in distribution for any order.
From these discussions and examples, it is clear that "covariance stationary" is a much
weaker form of stationarity than are "strictly stationary" or "stationary in distribution." We
often work, however, with covariance stationary or second-order weakly stationary processes
10
in time series analysis because it is relatively simple to check the first two moments. Henceforth, unless mentioned otherwise, we use the term stationary to refer to all processes that
are covariance stationary. =th this, the following important remark is in order.
A stochastic process is said to be a normal or Gaussian process if its joint probability
distribution is normal. Because a normal distribution is uniquely characterized by its first
two moments, strictly stationary and weakly stationary are equivalent for a Gaussian
process, Unless mentioned otherwise, processes which we discuss are assumed to be Gaussian. Like other areas in statistics, most time series results are established for Gaussian
processes. Thus, the autocorrelation functions and the partial autocorrelation functions discussed in the next two sections become fundamental tools in time series analysis.
2.2 The Autocovariance and Autocorrelation Functions

For a stationary process (Z,}, we have the mean E(ZJ = p and variance Var(Zt) = E(Zl p12 = vZ,which are constant, and the covariances Cov[Z,, Z,), which are functions only of
the time difference It - st. Hence, in this case, we write the covariance between Zt and Z,+k
as
Yk =
Cov(Z,, Zt+k) = EVt - ~)(Zt+k- EL),
(2.2.1)
and the correlation between 2, and Zr+k as
where we note that Var(Z,) = Var(Zt+k) = yo. As functions of k, yk is called the autocovariance function and pk is called the autoconelation function (ACF) in time series analysis
because they represent the covariance and correlation between 2, and Zitk from the same
process, separated only by k time lags.
It is easy to see that for a stationary process, the autocovariance function yk and the autocorrelation function pk have the following properties:
I, 7 0 = Var(Z,); po = 1.
2. I ~ k l 70;lpkl 1
3. yk = y-k and pk = p-n for all k, i.e., yk and pk are even functions and hence symmetric
about the lag k = 0. This property follows from the time dBerence between Zl and Z,+k
and Z, and Zl-k being the same, Therefore, the autocorrelation function is often plotted
only for the nonnegative lags such as the one shown in Figure 2.1. This plot is sometimes calIed a conelogram.
4. Another important property of the autocovariance hnction yk and the autoco~relation
function pk is that they are positive semidefinite in the sense that
2.3
-1.0
The Partial Autocorreiation Function
11
FIGURE 2.1 Example of autocorrelation function (ACF).

and
for any set of time points tl, t2, . . . , t, and any real numbers at, a2, . . . , and an.By
the result (2.2.3) follows from
defining the random variable X =
zy=l
The similar result for pk in (2.2.4) follows immediately by dividing the inequality
(2.2.3) through by yo. Thus, it is important to know that not every arbitrary function satisfying properties (1) to (3) can be an autocovariance or autocorrelation function for a
process. A necessary condition for a function to be the autocovariance or autoconelation
function of some process is that it be positive semidefinite.
2,3 The Partial Autocorrelation Function

In addition to the autocorrelation between 2, and Zt+k, we may want to investigate the correIation between Z, and Z,+n after their mutual linear dependency on the intervening variables
Zr+2,. . . , and Zr+k-l has been removed. The conditional correlation
is usually referred to as the partial autocorrelation in time series analysis.
12
Consider a stationary process {Z,} and, without loss of generality, we assume that
Zt+2, . . . , and Z,+k-l be defined
E(Zt) = 0. Let the linear dependence o ~ - Z , on
+~
as the best linear estimate in theAmeansquare sense of Zt+kas a linear function of Zl+l,
Zt+2,. . . , and Z,+k-l. That is, if Zt+kis the best linear estimate of Z,+k, then
where a i ( l 5 i 5 k - 1) are the mean squared linear regression coefficients obtained from
minimizing
The routine minimization method through differentiation gives the following linear system
of equations
Yi =
alyi-1
+ (uZYi-2 +
'
' '
+ ak-lY;-k+l
( l ~ i s k -1).
(2.3.4)
'
' '
+ a k - I ~ i - R + l (1 5 i 5 k - 1 ,
(2.3S)
Hence.
Pi = Elpi-1
In terms of matrix notation, the above system of (2.3.5) becomes
Pk- 1
Pk-2
Similarly,
where Pi(l 5 i 5 k - 1) are the mean squared linear regression coefficients obtained by.
minimizing
Hence,
1
P1
Pl
1
p2
Pk-2
Pk-3
Pk-4
which implies that ai = Pi(l 5 i 5 k
- 1).
"'
Pl
"'
2.3 The Partial Autocorrelation Function
13
It follows that the partial a~toconelationbetyeen Zt and Z,,k will equal the ordinary
autocorrelation between (Z, - Z,) and (2, + k - Z,+k), Thus, letting Pk denote the partial
autocodation between Zt and Zt+k;we have
Now,
because all other remaining terms reduce to zero by virtue of Equation (2,3.4).Hence,
Next, using that mi
= Pi(l 5
i 5 k - l), we have
Therefore,
Solving the system in (2.3.6) for ai by Crarner's rule gives
14

as the ratio of two determinants. The matrix in the numerator is the same as the symmetric
matrix in the denominator except for its ith column being replaced by ( p l , pn,
,
Substituting crj in (2.3.14)to (2.3.13)and multiplying both the numerator and denominator
of (2.3.13) by the determinant
...
the resulting Pk in (2.3.13) can be easily seen to equal the ratio of the determinants in
(2.3.15)each expanded by its last column,
The partial autocorrelation can also be derived as follows. Consider the regression
model, where the dependent variable Zi+kfrom a zero mean stationary process is regressed
on k lagged variables Zi+kdl,Zt+k-2, . . . and Z,, i.e,,
where $ki denotes the ith regression parameter and et+k is an error term with mean 0 and
uncorrelated with Zt+k-j for j = l , 2, . . . , k. Multiplying Z,+k-j on both sides of the above
regression equation and taking the expectation, we get
hence,
For j = 1,2,
. . . , k, we have the following system of equations:
Using Cramer's rule successively fork = 1'2,
15
. . . , we have
Comparing Equation (2.3.19) with (2.3.15)' we see that i$kk equals Pk.Thus, the partial autocorrelation between 2, and Zt+k can also be obtained as the regression coefficient associated
with Z, when regressing Zr+k on its k lagged variables
ZI+k-2, . . . , and 2, as in
(2.3.16). Because
has become a standard notation for the partial autocorrelation between
2, and Z,+k in time series literature, we use this notation in our book. As st function of k, $kR
is usually referred to as the partial autocorrelation function (PACF).

A process {a,}is called a white noise process if it is a sequence of uncorrelated random variables from a fixed distribution with constant mean E(a,) = pa,usually assumed to be 0,constant
variance Var(a,) = a: and yk = ~ov(a,,at,& = 0 for all k # 0, By definition, it immediately
follows that a white noise process {a,) is stationary with the autocovariance function
16
FIGURE 2.2 ACF and PACF of a white noise process: Z, = p
+ a,.
the autocorrelation function
and the partial autocorrelation function
The ACF and PACF of a white noise process are shown in Figure 2.2.
= 1 for any process, when we talk about the autocorreBecause by definition po =
lation and partial autocorrelations, we refer only to pk and 4fi for k # 0. The basic phenomenon of the white noise process is that its ACF and PACF are identically equal to zero.
Although this process hardly ever occurs in practice, it plays an important role as a basic
building block in the construction of time series models, just like the roles played by the sine
and the cosine functions (sin(nx), cosf~lx)]in Fourier analysis. More precisely, it plays the
role of an orthogonal basis in the general vector and function analysis.
A white noise process is Gaussian if its joint distribution is normal, In the following discussion, unless mentioned otherwise, { a t ) is always referred to as a zero mean Gaussian
white noise process,
2.5 Estimation of the Mean, Autocovariances,

and Autocorrelations
A stationary time series is characterized by its mean p, variance a2,autocorrelations pk,
and partial autocorrelations +kt. The exact values of these parameters can be calculated if
the ensemble of all possible realizations is known. Otherwise, they can be estimated if
2.5
Estimation of the Mean, Autocovariances, and Autocorrelations
17
multiple independent realizations are available. In most applications, however, it is dificult or impossible to obtain multiple realizations, Most available time series constitute
only a single realization, which makes it impossible to calculate the ensemble average.
For a stationary process, howeverj we have a natural alternative of replacing the ensemble
average by the time average. In the following discussion, we examine conditions under
which we can estimate with good statistical properties the mean and autocovariances and
hence the autocorrelations by using time averages.
2.5.1 Sample Mean

With only a single realization, a natural estimator for the mean p = E(Zr) of a stationary
process is the sample mean
which is the time average of n observations. The question becomes whether the above estimator is a valid or good estimator. Clearly,
which implies that Z is an unbiased estimator of p , It can also be easily shown that
where we let k = (t - s). Thus, if
1.1
n-='.
[2
((I
- !)pk]
k=-[n-1)
is finite, then ~ a r ( Z-t

) 0 as n + co, and
is a consistent estimator for ,u.That is,
18
in mean square. The process is said to be ergodic for the mean if the result in (2.5.5) holds.
A sufficient condition far this result to hold is that pk 0 as k + oo,because pk 0 as k -+oo
implies that for any E > 0, we can choose an N such that Ipd < ~ / for
4 all k 3 N. Hence, for
n > (N + I), we have
-+
-+
where we choose an n large enough so that the first term in the next to last inequality above
is also less than 42. Thus, when pk 0 as k co,we have
-+
which implies that, in Equation (2.5.4),
Intuitively, these results simply say that if Z, and Zt+Rsufficiently far apart are almost uncorelated, then some useful new information can be continually added so that the time average
will approach the ensemble average.
2.5.2 Sample Autocovariance Function

Similarly, with only a single realization, we employ the following estimators using the time
average to estimate the autocovariance function yk, i.e.,
I n-k
?k =
ixr=
n-k
(2,- Z)(zt+.k-
2(Zt - Z) (&+k - 2).
+k = -
n-
z)
19

Now,
n-k
x ( z t
r=l
- Z)(~t+k
- z,
n-k
= x [ ( z t t=l
P) -
(z
I L ) ~ [ ( Z I-+ ~
n-k
t=I
(2 - P)]
n-k
(2,- P ) ( z t + k - P) -
(2 - P )
r= 1
( ~ -t
P)
n-k
2
(ZI
t=l
where we approximate the

(rt - k ) ( Z - p ) . Hence,
terms
P)(Zt+k -
P ) - (n - k ) ( Z - P ) ~ ,
2- )
and
(2.5.10)
z : ~ ( z ,-+p~)
by
It is clear that both of these estimators are biased. When we ignore the term ~ a r ( 2 )
representing the effect of estimating p,$k becomes unbiased but ?k is still biased. In general, $k has a larger bias than $, especially when k is large with respect to n, Hence, for a
given n, often at most ,114 estimates are suggested to be calculated in time series analysis.
If pk + 0 as k + cm and hence the process is ergodic for the mean and lim,,, ~ a r ( Z =
) 0
as shown in (2.5.7), however, then both estimators +k and ?k are asymptotically unbiased.
Some may argue that because both .jlk and ;k are biased, it would be more appropriate to
compare their mean square errors. It can be shown that for certain types of processes +k
has smaller mean square error than $k (see, e.g., Parzen f1961bl). In addition, the estimate $k is always positive semidefinite like yk, but $k is not necessarily so. As a result,
we use $k in (2.5.8) as the sample autocovariance function to estimate the autocovariance
function yk.
When the process {Z,) is Gaussian, Bartlett (1946) has shown the following approximate results:
and
20
Similarly,
and
Thus, the variance of +k is larger than the variance of ?k. In fact, from (2S.16) we see
that the variance of $k, v a r ( q k ) , can be substantial for Iarge k resulting in an unstable and
erratic estimate.
Next, we would like to ask when the process is ergodic for the autocovariance function
so that in mean square we have
A rigorous proof of the above statement is complicated. For our purpose, it suffices to note
that for any given k, because the sample autocovariance is an asymptotically unbiased
to be mean square consistent and the
estimator of yk, hence a sufficient condition for
process to be ergodic for the autocovariances is that the autocovariance is absolutely summable, i.e., 2>lyi[ < ca and hence h
,
,
,~ a r ( ? ~=)0.For relevant readings, interested
readers are referred to Gnedenko (1962), Hannan (1930, p. 201), and Fuller (1996, p. 30%
among others. Ergdicity is assumed t o hold for the remainder of this book.
Tk
tk
2.5.3 Sample Autocorrelation Function
For a given observed time series Z1, &, . , ,Z, the sample ACF is defined as
z:=I~tfn,
where Z =
the sample mean of the series. A plot of n; versus k is sometimes
called a sample conelogram.
For a stationary Gaussian process, Bar&lett(1946) has shown that fork > 0 and k j > 0,
21
For large n, ikis approximately normally distributed with mean pk and variance
For processes in which pk = 0 for k > m, Bartlett's approximation of (2.5.20) becomes
In practice, pi(; = 1, 2, . . . , rn) are unknown and are replaced by their sample estimates
ii,and we have the following large-lag standard error of jk;
To test a white noise process, we use
EXAMPLE 2.4 To illustrate the computation of the sample ACF, consider the following
ten values of a time series:
The sample mean of these ten values is Z = 10. Thus,
22
and we note that
In other words, the sample ACF is also symmetric about the origin k = 0.
2.5.4 Sample Partial Autocorrelation Function

The sample PACF $M is obtained by substituting pi by bi in Equation (2.3.19). Instead of
calcuLating the complicated de,terminants for large k in (2.3.19), a recursive method starting
with 411= for computing c $has
~ been given by Durbin (1960) as follows:
and
The method holds also for calculating the theoretical PACF 4kk.
It was shown by Quenouille (1949)~haton the hypothesis that the underlying process is
can be approximated by
a white noise sequence, the variance of 4Kli
Hence, &2/1/;; can be used as critical limits on

process.
+kk
to test the hypothesis of a white noise
2.6 Moving Average and Autoregressive Representations of Time Series Processes
23
EXAMPLE 2.5 Using the data in Example 2.4, we have from (2.3.19), (2.5.251, and
(2.5.26)
Thus, by (2.5.25) we have
Other
can be calculated similarly.

of Time Series Processes
In time series analysis, there are two useful representations to express a time series process.
One js to write a process Z, as a linear combination of a sequence of uncorrelated random
variables, i.e.,
xj=o
00
where I/+-, = 1, {ar) is a zero mean white noise process, and

@
; < W. Here and in the
following an infinite sum of random variables is defined as the limit in quadratic mean
(mean square) of the finite partial sums. Thus, Zt in (2.61) is defined such that
where 2, = Zt - p. By introducing the backshift operator 8,xt = x, -i, we can write (2.6.1)
in the compact form
where $ ( B ) =
CO
xj=o+,Bj.
24
It is easy to show that for the process in (2.6.1)
and
E(atZt-j) = {
u
'
'
0,
for j = 0,
f o r j > 0.
Hence,
and
Clearly, the autocovariance and autocorrelation functions in (2.6.6) and (2.6.7) are functions of the time difference k only. Because they involve infinite sums, however, to be stationary we have to show that yk is finite for each k. Now,
Hence, 2,zO+/
< m is a required condition for the process in (2.6.1) to be stationary.
The form in (2.6.1) is called a moving average (MA) representation of a process. Wold
(1938) proved that a stationary process that is purely nondeterministic (i.e., a process contains no deterministic component that can be forecast or predicted exactly from its own past)
can always be expressed in the form of (2.6.1). Hence, the representation is also known as
Wold's representation in the literature, and any process that can be represented in this form
is called a nondeterministic process. Sometimes the term linearprocess is also used to refer
to the process in (2~5.1).
2.6 Moving Average and Autoregressive Representations of Time Series Processes
For a given sequence -of autocovariances yk, k = 0 fI , 3 2 ,

generating function is defined as
25
. . . , the autocovariance
where the variance of the process, yo, is the coefficient of BO and the autocovariance of lag k,
yk, is the coefficient of both Bk and
Using (2.6.6)and stationarity, we can write (2.6.8)
where we let j = i k and note that tjj = 0 for j < 0. This method is a convenient way of
calculating the autocovariances for some linear processes. The corresponding autoconelation generating function will be
Another useful form is to write a process Z, in an autoregressive (AR) representation, in

which we regress the value of Z at time ton its own past values plus a random shock, i.e.,
or, equivalently,
V(B)Z, = a,,
+ Ej=l]4
(2.6.1 1)
where v ( B ) = 1 - ~ F ~ . r rand
~ ~1j ,
< CO. The autoregressive representation
is useful in understanding the mechanism of forecasting. Box and Jenkins (1976) calf a
process invertible if it can be written in this form. They argue that in forecasting, a noninvertible process is meaningless. It is easily seen that not every stationary process is invertible. For a h e a r process 2, = $(B)u, to be invertible so that it can be written in terms of the
AR representation, the roots of $(B) = 0 as a function of B must lie outside of the unit circle. That is, if P is a root of +(B), then PI > 1, where ] I is the standard Euclidian metric.
When /3 is a real number, /PI is equal to the absolute value of fl and when j3 is a complex
number, p = c id, then 1 ~ =1
03
w.
26
It should be noted that an invertible process is not necessarily stationary. By Wold's result, it follows that for the process presented in (2.5.11) to be stationary, the process must be
able to be rewritten in a MA representation, is.,
zFo+;
such that the condition

< co is satisfied. To achieve that, the required condition is
that the roots of a (B) = 0 all lie outside the unit circle. That is, if S is a root of m(B), we
have 181 > 1.
Although the autoregressive and moving average representations are useful, they are not
the model forms we use at the beginning stage of model building because they contain an infinite number of parameters that are impossible to estimate from a finite number of available
observations, Instead, in modeling a phenomenon, we construct models with only a finite
number of parameters.
In the autoregressive representation of a process, if only a finite number of a weights
are nonzero, i,e., .rrl =
7~~ = 5h2, . . . , vp= c $and
~ vk= 0 for k > p, then the resulting
process is said to be an autoregressive process of order p. This process is written as
Similarly, in the moving average representation, if only a finite number of weights are
nonzero, i.e.,
= -el, +Z = -02, . . . , Gq = -Oq and t+bk= 0 fork > 4, then the resulting
process is said to be a moving average process of order q, which is given by
If we restrict ourselves to the above finite-order autoregressive models in (2.6.13) and moving average models in (2.6.14), however, the number of parameters may still be prohibitively
large. A natural alternative is the mixed autoregressive moving average model
For a fixed number of observations, the more parameters in a model, the less efficient is
the estimation of the parameters. Other things being equal, in general, we choose a simpler
model to describe the phenomenon. This modeling criteria is the principle of parsimony in
model building recommended by Tukey (1967) and Box and Jenkins (1976). In the following
chapters, we discuss some very useful parsimonious time series models and their properties,
Linear Difference Equations

Linear di~erenckequations play an important role in the time series models discussed in this
book, In fact, all models of finite number of parameters mentioned above relate an output Z, to
an input a, in terms of linear difference equations. Thus, they are sometimes referred to as
27
linear difference equation models. The properties of these models often depend on the characteristics of the roots of these difference equations, To better understand these models, we give a
brief introduction to h e a r difference equations, especially to the method of solving these
equations. A general nth-order linear difference equation with constant coefficients is given by
where Ci, i = 0, 1, . . . ,n, are constants. Without loss of generality, we can set Co = 1. The
function e, in (2.7.1) is called a forcing function. Equation (2.7.1) is said to be nonhomogeneous (or complete) if e, # 0, and homogeneous if e, = 0.
Using the backshift operator and letting C(B) =. (1 CIB C ~ B. .~. C,,Bn),we
can rewrite (2.7.1) as
C(B)Zt
(2.7.2)
e,.
As a function of B, C(B) = 0 is called the auxiliary equation associated with the given linear
difference equation. The solution of linear difference equations rests primarily on the following lemmas, which can be easily proved using the definition of a solution and the property of
linearity.
Lemma 2.7.1. If z,[~)and

blzjl)
z/*)are
solutions of the homogeneous equation, then
+ hzi2)is also a solution for any arbitrary constants bl and bn.
is a particular
Lemma 2.7.2. If Z i H )is a solution to the homogeneous equation and
is
the
general
solution
of the
solution of the nonhomogeneous equation, then z , ( ~
f)
complete equation.
The particular solution of a nonhomogeneous difference equation depends on the form of
the forcing function. The solution of a homogeneous difference equation depends on the roots
of the associated auxiliary equation. As will be seen, most of the time series models considered
in this book are of the difference equation type and the behavior of the time series generated by
such models is governed by the nature of the roots of the associated auxiliary equation. In the
remainder of this section, we concentrate on the solution of a general homogeneous linear difference equation. When B is used as an operator, it operates on Lime index t.
Lemma 2.7.3. Let ( 1
- B)m Z, = 0, Then a solution is given by 2,

constant and j is a nonnegative integer less than na.
bti, where b is any
Proof For nr = 1, Z, = bP = b , Clearly, we have (I -B)Z, = (1 - B)b = b - b = 0, Now

assume that (1 - B ) ~ - I Zr = 0, where 2, = btj, j < (m- 1). Then for Zt = bti, j < m,
28
Now each term in the last expression involving t contains only integer powers less than
(I# - 1). Thus, by our induction hypothesis, each term is reduced to zero by the operator
i
(I - B)"-'. The lemma is proved.
It fotlows from Lemmas 2.7.1 and 2.7.3 that (1 - ~ ) ~ [ ~ ~ i =' 0
b for
~ tany
j ] constants bo, bl, . . . and bmll.
Lemma 2.7.4 Let (l-RB)m Zt = 0. Then a solution is given by Zt = ti Rt, where j is any
nonnegative integer less than In, and the general solution is given by 2, = ( ~ r $ ' b ~ t j ) ~ ~ ,
where bj is a constant.
Proof
First, we note that
Repeated application of the above result gives
which equals zero because (1 - B y ti = 0 by Lemma 2.7.3. The result then follows immediately from Lemma 2.7.1.
m
Finally, we have the following main result.
Theorem 2.7.1. Let C(B)Z, = 0 be a given homogeneous linear difference equation where
N
N
C(B) = 1 CIB + C ~ +B ~ + C,,Bn. If C ( B ) = ITi=i(l
- RiB)*i, where Zi=lnli
=n
and Bi= R y 1 i = 1,2, . . . , N) are roots of multiplicity mi of C(B) = 0, then
N
2 = 8 , = , Xb~# ~
t j ~~:In
. particular, if mi = 1 for all i and R ; ~(i = 1,2, . . . ,n) are all
distinct, we have 2, = X $ l b i ~ j .
Proof
The result follows immediately from Lemmas 2.7.1,2.7.3, and 2.7.4.
Note that for a red-valued linear difference equation, a complex root of C(B) = 0 must
appear in pairs. That is, if(c -F di) is a root, then its complex conjugate (c + do' = (c - d11
is also a root. A general complex number can always be written in the polar form, i.e.,
( c f di) = a(cos 4 f i sin +),
where
and
(2.7.3)
29
Because ( c & di)t = at(cos #t rt i sin $t), it follows that for each pair of complex roots of multiplicity m, the solution of the homogeneous difference equation should contain the sequences
d cos #t, d sin #f; td cos 4t, t d sin 4 t; . . . ;and tm-la' cos #Jt, tm-lat sin +t.
For an illustration, consider the second-order difference equation with the following
auxiliary equation
with
R1 = c
+ di = ~ ( C Oc)Sf i sin 4 )
Rz
and
c - di = a(cos c) - i sin +),
By Theorem 2.7.1, we have

Z, = el(c
+ di)' + ez(c - di)'
= el[a(cos 4
+ i sin +)Ir + e~[a(cos# - i sin #J)It,
where el and e2 are any (complex) constants. For a real-valued process, we claim that we can
always write
Z, = blat cos +t
+ &d sin #t,
where br and b2 are real constants. This result follows because we can choose e2 to be the
complex conjugate of el. Thus, if el = x + iy, then e2 = x - iy, and
2, = (el
+ e2)(atcos#t) + (el - e2)i(arsin 41)
= bldcos
4t
+ &at sin +t,
where bl and b2can be easily seen as real.
EXAMPLE 2.6 Let Z, - 2Z,-1
Zti2 = 0. Find the closed form solution for 2,.

The auxiliary equation is given by
R+'
1 is a root of multiplicity 2. Hence, by Theorem 2.7,I,
EXAMPLE 2.7 Find the solution for Z, - 22,-1

The auxiliary equation is given by
+ 1.5Zt-2 - .5Z,4 = 0.
30
Hence, B1 = R r l
1 is a root of multiplicity one; and
and B3 = R?' = 1 - i are a pair of complex roots of multiplicity one.

Now R1 = 1 and R2 = (1 -i)/2 and R3 = (1 i)/2. To express R2 and R3 in the polar
form, we have from (2.7.4) and (2.7.5)
Hence, by the remarks following Theorem 2.7.1, we have
Remark The Riused in the solution of C(B) = 0 in Theorem 2.7.1 is the inverse of the
root Bj, i.e., Ri = B;l. Thus, as shown in Example 2.7, to find Ri, we first find the root Bi of
C(B) = 0 and then calculate its inverse. This cumbersome procedure can be avoided by noting that if we let R = B-I and multiply the equation
by Rn = (B-I)",we get
It can be easily seen that Biis a root of (2.7.6) if and only if Ri is a root of (2.7.7).Thus,
sometimes we can compute Ri = Bidl needed in Theorem 2.7.1 more easily from solving
(2.7.7). The reader can easily verify that each Ri obtained in Example 2.7 is indeed a solution of R3 - 2~~ 1.5R - .5 =+ ( R ~ R f S)(R - 1) = 0.
EXAMPLE 2.8 Solve (1 - +B)(1 -
= 0.
Because R1= 4, and R2 = 1 of multiplicity 2, we have from Theorem 2.7.1,Z, = bl$'
b2 b3f.
2.1 Let Z,, where t is even, be a sequence of independent random variables defined as
2, = 1 or - 1 with equal probability of fi2, and Z, =
if t is odd, where t is
an integer,
Exercises
31
(a) Is the process first order stationary in distribution?

(b) Is it second order stationary in distribution?
2.2 Let 2, = U sin (2vt) V.cos (2?tt), where U and V are independent random variables, each with mean 0 and variance I.
(a) Is Z, strictly stationary?
(b) Is Z, covariance stationary?
2,3 Prove or disprove that the following process is covariance stationary:
(a) Zt = A sin (27rt 8), where A is a constant, and 6 is a random variable that is
uniformly distributed on [0,27r]
(b) 2, = A sin (27rr 4- 8), where A is a random variable with zero mean and unit
variance, and 8 is a constant.
(c) 2, = (- l)'A, where A is a random variable with zero mean and unit variance.
2.4 Is the following a valid autocorrelation function for a real-valued covariance

stationary process? Why?
2.5
Verify the following properties for the autocorrelation function of a stationary

process:
(a) Po = 1,
(b) Ipnt -= 1,
tc) Pk = P-k-
2.6 It is important to note that not every sequence satisfying the properties specified in
Exercise 2.5 is an ACF of a process. Show that the following function satisfies the
properties stated in Exercise 2.5, but is not an ACF for any stationary process:
2.7
Given the time series 53, 43, 66, 48, 52, 42, 44, 56, 44, 58, 41, 54, 51, 56, 38, 56,
49,52,32,52,59,34,57,39,60,40,52,44,65,43:
(a) Plot the series.
(b) Can you guess an approximate value for the first lag autoco~elationcoefficient
p l based on the plot of the series?
(c) Plot 2, against Z,,[, and try again to guess the value of pl.
32
(d) Calculate and plot the sample ACF, &, fork = 0, 1,2, 3,4,5.
(e) Calculate and plot the sample PACF,
fork = 0,1,2,3,4,5.
aRli,
2.8 Show that the estimate $k is always positive semi-definite, but
Gkis not necessarily so.
2.9 Consider a stationary series with theoretical autocorrelation function
Find the variance of jkusing Bartlett's approximation,

2.10
Let Z,
+ xj=o$ja,-j
M
or, equivalently, Zt = p
XpO+jRj, t,ho = 1, XFO1$jl <

variance 5;.Prove that
process 2,.
2.11 Prove that n Var(Z)
03,
+ +(B)a,,
'33
as n
m, if
CO
&=-&'kt <
00-
2.12 Find a closed form solution for the following difference equations:
(a) 2, - 1.12,,32,-2 = 0,
(b) Z,-Z*-I + Z , - 2 - Z r - 3 = 0 ,
(c) Z, - 1.82,- + .81Z,-2 = 0,
-:
and the a, are white noise with mean 0 and

< m, where yj is the jth autocovariance of the
~p-l~~l
xk=-yk
where $ ( B )
Stationary Time
Series Models
Limited by a finite number of available observations, we often construct a finite-order parametric model to describe a time series process. In this chapter, we introduce the autoregressive
moving average model, which includes the autoregressive model and the moving average
model as specid cases. This model contains a very broad class of parsimonious time series
processes found useful in describing a wide variety of time series. After giving detailed discussions on the characteristics of each process in terms of the autocorrelation and partial autocorrelation functions, we illustrate the results with examples.

As mentioned earlier in Section 2.6, in the autoregressive representation of a process, if only
a finite number of ~ T Tweights are nonzero, i.e., a1 = + I , .rm = +2, . . . , vp=
and rk= 0
for k > p, then the resulting process is said to be an autoregressive process (model) of order
p, which is denoted as AR(p). It is given by
where &(B) = (1 - +lB - . . and % = 2, - p.

Because
= Zf=r[$jl5 W , the process is always invertible. To be stationary,
the roots of +p (3) = 0 must lie outside of the unit circle. The AR processes are useful in
describing situations in which the present value of a time series depends on its preceding
values plus a random shock. Yule (1927) used an AR process to describe the phenomena of
sunspot numbers and tbe behavior of a simple pendulum. First, let us consider the foliowing
simple models.
~-p~ln~l
33
34
CHAPTER 3 Stationary Time Series Models
1 The First-Order Autoregressive AR(1) Process
For the first-order autoregressiveprocess AR(l), we write
Z,= #,Zt-,
+ a,.
(3.1.3b)
As mentioned above, the process is always invertible. To be stationary, the root of (1 - #qB) = 0
must be outside of the unit circle. That is, for a stationary process, we have
< 1. The
AR(1) process is sometimes called the Markov process because the distribution of 2, given
it-*, . . .is exactly the same as the distribution of Z , given
ACF of the AR(1) Process The autocovariances are obtained as follows:
and the autoconelation function becomes
where we use that po = 1. Hence, when [ # 11 < 1 and the process is stationary, the ACF expo< 1, then all
If 0 <
nentially decays in one of two forms depending on the sign of
autocorrelations are positive; if - 1 < < 0, then the sign of the autoco~elationsshows an
alternating pattern beginning with a negative value. The magnitudes of these autocorrelations decrease exponentially in both cases, as shown in Figure 3.1.
PACF of the AR(1) Process For anAR(1) process, the PACF from (2.3.19) is
k = 1,
for k
2.
Hence, the PACF of the AR(1) process shows a positive or negative spike at lag 1 depending
on the sign of and then cuts off as shown in Figure 3.1.
EXAMPLE 3.1 For illustration, we simulated 250 values from an AR(1) process, (1 = .9, The white noise series a, are independent normal N(0, I)
#lB)(Z, - 10) = a,, with
random variables. Figure 3.2 shows the plot of the series. It is relatively smooth.
Table 3.1 and Figure 3.3 show the:ample ACF and the sample PACF for the series.
Clearly 5k decreases exponentially and 4Rtcuts off afier lag 1 because none of the sample
3.1
FIGURE 3.1 ACF and PACF of the AR(1) process: (1
Autoregressive Processes
- +,B]z,
35
= a,,
PACF values is significant beyond &at lag and, more important, these insignificant
do
not exhibit any pattern. The associated standard error of the sample ACF ;k is computed by
Zkk
and the standard error of the s a m p l e P ~ c ~ is set to be
which are standard outputs used in most time series software.
36
50
100
200
150
250
Time
FIGURE 3.2 A simulated AR(1) series, (1 - .9B)(Z, - 10)
= a,.
EXAMPLE 3.2 This example shows a simulation of 250 values fiom the AR(1) process
(1 - $lB)(Z, - 10) = a,, with
= -.65 and a, being Gaussian N(0, 1) white noise. The
series is plotted in Figure 3.4 and is relatively jagged.
The sample ACF and sample PACF of the series are shown in Table 3.2 and Figure 3,s.
We see the alternating decreasing pattern beginning with a xgative in the sample ACF and
= fil, 411
is also negative. Also, even
the cutoEproperty of the sample PACF. Because
though only the first two or three sample autocorrelations are significant, the overall pattern
clearly indicates the phenomenon of an AR(1) model with a negative value of 41.
&,
TGBLE 3.1 Sample ACF and sample PACF for a simulated series from
(1 - .9B)[Z, - 10) = a,.
FIGURE 3.3 Sample ACF and sample PACF of a simulated ARE])

series: (1 - .9B) (2, - 10) = a,,
-0
50
100
150
200
250
Time
FIGURE 3.4 A simulated AR(1) series (1
-i-.6SB) (Z, -
10) = a,.
37
38
TABLE 3.2 Sample ACF and sample PACF for a simulated series from
(1 ,65B)(Zt - 10) = a,.
FIGURE 3.5 Sample ACF and sample PACF of a sirnufated AR(1)

series (1 f .6SBj(Zt - 10) = a,.
In discussing stationary autoregressive processes, we have assumed that the zeros of
the autoregressive polynomial &(B) tie outside of the unit circle. In terms of the AR(1)
process (3.1.3a or b), it implies that
< 1. Thus, when I+lj 2 1, the process is regarded
as nonstationary because we have implicitly assumed that the process is expressed as a linear combination of present and past white noise variables. If we also consider a process
that is expressed as a linear combination of present and future random shocks, then there
exists an AR(1) process with its parameter
greater than 1 in absolute value, which is
still stationary in the usual sense of the term as defined in Section 2.1. To see that, consider the process
where {a,} is a white noise process with mean zero and variance a:. It is straightforward to
verify that the process 2, in (3.1.9) is indeed stationary in the sense of Section 2.1 with the
39
ACF, pk = (.5)w. Now, consider the process (3.1.9) at time (t - 1) and multiply both of its
sides by 2, i.e.,
Thus, (3.19) leads to the following equivalent AR(1) model with 4I = 2,

2, - 2Zt-1
b,,
(3.1.11)
where b, = -24 - 1. Note, however, that although the 6, in (3.1.1 1) is a white noise process
with mean zero, its variance becomes 4mi, which is four times larger than the variance of a,
in the following AR(1) model with the same ACE pk = (.5)H,
which can be written as a linear combination of present and past random shocks, is.,
03
2, =
In summary, although a process with an ACF of the form $14, where ]$I < 1, can be
written either as
~,.,(.S)c-j.
Z,- $-lZt-l
= b,,
(3.1.14)
where both a, and b, are zero mean white noise processes, the variance of b, in (3.1.14) is
larger than the variance of a, in (3.1-13) by a factor of 4-2.Thus, for practical purposes, we
will choose the representation (3.1.13). That is, in terms of a stationary AR(1) process, we
always refer to the case in which the parameter value is less than 1in absolute value.
3.1.2 The Second-Order Autoregressive AR(2) Process

For the second-order autoregressive AR(2) process, we have
40

The AR(2) process, as a finite autoregressive model, is always invertible. To be stationary, the
roots of 4(B) = (1 - #lB - 6 ~
= 0~
mist )lie outside of the unit circle. For example, the
process (1 - 1.5B
. 5 6 ~ ~ )=~ ,a, is stationary because (1 - 1.5B + , 5 6 ~ ~=)
( I - .7B)(1 - .8B) = 0 gives B = 11.7 and B = 11.8 as the two roots, which are larger than 1
in absolute value. Yet (1 - .2B - . ~ B ~ )=z ,a, is not stationary because one of the roots of
(1 - .2B - .8LI2)= 0 is B = I, which is not outside of the unit circle.
The stationarity condition of the AR(2) model can also be expressed in terms of its
parameter values, Let B1 and B2 be the roots of (1 - 41B - # z ~ 2 ) = 0 or, equivalently, of
- 1 = 0. We have
5b2B2 +
and
The required condition lBil > 1implies that ll/Bil < 1 for i
1 and 2. Hence,
and
Thus, we have the following necessary condition for stationarity regardless of whether the
roots are real or complex:
For real roots, we need 4f
41
+ 4$2 2 0, which impf es that
or, equivalently,
< 0. Thus, in terms of the parameter valPor complex roots, we have 42< 0 and #f
ues, the stationarity condition of the AR(2) model is given by the following triangular region
in Figure 3.6 satisfying
ACF of the AR(2) Process We obtain the autocovariances by multiplying Zt-k on

both sides of (3.1.15b) and taking the expectation
Hence, the autocorrelation function becomes

Pk =
$ I P ~ - I + 42L'k-2,
FIGURE 3.6 Stationary regions for the AR(2) model.
1.
42
Specifically, when k
1and 2
which implies that
and pk for k e 3 is calculated recursively through (3.1.19).

The pattern of the ACF is governed by the difference equation given by (3.1.19), namely
- &B*) pk = 0.Using Theorem 2.7,1, we obtain
(1 -
where the constants bl and b2 can be solved using the initial conditions given in (3.1.20) and
(3.1.21). Thus, the ACF will be an exponential decay if the roots of (I - +IB - &B2) = 0
are real and a damped sine wave if the roots of (1 - i$qB - #2B2) = 0 are complex.
The AR(2) process was originally used by G . U. Yule in 1921 to describe the behavior
of a simple pendulum. Hence, the process is also sometimes called the Yule process,
PACF of the M(2) Process For the AR(2) process, because
fork 2 1 as shown in (3.1.19), we have, from (2.3.19),
43
because the last column of the numerator is a linear combination of the first two columns.
= 0 for k 2 3. Hence, the PACF of an AR(2) process cuts
Similarly, we can show that
off after lag 2. Figure 3.7 ilIustrates the PACF and corresponding ACF for a few selected
AR(2) processes,
EXAMPLE 3.3 Table 3.3 and Figure 3.8 show the sample ACF and the sample PACF for
a series of 250 values simulated from the AR(2) process (1 + .5B - . ~ B ~ )=z a,,
, with the a,
being Gaussian N(0, 1)white noise. The oscillating pattern of the ACF is similar to that of an
AR(1) model with a negative parameter value. The rate of the decreasAng of the autocorrelations, however, rejects the possibility of being an AR(1) model. That 4kacuts off after lag 2,
on the other hand, indicates an AR(2) model.
EXAMPLE 3.4 To consider an AR(2) model with the associated polynomial having
complex roots, we simulated a series of 250 values from (1 - B .5B2)2,= a,, with the
a, being Gaussian N(0, 1) white noise, Table 3.4 and Figure 3.9 show the sample ACF
and the sample PACF of this series. The sample ACF exhibits a damped sine wave, and
the sample PACF cuts off after lag 2. Both give a fairly clear indication of an AR(2)
model.
44
FIGURE 3.7 ACF and PACF of AR(2) process: (1 -
- 42112)2,= at.
45
TABLE 3.3 Sample ACF.and sample PACF for a simulated series from
(1 + ,5B - .3B2)Z, = a,.
FIGURE 3.8 Sample ACF and sample PACF of a simulated AR(2)

series: (1 + .5B - .3B2fZ, = a,.
3.1.3 The General pth-Order Autoregressive AR(p) Process

The pth-order autoregressive process AR(p) is
(1 - # 1 B - # 2 ~ 2 - . - - -
# p ~ ~ ) z=t
at
(3.1.24a)
or
&
= $l~t-l
+ 42&-2 + - + &it,+ a,
(3.1.24b)
ACF of the General AR(p) Process To find the autocovariance function, we multiply Zt-R on both sides of (3.1.24b)
.&-,&
= +l~t-k~t-l
' ' '
+ ilmkat
and take the expected value

Yk
41~n-1+
+ #p~k-p,
k > 0,
(3.1.25)
46
(1 - B .5B2)zt
= 0,.
1.o
1.0
0.8
0.6 0.4 0.2 0.0 .
-0.2
-0.4
-0.6
I'
- 1 , .
rot
0.8 0.6 0.4 0.2 *k 0.0

-0.2 -0.4
-0.6
. ,
10
, a r k
-0.8 -1.0 -
-0.8 -
-1.0 -
FIGURE 3.9 Sample ACF and sample PACF of a simulated 4 2 )

series: (1 - 3 + .5B2)Z, = a,.
where we recall that E(a,ZtZ,_k)
= 0 for k > 0. Hence, we have the following recursive
relationship for the autocorrelation function:
From (3.1.26) we see that the ACF, PA, is determined by the difference equation
$(B)pk = (1 -. # 1- ~
$ 9 -~ .~. - 4 p ~ ) p =
k o for k > 0.NOW, we can write
47
where Xitn=di
= p, and G;l (i = 1,2, . , ., m) are the roots of multiplicity diof $(B) = 0,
Using the difference equation result in Theorem 2.7.1, we have
If di= 1for all i, then GiW1ate all distinct and the above reduces to
For a stationary process, IGT'~ > 1 and iGil < 1. Hence, the ACF pk tails off as a mixture of
exponential decays .or damped sine waves depending on the roots of &(B) = 0. Damped
sine waves appear if some of the roots are complex.
PACF of the General G R [ p ) Process Because pk = +lpt-l + c $ ~ P ~+- ~- - +

+ppk-p for k > 0, we can easily see that when k > p the last column of the matrix in the numerator of +u. in (2.3.19) can be written as a linear combination of previous columns of the
same matrix. Hence, the PACF
will vanish after lag p. This property is useful in identifying an AR model as a generating process for a time series discussed in Chapter 6.
In the moving average representation of a process, if only a finite number of @ weights are
nonzero, i.e., t,hI = --dl,
= -02, , , ,+q = -dq, and @k -- 0 for k > q, then the resulting
process is said to be a moving average process or model of order q and is denoted as MA(q).
It is given by
where
+ + -+
Because 1 8; .
0; < m, a finite moving average process is always stationary.
This moving average process is invertible if the roots of B(B) = 0 lie outside of the unit
circle. Moving average processes are useful in describing phenomena in which events produce an immediate effect that only lasts for short periods of time. The process arose as a
result of the study by Slutzky (1927) on the effect of the moving average of random
events. To discuss other properties of the MA(q) process, let us first consider the following
simpler cases.
48
3.2.1 The First-Order Moving Average MA(1) Process

When O(B) = (1 - OIB), we have the first-order moving average MA(1) process
where {a,] is a zero mean white noise process with constant variance uz.The mean of 12,)
is ~ ( 2 ,=) 0, and hence E(Z,) = p.
ACF of the MAC11 Process The autocovariance generating function of a MA(1) process
is, using (2,6.9),
Hence, the autocovariances of the process are
The autocorrelation function becomes
which cuts off after lag 1,as shown in Figure 3.10.

Because 1 Of is always bounded, the MA(1) process is always stationary. For the
process to be invertible, however, the root of (1 - BIB) = 0 must lie outside the unit circle.
Because the root B = 1/O1, we require ldll < 1 for an invertible MA(1) process.
Two remarks are in order.
1. Both the process 2, = (1

autocorrelation function
- .4B)a, and the process 2, = (1 - 2,5B)a, have the same
- BIB)a, and 2, = (1 - 1/61B)a, have the

same autocorrelations. If the root of (1 - BIB) lies outside the unit circle, however, then
the root of (1 - l/BIB) = 0 lies inside the unit circle, and vice versa. In other words,
among the.two processes that produce the same autocorrelations, one and only one k
invertible. Thus, for uniqueness and meaningful implication for forecasting discussed
in Chapter 5, we restrict ourselves to an invertible process in the model selections.
2. From (3.2.41, it is easy to see that 2Ipnl 4 1. Hence, for an MA(1) process, (pk[< ' 5 .
In fact, more generally, for any 01, 2, = (1
3.2
Moving Average Processes
49
FIGURE 3.10 ACF and PACF of MAEI) processes: Z, = (I - BIB)a,.
PACF of the MA(1) Process Using (2.3.19) and (3.2.4), the PACF of an MA(1) process
can be easily seen to be
50
In general,
-ef(i - e;L)
h k
fork
= 1 - gf(k+l) '
1.
Contrary to its ACF, which cuts off after lag 1, the PACF of an MA(1) model tails off exponentially in one of two forms depending on the sign of & (hence on the sign of pl). If alternating in sign, then it begins with a positive value; otherwise, it decays on the negative side,
as shown in Figure 3.10. We also note that
<
i.
EXAMPLE 3.5 The sample ACF and sample PACF are calculated from a series of 250
values simulated from the MA(1) model 2, = (1 - .5B)a,, using a, as Gaussian N(0,1)
white noise. They are shown in Table 3.5 and plotted in Figure 3.1 1. Statistically, only one
autoconelation
and two partial autocorrelations
and
are significant. From the
overall pattern, however, clearly cuts off after lag 1 and 4fi tails off, which indicate a
clear MA(1) phenomenon.
Fk
all h2
Z, = (I - .5B)a,.
FIGURE: 3.1 1 Sample ACF and sample PACF of a sirn~llatcdMA(1)

series: Z, = (1 - .5B)a,.
3.2
51
3.2.2 The Second-Order Moving Average MA(2) Process

When B(B) = (1 - OIB - O2B2),we have the second-order moving average process
where (a,) is a zero mean white noise process. As a finite-order moving average model, the
MA(2) process is always stationary. For invertibility, the roots of (1 - OIB - e2B2) = 0 must
lie outside of the unit circle, Hence,
which is parallel to the stationary condition of the AR(2) model, as shown in (3.1.18).
ACF of the MA(2) Process The autocovariance generating function via (2.6.9) is
Hence, the autocovariances of the MA(2) model are
and
The autocorrelation function is
which cuts off after lag 2.
52
PACF of the MA(2) Process From (2.3.19), using that pk = 0 fork 2 3, we obtain
The MA(2) process contains the MA(1) process as a special case. Hence, the PACF tails off as
an exponential decay or a damped sine wave depending on the signs and magnitudes of and
d2 or, equivdently, the roots of (1 - BIB - 02B2) = 0.The PACF will be damped sine wave if
the roots of (1 - BtB - 8 2 ~ 2=) 0 are complex, They are shown in Figure 3.12 together with
the comsponding ACE
EXAMPLE 3.6 A series of 250 values is simulated from the MA(2) process 2, = (1 .65B - .24~*)a,with a Gaussian N(0,1) white noise series a,. The sample ACF and sample
PAC? are in Table 3.6 and plotted in Figure 3.13. We see that 6kclearly cuts off after lag 2
and #kk tails off as expected for an MA(2) process.
3.2.3 The General qth-Order Moving Average MA(q) Process

The general qth-order moving average process is
For this general MAfq) process, the variance is

a
where O0 = 1, and the other autocovariances are
Hence, the autocorrelation function becomes
The autocorrelation function of an MA(q) process cuts off after lag q. This important property enabla us to identify whether a given time series is generated by a moving average process.
3.2
53
FIGURE 3.12 ACF and PACF of MA(2) processes: 2, = (I - GIB - O ~ B ~ ) ~ , .
54
TABLE 3.6 Sample ACF and sample PACF for a simulated MA(2) series from
2, = (1 - .65B - .24B2)a,.
FIGURE 3.13 Sample ACF and sample

PACF of a simulated MA(2)
'
series: 2, = (1 - .65B - ,24B2)al.
From the discussion of MA(1) and MA(2) processes, we can easily see that the partial autocorrelation function of the general MA(q) process tails off as a mixture of exponential decays
andlor damped sine waves depending on the nature of the roots of (1 - BIB . - 0484) = 0.
The PACF will contain damped sine waves if some of the roots are complex.
- .
3.3 The Dual Relationship Between AR(p) and MA(q) Processes

For a given stationary AR@) process,
= a,,
where $(B) = ( I - #lB -
..-
- +BP), we can write
3.3 The Dual Relationship Between ARlp) and WCq) Processes
with $(B) = (1
55
+ $ I B + $*B2 f . .) such that
The $ weights can be derived by equating ihe coefficients of Bj on both sides of (3,3.3). For
example, we can write the ARQ) process as
which implies that
Thus, we obtain the $is as follows:
Actually, forj 2 2, we have
where $o = 1.In a special case when 42 = 0,we have $j
+{ for j 2 0. Therefore,
This equation implies that a finite-order stationruy AR process is equivalent to an infiniteorder MA process,
Given a general invertible MA(q) process,
56
with fl,(B) = (1 - OIB -
- - . - e,B4), we can rewrite it as
where
For example, we can write the MA(2) process as
where
Thus, the .n weights can be derived by equating the coefficients of Bj as follows:
In general,
9 = n;.-161
-t ~ ~ - ~ tforj
3 ~ 2, 3.
(3.3.11)
When O2 = 0 and the process becomes the MA(1) process, we have .rrj
and
(I
1
Zt = a,.
+ OIB + B T B ~ + - - -)it
=
t 1 - $1B)
*
-8: for j 2 1,
3.4 Autoregressive Moving Average ARMA@, q ) Processes
57
Thus, in terms of the AR representation, a finite-order invertible MA process is equivalent to

an infinite-orderAR process.
In summary, a finite-order stationary AR(p) process corresponds to an infinite-order
MA process, and a finite-order invertible MA(q) process corresponds to an infinite-order
AR process. This dual relationship between the AR(p) and the MA(q) processes also exists
in the autocorrelation and partial autocorrelation functions. The ARCp) process has its autocorrelations tailing off and partial autocorrelations cutting off, but the MA(q) process has
its autocorrelations cutting off and partial autocorrelations tailing off.

A natural extension of the pure autoregressive and the pure moving average processes is the
mixed autoregressive moving average process, which includes the autoregressive and moving
average processes as special cases. The process contains a large class of parsimonious time
series models that are useful in describing a wide variety of time series encountered in practice.
3.4.1 The General Mixed ARMA(p, q] Process

As we have shown, a stationary and invertible process can be represented either in a moving
average form or in an autoregressive form. A problem with either representation, though, is
that it may contain too many parameters, even for a finite-order moving average and a finiteorder autoregressive model because a high-order model is often needed for good approximation. In general, a large number of parameters reduces efficiency in estimation. Thus, in
model building, it may be necessary to include both autoregressive and moving average
terms in a model, which leads to the following useful mixed autoregressive moving average
(ARMA)process:
where
and
For the process to be invertible, we require that the roots of Bq(B) = 0 lie outside the unit
circIe, To be stationary, we require that the roots of $(B) = 0 lie outside the unit circle.
Also, we assume that C$~(B)
= 0 and Bq(B) = 0 share no common roots. Henceforth, we refer
to this process as an ARMA(p, q) process or model, in which p and q are used to indicate the
orders of the associated autoregressive and moving average polynomials, respectively.
58
The stationary and invertible ARMA process can be written in a pure autoregressive
representation discussed in Section 2.6, i,e., (3,4,2)
T(B)Z~= at,
where
This process can also be written as a pure moving average representation,
(3.4.4)
2, = +(B)a,,
where
ACF of the ARMAEp, q) Process To derive the autocovariance function, we rewrite

(3.4.1) as
and multiply by
+ #+z,-~+ at - B l ~ , - l - . . . - d,at-,
k, =
on both sides
z ~ - =~ +zl ~~l - k ~ t - l
-k
- + +$l-gt-,+
- d,&-,aH
. . - d&t-kat-q.
'
We now take the expected value to obtain

yk = + l ~ R - l+
--
4- +pyk-p + ~ ( i r - ~ a-~OIE(Z,-kat-l)
)
- ..
6@(&-flt-,).
Because
E ( Z , - ~ ~ , - ~=) 0
fork > i,
we have
Yk = 4 1 ~ k - l +
and hence,
' ' '
+$~k-p
(q +
' 1 9
(3.4.6)
3.4 Autoregressive Moving Average ARMA@,q ) Processes
59
Equation (3.4.7) satisfies .the pth-order homogeneous difference equation as shown in

(3.1.26) for the ARlp) process. Therefore, the autocorrelation function of an ARMA(p, q)
model tails off after lag q just like an ARlp) process, which depends only on the autoregressive parameters in the model. The-fist q autocorrelations pq, pq-l, . . . ,pl, however, depend
on both autoregressive and moving average parameters in the model and serve as initial values for the pattern. This distinction is useful in model identification.
PACF of the m ( p , q ) Process Because the ARMA process contains the MA

process as a special case, its PACF will also be a mixture of exponential decays or damped
sine waves depending on the roots of 4JB) = 0 and 04(B) = 0.
3.4.2 The ARMA(1, 1) Process

(1 -
+ 1 ~ ) ~=, (1
- OIB)at,
For stationarity, we assume that [1$~1 < 1, and for invertibility, we require that [Bll < 1.
= 0, it is reduced to
= 0, (3.4.8a) is reduced to an MA(1) process, and when
When
an AR(1) process, Thus, we can regard the AR(1) and MA(1) processes as special cases of
the ARMA(1,l) process.
In terms of a pure autoregressive representation, we write
T(B)Z, = a,,
where
By equating coefficients of B] on both sides of the above equation, we get

?ri =
j- 1
Or),
forj r 1.
(3.4.9)
60
We can write the ARhM(1, 1) process in a pure moving average representation as
We note that
lee.,
[l +
($1
- 91)B +
($2
- $1#1)3~
+ . . .] = (1 - BIB).
Hence,
=(
forj
6 ,
1.
(3.4.10)
ACF of the A W ( 1 , I) Process To obtain the autocovariance for {Z,], we multiply

Zf+ on both sides of (3.4.8b),
- +lif-ardl
+
~ ~ - -~ ~a ~, i ~ - ~ a ~ - ~ ,
and take the expected value to obtain
More specifically, when k = 0,
Recall that ~ ( 2 ~ =
a u:.
~ ) For the term ~ ( i ' , a ~ - ~we) ,note that
Hence,
Yo
= + 1 ~ 1+
@a2
- 81)~,2.
4(+l
When k = I, we have from (3,4,11)

Y1 = 4 1 ~ -0
'
61
Substituting (3.4,13) in (3.4.12), we have
YO =
- +lelm: + ~2 - 4 1 e ~+~~TU:,
;
nus,
For k 2 2, we have from (3.4.11)
Hence, the ARMA(1, 1) model has the following autocorrelation function:
Note that the autocorrelation function of an ARMA(1, 1) model combines characteristics of both AR(1) and MA(1) processes. The moving average parameter 61 enters into the
calculation of pl. Beyond p l , the autocorrelation function of an ARMA(1, 1) model follows
function of an AR(1) process.
the same pattern as the autoco~~elation
PACF of the ARMA(1, 1) Process The general form of the PACF of a mixed model
is complicated and is not needed. It suffices to note that, because the ARMA(1, 1) process
contains the MA(1) process as a special case, the PACF of the ARMA(1,l) process also tails
off exponentially like the ACF, with. its shape depending on the signs and magnitudes of cP1
and el.Thus, that both ACF and PACF tail off indicates a mixed ARMA model. Some of the
ACF and PACF patterns for the -(I,
1) model are shown in Figure 3.14. By examining
Figure 3.14, it can be seen that due to the combined effect of both and 81, the PACF of the
ARMA(1, 1) process contains many more different shapes than the PACF of the MA(1)
process, which consists of only two possibilities.
62
FIGURE 3.14 ACF and PACF of ARMACl, 1) model (1
c#J1~)2,
= fl - OlB)a,.
EXAMPLE 3.7 A series of 250 values is simulated f?om the ARMA(1, 1) process (1 .9B)Z, = (1 - .5B)a,,with the a, being a Gaussian N(0,1) white noise series. The sample ACF
and sample PACF are shown in Table 3.7 and also plotted in Figure 3.15. That both and $kk
tail off indicates a mixed ARMA model. To decide the proper orders of p and q in a mixed
model is a much more difficult and challenging task, sometimes requiring considerable experience and skill. Some helpful methods are discussed in Chapter 6 on model idenacation. For
now, it is sufficient to identify tentatively from the sample ACF and sample PACF whether the
phenomenon is a pure AR, pure MA, or mixed ARMA model. Solely based on the sample
PACF as shown in Table 3.7, without looking at the sample ACF, we know that the phenomenon
-1.0
-1.0
63
FIGURE 3.14 (Continzred)

cannot be an MA process because the MA process cannot have a positively exponentially
decaying PACF.
EXAMPLE 3.8 The sample ACF and PACF are calculated for a series of 250 values as
shown in Table 3.8 and plotted in Figure 3.16. None is statistically significant from 0, which
would indicate a white noise phenomenon. In fact, the series is the simulation result from the
ARMA(1, 1) process, (1 - I#~B)Z,= (I - dlB)a,with = .6 and O1 = .5. The sample ACF
and sample PACF are both small because the AR polynomial (1 - -63)
and the MA polynomial (1 - .5B) almost cancel each other out. Recall from (3.4.14) that the ACF of the
64
TABLE 3.7 Sample ACF and sample PACF for a simulated ARMA(1, If series from
(1 - .9B)Zt = (1 - .5B)a,.
k
. 9
10
FIGURE 3.15 Sample ACF and sample PACF of a simulated ARMAfl, 1)

series: (I - .9B)Zt = (1 - .5B)a,.
ARMA(1, 1) process is pk = 41k-1(+l
- +rB1)/(1 + 8: - 2.4181) for k 2 1 ,
which is approximately equal to zero when 4l= dl, Thus, the sample phenomenon of a
white noise series implies that the underlying model is either a random noise process or an
ARMA process with its AR and MA polynomials being nearly equal. The assumption of no
common roots between 4 J B ) = 0 and O,(B) = 0 in the mixed model is needed to avoid this
confusion.
Before closing this chapter, we note that the ARMA(p, qg)model in (3.4.1), i.e.,
can also be twitten as
3.4 Autoregressive Moving Average ARMACp, q) Processes
65
TABLE 3.8 Sample ACF and sample PACF for a simulated series of the ARMA(1, 1)
process: (1 - .6B)Z, = (I - .5B)a,.
FIGURE 3.16 Sample ACF and sample PACF of a simulated ARMA(1, 1)

series: (I - .6B)Zt = f l - .5B)a,.
where
In terms of this form, the AR(p) model becomes

(1
+lB
- . . - $@P)Z,
and the MA(q) model becomes
It is clear that in the MA(q) process, Oo = 1.1..
= 8, 4- a,
66
3.1 Find the ACF and PACF and plot the ACF pk for k = 0, 1,2, 3,4, and 5 for each of
the following models where the a, is a Gaussian white noise process.
(a) Zl - .5Z,-1 = a,,
(b) Z, + ,98Zt-1 = a,,
(c) Z, - l.?Z1-l 3- .4ZtW2= a,,
(d) Z, - 1,2Z,-1 .8Zl-z = al.
3.2 Consider the following AR(2) models:

(i) 2, - .6Z1-1- .3Ztd2= a ,
(ii) Z, - .8Zt-1 .5Z,-2 = a,.
(a) Find the general expression for pk.
(b) Plot the pk, for k = 0, 1, 2, . . . , 10.
(c) Calculate r+2by assuming that cr; = 1,
3.3 Simulate a series of 100 observations from each of the models with a: = 1 in
Exercise 3.1. For each cas?, plot the simulated series, and calculate and study its
sample ACF jkand PACF
fork = 0, 1, . . . ,20.
3.4 (a) Show that the ACF pk for the AR(1) process satisfies the difference equation
(b) Find the general expression for pk.
- .25Z,-2 + a,.
3.5 Consider the AR(2) process Z, =
(a) Calcufate pl.
(b) Use pol p1 as starting values and the difference equation
to obtain the general
_
expression for pk.
(c) Calculate the values pk for k = 1,2, . . . , 10.
4
3.6 (a) Find the range of a such that the AR(2) process
is stationary.
(b) Find the ACF for the model in part (a) with a =
-4.
3.7 Show that if an AR(2) process is stationary, then
3.8 Consider the MA(2) process 2, = (1 - 1.2B $. . 5 ~ ~ ) a , .

(a) Find'the ACE using the definition of autocovariance function.
(b) Find the ACF using the autocovariance generating function.
(c) Find the PACF 4; for the process.
Exercises
67
3.9 Find an invertible process which has the following ACF:

po = 1,
P I = .25,
and
pk = 0
fork r 2.
3.10 (a) Find a process that has the follgwing autocovariance function:
yo = 10,
y2 = -4,
yl = 0,
yk = 0,
for lk] > 2.
(b) Examine stationarity and invertibility for the process obtained in part (a).
3.H Consider the MA(2) process 2, a, - .la,-l 3- . 2 1 ~ , - ~ .

(a) Is the model stationary? Why?
(b) Is the model invertible? Why?
(c) Find the ACF for the above process,
=I
3.12 Simulate a series of 100 observations from the model with rr; = 1 in Exercise 3 3 .
Plot the simulated series and calculate and study its sample ACF, j ? ~ ~and
, PACF, &
fork = 0, 1, , . . ,20.
3.13 One can calculate the PACF using either (2.3.19) or (2.5.25). Illustrate the computation of +11, 422,and $33 for the MA(1) process using both procedures.
3.14 Consider each of the following models:

(i) (1 - R)Z, = (1 - 1.5B)at,
(ii) (1 - .8B)Zt = (1 - .5B)al,
(iii) (1 - 1. l B 4- . ~ B ~ ) z=, (1 - 1.7B + . 7 2 ~ ~ ) a , ,
(iv) (1 - .6B)Z, = (1 - 1,2B + . 2 ~ ~ ) a ,
(a) Verify whether it is stationary, or invertible, or both.
(b) Express the model in an MA representation if it exists.
(c) Express the model in an AR representation if it exists.
3.15 Consider each of the foilowing processes:

(i) (1 - .6B)Z, = (1 - .9B)a,,
(ii) (1 - l.4B . ~ B ~ ) z=, (1 - .8B)a,

(a) Find the ACF pk
(b) Find the PACF #ldi
for k = 1,2,3.
(c) Find the autocovariance generating function.
3.16 Simulate a series of 100 observations from each of the models with oh2 = 1 in
Exercise 3.15. For each sipulated series, plot the series, calculate, and study its
sample ACF bk and PACF & for k = 0, 1, . . . ,20.
Nonstationary Time
Series Models
The time series processes we have discussed so far are a11 stationary processes, but many
applied time series, particularly those arising from economic and business areas, are nonstationary. With respect to the class of covariance stationary processes, nonstationq time
series can occur in many different ways. They could have nonconstant means p,, timevarying second moments such as nonconstant variance u:, or both of these properties. For
example, the monthly series of unemployed women between ages 16 and 19 in the United
States from January 1961 to August 2002 plotted in Figure 4.1 clearly shows that the mean
level changes with time. The plot of the yearly U.S. tobacco production between 1871 and
Jan 1960
Jan 1970
Jan 1980
Year
Jan 1990
Jan 2000
FIGURE 4.1 Monthly series of unemployed women between ages 16 and 19

in the United States from January 1961 to August 2002.
4.1
I860
1880
1900
1920
Year
1940
Nonstationarity in the Mean
1960
1980
69
FIGURE 4.2 Yearly U.S. tobacco production between 1871 and 1984.
1984 shown in Figure 4.2 indicates both that the mean level depends on time and that the
variance increases as the mean level increases,
In this chapter, we illustrate the construction of a very useful class of homogeneous
nonstationary time series models, the autoregressive integrated moving average ( A m )
models. Some useful differencing and variance stabilizing transformations are introduced to
connect the stationary and nonstationary time series models.
4.1 Nonstationarity in the Mean

A process nonstationary in the mean could pose a very serious problem for estimation of the
time-dependent mean function without multiple realizations. Fortunately, there are models
that can be constructed from a single realization to describe this time-dependent phenomenon. Two such classes of models that are very useful in modeling time series nonstationary
in the mean are introduced in this section.
4.1.1 Deterministic Trend Models

The mean function of a nonstationary process could be represented by a deterministic
trend of time, In such a case, a standard regression model might be used to describe the
70
CHAPTER 4 Nonstationary Time Series Models
phenomenon, For example, if the mean function p, follows a linear trend, p, = a 0

then one can use the deterministic linear trend model
2, = a. t a l t
+ a,,
+ nit,
(4.1+1]
with the a, being a zero mean white noise series. For a deterministic quadratic mean function, p, = CYO alt a2f2,one can use
More generally, if the deterministic trend can be described by a kth-order polynomial of

time, one can model the process by
Z, = cue
+ cult f ... + aktk+ a,.
(4.1-3)
If the deterministic trend can be represented by a sine-cosine curve, one can use
2, = v0
= YO
+ v COS(O~
+ 6) -t- a,
+ COS(W~)
iP sin(wt) + a,,
CY
(4.1 -4)
(4.1.5)
where
a = v cos 6,
-v sin 6,
and
We call v the amplitude, w the frequency, and 8 the phase of the curve. More generally,
we can have
rn
2, = v0
+ 2 (ajcos djt + Pj sin ujt) + ul,
j= 1
which is often called the model of hidden periodicities..These models can be analyzed using
standard regression analysis and are discussed again later in Chapter 13.
4.1.2 Stochastic Trend Models and Differencing

Although many time series are nonstationary, due to some equilibrium forces, different parts
of these series behave very much alike except for their difference in the local mean levels.
Box and ~enkins(1976, p. 85) refer to this kind of nonstationary behavior as homogeneous
nonstationary. In terms of the ARMA models, the process is nonstationary if some roots of
its AR polynomial do not lie outside the unit circle. By the nature of homogeneity, however,
4.2
Autoregressive Integrated Moving Average (ARIMA) Models
71
the local behavior of this kind of homogeneous nonstationary series is independent of its
level. Hence, by letting q ( B ) be the autoregressive operator describing the behavior, we have
for any constant C, This equation implies that q ( B ) must be of the form
for some d > 0, where 4 ( B ) is a stationary autoregressive operator. Thus, a homogeneous

nonstationary series can be reduced to a stationary series by taking a suitable difference of
the series. In other words, the series (Z,) is nonstationary, but its dth differenced series,
((1 - B ) ~ z , ) for Some integer d 2 1, is stationary. For example, if the dth differenced
series foilows a white noise phenomenon, we have
(3
B ) ~ z =, a,.
(4.1.12)
To see the implication of this kind of homogeneous nonstationary series, consider d = 1

in (4.1.12)>ie.,
2, = Z,-I
Given the past information Zt-l, Zt.+
+ a,.
(4.1.13b)
. . . , the level of the series at time t is
which is subject to the stochastic disturbance at time (t - 1).In other words, the mean level
of the process Zt in (I. - B ) ~ z for
, d r 1 changes through time stochastically, and we characterize the process as having a stochastic trend. This result is different from the deterministic
trend model mentioned in the previous section, where the mean level of the process at time t
is a pure deterministic function of time.
4.2 Autoregressive Integrated Moving

Average (ARIMA) Models
A homogeneous nonstationary time series can be reduced to a stationary time series by taking a proper degree of differencing. The autoregressive moving average models are useful in
describing stationary time series, so in this section, we discuss the use of differencing to
build a large class of time series models, autoregressive integrated moving average models,
which are useful in describing various homogeneous nonstationary time series,
72
CHAPTER 4 Nonstationary l i m e Series Models
4.2.1 The General ARIMA Model

Obviously, the stationary process resulting from a properly differenced homogeneous nonstationary series is not necessady white noise as in (4.1.12). More generally, the differenced
series (1 - B I ~ Zfollows
,
the general stationary ARMAb, q) process discussed in (3.4.1) of
Chapter 3. Thus, we have
and the invertible MA operwhere the stationary AR operator &(B) = (1 - #lB - * ator 8&B) = (1 - BIB - * - BqB(I)share no common factors. The parameter 90 plays very different roles for d = 0 and d > 0. When d = 0, the original process is stationary, and we recall
from (3.4.16)that O0 is related to the mean of the process, i.e,, O0 = p(1 - +l - ' When
d 2 1, however, Bo is called the deterministic trend term and, as shown in the next section, is
often omitted ftom the model unless it is really needed,
The resulting homogeneous nonstationary model in (4.2.1) has been referred to as the
autoregressive integrated moving average model of order (p, d, q) and is denoted as the
ARIMAfp, d, q) model. When p = 0, the ARIMAb, d, q) model is also called the integrated
moving average model of order (d, q) and is denoted as the IMA(d, q) model. In the following discussion, we illustrate some commonly encountered ARMA models.
a
..
4.2.2 The Random Walk Model

In (4.2.1),ifp = 0, d = 1, and q = 0,then we have the well-known random walk model,
Z,= Z,-I 4- a,.
(4,2,2b)
This model has been widely used to describe the behavior of the series of a stock price. In
the random walk model, the value of Z at time t is equal to its value at time (t - 1) plus a
random shock. This behavior is similar to following a drunken man whose position at time t
is his position at time (t - 1) plus a step in a random direction at time t.
Note that the random walk model is the limiting process of the AR(1) process (1 @)Z, = a, with 4 + 1. Because the autocorrelation function of the AR(1) process is pk = 4':
as 4 1, the random walk model phenomenon can be characterized by large, nonvanishing
spikes in the sample ACF of the original series (Z,} and insjgn%cant zero ACF for the differenced series ((1 - B)z,}.
Next, consider the following simpfe modifkation of (4.2.2a) with a nonzero constant t&
Z, = ZIII
+ Oo + a,.
(4.2.4)
73
With reference to a time origin k, by successive substitution, we have
z,+,4- 6, + a,
= Ztd2 + 20a + 'a, + at-l
2, =
+ ( t - k)Oo +
= Zk
aj,
j=kf 1
fort
> k.
Thus, it is dear that 2, contains a deterministic trend with slope or drift 19,. More generally, for the model involving the dth differenced series ((1 - B ) ~ z , ) ,the nonzero Oo in
(4.2.1) can be shown to correspond to the coefficient cud of Id in the deterministic trend,
q,+ a l t 4%id. For this reason, when d > 0, 80 is referred to as the deterministic
trend term. For large t, this term can become very dominating so that it forces the series
to follow a deterministic pattern. Hence, in general, when d > 0, we assume that Oo = 0
unless it is clear from the data or the nature of the problem that a deterministic component
is really needed.
The process in (4.2.3) with 8, # 0 is usually called the random walk model with drift.
Given Z,-r, Z,-2, , , ,by (4.2.4) the mean level of the series 2, at time t is
. -+
a
which is influenced by the stochastic disturbance at time (t - 1) through the term Zt-l as
well as by the deterministic component through the slope go. When Oo = 0, we have a model
with only a stochastic bend.
W P L E 4.1 To illustrate the results of the random walk mode1 discussed in this
section, we simulated 100 observations each from the model (1 - B)Zt = a, and the model
(1 - B)Z, = 4 + a,, where the a, in both models are i.i,d, normal N(0, 1) white noise.
The sample ACF and PACF of the original series are shown in Table 4.1 and Figure 4.3.
That the ACF for both series decays very slowly indicates that they are nonstationary, To
identify the model properly, we calculate the sample ACF and PACF of the differenced series
(1 - BIZ, as shown in Table 4.2 and Figure 4.4. A? expected, both exhibit phenomena of a
white noise process. In fact, the patterns of ;k and +&are identical for the two models. Then
how can we tell the difference between the regular random walk model and the random walk
model with drift? We cannot if we rely only on their autocorrelation structures, although the
sample ACF of the original series fiom a random walk model with drift generally decays
more slowly. If we look at their behaviors as plotted in Figure 4.5, however, then the difference is striking. The series of the random walk model with drift is clearly dominated by
the deterministic linear trend with slope 4. On the other hand, the nonstationarity of the
random walk model without drift is shown through a stochastic trend and its values are free
to wander.
74
C W T E R 4 Nonstationary Time Series Models
TkBLE 4.1 Sample ACF and sample PACF for the original series 2, simulated
from random walk models.
(a) For 2,from (1 - B)Zt = at
10
10
@] For ZZ,from (1 - BIZ, = 4 + a,

4
4.2.3 The ARfMA(0, 1, 1) or IMA(1, 1) Model

When p = 0, d = 1, and q = 1, the model in (4.2.1) becomes
where -1 < 6 < 1. This IMA(1,l) model for Zt is seduced to a stationaty MA(I) model for the
first differenced series, (1 - B)Z,, The random walk model is a special case of this IMA(1, 1)
model with 6 = 0. Thus, the basic phenomenon of the IMA(1,I) model is characterized by the
sample ACF of the original series failing to die out and by the sample ACF of the h t differenced series exhibiting the pattern of a fitst-order moving average phenomenon.
For-1<6<1,
-
(1
- OB)
= (1 - B)(1
+ BB + o2Il2 +
FIGURE 4.3 Sample ACF and PACF of the random walk model. (a) For Z, from
(1 - BIZ, = a,. (b) For 2, from (1 - B)Z, = 4 -t a,,
where a = (1 - 0). Hence,
75
78
CHAPTER 4 Nonstationary 3 m e Series Models

(a)
-10
20
40
60
Time
80
100
Time
FIGURE 4.5 Simulated series from random walk models. [a) A simulated random walk
series: (1 - B)Z, = a,. (bf A simulated random walk series with drift: (1 - B)Z, = 4 f a,.
TABLE 4.3 Sample ACF and sample PACF for the original series 2, simulated from
three ARIMA models.
fa) For Z, from
(1 - .8B)(l
5
(b) For Z, from (l
(c) For Zt from (1

3
- B)Z, = a,
6
- BIZ, = (1
10
. 8
10
10
- ,753~2,
- .9B)(1 - B)Z,= (1 -
.5B)a,
7
FIGURE 4.6 Sample ACF and PACF from three ARIMA models. (a) For Z, from
(1 - .8B)(1 - B)Zr = a,. (b) For 2, from (1 - BIZ, = jl - .75B)ar. (c) For 2, from
(1 - .9B)(1 - BIZ, = (1 - .5B)a,.'
79
80
T B L E 4.4 Sample ACF and sample PACF for the differenced series W, = (I
simulated from three ARIMA modeIs. .
- B)Z,
'
(a) For W , = (1 - B)Z, from { l - .8B)(1 - B)Zt = a,

2
3
4
5
6
7
8
(b) For W,= (1 - B)Z,from (1

3
4
5
10
- B)Z, = (1 - .75B)a,
6
[c) For W, = (1 - B)Z, from (1 - .9B)(1 - B)Z, = (1
10
10
- .5B)at
8
one shows the same phenomenon of the sustained large ACF and an exceptionally large first
lag PACF. The dominating phenomenon of nonstationarity overshadows the fine details of the
underlying characteristics of these models. To remove the shadow, we take differences for
each of these series. The sample ACF and PACF for these differenced series are shown in
Table 4.4 and Figure 4.7, Now, all the fine details are evident.*TheACF jkof W, = {I - B)Zt
from the ARIMA(1, 1, 0)model tails off, whereas its PACF $& cuts off after lag 1; the ACF
ti of W, = (1 - B)Z, from the ARIMA(0, 1, 1) model cuts off after lag 1, whereas its PACF
$&tails off; and both the ACF jkand PACF
of Wt = ( 1 - B)Z, from the ARIMA(l,l, 1)
model tail off as expected from their characteristics discussed in Chapter 3.
&

Differencing can be used to reduce a homogeneous nonstationary time series to a stationary
time series, Many nonstationary time series, however, are nonhomogeneous. The nonstationarity of these series is not because of their time-dependent means but because of their
FIGURE 4.7 Sample ACF and PACF for differenced series from three AFUMA
models. (a) For W, = (1 - BIZ, from (1 - .83) (1 - BIZ, = a,. (b) For W, =
( 1 - BIZ, from ( I - 312, = (1 - .75B)a,. (c) For W, = [l - B]Z, from
(1 - .9B)(I - B)Z, = (1 - .5B)a,.
81
82
CHAPTER 4 Nonstationary 'Erne Series Models
time-dependent variances and autocovariances. To reduce these types of nonstationarity,

we need transformations other than differelicing.
4.3.1 Variance and Autocovariance of the ARf MA Models

A process that is stationary in the mean is not necessarily stationary in the variance and the
autocovariance. A process that is nonstationary in the mean, however, will also be nonstationary in the variance and the autocovariance, As we have shown in the previous section,
the mean function of the ARIMA model is t h e dependent. We now show that the ARXMA
model is also nonstationaty in its variance and autocovariance functions.
First, we note a very fundamental phenomenon about the ARIMA model. That is,
although the model is nonstationary, the complete characteristic of the process is determined for all time only by a finite number of parameters, i.e., the & the Oj, and a:.
Thus, the complete future evolution of the process can be developed from the ARIMA
model fitted to a given data set, (Z1, &, . , , 2,). For example, suppose that we fit the
IMA(1, 1) model
to a series of no observations. With reference to this time origin no, for t

by successive substitutions,
=Z
,,
Similarly, for t
+ a, f
(I
- B)a,-, + . . + (1 - d)u,,+l
- k > no,
Hence, with respect to the time origin no,
> 110, we can write
- 60%.
(4.3.2)
83
where we note that Z, and a, are known values with respect to the time origin no, and
Now, horn (4.3.4) to (4.3.7), we have the foliowing important observations:
1. The variance, Var(Z,), of the ARTMA process is time dependent, and Var(Zt) # Var(Zt-k)
fork # 0.
2. The vafiance Var(Z,) is unbounded as t 4 co.
3. The autocovariance COV(Z,-~,
2,) and the autocorrelation COK(Z,-~,Z,) of the process
are afso time dependent and hence are not invariant with respect to time translation. In
other words, they are not only functions of the time difference k but are also functions of
both the time origin t and the original reference point no.
4, If r is large with respect to no, then from (4.3.7),C0rr(2,-~,2,) 1. Because lcorr(~,-~,
z,)[5 1, it implies that the autocorrelation function vanishes slowly as kjncreases.
In general, with only a single realization, it is difficult or impossible to make the statistical inferences of a process that is nonstationary in both the mean and the autocovariance or
autoco~elationfunction, FortunateIy, for the homogeneous nonstationary process, we can
apply a proper differencing to reduce it to stationary process. That is, although the original
series Zt is nonstationary, its properly differenced series Wt = (1 - B)~z, is stationary and
can be represented as an ARMA process
where
$(B)
(1 -
41B -
. - $+,BP)
and
e ( B ) = (1 - d l B -
.-
- 8,Bq)
have all their roots outside of the unit circle. Thus, the parameters #i, Bj, and a2 that control
the evolution of the nonstationary phenomenon of 2, can be estimated from the differenced
series W, in exactly the same way as the stationary case discussed in Chapter 7.
4.3.2 Variance Stabilizing Transformations

Not all nonstationary series can be transformed to stationary series by differencing. Many
time series are stationary in the mean but are nonstationary in the variance. To overcome this
problem, we need a proper variance stabilizing transformation.
It is very common for the variance of a nonstationary process to change as its level
changes. Thus,
84
for some positive constant c and function f. How do we find a function T so that the transformed series, T(Zr), has a constant variance? To illustrate the method, we approximate the
desired function by a first-order Taylor series about the point pt.Let
where T'(pJ is the iirst derivative of T(Z,) evaluated at p,. Now
Thus, in order for the variance of T(Z,) to be constant, the variance stabilizing transformation
T(Z,) must be chosen so that
Equation (4.3.12) implies that
For example, if the standard deviation of

Var(Z,) = c$, then
st
series is proportional to the level so that
Hence, a logarithmic transformation (the base is irrelevant) of the series, ln(Zt), will have a
constant variance.
Next, if the variance of the series is proportional to the level so that Var(Zt) = cp,, then
fi,
Thus, a squxe root transformation of the series,

will have a constant variance.
Third, if the standard deviation of the series is proportionai to the square of the level so
that Var(Z,) = cp;', then
Therefore, a desired transformation that has a constant variance will be the reciprocal IjZ,.
85
More generally, we can use the power transfonnation
introduced by Box and Cox (1964). This class of transformations contains many of the previously discussed transformations as special cases. For example, the following table shows
some commonly used values of A and their associated transformations.
Values of A (lambda)
Transformation
hZ,
2, (no transformation)
To see why X = 0 corresponds to the logarithmic transformation, we note that
One great advantage of using the transformation in (4.3.17) is that we can treat A as a transfarmation parameter and estimate its value from data. For example, we can include A as a parameter
in the model, (1 - +IB p ) = (1 - BIB - B,Bq)a, and
choose the value of X that gives the minimum residual mean square error. Box and Cox (1964)
show that the maximum likelihood estimate of A is the one that minimizes the residual mean
square error calculated from the fitted model on the 'hormalized" data
- - . $ p ~ ~ ) ( ~ -i A )
where
is the geometric mean of the data coming from the Jacobin of the transformation. Thus, for
A = 0, the residual mean square error is computed from the fitted model on
86
In a preliminary data analysis, one can use an AR model as an approximation to obtain the
value of A through an AR fitting that minimizes the residual mean square error. The choice
of the optimum value for A is based on the evaluations of the residual mean square error on a
grid of A values.
Some important remarks are in order.
1. The variance stabilizing transformations introduced above are defined only for positive
series. This definition, however, is not as restrictive as it seems because a constant can
always be added to the series without affecting the correlation structure of the series.
2. A variance stabilizing transformation, if needed, should be performed before any other
analysis such as differencing.
3. Frequently, the transfornation not only stabilizes the variance, but also improves the
approximation of the distribution by a normal distribution.
4.1 Consider the model
(a) Is the model for Zt stationary? Why?

(b) Let W, = (1 - B)~z,,Is the model for Wt stationary? Why?
(c) Find the ACF for the second-order differences Wt.
4.2
Consider the following processes:

(a) (1 - B I ~ Z=
, a, - .81at-.1 .38at-*,
(b) (I - B)Zt = (1 - .5B)a,.
Express each of the above processes in the AR representation by actually finding and plotting the .n weights.
4.3
(a) Simulate a series of 100 observations from each of the following models:
(i) (I - B)Z, = (1 - ,6B)a,,
(ii) (1 - BIZ, = .5 f (1 - .6B)a,,
(Ei) (1 - .9B)(1 - BIZ, = a,,
(iv) (1 - .9B)(1 - B)Z, = ( 1 - .5B)a,.
(b) Plot the simulated series.
and PACF,
(c) Calculate and discuss the behavior of the sample ACF,
k = 0, 1, . . , 2 0 for each simulated series and its differences.
4.4
&,
Suppose that Z, is generated according to Z, = a, -I- cat-1 -I- . . + cal, for t

where c is a constant,
(a) Find'the mean and covariance for 2,. Is it stationary?
(b) Find the mean and covariance for (1 - B)Z,. Is it stationary?
at
1,
Exercises
87
4.5 Consider the simple white noise process, 2,
= a,. Discuss the consequence of

overdifferencing by examining the ACF, PACF, and AR representation of the differenced series, jVt = 2, - Z,-l.
4.6
Let ZI, &, . . . , Znbe a random sample from a Poisson distribution with mean p.
(a) Show that the variance of Z, depends on its mean p.
@) Find a proper transformation so that the variance of the transformed variable is
independent of p.
(c) Find the variance of the transformed variable.
4.7 Let r, be the Pearson correlation coeff~cientfrom a sample size of n. It is

known that r,, is asymptotically distributed as N (p, (1 - p2)2/~t),where p is the
population correlation coefficient. Show that Fisher's z-transformation
Z = ln((1 r n ) / ( l - r,)) is actually a variance-stabilizing transformation.
Forecasting
Uncertainty is a fact of life for both individuals and organizations. Forecasting is essential for
planning and operation control in a variety of areas such as production management, inventory systems, quality control, financial planning, and investment analysis. In this chapter, we
develop the minimum mean square error forecasts for a time series following the stationary
and nonstationary time series models introduced in Chapters 3 and 4. Formulas will also be
developed to update forecasts when new information becomes available. We also discuss the
implication of the constructed time series model in terms of its eventual forecast function.
5.1 Introduction
One of the most important objectives in the analysis of a time series is to forecast its future
values. Even if the final purpose of time series modeling is for the control of a system, its
operation is usually based on forecasting, The termforecasrirlg is used more frequently in
recent time series literature than the term prediction. Most forecasting results, however, are
derived from a general theory of lineaf prediction developed by Kolmogorov (1939, 1941),
Wiener (1949), Kalrnan (1960), Yaglom (1962), and Whittle (1983), among others.
Consider the general ARIMA@, d, q) model
'
O(B) = (1 - BIB - . . - 8,Bq), and the series a,

where 4(B) = (1 - #qB - . . - ~,I~~BP),
is a Gaussian N(0, cr:) white noise process. The deterministic trend parameter Go is omitted
for simplicity but no loss of generality. Equation (5,l.l) is one of the most commonly used
models in forecasting applications. We discuss the minimum mean square error forecast of
2, following this model for both cases when d = 0 and d 0.

In forecasting, our objective is to produce an optimum forecast that has no error or as little
error as possible, which leads us to the minimum mean square error forecast. This forecast
will produce an optimum future value with the minimum error in terms of the mean square
error criterion.
89
5.2.1 Minimum Mean Square Error Forecasts for ARMA Models

To derive the minimum mean square enor forecasts, we first consider the case when d = 0
and ,u = 0, i.e., the stationary ARMA model
Because the mode1 is stationary, we can rewrite it in a moving average representation,
where
Suppose that at time t = n we have the observations Z,, Z,-I, Zn-2, . . , and wish to forecast I-step ahead of future value
as a linear combination of the observations Z,, ZR-l,
Zn-*, . . . . Because 2, for t = n, n - 1, a - 2, . . . %an alI be written in the form of (5.2.2),
we can let the minimum mean square error forecast Z,(l) of Zn+lbe
where the 1c,f are to be determined, The mean square error of the forecast is
which is easily seen to be minimized when
k ( ~= )$10,
Using (5.2.4) and that
we have
$rq
Hence,
+ $~+I%-I+ '!'r+zan-2
'
90
CHAPTER 5 Forecasting
Thus, the minimum mean square error forecast of Z,, is given by its conditional expectation. That is,
.?n(l)is usually read as the I-step ahead forecast of Zntr at the forecast origin n.
The forecast error is
Because ~ ( e , ( l ) ] ~t, ,5 n) = 0, the forecast is unbiased with the error variance
For a normal process, the (1 - a) 100% forecast limits are
where Nao is the standard normal deviate such that P(N > Nan) = a/2,
The forecast error e,(l) as shown in (5.2.8) is a linear combination of the future random
shocks entering the system after time n. Specifically, the one-step ahead forecast error is
Thus, the one-step ahead forecast errors are independent, which implies that Zn(l) is indeed
the best forecast of Zntl. Otherwise, if one-step ahead forecast errors are correlated, then one
can construct a forecast & + l
from the available errors a,, a,-l, an-?,. . , and hence improve the forecast of Zntl by simply using
&,+l as the forecast. The forecast errors for
longer Icad times, howevel; are correlated, This correlation is true for the fdrecast errors
in(l)+
and
which are made at the same lead time 1 but different origins n and n - j for j c I, It is
also true for the forecast errors for different lead times made from the same time origin.
For example, .
5.2
Minimum Mean Square Error Forecasts
91
Minimum Mean Square Error Forecasts for AFUMA Models

We now consider the general nonstationary ARJMA(p d, q) model with d P 0,is.,
5.2.2
..-
where 4 ( B ) = (1 - 4 E B - - $$$) is a stationary AR operator and 613) = (1 8 , -~ ... - 6,Bq) is an invertible MA operator, respectively. Although for this process
the mean and the second-order moments such as the variance and the autocovariance functions vary over time, as shown in Chapter 4, the complete evolution of the process is
completely determined by a finite number of fixed parameters. Hence, we can view the
forecast of the process as the estimation of a function of these parameters and obtain the
minimum mean square error forecast using a Bayesian argument. It is well known that
using this approach with respect to the mean square error criterion, which corresponds to a
squared loss function, when the series is known up to time n, the optimal forecast of Z,+) is
given by its conditional expectation
Zn-l, . . .).The minimum mean square error
forecast for the stationary ARMA model discussed earlier is, of course, a special case of
the forecast for the ARIMA(p, d, q) model with d = 0.
To derive the variance of the forecast for the general ARIMA model, we rewrite the
model at time t + I in an AR representation that exists because the model is invertible. Thus,
E(z, +~Iz, ,
where
or, equivalently,
Following Wegman (1986), we apply the operator
+ i,h13+ . . + $ l - l ~ i - l
to (5.2.18) and obtain
where no = - I and i,ho = 1. It can be easily shown that
92
Choosing $ weights so that
we have
where
Thus, given Z, for t 5 n, we have
because E(a,,+j[ Z,, t I11) = 0, for j > 0. The forecast enor is
where the $j weights, by (5,2,21), can be calculated recursively h m the
weights as follows:
Note that (5.2.25) is exactly the same as (5.2.8). Hence, the results given in (5.2.7) through
(5.2.14) hold for both stationary and nonstationary ARMA models,
1-1 2
For a stationary process, liml,,
E:l=o$j
exists. Hence, from (5.2.10), the eventual forecast limits approach two horizontally parallel lines as shown in Figure 5.l(a). For a nonstationary
process, liml,,
does not exist. In fact,
increases and becomes unbounded as
I -+ oo. Thus, the forecast limits in this case become wider and wider as the forecast lead 1
becomes larger Ad larger as shown in Figure 5.l(b). The practical implication of the latter case
is that the forecaster becomes less certain about the result as the forecast lead time gets larger.
For more discussion on the properties of the mean square error forecasts, see Shaman (1983).
x;J;$~
$.:~i#f

(a)
X Actual Observation
Forecast
+ Forecast Limit
FIGURE 5.1 (a] Forecasts for stationary processes. @) Forecasts for

nonstationary processes.
93
94
5.3 computation of Forecasts

A
From the result that the minimum mean square error forecasts &(I) of Zn+[at the forecast
origin n is given by the conditional expectation
we can easily obtain the actual forecasts by directly using the difference equation form of
the model. Let
The general ARIMA(p, d, q) model (5.2.15) can be written as the following difference equation form:
For t = n + I, we have
Taking the conditional expectation at time origin n, we obtain
where
and
W P L E 5.1 To illustrate the previous results, we consider the l-step ahead forecast
Zn(I)of Zn+[for'the following ARIMA(I,O, 1) or ARMA(1, 1) model:
5.3 Computation of Forecasts
95
1. Calculate the forecast. kn(l)as the conditional expectation from the difference equation form.
For t = n + I, we can rewrite the above model in (5,3.4) as
Hence,
and
2. Calculate the forecast error variance Var(e,(l)) = CT;$I~

$f.
When
< 1, we can calculate the $ weights from the moving average representation (5.2.2)with +(B) = (1 - 4B) and B(B) = (1 - BB). That is,
I+]
(1 - r,bB)(l
+ $lB + $
+~ . .~ .) = (1 - BB).
(5.3.7)
Equating the coeEcients of Bj on both sides gives
Hence, the forecast error variance becomes
which converges to q:[l
+ ( 4 - 0 ) ~ / ( 1 - @)] as l
-+
m.
When 4 = 1, which corresponds to taking the first difference, the model in (5.3.4)
becomes an IMA(1,l) process
i
needed for the forecast variwhere we note that (1 - B)p = 0. To calculate the $weights
ance, because the MA representation does not exist, we first rewrite (5.3.10) in an AR form
when time equals t + 1, i.e.,
96
where
n(B) = 1 - .rrlB- r n B 2 -
( 1 - B)
(1 - dB)
or
(1
- B)
- (nl + B)B - ( r 2- T ~ O ) B-~(
7 ~
r r 2 B ) ~ 3-
..
Equating the coefficients of ~i on both sides gives

=
(1 - ) * ,
j r 1.
(5.3.1 1)
Now, by applying (52.26) we obtain

$1
$2
a1
(1 - B ) ,
+ nl+l= (1 - 8)8 + (1 - 19)~= (1 - d),
That is, we have Gj = (1 - 8), 1 5 j

becomes
1 - I. Thus, the variance of e,(E), from (5.2,9),
which approaches w as I + oo.

As expected, (5.3.12) is the limiting case of (5.3.9) when 4 + 1, Thus, when 4 is close
to 1, the choice between a stationary ARMA(1, 1) model and a nonstationary MA(1, 1)
model has very different implications for forecasting.Jt can be seen even more clearly hfom
the limiting behavior of the I-step ahead forecast &(I) in (5.3.6b). For
4 1, Zn(l)
approaches the meat, p, of _the process as b +a.When # + 1, the first equation of
(5.3.6b) implies that Z,(l) = Z,(Z - 1) for all I. That is, the forecast is free to wander, with
no tendency for the values to remain clustered around a fixed level.
]#I
5.4 The ARIMA Forecast as a Weighted Average

of Previous Observations
Recall that we can always represent an invertible ARIMA model in an autoregressive representation. In this representation, Z, is expressed as an infinite weighted sum of previous
observations plus a random shock, i.e.,
or, equivalently,
5.4
The ARIMA Forecast as a Weighted Average of Previous Observations
97
where
Thus,
By repeated substitutions, we see that & ( I ) can be expressed as a weighted sum of current
and past observations Z, for f 5 n. For example,
where
More generally, it can be shown by successive substitutions that
where
and
T)" = Tj,
Thus, many smoothing results, such as moving averages and exponential smoothing, are
special cases of ARIMA forecasting. ARIMA models provide a very natural and optimal
way to obtain the required weights for forecasting. The user does not have to specify either
the number or the form of weights as required in the moving average method and the exponential smoothing method. Note dso that the ARlMA forecasts are minimum mean square
error forecasts. This optimal property is not shared in general by the moving average and the
exponentiaf smoothing methods.
98
For an invertible process, the .rr weights in (5,4.3) or (5.4.4) form a convergent series,
which impties &at for a given degree of accuracy, Z,,(I) depends only on a finite number of
recent observations. The associated ?r weights provide very useful information for many important managerial decisions.
EXAMPLE 5.2 For the ARMA(1, I) model in (5.3.4) with 181 c 1, we have, from (5.4.2),
with d -: 0,
Equating the coefficients of ~j on both sides gives
Assuming that p = 0,from (5.4.3), we have
&(r)
x(#
03
- e)ej-lk(r
- j).
j= 1
When I = 1, we get
For 1 = 2, from (5.4.4) and (5.4.5),
Again, in(2)is a weighted average of the previous observations with tbe ?rF)
weights given
by
= #(# - B ) B j - '
for j a 1. Note that by comparing (5.4.9) and
(5.4.10), we see that
iry)
which, as expected, agrees with (5.3.6b) with 1 = 2 and p = 0.
99
To see the implications bf these weights, let us examine (5.4.9) more carefully. For 161 < 1,
$-I(#8) converges to 0 as j * m, which implies that the more recent observations
have a greater influence on the forecast. For 2 1, although the model is still stationary, its
AR representation does not exist. To see the trouble, note that if 161 > 1, then the a weights
rapidly approach -# or +m as j increases, which iniplies that the more remote past observations have a much greater influence on the forecast. When = 1, the .rr weights become =
(4 - 1) for 0 = 1 and 9 = (- l)j-l(I + 4) for 8 = -1, which have the same absolute value
for all j. Hence, all past and present observations are equally important in their effect on the
forecast. Thus, a meaningful forecast can be derived only from an invertible process. The mod> 1 and 181 1 are both noninvertible.
els corresponding to /$[
=
I$(
]%I

Recall that when a time series 2, is available for t 5 n, the 1-step ahead minimum mean
square forecast error for the forecast origin n using the general ARMA model is obtained in
(5.2.25), which, for convenience, is listed again in the following:
In particular, the one-step ahead forecast error is
Clearly, the forecast error at the forecast origin (11 - 1) is

*
zn- zndI(l)
= an.
From (5.5, I), it is clear that
where
and
Hence, after substituting, rearranging, and using (5.5.33, we have the following updating
equation:
100
or, equivalently,
The updated forecast is obtained by adding to the previous forecast a constant multipIe
of the one-step ahead forecast error a,+l = Zn+l - Zn(l), which is certainly sensible. For
example, when the value ZM1 becomes available and is found to be higher than the previous
forecast (resulting in apositive forecast error a,+l =
&(I)), we wiU naturally modify
the forecast in(11) made earlier by proportionalty adding a constant multiple of this error,
5.6 Eventual Forecast Functions

Let the ARTMA model be
where $(B)
=;
4(B)(l - B ) ~Recall
.
from Equation (5.3.3) that
When I > q, 2,,(1)becomes
where B is now operating on I such that 23gn(l)= in([

- 1). That is, Zn I for 13 q satisfies
the homogeneous difference equation of order (p -I-d). Let P ( B ) = Jli=l(l- RiB)mt with
N
xi=lnri
= (p 4- d). Then, from Theorem 2.7.1, the general solution is given by
A
L)
which, as seen shortly, holds actually for all 1 2 (q - p - d + 1) and hence is often referred
to as the eventual forecast function. The
are constants that depend on time origin n. For
a given forecast origin n, they are fixed constants for all forecast lead times 1. The constants
change when the forecast origin is changed.
The reason that Equation (5.6.2) holds for all I 2 (g - p - d + 1) foltows from (5.6.2)
keing the splution of Equation (5.6.1) for I > q. Thus, when I = q + 1, the points
Z,,(q), . . ,Zn(q - p - d 1) actually iie on the curve represented by Equation (5.6.2).
5.6
Eventual Forecast Functions
101
In other wozds, theAfunction.in(5.62) is the unique curve that pass? through the (p + d) values
given by Z,(q), Zn(g - I), . . . , Zn(q - p - d + I), where Zn(-j) = Znlj for j 2 0.
These (p + d) values can serve as initial conditions for solving the constants in (5.6.2).
EXAMPLE 5.3 For the ARIMA(1, 0, 1) model given in (5.3.4),
the forecast &(I) satisfies the difference equation (1 - # B ) ( ~ ~ (-I )p )

eventual forecast function is given by
0 for 1 > 1. The
which holds for 1 r (q - p - d +- 1) = 1. Thus, the function passes through Zn(l), i.e.,
in(l)= a
and c?)= (&(l) - p)+d. Thus, & ( l ) = + [ i n ( l ) - p]$'-'.
As 1 -+ co, Zn(l) approaches the mean p of the stationary process as expected.
+ &'+
A
EXAMPLE 5.4 Consider the -A(
1, I , 1)model
The eventual forecast function is the solution of (1 - @)(I
- B)Z,(l)
= 0 and is given by
which holds for 1 2 (q - p - d t 1) = 0. Thus, the function passes through the

first forecast i n ( l ) and the last observation Z,, and they can be used to solve the constants. Because i n ( l ) =
+
and Z, = &(o) = df')+&), we have
cP) c F ~
cIn'=(in(1) - +zn)/(l
9), c?)=
- &(l))/(l
EXAMPLE 5.5 Consider the ARUIA(O,2,1)model
- 4); hence,
102
The eventual forecast function is the solution of ( 1 - B ) ~ & ( z )= 0 for 1

(5.6.2) it is given by
in(l) =
CY) + C$)'I,
> 1, and from

(5.6.6)
which holds for I r ( q - p - d + 1) = 0. Thus, the function is a straight line passing

through i n ( l ) and Z,. Because i , ( l ) = ~ j " )
and Zn = &(0) = Cj"), we have
Cln) = Zn,
= & ( I ) - &, andi,,(l) = Zn + (.?,,(I) - Zn)l.
+ cF)
CP'
ExiihfPLE 5.6 Consider the followingARlMA(1, 1, 1) model with a deterministic trend term
The eventual forecast function is the solution of (1 - +B)(l - B)$(I) = Bo for 1 > 1.
The solution for the homogeneous equation is given in Equation (5.6.4). For a particular solution, we note from the discussion in Section 4,2,2 that the deterministicterm do in this case
is related to the coefficient of a first-order deterministic time trend,
u!;l")l.
Thus,
CUP)+
Because ( I - B)(a[")+ ar'l) = ap! it follows that (1 - +B)aj")= Bo and ap)=

00 /(1 - +), and a particular solution becomes
which holds also for 1 > (q - p - d + 1) = 0. Combining the solution (5.6.4) of the homogeneous equation and a particular solution given in (5.6.9), we get the eventual forecast function
which passes through i n ( l ) and Zn, where b p ) and b(;) are combined new canstants
to be determined. Because Z (1)
+ b?) 4 f Bo/(l - 4) and Zn = Zn(O) =
b y ) by), we have bvJ = [(&(I) - 4Zn)(l - 4 ) - Bol/(l - $)',
by) =
[(Zn - i n ( l ) ) ( l - 4) + 001/(1 - +I2, and
=!?)
The difference between Equations (5.6.1 1) and (5.6.5) is drastic. When the forecast lead
time 1 is large, the forecast from model (5.6.7) is clearly dominated by the last term related
to a deterministic trend.
5.7
A Numerical Example
103
5.7 A Numerical Example

As a numerical example, consider the AR(1) model
with 4 = .6, p = 9, and cr: = .I. Suppose that we have the observations Zg7 = 9.5, Zgg = 9,
299 = 9, Zlm = 8.9, and and want to forecast Zlol, Zlm, Zlm, and Zlol with their associated
95% forecast limits. We proceed as follows:
1. The AR(1) model can be written as
and the general form of the forecast equation is
it[l)= P + + ( i t ( l - 1) - 4
=p
+ +yz, - p),
1 2 1.
Thus,
2. To obtain the forecast limits, we obtain the )t weights from the relationship
That is,
The 95% forecast limits forZlol fiom (5.2.10)are
The 95% forecast limits for Zroz.are
The 95% forecast limits for ZlO3 and ZlO4can be obtained similarly. The results are
shown in Figure 5.2.
104
8.2
Forecast
+ Forecast Limit
+
4-
8-4
FIGURE 5.2 Forecasts with 95% forecast limits for an ARC11 process.
3. Suppose now that the observation at t = 101 turns out to be Zlol = 8.8. Because +r =
#' = (.6)[,we can update the forecasts for ZIo2,Zlm, and Zloj by using (5.5.5) as follows:
The earlier forecasts for ZIM,ZIM,and zIo4

made at t
to the negative forecast error made for Zlol.
100 are adjusted downward due
This forecasting is based on the assumption that the parameters are known in the model.
In practice, the parameters are, of course, unknown and have to be estimated from the given
Exercises
105
observations {Z1, &, . . . ;Zn}, With respect to the forecasting origin t = n, however, the
estimates are known constants; hence, the results remain the same under this conditional
sense. Estimation of the model parameters is discussed in Chapter 7.
5.1 Consider each of the following models:

(0 (1 - 41BI(Zt - PI = at,
(i) (1 - 4 4 - # 2 ~ 3 ( ~-t P) = at,
(iii) (1 - 41B)(l - BIZ, = (1 - OIB)at.
(a) Find the I-step ahead forecast & ( I ) of Zn,l.
(b) Find the variance of the 6-step ahead forecast error for 1 = 1,2, and 3.
5.2 (a) Show that the covariance between forecast errors fiom different origins is given by
(b) Show that the covariance between forecast errors from the same origin but with
different lead times is given by
(a) Compute and plot thecomlations between the error of the forecast & ( 5 ) and
those of the forecasts Ztlj(5) for j = 1,2, . . . , 5 .
(b) Compute-and plot the correlations between the error of the forecast ~ ~ ( and
3 )
those of Z,(I) for I = 1,2, , , , ,5,
5.4 Verify Equation (5.2.20).

5.5 A sales series was fitted by the ARIMA(2, 1,O) model
where &: = 58,000 and the last three observations are Zn-2 = 640, Zn-l = 770,
and 2, = 800.
(a) Calculate the forecasts of the next three observations.
(b) Find the 95% forecast limits for the forecasts in part (a).
(c) Find the eventual forecast function.
106
5.6
Consider the IMA(1, 1) model
(a)
(b)
(c)
(d)
Write down the forecast equation that generates the forecasts, Z,(I).
Find the 95% forecast limits for & ( l ) .
Express the forecasts as a weighted average of previous observations.
Discuss the connection of this forecast ~ J z )with the one generated by the simple
exponential smoothing method.
5.7 (a) Show that (5.2.23) and (5.4.5) are equivalent.

(b) Illustrate the equivalence of (52.23) and (5.4.5) using the model in Exercise 5.6.
5.8 Consider the ARZMA(0, 1, 1) model
(a) Find the rr weights for the AR representation

Z, = Z, -t a,,
2,= 2;
(X1
and show that Xj. l q = 1.

be the two-step ahead forecasts of Z,+ 2at time t.
(b) Let i ( 2 ) =
Express ?rj2)in terms of the weights TI-)
where
5.9
= 1.2,
Consider an AR(2) model (1 - &B - &B2)(Z, - p) = a,, where
49 = - .6, p = 65, and vi = 1. Suppose that we have the observations q6= 60,4,
Zn = 58,9, = 64.7, ZT9=: 70.4, and Zso= 62.6.
(a) Forecast GI,Zg2,Zg3,and Zg4.
(b) Find the 95% forecast limits for the forecasts in part (a).
(c) Suppose that the observation at t = 81 turns out to be Zal = 62.2. Find the
updated forecasts for Za2,ZE3,and ZW.
= 33.4, ZSo= 33.9, and rri = 2.

and the observations
(a) Compute the forecast Z50(l), for 1 = 1,2, 3 and their 90% forecast limits.
(b) What is the eventual forecast function for the forecasts made at t = 50?
(c) At time t = 51, ZS1 became known and equaled 34.1. Update the forecasts
obtained in part (a).
5.11 Obtain the eventual forecast function for the following models:
(a) (1 - .6B)Z, = (1 - .8B .3~')a,,
(b) (1 - .3B)(1 - B)Z, = .4 f a,,
Exercises
107
(c) (1 - 1.2B - 6 ~(Zt

~) 65) = n,,
(d) (1 - B)~z,= 50 -I-(1 - .2B - .3ft2)at,
(e) (1 - .6B) (1 - B)~z, = (1 - .75B + . 3 4 ~ ~ ) a , .
5.12 Consider the AR(2) model, 2, = -2+ 1.8Z,-1 - .81Zrd2+ a,, where the a, is the
Gaussian white noise with mean 0 and variance 4, and Z47= 19, Z48 = 22, Za = 17,
and Z50= 21,
(a) Find the mean of the process.
(b) Find the variance of the process.
(c) Find the forecasts, ?50(l), for E = 1,2, and 3.
(d) Find the 95%forecast limits in part (c).
(e) Find the eventual forecast function for the forecast made at f = 50 and its h i t ing value,
Model Identification
In time series analysis, the most crucial steps are to identify and build a model based on the
available data. These steps require a good understanding of the processes discussed in
Chapters 3 and 4, particularly the characteristics of these processes in terms of their ACF, pk,
and PACF, I $ ~In. practice, these ACF and PACF are unknown, and for a given observed
time series ZI, &, . . . , and Z,, they have to be estimated by the sample ACF, kk, and PACF,
J M , discussed in Section 2.5. Thus, in model idenacation, our goal is to match patterns in
the sample ACF, Pk, and sample PACF,
with the known patterns of the ACF, pk, and
PACF, &, for the ARMA models. For example, because we know that the ACF cuts off at
lag 1 for an MA(1) model, a large single signscant spike at lag 1 for Fk will indicate an
MA(1) model as a possible underlying process.
After introducing systematic and useful steps for model identification, we give illustrative examples of identifying models for a wide variety of actual time series data. We also
discuss some other identification tools such as the inverse autocorrelation and extended sample autocorrelation functions.
aM,
6.1 Stem for Model Identification

-
To illustrate the model identification, we consider the general ARIMA(p, d, q) model
Model identification refers to the methodology in identifying the required transformations, such as variance stabilizing transformations and differencing transformations, the
decision to include the deterministic parameter Bo when d 2 1, and the proper orders of
p and q for the model. Given a time series, we use the following useful steps to identify a
tentative model.
Step 1. Plot the time series data and choose proper transformations.
In any time series analysis, the first step is to plot the data. Through careful examination
of the plot, we usually get a good idea about whether the series contains a trend, seasonality,
outliers, nonconstant variances, and other nonnormal and nonstationary phenomena. This
understanding often provides a basis for postulating a possible data transformation.
6.1 Steps for Model Identification
109
In time series analysisj the most commonly used transformations are variance-stabilizing
transformations and differencing. Because variance-stabilizing transformations such as the
power transformation require non-negative values and differencing may create some negative
vdues, we should always apply-variance-stabilizing transformations before taking differences. A series with nonconstant variance often needs a logarithmic transformation. More
generally, to stabilize the variance, we can apply Box-Cox's power transfornation discussed
in Section 4.3.2. Because a variance-stabilizing transformation, if necessary, is often chosen
before we do any further analysis, we refer to this transformed data as the original series in
the following discussion unless mentioned otherwise.
Step 2. Compute and examine the sample ACF and the sample PACF of the original series
to further confirm a necessary degree of differencing so that differenced series is stationary.
Some general rules are the following:
1, Lf the sample ACF decays very slowly (the individual ACF may not be large) and the
sample PACF cuts off after lag 1, then it indicates that differencing is needed. Try taking
the fist differencing (1 - B)Z,. One can also use the unit root test proposed by Dickey
and Fuller (1979). In a borderline case, differencing is generally recommended (see
Dickey, Bell, and Miller [1986]).
2. More generally, to remove nonstationarity we may need to consider a higher-order differencing (1 - B ) ~ z ,for d > 1. In most cases, d is either 0, 1, or 2. Note that if
(1 - B ) ~ z is
, stationary, then (1 - B ) ~ + ~for
z , i = 1, 2, , , are also stationary. Some
authors argue that the consequences of unnecessary differencing are much less serious
than those of underdifferencing, but do beware of the artifacts created by overdifferencing so that unnecessary overparameterization can be avoided.
Step 3. Compute and examine the sample ACF and PACF of the properly transformed and differenced series to identify the orders of p and q where we recall that p is the highest order in the
autoregressive polynomial (1 - #lB - &BP), and q is the highest order in the moving
average polynomial (1 - fllB - . - - B,Bq). Usually, the needed orders of these p and q are
less than or equal to 3. TabIe 6.1 summarizes the important results for selectingp and q.
A strong duality exists between the AR and the MA models in terms of their ACFs and
PACFs. To identify a reasonably appropriate ARSMA model, ideally, we need a minimum of
n = 50 observations, and the number of sample lag-k autocorrelations and partial autocorrelations to be calculated should be about 11/4, although occasionally for data of good quality
one may be able to identify an adequate model with a smaller sample size.
TABLE 6.1 Characteristics of theoretical ACF and PACF for stationary processes.
Process
ACF
AREP )
Tails off as exponentia1 decay or

damped sine wave
Cuts off after lag p
MA(q)
Cuts off after lag q
Tails off as exponential decay or

damped sine wave
ARMA(P, q )
Tails off after lag (q - p)
Tails off after lag (p - q )
PACF
110
CHAPTER 6 Model Identification

We identify the orders p and q by matching the patterns in the sample ACF and PAW with
the theoretical patterns of known models. Thl: art of a time series analyst's model identification
is very much like the method of an FBI agent's criminal search. Most criminals disguise themselves to avoid being recognized, ~vhichis also fnte of ACF and PACF. The s m p h g variation
and the cornlation among the sample ACF and PACF as shown in Section 2.5 often disguise the
theoretical ACF and PACF patterns. Hence, in the initial model idenGcation we always concentrate on the general broad features of these sampleACF and PACF without focusing on the fine
details. Model improvement can be easily achieved at a later stage of diagnostic checking.
The estimated variances of both the sample ACF and PACF given in (2.5.21) and (2.5.27)
are very rough approximations. Some authors recommend that a conservative threshold of
1.5 standard deviations be used in checking the significance of the short-term lags of these
ACF and PACF at the initial model identification phase, especially for a relatively short series.
Step 4. Test the deterministic trend term Oo when d > 0.

As discussed in Section 4.2, for a nonstationary model, q!t(B)(l - B)d Zt = f B(B)a,, the
parameter Bo is usually omitted so that it is capable of representing series with random changes
in the level, slope, or trend, If there is reason to believe that the diEerenced series contains a
deterministic trend mean, however, then we can test for its inclusion by comparing the sample
mean
of the differenced series W, = (1 - B ) 2,
~ with its approximate standard error Ss.
To derive Sc,we note kom Exercise 2.11 that lim,,,n
~ a r ( m )= E;, yj; hence,
where y(B) is the autocovariance generating function defined in (2.6.8) and y(1) is its value
at B = 1. Thus, the variance and hence the standard error for
is model dependent, For
example, for i h e m A ( 1 , d, 0) model, (1 - +B)Wt = a,, we have, from (2.6.9),
so that
where we note that a$ = mj/(1 -
42)The
. required standard error is
Expressions of Sv for other models can be derived similarly. At the model identification
phase, however; because the underlying model is unknown, most available software use the
where
.Go is the sample variance and ;I,. ..,$k
11 1
are the fist k significant sample ACFs of
{ W,}.
Under the null hypothesis pk = 0 fork 2 1, Equation (6.1,5) reduces to
Alternatively, one can include do initially and discard it at the final model estimation if the
preliminary estimation result is not significant. Parameter estimation is discussed in Chapter 7.

In this section, we present a variety of real-world examples to illustrate the method of model
identiiication. Many computer programs such as AUTOBOX, EVIEWS, MINITAB, RATS,
SAS, SCA, SPLUS, and SPSS are available to facilitate the necessary computations. Some
programs are available for both the mainframe and personal computers.
EXAMPLE 6.1 Figure 6.1 shows Series Wl, which is the daily average number of defects
per truck found in the final inspection at the end of the assembly line of a truck manufacturing
plant. The data consist' of 45 daily observations of consecutive business days between
FIGURE 6.1 Daily average number of truck manufacturing defects

(Series W l ) .
112
TABLE 6.2 Sample ACF and sample PACF for the daily average number of truck
manufacturing defects (Series Wl).
November 4 to January 10, as reported in Burr (1976, p. 134). The figure suggests a stationary process with constant mean and variance. The sample ACF and sample PACF ate reported
in Table 6.2 and plotted in Figure 6.2. That the sample ACF decays exponentially and the
sample PACF has a single significant spike at lag 1 indicates that the series is likely to be
generated by an AR(1) process,
EXAMPLE 6.2 Series W2 is the classic series of the Wolf yearly sunspot numbers between
1700 and 2001, giving a total of n = 302 observations. Scientists believe that the sunspot numbers affect the weather of Earth and hence human activities such as agriculture, telecommunications, and others. The Wolf sunspot numbers have been discussed extensively in time series
literature, for exampte,YuIe (19271, BartIett (1950), Whittle (1954), and Brillinger and Rosenblatt
(19671, This series is also known as the Wolfer sunspot numbers, named after Wolfer, who was a
student of Wolf. For an interesting account of the history of the series, see Izenman (1985).
FIGURE 6.2 Sample ACF and sample PACF of the daily average number
of truck manufacturing defects (Series Wl).
6.2
2700
1750
1800
1850
Year
1900
Empirical ExampIes
1950
2 13
2000-
FIGURE 6.3 Wolf yearly sunspot numbers, 2 700-2001 (Series W2).

The plot of the data is given in Figure 6.3. It indicates that the series is stationary in the
mean but may not be stationary in variance. To investigate the required transformation for variance stabilization, we use the power transformation analysis discussed in Section 4.3.2 with the
result given in Table 6.3. It suggests that a square root transformation be applied to the data.
The sample ACF and sample PACF of the transformed data are shown in Table 6.4
and Figure 6.4. The sample ACF shows a dmping sine-cosine wave, and the sample
PACF has relatively large spikes at lags 1,2, and 9, suggesting that a tentative model may
be an ARf2),
TABLE 6.3 Residual mean square

errors in the power transformation.
h
Residual mean square

error
1.O
0.5
0.0
-0.5
-1.0
277.81
241.26
256.43
6,964.83
2,040,887.01
114
CHAPTER 6 Mode1 Identification
TGBLE 6.4 Sample ACF and sample PACF for the square root transformed sunspot
numbers [Series W2).
FIGURE 6.4 Sample ACF and sample PACF for the square root transformed
sunspot numbers (Series W2).
By ignoring the values of $fi beyond lag 3, Box, Jenkins, and Reinsel (1994) suggest that an
- C) = (1,. is also possible, even though
AR(3) model, (1 - # J ~ -B ~
6.2
Empirical Examples
11 5
their analysis was based on the nontransformed data between 1770 and 1869. Because of the
large autocorrelation of .67 at lag 11,many scientists believe that the series is cyclical with a
period of 11 years. We examine this series more carefully in later chapters.
EXAMPLE 6.3 Series W3 is a laboratory series of blowfly data taken from Nicholson (1950).
A fixed number of adult blowflies with balanced sex ratios were kept inside a cage and given a
fixed amount of food daily. The blowfly popufation was then enumerated every other day for approximately two years, giving a total of n 364 observations. BriIiinger et al. (1980) h t applied
time series analysis on the data set. Later Tong (1983) considered the following two subseries:
Blowfly A:
for Z, between 20
Blowfly 13:
for Z, between 218
t 5 145,
5 f 5
299,
and argued that the series Blowfly A is possibly generated by a nonlinear modei. Series W3
used in our analysis is the series Blowfly B of 82 observations as shown in Figure 6.5.
The data plot suggests that the series is stationary in the mean but possibly nonstationary
in variance. The power transformation analysis as shown in Table 6.5 indicates that no
transformation is needed and the series is stationary in variance. The sample ACF_and PACF
are shown in Table 6.6 and Figure 6.6. The sample decays exponentially and #Akcuts off
after Jag 1. Thus, the following AR(1) model is entertained:
Fk
FIGURE 6.5 Blowfly data (Series W3).
116
TMLE 6.5 Results of the power

transformation oii blowfly data.
Residual mean square error
TABLE 6.6 Sample ACF and sample PACF for the blowfly data (Series W3).
10
FIGURE 6.6 Sample ACF and sample PACF for the blowfly data (Series W3).
EXAMPLE 6,4 Series W4 is the monthly number of unemployed young women between
ages 16 and 19 in the United States from January 1961 to August 2002 as shown in Figure
4.1. It is clearly nonstationary in the mean, suggesting differencing is needed. This conclusion
is further confumed by the sustained large spikes of the sample ACF shown in Table 6.7
and Figure 6.7. The differenced series as shown in Figure 6.8 seems to be stationary.
117
TABLE 6.7 Sample ACF and sample PACF of Series W4: the U.S. monthly series of
unemployed young women between ages 16 and 19 from January 1961 to August 2002.
FIGURE 6.7 Sample ACF and sample PACF of Series W4: the U.S. monthly series of
unemployed young women between ages 16 and 19 from January 1961 to August 2002.
The sample ACF and sample PACF for this differenced series are reported in Table 6.8 and
also plotted in Figure 6 9 . The sample ACF now cuts off after lag 1 while the sample PACF
tails off. This pattern is very similar to the ACF and PAW for MA(1) with positive O1 as
shown in Figure 3.10, suggesting an MA(1) model for the differenced series and hence an
IMA(1, 1) model for the original series. To test whether a deterministic trend parameter Oo is
needed, we calculate the t-ratio ~ / S E= .3707/1.925 = .1926, which is not significant.
Thus, our proposed model is
EXAMPLE 6.5 The death rate is of major interest for many state and federal governments.
Figure 6.10 shows Series W5, which is the yearly cancer death rate (all forms, per 100,000
population) of Pennsylvania between 1930 and 2000 published in the 2000 Pennsylvania Vital
Statistics Annual Report by the Pennsylvania Department of Health. The series is clearly
nonstationary with an increasing trend. This nonstationarity is also shown by the slowly
decaying ACF in Table 6.9 and Figure 6.11. Both Figure 6.11 and the evaluation of power
118
1960
1970
1980
Year
1990
2000
FIGURE 6.8 The differenced monthly series, W, = (1 - B)Z,, of the U.S. rnonthIy
series of unemployed young women between ages 16 and 19 from January 1961 to
August 2002 (Series W4).
TGBLE 6.8 Sample ACF and sample PACF of the differenced U S . monthly series,
Wt = (1 - BIZ, of unemployed young women between ages 16 and 19 (Series W4).
transformations indicate no transformation other than differencing is needed. The sample

ACF and PACP of the differenced data shown in Table 6.10 and Figure 6.12 suggest a white
noise process for the differenced series. The t-ratio, W/sW= 2,0543/.3336 = 6.158, implies
that a determi-nistic trend term is recommended. Hence, the following random walk model
with drift is entertained:
(1
- BIZ, = O0 + a,.
(6.2.6)
6.2
FIGURE 6.9
W, = (1
Empirical Examples
1 19
Sample ACF and sample PACF of the differenced U.S. monthly series,
of unemployed young women between ages 16 and 19 [Series W43.
- BIZ,
194
1960
1980
2000
Year
FIGURE 6.10 Series W5, the yearly Pennsylvania cancer death rate
between 1930 and 2000.
Based on the sample ACF and PACF of the original nondifferenced data in Table 6.9, one
may also suggest an alternativeAR(1) model
to be close to 1. We will investigate both

The clear upward trend, however, suggests
models in Chapter 7 when we discuss parameter estimation.
120
TGBLE 6.9 Sample ACF and sample PACF of the Pennsylvania cancer death rate
between 1930 and 2000 [Series W5).
FIGURE 6.1 1 Sample ACF and sample PACF of Series WS, the yearly
Pennsylvania cancer death rate between 1930 and 2000.
EXAMPLE 6.6 We now examine Series W6, which is the yearly U.S. tobacco production
from 1871 to 1984 published in the 1985 Agricultural Statistics by the United States DepaEtment of Revenue and shown in Figure 4.2, The plot indicates that the series is nonstationary
both in the mean and the variance, In fact, the standard deviation of the series is changing in
time roughly in proportion to the fevel of the series. Hence, from the results of Section 4,3.2,
a logarithmic transformation is suggested, These transformed data are plotted in Figure 6.13
and show an upward trend with a constant variance.
The very slowly decaying ACF as shown in Table 6.11 and Figure 6.14 further supports
the need for dierencing.Hence, the sample ACF and PACF for the differenced data, W,=
(I - B ) h Z, are reported in Table 6.12 with their plots in Figure 6.15. The ACF cuts off after
lag 1, and the PACF tails off exponentially, which looks very similar to Figure 3.10 with 81 > 0.
It suggests that an IMA(1, 1) is a possible model. To determine whether a deterministic trend
term Bo is needed, we examine the t-ratio, t = R/sw = .0147/.0186 = ,7903, which is not
significant. Hence, we entertain the following IMA(1,l) model as our tentative model:
12 1
TABLE 6.10 Sample ACF and sample PACF for the differenced series of the
Pennsylvania cancer death rate between 1930 and 2000 [Series W53.
FIGURE 6.12 Sample ACF and sample PACF for the differenced series
of the Pennsylvania cancer death rate between 1930 and 2000 (Series WS].
EXAMPLE 6.7 Figure 6.16(a) shows Series W7, the yearly number of lynx pelts sold by
the Hudson's Bay Company in Canada between 1857 and 1911 as reported in Andrew and
Herzberg (1985). The result of the power transformation in Table 6.13 indicates that a logarithmic transformation is required. The natural logarithm of the series is stationary and is
plotted in Figure 6,16(b),
The sample ACF in Table 6.14 and Figure 6.17 show a cle,ar sine-cosine phenomenon
indicating an AR@) model with p r 2. The three significant +w, strongly suggest p = 3.
Thus, our entertained model is
(1 - +lB - & B ~- &B3)(lnzt - p) = a,.
(6.2.9)
A related series that was studied by many time series analysts is the number of
Canadian lynx trapped for the years from 1821 to 1934. References include Campbell and
Walker (19771, Tong (1977), Priestley (I 981, Section 5 . 3 , and Lin (1987). The series of
lynx pelt sales that we analyze here is much shorter and has received much less attention in
the literature. We use this series extensively for various illustrations in this book.
122
1870
1890
1910
1930
1950
1970
1990
Year
FIGURE 6.13 Natural logarithms of the U.S. yearly tobacco production in million
pounds (Series W6).
TABLE 6.1 1 Sample ACF and sample PACF for natural logarithms of the U.S. yearly
tobacco production (Series W6).
6.3 The Inverse Autocorrelation Function (IACF)

Let
be an -A@,
q) model where 4 J B ) = ( 1
autoregressive operator, B,(B) = (1 - BIB - .
#qB -
- . - B,Bq
. . . - &BP) is a stationary
is an invertible moving average
123
FIGURE 6.14 Sample ACF and sample PACF for natural logarithms of the U.S.
yearly tobacco production (Series W63.
TABLE 6.12 Sample ACF and sample PACF for the differenced series of natural
logarithms of the U,S. yearly tobacco production (Series W6).
operator, and the a, is a white noise series with a zero mean and a constant variance ui,
We can rewrite Equation (6.3.1) as the moving average representation
where +(B) = Bq(B)/$,(B). From (2,6,9), the autocovariance generating function of this
model is given by
124
FIGURE 6.15 Sample ACF and sample PACF for the differenced natural
logarithms of the tobacco data (Series W6).
Year
Year
FIGURE 6.16 The Canadian lynx pelt sales data (Series W7). [a) The yearly number
of lynx pelts sofd in Canada between 1857 and 1911. @) The natural logarithms
of yearly numbers of lynx pelts sold in Canada between 1857 and 1911.
Assume that y(B) # 0 for a11 1
3
1 = 1 and let
125
TABLE 6.13 Results of the power

transformation on the lynx pelt sales.
h
Residual mean square error
TABLE 6.14 Sample ACF and sample PACF for natural logarithms of the yearly
number of lynx pelts sold (Series W7).
and Bq(B), the coefficients {&)} in (6.3.4) form a posiClearly, &om the assumptions on
tive definite, square surnmable sequence that is symmetric about the origin. Hence, they are
proper autocovstriances for a process. With respect to the model given in (6.3.11, the process
having #')(B) as its autocovariance generating function is referred to as the inverse process.
Hence, y(I1(B) is also referred to as the inverse autocovariance generating function of {Z,). Natwdly, the inverse autocorrelation generating function of {Z,), from (2.610);is given by
The kth lag inverse autocorrelation is defined as
which, of course, is equal to the coefficient of B~ or B - ~

in p ( o ( ~ )As
. a function of k,
called the inverse autocorrelation function (IACF).
pi!is
126
C M E R 6 Model Identification
FIGURE 6.17 Sample ACF and sample PACF for naturaI logarithms of the yearly
number of lynx pelts sold (Series W7),
From Equations (6.3.3) and (6.3.4), it is clear that if (Z,) is an ARMA (p, q) process,
then the inverse process will be an ARMA(q, p) process. Specifically, if (Z,] is an AR(p)
process with autocorrelations tailing off, then the inverse process will be an MA@)
process with its autocorrelations cutting off at lag p, I n other words, the inverse autocorrelation function of an AR@) process will cut off at lag p. Similarly, an MA(q) process
with autocorrelations cutting off at lag q will have its inverse autocorrelations tailing off.
Hence, the inverse autocorrelation function of a process exhibits characteristics similar to
the partial autocorrelation function and can be used as an identification tool in model
identification.
For an M(p)process, it is easily seen that the inverse autocorrelation function is given by
Because one can approximate an ARMA model by an AR(p) model with a suficiently largep,
one method to obtain a sample inverse autocorrelation
is to replace the AR parameters of
the approximate AR model by their estimates in (6.3.7); i.e.,
127
The parameter estimation is discussed in Chapter 7. Under the no11 hypothesis of a white
noise process, the standard error of $) is given by
Thus, one can use the limits f2 1 6 to assess the significance of the sample inverse autocorrelations.
EXAMPLE 6.8 As illustrations, Table 6.15 shows sample inverse autocorrelations for two
time series examined in Section 6.2, Table 6.15(a) is the sample inverse autocoxrelation for
the daily average series of truck manufacturing defects discussed in Example 6.1 in which
we entertained an AR(1) model for the series. Note that although the inverse autocorrelation
function seems to cut off at lag 1, it is not statistically significant at a = -05.Table 6,15(b) is
the sample inverse autocorrelation for the natural logarithms of the Canadian lynx pelt
sales that was identified as an AR(3) model based on the three significant PACF examined in
Example 6.7. In terms of the LACF, however, the maximum AR order will be 2. In fact,
because the standard error of the sample IACF is -14, the model implied by the IACF may be
an AR(l), which is clearly not appropriate from the sample ACF in Table 6.14(a).
It is not unusual that the values of the IACF are smaller than those of PACF in the above
= p f ) = 0 for k > p.
examples. If the underlying modd is an A&!@), then we know that
Equation (6.3.7), however, implies that
and from the discussion in the closing paragraph of Section 2.3, we have
TGBLE 6.15 The sample inverse autocorrelation function (SfACFJ.
(a) The daily average number of truck manufacturing defects (Series W1)
1
2
3
4
5
6
7
8
9
10
(b] The natural logarithms of Canadian lynx pelt sales (Series W7)
1
2
3
4
5
6
7
8
9
10
128
Hence, I#,l
> Ipf)l, and values of the sample IACF in general are smaller than values of the
sample PACF, particularly at lower lags. In one study, Abraham and Ledolter (1984) conclude
that, as an identification aid, the PACF generally outperforms the IACE Some computer programs such as SAS (1999) and SCA f 1992) provide both PACF and IACF options for analysis.
The inverse autacorrelation function was &st introduced by Cleveland (1972) through
the inverse of a spectrum and the relationship between the spectrum and the autocorrelation
function, leading to another method to obtain a sample inverse autocorrelation function. We
return to this point in Chapter 12.
6.4 Extended Sample Autocorrelation Function

and Other Identification Procedures
The autocorrelatian function, the partial autocorrelation function, and the inverse autoconelation function are useful in identifying the orders of AR and MA models. In this section, we
introduce the extended sample autocodation function and orher procedures that wiI1 help
us identify the orders of mixed ARMA models.
6.4.1 The Extended Sample Autocorrelation Function (ESACF)

From the previous empirical examples, it seems clear that, due to the cutting off property of
the PACF and MCF for AR models and the same cutting off property of the ACF for MA
models, the identitication of the order p of an AR model and the order q of an MA mode1
through the sample ACF, PACF, and WCF are relatively simple. For a mixed ARMA
process, however, the ACE PACF, and IACF all exhibit tapering off behavior, which makes
the identification of the orders p and q much more difficult. One commonly used method
considers that if Z, follows an ARMA(p, q) model
or, equivalently,
then
follows an
MA(^) model
=
( 1 - e,B
- ..-- 9, Bq)a,,
6.4 Extended Sample Autocorreiation Function and Other Identification Procedures
129
where without loss of generality we assume Bo = 0. Thus, some authors such as Tiao and
Box (1981) suggest using the sample ACF of the estimated residuals
from an ordinary least squares (OLS) AR fitting to identify q and hence obtaining the orders p
and q for the ARMA(p, q) model. For example, an MA(2) residual process from an AR(1) fitting implies a mixed ARMA(l,2) model. But as we show in Section 7.4, however, because the
OLS estimates of the AR parameter #is in (6.4.4) are not consistent when the underlying
model is a mixed ARMA(p, q) with q > 0, the procedure may lead to incorrect identification.
To derive consistent estimates of
suppose that n observations adjusted for mean are
available from the ARMA(p, q) process in (6.4.Ia).If an AR(p) model is fitted to the data,
i.e., if
#J~,
where e, represents the error term, then the OLS estimates

inconsistent and the estimated residuals
of +i, i = 1, . . . ,p, will be
zFi,
will not be white noise. In fact, if q 2 1, then the lagged values

i = I, . . . ,q, will contain some information about the process Z,, which leads to the following iterated regressions.
First, we consider an AR(p) regression plus an added term i?:?
i.e.,
,,
where the superscript (1) refers to the fi5t iterated regression and e;') represents the correspondingarror term. The OLS estimates
will be consistent if q = 1. I f q > 1, however,
then the #(" will again be inconsistent, the estimated residuals Zjf) are not white noise, and
the lagged values 2!?i will contain some information about Z,, We thus consider the second
iterated AR(p) regression
+I1'
$y)
The OLS estimates

will be consistent if q = 2. For q > 2, the
will again be inconsistent. Consistent est@ates can be .obtained by repeating the above iteration, however. That
is, the OLS estimates #!q) obtained from the following qth iterated AR(p) regression will be
consistent:
130

where Lii) = 2, - X C ~ $ ~ ~ ) Z-, 861[{"
-~
?'i-;Ois the estimated residual of the fi
iterated AR(p) regression and the Jim and
are the corresponding least squares estimates.
In practice, the true order p and q of the ARMAO-,, q) model are usually unknown and
have to be estimated. Based on the preceding consideration, however, Tsay and Tiao (1984)
suggest a general set of iterated regressions and introduce the concept of the extended sample autocorrelaJion function (ESACF) to estimate the orders p and q. Specifically, for m = 0,
1, 2, . . . , let #, i = 1, . . . , nt, be the OLS estimates from the jth iterated AR(m) regression of the ARMA process Z,. They define the mth ESACF at lag j,
of Z, as the sample
autocorrelation function for the transformed series
PY
by),
in a two-way table as shown in Table 6.16 where the first row

It is useful to mange
corresponding to $) gives the standard sample ACP of Z,, the second row gives the
first ESACF ;jl), and so on. The rows are numbered 0, 1, . . . to specify the AR order, and the
columns are numbered in a similar way for the MA order.
is a function of r l , the number of observations, even though
Note that the ESACF
it is not explicitly shown. In fact, it can be shown (see Tsay and Tiao [1984]) that for an
ARMA(p, q ) process, we have the following convergence in probability, i.e., form = 1,2,
. . ,and j = 1,2, . . . ,we have
+ 0,
OIm-pj-q,
otherwise.
Thus, by (6.4.10), the asymptotic ESACF table for an ARMA(1, 1) model becomes the one
shown in Table 6.17. The zeros can be seen to form a triangle with the vertex at the (1, 1)
position. More generally, for an AIZMA@, q) process, the vertex of the zero triangle in the
asymptotic ESACF will be at the (p, q) position, Hence, the ESACF can be a useful tool in
model identification, particularly for a mixed ARMA model.
Of course, in practice, we have finite samples, and the ;@) for 0 5 m - p < j - q may
X can be approximated using
not be exactly zero. However, the asymptotic variance of
Bartlett's formula or more crudely by (ti - m - j)-I on the hypothesis that the transformed
pfm),
TABLE 6.16 The ESACF table.
6.4 Extended Sample Autocorrelation Function and Other Identification Procedures
131
TMLE 6.17 The asymptotic ESACF for

an ARMA(1, 1) model.
MA
AR
..
series ~ l j of) (6.4.9) is white noise. The ESACF table can then be constructed using indicator
symbols with X referring to values greater than or less than 33 standard deviations and 0 for
values within f2standard deviations.
EXAMPLE 6.9 To illustrate the method, we use SCA to compute the ESACF for the natural logarithms of Canadian lynx pelt sales discussed in Example 6.7. Table 16.18(a) shows
the ESACF and Table 6.18(b) the corresponding indicator symbols for the ESACF of the
series. The vertex of the triangle suggests a mixed ARMA(2, 1) model. This model is different from an AR(3) model that we entertained earlier based on the characteristic of the PACE
We further examine the series in Chapter 7.
TABLE 6.18 (a) The ESACF for natural logarithms of Canadian lynx pelt sales.
(a) The ESACF
(b] The ESACF with indicator symbols
132
TABLE 6.19 The ESACF of Series C in Box, Jenkins, and Reinsel.
0
1
2
3
4
5
X
X
0
X
X
X
X
0
X
X
X
X
0
X
X
0
0
0
0
X
X
0
0
0
0
X
x
0
X
X
0
0
0
0
0
0
X
X
0
EXAMPLE 6.10 As another example, Table 6.19 shows the ESACF for Series C of 226
temperature readings in Box, Jenkins, and Reinsel (1994). Because the vertex of the zero triangle occurs at the (2,O)position, the ESACF approach suggests an AR(2) model for this series.
OLS estimation is used in the iterated regression, Because the OLS regression estimation
can always be applied to a regression model regardless of whether a series is stationary or nonstationary, invertible or noninvertible, Tsay and Tiao (1984) allow the roots of AR and MA
polynomials to be on or outside the unit circle in their definition of the ESACE Hence, the
ESACF is computed fiom the original nondifferenced data. As a result, the ARIMA(p d, q)
process is identified as the ARMA(e q) process with P = p d. For example, based on the
sample ACF and PACF of Series C as given in Table 6.20, Box, Jenkins, and Reinsel (1994)
suggested an ARIMA(1, 1,0) model or an ARIMA(O,2, Of model. Both these ARIMA(1, I, 0)
and ARIMA(0, 2, 0) rnodefs, however, are identified as an ARMA(2, 0)model in term of
the ESACF,
Because the ESACF proposed by Tsay and Tiao (1984) is defined on the original series,
they argue that the use of ESACF eliminates the need for differencing and provides a unified
identification procedure to both stationary and nonstationary processes. To see whether a
series is nonstationary, they suggest that for given specified values of p and q, the iterated
AR estimates can be examined to see whether the AR polynomial contains a nonstationary
factor with roots on the unit circle, For example, i? the previous ~e'i,& C in Box, Jenkins,
y d Reinsel (1994), the iterated AR estimates are 41') = 11.1 and 410) = -32. Because
+to)(3) (1 - 3)(1 - ,8B), it indicates that the AR polynomial contains the factor (1 B). Other than a few nice exceptions, however, the task of identifying nonstationarity
through this approach is generally difficult. The real advantage of the ESACF is for the identification of p and q for the mixed ARMA models. This advantage can be used much better if
the ESACF is used for a properly transformed stationary series, particularly because a tentatively identified model will be subjected to more efficient estimation procedures (such as the
maximum likelihood estimation), which generally require stationarity.
Due to sampling variations and correlations among sample ACF, the pattern in the
ESACF table from most time series may not be as clear-cut as those shown in the above
examples. From the author's experience, though, models can usually be identified without
much difficulty through a joint study of ACF, PACF, and ESACF.
6.4 Extended Sample Autocorrelation Function and Other Identification Procedures
133
TABLE 6.20 Sample ACF and PACF of Series C in Box, Jenkins, and Reinsel (1994).
Some computer programs such as AUTOBOX (1987) and SCA (1992) provide the option
to compute the ESACF in the model identification phase.
6.4.2 Other Identification Procedures

Other model identification procedures include the information criterion (AIC) proposed by
Akaike (1974b); the R-and-S-array introduced by Gray, Kelley, and Mchtire (1978); and the
comer method suggested by Beguin, Gourieroux, and Monfort (1980). The statistical properties of the statistics used in the R-and-S-array approach and the corner method are still
largely unknown, and the software needed for these methods is not easily avaitable. Interested readers are referred to their original research papers listed in the reference section of
this book. The information criterion,is discussed in Chapter 7.
At this point, it is appropriate to say that model identification is both a science and an
art, One should not use one method to the exclusion of others. Through careful examination
of the ACF, PACF, IACF, ESACF, and other properties of time series, model identification
truly becomes the most interesting aspect of time series analysis.
134
6.1 Identify appropriate AMMA models from the sample ACF below, Justify your
choice using the knowledge of the theoretical ACF for ARIMA models.
(a) n = 121, data = 2,
k
1
2
3
4
5
6
7 8
9 1 0
?k
,15 -.08
.04 .08
-08 -03 .02 .05 .04 -.I1
(b) n = 250, data = 2,
k
1
2
3
;k
-.63
.36 -.I7
4
5
.09 d.07
(c) n = 250, data =: 2,

k
1
2
3
4
6 k -.35 -.I7
.09 -.06
(d)
6
7
-06 -.08
5
6
.01 -.01
7
-.04
8
9
.10 -.I1
10
.06
8
9
.07 -.07
10
.09
W = 2,5, S$ = 20.
7
8 9 1 0
6
3 4
5
1
2
-99 -98 .98 .97 .94 .91 .89 -86 3 5 .83
C W ( ~ ) .45 -.04 .12 .06 -.I8 .16 -.07 .05 .10 .09
12
= 100, data = Z,, Wt = (1 - B)Z,,
k
;z(k)
= 1500
(e) n = 100, data = Z,, W,= (1 - B)Z,, = 35,
5
6
7
8 9 1 0
1
2
3
4
k
k(k) -94 .93 .90 -89 -87 $6 .84 .81 .80 .80
bw(k) .69 ,50 .33 .19 .10 .08 .03 .O1 -01 .00
6.2 Identify proper ARlMA models for the following data sets (read across):
(a) -2.401
-.574
,382 -.535 -1.639
-.960 -1.118
.645
.I17 -.493 -2.282 -1.823
-.719 -1.236
-,531
-1.891
,370
.082
.589
1.413
-.I79
-.790 -1.476 -2.491 -4.479 -2.809
-,961
-.865
,740
.907
-2.154 -1.532 -2.119 -3.349 -1.588
2.463 2.714
2.622
.701
.557 2.259
1.540
3.186
3.192 2,939 .3.263
3.750 4.322
2,089
1.060
.370
1.912
.295
,227 1.356
3.279
3.960
.610
2.126
-.I95
.340
1,084 1.237
-,769
.595
.I40 -.082
1,292
3.317 2.167
-A55
2.193
1.353 -.600
1,551 2.610
370
1.604 2.033
1.875 2.082
1.367
,203 1.472
,343 -.334
1.153
1.368
.676
3.746 2.954
2.896
1.970 2.296
.968
1.328 1.325
1.041
1.918
1.569
(b) -1,453
-.048
-.566
.I74
.469
.867
-.SO5
.727 -.765
,858 - 3 3
.697
1.060 -.478
-.270 -1.002
-.289
-.239 -1.200 -2.077
.024 -.542
-1.317
-.454
1.738
-1.986
-.I40
-,581 -1.572
-.I22
-1.605 -2.984
1.693
.463
.421
Exercises
(c)
(d)
1 35
3.485
4.598
3.778
4.613
3.452
4.404
4.311
6.059
4.777
6.358
7.204
6.637
9.012
8.219
7.501
.315
- 1.054
- .999
- 1.749
-2.822
.718
.769
.677
.653
-2.047
-.935
4.419
1.424
- 1.014
- 1.561
6.3 Identify models for Series W1 through W7 using ESACF and compare them with
the models suggested in the book using ACF and PACE
Parameter Estimation,
Diagnostic Checking,
and Model Selection
After identifying a tentative model, the next step is to estimate the parameters in the model.
With full generality, we consider the general ARMA(p, q) model. That is, we discuss the
estimation of parameters 4 =
&, . . . , & ) I , p = E(Z,), 8 = (dl, 02, . . . , dq), and
a: = ~ ( a : in
) the model
where Z, = Z, - p, Z, (t = 1, 2, . . . , n) are 11 observed stationary or properly transformed

stationary time series and { a t ) are i.i.d. N(0, r+;) white noise. Several widely used estimation
procedures are discussed.
Once parameters have been estimated, we check on the adequacy of the model for the
series. Very often several models can adequately represent a given series. Thus, after introducing diagnostic checking, we also present some criteria that are commonly used for model
selection in time series model building.
7.1 The Method of Moments

The method of moments consists of substituting sample moments such as the sample mean, Z,
sample variance To, and sample ACF j;ifor their theoretical counterparts and solving the resultant equations to obtain estimates of unknown parameters. For example, in the AR(p) process
the mean p = E(z,) is estimated by 2. TO estimate +, we h t use that pk = 41~k-l

+ & P ~ - +~
. . + $pk-p for k 2 1to obtain the following system of YuleWalker equations:
7.1
The Method of Moments
ck,
137
Then, replacing pk by
we obtain the moment estimators 41,#*, . . . , 4pby solving the
above Iinear system of equations. That is,
These estimators are u_suailycallec Yule-Walker estimators.

Having obtained 41,49,. .. , #p, we use the result
and obtain the moment estimator for a

: as
EXAMPLE 7.1 For the AR(1) model
the Yule-Walker estimator for
from (7,1,3), is
The moment estimators for p and o-2 are given by
$=Z
and
respectively, where To is the sample variance of the Z, series.

Next, Iet us consider a simple MA(1) model
138
CHAPTER 7 Parameter Estimation, Diagnostic Checking, and Model Selection
Again, p is estimated by 2.For GI, we use that
and solve the above quadratic equation for BI after replacing pl by
The result leads to
= i . 5 , then we have a unique solution

= kl,which gives a noninvertible estimated
model. If Ifill > .5, then the real-valued moment estimator does not exist. This result is ex< .5, as discussed in Section 3.2,l.
pected, because a real-valued MA(1) model always has
For
< .5, there exist two distinct real-valued solutions and we always choose the one that
satisfies the invedbility condition, After having obtained
we calculate the moment estimator for w: as
Ipl l
sl,
This example of the MA(1) model shows that the moment estimators for MA and
mixed ARMA models are complicated. More generally, regardless of AR, MA, or ARMA
models, the moment estimates are very sensitive to rounding errors. They are usually used
to provide initial estimates needed for a more efficient nonlinear estimation discussed later
in this chapter, particularly for the MA and mixed ARMA models. The moment estimators
are not recommended for final estimation results and should not be used if the process is
close to being nonstationary or noninvertible.
7.2 Maximum Likelihood Method

Because of its many nice properties, the maximum likelihood method has been widely used
in estimation. In this section, we discuss several alternative maximum likelihood estimates
that are used in time series analysis.
7.2.1 Conditional Maximum Likelihood Estimation

For the general stationary ARMA(p, q) model
139
where 2, = Z, - p and {ci,} are i,i,d. N(0, G:) white noise, the joint probability density of
a = (al, az, , ,a,)' is given by
..
Rewriting (7.2.1) as
we can write down the likelihood function of the parameters ( 4 , p , 8, m i ) .

Let Z = (Zl, &, . , . ,Zn)' and assume the initial conditions ZI = (ZI-pJ . . . , L1,
and a* = (al-,, . , ,adl, Q)' are known. The conditionaI log-likelihood function is
a'
where
is the conditional sum of squares function. The quantities of 4, b, and 0, which maximize Function (7.2.4), are called the conditional maximum likelihood estimators. Because
lnL* (4,p , 0, g;) involves the data only through S*(+, p, 0), these estimators are the same
as the conditional least squares estimators obtained from minimizing the conditional sum of
squares function S+(+, p , O), which, we note, does not contain the parameter cr:.
There are a few alternatives for specifying the initial conditions Z* and at. Based on the
assumptions that {Z,} is stationslry and (a,) is a series of i.i.d. N(0, uz),random variables, we
can replace the unknoxm 2, by the sample mean and the unknown a, by its expected value of 0.
For the model in (7.2.1), we may also assume ap = ap-1 =
= ~ l p + l -=
~ 0 and calculate a,
fort 2 @ 1)using (7.2.3). The conditional sum of squares function In (7.2.5) thus becomes
which is aIso the form used by most computer pzograms.

After obtaining the parameter estimates #, fi, and
lated from
---
a, the estimate 3; of a: is calcu-
whe_retheAnumberof degrees of freedom d.f, equals the number of terms used in the sum of
S*(& b, 8 ) minus the number of parameters estimated. Lf (7.2.6) is used to calcufate the
140
sum of squares, d.f. = (n - p) - ( p

d.f. should be adjusted accordingly.
+ q + 1) = n - (2p + q + 1). For other models, the
7.2.2 Unconditional Maximum Likelihood Estimation

and Backcasting Method
As seen from Chapter 5, one of the most important functions of a time series model is to
forecast the unknown future values. Naturally, one asks whether we can back-forecast or
backcast the unknown values Zr = (Zllp, . . . , Z - l , &)' and a* = (al-,, + . , a-1, q)'
needed in the computation of the sum of squares and likelihood functions, Indeed, that is
possible because aay ARMA model can be written in either the forward form
or the backward form
where FjZ, = 2, Because of the stationarity, (7.2.8) and (7.2.9) should have exactly the same
autocovariance structure, which implies that (e,} is also a white noise series with mean zero
and variance cr;. Thus, in the same way as we use the forward form (7.2.8) to forecast the
unknown future values
for j > 0 based on the data (Zl, &, . . . , Z,,), we can also use the
backward form (7.2.9) to backcast the unknown past values 3 and hence compute aj for j 5 0
based on the data {Z,, Z,,-l, , ,Z1). Therefore, for a further improvement in estimation, Box,
Jenkins, and Reinsel (1994) suggest the follo~vingunconditional log-likelihood function:
..
where S(4, p , 0) is the unconditional sum of squares function given by
and E{a, 4, p , 8, Z) is the conditional expectation of a, given 4, p, 8, and Z. Some of these

terms have to be calcylated using backcasts illustrated in Example 7.2.
The quantities 4, $, and 8 that maximize Function (7.2.10) are called unconditional
maximum likelihood estimators. Again because lnL(#, p, 6,uz) involves the data only
through S(4, p , %),these unconditional maximum likelihood estimators are equivalent to the
unconditiond least squares estimators obtained by minimizing S{+, p, 8). In practice, the
summation in (7.2.1 1) is approximated by a finite fonn
where M is a sufficiently large integer such that the backcast increment JE(z, 4 , p, 6,Z) E(Z,-I 4, p, 8, z)]is less than any arbitrary predetermined small E value for t 5 - (M + 1).
141
This expression implies that E[Zt 1 4 , p, 0, 2)C= p; hence, E(at 4 , p, 8 , Z ) is negligible for
t 5 - (M -t- 1).
After obtaining the parameter estimates 4,b, and 0, the estimate 6; of uz can then be
calculated as
A
For efficiency, the use of backcasts for parameter estimation is important for seasonal
models (discussed in Chapter 81, for models that are close to being nonstationary, and
especiaIly for series that are relatively short. Most computer programs have implemented
this option.
.
EXAMPLE 7.2
To illustrate the backcasting method, consider the AR(1) model that can
be written in the forward form
or, equivalently, in the backward form
where, without loss of generality, we assume that E(Z,) = 0. Consider a very simple example
with ten observations, Z = (ZI, &, . . ,ZIO)from the process, which are listed in Table 7.1
TABLE 7.1 CaIcufation of St4 = .3) for (1

method.
- +BIZ, = a, using the backcasting
142
under the column E(Z, 2)for f = 1,2, , . . , 10. Suppose that 4 = .3 and we want to calculate the unconditional sum of squares
IE
where M is chosen so that (2, 4 = .3, Z) - E(ZIdl 4 = .3, z)[< .005 for t 5 -(M
1).
To simplify the notations for this example we write E (a, 4 .= .3, Z) as E(ar Z) and
E(ZI f q5 = .3, Z) as E(Zt Z).
To obtain E(aI Z) we use (7.2.14) and compute
This computation of E(a, Z) for f 5 1, however, involves the unknown Z, values for t
which need to be backcasted. To do so, we use the backward form in (7.2.IS), i.e.,
0,
First, we note that in terns of the backward form, e, for t 5 O are unknown future random
,&, and Zl. Hence,
shocks with respect to the observationsZ,, Z,,_l,
. ..
E(et 1 2) = 0,
fort
0.
(7.2.19)
Therefore, for cfJ = .3, we have from (7.2.18)
Because ] E ( z - ~ Z) - E(Z-:! z)]= .00378 < ,005, the predetermined e value, we choose
M = 2.
Now, with these backcasted values 2, for t 5 0, we can return to the forward form in
(7.2.17) to compute E(at Z) for 4 = .3 ftom t = -2 to t = 10 as follows:
143
All the above computations can be carried out systematically as shown in Table 7.1 and
we obtain
Similarly, we can obtain S(4)for other values of 4 and hence find its minimum.
For the AR(1) model we do not need the value E(e, Z) for t 2 1. For other models, they
may be required, but the procedure is the same, For more detailed examples, see Box, Jenkins,
and Reinsel (1994, p. 233).
7.2.3 Exact Likelihood Functions

Both the conditional and unconditional likelihood functions (7.2.4) and (7.2.10) are approximations. To illustrate the derivation of the exact likelihood function for a time series model,
consider the AR(1) process
z, = 4zt-1 + a,
where 2, = (2, - p), ]I$] < 1 and the a, are i.i.d. N(0,u:). Rewriting the process in the
moving average representation, we have
144
Clearly, the 2, will be distributed as N(O, rr:/(1 - $2)). The z,, however, are highly conelated. To derive the joint probability density function P ( z ~ i2,.
, . ,~ ~ 5 of
, ) (&, &, ,in)
and hence the likelihood function for the parameters, we consider
...
Note that el follows the normal dis&ibution~ ( 0~,: / ( 1- $2)), at, for 2 5 t 5 n, follows
the normal distribution N ( 0 , w;), and they are all independent of one another, Hence, the
joint probability density of (el, a*, . , a,) is
..
Now consider the following transformation:
z,= #Zn-, -t-
a,.
The Jacobian for the transformation, from (7.2.221, is
7.3
Nonlinear Estimation
145
It follows that
P(z,,
i,,. . . ,
. . . , a,)
= P(el, a*,
Hence, for a given series (zl, &,
. . . ,z,)
we have the following exact log-likelihood function:
where
is the sum of squares term that is a function of only 4 and p.

The exact closed form of the likelihood function of a general ARMA model is complicated.
llao and Ali (1971) derived it for an W ( 1 , 1) model. Newbold (1974) derived it for a general W ( p , q) model. Interested readers are advised also to see Ali (1977), Ansley (1979),
Hillmer and T5ao (1979), Ljung and Box (1979), and Nicholls and Hall (1979), among others
for additional references.

It is clear that the maximum likelihood estimation and the least squares estimation
involve minimizing either the conditional sum of squares St(+, p, 9 ) or the unconditional
sum of squares S(#, p , e), which are the sums of squares of the error terms at's. For an
AR(p) process,
and at is clearly linear in parametefs. For a model containing an MA factor, however, the a,
is nonlinear in parameters. To see that, consider a simple ARMA(1, 1)model
146

To calculate a , we note that
This equation is clearly nonlinear in the parameters. Hence, for a general ARMA model, a
nonlinear least squares estimation procedure must be used to obtain estimates.
The nonlinear least squares procedure involves an iterative search technique. Because a
linear model is a special case of the nonlinear model, we can illustrate the main ideas of the
nonlinear least squares using the following linear regression model:
. . , 11, where the et's are i.i.d. N ( 0 , u:) independent of all the Xti. Let Y =
(Yl, Y2, . . . , Yn)',a = (a1,
a2, . . . , ap)'and be the corresponding matrix for the independent variables Xti's. From results in linear regression analysis, we know that the Ieast
squares estimators are given by
for t = 1, 2,
which follows a multivariate normal distribution MN(a, V(&)) with
The minimum residual (error) sum of squares is
The least squares estimates in (7.3.5) can also be obtained by the following two-step procedure discussed in Miller and Wichern (1977).
Let Z = (G,E2,.. , ap)' be an initial guess value of a = (al, a2, . . . , (~p)'. We can
rewrite the minimum residual sum of squares in (7.3.7) as
.-
147
where the Z,'s are estimated residuals based on the initial given values G and S = (6 - Z).
Now, S(S) in Equation (7.3.9) and S ( & ) in (7.3.7) are in the same form. Hence, the least
squares value of 8 is
(zl,.
.., e,)'. Once the values S = (81, S2,

where =
squares estimates are given by
Note that the residual Zt is calculated as I;
. . . , 8,)
are calculated, the least
- 4, where
represents a guess of the regression equation obtained by using the original model and the given
initial values Ziys.Moreover, it is clear from Equation (7,3.4) that
for i = 1,2, . . ,p and t = 1, 2, . . . ,n. Hence, the matrix used in the equation of the
least squares estimates in (7.3.5) and (7.3.10) is actually the matrix of the partial derivatives
of the regression function with respect to each of the parameters.
Now, consider the following model (linear or nonlinear):
where X, =. (X,g,Xt2, . . . , Xtp) is a set of independent variables corresponding to the observations, a = (a1,
a2,, , . ,+)' is a vector of parameters, and e, is a white noise series having
zero mean and constant variance (T: independent of X,. Let Y = (Yl, Yz, . . . , Y,)' and
f(a)= XI, a),
f(X2, a),. . . ,f(X,, a)]'.From the previous discussion, the least squares
estimators (linear or nonlinear) can always be calculated iteratively as follows,
Step 1. Given any vector of initial guess values Z, compute the residual Z = (Y - Y) and
the residual sum of squares
where ? = f(Z} is a vector of

values obtained by replacing the unknown parameters
by the initial guess values. Approximate the model f(Xt, a)with the frrst-order Taylor series
expansion about the initial value i'. That is,
148
z)
where S = (a and gz = [Xlj] is the n X p matrix of the partial derivatives at ii;: i n

the above linear approximation. That is,
Then we calculate
zzis fixed and equals g;for a nonlinear model, this x8.
Note that for a linear model the

changes from iteration to iteration,
Step 2. Obtain the updated least squares estimates
and the corresponding residual sum of squares S(&). We note that Si in S represents the difference or change in the parameter values. For a Enear model, step 2 gives the find least squares
estimates. For a nonlinear model, step 2 only leads to new initial values for further iterations.
In summary, for a given general ARMA(p, q) model, we can use the nonlinear least squares
procedure to iind the least squares estimates that minimize the error sum of squares S* (4,p, 0)
or S(4,p, 0). The nonlinear least squares routine starts with initial guess values of the parameters. It monitors these values in the direction of the smaller sum of squares and updates the initid
guess values. The iterations continue until some specified convergence criteria are reached.
Some convergence criteria that have been used are the relative reduction in the sum of squares,
the maximum change in the parameter values less than a specified level, or the number of iterations greater than a certain number.To 'achieve a proper and faster convergence, many search
algorithms are developed. One of the algorithms commonly used is due to Marquardt (1963). It
is a compromise between the Gauss-Newton method and the method of steepest descent. For
more discussions on nonlinear estimation, see Draper and Smith (1981), among othefs.
Properties of the Parameter Estimates Let a = (4, p, O), ii be the estimate of a and
X g be the matrix of the partial derivatives in the h a l iteration of the nonlinear least squares
-
procedure. We know that asymptotically & is distributed 2 a multivariate normal distribution

MN(a,V ( & ) ) .The estimated variance-covariance matrix V ( 6 ) of ii is
where $2 is estimated as in (7.2.7) or (7.2,13) and

is the sample covariance between $
and Gj. We can test the hypothesis Ho: ai= rrio using the following t statistic:
149
+ +
with the degrees of freedom equaling n - ( p q

1) for the general ARMA model in
(7.2.1). (More generally, the degrees of freedom equals the sample size used in estimation
minus the number of parameters estimated in the modei.) The estimated correlation matrix
of these estimates is
where
A high correlation among estimates indicates overpararneterization, which should be avoided

as it often causes daculties in the convergence of the nonlinear least squares.
The results presented in this section are perfectly general. They hold for both linear and
nonlinear models. We illustrate their uses in the following simple example.
EXAMPLE 7.3 Consider the AR(1) model

Z , = #Z,-l
To put the estimation of
where Y, = Z,, f(Z,-l,
+ a,.
(7.3.22)
4 in the context of this section, we use (7.3.13) to rewrite (7.3.22) as
4) =
and e, = a,, for t = 2, . . . ,n. Thus,
Applying &%nonlinearestimation to (7.3.231, we choose, with no loss of generality, the initial

guess value $ of 4 to be 0.It follows from steps 1 and 2 of the procedure that
where
x&is the
(la
- 1) X
1 matrix of partial derivatives, That is,
150
and
Thus,
Properties of
t$
The
timitor
~ ( $ 1= c ~ ( ~ ~ X $=) -U' ~
) is dishibuted agymptofically as N(+,v(+)), where
Z ~ = ~ Z ~is-the
I )variance-covariance
- ~
matrix
of
4. The es-
timated variance-covariance matrix is
x:=2(~I
+
where 6; =
-4~~-~)~ /1 () .1Conditional
1
on (Z,, r a ~ a ) , we can test the
hypothesis Ho: = c $ ~using the standard normal statistic if u: is known,
or the t statistic if a: is unknown and estimated,
which follows a t-distribution with (la - 1 ) degrees of freedom.

For the asymptotical distribution of
we note that
G($- 4) -% N ( 0 , n~($)). Now, if 101< I, then we have
4,
Thus, asymptotically, when 141 < 1, we have

alently,
N ( 4 , (1 - # 2 ) / n ) .
G ( b - 6) D
$5
N(+,v($)) or
N(O, (1
-+
- @)) Or, equiv-
7.4 Ordinary Least Squares (OLS) Estimation in Time Series Analysis
151
7.4 Ordinary Least Squares (OLS) Estimation

in Time Series Analysis
Regression analysis is possibly the most commonly used statistical method in data analysis.
As a result, the ordinary least squares (OLS) estimation developed for linear regression models is perhaps also the most frequently used estimation procedure in statistics. In this section,
we discuss some problems of OLS estimation in time series analysis.
Consider the simple linear regression model
Under the following basic assumptions on the error term e,:
1,
2.
3.
4.
Zero mean: E(eJ = 0

Constant variance: E(e:) = ~2
Nonautocorrelation: E(et ek) = 0 for t f k
Uncorrelated with explanatory variable X,:E(X,eJ = 0
it is well known that the OLS estimator
is a consistent and the best linear unbiased estimator of 4. Note however, that assumption 4 is
crucial for this result to hold. Assumption 4 automatically follows if the explanatory variables
are nonstochastic. In a noncontro1lable study, however, particularly when time series data are
involved, the explanatory variables are usually also random variables.
Now, consider the foliowing time series model:
The OLS estimator of 4, based on available data, is
We would like to ask whether

is still unbiased and consistent in this case when the
explanatory variable is a lagged dependent variable. The answer depends on the stochastic
nature of the error term e,. To see that, we rewrite 4 as
and consider the following two cases.
152
CHAPTER 7 Parameter Estimation, Diagnostic Checlung, and Mode1 Seiection

Case I :
e, = a,. That is, the e, is a zero mean white noise series of constant variance a:. In this
case, it is easy to see that in (7.4.4) is equivalent to the first lag sample autocorrelation jl for
the series 4.If 141 < 1 and hence 2, becomes an AR(1) process with an absolutely summable
autocone~ationfunction, then by Section 2.5 is a consistent estimator of p l , which is equal to
$. Thus, $ in (7.4.4) is an asymptotically unbiased and consistent estimator of 4.
e, = (1 - BB)a,, where the a, is a zero mean white noise series of constant

variance t ~ z hence
;
e, is an MA(1) process. Under this condition, the series 2, becomes an
ARMA(1, 1) process
Cme 2:
and
This result shows that autocorrelation in the error term not only violates assumption 3 but also
causes a violation of ?sumption 4 when the explanatory variables contai? a lagged dependent
variabfe. In this case, 4 is no longer a consistent estimator of 4 because 4
is a consistent
estimator for p l , and for an ARMA(1, 1)process, by (3.4.14),
In summary, the OLS estimator for the parameter of an explanatory variable in a regression model will be inconsistent unless the error term is unconelated with the explanatory
variable. For ARMA(p, q) models, this condition usually does not hold except when q = 0.
Estimation methods discussed in Sections 7.2 and 7.3 are more efficient and commonly used
in time series analysis.
7.5 Diagnostic Checking

Time series model building is an iterative procedure. It starts with model identification and
parameter estimation. After parameter estimation, we have to assess model adequacy by
checking whether the model assumptions are satisfied. The basic assumption is that the {a,]
are white noise. That is, the a,'s are uncorrelated random shocks with zero mean and constant variance. For any estimated model, the residuals 2,'s are estimates of these unobserved
white noise at's. Hence, model diagnostic checking is accompLished through a careful
analysis of the residual series (6,). Because this residual series is the product of parameter
estimation, the model diagnostic checking is usually contained in the estimation phase of a
time series package.
To check whether the errors are normally distributed, one can construct a histogram of
the standardized residuals ;,/Go and compare it with the standard nonnal distribution using
7.6 Empirical Examples for Series WI-W7
153
the chi-square goodness of fit test or even Tukey's simple five-number summary. To check
whether the variance is constant, we can examine the plot of residuals. To check whether the
residuals are approximately white noise, we compute the sample ACF and sample PACF (or
IACF) of the residuals to see whether they do not form any pattern and are all statistically
insignificant, i.e., within two standard deviations if a = -05.
Another useful test is the portmanteau Iack of fit test. This test uses all the residual sample
ACFs as a unit to check the joint null hypothesis
with the test statistic
This test statistic is the modified Q statistic originally proposed by Box and Pierce
(1970). Under the null hypothesis of model adequacy, Ljung and Box (1978) and Ansley
and Newbold (1979b) show that the Q statistic approximately follows the X ~ ( K- rn)
distribution, where m = p q.
Based on the results of these residual analyses, if the entertained model is inadequate, a
new model can be easily derived. For example, assume that the entertained AR(1) model
produces an MA(1) residual series instead of a white noise series, i.e.,
Then we should re-identify an ARMA(1, 1) model
and go through the iterative stages of the model building until a satisfactory model is
obtained. As mentioned earlier, if OLS estimation is used and the model should be indeed a
mixed model, then the OM estimates of the AR parameters based on a misidentified model
are inconsistent, Although that may sometimes cause problems, the above procedure of
using the residuals to modify models usuaIIy works h e .
7.6 Empirical Examples for Series W1-W7

For an illustration, we estimated the AR(3) model identified in Example 6.7 for Series W7-the
yexly numbers of lynx pelt sales-and obtained the following result:
154
and &: = .124, where the values in the parentheses under the estimates refer to the standud
errors of those estimates. They are all significant except for +z, and the model can be refitted
with 42removed if necessary.
To check model adequacy Table 7.2 gives the residual ACF and PACF and the Q statistics. The residual ACF and PACF are all small and exhibit no patterns, For K
24, the Q
statistic is Q = 26.7, which is not significant as x i 5 ( 2 I ) = 32.7, the chi-square value at the
significance level a = .05 for the degrees of freedom = K - rn = 24 - 3 = 21. Thus, we
conclude that the AR(3) model is adequate for the data.
Similarly, we use the nonlinear estimation procedure discussed in Section 7.3 to fit
the models identified in Section 6.2 for Series W1 to W6. The results are summarized in
Table 7.3.
Diagnostic checking similar to the one for Series W7 was performed for each model
fitted in Table 7.3. All models except the AR(2) for Series W2 are adequate. Related tables
are not shown here. Instead, we recalt that in Example 6.9, Section 6.4.1, Series W7 was
alternatively identified as an ARMA(2, f ) model using the ESACF. The estimation of this
model gives
with all parameters being significant and 6; = .116. The result was also presented in
Table 7.3. The residual autoconelations from this ARMA(2, 1) model shown in Table 7.4
also indicate the adequacy of the model, In fact, both AR(3) and ARMA(2, 1) models fit
the data almost equally well, raising the question of model selection to be discussed next.
TABLE 7.2 Residual ACF and PACF for the AR(33 model for Series W7.
155
7.6 Empirical Examples for Series Wl-W7
TABLE 7.3 Summaryof models fitted to Series W1-W7 (values in the parentheses
under estimates refer to the standard errors of those estimates).
Series
No. of
observations
Fitted models
6:
TABLE 7.4 Residual autocarrelations, Fk, from the ARMA(2, 1) model for Series W7.
156

In time series analysis or more generally in any data analysis, several models may adequately
represent a given data set. Sometimes, the best choice is easy; other times the choice can be
very difficult. Thus, numerous criteria for model comparison have been introduced in the literature for model selection. They are different from the modd identification methods discussed
in Chapter 6. Model identification tools such as ACF, PACF, IACF, and ESACF are used only
for identifying likely adequate models. Residuals from all adequate models are approximately
white noise and are, in general, indistinguishable in terms of these functions. For a given data
set, when there are multiple adequate models, the selection criterion is normally based on summary statistics from residuals computed fYom a fitted model or on forecast errors calculated
&om the out-sample forecasts. The latter is often accomplished by using the first portion of the
series for model construction and the remaining portion as a holdout period for forecast evduation. In this section, we introduce some model selection criteria based on residuals. Criteria
based on out-sample forecast errors are discussed in Chapter 8.
I. Akaike's M C and BXC Assume that a statistical model of M parameters is fitted to

data. To assess the quality of the model fitting, Akaike (1973, 1974b) introduced an information criterion. The criterion has been called AIC (Akaike's information criterion) in the literature and is defined as
AIC(M) = -2 ln[maximum likelihood]
+ 2M,
(7.7.1)
where M is the number of parameters in the model. For the ARMA model and n effective
number of observations, recall from (7.2.10) that the log-likelihood function is
Maximizing (7.7.2) wih respect to 4, p., 0, and ui, we have, from (7,2.13),
Because the second term in (7.7.3) is a constant, the AIC criterion reduces to
The optimal order of the model is chosen by the value of M, which is a function of p and q,
so that AIC(M) is minimum.
Shibata (1976) has shown that the AIC tends to overestimate the order of the autoregression, Akaike (1978, 1979) has developed a Bayesian extension of the minimum AIC
procedure, called the Bayesian information criterion (BIC), which takes the fom
BIC(M) = ~ l l n & z -(n - M)ln
+Mlnn
+ Mln
>I
1 M
(7.7.5)
157
where Gi is the maximum likelihood estimate of a i , A4 is the number of parameters, and

r?$ is the sample variance of the series. Through a simulation study Akaike (1978) has
claimed that the BIC is less likely to overestimate the order of the autoregression. For
further discussion on the
of ATC, see FindIey (1985).
2. Schwartz's SBC Similar to Akaike's BIC, Schwartz (1978) suggested the following
Bayesian criterion of model selection, which has been called SBC (Schwartz's Bayesian
criterion):
Again in (7.7,6), &: is the maximum Ekelihood estimate of r+z, M is the number of parameters in the model, and n is the effective number of observations that is equivalent to the number of residuals that can be calculated from the series.
3. Parzen's CAT Parzen (1977) has suggested the following model selection criterion,
which he called CAT (criterion for autoregressive transfer functions):
where 6; is the unbiased estimate of uz when an AR(j3 model is fitted to the series and n is
the number of observations. The optimal order of p is chosen so that CAT(p) is minimum.
We have introduced only some commonly used model selection criteria. There are many
other criteria introduced in the literature. Interested readers are referred to Hannan and
Quinn (1979), Stone (19791, Hannan (1980), and others.
EXAMPLE 7.4 The AIC has become a standard tool in time series model fitting, and its
computation is available in many time series programs. In Table 7.5, we use the SAS/ETS
(1999) software to compute AIC for Series W7, Canadian lynx pelt sales. From the table, it
is clear that the minimum AIC occurs for p = 2 and q = 1. Hence, based on the AIC, an
ARMA(2, I) model should be selected for the data. Note that a competitive AR(3) model
that we fitted earlier to the same data set gives the second smallest value of AIC.
TDLE 7.5 AIC values for Series W7.
158
7.1 Assume that 100 obsirvations from an ARMA(1, 1) model
gave the following estimates: G: = 10,

01,and IT:.
moments estimates for
= ,523, and
i2= .418. Find method of
7.2 Assume that 100 observations from an AR(2) model
gave the following sample ACF:
= .8, F2 = ,5, and j3= .4. Estimate
and &.
7,3 Given the set of observations 2.2,4.5,2.5,2.3, 1.1, 3.0, 2.1, and f .O,calculate the conditional sum of squares S(OI,&) for the MA(2) process with
- .5 and 62 = -2.
7.4
Given the set of observations 6 , 2 , 4 , 5 , 3 , 4 , 2 , and 1, illustrate how to calculate the

conditional sum of squares function S(#Q, B1) for theARMA(1, I) model.
7.5
Consider the following observations from an ARIMACO, 1, 1) model with 8 = .4:
(a) Calculate the conditional sum of squares (with a0 = 0).

(b) Calculate the unconditionaj sum of squares using the backcasting method as
shown in Table 7.1.
,
7.6 Simulate 100 observations from an ARMA(1, 1) model.

(a) Fit the simulated series with an AR(1) or an MA(1) model. Carry out diagnostic
checking, and modify your fitted model from the result of residual analysis.
(b) Estimate the parameters of your modified model, and compare with the m e
parameter values of the model.
7.7 A summary of models fitted for the series W1 to W7 is given in Table 7.3. Perform
residual analysis and model checking for each of the fitted models.
7.8
Use ~kto find a model for each of the series W l to W7, and compare it with the
fitted model given in Table 7.3.
Exercises
7.9 Suppose that (1 --@)Z, = (1
159
BB)a, is a tentatively entertained model for a
process. Given
calculate the unconditional sum of squares for I$ = .4 and 0 = .8.
7.10
Consider the AR(1) model

(1 - W ( Z t -
PI
= at-
(a) For p = 0, find the maximum likelihood estimator for # and its associated variance.
(b) Find the maximum likelihood estimators for # and p when p # 0.
(c) Discuss the relationship between the ordinary least square estimator and the
maximum likelihood estimator for # in the given model.
7.11 Illustrate the nonlinear estimation procedure discussed in Section 7.3 using the following models:
(a) The MA(1) model Zt = (1 - BB)a,.
(b) The AR(2) model Zt = $lZf-~+ $2Zr-2 $. a,.
7.12
Derive the joint asymptotical distribution of the least squares estimators,

for the AR(2) model
[GI,
7.13 Consider the yearly data of lumber production (in billions of board feet) in the
United States given as follows:
Year
Production
(a) Plot the data and perform necessary analysis to construct an appropriate model
for the series.
(b) Find and plot the forecasts for the next four years, and calculate 95% forecast
limits.
(c) Update your forecasts when the 1983 observation became available and
equaled 34.6.
Seasonal Time Series Models

Because of their common occurrence in our daily activities, we devote a separate chapter to
seasonal t h e series. After a brief introduction of some basic concepts and conventional
methods, we extend the autoregressive integrated moving average models to represent seasonal series. Detailed examples are given to illustrate the methods,
8.1 General Conce~ts

Many business and economic time series contain a seasonai phenomenon that repeats
itself after a regular period of time. The smallest time period for this repetitive phenomenon is called the seasonal period. For example, the quarterly series of ice-cream sales is
high each summer, and the series repeats _thisphenomenon each year, giving a seasonal
period of 4, Similarly, monthly auto saIes and earnings tend to decrease during August
and September every year because of the changeover to new models, and the monthly
sales of toys rise every year in the month of December. The seasonal period in these later
cases is 12. Seasonal phenomena may stem from factors such as weather, which affects
many business and economic activities like tourism and home building; cultural events
like Christmas, which is closely related to safes such as jewelry, toys, greeting cards, and
stamps; and graduation ceremonies in the summer months, which are directly related to
the labor force status in these months,
As an illustration, Figure 8.1 shows the U.S. monthly employment figures (in thousands) for young men aged between 16 and 19 years from 1971 to 1981. The seasonal nature
of the series is apparent. The numbers increase dramatically in the summer months, with
peaks occurring in the month of June when schools are not in session, and decrease in the
fall months when schools reopen. The phenomenon repeats itself every 12 months, and thus
the seasond period is 12.
More generally, suppose that the series { Z , ) is seasonal with seasonal period s. To analyze
the data, it is helpful to arrange the series in a two-dimensional table as shown in Table 8.1
according to the period and season and including the totals and averages. Wold (1938) credits
these arrangements of the table to Buys-Ballot (1847); hence, the table has also been called the
Buys-Ballot table in the literature.
8,l General Concepts
I970
1972
1974
1976
Year
1978
1980
1982
16'1
'.
FIGURE 8.1 U.S. monthly employment figures (in thousands) for young
men aged between 16 and 19 years from 1971 to 1981.
TABLE 8.1 Buvs-Ballot table for a seasonal time series.

2
...
Total
Average
z1
z
2
ZJ+2
z
3
zr+3
.. .
,..
zs
Zs+l
2,
E1
T.2
Z.
7.2
z(n-l>+3
* ' '
zm
T.n
Z,
...
...
2,.
Z[n-1>+l
Total
Average
TI.
zl.
z(n-l>+2
T2.
z2
T3.
z3.
Tj. = the total for the jth season,

= the average for the jth season,
T j = the total for the jth'period,
Z.] = the average for the jth period,
T = the grand total of the series,
TW
T/s
Tltl
T/w
162
CHAPTER 8 Seasonal Time Series Models
8.2 Traditional Methods

Conventionally, time series have been thought to consist of a mixture of trend-cycle (P,),
seasonal (S,), and irregular components (eJ. If these components are assumed to be additive,
then one can write the time series Z, as
2, = P,
+ St + e,.
(8.2.1)
To estimate these components, several decomposition methods have been introduced in the
literature.
8.2.1 Regression Method

In the regression method, an additive seasonal time series is written as the following regression model:
where P, = CYO + XyL1cyiq,,

and the Wit are the trend-cycle variables; S, = Zj=lPjl$,, and the
yl are the seasonal variables. For example, a linear trend-cycle component P, can be written as
k
More generally, a trend-cycle component can be written as an mth-order polynomial in time as
S i d a r l y , the seasonal component St can be described as a lineat combination of seasonal

dummy (indicator) variables or as a linear combination of sine-cosine functions of various
frequencies. For example, a seasonal series of period s can be written as
where Dj, = 1 if t corresponds to the seasonal period j and 0 othenvise. Note that when the
seasonal period equals s, we need only (s - I) seasonal dummy variables. In other words, /3,
is set to be 0 so that the coefficient pi, j # a, represents the seasonal effect of thejth period
as compared with the period s. Alternatively, S, can be written as
163
8.2 Traditional Methods
where [sD]is the integer portion of sI2. A model of this type is discussed hrther in Chapter 13.
Thus, the model (8.2.2) bticomw
For a given data set Z, and specified values of~?t

and s, the standard least squares regresand yj of the parameters ai, Pi, and yp
sion method can be used to obtain estimates Gi,
The estimates of P,, S,, and e, for Equation (8.2.7) are then given by
and
2t
= zt - P, - S,.
For Equation (8.2.8)they are given by
and
8.2.2 Moving Average Method

The moving average method is developed based on the assumption that an annual sum of
a seasonal time series possesses little seasonal variation. Thus, by letting N, = P, e, be
164
the nonseasonal component of the series, an estimate of the nonseasonal component can
be obtained by using a symmetric moving average operator. That is,
where nt is a positive integer and the $ are constants such that I+ = h-! and X%-,Ji = 1. An
estimate of the seasonal component is derived by subtracting N, from the original series, i.e.,
A
These estimates may be obtained iteratively by repeating various moving average operators.
A saccessful example of using this moving average procedure is the Census X-12 method,
which has been widely employed in government and industry.
The series with seasonal component removed, i.e., Zt - S,, is referred to as the seasonally adjusted series. Hence, the decomposition procedures are also known as seasonal adjustment. Based on prevailing belief that the seasonal component is a well-behaved phenomenon
that can be predicted with a reasonable accuracy, there is great interest in government and
industry to adjust the series seasonally. The topic is presented only briefly here. Interested
readers are referred to an excellent collection of articles edited by Zellner (1978) and to
some research papers on the topic such as Dagum (1980), Pierce (1980), Hillmer and Tiao
(1982), Bell and Hillmer (1984), and Cupingood and Wei (1986).
A
8.3 Seasonal ARIMA Models

Traditional methods presented in Section 8.2 are based on assumptions that the seasonal
component is deterministic and independent of other nonseasonal components. Many time
series, however, are not as well behaved. More likely, the seasonal component may be stochastic and correlated with nonseasonal components. In this section, we extend the stochastic ARIMA models discussed in previous chapters to model seasonal time series.
For an illustration, let us examine the U.S. monthly employment figures (in thousands) for
young men aged between 16 and 19 years &om 1971 to 1981 using the Buys-Ballot table shown
in Table 8.2. The table indicates that the employment figures are not only related from month to
month but also related from year to year. Thus, to forecast the employment level for the month
of June, one should examine not only the employment levels in the n e i g h b o ~ gmonths such as
May and July, but also the employment levels for the month of June in the previous years.
In general, the Buys-Ballot table implies that (2,)contains within periods and between
periods relationships. Within periods relationships represent the correlation among
Zt-2,
Z,-,, Z,,
Zit*, . . and between periods relationships represent correlation among . . .
~t-25,Zf-s1ZI,zits, &+a, ' ' ' '
Suppose that we do not know that the series {Z,} contains the between period seasonal
variation and fit a nonseasonal ARIRiIA model for the series, i.e.,
...
ys-Ballot table for the U.S. monthly employment figures (in thousands) for young men aged between 16 and 19 years.
Mar.
8,805
Apr. May
June
July
7,986 7,590 12,048
11,052
Aug.
8,770
Sept.
8,288
Oct.
8,181
Nov.
8,971
Dec
8,818
Total
109,094
Avg.
9,091.17
166
Then obviously the series { b , ) will not be white noise as it contains unexplained between
period (seasonal) correlations. Let
be the autocorrelation function for {b,] representing the unexplained between periods relationship. It can be easily seen that this between periods relationship can also be represented as
an AF3MA model
where
and
are polynomials in BS with no common roots. The roots of these polynomials lie outside of
the unit circle, and {a,] is a zero mean white noise process.
For illustration, suppose that in (8.3.3) P = 1, s = 12, D = 0, and Q = 0. Then
( I - @ ~ ' ~ )=b a,.
,
(8.3,4)
Zf @ = .9, then the autocorrelation function of (b,] becomes pxlz) = (.9)j as shown in Figure 8.2.
Similarly,if P = 0, s = 12, D = 0, and Q = 1,we have
E 0 = -8, then the autocorrelation function becomes
which is shown in Figure 8,3.

Combining (8.3.1) and (8.3.3) we get the well-known Box-Jenkins multiplicative seasonal ARZMA inodel,
FIGURE 8.2 ACF for (I
2 67
.9BE)bt = a,.
FIGURE 8.3 ACF for 6, = (1
.8B12)a,.
where
ifd=D=O,
otherwise.
For convenience, we often call &(B) and d4(B) the regular autoregressive and moving
average factors (polynomials) and Dp(BS)and Bp(BS)the seasonal autoregressive and moving average factors (or polynomials), respectively. The model in (8.3.6) is often denoted as
ARIMA(p, d, q ) X (R D, Q),, where the subindex s refers to the seasonal period.
168
C m E R 8 Seasonal Time Series Modeb
EXAMPLE 8.1 Let us consider the AWMA(0, 1, 1) X (0, 1, l)lz model
This model has been found very useful to represent a variety of seasonal time series such as auline data and trade series. The model was first introduced by Box and Jenkins to represent the
international air travel data. Hence, it is also referred to as the airline model in the literature.
The autocovariances of \Vr = (1 - B)(1 - B ' ~ ) z can
, be easily found to be
(1 -t d2)(1
yo
71= -6(1
YII
+- 82)u,2,
+ e2)a;,
= 60~2,
- e(l + e2)c:,
7 1 3 = eed,
ylz =
yj = 0,
otherwise.
Hence, the ACF becomes
pj
= 0,
otherwise.
(8.3.9)
For 0 = .4 and 0 = .6, the plot of pk is shown in Figure 8.4.
The General AFUMA Models More generally, we can write a general ARWA model
as follows:
Thus, the model may contain K differencing factors, M autoregressive factors, and N moving
average factors. This extension is usefuf in describing many nonstandard time series that, for example, may contain a mixture of seasonal phenomena of different periods. Because it is this general form that most time series software use, we now explain this general model in more detail.
The ith difi'erencing factor is
FIGURE 8.4 ACF for W, = (1 - .4B)(1
169
- .6B1*)a,.
with the order si (the power of B) and the degree di.If K = 0, then 2, = 2, - p. Otherwise,
Z, = Z,, as the mean of the series, does not exist. The parameter O0 represents the deterministic
trend and is considered only when K # 0.The jth autoregressive factor
contains one or more autoregressive parameters, cjjm.

The kth moving average factor is
and contains one or more moving average parameters, dh. In most applications, the values of
K, M, and N are usually less than or equal to 2. Parameters in autoregressive and moving
average factors beyond the first factor of each kind are normally considered seasonal
parameters in the model. Clearly, one may use the factors in any desired order-for example,
making the first factors in each case as the "seasonal factors"-if so desired. The same is
true for the differencing factors.
PACE IACF, and ESACF for Seasonal Models The PACF and IACF for seasonal
models are more complicated. In general, the seasonal and nonseasonal autoregressive components have their PACF and IACF cutting off at the seasonal and nonseasonal lags. On the
other hand, the seasonal and nonseasonal moving average components produce PACF and
IACF that show exponential decays.or damped sine waves at the seasonal and nonseasonal
lags. The computation of the ESACF for seasonal models is time consuming, and the patterns
are in general very complicated. Besides, because the ESACF provides only the information
about the maximum order of p and q, its use in modeling seasonal time series is very limited.
We have found that in identifying seasonal time series, the standard ACF analysis is still the
most useful method.
170
Model Building and Forecasting for Seasonal Models Because seasonal models
are special forms of the ARMA models introduced in Chapters 3 and 4, the model identification, parameter estimation, diagnostic checking, and forecasting for these models follow
the same general methods introduced in Chapters 5, 6, and 7. The description of these
methods is not repeated here. Instead, we illustrate the application of these methods to several seasonal time series in the next section.

EXAMPLE 8 2 This example shows a simulation of 150 values from an W ( O , I, 1) X
(0, 1, model
with 8 = .8, 0 = .6, and a, being Gaussian N(0,1) white noise. The series is listed as Series
W8 in the appendix. As shown in Figure 8.5, it is clearly seasonal with an upward trend,
Table 8.3 and Figure 8.6 show the sample ACF and PACF for the series. The ACF is large
and decays very slowly, whereas the PACF has a single large spike at lag 1.These values indicate that the series is nonstationary and that differencing is called for.
30
60
90
120
150
Time
FIGURE 8.5 A simulated ARIMA(0, 1, 1) x (0, 1,

(1 - B4fZt = ( 1 - .8B) ( 1 - .6B4]a,.
series (1 - B)
171
(1 - B)(1 - B4)Z, (I - .8B)(1 - .6B4)a,.
FIGURE 8.6 Sample ACF and sample PACF for a simulated

series from (I. - B)(1 - B4fZ, = (I - .8B)(1 - .6lI4)a,.
To remove nonstationarity, the series is differenced and the sample ACF and PACF of
the differenced series (1 - B)Z, are computed as shown in Table 8.4(a) and Figure 8.7(a).
That the ACF decays very slowly at multipIes of seasonal period 4 implies that a seasonal
differencing ( I - B ~ is) also needed so as to achieve stationarity. Thus, we examine the
sample ACF and sample PACF for (1 - B)(1 - B ~ ) z as
, given in Table 8.4(b) and Figure
8,7(b). It is known that the ACF of an ARIMA(0, 0, 1) X (0, 0, 1j4model is zero except
at lags 1, 3, 4, and 5. Hence, the sample ACF of (1 - B)(1 - B ~ Zin, Table 8.4(b) and
Figure 8.7(b) implies an ARIMA(0, 1, 1) X (0, 1,
model for the original series {Z,}.
That is,
Parameter estimation procedures can now be used to estimate the values of 8 and 8.
172
CHAPTER 8 Seasonal 'Erne Series Models
TABLE 8.4 Sample ACF and sample PACF for the differenced series simulated from
( 1 - EI4)(1 - BIZ, = (1 - .8B)fl - .6B4fat.
(a) For Wt = (1 - B I Z ,
3
4
5
(m= 1.10, ST, = 3.87, n = 149)

6
10
11
12
EXAMPLE 8.3 We now examine the U.S. monthly employment figures for young men
between 16 and 19 years of age f ~ o m1971 to 1981, which is listed as Series W9 in the
appendix. The series was shown earlier in Figure 8.1.
Model Identificatiotz The columns and thcassociated marginal column totals of the BuysBallot table in Table 8.2 indicate a strong seasonal variation with seasonal period 12. The row
totals over the years, on the other hand, imply that there is a stochastic trend in the series. This
trend has to be removed by differencing before a model can be identified. The sample ACF for
the original and differenced series are computed and given in Table 8.5. The need for both
is clear &om Table 8.5(b) and
regular differencing (1 - B ) and seasonal differencing (1 (c). Now, the ACF for (JK = (1 - B)(1 - 312)Z,} as shown in Table. 8.5(d) has only one
= .66, S, = 71.85, n = 119, the t-value of
significant spike at lag 12. Because
W = .66/(7 1.85/*)
= .1, which is not significant, and thus the deterministic &end term
Bo is not needed. Hence, we identify the following ARIMA(0, 1, 0) X (0, 1,
process as a
tentative model for the series
Parameter Estimation and Diagnostic Checking Using the standard nonlinear estimation procedure, we obtain the following results:
8.4
Empirical Examples
173
FIGURE 8.7 Sample ACF and sample PACF of the differenced series,
(a) For W, = (1 - B)Z,. fb) For Wr = (1 - B)(l - B4]Zt.
with &: = 3327.3. The residua1 ACF for the fitted model as shown in Table 8.6, however,
has a significant spike at lag 1. Thus, we modify the model to an ARlMA (0,1, 1) X
(0,1, 1112, j.e.,
174
TABLE 8.5 Sample ACF for Series W9.

la] ;L for
{Z1)(Z= 826.47, Sz = 161.81, n = 132)
(b) jkfor {Wt

= (1
- B ) z , ) ( ~= 22.6, Sly = 148.89, n = 131)

Ti
(c) ik for {Wt = ( 1
(d) ikfor {TV, = (1
- B ' ~ ) Z ,(w
) = 26.98, Sw = 114.22, n = 120)
- B)(1 - B ~ ~ ) Z , ) [=W.66, Sw = 71.85, n = 119)
Parameter estimation gives
with Gz = 3068. Values of the residual ACF of this modified model as shown in Table 8.7 are
all small and exhibit no patterns. For K = 24, the value 20.7 of the Q statistic is not significant
175
TABLE 8.6 Residual ACF, &, from the fitted model

(1 - Bf(1 - BI2)Z, = (1 - ;79B12)a,.
because X$5(22) = 33.9. Thus, the fitted ARIhllA(0, 1, 1) X (0, 1,

judged adequate for the series.
model in (8.4.5) is
Forecasting Because the model in (8.4.5) is adequate, we can use it to forecast the future
employment figures. As discussed in Chapter 5, for a given forecast origin, say t = 132, forecasts can be calculated directly from the difference equation. For Equation (8.4.5)we have
Thus, the 1-step ahead forecast from the time origin t = 132 is given by
where
and
TABLE 8.7 Residual ACF, ;k, from the fitted model (1 - B](1
(1 - .33B) [l - .82fI1*)a,.
- B'2]Z, =
176
CHAPTER 8 Seasonal Erne Series Models
To derive the forecast variance, we first rewrite the model in the following AR representation because the model is nonstationary but invertible:
where
and, hence,
By equating the coefficients of Bj on both sides of (8.4.7), we have
Now, from (5.2.26), the $j weights that are needed for calculating the forecast variance can
be easily obtained as folloivs:
and
where the .nj weights are given in (8.4.8). By (5,2,9),the variance of the forecast error is
and the 99% forecast h i t s , by (5.2.10), are given by
177
The first 12 forecasts, i.&, ?132(~), for 1 = 1, 2, . . . , 12, and their standard errors are given
in Table 88.. These forecasts and their 95% forecast limits are also plotted in Figure 8.8.
Because the process is nonstationary, the limits become wider as the forecast lead time 1
becomes larger.
TGBLE 8.8 Forecasts for youth employment at the

origin of December 1981.
A
Lead time
Forecasts Zk32(I)
Forecast
+ Forecast Limit
FIGURE 8.8 Forecasts and their 95%confidence limits

for youth employment at the origin of December 1981.
Standard error
178
FIGURE 8.9 Quarterly beer production, in millions of barrels, between

1975 and 1982 (Series W10).
EXAMPLE 8.4 Figure 8,9 shows the 32 consecutive quarters of U.S. beer production, in
millions of barrels, from the first quarter of 1975 to the fourth quarter of 1982. The series is
listed as Series W10 in the appendix, The numbers of sample ACF and PACF values used to
identify a seasonal model with a seasonal period s should be at least 3s. Series WlO may be
too short. We choose it because sometimes we do need to construct a model rom a relatively
short series.
The First Tentative Model Based on the Original Series To assess the forecasting
performance of a model, in this example, we use only the first 30 observations of the series
for model construction. The power transformation analysis of these 30 observations indicates that there is no need for a power transformation. The sample ACF and sample PACF
as shown in Table 8,9 demonstrate a strong seasonal pattern of period 4. Although the
sample ACF at multiples of 4 are larger than their neighboring values, they are decreasing
and not large. It is not evident that we need a seasonal differencing. Thus, we start with a
tentative model of the form
(1 -
w)(z,
- IJ.) = a,.
(8.4.11)
The estimation of the above model gives 6 = -93 with the standard error of .07. The
95% confidence interval of @ clearly contains 1 as a possible value. The residual ACF
has a significant spike at lag 4 (table not shown); hence, the model is inadequate.
179
TABLE 8.9 Sample ACF and sample PACF for U.S.

quarterly beer production (Series W10).
(a)
bk for {Z,)
(2 = 44.92, S,
= 5.67,
n = 30)
Models Based on the Seasortally DllfferencedSeries As a result of the previous analysis,

we consider the seasonally differenced series W,= (1 - B ~ } zwith
, its sample ACE and sample
PACF shown in Table 8.10.
That the sample ACF of the series (1 - BJ)2, as shown in Table 8,10(a) has only one
significant spike at lag 4 suggests the following tentative model:
where Bo is included because

Parameter estimation gives
SF = 1 . 3 6 / ( 2 . 0 3 / 6 )
= 3.42, which is significant.
and :?t = 2.39, where values in parentheses below the parameter estimates are the associated standard errors. The residual ACF as shown in Table 8.11 indicates no model inadequacy.
Careful readers may notice, howver, that the sample PACF of (1 - B ~ ) z ,as shown in
Table 8.10(b) also has only one significant spike at lag 4, with its magnitude approximateIy
180
Sample ACF and sample PACF for the

seasonally differenced series of U.S. quarterly beer
production.
TABLE 8.10
(a]
h for {w,= (1 - B ~ ) z , )(F= 1.36, Sw = 2.03, n = 26)
TABLE 8.11 Residual ACF, jk,from the fitted model

[l - B4)Z, = 1.49 + (1 - .87B4)a,.
equal to the sample ACF at the same lag. This fact suggests that the following purely
seasonal AR process could also be a good tentative model:
(I
@ ~ j ) (-l E4)Z, = do
+ a,.
(8.4.14)
181
TABLE 8.12 Residual ACF, ;k, from the fitted model

(1 .66114)(l - B4)2, = 2.51 + a,.
The estimation results are
with G: = 3.06. The residual ACF as shown in Table 8,12 also indicates model adequacy.
Model Selection Based on Forecast Errors It is not uncommon that several adequate
models can be found to represent a series, The ultimate choice of a model may depend on the
goodness of fit, such as the residual mean square or criteria discussed in Chapter 7. If the main
purpose of a model is to forecast future values, then alternative criteria for model selection can
be based on forecast errors. Let the 1-step ahead forecast error be
where rz is the forecast origin larger than or equal to the length of the series so that the evaluation is based on outsample forecasts. The comparison is usually based on the folIo\ving
summary statistics,
1. The mean percentage error, which is also referred to as bias because it measures forecast bias, is
MPE =
2.
-z2+)loo%,
-
Tfie mean square enor is

1 "
MSE = - 2 e f .
Ml=l
182
TABLE 8.13 Comparison of the forecasts between models (8.4.133 and (8.4.1s).
Lead time
Actual value
Model (8.4.13)
Forecast value
Error
Model (8.4.15)
Forecast value
Error
MPE
MSE
MAE
W
3. The mean absolute error is
MAE
- -z
1
lej[,
Ml=l
4. The mean absolute percentage error is

MAPE =
-x I
-
)loo%.
For illustration, we calculate the one-step ahead and two-step ahead forecasts &O(l) for 1 =
1 and 2 from the forecast origin 30 for the two competing models (8.4,13) and (8.4.15). The
forecast errors and the corresponding summarystatistics are shown in Table 8.13. The results indicate that Equation (8.4.13) slightly outperforms Equation (8.4.15) in terms of forecasts.
8.1 Find the ACF for the following seasonal models:

(a) Z, = (1 - BIB)(l - BIBS)a,,
(b) (1 - (PIBS)Z,= (1 - B1B)ar,
(c) (1 - aIP)(1
- #J$)& = at,
8.2 Identify appropriate time series models for the data from the following sample autocorrelations:
(a) n = 56, data = Z,, VZ, = f 1 - B)Z,, VV4Zl = (1 - B)(1 - B4)Z,
where W is used to refer to the corresponding series Z, VZ,, or VV4Zt.
Exercises
(b)
102, data = (1 - B)(1 - ~ ' * ) l nZ,, W
.24, s&
183
102.38.
?k
8.3 Consider the following model:
( 1 - ~ ' ~ )-( B)Z,

1
( 1 - B I B ) ( l - 81B12)a,
withdl = . 2 , @ = .8,anduz = 1.
(a) Express the model in terms of the AR form. Compute and plot the rr weights.
(b) Compute and plot the IC, weights that are needed to evaluate the forecast variance.
(c) Find the forecasts and their 95% forecast limits for the next 12 periods,
8.4 Consider the following model:
(a) Find the one-step ahead forecast in terms of past observations.

(b) Find the eventual forecast function.
8.5
Consider the number of visitors (in thousands) to a Colorado ski resort given as follows:
Year
Winter
Spring
Summer
Falf
(a) Plot the series, and examine carefully the trend and seasonal phenomenon contained in the data.
(b) Give a preliminary evaluation of the series using Buys-Ballot table.
(c) Fit a proper regression'model such as Equation (8.2.8) to the data using only
observations between 1981 and 1985.
(d) Fit a seasonal ARIMA model to the data using oniy observations between 1981
and 1985,
(e) Calculate the forecasts for the next 4 periods using the models obtained in
parts (c) and (d); compare two models based on forecast errors.
184
8.6 Consider the following U.S. liquor sales (in millions of dollars):
Year
Jan.
Feb.
Mar.
Apr.
June
May
July
Aug.
Sept.
Oct.
Nov.
Dec.
(a) Build a seasonal ARIMA model for the series.

(b) Forecast the next 12 observations, and find their 95% forecast limits.
8,7 Consider the following U.S. personal consumption of gasoline and oil (in billions
of dollars) between 1967 and 1982:
Year
I1
I11
1V
(a) Build a seasonal ARIMA model for the series.

(b) Forecast the next 4 observations and find their associated 95% forecast limits.
Exercises
185
8.8 Consider the following monthly employment figures for young men between ages 16
and 19 in the United States from January 1982 to November 2002 (in thousands):
Year
Jan.
Feb.
Mar.
Apr.
May
June
July
Aug.
Sept.
Oct.
Nov.
Dec.
(a) Build two alternative seasonal models for the series using observations from
January 1982 to December 2001,
(b) Compare your models with the model discussed in Example 8.3.
(c) Forecast the next 12 observations, and comment on a model selection in terms of
forecasting performance.
Testing for a Unit Root

As discussed in Chapter 4, a nonstationary time series can often be reduced to a stationary
series by differencing. To decide whether to take the difference, so far we have relied on our
own judgement after examining the plot of the series and the pattern of its autocorrelation
function. For a formal treatment, we present several unit root tests for nonseasonal and
seasonal time series i n this chapter. Empirical examples are used to illustrate the tests.
9.1 Introduction
In Chapter 4, we introduced a homogeneous nonstationary time series that can be transformed to a stationary series by taking a proper order of differencing, Specifically, if a
time series Z, is nonstationary and its dth difference, Ad Z, = (1 - B ) ~ z , , is stationary
and also can be represented as a stationary AfZMA(p, q) process, then the Z, is said to follow an ARIMA(p, d, q) model. The Z, in this case is referred to as an integrated process
or series.
In model identification discussed in Chapter 6, the decision for differencing is based
on an informal procedure relying on a subjective judgment of whether the sample ACF of
a time series fails to decay rapidly. If we decided that was the case, then we would fit the
series with an ARXMA model. Because an ARIMA model can be regarded as a more general ARMA model with a unit root in the associated AR polynomial, for a more formal
procedure we can introduce tests to test for a unit root in our model fitting, Most of these
tests can be performed by simply using the output from a standard multiple regression
procedure.
9.2 Some Useful Limiting Distributions

To develop a desired test statistic and discuss its properties, we will use the approach
employed by Chan and Wei (1988) and begin with the following useful notions. Alternative proofs of.some results presented in this chapter can also be found in Fuller (1996,
Chapter 10).
A process, SV(t), is continuous if its time index t belongs to an interval of a real line. For
distinction, we often write a continuous process as W(t)rather than TV,. A process W ( t ) is
9.2
Some Useful Limiting Distributions
187
said to be a Wiener process (also known as Brownian motion process) if it contains the
following properties:
1,
2.
3.
4,
W(0)= a;
E [ W ( t ) ] = 0;
W(t) follows a nondegenerate normal distribution for each t; and
W(t) has independent increments, i.e. [W(t2) - W(tl)] and [W(t4) - W(t3)] are
independent for any nonoverlapping time intervals (tl, t 2 ) and (tg, t4).
W~thno loss of generality, we consider tin the closed interval between 0 and 1, i-e., t E 10, 11.
Furthermore, if for any t, W(t) is distributed as N(0, t), then the process is also called standard
Brownian motion.
Given the i.i.d. random variables a,, for t = 1, . . . , n, with mean 0 and variance uz,define
for 0 5 x < l/n,
for l/n 5 x < 2/11,
for 2/11 5 x < 3/n,
That is,
where x
[O,l] and [xn] represents the integer portion of (xlz). Now,
Because, as
--r
m,
6 and ~ z a , / t / i ; ; ; jconverges
to a
N(0,a:) random variable by the central limit theorem, it foflorvs that F,,(x) converges in
distribution to f i N ( 0 , u:) = N(0, X U $ ) as n +00, which wjlI be denoted by
D
P
Fn(x)
N(0, x u ; ) , In the following discussion, we also use the notation X, --+X to
indicate the convergence of X, to X in probability as n
m,
It can be easily seen that the limit of the sequence of the random variable Fn(x)/u, can
be described by a Wiener process, i.e.,
188
where W(x)at time t = x follows an N(0, x). Specifically,
where W(1)follows an N(0,I).

Let Zt = al
+ a* + - - - + a, and Zo = 0.We can rewrite Fn(x) in (9.2.1) as

for0 5 x < l/n,
for l/n 5 x < 2/n,
for 2/12 5 x < 3/n,
\ z"/fi,
forx
(9.2.6)
I.
Then, (9,2,5)implies that
and
Also, it is clear that the integral j:Fn(x) dx is simply the sum of the area defined in (9.2.6),i.e.,
Thus, by (9.2.4),
Similarly,
and
Next,
+ a,)2 = 2:-1 + 2Z,-la, + a:,
2: =
zt-,a,
1
?[z:
z:-,- a:],
9.3 Testing for a Unit Root in the AR(1) Model
189
and summing from I to n gives
Therefore,
'cT.[~(l)]'
2
,.:
1
1
= - f l : { [ ~ ( l ) ] ~ - I),
2
(9.2.1 1)
which follows from (9.2.8) and that [ ~ : = l a f / n ]
o:.

The simplest model that may contain a unit root is the AR(1) model. We discuss three cases
for this model: a model without a constant term, a model with a constant term, and a model
with a linear time trend.
9.3.1 Testing the AR(1) Model without a Constant Term

For the simple AR(I) model
with t = 1, . . . , n and Zo = 0, where the a, is the Gaussian N(0, a:) white noise process,
the unit root test implies a test for a random walk model, HO: 4 = 1. The alternati-ve is that
the series is stationary, i.e., HI : #I < 1. Thus, if tbe test is based on the statiGic 4 used to
estimate 4, the null hypothesis will be rejected if #I is much less than 1 or if ( 4 - 1) is too
negative. With the assumed Zo and following (7.4.4) we obtain the OLS estimator of I$ in
(9.3.1) as.
It is tempting to use the normal distribution given in (7.3.26) or the t-distribution given in
(7.3.27) to test the above hypothesis. These distributions, however, only hold when ]#I < 1.
Under the hypothesis Ha : 4 = 1, we note that
190
Thus,
The result folfows from (9.2.10), (9.2.1 1) and that under Ho, (9.3.1) becomes a random walk
model that can be written as
Zr = a l
+ az+
* * '
Recall from (7.3.26) and (7.3.27) that when 1$1
fa,.
(9.3.5)
< 1, we can test the hypothesis Ho : 4 =
t$ = 1, however, we can no longer use

these distributions. In fact, in (9.3.4), lV(1) is known to be an N(0, 1) random variable.
Hence, fW(1)12 follows the chi-square distribution with one degree of freedom, i.e., x2(1).
The probability that a X2(1) random variable is_less than I is ,6827, Because the denominator is always positive, thejrobability that 11(4 - I ) < 0 approaches ,6827 as n becomes
large. The OLS estim_ator4 clearly underestimates the true value in this case, and the limiting distribution of n(+-- 1) is clearly skewed to the left. As a result, the null hypothesis is
rejected only when n ( 4 - 1) is really too negative, i.e., much less than the rejection limit
when the normal or the t-distribution is used.For any given probability density function f(x), if we wish to calculate
+o using either the normal or the t-distribution. When
and the integration cannot be solved algebraically, we can use the Monte Carlo method to
approximate the integral by a sum in the following way. First, generate n independent uniform random variables XI,, , X,,on fO,I]. Then compute
..
By the law of large numbers, this sum converges to E[f(X)] as n becomes large, which
is simply
The percentiles for the empirical distribution of n($ - 1) given in (9.3.4) were constructed
by Dickey (1976) using the Monte Carlo method and reported in Fuller (1996, p. 641). They
are reproduced in Table F(a) of the appendix.
191
The commonly used t-statistic under Ho is
where
s4 =
[ ~ ~ (]e ~ - ~ )x
-1 ID
and
i+;
,=I
t=1
(Zt - $2,-d2
(11 - 1)
'
It can be easily seen from (9.3.3) that
which follows from (9.2.10), (9.2.1 1) and that $2 is a consistent estimator of 6;.
The nature of the distribution of T is similar to that of n ( 4 - 1). We reject Ho if Tis too
large negatively. The percentiles for the empiricaf distribution of T were also constructed by
Dickey (1975) using the Monte Carlo method and reported in Fuller (1996, p. 642). They are
reproduced in Table G(a) of the appendix.
The use of the assumed initial value Zo = 0 is purely for the convenience of deriving a
limiting distribution. In actual data analysis, we normally use only real data points; hence
(9.3.1) is computed only for t = 2, , . , 11. As a result, we often use only (11 - 1) sums of
squares and produc9 when estimating the regression parameter in (9.3.2).AWiththat in mind,
the test statistic n(4 - 1) given in (9.3,3) can be repla_ced by (rt - I)(+ - 1). It is clear
that the large sample results of ( a - 1)(4 - 1) and n ( 4 - I) are exactly the same.
EXAMPLE 9.1 As shown in Table 7.3 of Chapter 7, we fit Series W5 with two alternative
models, a random walk model and an AR(1) model with a large value of 4. Now let us more
rigorously test for a unit root. To facilitate the test, we consider the series Z,, which is the
mean adjusted data set of the 71 annual cancer death rates of Pennsylvanians between 1930
and 2000. That is, Z, = - P, where Y, ls the annual cancer death rate and is the mean of
the series. To test whether the underlying process contains a unit root, we fit the following
regression model
2, = $Z,-,
+ a,.
(9.3.8)
192
The OLS regression equation is
4.
where theAvaluegiven in the parenthesis is the standard error of

With n = 71, we have
( n - I)(+ - 1) = 70(.985 - 1) = -1.05, which is not less than the 5% critical value
-7.7 for n = 50 or -7.9 for n = 100 given in Table F(a). Thus, the null hypothesis, Ho : 4 = 1,
cannot be rejected, and we conclude that the underlying process for the data contains a unit root.
The t statistic is given by
which again is not less than the 5% critical value -1.95 given in Table G(a). Hence, we
cannot reject the null hypothesis, Ho : 4 = 1, and obtain the same conclusion that the underlying model contains a unit root.
9.3.2 Testing the AR(1) Model with a Constant Term

The mean, p = E(Zt), of the AR(1) process given in (9.3.1) was assumed to be 0. For a
nonzero mean AR(1) process, we consider the foflowing model with a constant term
Z,
+ 4Zi-, + a,,
(9.3.9)
for t = I, . . . ,n, where werecall from (3.4.16) that a = p(1 - $1.

First, note that in terms of the matrix form of a standard regression model, we can
rewrite (9.3.4) as
where
The OLS estimator,
b, of P is given by
= (x'x)-'X'Y,
Thus,
aP
193
where following Dickey And Fuller (1979), we use the subscript p in

to denote that the
estimate is based on (9.3.9), with a constant term related to the mean of the process, Because
- 8 ) = (x'x)-'X'Eand under Ho : # = 1, we have a = 0,it follows that under the null
hypothesis we have
(b
Thus,
by (9.2.5),(9.2.9), (9.2.10), and (9.2.11). It follows that
194
The corresponding t statistic is
where from (7.3,19),
S$p =
"[o
I]
~ ~ l - ~ ] , ; , ] l D ,
Czt-1 cz:-1
1=1
t=l
For the sampling distribution of T,, we first note that
hence, from (9.2.9) and (9.2.10)
Rewriting T, as
and using (9.3.1 1) and (9.3.13), we have that
Again, the-subscript ,u is used in T,, to denote that (9.3.9), with a constant term related
to the mean of the series, is used. The empirical percentiles of n ( 4 , - 1) and T, were constructed by Dickey (1974) and reported in Fuller (1996, pp. 541-642). They are reproduced
in Table F(b) and Table G(b), respectively.
195
EXAMPLE 9.2 Let us consider Series W6, the 114 yearly observations of U.S. tobacco production from January 1871.to 1984. To test for a unit root, we fit the foflowing regression model
Z , = cr
+ 4Z,-1 + a,.
(9.3.16)
where the_ values in the parentheses are the standard errors. With ti = 114, we have
(ti - I)(#, - 1 ) = 113(.9143 - 1) = -9.6841, which is not less than the 5% critical
value - 13.7 for n = 100 and - 13.9 for n = 250 given in Table F(b). Thus, the null hypothesis, Ho : # = 1, cannot be rejected, and we conclude that the underlying process for the data
contains a unit root with a = 0.
The t statistic in this case becomes
which is not less than the 5% critical value -2.90 for 31 = 100 and -2.88 for n = 250 given
in Table G(b).Thus, again the null hypothesis, Ho : 4 = 1, cannot be rejected,
9.3.3 Testing the AR(I) Model with a Linear Time Trend

Sometimes a time series process may contain a deterministic linear trend. One example is
Z, = a
+ St + 4Z,-I + a,.
(9.3,17)
h
The test statistic based on the OLS estimator of 4 , denoted by #J,,

and its limiting distribution can be derived using_the same procedure as in previous cases. The empirical percentiles
of the distributions of ? I ( + , - 1) and I; = (4, - 1)/S& under the hypothesis that 4 = 1 and
S = 0 were constructed by Dickey (1976) and reported in FulIer (1996, pp. 641-642). They
F e reproduced in Table F(c) and Table G(c), respectively. Again, the subscript t is used in
4, and T, to denote that (9.3.17) with a linear time trend is used,
EXAMPLE 9.3 Let us again consider Series W5, the 71 annual cancer death rates of
Pennsylvanians between 1930 and 2000. It was shown in Example 9.1 that the underlying
process for the series contains a unit root. Recall fiom Table 7.3 that a random walk model
with drift was used to fit the series: As shown in Section 4.2.2, the implication is that the
model may contain a deterministic component, Thus, we will now try to fit the series that is
presently denoted as 2,with the model
Z, = a
+ fit + #JZrll 4- a,.
196
where the values in the pahrentheses are the standard errors. Now the value of the test
statistic becomes ( n - I)(+, - 1) = 70(.9013 - 1) = -6.909 which is not less than the
5% critical value -19.7 for n =: 50 or -20.6 for n = 100 given in Table F(c). Thus, the null
hypothesis, Ho : 4 = 1, cannot be rejected, and we conclude that the model for the series of
annual cancer death rates contains a unit root.
which is not less than the 5% critical value -3.50 for n = 50 or -3.45 for n = 100 given in
Table G(c). Thus, again the null hypothesis, Ho : 4 = 1,cannot be rejected.
It is important to note the difference between (9.3.9) and (9.3.17). Not rejecting Ho :
= 1 in (9.3.9) simply implies that the model contains a unit root because 4 = 1 also indicates that cu = 0. In (9.3.17), however, not rejecting the null hypothesis, Ho : 4 = 1, does not
necessxily imply that ru = 0. Thus, the conclusion in this case is that the model contains a
unit root with or without a deterministic tern, depending on whether the intercept term is
statistically significant. As shown in (9.3.181, the constant term is statistically significant at
the 10% level. In a preliminary analysis, a random walk modef with drift could be used to fit
the series.
9.4 Testing for a Unit Root in a More General Model

Note that the ARfl) model without a constant term in (9.3.1) is reduced under the null hypothesis of a unit root to a random walk model
(1 - B)Z, = a,,
where a, is the Gaussian white noise process. As shown in (9.3.5),Z, in this case becomes the
sum o f t independent and identically distributed random variables, i.e., 2, =: a, a2 . +
a,. More generally, we may have
+ + -
where X,is a nonwhite noise stationary process. For the following discussion, we assume
that X,is a general linear process

03
197
CO
where @ ( B ) =
$j~i,'@
= ~0, and Ej=ojl$jl < 00.
To test for a unit root in this general case, we can fit the following OLS regression
and consider the estimator
Under the null hypothesis, Ho: # = 1, we have
and
For the sampling distribution of (9.429, we note that by substitution (9.4.1) becomes
Now,
Hence,
198
where
I = -~ , ~ o ( ~ ~ o @ j + l + =
i )~
a ~f .f, i o a i a , -and
i ori = - 2.
J=O I
Thus,
Because
it follows that the process Y, is stationary. Clearly, I$(l)I < a. Thus, the nonstationq
process 2, given in (9.4.1) is actually the sum of a random walk, a stationary process, and the
initial conditions,
Let
for0 s x < l/n,
for 1/12 I x < 2/11,
for 2/n 5 x < 3/14
(9.4.8)
forx = 1,
where Z, = @(l)[al+ a2 + . a,] 4- Yt - Yo &. Because [Y, - + zo]/Ih+

0,
comparing (9.4.8) with (9.2.1) or (9.2.61, we see fiom (9.2.4) that under (9.4.1), the limiting
distribution of+H,(x)is given by
%
Specifically,
%
- Hn(l)
fi -
199
(9.4.10)
ua$(l)W(l).
(6)
F,y=
It is easily seen from (9.4.7) that the limiting distributions of ~ , and

/fi
are
the same. Thus, we also have
or, equivalentlyy
which is often known as the central limit theorem for stationary processes.
It follows similarly from (9.2.9) and (9.2.10) that
n - 3 / 2 e ~ , -=
,
r-1
IX(..
.I-ua$(l)
--+
~(x)&,
0
and
Because
summing from I to rt we get
Hence,
which follows from (9.4.10) and that Z@Z
P
---+
0 and
[8:=Iz/~]
ob
P
(9.4.13)
200
It follows from (9.4.14) and (9.4.15) that
Note that
zF
the- ~
variance
]
of the estimator
which follows from (9.4.14). Further note that u $ / [ L ~ = ~ is
in tbe OLS regression in (9.4.3). Hence, following Phillips and Perron (1988), we can
use the following modified statistic,
4,$8,
where &$ and ?(I) are consistent estimates of u$ and y(l), in our test, Here we note fiom
(2.6.8) and (2.6.9) that y ( B ) = u$@(B)$(B-') is the autocovariance generating function, Thus,
where yk is the kth autocovariance of the stationary process X,. With the modified statistic
(9.4.17), we can use the same Table F of the appendix for the significance test.
9.4
201
Testing for a Unit Root in a More General Model
In practice, we can use residual mean square error from the OLS regression fitting in
(9.4.3) for G$,and
where the +k's are the sample autocovariances of the OLS regression residuals. The value of
m is chosen such that Iqkl is negligible for k > m.

Similarly, the t statistic from the OLS regression fitting in (9.4.3) can also be modified so
that the same Table G of the appendix can be used for the significance test. For more detailed
discussion on the procedure of Phillips and Perron, we refer readers to their 1988 paper and to
Hamilton f 1994),
,
Alternatively, for any given accuracy, because any stationary process can be approximated by an autoregressive model, Dickey and Fuller (1979) suggest the use of a higher-order
AR model to solve the problem,
Suppose that the series is generated from the A R ( p ) model
where the a, is the Gaussian N(O,c$) white noise process with ~ ( a ! <
) cu, and aP(B) = 1 Q ~ B - ,. , - @, BP, lvhich may contain a unit root.
To test for a unit root, we assume that
(B) = 6-1(B)(l - B), where 6-1
(B) = ( 1 p l ~ - . - p P - ~B P - ~has
) roots lying outside the unit circle, This assumption implies that
..
P-1
q p - l f B ) ( l - B)Zt = (Z! -
C V ( Z-~Zt-j-l)
-~
a,.
(9.4.21)
j= 1
Thus, testing for a unit root is equivalent to testing c$ = 1 in the model

P- 1
+ j=1
Cqj~ztAj
+ a,,
Z, = 4zrU1
where
where
= (Zt-j
(9.4.22)
- Zt-j-l). Tn terms of the matrix form, we can rewrite (9.4.22) as
202
The OLS estimator,
=
Because = (X'X)-~X'Y
8, of @ is given by = (x'x)-l~'~.
+ (X'X)-~X'E,
it follows that P - P = (x'X)-~X'E.
Thus,
(X'X)-l~'(~#l
+ E) = p
under the null hypothesis of a unit root, we have
Let
We note that
203
Hence,
From (9.4.21), it is clear that under the null hypothesis we have

( 1 - B ) Z , = AZ, = X, = $(B)a,,
(9.4.27)
where I
)
@
) = l/cp,-l (B) is assumed to satisfy stationarity and the conditions given in
(9.4.2).Thus, from (9.4.14) and (9.4.15), we have
and
204
CHAF'TER 9 Testing for a Unit Root
for j = 1, . . . , (p - 1).Furthermore, from (9.4.7), we have
s:+p+l
where we use (9.2.1 1) and that

(5-1 - YO Zo)a,/n
0.
Applying (9,4.27), (9.4.28), (9.4.29), and (9.4.30) to (9.4.26), we obtain the following
interesting results:
1. Asymptotically, the estimators of and 9's are independent as shown in (9.4.29).

2. The qj's are parameters on the stationary regressors (AZt-l, . . . , AZ,-,+l). The Imiting distribution of their OLS estimators Gj's obtained from (9.4.22) is the same as
the standard asymptotic distribution for the OLS estimators obtained by regressing the
differenced series AZ, on AZt-l, . . , AZi-p+l. Hence, they can be tested using the
standard t statistics provided in the regression output.
3. The parameter q5 relates to the unit root test, and asymptotically by (9.4.28) and (9.4.30).
or, equivalently,
Comparing (9.4.31) with (9,3.4), we see that Table F(a) can be used when applying the
statistic n($ - 1)$(1).
Similarly, following (9.3.7) and from (9.4.26), (9.4.28), and (9.4.301, we see that the
"t statistic" in this case is
9.4 Testing for a Unit Root in a More Genera1 Model
205
As a result, the Table G(a) can be used in this general case. Similar results extend to the use of
Tables F@)and (c) and G(b) and (c), for the model with a constant tern and the model with a
line_ar time trend, respectively. Again we often use ( ~ t- p ) ( $ - l ) $ ( l ) instead of
tt($ - 1)+(1) if no initial values are assumed in Equation (9.4.22) and the regression is computed for [IZ - p) observation:, i.e., for Z,, t = ( p -it- I), . . . ,n. The limiting distributions of
(11 - p ) (# - 1)$(1) and n(# - l)$( I ) are exactly the same. The tests such as (9.4.31) and
(9.4.32) resulting &om the use of a higher-order AR model are often referred to as the augmented
Dickey-Fuller tests in the literature.
EXAMPLE 9.4 As an illustration, let us consider the 54 yearly advertising expenditures

of the Lydia Pinkham data listed as Series W12 in the appendix. Based on the ACFs and
PACPs of the original series and its differencing, we can show that a reasonable model for
the series is an AR(2) after differencing. More rigorously, we can assume an AR(3) model
and test for a unit root. Thus, we fit (9.4.22) forp = 3 with a constant, i.e.,
where AZ, = (Z, - Ztll). Without using assumed initial values, we will fit the model with
t -. 4, . ,54. The OLS regression equation becomes
. .
where the values in the parentheses are the standard errors. To compute the test statistic, we
note that under the null hypothesis (9.4.33) becomes
(1
where p = &/(I
mus,
and
- B)Z, = p + X,,
- QI - 0and Xt = $@)a,
= [ l / q ~ ~ - ~ ( B )=) a[1/(1
~
- plB - *B2))a,,
206
which is not less than the 5% critical value -13.3 for rt = 50 given in Table F(b). Thus, the
null hypothesis, Ho : C#J = 1, cannot be rejected, and we conclude that the process for yearly
advertising expenditures can be approximated by an AR(3) model with a unit root.
which is not less than the 5% critical value -2,93 for n = 50 given in Table G(b). A g i n the
null hypolhesis, Ho : = 1, cannot be rejected.
Some remarks are in order.
1. It is clear that Equation (9.4.22) can also be written as
where a = 4 - 1. Thus, in testing a unit root, we can also equivalently run the regression (9.4.34) and test Ho : a = 0 against HI : a < 0.For the AR(1) model, Equation
(9.4.34)becomes
AZ, = aZ,-I + a , .
(9.435)
2. If the null hypothesis is not rejected, we conclude that the model contains a unit root.
This conclusion implies, however, only that the series Zt is integrated of order
d greater than or equal to 1, To find out the order of required differencing, we can
repeat the above test on the series AZ,, A2Zt, and so on until an order of integration is
reached.
3. A nonstationary series may not be homogeneous and no differencing of any order will
transform it into a stationary series. In such a case, some other transformations such
as those discussed in Chapter 4 may be necessary. In practice, however, for most
homogeneous nonstationary series the required order of differencing is rarely greater
than 2.
It is clear that with the general structure introduced in this section, the test can be used for
any general mixed ARMA model. For more references, see Said and Dickey (1985) or Phiuips
and Penon (1988), among others.
9.5 Testing for a Unit Root in Seasonal Time Series Models

As discussed in Chapter 8, a seasonal time series can also be nonstationary and requires a
seasonal differencing to remove its nonstationarity. Recall that if 2, is a seasonal time series
with a seasonal period s, then its seasonal differencing is (1 - BS)Z, = 2, 2,-,. In such
9.5
Testing for a Unit Root in Seasonal Time Series Models
207
a case, we say that the associated time series model contains a seasonal unit root. A formal
testing procedure is introduced in this section.
9.5.1 Testing the Simple Zero Mean Seasonal Model

Consider the seasonal time series model (1 - (PBS)Z,= a, or, equivalently,
Z, = @Zt-,
+ a,,
(9.5.1)
where Zl-,, Zz-,, . . , , Zo are initial conditions and the a, are i.i.d, random variables with
mean 0 and constant variance a:. The OLS estimator of @ is given by
which is also the maximum likelihood estimator when a, is Gaussian. The Studentized statistic for testing the null hypothesis Ho : @ = 1 is
where
and
The sampling distributions of n ( 6 - 1) and T, under the nut1 hypothesis Ho : Q = 1, are

discussed by Dickey, Hasza, and Fuller f1984), and their percentiles for various sample sizes
and for s = 2,4, and 12 are reproduced in Tables H and I.
9.5.2 Testing the General Multiplicative Zero Mean Seasonal Model

For the more general multiplicative seasonal model, consider
where the roots of (1 - B - . . . - 4pBP) = 0 lie outside of the unit circle. We note
4) where 4 = (+I, . , &). The linear
that the a, in (9.5.6) is a nonlinear function of (a,
--
208

A
approximation of this function a,(@, t$), evaluated at (@, I$), using Taylor's expansion
gives
where Rt is the Taylor series remainder. Equivalently,
where e, = a,(@, 4) - R,.

To test the null hypothesis Ho : 0 = 1, Equations (95.6) and (9.5.7) suggest the following
two-step regression procedure:
A
1, Regress ASZt on
. . . ,ASZ,-, to obtain an initial estimator 4 of 4, where ASZ,
(1 - B3Zt. This estimator can be easily seen as consistent because under the null
hypothesis Ho : @ = 1, Equation (9.5.6) becomes
which is a stationary AR(p) model for the seasonally differenced series ASZI.
2. Compute a t ( l ,
from (9.5.8) and use Equation (9,5.7) to regress a,(l,
on
[(I - ),B $Z,-l,
AsZ,-2,., . . . , A S Z ~ - ~to] obtain the OLS esti,rPp - + p ) , The estimator of a =. (@ - 1) can
mator of (0- I) = a! and
be used to test the null hypothesis No: <P = 1. Under the null hypothesis, we have a = 0. If'
a is much less than 0, then it implies that < 1 and the process is stationary. Hence, we wiU
reject Ho : @ = 1if the estimator of a = (@ - I) is significantly negative.
6)
--. Gp~~)~,-s,
4)
Dickey, H%sza,and Fuller (1984) show that under Ho : @ = 1, the percentiles of Tables
H and I for n(@ - 1) and Tin (9.5.1) are also applicable for the general multiplicative seasonal model in (9.5.6).
The models in (9.5.1) and (9.5.6) imply that E(Z,) = 0. The extension to the seasonal
model with a nonzero mean is straightforward from the results of Section 9.3. For most practical purposes-a purely seasonal model without any regular ARMA component is rare. Most
of the time the question of a zero or nonzero constant and a zero or nonzero mean arises
when one needs to decide the proper regular differencing discussed in Sections 9.3 and 9.4.
In other words, in testing a unit root with a nonzero constant or a nonzero mean, we usually
9.5 Testing for a Unit Root in Seasonal Time Series Models
209
apply the results given in Sections 9.3 and 9.4. Thus, we will not present the extension of the
purely seasonal mode1 with a nonzero constant here. Instead, we refer the interested reader
to Dickey, Hasza, and Fuller (1984),
In actual data anatyzis, when there are no good initial values of Zl-,, Gms,
. . . , and &
available, the estimator Q, of @ and its associated statistics presented earlier should be modified accordingly. Thus,
and
In testing the hypothesis I& : Q, = 1, we will use (rr - s)@ - 1) instead of

although the limiting distributions of the two statistics are exactly the same.
n ( 6 - I),
EXAMPLE 9,s Consider the series, Z,, which is the mean adjusted data set of Series W10
of the 32 quarterly U.S. beer productions fiom the fhst quarter of 1975 to the fourth quarter of
1982, which we studied i n Example 8.4. That is, Z, = I; - F, where Y, is quarterly U.S.beer
productions and is the mean of the series. With no knowledge of the underlying model, we
can begin with the test of whether the model contains a unit root at the seasonal period, s = 4.
Thus, we fit the regression model
for t = 5, . . . ,32, without assuming any initial values for Z, with t < 1. The OLS regression
becomes
where the value in^ the parenthesis is the standard error. The value of the test statistic
becomes (tt - s)(@ - 1) = (32 4)(.9014 - 1) = -2.7608, which is not less than the
5% critical value -8.67 for n = 40 given in Table H. Thus, the null hypothesis, Ho : CD = 1,
cannot be rejected, and we condude that the model for the series of quarterly beer productions contains a unit root at the seasonal period of 4.
2 20
which is not less than the 5% critical value -1.87 for n = 40 given in Table I. Thus, again
the null hypothesis, Ho : @ = 1, cannot be'rejected.
The above test result suggests that a seasonal difference, (1 - B~)z,,is called for. TO
see whether the second-order seasonal difference is required we can let bV, = (1 - B ~ z , ,
for t = 5, . . , 32, with a total of 28 W,'s, and test the hypothesis,
: @ = 1, i n the
regression model
The OLS regression results are
The value of the test statistic becomes (n - s)(& - 1) = (28 - 4)(.0628 - 1) =

-22,4928, which is much less than the 5% critical value -8.67 for s = 4 and n = 40 given
in Table H. Thus, the null hypothesis, No : Q, = 1, should be rejected, and we conclude that
the model for the first seasonal differences of quarterly beer productions no longer contains
a unit root at the seasonal period. The t statistic in this case becomes
which is also much less than the 5% critical value -137 for s = 4 and n = 40 given in
Table I, and we should reject the null hypothesis, Ho : iP = 1.
The above results imply that the series requires one seasonal difFerencing of period 4.
To see whether the series also needs a regular differencing, we can apply the results in
Section 9.3.1 and test the hypothesis, Ho : 4 = 1, in the regression mode{
IK
#W-l + a,,
(9.5.14)
...
for t = 1,
, 28, where for convenience we reindex the 28 available differences starting
from time 1. From the OLS regression results
- 1) = (28 - 1)(.336 - 1) = -17.928, which is much less than the

5% critical value -7.3 for n = 25 or -7.7 for n = 50 given in Table F(a). Thus, the null
hypothesis, Ho : 4 = 1, should be rejected, and we conclude that the model for the first seasonal differences of quarterly beer production no longer contains unit roots, The t statistic in
this case becomes
we have fta - I)($
Exercises
21 1
which is also much less than the 5% critical value - 1.95 given in Table G(a); hence, we
reach the same conclusion. The implication is that we can identify a stationary model for the
quarterly differences, (1 - B~)z,.
9.1 Let Z,
+~,~iI&j
xJzo
or, equivalently, Z = p
(@a, where
z,zol$jI
$(B) =
$ j ~$0 ~=, I, and
< m.
(a) Asgme that the a, are white noise with mean 0 and variance (T:. Show that
&=-Yj
= u1[I&(1)l2, where yj is the jth autocovariance of the process Z,.
(b) Assume that
xJ:i+j
variables. Prove that
# 0, and the a, are i.i.d. (0, u:) random

D
f i ( Z - a) 3 ~
( 0u~[4(1)]').
,
(c) Show that if Z, is anAR(1) process, Z, = #z,-~ + a,, where zt = Z,
- p and
1#1 < 1, then f i ( z - p ) - f l r ~ ( 0ui(l

,
- @)-l).
9.2 Derive the limiting distribution of ,t(&
- 11) for the model given in (9.3.17).
9.3 Derive the limiting distribution of T, for the model given in (9.3.17).
9.4 (a) Test for a unit root on Series W7.

(b) Test for a unit root on Series W16.
9.5
(a) Find the limiting distribution of n(4, - 1) in testing a unit root for the model
where -1 < 8 < 1.

(b) Test for a unit root on Series W6,
9.6 Derive the limiting distribution of n(&
- 1) for the model given in (9.5.1).
9.7 Derive the limiting distribution of Tfor the model given in (9.5.1).
9.8 Perform unit root tests on the series of U.S, personal consumption of gasoline
and oil given in Exercise 8.7 and determine the required differencing to make the
series stationary.
9.9 Perform unit root tests on Series W14 and determine the required differencing to
make the series stationary.
Intervention Analysis
and Outlier Detection
Time series are frequently affected by certain external events such as holidays, strikes, sales
promotions, and other policy changes. We call these external events i~zterventions.In this
chapter, we introduce the technique, called iritemention analysis, to evaluate the effect of
these external events. Intervention analysis has been successfully used to study the impact of
air pollution control and economic policies (Box and Tiao, 1975), the impact of the Arab
oil embargo (Montgomery and Weatherby, 1980), the impact of the New York blackout
(Izenman and Zabell, 1981), and many other events. We first discuss the analysis when the
timing of the interventions is known. The method is then generalized to study the impact of
the events when the timing of interventions is uoknown and hence leads to the general time
series outlier analysis.

Given that a known intervention occurs at time T, is there any evidence of a change in the
time series (such as the increase of the mean level) and, if so, by how much? One may initially think that the traditional two-sample t-test could be used to analyze this problem in
terms of comparing the pre-intervention data with the post-intervention data, The t-test,
however, assumes both normality and independence. Even though the t-test is known to be
robust with respect to the normality assumption, it is extremely sensitive to the violation
of the independence assumption as shown by Box and Tiao (1965), who developed the
intervention analysis to study a time series structural change due to external events (Box
and Tiao, 1975).
There are two common types of intervention variables. One represents an intervention occurring at time T that remains in effect thereafter. That is, the intervention is a step
function,
213
The other one represents an intervention taking place at only one time period. Tfius, it is a
pulse function,
Note that the pulse function can be produced by differencing the step function s J ~That
. is,
plT) =
- Sig = (1 - B)$q. Therefore, an intervention model can be represented
equally well with the step function or with the pulse function. The use of a specific form is
usually based on the convenience of interpretation.
There are many possible responses to the step and pulse interventions. We illustrate
some commonly encountered ones.
sI~
1. A fixed unknown impact of an intervention is felt b periods after the intervention. Thus,
depending on the type of intervention, the impact is
2. An impact of an intervention is felt b periods after the intervention, but the response is
gradual. For a step input, we have
and for a pulse input,
where 0 5 6 5 1. For 6 = 0, (10.1.5) and (10.1.6) reduce to (10,1,3) and (10.1.4),

respectively. If 6 = 1, the impact increases linearly without bound. For most cases, we
have 0 < S < I, and the response is gradual.
For illustrations, we plot the above interventions with b = 1 and 0 < S < 1 in Figure 10.1.
Note that various responses can be produced by different combinations of step and pulse
inputs. For example, we may have the response
214

I
n
J
l
u
J
Response
UB S p
WEpt(T)
FIGURE: 10.1 Responses t o step and pulse inputs.

as represented in Figure 10.2. As mentioned-earlier, however, because p133 = (1 - B),!$~!
response (10.1.7) can also be written as
This model is useful to represent the phenomenon in which an intervention produces a

response that tapers off gradually but leaves a permanent residue effect in the system. The
impact of an intervention such as advertising on sales can be represented as shown in
Figure 10.2(a), and the effect of a price or a tax increase on imports may be represented in
Figure 10.2(b).
FIGURE 10.2 Response t o combined inputs, [woB/(l

(a) w g > Oand w l > 0. (b) oo < Oand wl < 0.
S B ) ] P ~+)
o,~~IT),
10.2 Exampfes of Intervention Analysis
215
More generally, a response may be represented as a rational function
where a(B) .= o o - wlB - . , . - osBSand 6(B) = 1 - aLB - . . . - 6, Br are polynomials in B, b is the time delay for the intervention effect, and the weights mi's in the polynomial
o(B) often represent the expected initial effects of the intervention. The polynomial S(B), on
the other hand, measures the behavior of the permanent effect of the intervention. The roots
of 6 ( B ) = 0 are assumed to be on or outside the unit circle. The unit root represents an
impact that increases linearly, and the root outside the unit circle represents a phenomenon
that has a gradual response.
For multiple intervention inputs, we have the following general class of models:
.. .
where lit,j = 1, 2,
, k are intervention variables. These intervention variabfes can be
either step or pulse functions. More generally, they can be proper indicator variables, as
shown later in Examples 10.2 and 10.4. The form W ~ ( B ) B ~ ~ / Sfor
~ ( the
B ) jth intervention is
postulated based on the expected form of the response given knowledge of the intervention.
The main purpose of intervention models is to measure the effect of interventions. Thus,
with respect to the intervention variables lit, the time series free of intervention is called
the noise series and is denoted by N,, and its model is hence known as the noise model, The
noise model [B(B)/!P(B)]~,is usually identified using the univariate model identifleation
procedure based on the time series 2, before the date of intervention, i.e., (2, : t < T } . If
diagnostic checking of the model reveals no model inadequacy, then we can make appropriate inferences about the intervention. otherwise, appropriate modifications must be made to
the modd and estimation and diagnostic checking repeated. For a nonstationary process, the
model in (10.1.10) normally does not contain a constant term Bo.
10.2 Examples of Intervention h a l y s i s

EXAMPLE 10.1 A utility company provides electricity, light and power, gas and gas
transmission, telephone and telegraph services, and water. Its business is regulated by
federal and state authorities and normalfy is given an exclusive right to perform its services in a specific area, free from any competition. As the result, its revenue-generating
ability is relatively stable, and its stock does not fluctuate in value as much as most
common stocks. It generally pays a high rate of dividend in addition to offering moderate
growth potential. Therefore, its stock becomes an important component in many investment portfolios.
Duke Energy Corporation is an utility company headquartered in Charlotte,North Carolina.
It contains many business units, including Franchised Electric, Natural Gas Transmission, Field
21 6
Services, Duke Energy North America, International Energy, and Other Operations. Through
these units, it engages in businesses not only in North Carolina but also other areas inside and
outside the United States. The company's shares are traded in the NeivYork Stock Exchange as
DUK, Like all other companies, it issued statements fiom time to time about its business and
financial conditions.
On July 9, 2002, Schiffrin & Barroway, LLP, a law firm located outside Philadelphia,
Pennsylvania, that specializes in representing shareholders and consumers in many classaction litigations throughout the United States, fiIed a compfaint in the United States District
Court for the Southern District of New York against Duke Energy Corporation alleging that
the company misled investors about its business and financial conditions and, consequently,
about the true value of DUK stock between July 22, 1999, and May 17,2002. The law firm
tried to recover damages for the company's breaches of its fiduciary duties and violations of
the federal securities laws on behalf of at1 investors who bought Duke Energy Corporation
stock during the period. Following the announcement, the stock of Duke Energy Corporation
fell $1.25 and closed at $29.06 per share that day.
In this example, we perform an intervention analysis to assess the effect of this classaction securities litigation announcement on Duke Energy Corporation's stock prices.
Figure 10.3 shows Series W11, the daily closing stock price of Duke Energy Corporation
between January 3,2002, and August 31,2002. 'There are 166 observations in total, and the
closing price, $29.06, of July 9,2002, corresponds to the 129th observation.
Because the announcement was made late in the day and July 10, 2002, was the first
trading day after the announcement, the period between January 3, 2002, and July 9,2002, is
regarded as the noise series containing no major intervention that would affect the stock
FIGURE 10.3 Daily stock price of Duke Energy from January 3, 2002,
to August 31, 2002.
10.2
217
Examples of Intervention Analysis
TABLE 10.1 Sample ACF and PACF of Duke Energy Corporation stock prices
price. The sample ACF and sample PACF of the original and the differenced noise series as
shown in Table 10.1 suggest the random walk model
(1 - B)N, = a,.
(10.2.1)
By assuming that the effect of the class-action litigation against the company is to cause
an immediate level change in stock prices, one could propose the response hnction
where
represents the impact of the litigation announcement and
0,
1,
t < 130 (July 10,2002),

t r 130 (July 10,2002).
The intervention model becomes

Parameter
.
Estimate
St. Error
2 18
CHAPTER 10 Internention Analysis and Outlier Detection
The sample ACF for the residuals show no signs of model inadequacy, and the estimate of the
intervention parameter is highly significant. Tfius, there is evidence that the announcement of the
litigation against Duke Energy Corporation did cause a downward effect on its stockprices.
EXAMPLE 10.2 Los Angeles has been known to be plagued by a special air pollution
problem. The problem comes from substances produced by chemical reactions in sunlight
among some primary pollutants such as oxides of nitrogen and reactive hydrocarbons. The
products of these chemical reactions are responsible for the notorious Los Angeles smog,
which causes such health hazards as eye irritation and lung damage. One measured product
that is indicative of the degree of this photochemical pollution is ozone, often denoted by 03.
Figure 10.4 shows the monthly average of hourly readings of O3 in parts per hundred million
(pphm) in downtown Los Angeles from 1955 to 1972.
To ease the air poliution problem, different methods were instituted, including the diversion of traffic in early 1960 by the opening of the Golden State Freeway and the inception
of a new law (Rule 63) that reduced the allowable proportion of reactive hydrocarbons in
the gasoline sold locally. Also, after 1966 special regulations were implemented to require
engine design changes in new cars in order to reduce the production of 03.
It was through
the study of the effect of these events on the pollution problem that Box and Tiao (1975)
introduced the intervention analysis.
FIGURE 10.4 Monthly average of hourly readings of O3 (pphm)

in do~vntownLos Angeles, 1955-1972.
10.2 Examples of Intervention Anaiysis
219
The period from 1955.to 1960 is assumed to be free of intervention effects and is used
to estimate the noise model for.N,. The sampleACE; within this period suggests nonstationary and highly seasonal behavior. The ACF of the seasonally differenced series (1 - B'~)N,
has significant spikes only at lags 1 and 12, which implies the following noise model:
Box and Tiao (1975) suggest that the opening of the Golden State Freeway and the implementation of Rule 63 in 1960 represent an intervention I,, which might be expected to produce
a step change in the O3 level at the beginning of 1960. Intervention I2 would be represented by
the regulations implemented in 1966 requiring engine changes in new cars. The effect of
I2would be most accurately measured by the proportion of new cars having specified engine
changes in the car population over time. Unfortunately, no such data are available. One might,
however, represent the effect of I2 as an annual trend reflecting the effect of the increased
proportion of new design vehicles in the population. Because of the differences in the intensity
of sunlight and other meteorological conditions between the summer months and the winter
months, the effect of I2 would be different in these two seasons. Thus, Box and Tiao (1975)
proposed the model
where
Ilt
13'
0,
1,
t<January1960,
t 2 January 1960,
1,
0,
"summer" months June-October beginning in 1966,

otherwise,
{I'
0,
"winter" months November-May beginning in 1966,

otherwise.
Estimation results of the above model are

Parameter
Estimate
St. Error
220
Residuals of 6, show no obvious inadequacies in the model. Thus, there is evidence that
(1) intervention l1 reduces the level of O;, and (2) associated with intervention 12, there is
a reduction of O3 level in the summer months but not in the winter months.
EXAMPLE 10.3 The Arab oil embargo in November 1973 greatly affected the supply of
petroleum products to the United States. The prices of petroleum products rocketed, and the
cries for energy conservation were loud and clear everywhere in the United States. It is
generally believed that the rate of growth of energy consumption following this event was
lower for the post-embargo years, To test the validity of this belief, Montgomery and Weatherby (1980) applied an intervention model to the natural logarithm of monthly electricity
consumption from January 1951 to April 1977 using a total of 316 observations.
Although the embargo started in November 1973, Montgomery and Weatherby (1980)
assumed that the effect of this embargo was not felt until December 1973, Thus, they used
275 months of data from January 1951 to November 1973 to model the noise series and
obtained the folloiving ARZMA(0, 1,2) X (O,1, lIlzmodel:
By assuming that the effect of the embargo is to cause a gradual change in consumption, one
could propose the intervention model
where oo represents the initial impact of the oil embargo and
The estimates and the associated standard errors are

Parameter
Estimate
St. Error
Because the estimate of S1is not statistically significant, Montgomery and Weatherby
dropped the parameter S1 in the foIlowing modified model:
10.2 Examples of InterventionAnalysis
221
where I, was defined as in (10.2.7). The estimation results are

Parameter
Estimate
St. Error
The estimates are all statistically significant. The residual ACF does not exhibit any model
inadequacy. Thus, the intervention model in (10.2.8) is satisfactory. The result implies that the
embargo induced a permanent level change on electricity consumption. Because the model was
built using the natural logarithm of electricity consumption, the estimate of the intervention
effect in terms of the original megawatt hours (MWH) metric is e'o = e-,07 = .93. Therefore,
the post-intervention level of electricity consumption was 93% of the pre-intervention level, or,
equivalently, the effect of the Arab oil embargo reduced electricity consumption by 7%.
EXAMPLE 10.4 At exactly 5:27 P.M., November 9, 1965, most of New York City was
plunged into darkness due to a massive power failure. The blackout was long, and most of the
city remained dark during the night. On Wednesday, August 10, 1966, the New York E m s carried a front-page article with the headline "Births Up 9 Months After the Blackout." Numerous
articles were published afterward in newspapers and magazines both inside and outside the
United States alleging a sharp increase in the city's birthrate. A number of medical and demographic articles then appeared with contradictory statements regarding the blackout effect.
Using a total of 313 weekly births in New York City from 1961 to 1966, which is plotted in
Figure 10.5, Izenman and Zabell(1981) applied the intervention analysis to the phenomenon.
Because the blackout, which occurred on November 9, 1965, falls in the middle of the
254th week, the first 254 weekly birth totals are used to model the noise series with the
following process:
Furthermore, the obstetrical and gynecological studies show that the mode of the gestational interval, which is defined as the time from onset of the last menstrual period (LMP)
to birth, occurs at 40 or 41 weeks after the LMP. Izenman and Zabell(1981) suggested the
intervention form
where wo is used to represent the effect of the blackout and

1,
It = (0,
t = 292,293,294,295,
otherwise.
222
2500
1960
1962
1964
1966
1968
Year
FIGURE 10.5 Total weekly births in New York City, 1961-1966.

The previous intervention variable I, was introduced to take the value 1 for weeks 38 through
41 after the blackout to give the model the maximum possible chance of detecting the effect.
Thus, we have the intervention model
where I, is defined as in (10.2.11).

The estimates and associated standard errors are
Parameter
Estimate
St. Error
The residual ACF shows no signs of model inadequacy. Because the parameter estimate of wo
is not statistically significant, we conclude that intervention analysis of the New York City
birth data does not detect a significant increase in births that can be ascribed to the blackout.
Here, we note that the model estimated in (10.2.12)is slightly different from the model given
in Izenman and Zabell(198i).
223

Time series observations are sometimes influenced by interruptive events, such as strikes,
outbreaks of war, sudden political or economic crises, unexpected heat or cold waves, or
even unnoticed errors of typing and recording. The consequences of these interruptive
events create spurious observations that are inconsistent with the rest of the series. Such
observations are usually referred to as outliers. When the timing and causes of intemptions are known, their effects can be accounted for by using the intervention model discussed in Sections 10.1and 10.2. In practice, however, the timing of interruptive events is
sometimes unknown. Because outliers are known to wreak havoc in data analysis, making
the resuitant inference unreliable or even invalid, it is important to have procedures that
will detect and remove such outliers effects, The detection of time series outliers was first
studied by Fox (1972), where two statistical models, additive and innovational, were introduced. Other references on this topic include Abraham and Box (1979), Martin (1980),
Chang and Tiao (19831, Hillmer, Bell, and Tiao (1983), Tsay (1986), Chang, Tiao, and
Chen (1988).
10.3.1 Additive and Innovational Outliers

With no loss of generality, consider a zero mean stationary process. Let Z, be the observed
series and X, be the outlier-free series. Assume that (X,)follows a general ARMA(p q)
model
- -
where +(B) = l-+lB - . . - +p BP and B(B) = (1 -BIB - . - B,B4) are stationary

and invertible operators sharing no common factors and (a,) is a sequence of white noise,
identically and independently distributed as N(0, a:). An additive outlier (AO) model is
defined as
where
224
C W T E R 10 Intervention Analysis and Outlier Detection
is the indicator variable representing the presence or absence of an outlier at time T. An

innovational outlier ( 1 0 ) model is defined as'
Hence, an additive outlier affects only the Tth observation, Z n whereas an innovational outlier affects all observations ZT, ZT+l,
, . , beyond time T through the memory of the system
described by O(B)/t$(B).
More generally, a time series might contain several, say k, outliers of different types, and
we have the follo~vinggeneral outlier model:
where X, = ( 6 ( B ) / + ( B ) ) a,, v j ( B ) = I for an AO, and vj(B) = O(B)/+(B) for an I 0 at

time t = Tj.
10.3.2 Estimation of the Outlier Effect When the Timing

of the Outlier Is Known
To motivate the procedure for detecting A 0 and 10, we consider a simpler case when T and
all parameters in (10.3.1) are known. Letting
and defining
e, = .rr(B)Z,,
we have from ( 1 0 . 3 . 2 ~and
) (10.3,3b)that
~ ) a,
AO: e, = W T ( B ) I ~ +
and
10.3
Time Series Outliers
225
For n available observations, the A 0 model in (10.3.7) can be written as
Let hT
be the least square estimator of w for the A 0 model, Because (a,) is white noise,
from the least squares theory, we have that
AO;
&,=
e~ - x;l:wjeT+j
2;::7Tj
where a * ( F ) .= (1 - a l F such that Fe, = ertl, and T~ =
. . . - T,-~F"-~),
F is the forwatd shift operator
7 ~ ~ -1 ; ~
n-T 2
zj=O
rj.The variance of the estimator is
f
f
:
--
(10.3.1 1)
T ~ '
Similarly, letting CjlT be the least squares estimator of o for the I 0 model, we have
10:
and
Gr
= e~
(10.3. f 2)
226

Thus, the best estimate of the effect of an I 0 at time T is the residual en whereas the best estimate
of the effect of an A 0 is a linear combination of en e ~ +. ~
. . ,,and e,, with weight depending on
the sbvcture of the time series process. It is easily seen that V a r ( h T )5 Var(GT)= vi,and
in some cases, Vw(GAT)can be much smaller than crz.
Various tests can be performed for the hypotheses
No:
ZT is neither an A 0 nor an I 0
H2:
Zr is an 10.
The likelihood ratio test statistics for A 0 and 10 are
and
Under the null hypothesis Ho, both A1,=and
are distributed as N(0, I),
10.3.3 Detection of Outliers Using an Iterative Procedure

If' T is unknown but the time series parameters are known, we can proceed to calculate A1,, and
hZrfor each t = 1,2, . ,n, and then make decisions based on the above sampling results. In
practice, however, the time series parameters 4j,Oj, q,and a; are usually unkno\vn and have
to be estimated. It is known that existence of outliers makes the estimates of the parameters
seriously biased. In particular, as expected, a! will tend to be overestimated. Chang and Tiao
(1983) proposed an iterative detecting procedure to handle the situation when an unknoivn
number of A 0 or I 0 may exist.
..
Step 1, Model the series (2,) by assuming that there are no outliers. Compute the residuals
from the estimated model, i,e,,
*;
w h e r e d ( ~ )= (1
- &B-
be the initial estimate of (I.:.
$(B)Z,
- $ p ~ ~ ) a n d i =( ~(1) -
a l -~ - . . - 4
~ q )~
.
e t
227
Step 2. Calculate hl,, and,h2,,for i = 1 , 2 , . . . ,n, using the estimated model. Define
A
where T denotes the time when the maximuh occurs. 1f AT = /A1,TI > C, where Cis a predetermined positive constant usually taken to be some value between 3 and 4, then there is an
A 0 at time T with its effect estimated by GAT.One can modify the data using (10.3.2bf as
and define the new residuals using (10.3.7) as
If IT =
> C, then there is an I 0 at time T with its effect being hT,
This I0 effect can
be removed by modifying the data using f 10.3.3a),i.e.,
and defining the new residuals using (10.3.8) as
A new estimate
is then computed from the modified residuals.
x,
Step 3. Recompute ir,,

and i2,,
based on the modified residuals and
and repeat step 2
until all outliers are identified. The initial estimates in m(B) remain unchanged.
Step 4. Suppose that step 3 terminated and k outliers have been tentatively identified at
times T I , T2, . . . , and Tk. Treat these times as if they are known, and estimate the outlier
parameters w l , oz,. . . ,ukand the time series parameters simultaneously using the model
where vj (B) = 1for an A 0 and vj(B) = 6(B)/#(B) for an I 0 at t = Ti. This result leads to
the new residuals
A revised estimate of a: can then be caIculated.
228
CHAPTER 10 Intervention AnaIysis and Outlier Detection
Steps 2 through 4 are repeated until all outliers are identified and their impacts simultaneously estimated. Thus, we have the fitted outlier model
where $ , $ ( B ) = (1 obtained in the final iteration.
. . - - &BP)
and ; ( B ) = ( 1
- 8 1 -~
i q ~ q )are

The above outlier detection procedure can be easily implemented in any existing intervention analysis software or linear regression package. Both time series software SCA (1992)
and AUTOBOX have implemented the procedure and make the analysis easier. A computer
program based on a modification of the above procedure was also written by Bell (1983). We
illustrate the following examples by using AUTOBOX and a standard regression package.
EXAMPLE 10.5 The outlier detection procedure discussed in Section 10.3 has been applied
using software AUTOBOX to the U.S. quarterly series of beer production between 1975 and
1982, which was fitted earlier by a seasonal ARIMA(0, O7 0) X (0, 1,
model in Chapter 8.
The result indicates that there are no outliers evident with a significance level .05 in the series.
To check the efficiency of the proposed procedure, we artificially contaminated the series by
replacing the original observation ZI2 = 36.54 with a new observation ZI2 = 56.54, which
could be due to a typing error, We then apply the procedure to this outlier contaminated series,
which is plotted in Figure 10.6.
The analysis produces the following result:
Iteration
Detected Outliers
Time
me
Magnitude (G)
Thus, the procedure correctly identifies the A 0 at t = 12 in the first iteration. Although &
has also been detected as an I 0 in the second iteration, the effect is much smaller.
EXAMPLE 10.6 Series W1 was analyzed in Sections 6.2 and 7.6 resulting in an AR(1) model
with 6%= .21. Inspection of the residuals from the fitted model indicates the possible existence of a number of outliers. The series is the daily average number of defects per truck
1975
1977
1979
1981
1983
229
Year
FIGURE 10.6 Outlier contaminated U.S. quarterly series of beer

production between 1975 and 1982.
found in the final inspection at the end of the assembly h e of a truck manufacturing plant.
To ensure the maintenance of quality, the detection of outliers is always an important task in
quality control. In this example, we apply the previous outlier detection procedure to the
data set and obtain the following results:
Detected Outliers
Iteration
Time
Thus, we consider the following outlier model:
VP~
230
CHAPTER 10 Intervention Analysis and Outfier Detection
Simultaneous estimation of the previous parameters yields
2, = 1.14 4- 1.391$~~).991j7)
(.11)
(.37)
(1
- .28B)
(.11)
(-,61ij9)
(-19)
+ .661j4) + a,)
(.31)
(10.4.3)
and 6; = .II. Comparing (10.4.3) with (10.4.1), we see that a reduction of about 100% in
the estimated variance 6: of the at's occurs, from ,21 to .11, when the effects of the four
outliers are taken into account, Furthermore, the change in the estimate of the autoregressive
parameter is also substantial, decreasing from .43 to a much smaller value of .28. For a production process under proper quality control, one would expect the series of defects to be
random white noise. That would occur if a smaller critical value C were chosen and a few
more additional outliers were identified.

As indicated earlier, the intervention analysis is used when the timing and causes of intermplions are known. Because the timing and causes of interruptions are sometimes unknown, in
addition to the additive outlier (AO) and innovational outlier (10),the following level shift
(LS) and temporary change (TC) can also be incorporated in the iterative outlier detection
procedure discussed in Section 10.3:
Ls: 2, = xt+ (1
1
-
B)o~l!n
and
TC: 2, = X,
+ (I -1SB) Wlf(TI
9
where we recall that X, is the underlying outlier-free process. For empirical examples that
contain possible LS and TC situations, we refer readers to Chen and Liu (1991, 1993).
Other than the model based iterative detection procedure, many other h e series outlier
detection procedures have been introduced in the literature. For example, because the underlying outlier-free model is often unknown and outliers can distort model identification as shown
in the following, Lee and Wei (1995) proposed a model-independent outlier detection procedure, Other detection procedures include Tsay (19881, M o l t e r (19901, and Ljung (1993).
After outliers are identified, one can adjust data using (10.3.18) or (10.3.20) and then
pursue the analysis based on the adjusted data. A more hitfuf approach, however, is to search
for the causes of the identified outliers and further fine-tune the fitted model in (10.3.24).This
process is appropriate not only for parameter estimation but also for model checking and forecasting. In searching for the causes of an outlier, one may find the nature of the disturbance.
231
For example, some outliers may tum out to be important intervention variables due to some
policy changes with which the analyst was unfamiliar and hence were overlooked during the
preliminary stage of data analysis. Thus, instead of using the adjusted data by removing the
effects of outliers, the analyst should incorporate the information into the model by introducing
proper intervention variables and response.functions as discussed in Sections 10.1 and 10.2.
This explicit form of a combined intervention-outlier model is usually more useful in forecasting and control than the uaivariate fitted model based on the outlier-adjusted data.
EXAMPLE 10.7 As an example let us consider Series W14, the monthly airline passengers in the United States from January 1995 to March 2002 plotted in Figure 10.7.
Without looking at the plot and blindly applying the outlier detection method introduced
in the chapter to the series, we obtain the following result:
Detected Outliers
Iteration
Time
PP~
If we use a significance level Iess than .01, then the only outlier found is the observation at
time 81 that corresponds to September 2001, the month of the World Trade Center tragedy in
New York City, which is clearly an intervention event. The outlier procedure not only detects
the event but also suggests the form of the intervention.
1996
1998
2000
2002
Year
FIGURE 10.7 The monthly airline passengers in the United States

from January 1995 to March 2002 (Series W14).
232
The standard time series modeling on the subseries from January 1995 to August 2001
suggests the ARZMA(2,0, 0) X (0,1, O)lz seasonal model:
Thus, we will combine model (10.5.3) and the information about the observation at time 81
in the intervention model
where
0,
1,
t<81(Sept.2001)
t 2 81 (Sept. 2001).

Parameter
Estimate
St. Error
The effect of the September I f , 2001, tragedy on the airline industry is clearly devastating.
The iterative outlier detection procedure is based on the assumption that the underlying model for the outlier-free series is either known or can be identified. In practice, however, the underlying model is usually unknown and has to be identified through the
standard analysis of sample statistics, such as sample ACF, sample PACF, sample LACF,
and ESACF, Thus, it should be emphasized that the detection procedure of Section 10.3
works well only when the outlier effect is moderate and will not overshadow and obscure
the underlying pattern contained in the sample statistics of an outlier-free series. In more
serious cases, outIier contamination may make model identification impossible. For
example, the blowfly data exhibit a very clear autocorrelation pattern of AR(1) process,
as shown in Table 6.6. The estimate of the AR(1) parameter
was found to be .73 in
Table 7.3. The data are listed as Series W3 in the appendix. Now, suppose, due to a
typing error, that the value 22221 was used for Z20instead of 2221, Figure 10.8 shows
the series contaminated by this outlier at Zzo. Its sample ACF, sample PACF, and ESACF
are computed and shown in Table 10.2. The results all indicate a white noise phenomenon. The underlying AR(1) characteristics have been completely washed out by this single outlier. A white noise series is itself an outlier in empirical time series. Thus, when
encountering a white noise phenomenon in studying sample statistics such as sample
ACF, one should first examine whether there is an obvious outlier in the data. Such an
outlier can be easily detected by plotting the series, which is always the first aid for any
data and outlier analysis.
10.5 Model Identification in the Presence of Outhers
233
To reduce the adverse effect of outliers on the parameter estimation, a number of

robust estimation methods have been introduced in the literature. In terms of estimation of
the ACF, Chan and Wei (1992) introduced the following a-trimmedsample autocorrelation
function (TACF).
Let Z(l) 5 Z(2), . . , I
be the ordered observations of the given time series
Z1, &, . . . , Z., The a-trimmed sample autocorrelation function is defined by
where
FIGURE 10.8 Outlier contaminated blowfly data.
234
TABLE 10.2 Sample ACF, sample PACF, and ESACF for an outlier contaminated M(1) series.
AR
d"l={O'
1,
[c)
ESACF
10
ifZ~Z(g)~rZ,~Z(.-ptl),
otherwise,
and g is the integer part of [an]and O 5 a 5 .05. In other words, the TACF
in (10.5.5)
trimmed a% extreme values from the computation of the sample ACE
Because {L?)) is a deterministic sequence of real numbers (0 or 11, the asymptotic
theory for time series containing amplitude modulated observations can be applied to the
TACF, Follo~vingDunsmuir and Robinson (19811, under the white noise model it can be
easily shown that the
for k 2 1 are asymptotically independent normal random
variates with asymptotic variances l/v(k),where
Thus, we can approximate the standard error of the TACF by
Exercises
235
where
Note that f (k) < 1 when a > 0. Thus, Sip) > Sikfor the white noise model, which is expected
because the information in the trimmed series is always less than that in the original series under
the outlier-free situation.
The choice of a is very important for the TACF. If we choose a large value for a,the
TACF is no longer an efficient estimator because G(k) becomes smaller as a increases. On
the other hand, if we select a very small value for a , the purpose of the estimator (to trim out
possible outliers in the series) might not be fully fulfilled. In practice, we recommend using
a = 1 to 2% for general series, a = 3 to 5% for medium contaminated series, and a = 6 to
10% for heavily contaminated series.
Other robust estimations of ACF include the jackknife estimator proposed by Quenouilli
(1949) and the robust sample p d a l autocorrelation and autocorrelation functions by Masarotto
(1987). Chan and Wei (1992) show that in estimating the ACF, the simple TACF in (10.5.5) not
only compares favorably with other robust estimators in terms of the root mean squared error,
but it also performs well in the presence of both additive and innovational outliers.
When there are outliers in the series, all the proposed robust ACF estimations outperform
the standard sample ACF, Because, in practice, we may not know whether outliers exist, for
proper model identification the use of robust estimates of autocorrelations is recommended.
There are many studies on time series outliers. For other useful discussions on robust
methods and model identification in the presence of outliers, we refer readers to Tsay (1986),
Abraham and Chuang (1989), Chen and Liu (1991,1993), Wei and Wei (1998), and Li (2003),
In closing the chapter, we note that internention analysis is a useful technique to study
the impacts of intervention and outliers that normally cause the level shift of a time series.
On occasion, however, the shift occurs in the variance of a series. Different techniques are
needed to study this problem. Because of the length of the chapter, we will not discuss this
interesting problem and refer readers to Wichern, Miller, and Hsu (1976) and Abraham and
Wei (1984).
10.1 Consider the following response functions of interventions:

(1) U O P P
WI
(4) [(I Y 8 B )
(1 - B ) ] P i g
236
Wo
(27
( I - SB)(1 - B ) P'
(a) Plot the above response functions.
(b) Discuss possible applications of the various interventions.
10.2 Find a time series that was affected by some external events. Carry out intervention
analysis and submit a written report of your analysis.
10.3 Perform and report an iterative outlier analysis for the following time series (read across):
10.4 Find a time series of your interest, which was likely contaminated by some outliers or
10.5
interventions.
(a) Fit a best AREVIAmodel for the series.
(b) Detect all possible outlier and intervention variables, and build an outlierintervention model for the series.
(c) Compare and discuss your findings in parts (a) and (b).
(a) Find a time series model for the data given in Exercise 10.3 using the sample
ACF, I;k.
(b) Find a time series model for the same data in part (a) using the 5% trimmed
sample autocorrelation function,
(c) Compare and comment your results in parts (a) and (b).
;p!
Fourier Analysis
The time series approach presented in earlier chapters, which uses functions such as autocorrelations and partial autocorrelations to study the evolution of a time series through parametric
models, is known as time domaitt analysis. An alternative approach, which tries to describe the
fluctuation of time series in term of sinusoidal behavior at various frequencies, is known as
freqi~ertcydomain analysis. In this chapter, we introduce some basic concepts of Fourier analysis, which are fundamental for the frequency domain analysis.
1 1.1 General Concepts

Often it is more convenient to represent a function by a set of elementary hnctions called
a basis such that all functions under study can be written as linear combinations of the
elementary functions in the basis. One very useful set of these elementary functions involves
the sines and cosines or complex exponentials. In this chapter, we study how to construct
an arbitrary function using these sinusoids. The study of this subject is often referred to as
Fourier analysis due to the 18th-century French mathematician J. B. J. Fourier, who claimed
in 1807 that any periodic function could be represented as a series of harmonically related
sinusoids, Later he also obtained a representation for nonperiodic functions as weighted integrals of sinusoids that are not all harmonically related.
The development of the techniques of Fourier analysis has a long history involving
many mathematicians, including E. Euler, D. Bernoulli, P. S. Laplace, J. L. Lagrange, and
P. L. DiricNet, in addition to Fourier. Although the original work of Fourier analysis was solely
concerned with phenomena in continuous time, such as the physical motion of vibrating string
and heat propagation and diffusion, the essential ideas of Fourier analysis carry over to phenomena in discrete time. For our purposes, we concentrate on Fourier analysis of discrete t h e
functions or sequences. Specifically, we establish the orthogonality property of sine-cosine
functions and the complex exponentials in Section 11.2, Using this property, we develop the
Fourier series representations for finite sequences and periodic sequences in Sections 11.3 and
11.4. The Fourier transform of an aibitrary nonperiodic sequence is discussed in Section 11.5.
The Fourier representation of a continuous function is presented in Section 11.6. The fast
Fourier transform used in the actual computation of the Fourier representation is presented
in Section 11.7. With the proper understanding of these techniques, we are then ready for the
frequency domain approach of time series analysis in Chapters 12 and 13.
238
CHAPTER 11 Fourier Analysis
1 1.2 Orthogonal Functions

Let &(t) and #j(t) be compfex valued functions defined on the domain D, which is a subset
of the real line. Discrete-time functions #k(t) and 4j(t) defined on a discrete set are said to be
orthogonal if
where @(t) denotes the complex conjugate of +,(t). Continuous time functions 4k(tt) and
#j(l) defined on an interval of the real line are orthogonal if
In this book, we concentrate on the discrete-time functions, which are also called sequences.
There are many kinds of orthogonal functions. In this section, we introduce two equivalent
systems of orthogonal functions that are useful in time series analysis.
Suppose that the trigonometric sine and cosine functions sin (2vktln) and cos ( 2 ~ k t j n )
are defined on a finite number of 11 points, i.e,, for t = 1,2, . . ,n. Fork = 0, 1,2, . . , [11/2],
where [xlis the greatest integer less than or equal to x, the system
contains exactly n nonzero functions, i.e., none of which is identically zero, which follows
from the sine function being identically zero for k = 0 and for k = [n/2] if n is even.
More specifically, [11/2] = n/2 when n is even, and the system contains cos(2~0tln)= 1,
sin(2rktln) and cos(2?rkt/n) for k = 1,2, . . , 1112 - I, and cos(2r(n/2)t/rr) = cos(?rt) =
(- 1)'. If n is odd, then [n/2] = ( ~ r- 1)/2 and the system consists of cos(2mOt/n) = 1,
sin(2?rkt/ts) and cos(27~ktln)for k = 1, 2, . . . , (11 - 1)/2. In either case, it has exactly n
functions in the system,
Next, we show that the system (11.2,3) is actually a collection of orthogonal functions.
To do so, we use the Euler relation
eim = cos w
+ i sin o
and the identities

,io
sin w =
_
2i
e-i~
'
e i ~+ e - i ~
COS w
'
(1 1.2.4)
1l.2 Orthogonal Functions
239
Then
But
zeimi
= zcos
R
r=l
wt
t=l
+i
sin wf.
t=1
Hence, we have
(1 1.2.8)
t=
sin ot = sin(
w(n
t=l
+ 1)
sin(wrt/2)
sin(w/2)
(1 1.2.9)
'
Let w = 2/rrk/n. Because
sin(mn/2) - sin(ak)
rt,
sin(w/?)
sin(~k/kln) = (0,
k
k
=.
0,
0,
and cos o = 1 and sin w = 0 for w = 0,Equations (1 1.2.8) and (1 1.2.9) imply that
if,
0,
5 sin(?)
t=l
O,
k=0,
k # 0;
k = O, I , .
(11.2.10)
. . , [n/21.
(11.2.11)
240
Next, using the trigonometric identities

cos o cos h = ${cos(o
+ A) + cos(o - A)),
sin o sin A = i{cos(w - A ) - cos(o

sin w cos h = ${sio(o
+ A)),
+ A) + sin(@ - A)),
and from Equations (11.2,10) and (11.2.1I), we obtain
2vkt
c o s ( ) c o s ( ) = { 2 ,
t= 1
0,
5 sin(2:kf)
sin(?)
= {:;2,
r= 1
0,
k = j = 0 or 142 ( n even),
k=jiOnrii/2(i~ermJ,
k#j;
(112.13)
k = j = 0 or n/2 (n even),
k = j + 0 or n/2 ( n even),
k # j;
(11.2.14)
= 0 for ail k and j.

1-1
This result shows that the system (1 1.2.3) is a set of orthogonal functions.
From Equations (11.2.5) and (11.2.6), it is clear that the system of the trigonometric
functions in (1 1.2.3) can also be represented in complex form. This form is much simpler
and compact and is very useful in certain applications. In terms of this representation, the
corresponding system contains the following complex exponentjals:
II
+ 1 5 k 5 -if
2
2
er2?rh/n: - -
ti
(12 - I )
( n - 1)
is even and - -s k s 2
i f n is odd
which again consist of exactly n functions. To show the orthogonal property of these complex
exponentials, we note that
where we use that
11.3 Fourier Representation o f Finite Sequences
241
and (eamkln)"= eilmk =. 1. Applying Equation (11.2.17) we immediately obtain the

following result:
That is, the system (I 1.2.16) is orthogonal.
11.3 Fourier Representation of Finite Sequences
...
Let ZI, &,

,Zn be a sequence of n numbers. This sequence can be regarded as a set of coordinates of a point in an n-dimensional space. In vector analysis, we often construct a set of vectors called a basis so that every vector in the space can be written as a linear combination of the
elements of the basis. For a given n-dimensional space, it is known that any set of n orthogonal
vectors forms a basis. Thus, the given sequence of n numbers, (Z,}, can be written as a linear
combination of the orthogonal trigonometric functions given in the system (11.2.3). That is,
Equation (11.3.1) is called the Fourier series of the sequence Z, The akand bkare called Fourier
coefficients. Using the orthogonal property of the trigonometric functions given in (11.2.13) to
(1 1.2.15), we obtain the akand bkby multiplying cos(2mkt/n) and sin(27rkfln) on both sides of
f 11.3.1), respectively, and then summing over f = 1,2, . . . ,la. To avoid possible confusion, the
reader can replace the index k by j in Equation (11.3.1) in this operation. Thus, we have
ak =
eZ,
2 ez,
1
n ,=1
n
t=l
cos(F),
cos(~),
k = 0, and k = ,112 if tt is even,

=
2,.
..,
[%I,
(1 1,3,2)
Let ok= 2?rk/n, k = 0,1, . . . , [n/2]. These frequencies are called the Fourier frequencies.
Using the system of complex exponentials given in (11.2.16), we can also write the Fourier
series of Zt as
In-1)/2
2, =
k=- n-1)/2 ckeiUkr,

{k=8+;keiukf,
if n is odd,
8 n is even,
242
CHAPTER I1 Fourier Analysis
where the Fourier coefficients ckare given by
Equations (1 1.3.1) and (11.3.3) imply that

[n/2J
Z,
2(akcos wkt + bksin wkt)
k=O
qeiwkt,
if n is odd,
ckei"kt,
if 11 is even,
Thus, from the relationships (11.2.5) and (11.2.6), the Fourier coefficients ak,bk, and ck are
easily seen to be related as
co = %; cnp =
an/;!(even n),
ak - ibk
ck = -
'
The coefficient co = a0 = x : = l ~ , / nis often referred to as the d.c. value, i.e., the constant average value of the sequence.
The above material shows that any finite sequence can be written as a linear combination of the sine-cosine sequences or the complex exponentials.
1 1.4 Fourier Representation of Periodic Sequences

A general functionf ( t ) is said to be periodic with period P if there exists a positive constant
P such that
for all t. It is dbvious that a function that is periodic with period P is d s o periodic with
periods 2P, 3P, . . . . The smallest positive value of P for which (11.4.1) holds is called the
fundamental period or simply the period of the function.
11.4 Fourier Representation of Periodic Sequences
243
Suppose that the sequence (or the discrete time function) 2, is periodic with period
where n is a positive integer. That is,
11,
for all integer t. A fundamental phenomenon of a periodic function is that the function is
uniquely determined by its pattern in a range of one period. Outside that range the function
is just a repetition of the pattern in that range. Thus, a periodic sequence with period n is
,and n.
uniquely defined by its values at t = 1,2,
Following the result of the Fourier series of a finite sequence in Section 11.3, we can
write 2, for t -; 1,2, . . ,n as the linear combination of the orthogonal sine and cosine functions as follows:
.. .
["/21
Z, =
2 (akcos wkt f bk sin okt)
k=0
bk
CZ,sin wkt,
=IL t = l
,.,[?I
k = 1,2
with ok = 2sk/n. Similarly, we can also write Z, for t = 1,2,

tion of complex exponentials
. . . , n as the linear cornbina-
The Fourier coefficients ak and bk in (11.4.4) and (11.4.5) and ck in (11.4.7) are related by
the equations in (1 1.3.5).
It is easy to see that the sine-cosine functions sin o k t and cos wktin (11.4.3) and the
complex exponentials eiwkt in (11.4.6)are periodic with period n, which leads to the desired
consequence that
244
CHAPTER 1 1 Fourier Analysis
for all integers t and j. In other words, the Equations (11.4.3) and (11.4.6) are valid for all
integers t.
We have demonstrated how to represent an arbitrary periodic sequence of period n as a
linear combination of n trigonometric sequences or n complex exponentials. The smallest
positive value of n for which the Fourier series representation in (1 1.4.3) and (11.4,6) holds
is called the fundamental period, and the corresponding value 2 n / n is called the fundamental frequency. The terms for k = 1 and k = - 1 in the above representation both
have the same fundamental period equal to n (and hence the same fundamental frequency
2.rrlrt) and are collectively referred to as the fist harmonic components. More generally, the
terns for k = +j and k = - j both have the frequency j(2rfn) and are referred to as the jth
harmonic components. Thus, all the terms in the Fourier series representation have frequencies that are multiples of the same fundamental frequency, 2 4 ~ 1 and
, hence are harmonically related.
For a given periodic sequence Z, of period 11, the energy associated with the sequence in
one period is defined as
Now, multiplying by Z, on the both sides of (11.4.3), summing from t

using the relation (11.4.4) and (11.4.5), we have
%z: = [
11'
~ [ a k ~cos
z okt
l
k=O
[(n-1)/2]
1 1 ~ 8+ ntri
1 to t = n, and
+ bk$z1
sin mkt
l
1-
1=1
fag
+ bj),
if n is odd,
k=l
(11.4.9)
,t [(fi-1)/2]
+ -2
k=l
(az 4- b$)
+ no$*,
if n is.even.
Equation (11.4.9) is known as Parseval's relation for Fourier series. Equivalently, using
Equations (1 1.4.6) and (1 1.4.7), we have the corresponding form of Parseval's relation as
(n-1)/2
k=-(n- 1)/z
42
k=-n/2+1
where [ckI2= eke$.
IckI2,
ifnjsodd,
j ck12,
i f n is even,
11.4 Fourier Representation of Periodic Sequences
245
Equations (1 1.4.9) and (11.4.10) imply that the total energy for a periodic sequence over
the whole time horizon t = 0, 4- 1, 4-2, . . . is infinite. Hence, we consider its energy per unit
time, which is called the power of the sequence. This equation is given by
or, equivalently,
k=- n-1)/2
k=-n/2+1
Is p,
if n is odd,
(11,4.12)
I ckp,
if n is even.
As noted above, the jth harmonic components include the terms for both k =: +j and k =: - j
as they correspond to the same frequency j ( 2 4 n ) . Hence, we can interpret the quantity
from the term in the Fourier series of 2, at the kth frequency ok= 2.rrkln as the contribution
to the total power. The quantity fk plotted as a function of wk shown in Figure I f . I is called
the power spectrum and describes how the total power is distributed over the various hequency components of the sequence 2,.
EXAMPLE1l.I L e t Z l = 1 , Z 2 = 2 , & = 3 , a n d Z t + 3 j = Z , , t = 1 , 2 , 3 , j = f l , f 2 ,
+3,
. , . .Clearly, the function is periodic, with period 3 as shown in Figure 11.2.
Now, the period a

(11,4.3)becomes
3, which is odd, and fn/2] = I. The Fourier representation using
Z,
.
a
+ al cos (- + b sin ( 2,
;t)
FIGURE 1 1.1 Power spectrum.
246
FIGURE 1 1.2 Periodic sequence in Example 11.1.

where the Fourier coefficients are obtained through (11.4.4)as
That is,
Similarly, in terms off11.4.6),we can also represent Z, as
where the coefficients are calculated using (1 1.4.7):
That is,
+ + $(I + .57735'03i)e~~~~/~,
t = 1,2,3,4, . .
Z, = i(1 - .5773503i)e-~~'"/~ 2
The coefficients ak, bk,and ck are clearly related as shown in (1 1.3.5).
11.5
Fourier Representation of Nonperiodic Sequences
247
Ff GURE 11.3 Power spectrum of Example 1 1 . 1 .
Using (1 1.4.13),the power spectrum of the sequence is given by
(-.577350312] = $,
k = 0,
k = 1,
otherwise.
The plot is given in Figure 11.3.
1 1.5 Fourier Representation of Nonperiodic Sequences:

The Discrete-Time Fourier Transform
Consider a general nonperiodic sequence or discrete-time function, Z,, of finite duration such
that Z, = 0 for I t / > M. Let n = (2M 1) and define a new function
which is clearly periodic with period n. Thus, using (1 1.4.6), we can express (1 1.5,l)in
a Fourier series of the form
248
In the interval -(rt - 1)/2 5 t 5 (ti - 1112, however, Yr= Zt. Therefore,
where we have used that Z, = 0 for t < -(n - 1)/2 or t > (n - 1)/2. Let
We have
where Aw = 2 ~ / nis the sample spacing of frequencies, Combining (11.5.2) and

(11.5.6) yields
Because Ao = 2.ir/n, (11.5.7)can be written as
Now each term in the summation of (11.5.8) represents the area of a rectangle of height
f(kAw)eikA"' and width Ao.As n * ~ , we have I;-+ 2, and A o * 0. Thus, the limit of the
summation becomes an integral. Furthermore, because the summation is carried out over n
consecutive values of width Ao = 2 4 n , the total interval of integration will always have a
width of 2 ~Therefore,
.
11.5 Fourier Representation of Nonperiodic Sequences
249
~ e c a u sf(w)eiwt,
e
as a function of w , is periodic with period 27~,the interval of integration
can be taken as any interval of length 27r. Specifically, we can consider -71- 5 w 5 71-. Thus,
we have the following:
and
The function f(w) in (11.5.11) is usually referred to as the (discrete time) Fourier transform
of Z,, and 2, in (1 1.5.10) is often referred to as the (discrete time) inverse Fourier transform
of flu). They form a Fourier transform pair.
From the above discussion, we see that we can also define the Fourier transfomf(w) as
instead of the given (11.5.5). Tbese modified definitions offlu) lead to the following Fourier
transform pairs:
and
respectively, which may be found in other books. in this book, we use the Fourier transform
pairs as given in (11.5,10) and (11.5.11).
250

The derivation of the integral in (11.5.10) as the limiting form of a sum implies that the
inverse Fourier transform given in (1 1.5.10) represents the sequence 2, as a linear combination of complex sinusoids infinitesimally close in frequency with amplitudes 1f(o)l(do),
Thus, the quantity I f(o)l is often referred to as the spectrum or amplitude spectrum of the
sequence, as it provides us with the information on how Z, is composed of complex sinusoids at different frequencies,
The energy associated with the sequence is given by the following Parseval's relation:
2 ~ , e - ' ~do'
f (o)
-
2~ J-f *(w)f(w) d o
7s
= Z1.L
If(@) 1' do.
(11.5,16)
Parseval's relation thus relates energy in the time domain to energy in the frequency domain.
In other words, the energy may be determined either by computing the energy per unit time
IZ,~~
and summing over aU time or by computing the energy per unit fkequency 27rlf(o)[* and
integrating over all frequencies. Hence, g(w) = 2?rlf(w)12, as a function of o , is also referred
to as the energy spectrum or the energy spectral density function.
In the above construction, Z, was assumed to be of arbitrary finite duration. Equations
(1 1.5.10) and (1 1.5.11) remain valid for a general nonperiodic sequence of infinite duration.
In this case, however, we must consider the question of convergence of the infinite summation in (11.5.11). A condition on Z, which guarantees the convergence of this sum is that the
sequence {Z,) is absolutely summable. That is,
In fact, the above theory holds also when 2, is square summable, i.e.,
The proof of the result using a weak condition depends on an alternative presentation of the
Fourier transform and is omitted. For our purposes, it suffices to note that the condition in
Equation (1 1.5.17) implies the condition in Equation (1 1.5.18), but the converse is not true.
11.5 Fourier Representation of Nonperiodic Sequences
251
Important differences. between the frequency domain properties of periodic and nonperiodic sequences exist:
1. The spectrum'frequenciesof periodic sequences are harmonically related and form a finite
discrete set, whereas those of nonperiodic sequences form a continuum of frequencies.
2. The energy over whole time horizon t = 0, f1, f2, . . . for periodic sequences is infinite.
Hence, we study their properties in terms of the power spectrum over a finite set of harmonically related frequencies. The resulting spectra are hence sometimes referred to as
line spectra. The energy over the whole time horizon for nonperiodic sequences is finite
as guaranteed by the condition in Equation (11.5.17). Hence, we describe their properties
in terms of the energy spectrum over a continuum of frequencies.
EXAMPLE 11.2 Consider the sequence
which is graphed in Figure 11.4.

Now
Hence, we can write
FIGURE 11.4 Nonperiodic sequence in Example 11.2,
252
CHAPTER 7 1 Fourier Analysis
Frequency (0 to T)
FIGURE 11.5 The spectrum of the sequence in Example 11.2.
The function f(o)is graphed in Figure 11.5. It shows that the spectrum is a positive and
symmetric function dominated by low frequencies.
EXAMPLE 11.3 Let the sequence be
as shown in Figure 11.6.
FIGURE 11.6 Symmetric oscillating sequences in Example 11-3.
11.5 Fourier Representation of Non~eriodicSequences
253
Following the same arguments as in Example 11.2, we obtain
3
2 ~ ( 5 4 cos o)'
-5-
I0 5 T.
The spectrumf(o) as shown in Figure 11.7 is again positive and symmetric. Now however, it
is dominated by high frequencies. This phenomenon is typical for a sequence that tends to
oscillate rapidly.
EXAMPLE l f .4 In this example, we find the corresponding sequence Z, for the spectrum
Am) =
1 + cos w
27r
'
- - ' T T ~ O S T
For a given spectrum, we can calculate the corresponding sequence by using inverse
Fourier transform given in (11.5.10).Thus,
-/
1
2w
"
-r
(1
+ cos w)eimr dw
Frequency (0 to ?r)
FIGURE 11.7 The spectrum of the oscillating sequence in Example 11.3.
254
CHAPTER 1I
Fourier Analysis
( 0,
otherwise.
1 1.6 Fourier Remesentation of Continuous-Time Functions

1 1.6.1 Fourier Representation of Periodic Functions
We now consider a function Kt) that is defined for all real values t. First, we assume thatfit)
is periodic with the fundamental period P, i.e,,
f ( t ) = f (t
+ j P ) for all real values t and integers j.
(1 1.6.1)
This periodic function is completely characterized by its behavior in any interval of length P.
Thus, we only need to consider the function for t in the interval [-P/2,P / 2 ] . Now recall that
the fundamental period P and the fundamental frequency wo are related as follows:
For this fundamental frequency, consider the set of harmonically related complex exponentials
Clearly,
Hence, the complex exponentials in (11.6.3) are periodic with the same period P = 2 4 0 0
as the functionflt). Also,
11.6 Fourier Representation of Continuous-TimeFunctions
Equation (11.6.5) follows because if k
= j,
255
then the integral becomes
if k =#j, then the integral is
Thus, the system in (11.6.3) is a periodic system with period P that is orthogonal on any
interval of width P. Hence, we might represent the functionAt) as the following Fourier series:
If the representation is valid, i.e., if the series in (11.6.6) converges uniformly to At), then
term-by-term integration is permissible and the Fourier coefficient can be easily shown to be
For the representation in (11.6.6) to be valid, we need to show that the series uniformly
converges to the functionAt). There are a number of necessary and sufficient conditions for
the convergence of the series. For our purposes, it suffices to note the following sufficient
conditions proposed by Dirichlet.
Dirichlet Conditions for Periodic Functions IfAt) is a bounded periodic function

with period P and has at most a finite number of maxima and minima in one period and a
finite number of discontinuities, then the Fourier series converges toflf) at every continuity
point of flt) and converges to the average of the right-hand and left-hand limits ofAt) at
each point whereflt) is discontinuous,
The Parseval's relation for (1 1.6.6)is easily seen to be
256
C W T E R 11 Fourier Analysis
11.6.2 Fourier Representation of Nonperiodic Functions:

The Continuous-Time Fourier Transform
For a general nonperiodic functionf(f), which is defined on all real value t but of finite duration, i.e.,f(t) = 0 for It\ > P/2, we define a new function that is identical to f(t) in the interval
[-P/2, P/2], but is periodic outside this interval. That is,
Clearly, g(t) is periodic with period P. Thus, we can write
and
where oo = 2nlP.Now g(t) = f(t) in the interval [ - P / 2 , P/2]. Hence, in this interval,
we have
and
11-6 Fourier Representation of Continuous-Time Functions
257
where
Now each term in the summation of (11.6.12) represents the area of a rectangle of height
H(kwo) and width wo. As P +W, g(t) -+f(t), oo +0, and the limit of the summation
becomes an integral, Thus,
and
where we note that in the limit the quantity kwo becomes a continuous variable, which we
denote by w. That is,
and
The function F ( w ) in (1 1.6.14) is called the Fourier transform or Fourier integral of f(t),
and f(r) in (11.6.13) is called the inverse Fourier transform or inverse Fourier integral
258
of F ( o ) .They form a Fourier transform (integral) pak Similarly, by the same arguments as
in Section 11.5, the following alternative forms of the Fourier transform pairs,
and
may also be found in other books.

The Parseval's relation for the Fourier transform pair in (1 1.6.13) and (11.6,14)can be
easily shown to be
In the above construction,f(#)is assumed to be of finite duration. Although the results are
valid for some functions of infinite duration, not all functions of infinite durations have a
Fourier transform. The following Dirichlet conditions provide a useful set of sufficient conditions of the existence of T ( w ) ,
Dirichlet Conditions for Nanperiodic Functions
lz 1
1, f (t)is absolutely integrable, i.e.,

f ($1df < m.
2, f(t) has only a finite number of maxima and minima and a finite number of discontinuities
in any finite interval.
1 1.7 The Fast Fourier Transform

Because, in actual time series analysis, we are given a sequence of 11 values Z1, Zz, . . , Z,, we
use (1 1.3.4) to calculate the following Fourier coefficients, also known as Fourier transforms:
1 1.7 The Fast Fourier Transform
259
where we assume that fa is even and recall that wk = 2mkl1~.Rewriting (1 1.7.1) in the form
we see that for each k, the computation of ck requires approximately n complex multiplications and additions if we ignore that for some values o f t and k, e-'"tf is equal to 44 and does
not, strictly speaking, require a complex multiplication. Hence, the direct computation of the
whole set {q : k = -n/2
1, -n/2 2,
,0, . , ,112) using (11.7.1) requires approximately 1z2 complex multiplications and additions. When JZ is large, the required number of
operations is clearly substantial. Additionally, there is a corresponding storage problem. For
example, if n =; 450, then the direct computation requires 202,500 complex multiplications
and additions and storage locations. For large values of n, this computation may become formidable if not impossible. Hence, an efficient algorithm, known as the fast Fourier transform
0,
was developed to compute these Fourier transforms. Major contributions to the
method include Good (1958) and Cooley and Tukey (1965).
To illustrate the algorithm, we consider an even value of n, and we let X,= Z2' represent
the even-indexed values of {Z,} and Y, = 22t-1represent the odd-indexed values of (2,).
Furthermore, to simplify the notations we let
+ ... . .
Then, the n-point Fourier transforms in (11.7.1) become
where
260
Note that fk and gk in (11.7.5) and (11.7.6) are clearly the (n/2)-point Fourier transforms of
X,and Y,, respectively. Hence, Equation (11.7.4) implies that the n-point Fourier transforms
can be calculated in terms of the linear combination of the simpler Fourier transforms of
length n/2. Because
we have
Thus, in Equations (11.7.5) and (11.7.6), we only need to calculate fk and gk for
k = 1, 2, . . . , r1/2. The direct computation of these fi and g k through (1 1.7.5) and (1 1.7.6)
requires approximately 2(n/212 complex multiplications and additions. Then the linear combination of these two (n/2)-point Fourier transforms through (11.7.4) requires rt complex
multiplications corresponding to multiplyin gk by A? and also n complex additions corresponding to adding the product ;,4ikgk
to fk. Consequently, the total number of complex
multiplications and additions required in the computation of Equation (11.7.4) is
If n/2 is also even, then by repeating the same process, each of the (nJ2)-point Fourier
transforms fk and gk in (11.7.5) and (11.7.6) can be computed via two (ti/4)-point Fourier
transforms. To find the number of required operations to compute each of the (n/2)-point
transforms via the (11/4)-point transforms, we simply follow the same logic as described in
the previous paragraph and note that it is equal to n/2 2(n/4)2. Hence, the required total
number of operations for the overall computation leading to ck's is
If n = 2'with r being a prime number, then we can continue this process for r iterations and end with a set of 2-point Fourier transforms. The total number of required
complex multiplications and additions for the overall computation is seen to be approximately equal to
Exercises
261
Clearly, for a large value of 11, the reduction in the required operations from n2 to n fog2ti is
very substantial. For example, if n = 2048 = 211,then the direct computation using (1 1.7.1)
requires approximately 4,194,304 operations, whereas the fast Fourier transform requires only
22,528 operations. The reduction-ismore than 99%. Furthermore, because h2 --- e-i2n/2 = -1,
the 2-point Fourier transforms in the final iteration are actually obtained only by simple
addition and subtraction.
In essence, the FEPr is an iterative algorithm that uses the properties of trigonometric
functions and complex exponentials in reducing the computation of n-point Fourier transforms to simpler transforms. In this section, we illustrated the algorithm when n = 2'. In
practice, the actual length of a sequence is, of course, rarely of the form 2'. One can, however, always add zeros to the sequence (known as padding) until it reaches the required
form. It is clear that for a given frequency wk these appended zeros will not affect the
nurneticaf values'of the Fourier transforms. After we add zeros to increase the length of a
sequence from n to rn (say), however, then the transforms will be calculated at the frequencies wk = 2ak/m rather than at o k = 2mk/n. To attenuate potential undesirable effects,
some authors have recommended that the unpadded portion of a modified sequence be
"tapered" to smooth the transition from nonzero to zero values (see, e.g., Tukey [1967j,
Godfrey [1974], and Brillinger [1975]).
AIthough n is not necessarily of the f o m n = 2', we know Erorn number theory that an
integer can always be written as the product of prime numbers, i.e., we can always write
. .
where rl, r2, , , rm are prime numbers not necessarily equal to 2. It is clear that the FTT
for this form of n can be similarly developed. Because most available computer software
for fast Fourier transforms is based on the form n = 2' introduced in this section, we will
not go into the method with other forms of n. We leave as an exercise the simple case when
n is the product of two integers, say n = nln2. For other cases, we refer readers to Bloomfield (2000).
11.1 Prove the identity
11.2 Consider the following finite sequence:
Find the Fourier representation of the sequence.
262
11.3 Find the Fourier representation for each of the following sequences:
if 1 5 t 5 m,
(a) Z r =
i f m < t r n,
m 2).
(b) Z, = tt - t , for 1 5 t 5 11.
{i:
Xke x p ( - i ( 2 n k t ) l p ) . Show that Zt is periodicwith periodp.
11.4 Consider 2, =
11.5 Let Z1 = 4, Z2 = 3, Z3 = 2, Z4 = I , and Zt+4j= Z,, for t = 1,2, 3 , 4 and j = 3 1,

f2,.. . .
(a) Find the Fourier representation of Z,.
(b) Find the Fourier representation of 2, in terms of complex exponentials.
(c) Find and plot the power spectrum of 2,.
11,6 Consider the following sequence:
(a) Is the sequence absolutely summable?

(b) Find the Fourier transform of 2,.
(c) Plot and discuss the spectrum of the sequence.
11.7 Find the corresponding sequence Z, for each of the following spectra:
1 - cos 0 cos 2w
(a)
f (0.3)
(c)
f ( 4=
297
+ cos 40
2rr
'
-?r TT
-rr 5
(O
0.3 5
m.
a.
118 For absolutely integrable functions g ( x ) and h(x) defined on the real line, the function
is called the convolution of gfx) and h(x). Let X, and Yt be absolutely summable
sequences, and define 2, = X,Y,. Show that the spectrum of 2, is the convolution of
the spectrums for Xi and Y,.
11.9 Find the Fourier representation for each of the foIfawing functions:
(a) f ( t ) = 1, -?r 5 t 5 IT, andf ( t ) = 0 otherwise.
(b) f ( t ) = cos +t, where #is not an integer,
(c) f(t) '= sin 4t, where 4 is not an integer.
11.10 Show the Parseval's relation given in Equation (11.6.17).
Exercises
263
11.11 A generalized function S ( t ) represents a sequence of functions 6,(t) for ti = 1, 2, . . .

such that
provided that f(t) is continuous at t = to. The function S(t) is also called Dirac's
delta function.
(a) Show that the sequence s,(t) = ( ~ l / a )efnf2,
' / ~ -m < t < m, yields a Dirac's
delta function.
(b) Find the Fourier transfom pair of a delta function.
11.12 Discuss the fast Fourier transform when the number of observations n is the
product of two integers 111 and 112, i.e., ri = nl n2.
Spectral Theory of
Stationary Processes
With the proper background of the Fourier transform discussed in Chapter 11, we are
now ready to study the frequency domain approach of time series analysis. In fact, the
spectrum of a stationary process is simply the Fourier transform of the absolutely summable autocovariance function of the process. More generally, a stationary process can
always be represented by a spectral distribution function. After developing the general
spectral theory, we examine the spectrum of some common processes, such as the spectrum of ARMA models. Related questions of the effect of Iinear filters and aliasing are
also addressed.
12.1 The Spectrum

12.1.1 The Spectrum and Its Properties
Let 2, be a real-valued stationary process with absolutely summable autocovariance
sequence yk. Then, from the results in Section 11.5, the Fourier transform of yk exists and by
Equation (11.5.11) equals
where we have used the properties that yk = Y-R, sin 0 = 0, sin a ( - k ) = - sin ok, and
cos w(-k) = cos ok. The sequence yk can be recovered from f(o) through the inverse
Fourier transform
264
The function f(o) has the following important properties.
1, f ( o ) is a continuous real-valued nonnegative function, i.e., f(o)/= f(wf. Thus, from

the discussion following Equation (11.5.15) in Section 11.5, f(w) is also referred
to as the spectrum of the autocovariance sequence yk or the corresponding stationary process Z,. It is obvious that f ( o ) is a continuous real-valued function. To
show it is nonnegative, we note that yk, as an autocovariance function, is positive
semidefinite, i.e.,
for any real numbers ci, cj and any integers kj and kj. In particular, let ci
cj = cos wj, ki = i, and kj = j. We have
n
C CY(i-j)
cos mi cos oj r O.
i=l
j=]
Likewise,
n
2 j=l2y(i-j) sin wi sin w j r 0.

Thus,
n
y(i-j) [cos oi cos oj

i-1 j-1
+ sin wi sin oj]
2 j=1
cY(i-j)
cos w(i - j)
n
.. -
k=-(n-1)
where k = (i - j). Therefore,
(n
Ikl)yk cos ok
cos oi,
266
CHAPTER 1 2 Spectral Theory of Stationary Processes
Now, that ys; is absolutely sumable implies that for any given E
N > 0 such that
> 0, we can choose an
Thus, for n > N, we have
Clearly, for any fixed N,
Because E is arbitrary, it follows that
Hence,
because yk cos wkl5l ykl.Therefore, by (12.1.7) and (12.1.Q
1
f (o) = 2?r k
2= 4 yk cos ok
CQ
2. f ( o ) = f(o f 2v) and hence f ( o ) is periodic with period 27r. Furthermore, because
f ( o ) = f(-o), i.e., f ( o ) is a symmetric even function, its graph is normally presented
only for 0 5 w 5 rr.
267
12.1 The Spectrum
TI
Frequency (0 to rr)
FIGURE 12.1 An example of a spectrum.

3. From (12.1.4),we have
TI
-77
which shows that the spectrum f(w) may be interpreted as the decomposition of the variance of a process. The term f(o)dw is the contribution to the variance attributable to the
component of the process with frequencies in the interval (o,
o
do). A peak in the
spectrum indicates an important contribution to the variance from the components at
frequencies in the corresponding interval. For example, Figure 12.1 shows that the components near frequency ooare of greatest importance and the high-frequency components near rr are of Little importance.
4. Equations (12.1.1) and (12.1.4) imply that the spectrum f(w) and the autocovariance
sequence yk form a Fourier transform pair, with one being uniquely determined from the
other. Hence, the time domain approach and the frequency domain approach are theoretically equivalent. The reason to consider both approaches is that there are some occasions when one approach is preferable to the other for presentation or interpretation.
12.1.2 The Spectral Representation of Autocovariance Functions:

The Spectral Distribution Function
Note that the spectral representation of yk given in (12.1.1) and (12.1.4)holds only for an
absolutely summable autocovariance function. More generally, for a given autocovariance
function yk, we can always have its specttal representation in terms of the following
Fourier-Stieltjes integral:
yk = j:aimk
dF(o),
(12.1.11)
268
where F ( o ) is known as the spectral distribution function. Equation (12.1.11) is usually

referred to as the spectral representation of jhe autocovariance function yk.Like any statistical distribution function, the spectral distribution function is a nondecreasing function that
can be partitioned into three components: (1) a step function consisting of a countable number of finite jumps, (2) an absolutely continuous function, and (3) a "singular" function. The
third component is insignificant and is ignored in most applications. Thus, we may write the
spectral distribution function as
where Fs(u)
is the step function and F,(o) is the absolutely continuous component. Equation
(12.1.4) shows that for a process with absolutely summable autocovariance function, F(w) =
Fc(w) and dF(w) = f(o)d o .
To illustrate a step spectral distribution function, consider the general linear cyclical
model
where the Ai are constants and the Oi are independent uniformly distributed random
variables on the interval t-T, T ] .The oi are distinct frequencies contained in the interval
[-T, T ] .Then
and
where we note that
12.1 The Spectrum
269
Clearly, the autocovariance function yk in (12.1.15) is not absolutely summable. Hence,

we cannot represent it in the form of (12.1.4). We can, however, always represent it in the
form of (12.1.11). To do that we make the following important observations:
1. 2, is the sum of M independent components.
2. wi can be anywhere in the interval [-'T? TI, and yk in (12.1.11) is also the autocovariance function for the process given in (12.1.13) at frequencies -wi.
and hence the variance A ; F / ~of the ith component is its contribution to the total variance
of 2, at both frequencies -mi and oj.In other words, one half of the variance of the ith
component is associated with the frequency -wi, whereas the other half is associated
with the frequency wi. Thus, Equation (12.1.15) can be rewritten as
where
or, equivalently,
where F(w) is a monotonically nondecreasing step function with steps of size Af/4 at
w = fw i# 0 and a step of size A?/2 for o = wi = 0 for i = 1, , . , ,M,
Figure 12.2 illustrates the spectral distribution function for a process with M = 2, Al = 1,
A2 = 2, and nonzero frequencies oland 02.
The corresponding spectrum representing the variance contributions at different frequencies is a set of discrete lines and is shown in Figure 12.3.
Because (12.1.17) is clearIy periodic, this result is certainly expected. As shown in
Section 11.5, the spectrum for a periodic sequence is a discrete or line spectrum.
Although the spectral distribution function F(w) is a nonnegative and nondecreasing
function, it does not quite have the properties of a probability distribution function because
dF(w) = yo, which is not necessarily equal to 1. If we define
270
CHAPTER 1 2 Spectral Theory of Stationary Processes
FIGUFE 1 2 2 Spectral distribution function for

2
Z, = 2 i = I A jsin(@&+ O i ) ,Al = 1, Az = 2.
FIGUY 12.3 Line spectrum for

Z, = Zi=*Ai sin(wit + Of),
A1 = 1, Az
however, then G.(o) r 0 and
2.
Jz
d G ( w ) = 1. When dF(o) =f (o)

do,we have
p ( o ) do =
dC(o)="f do.
Yo
(12.1.20)
12.1 The Spectrum
271
Hence, fkom (12.1.1) and (12,1.4),we get the following corresponding Fourier transform pair:
and
The function p(w) has the properties of a probability density function over the range [ - T , T]
and is often referred to as the spectral density function, although the same term has also been
used to refer to the spectrumf(o)in the literature.
12.1.3 Wold's Decomposition of a Stationary Process

Note that the process 2, in (12.1.13) is clearly stationary because E(Z,) = 0 and
E(Z,Z,+k) = 4 ~ ~ ~ I ~o i fk c= o7 s~ There
.
is, however, a very important difference
between this process and the processes discussed in Chapters 2 through 10. The iinear cyclical model given in (12.1.13) is deterministic in the sense that it can be predicted without
error from its past record. A process such as the ARIMA model that does not have this praperty is termed nondeterministic. Obviously, a process may contain both deterministic and
nondeterministic components. For example, the model
where A is a constant, A is a uniformly distributed random variable in I-v, 4,and [B(B)/+(B)]

a, is a stationary and invertible A W A model, contains both deterministic and nondeterministic components. In fact, Wold (1938) showed that any covariance stationary process can
be represented as
where
is a purely deterministic component and ~ j " is
) a purely nondeterministic component. This representation is known as Wold's decompositionof a stationary process. This decomposition is similar to the decomposition of spectral distribution function given in (12.1.12)with
2fd)having a step spectral distribution function and zIn)having an absolutely continuous spectral distribution function.
272
12.1.4 The Spectral Representation of Stationary Processes

Using exactly the same argument that an autocovariance sequence yk can always be represented in the form given in (12.1.1 I), a given rime series realization Zt can be written as the
folIowing FourierStieltjes integral:
where U(o) plays a similar role to F(o)in Equation (12.1.11) for a single realization,
Because every realization can be represented in this form, however, we certainly should
not expect the function U(o) to be the same for all realizations. Instead, the function
U(u) should, in general, vary from realization to realization. In other words, corresponding to each realization of the process Z, there will be a realization of U(o). If
(12.1.27) is used to represent the process Zt of all possible realizations, then U ( o ) is
itself a complex valued stochastic process for each o.The integral on the right-hand
side of (12.1.27) thus becomes a stochastic integral, and the equality is defined in the
mean-square sense, i.e.,
The relation (12.1.27) is called the spectral representation of the stationary process 2,.
To study the properties of U(u), we assume, with no loss of generality, that Z, is a zeromean stationary process. Hence,
E [ d U ( w ) ] = 0,
for all o.
(12.1.29)
Next, we note that for a zero-mean complex valued stationary process Z,, the autocovariance
function yk is defined as
where Z;+k denotes the complex conjugate of Z,+k. This general definition is necessary
so that the variance of Zr will be real. Clearly, when Z, is a real valued process, we have
Z;+k = Zt+k and hence yk = E(Z,Z,+k). Taking the complex conjugate of both sides of
(12.1.27), we obtain
Hence, Zt is real-valued if and only if for all o
12.1 The Spectrum
273
Now, consider
which should be independent of t as 2, is assumed to be stationary. This result implies that

the contribution to the double integral in (12.1.33)must be zero when o A, i.e.,
E [ d U ( o )dU"(A)] = 0,
for all w # A.
(12.1.34)
In other words, U ( o )is an orthogonal process. Letting h = w in (12.1.33),we get
Ti-
= Le"dF(w),
where
dF(w) = ~ [ i d U ( w ) l=~ E] d ~ f w ) d ~ * ( w ) ,for -rr
5 w 5 .rr,
(12.1.36)
and F(o)is the spectral distribution function of 2,. If 2, is purely nondeterministic so that
F(o) = F,(o), then we have
f ( ~ ) d w= dF(w) = ~ [ [ d ~ ( w ) / * ] , for -T
(12.1.37)
o 5 T,
and
Ti-
yk
JTe "kf(w)d o .
The above result shows that a stationary process can always be represented as the Iimit of
the sum of complex exponentials or, equivalently, sine and cosine functions with uncorrelated
random coefficients. This representation is also known as Cramer's representation. The derivation leading from (12.1.27)to (12.1.38)provides a mare explicit physical interpretation of the
spectrum discussed earlier in (12.1.10).That is, f(w) d o is the contribution to the variance of 2,
attributable to the component Z,(w) of 2,having frequencies in the range (w, w do).
274
12.2 The Spectrum of Some Common Processes

12.2.1 The Spectrum and the Autocovariance Generating Function
Recall from Section 2.6 that, for a given sequence of autocovariances yk, k = 0, i l , 3 2 ,
the autocovariance generating function is defined as
. ,. ,
where the variance of the process yo is the coefficient of BO = 1 and the autocovariance of
lag k, yk, is the coefficient of both B~ and B-k. Now, we know that if the given autocovariance
sequence yk is absolutely summable, then the specbxm or the spectral density exists and equals
1
2rT k=,
f (o)= -
C yke-lUk.
Comparing Equations (12.2.1) and (12.2,2), we note that for a process with an absolutely
summable autocovariance sequence, its spectrum and autocovariance generating function
are related as follows:
1
f (o)= -y(e-io).
2a
This result is very useful in deriving the spectrum for many commonly used processes.
12.2.2 The Spectrum of ARMA Models

Any nondeterministic stationary process Z, can be written as
zFo
$ j ~ j ,t,bo = 1 . Without loss of generality, we assume that E(Z,) = 0.

where $ ( B ) =
From (2.6.9), the autocovariance generating function of this linear process is given by
Now, a general stationary ARMA(p, q) model
with +JB) = (1. - I$~B- . . - &B*) and 04(B) = (1 - OIB common factors can be written as
..-
- BqB4) sharing no
275
with $(B) = 8q(B)/4p(B).Hence, the autocovariance generating function of the ARMA(p, q)

model becomes
When the model is stationary, the roots of 4p(B) = 0 Lie outside of the unit circle, guaranteeing the absolute sumability of the autocovariance function. Consequently, the spectrum of
a stationary ARMA(p, q) model is given by
which is also known as a rational spectrum.

If the model is invertible, i.e., the roots of flq(B) = 0 lie outside of the unit circle,
0,(e-iW)9,(eio) will not vanish. Hence, the inverse of f(o) exists and is given by
which can be easily seen to be the spectrum of an ARMA(q, p) process. Applying the inverse
Fourier transform offl(w), we get
which is known as the inverse autocovariance function. Naturally,
is called the inverse autocorrelatio~function, a concept due to Cleveland (1972). As discussed in Section 6.3, the inverse autoconelation function can be used as an identification
tool that has similar characteristics as the partial autocorrelation function. Equations (12.2.9)
and (12.2.1 1) provide an alternative method to calculate the sample inverse autoconelation
function,
276
The Spectrum of a White Noise Process A white noise process
is a series of uncorrelated random variables with autocovariance function
and autocovariance generating function ~ ( 3=) a:. Hence, by (12.1.l) or equivalently by

(12.2.31, its spectrum is
which is a horizontal straight line as shown in Figure 12.4. This result implies that the contrjbutions to the variance at all frequencies are equal. It is well known that white light is produced
when the light spectrum at aU frequencies is the same. In fact, the white noise process is named
because of its similarity to white light in terms of the spectrum. In model building, the diagnostic checking for white noise can be performed by examining whether the spectrum of residuals
is relatively flat.
The Spectrum of an AR(I) Process The autocovariance generating function of a

stationary AR(1) process, (1 - +B)Z, = a,, is given by
7r
Frequency (0 to ?r)
FIGURE 12.4 The spectrum of a white noise process.
277
97
Frequency (0 to .rr)
Frequency (0 to q)
FIGURE 12.5 Spectrum for the ARCl) process.
Thus, by (12.2.8a) the spectrum is
The shape of the spectrum depends on the sign of 4. As shown in Figure 12.5, when # > 0
and hence the series is positively autoconelated, the spectrum is dominated by low-frequency
(long-period) components. If t$ < 0,then the series is negatively autocorrelated and the spectrum
is dominated by high-frequency (short-period) components. In other words, a low-frequency
dominating spectrum indicates a relatively smooth series, whereas a high-frequency dominating
spectrum implies a more ragged series. We remind the reader that these phenomena were, in fact,
iuustrated earlier in Examples 11.2 and 11.3.
As t$ approaches I, the ARfl) process in the limit becomes a random walk model; hence,
c u
f(&) =
G(1- cos
0)
can be taken to be the spectrum of a random walk model, although, strictly speaking, a random walk does not have a spectrum because its autocovariance sequence is not absolutely
summabfe. The function (12.2.17) however, is a well-behaved nonnegative even function
other than a possible problem at o =: 0 where the peak of the function becomes infinite in
height. The implication of this phenomenon in data analysis is that a sample spectrum with
strong peak near zero frequencies indicates a possible need for differencing.
The Spectrum of an MA(1) Process The autocovariance generating function of a

moving average process of order 1, Z, = (1 - OB)a,, is
278
and its spectrum is
Again, the shape of f(o) depends on the sign of 8 .

Recall that
( 0 9
otherwise.
Thus, when 0 is positive, the series is negatively autocorrelated and hence relatively ragged,
which is represented by a spectrum with higher values at the higher frequencies. On the
other hand, when 6 is negative, the series is positively autocorrelated and hence relatively
smooth. The corresponding phenomenon in terns of the spectrum is its dominating values at
the lower frequencies. Both cases are shown in Figure 12.6. As expected, the autocorrelation
function in the time domain and the spectrum in the frequency domain contain exactly the
same information about the process.
12.2.3 The Spectrum of the Sum of Two Independent Processes

Consider the process Z,, which is the s&n of two independent stationary processes X,and Y,,
Frequency (0 to ?r)
FIGURE 12.6 Spectrum for the MA(1) process.
Frequency (0 to ?r)
1 2.2 The Spectrum of Some Common Processes
279
Let yZ(B),yx(B), and yy(B) be the autocovariance generating functions for the Z,, X,, and Y,
processes. Then,
Hence, it follows that
That is, the spectrum of the sum of two independent processes is the sum of the two independent spectra,
12.2.4 The Spectrum of Seasonal Models

From the previous results in Section 11.4 and Section 12.1.2, we know that a deterministic
seasonal component with seasonal periods will have a discrete (line) spectrum at the harmonic
Fourier) frequencies ok= 27~k/s,k = 4Z1, 4Z2, , . , *[s/Z]. For example, the model
where the a, represent a white noise error process, can be used to fit a monthly series with a
cyclical trend. The spectrum of 2, in (12.2.23), from the result in Section 12.2.3, will be the
sum of two independent spectra. One represents the deterministic component in (12.2.23),
and the other represents the error process a,. Because the spectrum of the white noise emor
process is a horizontal flat line, the resulting spectrum will have spikes at the harmonic seasonal frequencies ok= 27412 for k = fl , f2, . ,f6 ,
Now consider the following seasonal ARMA model with seasonal period 12:
..
(1 - @B'*)z,
= a,.
(12.2.24)
280

The autocovariance generating function is
If the process is stationary, then its spectrum exists and equals
Note that cos(l2o) = - 1 at o = v(2k - 11/12 for k = 1,2,3,4, $ 6 , and cos(l2o) = 1for
w = 0 and w = 2mk/12, k = 1,2, 3,4,5,6. Thus, for Q, > 0, other than a peak at o = 0, the
spectrum will also exhibit peaks at seasonal harmonic frequencies 2~rk/12fork 1,2,3,4,5,
and 6 and troughs at frequencies w = ~ ( 2 k- 1)/12 for k = 1, 2, 3,4,5, and 6 as shown in
Rgure 12.7. The spectrum for Qt < 0 can be derived similarly,
Consider a multiplicative seasonal ARMA model having both seasonal and nonseasonal
ARMA components,
(1 - @ B ' ~(1
) - +B)Z, = a,.
(12.2.27)
From (12.2.16) and (12.2.26), the spectrum can easily be seen to be
f ( ~=
)
?;;(1 + rp2 - 2 0 cos 120) (1 4- f#2 - 24 cos 0 ) .
The general shape of the spectrum for @ > O and 4 > 0 is shown in Figure 12.8.
Frequency (0 to ?r)
FIGURE 12.7 Spectrum for (1
- (PB'2]~, = at.
(12.2.28)
281
Frequency (0 to rr)
FIGURE 12.8 Spectrum for (1
@Bl2)[1
+BIZ, = a,.
The sample spectrum discussed in the next chapter does not behave as regularly as those
in Figures 12.7 or 12.8. If the data indeed contain a seasonal phenomenon, though, then the
spectrum should exhibit peaks or bumps at the harmonicafly related seasonal frequencies.
Again, we remind the reader about the difference between the spectra for the Equations
(12.2.23) and (12.2.24). Although both are purely seasonal models, the cyclical trend term
in (12.2.23) is deterministic and hence has a discrete (line) spectrum, whereas the seasonal
phenomenon in (12.2.24) is nondeterministic with a continuous spectrum. This difference
implies that, in data analysis, a sharp spike in the sample spectrum may indicate a possible
deterministic cyclical component, whereas broad peaks, unless due to the smoothing effect
to be discussed in the next chapter, often imply a nondeterministic seasonal component
similar to the multiplicative seasonal AlUMA process. In either case, it is generally impossible to determine the exact form or the order of the underlying model from a spectrum,
In terms of the ARIMA model building, the autoconelations and partial autocorrelations
are, in general, more useful. In any event, for model building the autocorreIation and the
spectrum should complement rather than compete with each other.

12.3.1 The Filter Function
Let Z, be a stationary process with' an absolutely summable autocovariance sequence yZ(k)
and corresponding spectrum
282
Consider the following linear filter of 2, given by
CO
where a(B) = Zj=-aj@and

Xj=-tql < 00.
Without loss of generaIity, assume that E(Z,) = 0 and hence E(Y,)= 0.We have
E(,,+,)
= E[(
h = - a ah+-.)
whereweletl=k-j+h.
Now, y y (k) is absolutely summable, because
Hence, we have
(2
j=-Ol
q Z t +k- j ) ]
283
where
and which is often known as the f2ter function or the transfer function. This filter function
can be used to measure the effect of applying a filter on a series. In many studies, one important objective is to design a good filter so that the output series or signals satisfy certain
desired properties.
12.3.2 ~ f f e c of
t Moving Average
Consider the simple moving average
where a ( B ) =
( E Y L ~ ' B ~Then,
) / ~ . from (12.3.4),
- -1 1
- cos mw
m2 1 - c o s o
'
Note that cos mw = -1 when mw = (2k - 1)1T, i.e., w

(2k - l)m/m for k = 1 , . . . ,
[(m 1)/2],and cos mw = 1 when mo = 2k .rr or w = 2k ?rim for k = 1, . . , [m/2].Also,
Hence, for ni r 2, the filter function has its absolute maximum at w = 0 and relative maxima
at frequencies w = (2k - I)+ for k = 1, , , [(ni 11/21, The function vanishes at frequencies o = 2 k d n t for k = 1, . . . , [mn/2].Because (1 - cos o)is an increasing function
between 0 and v , the overall shape of the filter function looks like that shown in Figure 12.9.
When m is large, the first peak around the zero frequency becomes dominating and the relative maxima at other frequencies become negligible, which implies that Y,will mainly contain
..
284
CHAPTER I2 Spectral Theory of Stationary Processes
77
Frequency (0 to d
FIGURE 12.9 Filter function for
Icy(eiw) 1
l/m2[(l - cos mo]/(l
- cos w ] ] .
only the low frequency component of the original series 2,. Because a monotonic trend has an
infinite period that corresponds to a zero frequency, the moving average bas often been used
to estimate the monotonic trend component of a series. A filter like this one, which preserves
the low-frequency component and attenuates or eliminates the high-frequency component of a
series, is called a low pass filter.
12.3.3 Effect of Differencing

Next, we consider the differencing operator
where n(B) = (1 - B). From (12.3.4), we have
[a(eio)t2= (1 - eiw)( 1
- e-iu)
= 2(1 - cos w ) .
(12.3.8)
Clearly, Ia(ei")12is 0 at w = 0.It increases between 0 and rr and attains its maximum at o = a,
as shown in Figure 12.10. Thus, the differencing filter preserves the higher-frequency component and essentially eliminates the low-frequency component of a series. Such a filter is called
a high pass fitter and is often employed to remove the trend of a series. The seasonal differencing has an effect similar to a high pass filter and is left as an exercise.
12.4 Aliasing
285
7l
Frequency [O to T )
FIGURE 12.10 Filter function for differencing
1 a(eiw) l2= 211 - cos o).
12.4 Aliasing
Consider the following cosine curves:
eos[(o -t 2 n j / ~ t ) t ] , forj
0, +1, &2, . . .
and
-m
< t < m.
(12.4.1)
For any given o,the curves cos(ot) and cos[(w f2 ~ r j / A t ) ffor

] j # 0 are clearly different, as
shown in figure 12.11 for cas(ot) and cos[(w 2 ~ r / A t ) twith
] o = 7r/2 and At = 1. Now,
suppose that we study the curve cos(ot) as shown in the dotted line and observe it only at the
time points k Af for k = 0,f1, +2, . . . , where A t is the sampling interval. These curves
become indistinguishableat the time points t = k A t for k = 0,Ifr 1, f2, . . . because
coa[(w
k)*:
FIGURE 12.11 The aliasing.
At]
cos(uk At i 2 ~ r j k )= cor(uk At),
(12.4.2)
286
The same problem occurs for the sine and exponential functions. Thus, in Fourier representation, if we observe a process 2, at t = k At, for k = 0,fI , f2, . , then the components of 2, with frequencies o 3~ 2 a j / A t , for j = k l , f2 , . . will all appear to have
frequency o,These frequencies are said to be aliases of w. The phenomenon, in which a
high frequency component takes on the identity of a lower frequency, is called aliasirig,
For a given sample interval At, if we observe a process at time points k At for k = 0,
f1, 2t2, , . . , the fastest observable oscillation goes above the mean at one time point and
below the mean at the next time point. The period of this fastest oscillation is 2At. Thus, the
highest frequency used in Fourier transform is 2?r/2At -- ?r/At, which is also known as the
Nyquist frequency with the given sampling interval At. With this Nyquist frequency v / A t , if
o < m/At, then, as discussed above, the components with frequencies w A 2vj/At, for j = 0 ,
f1, f2, . . , are all confounded and are aliases of one another, and the sum of all their contributions will appear at frequency o in F(w) orf(o), making the estimation of contributions
of different components difficult or impossible. To avoid this confounding or aliasing problem, we thus should choose the sampling interval so that the frequencies under study are all
less than the Nyquist frequency. For example, if a phenomenon is a linear combination of
sine-cosine components with frequencies wl, 02,
. . . , and om,then we shouId choose At SO
that loj]< (v/At) for j = 1,2, . , m. Otherwise, if v / A t < 0.9,say, then will be aliased
with some other gequency in the interval fO,?r/Af] and hence make the estimation of the
component at oj impossible. In data analysis, we often take At = 1.
..
..
12.1 Consider the process
where A1 and A2 are independent random variables, each with mean 0 and variance T.
(a) Find the autocovariance function ykfor 2,.
{b) Is yk absolutely summable? Find the spectral distribution fuhction or the spectrum of 2,.
where A is a constant, A is a uniformly distributed random variable on [-a,v],

and the at's are i.i.d. random variables with mean 0 and variance 271.. Also, h and a,
are independent for all t.
(a) E n d the autocovariance function yk for 2,.
(b) Is yk absolutely summable? Find the spectral distribution function or the spectrum
of Z,.
Exercises
287
12.3 A spectral distribution function G(w) can be decomposed in the form

G(o) = pGa(o)
+ q~d(o),
where p and q are nonnegative, p $; q = 1, Ga(w) is an absolutely continuous spectral

w a) discrete or step spectral distribution function,
distribution function, and ~ ~ ( is
Express the following specttd distribution function in the above decomposition form:
12,4 Given the following spectral distribution of Z,,
Find the autocovariance function yh for Z,.

12.5 Find and discuss the shape of the spectra for AR(1) processes, (1 - $B)Z,
with the following values of 6:
a,,
fa) 4 = .6,
(b) 4 = .99,
(c)
+ = -.5.
12.6 Find and discuss the spectrum for each of the following processes:
(a) (1 - .7B + . 1 2 ~ ~=) a,,
~,
(b) 2, = (1 - .7B - .18B2)a,,
(c) (1 - .5B)Zt = (1 .6B)aj,

(d) Z, = (1 - .4B)(1 - .8&)a,, where the a,'s are i.i.d. N(0, 1).
12.7 Find and discuss the shape of the spectrum for

(1 - @ B ~ ) z =
, a,,
where the a, is a Gaussian white noise process with mean 0 and variance 1.
12.8 Let X, and Y, be independent stationary processes with the spectra fx(w) and fy(o),
respectively. Assume that Y,follows the AR(2) model, (1 - .6B - .2B2)Y, = a,,
and X, and a, are independent white noise processes, each with mean 0 and variance cr2. If Z, = X,+ Y,, then find and discuss the shape of the spectrum of 2,.
288
12.9 Consider the following linear filter of 2,:
where Z, is a stationary process with an absolutely summable autocovariance function. Find and discuss the filter function defined in (12.3.4).
12.10 Differencing operators have been used widely to reduce a nonstationary process to a
stationary process. Discuss the effect through the filter function for each of the
following differencing operators:
fa) Wt = (1 - B > ~ z ~ ,
(b) I.V, = (1 - B~)Z,,
(c) IV, = (I - B)(1 - B ' ~ ) Z ~ .
Estimation of the Spectrum

In this chapter, we'discuss estimation of the spectrum. We begin with periodogram analysis,
which is a useful technique to search for hidden periodicities. The smoothing of the sample
spectrum and related concepts such as the lag and spectral windows are discussed. Empirical
examples are presented to illustrate the procedures and results.

13.1.1 The Periodogram
Given a time series of n observations, we can use the results in Section 11.3 to represent the
n observations in the following Fourier representation:
InPI
2, =
2 (akcos o k t + bk sin o k t ) ,
k=O
where o k = 2ak/n, k = 0, 1, . . . , [11/2], are Fourier frequencies, and
and
are Fourier coefficients. It is interesting to note the close relationship between the above
Fourier representation and regressidn analysis. In fact, the Fourier coefficients are essentially
the least squares estimates of the coefficients in fitting the following regression model:
mn/21
Z, =
k=O
(akcos o k t + bk sin wkt) + el,
290
where the ok are the same Fourier frequencies. The fitting will be perfect, but we can use the
notion from regression analysis to regard the Parseval's relation
as an analysis of variance presented in Table 13.1.

The quantity I(wn) defined by
mi,
n
- (a: 4-
bz),
nu52 ,
0,
k = 1, .
. . ,[(n - 1)/2],
(13,1.4)
?I
k = - when n is even,
2
is called the periodograrn. It was introduced by Schuster (1898) to search for a periodic
component in a series.
Sampling Properties of the Periodogram

Assume that Zl, Z2, . . . , and Z, are i.i,d.N ( 0 , u2).Then
13.1.2
TABLE 13.1 Analysis of variance table for periodogram analysis.

Source
Degrees of freedom
Sum of squares
--
Frequency wo = 0 (Mean)
Frequency w~ = 2m/n
11
-(a:
2
+ b:)
Frequency oz = 4r/n
Frequency W f [ n - u / 2 ] = [(n-l)/2]2r/n
Frequency w , / ~= .rr (exists only for even n)
Total
2
2
y ( a [ ( n - ~ ) / ~+] b[(n-1)/21)
291
and
where we use the fact that ak and aj are independent for k # j because of (11.2.13).
Hence, the ak for k = 1, 2, . , [(n - 1)/2], are i,i.d, N(0, 2g2/1z), and the
naf/2u2, k = 1,2,. , [ ( n - 1)/2f, are i,i.d. chi-squares with one degree of freedom.
Similarly, the nb$/2cr2, k = 1,2, , , [ ( t i - 1)/2], are i.i.d. chi-squares with one degree
of freedom. Furthermore, naj/2u2 and nbj2/2u2 are independent for k = 1, . . . ,
[(n - 11/21 and j = 1, 2, . , [(n - 1)/23, because, ak and bj are normal and by the
orthogonality property of the cosine and sine system, we have
..
..
..
. .
Ii
Z, cos mkt
=
4 "
C [E(z?)
n2
4ff2
112 1=1
=0
cos wkt sin wit]
C" [COSukt
=-
.C
Zu sin ojv
u=l
sin ojt]
for any k and j.
(13.1.5)
It follows that the periodogram ordinates
for k = 1,2, . . . , f(n - 1)/2] are i.i.d. chi-squares with two degrees of freedom, Clearly, by
u ~ even n) follows the chi-square distrithe same arguments, each of 1(0)/c~and I ( T ~ ) / (for
bution with one degree of fteedom. With the adjustment for Z(T) in mind, we assume,
without loss of generaiity, that the sample size n is odd in the following discussion.
Suppose that a time series of n observations contains a periodic component and can be
represented by
Z,
= a. 't
ak cos wkt
+ bksin o k t -k e,,
(13.1.7)
where ok = 27rk/n, k f 0, and the e, are i.i.d. N(0, (r2). To test the hypothesis in (13.1.7), it
is equivalent to test in Table 13.1
Ho:ak=bk=O
vs.
Hl:akSCO
or
bk+O
292
using the test statistic
(n - 3) (a: + bz)
2 f:f3 (a,? bj2)'
J#k
which follows the F-distribution F(2,11- 3) with 2 and (n - 3) degrees of fieedom. Neither
the numerator nor the denominator in (13.1.8) includes the term ao, corresponding to the
zero frequency. In fact, because the periodogram at frequency zero reflects the sample mean
and not the periodicity of the series, it is usually removed from consideration in the analysis.
More generafly, we can test whether a series contains multiple m periodic components by
postulating the model
Zt
= a.
r..
(ak,cos uk1t+ bki sin mkit) + e,,
(13.1.9)
i= 1
where the e, are i,i,d. N(0, m2), wk, = 2?rk JII, and the set I = {ki : i = 1, . . , ma} is a subset
of {k : k = 1,2, . . . , [n/2]).The corresponding test statistic from Table 13.1
follows the F-distribution F(2m, n - 2m

freedom.
1) with 2m and (n - 2m - 1) degrees of
13.1.3 Tests for Hidden Periodic Components

In practice, even if we believe that a time series contains a periodic component, the underlying frequency is often unknown. For example, we might test
H o : a = /3 = 0
vs.
Hl:aZ 0
or
/3
for the model

Zt = p
+ a cos ~t + /3 sin w t + e,,
(13.1.11)
where {er] is a Gatmian white noise series of i.i.d. N(0, a*)and the kquency w is unknown.
Because the frequency w is not known, the F-distribution and the test statistic as discussed i n
293
the previous section are not directiy applicable. The periodogram analysis, however, can still
be used. In fact, the original purpose of the periodogram was to search for "hidden
periodicities." If the time series indeed contains a single periodic component at frequency w,
it is hoped that the periodogram I(o& at the Fourier &equency wk closest to o will be the
maximum. Thus, we can search out the maximum periodogram ordinate and test whether this
ordinate can be reasonably considered as the maximum in a random sample of [n/2J i.i.d.
random variables, each being a multiple of a chi-square distribution with two degrees of h e dom. In this case, a natural test statistic will be
where we use w(l,-toindicate the Fourier frequency with the maximum periodograrn ordinate.
Now, under the null hypothesis Ho, the { Z , ) are Gaussian white noise N(0, cr2). Hence,
the periodogram ordinates I(#R)/u2,k = 1, 2, . . . , [n/2],are i.i.d. chi-square random variabIes with two degrees of freedom, which have the probability density function
Thus, for any g 3 0, we have
If u2were hown, then we could use (13.1.14)to derive an exact test for the maximum ordinate.
In practice, however, u2is often unknown and has to be estimated. To derive an unbiased estimator of cr2, we note that under the null hypothesis
Thus,
294
is an unbiased estimator of a2,

and by replacing u2by &2 it leads to the test statistics
Using that
= 1,2,
. . . , [11/2] are independent, we have
It follows that G2is a consistent estimator of a2.Thus, for large samples, the distribution of
V can be approximated by the same distribution as ~ ( ~ ) ( o ( ~ , )i.e.,
/ u ~for
, any g '- 0,
An exact test for max{I(o&] was derived by Fisher (1929), based an the following statistic:
Under the null hypothesis of the Gaussian white noise process N(0, u2)for Z,, Fisher (1929)
showed that
where N = [r1/2], g > 0, and nt is the largest integer less than l/g. ~ h u sfor
, any given significance level a,we can use Equation (13.1.20) to find the critical value g, such that
If the T value calculated from the series is larger than g,, then we reject the null hypothesis
and conclude that the series Z, contains a periodic component. This test procedure is known
as Fisher's test,
The critical values of T for the sigdicance level a = .05 as given by Fisher (1929) is
shown in Table 13.2.
The third column in Table 13.2 is an approximation obtained by using only the first term
in (13.1.20),i.e.,
295
TABLE 13.2 . The critical values for a = .05 for the ratio of the
largest periodogram ordinate t o the sum.
N*
*N= fn
g, (by exact formula)
g, @y first term only)
- 1)/2 if n is odd and N = (n/2 - 1) if n is even.
The approximation is very close to the exact result. Hence, for most practical purposes, we
can use Equation (13.1.21)to derive the critical value g, for the test.
Under the null hypothesis Ho in (13.1,11), a significant value of ~ ( ' ) ( w (leads
~ ) ) to the
rejection of Ho, which implies that there exists a periodic component in the series at some
frequency o,This o is not necessarily equal to w ( l ) , because
is chosen only from
the Fourier frequencies and not from all the possible frequencies between 0 and rr. Hartley
(19491, however, has shown that the unknown w with the maximum periodic component
can be safely estimated by
and that the probability that lo - wcl,l > 29~/11is less
than the significance level of the test.
Let I ( ~ ) ( w (be
~ )the
) second large-st periodogram ordinate at Fourier fiequency ~ ( 2 )Whittle
.
(1952) suggested extending Fisher's test for this second largest ordinate based on the test statistic
where the distribution in (13.1.20)is taken as the distribution of T2 with N being replaced by
(N - 1 ) . The procedure can be continued until an insignificant result is obtained. It leads to
an estimate of m,the number of periodic components present in the series.
Alternatively, we can use standard regression analysis and construct the usual regression
test. For example, to examine whether a monthly time series is seasonal with period 12, we
might postulate the model
where the et are i.i.d. N(0, u2).

The test for seasonality is then performed by the F ratio
of the mean square for the six seasonal frequencies 277k/12, for k = 1 , 2, 3, 4, 5, 6, to the
residual mean square.
297
Frequency (0 to T)
FIGURE 13.1 Periodogram of the logarithms of the annual Canadian

lynx pelt sales,
nated by a very latge peak at frequency w6 = 21r(6)/55 = .68544, This fresuencycorresponds to
a period of P = 2v/o = 2v/w6 = 9.1667 years. It indicates that the data exhibit an approximate
nine-year cycle. Also included in Table 13.3 are the values of the F statistics given in (13.1.8) to
test the significance of the periodogram at each Fourier frequency. For significance level a = .O5,
F.05(2,52) = 3.19533, and the periodogram is significantonly at frequency m6 = .68544.
For Fisher's exact test for the maximum periodogram, we have
From Table 13.2, g,os= .22805 for N = 25, and g.05 = .I9784 for N = 30. More precisely for
N = 27, we can use the first term approximation given in (13.1.21). For a = .O5, we have
which gives g = .21493. Because T = .7518 >> .21493, the result is highly significant and
we conclude that the series contains a periodic component at the frequency 0)s = .68544.
298

In this section, we investigate the estimation of the spectrum of time series with absolutely
summabIe autocovariances.From (12.1,3),the spectrum is given by
03
= -(YO
2~
+ 2k=lx y kCOS OK),
-?r
5 6.l 5
v.
Based on sample data, it is natural to estimate f ( o ) by replacing the theoretical autocovariances yk by the sample autocovariances
For a given time series of n observations, however, we can only calculate fork = 0,l, . . . ,(a - l). Hence, we estimate f ( o ) by
Tk,
Tk
and call it the sample spectrum. Because

Section 2.5.2, we have
Tk is asymptotically unbiased, as discussed in
lim ~ ( f ( w )=) f ( w ) .
11-cO
Thus, j ( w ) is asyrnptaticaIly unbiased and looks very promising as a potential estimator

forf (w).
To further examine the properties of the sample spectrum, let us consider f ( w k ) at the
Fourier frequencies ok= Zn-k/n, k = I ,
,[11/2], At these Fourier frequencies, the sample
spectrum and the periodogram are closely related. To see, we note that
.. .
Z,(cos okt- i sin wkt)
2 " Z*(COS
okt 'i. i sin @kt)
where we use that

Because
x:=,
eiokt
299
2:-e-iwir = 0.
and letting j = f - s in (13.2.4), we get
Hence, fro~n(13.2.2b), we have
where we note that j(wr) = I(~.,~)/2?r = mn,J2?r if n is even.

It follo\vs that if Z,IS a Gausslan white noise series with mean 0 and constant variance
a2, then f(wk), fork = 1,2, . . . , (11 - 1)/2, are distributed independently and identically as
(a2/4.rr)x2(2) = ( ~ ~ / 2 1 ~ ) ~ ~ (and
2 ) /we
2 , denote it as
where x2(2) is the chi-square distribution with two degrees of freedom. Here, we note that
u2/27r in (13.2.7) is the spectrum of Z,. In generai, following the same arguments, it can be
then
shown that if Zt is a Gaussian process with the spectrumf(o),
Now,
300
and
which does not depend on the sample size n. Thus, although the sample spectrum cdculated at the Fourier frequencies is_unbiased, it is an unsatisfactory estimator because it is
not consistent as the variance of f fok)does not tend to zero as the sample size n tends to
infinity. Furthermore, by (13.2,6) and that the periodogram ordinates are independent, as
shown in (13.1.6), we have
for any two different Fourier frequencies wk and mi. More generally, even for other distinct
non-Fourier frequencies o and A, as the sample size n increases and the mesh of the Fourier
Frequency (0 to .n)
Frequency @ to .Ir)
Frequency (0 to T )
Frequency (0 to ?rf
FIGURE 13.2 Sample spectrum of a white noise process.
301
frequencies becomes finer and finer, the covariance between j ( w ) and j(h) still tends to
zero as TI e m, As a result, f ( ; ( o ) is unstable and tends to be very jagged, as illustrated in
Figure 13.2, Note that the pattern persists regardless of the size of the sample. The effort of
correcting these unsatisfactory properties leads to the smoothing of the periodogram and the
sample spectrum,

13.3.1 Smoothing in the Frequency Domain: The Spectral Window
A natural way to ieduce the variance of the sample spectrum is to smooth the sample spectrum locally in the neighborhood of the target frequency. In other words, the spectral estimator is the smoothed spectrum obtained from the following weighted average of m values to
the right and left of a target frequency ok,i.e.,
where the mk = 27rk/n, n being the sample size, k = 0, f1, . . . , f.[n/2], are Fourier frequencies; nt, indicates that m is a function of n, which is often chosen such that mn -+ m but
(m,/n) +0 as n +m; and W,(wj) is the weighting function having properties that
and
m"
Iim
IJ+M
2 w,2(wj) = 0,
j=-m,
The weighting function W n ( w j ) is called the spectral window because only some of the
spectral ordinates are utilized and shown in the smoothing summation. Iff(o) is flat and
approximately constant within the window, then we have
302
Also, by (13,2.10),
where we also use that the sample spectrum is independent at Fourier frequencies.
The property (13.3.4) of the spectral window implies that the variance of the smoothed
spectrum decreases as m, increases. The value of rn, represents the number of frequencies
used in the smoothing. This value is directly related to the width of the spectral window,
also known as the bandwidth of the window, As the bandwidth increases, more spectral
ordinates are averaged; hence, the resuiting estimator becomes smoother, more stable, and
has smaller variance. Unless f(w) is really flat, however, the bias will also increase as the
bandwidth increases because more and more spectral ordinates are used in the smoothing.
We are thus forced to compromise between variance reduction and bias, a common
dilemma with statistical estimators.
More generally, from (13.2.2a), the sample spectrum is defined at any frequency w
between -T to a,and not only at the Fourier frequencies. Thus, we can write the general
smoothed spectrum in terms of the following integral form:
where W,(A) is the spectral window satisfying the conditions
and
The spectral window is also known as the kernel in the literature.
303
If the spectrum is approximately constant within the bandwidth of the spectral window,
the condition (13.3.9) implies that
For the variance, we note that (13.3.8) can be approximated by the sum
.
where mk = 2 ~ k / 1 1Thus,
By condition (13.3,11), var(f;,(w)) +0 as n + w; hence, f,,(w) is a consistent estimator

Off(&).
Clearly, spectral ordinates at two different frequencies o and A are in general correlated
because they may contain some common terms in the smoothing. Because the sample spectral ordinates at different Fourier frequencies are independent, however, the covariance
between smoothed spectral ordinates is easily seen to be proportional to the amount of overlap between the spectral windows centered at o and A. The covariance will be large if the
spectral windows overlap considerably; the covariance will be small if the spectral windows
overlap only slightly.
A few other remarks are in order. In the actual computation of the smoothed spectrum,
the discrete form of (13.3.1) is used at the Fourier frequencies. Thus, the smoothing is
essentially applied to the periodogram. Because the periodogram is periodic with period 2 7 ~ ,
when the window covers frequencies that fail to lie entirely in the range between -v and a,
we can extend the periodogram and hence the sample spectrum using the periodic property.
Equivalently, we can fold the weights back into the interval -T to a.As the periodogram is
also symmetric about frequency zero, catculation is only necessary for the frequency range
between zero and a.Also, because the periodogram at frequency zero reflects the sample
mean of Z, and not the spectrum, it is not included in the smoothing, and tbe value at wl is
used in its place.
304
13.3.2 Smoothing in the Time Domain: The Lag Window

Note that the spectrum f(m) is the Fourier transform of the autocovariance function yk.
Hence, from (13.2.2a), an alternative to spectrum smoothing is to apply a weighting function
W ( k ) to the sample autocovariances,i.e.,
Tk
Tk
Because the sample autocovariance function is symmetric and because is less reliable
for larger k, the weighting function W(k) should also be chosen to be symmetric, with its
weights inversely proportional to the magnitude of k. Thus, we have
where the weighting function W,(k) is chosen to be an absolutely summable sequence
W, (k) = I V ( i ) ,
which is often derived from a bounded even continuous function W(x) satisfying
The value of M is the truncation point that depends on the sample size tt. This weighting
function W,(k) for the autocovariances is called the lag window.
The lag window and the spectral window are closely related because the autocovariance
is the iiyerse Fourier transform of the spectrum. From (13.2.2a), the inverse Fourier transform of f ( h ) is
Thus,
305
where
is the spectral window. It is clear from (13.3.21) that the spectral window is the Fourier
transform of the lag window and that the lag window is the inverse Fourier transform of the
spectral window, i.e.,
Hence, the fag window and the spectral window form a Fourier transform pair, with one
~ u spectral witldolv
being uniquely determined by the other. Both the terms lag ~ v f ~ d oand
were introduced by BIackman and Tukey (1959). The ~veightirtgfunctiort was the standard
term used in the earlier literature.
The variance expression given in (13.3.14) can also be written in terms of a lag window.
By Parseval's relation from (11.5.16),
hence,
Because the lag window is often derived from a bounded even continuous function as given
in (13.3.181, however, we have
306
CHAPTER I3 Estimation of the Spectrum
Thus,
1
vu[f
(o)]
" ;P(&)/
W2(x)&.
-1
This result implies that the variance of the smoothed spectrum for a given lag window is
related to the ratio M/n, which is the proportion of sample autocovarjances for which the
lag weights are nonzero.
13.3.3 Some Commonly Used Windows

With the above general background, we now introduce some lag and spectral windows commonly used in time series analysis.
The Rectangular Window The so-called rectangular or truncated lag window
where M is the truncation point less than (n - 11, is derived from the continuous function
From (13.3.21), the corresponding spectral window is given by
1
=-[I
2~
-
sin(oM/2)
+ 2 cos(o(M ~+h (1)/2)
O/2)
1 sin(o/2)
2%-
(using (11.2.8)),
+ [sin(o(M ci 112)) - sin(w/2)] ,

sin(o/2)
(using (11.2.12c)),
The rectangular lag and spectral windows are both shown in Figure 13.3,
307
FIGURE 13.3 The rectangular lag and spectral windows. (a) Rectangular
lag window. [b) Rectangular spectral window.
Note that the spectral window as shown in Figure 13.3 has a main lobe at w = 0 with a
height of (2M 1)/2a, zeros at o = f2j7~/(2M I), and side peaks (lobes) of decreasing magnitude approximately at o = 6f4j l)n/(2M
1) for j = 1,2, . . As a result, the smoothed
spectrum estimator using this window may lead to negative values for some fiquencies o.
Because the spectrum is known to be a nonnegative function, this result is certainly not desirable.
For a given spectral window W,(w) that attains its maximum at w = 0, one commonly
used definition for the bandwidth of the window is the distance between the half-power
points on the main lobe, as shown in Figure 13.4. That is,
Bandwidth = 2wJ,
where wl is such that W,(f wl) = ~ " ( 0 ) .
FIGURE 13.4 Bandwidth of the spectral windo\v.
..
(13.3.30)
308
Lf wll is the first zero of W,(w), the value of O.Q can be approximated by ;all; hence, the
bandwidth is approximately equal to oll. In terms of the above rectangular spectral window
~ i ( w ) the
, bandwidth is approximately equal to 2 d 2 M 1). In general, the bandwidth of
the spectral window is inversely related to the truncation point M used in the lag window.
Thus, as M increases, the bandwidth decreases; hence, the variance of the smoothed spectrum
increases although the bias decreases, as discussed in Section 13.3.1. On the other hand, as M
decreases, the bandwidth increases and the variance decreases while the bias increases.
BartIett's Window Bartlett (1950) proposed the lag window
based on the triangular function
Hence, the window is also known as the &iangular window. The corresponding spectral window
is given by
sin(o/2) cos(wM/2) sin(w(M - 1)/2)

sin(w/2)
by (11.2.9) and (11.2.8)
13.3
The Smoothed Spectrum
1
-(1 - cos w ) -t- -(cos o - cos wM)
2~rM[iin(w/2)]~2
2
{L(
309
by (11.2.12b)
1 - cos w M )
2~r~[sin(m/2>]~
Bartlett's triangular lag window and its corresponding spectral window are shown in
Figure 13.5, Because the spectral window Wf(w) is nonnegative, the Bartlett's spectrum
estimator is always nonnegative. Furthermore, a direct comparison of Equations (13.3.29)
and (13.3.33) shows that the side lobes of the Bartlett window are sm~flerthan the side lobes
of the rectangular window, The effect of large side lobes is to allow f (A) to make large contributions in the smoothing at frequencies distant from w. Hence, the resulting spectrum
FIGURE 13.5 BartIett's lag and spectral windows.

lag window. (b) Bartlett's spectral window.
(a]
Bartlett's triangular
310
estimate fw(o)may reflect significant spectral components at other frequencies different

from o.This phenomenon is known as leakage. Because lobes are produced by the Fourier
transform of sharp corners, the general principle of the sefection of lag windows is to avoid
functions with sharp corners.
The Blackman-Tukey Window Blackman and Tukey (1959) suggested the lag window
based on the continuous weighting function
W(x) =
1 - 2a
+ 2a cos vx,
1x1 5 1,
Ixt > 1.
Again, M is the truncation point for the sample autocovariance function and the constant a is
chosen in the range 0 < a 5 .25 so that t.v$(k)r 0 for all k. The corresponding spectral
window can be derived as follows:
where we have used (13.3.29). Thus, the Blackman-Tukey spectral window is a weighted linear combination of the spectral windows for the rectangular lag function at the frequencies
(o- ?r/M), o,and (o4- ?r/M), i.e.,
where W;(o) is given by (13.3.29)As

. a result, the Blackman-Tukey spectrum estimator may
also be negative at some frequencies o.
31 1
Blackman and 'I'ukey (1959) called the window in (13.3.34) with a = .23 as hamtning
after R.W.Hamming, one of nkey's coauthors who had worked on the subject. The resulting
window is
which is also known as the Tukey-Hamming window in the Literature. The window (13.3.34)
with a = .25 is called as hanni~tgafter the Austrian meteorologist Julius von Hann, who was
not directly related with the subject. The window is given by
which is also known as the Tukey-Hanning window or the Tukey window in the literature.
The Parzen Window Parzen (1961b) suggested the lag window
which is based on the continuous weighting hnction
The corresponding spectral window for an even value of M is given by

1
W,P(u)= -
";'
C W;(k) COS wk
2.rr k=-&f
+2 2
(1
- / k l / ~cos
) ~wk
M/2<lkl~df
where we refer readers to a more 'detailed derivation in Parzen (1963). When M is large,
(13.3.41) is approximated by
312
FIGURE 13.6 The Tukey-Hanning and the Parzen windows. (a] Tukey-Hanning
lag window (M = 5)- @) Parzen lag window [M = 53. (c) Tukey-Hanning spectral
window (M = 51, (d) Parzen spectral window (M = 5).
Both the Tukey-Hanning window and the P m n window are plotted in Figure 13.6.As can
be seen from Figure 13.6(b) and (d), for the same mncation point M, the bandwidth of the
Tukey-Hanning spectral window is narrower than the bandwidth of the Parzen spectral window.
Hence, the Tukey-Hanning estimator will have a smaller bias than the Parzen estimator.
Because W:(o) is always nonnegative, however, the Parzen window will produce a nonnegative and smoother spechum estimator.
The above windows are some commonly used ones, particularly in terms of the commercially available software. Many other windows are introduced in the literature. Interested readers are referred to an excellent book by Priestley (1981, Chapter 6) for more
details.
The quality of a smoothed spectrum is determined by the shape of the window (i.e., the
form of the weighting function) and the bandwidth of the window (or, equivalently, the truncation point). The spectrum estimates produced for the same window shape and different
bandwidths vary, Thus, in spectrum smoothing, we are concerned not only about the design
313
of spectral windows with desirable shapes (or "window carpentry" as John W. Tukey called
it), but also about the bandwidth of the windows. The latter concern often is a more crucial
and difficult problem in time series analysis because for a given window shape there is no
single criterion for choosing the optimal bandwidth. Additionally, wider bandwidths produce
smoother spectrums with smaller variance of the estimators; on the other hand, narrower
bandwidths Iead to smaller bias and smudging and hence to higher resolution of the resulting
spectrum estimates. A compromise is needed between high stability and high resolution. To
ease the difficulty, the following steps are often suggested. First, choose a spectral window
with an acceptable and desirable shape. Then initially calculate the spectral estimates using a
large bandwidth and then recalculate the estimates by using gradually smaller bandwidths
until the required stability and resolution are achieved. This procedure is often referred to as
"window closing."
Because the bandwidth of a spectral window is inversely related to the truncation
point M used in the lag window, Jenkins and Watts (1968, Chapter 7) suggest choosing
three values of M, say, M I , M2, and M3 with M3 = 4M1. The sequences of spectral estimates are then computed and examined with these M values to see whether an intermediate value between MI and M3, or a value smaller than MI, or a value larger than M3 should
be chosen. The spectral bandwidth can also be determined by choosing a truncation point
M such that ?(k) for k > Mare negligible. This choice of M has been found practical and
was used successfully in a number of cases, Alternatively, a truncation point M can also be
chosen as a function of n, the number of observations in a series, For a moderate size 11, M
can be chosen to be M = [n/10].
In principle, if we wish to be able to distinguish several peaks offlu) at frequencies XI, A2,
As, . . . ,the bandwidth of the spectral window should not exceed the minimum interval between
these adjacent peaks. Otherwise, the smoothed spectrum will blur these peaks, In fact, a commonly chosen bandwidth is equal to minj]hi, 1 - Ail.
EXAMPLE 13.2 Figure 13.7 shows the sample spectrum of the lynx pelt sales data
over the interval (0,T).Also shown are the smoothed sample spectrum functions using
the Bartlett window with truncation points M = 2, 5, and 10. The spectrum for M = 10
clearly is improper. To compare different estimates, Figure 13.8 shows the spectrum
estimates using various windows with the same truncation point M = 5. The smoothed
spectrum given by the rectangular window is clearly undesirable. The estimates using
the Bartlett, Parzen, and Tukey-Hanning windows all lead to the periodic component at
the frequency w = .68544, but the rectangular window does not lead to this periodic
component.
13.3.4 Approximate Confidence Intervals for Spectral Ordinates
Recall that for a given sarnple ZI, &, . . , Z,, from a process with the spectrum f(w), the
sample spectral ordinates f ( w k ) at Fourier frequencies u k = 2.rrk/n, with okf 0 and a,are
independent and identically distributed as
314

[a) Spectrum for pelt sales data
@) Bartlett window (M= 2)
Frequency (0 to T)
Frequency (0 to T)
(c) Barlleft
(d) Bartlett window (M = 10)
window (M= 5)
0.35
0.6
Frequency (0 to w)
Frequency (0 to ?r)
FIGURE 13.7 Sample spectrum of the logarithms of the Canadian lynx

pelt sales and Bartlett estimates for different values of M.
Thus, if we smooth the sample spectrum with the simple (2m
+ 1) term moving average, i.e.,
the smoothed spectral ordinate &(wk) will be distributed as
where the degrees of freedom DF = (4m 2) is simply the sum of the degrees of freedom of
(2m
1) x2(2) random variables. In fact, this property is the first smoothing method introduced by D a ~ d
f1946), one of the pioneers of spectrum estimation. The estimator in
31 5
@) Bartlett window (M= 5)
(a) Rectangular window (IM=

5)
Frequency (0 to T )
(c)
Tukey-Hanning window (M= 5)
(d) Parzen window (M = 5)

0.80
Frequency (0 to TF)
Frequency (0 to T)
FIGURE 13.8 Smoothed sample spectrum of the logarithms of the

Canadian lynx pelt sales using different windows.
(13.3.44) is also known as Daniell's estimator. However, this nice additive property no longer
holds for
if o is not a Fourier frequency or if the sample spectrum is smoothed by a
spectral window with nonuniform weights. Thus, w e can only approximate a general smoothed
spechm
by
3,,(0)
3,(4
where c and v are chosen such that
and
316
Now, for any given spectral window W,(w),from (13,3.12)and (13.3.14), we have
~ [ f w ( u l=
l f(m)
and
Hence,
cv = f (o)
and
giving
It folIows that
where v is often referred to as equivalent degrees of freedom for a smoothed spectrum.

Using (13.3.26) or (13,3.23) and (13.3.25), we can also write the equivalent degrees of
freedom in terms of the lag window
where W[x) is the continuous weighting function used in the associated lag window. The
equivalent degrees of fieedom v for some commonly used windows are given in Table 13.4,
317
TABLE 13.4 Equivalent degrees

of freedom for various windows.
Rectangular
Bartlett
Tukey-Hamming
Tukey-Hanning
Parzen
./M
3n/M
2.5 16?1/M
8n/3M
3.709nlM
By (13.3.51), we have
where X:(V) is the upper a% point of the chi-square distribution with v degrees of freedom.
Hence, a (1 - orj100% confidence interval f o r f f : ~is) given by
where Y is the equivalent degrees of freedom calculat2d by (13.3.49) or (13.3.52).

Recall that the asymptotic mean and variance of A,(&) are proportional to f(w) and f2(w),
respectively. Hence, from the discussion in Section 4.3.2, the logarithmic transformation of the
spectrum estimator lnfw(o) is often suggested. From (13.3,53), the (1 - a)l00% confidence
interval for lnfiw) is given by
Note that the width of the interval for?(@) as given in (13.3.53) is proportional to j,,(o) and
hence varies with frequency. The width of the confidence interval for lnf(w) as given in (13.3.54),
however, will be the same for any &equencyo.
EXAMPLE 13.3 For illustration, we calculate the 95% confidence interval for the spectrum
of the Canadian lynx pelt sales discussed in earlier examples using Parzen's window with
M = 5, For n = 55 and M = 5, we have, from Table 13.4, v = 3.709(55/5) = 40.79 = 41.
Because ~ . & ~ ( 4 =
1 )25.22 and ~ . & ( 4 1 ) = 60.56, the 95% confidence interval for f(w),
from (13.3.53), becomes
3 18
CHAPTER 1 3 Estimation of the Spectrum

where jw(o)
is the estimated specbum using Panen's window with M = 5 given in Figure 13.8.
Note that the maximum of fw(m) = .74 occ'm at w =: .69. Thus, the 95% confidence interval
forf(o)at w = .69 is given by
The confidence intervals for f@) at other frequencies w can be ca1culated similarly and are

As discussed in Chapters 2 and 3, any stationary process can be approximated by an AR(p)
model,
b,,
. . . , and 9 be the estimates of +I, . . . , &, and ui. A reasonable

for some p. Let
alternative to spectrum estimation is to substitute these parameter estimates in the theoretical
expression for the spectrum of the AR@) model discussed in Section 12.2.2, i.e.,
Frequency (0 to ?r)
FIGURE 13.9 The 95%confidence intervals for the spectrum of the

logarithms of the Canadian lynx pelt sales using Parzen's window.
31 9
where &,(e-'") = (1 - (Pie-I" - . . . - GPe-bu).This method of spectrum estimation

through an AR approximation was suggested by Akaike f 1969) and Panen (1974) and is often
known as autoregressive spectral estimation.
For large n, Parzen (1974) showed that
A
Thus, to control the variance, the order of p chosen to approximate the process should not be
too large. On the other hand, p should not be too small because an inadequate order of p
leads to a poor approximation and hence may increase the bias of the estimated spectrum.
Consequently, similar to the choice of bandwidth, truncation point, and different windows,
the determination of the order p in the autoregressive approximation becomes a crucial step
in the AR spectral estimation. Parzen (1977) suggested the use of CAT criterion in choosing
the optimal order p. If the AR order p is correctly determined, then asymptotic simultaneous
confidence bands for autoregressive spectra can be constructed using the method by Newton
and Pagano (1984),
More generally, we can approximate the unknown process by an ARMAb, q) model
al,
..
for some p and 4.Let

. . . ,&,, dl, . ,iq,and &: be the consistent estimates of #J~,
...,
(PP,01, . . . ,O,, and (ri,The spectrum for the underlying process can also be estimated by
h
where
and
Equation (13.4.5) is known as ARMA spectral estimation. Sitnilar to the AR spectral estimation, the quality of the ARMA spectral estimation relies on the proper choices of p and q
in the approximation. As discussed in Section 7.7, the choice of the orders p and q can be
determined using criteria such as Akaike's AIC.
320
C W T E R 13 Estimation of the Spectrum
EXAMPLE 13.4 Recall that we fitted the logarithms of Canadian lynx pelt sales with AF&lA
models in Chapter 7 and found that both the AR(3) model
with 6: = .I24 and the ARMA(2,l) model
with 62 = .116, where X, = (In 2, - .98), were adequate. Thus, the spectrum of the logarithms of Canadian lynx pelt sales can also be estimated by
Both the spectra (13.4.8) and (13.4.9) are plotted in Figure 13.10. They are extremely
similar, and both have the sharp peak at o = .68544 yielding a period of 9.17 years. After
carefully examining Figures 13.7, 13.8, and 13.10, one should find that the ARMA spectral estimation is comparable with other estimates for this data set.
(a) AR(3) spectrum
@f ARR.TA(2, I) spectrum
Frequency (0 to ?r)
Frequency (0to n)
FIGURE 13.10 ARMA spectral estimation for the logarithms of Canadian

lynx pelt sales.
Exercises
32 1
13.1 Consider the following data set (read across):
(a) Compute the periodogram.

(b) Perform Fisher's test for the maximum periodogram ordinate.
13.2 Consider each of the series W2 and W3 discussed in Chapters 6 and 7.

(b) Perform Fisher's test for the maximum periodogram ordinate.
(c) Perfom Whittle's test for the second largest periodogram ordinate.
13.3 Compute and discuss the smoothed spectrum for the data set given in Exercise 13.1
using Bartlett's window with truncation points M = 5, 10, and 15.
13.4 (a) Compute the smoothed spectrum for the data set given in Exercise 131 using the
Tukey-Hanning window with M = 10,
(b) Find the 95% confidence interval for the spectrum of the unknown underlying
generating process for the data set using the results obtained in part (a).
13.5 (a) Compute the smoothed spectrum for the Series W2 using Parzen's window with
a proper choice of M.
(b) Find the 95% confidence interval for the spectrum of the unknown underlying
generating process for Series W2 using the result obtained in part (a).
13.6 Consider the first differences of Series W4 discussed in Chapters 6 and 7.
(b) Compute and plot a smoothed spectrum using a proper spectral window,
(c) Find and plot an ARMA spectral estimate.
(d) Comment your results obtained in parts (b) and (c).
13.7 Consider the proper differenced sequence of Series W9 discussed in Chapter 8.
Repeat the analysis requested in Exercise 13.6.
13.8 Find and discuss the ARMA spectral estimates for Series W2 and W3.
Transfer Function Models

In earlier chapters, we were concerned with the time-domain and the frequency-domain
analysis of a univariate time series. In this chapter, we consider transfer function models
where an output series is related to one or more input series. For example, sales may be
related to advertising expenditures or daily electricity consumption series may be related to
certain weather variable series such as daily maximum outdoor temperature, and relative
humidity. We study the single-output linear system with both a single-input and a multipleinput Iinear system. After studying the basic characteristics of these transfer function models,
we discuss the identification, estimation, and diagnostic checking of these models. Forecasting using transfer function models is aIso discussed. Detailed step-by-step model building is
illustrated with an empirical example. The frequency domain approach of bivariate processes
is also presented.

14.1.1 General Concepts
Assume that x, and y, are properly transformed series so that they are both stationary. In a singleinput, single-output linear system, the output series yt and the input series x, are related through
a linear filter as
&=,w
where v(B) =
vl.@ is referred to as the transfer function of filter by Box, Jenkins,
, the noise series of the system that is independent of the input
and Reinsef (1994) and ~ z is
series x,. Recall from Section 12.3 that the term transferfrcilctiortis also used to describe the
fiequency response function in the literature. Box, Jenkins, and Reinsel call Quation
(14.1.1) the transfer function model. For an obvious reason, when x, and n, are assumed to
follow some AREvlA models, Equation (14.1.1) is also known as the ARMAX model.
The coefficients vj in the transfer function model (14.1.1) are often called the impulse
response weights. As a function of j, vj is also called the impulse response function. The
transfer function model is said to be stable if the sequence of these impulse response weights
is absolutely summable, i.e., if ZIv,l < 00. Thus, in a stable system, a bounded input
323
always produces a bounded output. The transfer function model is said to be causal if vj = 0
for j < 0. Thus, in a causal model, the system does not respond to input series until they
have been actually applied to the system. In other words, the present output is affected only
by the system's current and past input values. A causal model is also called a realizable
model as it appears that all real physical systems are causal. In practice, we often consider
only the stable and causal model
CO
cn
where v ( B ) = Zj=O
vjBj, Xj=O[vjl< 00, and X, and n, are independent. The system is
presented in Figure 14.1.
The purposes of transfer hnction modeling are to identify and estimate the transfer
function v(B) and the noise model for n, based on the available information of the input
series xt and the output series y,. The difficulties are that the information on x, and yt is finite
and the transfer function v(B) in (14.1.2) may contain an infinite number of coefficients. To
alleviate these difficulties, we represent the transfer function v(B)in the rational form
. .
..-
where o(B)= 00 - w l B - w,BS, 6(3) = 1 - alB - 6,Br, and b is a delay

parameter representing the actual time lag that elapses before the impulse of the input variable
produces an effect on the output variable. For a stable system, we assume that the roots of
S(B) = 0 are outside the unit circle.
FIGURE 14.1 Dynamic transfer function system.
324

The orders of s, r, and b and their relationships to the impulse response weights uj can
be found by equating the coef6cients of Bj in both sides of the equation
Thus, we have that
The r impulse response weights vb+,, vb+,-~, . . ,V ~ + ~ - ~ serve

+I
as starting values for the
difference equation
Tn other words, the impulse response weights for the system (14.1,3) consist of the following:
1.
2.
3,
4.
b zero weights vo, vl, . . . , vb-I

s - r 1 weights vb, v b + ~ , . , vb+s-rthat do not follow a fixed pattern
r starting impulse response weights v ~ + , - ~ +Vb+s-r+2r
I,
. . . ,and vb+,
vj, for j > b s, that follows the pattern given in (14.1.5)
..
In summary, b is determined by vj = 0 for j < b and vb # 0.The value r is determined

by the pattern of the impulse response weights in a similar manner to the identification of the
order p for a univariate ARTMA model through the pattern of the autocorrelation function,
For a given value of b, if r = 0, then the value of s can be easily found using that vj = 0 for
j > b s; if r # 0, then the value of s is found by checking where the pattern of decay for
impulse response weight begins.
14.1.2 Some Typical Impulse Response Functions

In practice, the values of r and s in the system (14.1,3) rarely exceed 2. Some typical transfer
functions are illustrated as follows:
Tjpe I . r = 0, In this case, the transfer function contains only a finite number of impulse
response weights starting with vb = 00and ending at vb + ,= -usas shown in Table 14.1.
Type 2. r = 1. In this case, the impulse response weights exhibit an exponential decay
starting from vb if s = 0, from v b + if s = 1, and from vb + XS = 2 as shown in Table 14.2.
14.2
The Cross-Correlation Function and Transfer Function Models
325
TABLE 14.1 Transfer function for r = 0.
(b,~)
Transfer function
%pica1 impulse weights
TAl3LE 14.2 Transfer function for r = 1.

Transfer function
Typicai impulse weights
o p e 3, r = 2. In this case, the impulse response weights exhibit either a damped exponential or a damped sine wave depending on the nature of the roots of the polynomial
S(B) = (1 - S I B - S2g2)= 0.They follow an exponential decay if the roots are real, i.e.,
if Si 4a2 r 0; they follow a damped sine wave if the roots are complex, i.e., if
Sf 4S2 < 0. The value of s is determined similarly as described earlier. Table 14.3
illustrates some examples of impulse response weights exhibiting a damped sine wave.
+-
14.2 The Cross-Correlation Function and Transfer

Function Models
14.2.1 The Cross-CorrelationFunction (CCF)
The cross-correlation function is a useful measure of strength and direction of correlation
between two random variables. Given two stochastic processes xt and y, for t = 0, f 1,
f2, , , , we say that x, and y, are jointly stationary if x, and y, are both univariate stationary processes and the cross-covariance function between x, and y,, Cov(xt, y,), is a function
326
TABLE 14.3 Transfer function for r = 2.

Typical impulse weights
Transfer function
of the time difference (s function between x, and y,:
t)
only. In such cases, we have the following cross-covariance
for k = 0, f1 , f2, . . where px = E(x,) and py = E(y,), Upon standardization, we have the
following cross-correlation function (CCF):
for k = 0, f1, *2, . . . , where axand my are the staadard deviations of x, and y,. It is important
to note that the cross-covariancefunction yq(k) and cross-correlation functions p,(k) are generalizations of autocovarimces and autocorrelation functions because y d k ) = yJk) and p d k ) =
p,(k). Unlike the autocorrelation function px(k), however, which is symmetric around the origin,
i.e., pJk) = px(-k), the cross-correlation function is not symmetric, i.e., p,(k) # p d - k ) .
Instead, because y,(k) = E(x, - p,)(yl + k - ~ c y )= E(y, + k h ) ( x t - p,) = y d - k ) , we have
p,(k) = pJ-k). Ttius, the CCF measures not only the strength of an association but also its
direction. To see the full picture of the relationship between the series x, and the series y, it is
important to examine the CCF, p,(k), for both positive lags k > 0 and negative lags k < 0. The
graph of the CCF is sometimes called the cross-conefogram,
EXAMPLE 14.1 Consider the simple AR(I) process

( 1 - @)Z, = a,,
(14.2.3)
where 141 < 1 and n, is a white noise series with zero mean and constant variance a:. For
time t k, we can rewrite Equation (14.2.3) as
14.2 The Cross-Correlation Function and Transfer Function Models
327
Zt+k =
( 1 - &3)
= at+k
+ $a,+k-l
+ +2at+k-2 + ' '
The cross-covariance function between a, and Z, is
Hence, the CCF becomes
where we recall that cr? = Var(Zt) = a;/(l - $2). The graph of this CCF for
Because a univariate ARMA model &(B)Zt = Oq(B)atcan be written as
G#I
= .5 is
the above example shows that the univariate ARMA model can be regarded as a special case of
the transfer function model without a noise term. In this formulation, the univariate time series 2,
is the output, the white noise series a, is the input, and the trmsfer function v(B)= Bq(B)/+p(B).
FIGURE 14.2 CCF between a, and 2, for (f
#B)Z,
= a, with 4 =
.5.
328
CHAPTER 1 4 Transfer Function Models
14.2.2 The Relationship between the Cross-Correlation

Function and the Transfer Function
For time t
+ k, the transfer function model (14.1-2)can be written as
Without loss of generality we assume that ~ 1=, 0 and py = 0. Multiplying by xt on both

sides of Equation (14.2.5) and taking expectations, we have
where we recall that y, (k) = 0 for all k. Hence,
The relationship between the CCF, pe(k) and the impulse response function vj is clearly
contaminated by the autocorrelation structure of the input series xt. Even if r = 0 in
(14.1.3) and hence the transfer function v(B) contains only a finite number of impulse
response weights, the use of Equation (14,2,7)to form a system of equations to solve
for vj as a function of p,(k) and p,(k) is difficult. The variance and covariance of the
sample estimate of p,(k) are clearly contaminated by the autocorrelation structure of
the input series x, as shown in Equation (14,2.7),making the identification of both p,(k)
and uk difficult.
If the input series is white noise, i.e., p,(k) = 0 for k # 0, however, then Equation
(14,2.7) reduces to
Thus, the impulse response function vk is directly proportional to the cross-correlation

function p,(k).
Some remarks are in order.
1 . The CCF, p,(k), is defined only when x, and y, are jointly bivariate stationary processes.
To achieve the required stationarity, some differencing operators and variance stabilization
transformations may be necessary. Thus, unless mentioned otherwise, the processes x, and
y, are assumed to be jointly stationary in our discussions.
2. In the general transfer function model
we assume that the input series x, foIlows an ARMA process
329
where a;is white noise. The series a,,
is often called the prewhitened input series. Applying the same prewhitening transformation to the output series y,, we obtain a filtered output series,
Letting E ,
B ; ' ( B ) ~ ~ ( B the
) ~ ,transfer
,
function model (14.2.9) becomes
The impulse response weights vj for the transfer function can therefore be found as
This result leads us to the fundamental steps of the identification of the transfer function
models to be discussed in the next section,

14.3.1 Sample Cross-Correlation Function
For a given set of time series data x, and y,, 1 5 t 5 n, the cross-coxelation function
is estimated by the sample cross-correlation function
sx=
V'xm,
Sy=
m,
and Z and are the sample means of the xt and y, series, respectively.
(14.3.3)
330
C W T E R 14 Transfer Function Models
To test whether certain values of the p,(k) are zero, we compare the sample CCF &(k)
with their standard errors, Under the normal assumption, Bartlett (1955) derived the approximate variance and covariance between two sample cross-correlations &(k) and ;,(k
j).
The covariance is given by
Hence,
Under the hypothesis that the two series xt and y, are uncorrelated and the x, series is white
noise, (14.3.4) becomes
It follows that
Thus, when the xt series is white noise, we can test the hypothesis that the two series xt and y,
are not cross-cqrrelated by comparing the sample CCF ;,(k) with their approximate standard errors I/-.
Tn practice, the x, series is usually not white noise, and we have to prewhiten it and also
filter the output series, as shown in the next section.
14.3 Construction of Transfer Function Modeb
331
14.3.2 Identification of Transfer Function Models

Based on the above discussions, the transfer function v(B) is obtained from the following
simple steps:
1. Prewhiten the input series,
where at is a white noise series with mean zero and variance

2. Calculate the filtered output series, That is, transform the output series y, using the
above prewhitening model to generate the series
3. Calculate the sample CCF, &(k)
between a,and P, to estimate ~ k :
The significance of the CCF and its equivalent Ck can be tested by comparing it with its
standard error (n - k)-'fi.
4. Identify b, S(B) = (1 - alB - . - 6,Br), and w(B) = ( o o - w l B - . - o , B S ) by
matching the pattern of Gk with the h o w n theoretical patterns of the v k discussed in
Sections 14.1.1 and 14.1.2. Once b, r, and s are chosen, preliminary estimates hj and
can be found from their relationships with vk, as shown in Equation (14.1.4). Thus, we
have a preliminary estimate of the transfer function v(B) as
ij
Identification of the Noise Model Once we obtain the preliminary transfer function,
we can calculate the estimated noise series
332
The appropriate model for the noise can then be identified by examining its sample ACF and
PACF or by other univariate time series identification tools, giving
Remarks on the Identification of the Transfer Function Model Combining

(14.3.12) and (14.3.14), we have the following entertained transfer function model:
Some important remarks are in order in the construction of this transfer function model.
1. In the construction of the model, it is assumed that the variables y,, x,, and a, are all stationary. Thus, for nonstationary time series, some variance-stabilization and differencing transformations should be used k s t to achieve stationarity.
2. In the above process of identifying the transfer function v(B), we prewhiten the input
series. The prewhitening model is applied to Hter the output series, but not necessarily to
whiten it. This method for constructing a causal transfer function model is nonnal and
simple. For constructing a possible noncausal system with feedback phenomenon, where
y, is influenced by x, and x, is also influenced by y,, however, both input and output series
should be prewhitened before examining its CCP. This process is often referred to as
double prewhitening. For a good reference and example of constructing a noncausal
system, see Granger and Newbold (1986). It is, however, generally more advantageous to
model a noncausal system using a vector process discussed in Chapter 16.
14.3.3 Estimation of Transfer Function Models

After identifying a tentative transfer function model
. .
we need to estimate the parameters 6 = (a1 , . ,a,)', o = (wO,W I , . . . , w,)',

+
,
)
I
, d = (dl, .
,O,)', and ui.We can fewrite (14.3.16) as
..
or, equivalently,
where
, ,,
333
and
Thus,
where ci, dj, and ek are functions of ai, uj, and 6,. Under the assumption that the a, are
N(0, a:) white noise series, we have the following conditional likelihood function:
&kp
L ( 6 , o, &fl,
d 1 b, x, Y, xo,yo, ao) = (2?rt~;)-"/~exp
(14.3.20)
where xo, yo, a0 are some proper starting values for computing a, from (14.3.19) similar
to the starting values needed in the estimation of univariate ARIMA models discussed in
Section 7.2.
In general, the estimation methods introduced in Chapter 7 can also be used to estimate
the parameters o , 6,#, 8, and (ri.For example, by setting the unknown a's equal to their
conditional expected values of zero, the nonlinear least squares estimate of these parameters
is obtained by minimizing
w f i e r e f o = m a x { p + r + 1 , b + p + s + I}.
Note that so far we have assumed that b is known, For given values of r, s, p, and q, if
we need aIso to estimate b, then (14,3.21)can be optimized for a likely range of values of b.
Then b is selected to be the value that gives the overall minimum of the sum of squares.
Because the ordinary least squares method is the most commonly used estimation procedure, in terms of the general transfer function model in (14.2.9), one may be tempted to
use it to estimate the parameters. As pointed out by Miiller and Wei (19971, however, the
OLS estimates in this case are not necessarily consistent. B e y introduce an iterative regression procedure to produce consistent estimates, and on the basis of these consistent estimates
they also propose an alternative method of model specification. We refer readers to Miiller
and Wei's paper for details,
334
14.3.4 Diagnostic Checking of Transfer Function Models

After the model has been identified and its parameters estimated, it is necessary to check the
model adequacy before we can use it for forecasting, contral, and other purposes. In the transfer
function model, we assume that the a, are white noise and are independent of the input series x,
and hence are also independent of the prewhitened input series q.Thus, in the diagnostic checking of a transfer function model, we have to examine the residuals 4 from the noise model as
well as the residuals cy, from the prewhitened input model to see whether the assumptions hold,
1. Cross-correlation check: To check whether the noise series a, and the input series x, are
independent. For an adequate model, the sample CCF, j z ( k ) , between Gt and or, should
show no patterns and lie within their two standard errors 2(n - k)-'/',
The following
portmanteau test can also be used:
which approximately follows a X2 distribution with (K 1) - M degrees of freedom,

where m = n - to
1, which is the number of residuals Ci, calculated, and M is the
number of parameters Siand wj estimated in the transfer function v(3)= o(B)/8(B).As
shown in Pierce (19681, the number of degrees of fkeedorn for Qo is independent of the
number of parameters estimated in the noise model.
2. A~ctoco~~relafion
check: To check whether the noise model is adequate. For an adequate
model, both the sample ACF and PACF of ii, should not show any patterns. A portmanteau Q test similar to the one in (7.5.1) can also be used. In other words, we can calculate
The Q1 statistic approximately follows a x2 distribution with (K - p - q) degrees of

freedom depending only on the number of parameters in the noise model.
In terms of model inadequacy, two situations may arise,
Case 1: The transfer function v(B) is incorrect. In this case, regardless of whether the noise
model is adequate, we have both p,,(k)
0 and p,(k) # 0 for some k. To see, suppose that
the correct model is
but we incorrectly choose a wrong transfer function vo(B)leading to an error term b,,
Then,
335
Thus, b, will not only be cross-conefated with x, and hence with a;, but also \viU be autocorrelated. That is hue even \& +o(B) = t$(B), i.e., when the noise model is correct. Hence, the Ernedy in this case is first to reidentify the transfer function v(B)and then to modify the noise model.
Case 2: The transfer function v(B) is correct, and only the noise model is inadequate, In
this case, p,,(k) = 0 for a11 k, but p,(k) # 0 for some k. The pattern of b;(k) can be used to
modify the noise model [B(B)/+(B)]a,.
In summary, for a transfer function model to be adequate, both ;,;(k) and { ; ( k ) should
be statistically insignificant and show no patterns. Because the noise model will be contaminated by an incorrect transfer function, in the diagnostic checking of transfer function models,
it is prudent to carry out the cross-correlation check first. In essence, this step is similar to the
analysis of a variance model with an interaction term where the test of an interaction effect
should be performed before the test of the main effects. If the cross-correlation test fails, there
is no need to check the autocorrelations for &, One should reidentify the transfer function
v(3)and repeat the iterative model-building steps until a satisfactory model is obtained.
14.3.5 An Empirical Example

The Data The Lydia Pinkham annual data listed as Series W12 in the appendix and plotted in Figure 14.3 provide an interesting case for studying the dynamic relationship between
advertising and sales, Lydia Pinkham's Vegetable Compound was first brewed and prepared
\ \/
011900
1910
1920
1930
Advertising
1940
1950
1960
Year
FIGURE 14,3 Lydia Pinkham annual advertising and sales data (1907-1960).
336
CHAPTER 1 4 Transfer Function Models
by the lady of the same name as a home remedy for her friends. The compound, an herbal
extract in atcoholic solution, was considered to be effective against "women's weakness."
Additional medical claims followed with the marketing success of the compound. The first
commercial sale of the medicine was in 1873, when sales reached $500,000. A part of the
profits was reinvested immediately into high intensity advertising campaigns. Records of a
court litigation involving a family argument produced a complete listing of the firm's annual
advertising expenditures and sales over the period 1908-1935. Palda (1964) was later able to
obtain the data for 1907-1960.
Based on Palda (1964), the product remained essentially unchanged in taste, form, and
action through the period in question, although in 1914 the solid content of the liquid compound was increased sevenfold at the prodding of the Internal Revenue Service, which was
considering an alcohol tax on the compound. There were no close product substitutes on the
market. The company relied exclusiveIy on advertising to increase primary demand for the
product; no other marketing instruments, such as a sales force, credit, or quantity discounts,
were used. The advertising budget was primarily for newspaper copy, but the company began to experiment with radio and television in the years following World War II. There were
some changes in the character of the advertising copy used because of orders fiom the U.S.
Food and Drug Administration in 1925 to cease and desist from continuing to make strong
medical claims. Furthermore, the U.S. Federal Trade Commission in 1938 was also examining the medical uses of the compound. Finally, the company was able to demonstrate the
possibility that the active herbal ingredients had some medical benefits, and in 1941 the FTC
permitted the company to strengthen its advertising copy.
A Transfer Function Model between Advertising and Sales A strong relationship has been demonstrated (Clarke, 1976) between current sales and present and past
advertising. Such lagged advertising effects on sales may result from delayed response to the
marketing effort, to a holdover of new customer demand, or to an increase in demand from
existing customers. In addition, advertising may have a cumulative effect on demand. A
transfer function model using sales Y,as an output series and advertising expenditures X,as
an input series was first suggested by Helmer and Johansson (1977) to model the Lydia
Pinkham data. It was reexamined later by Heyse and Wei (1986). For the purpose of forecasting comparisons, they hold back the last 14 observations and use only the first 40 observations (1907-1946) for model building, as adopted in the following.
1 . Prelvhiteriirtg of the advertising series. The first step in building a transfer function
model is to prewhiten the input series: advertising expenditures. Examination of the
sample ACF and PACF, shown in Table 14.4, indicates that the original series X,is nonstationary but its differences, x, = (1 - B)X,,are stationary and can be modeled as an
AR(2) model. The fitted model is
( 1 - .07B
+ .41~')x, = a,,
(14.3.27)
where x, = (1 - B)X, and the a,are the white noise with mean 0 and 2; = 52,601. Diagnostic checking does not reveal any inadequacies in the model. Thus, the prewhitening filter
for x, will be used to filter the differenced output series, y, = (1 - B)Y,, where we note that
the original series of sales is also nonstationary.
337
TABLE 14.4 Sample ACF and PACF of the advertising series X,.
(a) f i k and
4mkfor X,
7
(b) filr and t$bh for x, = (I - B)X,

5
6
7
8
10
11
12
10
11
12
TABLE 14.5 Sample CCF and impulse response function for the
prewhitened input and the filtered output series (i?, = 229.35 and
Gp = 287,54).
Lag (4
CCF(L,3(k))
Impdse response function (9,)
2, Identijcatiort of the imptrlse response functiort and trarrsferfunctiort. Table 14.5 gives
sample cross-correlation,
and the impulse response function, Gk = (Gfi/&a)bap(k),
for the prewhitened input senes
a, = (1 - . 0 7 ~ - +. 4 1 ~ ~ ) x , , where &, = 229.35,
and the filtered output series
p,
= (1 - .07B
+ .41g2)y,,
where G p = 287.54,
(14.3.28)
338
The pattern of the sample impulse response function Ck in Table 14.2 suggests a transfer
function with b = 0,r = 1, and s = 0, i,'e.,
If only the first two large impulse weights Go and GI were retained and the remaining
smaller ones were ignored, then, from Table 14.1, a transfer function with b = 0, r = 0,
and s = 1 is also a possibility. That is,
3. Ide~ifijcatioaof the noise niodel. Once a tentative transfer function is obtained, we can
obtain preliminary estimates of the parameters in the transfer function by utilizing the
relationships between the impulse response weights and the parameters given in Equation (14.1.4). For the transfer function given in (14.3.29), we have
Thus,
The estimated noise series is then obtained by
where y, = (1 - B)Y, and x, = (1 - BJX,.Table 14.6 shows the sample ACF and PACF
of the estimated noise series. The pattern indicates a white noise model. To be conservative, however, one can try an AR(1) model
TABLE 14.6 Sample ACF and PACF for the estimated noise series.
339
and check the result of estimation in the combined transfer function-noise model to see
whether = 0.
4. Estimation of the transfer fitnctiotz model. Combining the above results, the following
two tentative transfer function models are obtained.
and
These models were in fact originally suggested by H e h e r and Johansson (1977);.Estimates

of the parameters and their associated standard errors are as follows:
Equation (14.3.33)
Parameter
Estimate
St.E.
Equation (14.3.34)
Parameter
Estimate
The results indicate that the parameters S1, wl,and

models, and one could consider the following model:
(1
- B)I; = w o ( l - B)X, + a,.
St.E.
coufd be deleted from the
(14.3.35)
The fitted model becomes
with 6: = 46,935.
5. Diagnostic checking. Final adoption of a proposed model requires study of the residuals.
Specifically, one has to check whether the a, are indeed white noise and are independent
of the input series X,and hence independent of the prewhitened input series apThe first
11 lags of the residual ACF of ,& and the CCF between the prewhitened input a,and the
residuals 2, fiom Equation (14.3.33) are shown in Table 14.7. Inspection of these two
functions indicates that both the ACF and CCF are zero at these lags; hence, one might
conclude that Equation (14.3.33) is adequate. Similar results hold for the residual ACF
and the CCE; between the prewhitened input and the residuals from Equations (14.3.34)
and (14.3.35).
340
TABLE 14.7 Sample ACF and CCF for the residuals from Equation (14.3.333,
Reexamination of the Transfer Function Model Are the transfer function models in Equations (14.3,33), (14.3.34), and (14.3.35) really adequate for the Lydia
Pinkham data? Because in practice one often considers only a reatizable system, i.e., a
causal transfer function model, the above procedure relied only on nonnegative lags. This
approach has been widely used in empirical research. Unfortunately, these procedures are
wrong; hence, the resulting conclusions are dubious. Because the ACF is symmetric
about zero, one needs consider only its nonnegative lags. As mentioned in Section 14.2,
however, the CCF is generally asymmetric and measures both the strength and direction
of an association. To see the complete picture of the relationship between two time series
x, and y,, it is important to examine p,(k) for both positive lags k > 0 and negative lags
k < 0. A causal transfer function model exists between series x, and y, only when
pq(k) =: 0 for k > 0 or pv(k) = 0 for k < 0. Series x, is said to cause series y,, denoted
by x, -+ y,, if pq(k) # 0 for some k > 0 and p,(k) = 0 for all k < 0; series y, is said to
cause series x,, denoted by x, +- y,, if pq(k) # 0 for some k < 0 and p9(k) = 0 for all
k > 0. A feedback relationship between x, and y, exists, denoted by x , .H y,, if p,(k) # 0
for some k < 0 as well as for some k > 0. Two series x, and y, are said to have a contemporaneous relationship if p,(O) # 0. They are said to be related only contemporaneously
if p,(O) # 0 and pry(k) = 0 for all k # 0.
Now, fet us reexamine the Lydia Pinkham data. The cross-conelations between the
prewhitened input series a,and the filtered output series j3, at both negative and nonnegative
lags are shown in Table 14,8 and plotted in Rgure 14.4. The pattern clearly indicates a contemporaneous relationship, fiap(0) = .51, and the presence of a lagged association from
sales to advertising, &(- 1 ) = .43. Because pap(k) # 0 for k = 0 , - I and paB(k) = 0 for
k > 0 at a 5% significance level, we have sales (Y,)-t advertising (X,) rather than advertising (X,)+ sales (Y,).This phenomenon happens because advertising budgets are often set as
a percentage of earlier sales.
TABLE 14.8 Sample CCF bettveen the prewhitened input and the filtered output series.
&(k)
St. E.
.23
.18
.19
.18
.24
.I8
.16
.I7
.43
-17
.51
.17
.32
.17
.ll
.I7
-.09
.18
.00
.18
-.26
.18
341
FIGURE 14.4 Sample CCF between the prewhitened input and Htered
output series with hvo standard error limits.
TABLE 14.9 Sample CCF between the prewhitened input series and the residuals.
( k )
St.E.
.32
-18
.06
.18
-19
-18
-05
.17
.46
.17
-.01
-17
.02
.18
-.04
.18
-.I4
.18
-.I0
.18
-.31
.18
Equations (14.3.331, (14.3.341, and (14.3.35) are constructed without using the information contained in the CCF at negative lags. As a result, although the residual ACF indicates that the residuals are white noise, the more detailed CCF between the prewhitened
input series and the residuals from Equation (143,331 as shown in Table 14.9 strongly
suggests that the model is inadequate. The significant spike, ;,-,(-I),
at lag k = -1
implies that there is a feedback relationship from sales to advertising at one period which
is not properly accounted for by transfer function models such as (14.3.33), (14.3.34), or
(14.3.35).

Once a transfer function model is found to be adequate, it can be used to improve the forecast
of the output series Y, by using the past history of both the output series Y, and the associated
input series X,.That is particularly tnre if the input series is a leading indicator.
342
CHAPTER 14 Tsansfer Function Models
14.4.1 Minimum Mean Square Error Forecasts

for Stationary Input and Output Series
Suppose that Y,and X,are stationary and are related in the stable transfer function model
and
where w(B), S(B), OfB),+(B), +x(B), and O,(B) are finite order of polynomials in B; the roots
of 6(B) = 0, d ( 3 ) = 0, 4(B) = 0,+x(B) = 0, and B,(B) = 0 are all outside of the unit circle;
and a, and a, are independent zero mean white noise series with variances ui and C T ~ ,
respectively. Let
and
We can rewrite (14.4.1) as
2(1)
02
03
$,
= 1. Thus,
= Zj-o ujs+,-j + XjEo $ju,+l-i.
Let
=
*
CO
Zj=Oui+jat-j 4- Zj=o$ & j ~ , - ~be the I-step ahead optimal forecast of Y,+l. Then the fore-
where
00
cast error becomes
343
The mean square of the forecast error E[X+[ - i(l)j2

is given by
In other words, the minimum mean

which is minimized when
lcLj = ul+j and $f~i+~
=
*
square error forecast x(1) of Yt+[at time origin t is given by the conditional expectation of
Y,+,at time t. Because E(Y,+, - q(I)) = 0, the forecast is unbiased and the variance of the
forecast is given by
A
14.4.2 Minimum Mean Square Error Forecasts

for Nonstationary Input and Output Series
By the same argument used in Section 5.2.2 for a general A D A model, the minimum
mean square error forecast of Yt+[ at time f is also given by the conditional expectation of
Y,+Iat time t even if Y , and X, are nonstationary. That is true as long as Y,and X, can be
reduced to stationary series by some proper transformations and differencing so that they
follow some ARTNIA(p, d, q) models. Likewise, Equation (14.4.8) holds also for the nonstationary case, and the weights 19 and qj in (14.4.8) can be derived by the same method as
Equation (5.2,26) used to compute the rC;. weights for the ARIMA model. Specificafly, suppose that Y, and X, are nonstationary but that ( 1 - B ) ~ Yand
, ( 1 - B ) ~ x are
, stationary and
are related in the stable transfer function model
and
where w(B), 6(B),8(B), +#3),

(14.4.9), we have
8,(B), a,, and a, are defined as in (14.4.1)and (14.4.2).From
and Y, is the sum of two independent processes. The required uj weights come from the first
term ( ~ ( B ) / ~ ( B ) ) Band
~ x ,the
,
weights come from the second term (B(B)/#(B)(f- B ) ~ ) u , .
344
To derive the Gj weights, we let
and express el in an AR representation that exists because the roots of d(B) = 0 are outside
the unit circle. Thus,
T ( ~ ) ( L=
I )a,,
~,
(14.4.13)
where
By the results from (5,2.15) through (5.2.26),the II;.weights are calculated recursively from
the
weights in (14.4.14) as follows:
and $0 = 1.
To derive the uj weights, we first express Xt in (14.4.10)
in terms of its AR representation
that again exists because the roots of 8,fB) = 0 are assumed to be outside of the unit circle. Thus,
T ( ~ ) ( B ) X=, a,,
(14.4.16)
where
Next, we use the result (5.2.26)to compute the following +)'I
weights:
. ..
= I. The required ui weights for j = 0, 1,

, (1 - I),are thus equal to the coeffiand
cient of B] in the following expression that exists because w(B) is a finite-order polynomiai
in B and the roots of 6(B) = 0 are outside of the unit circle:
It can be easily seen that if b # 0,then uj
in o(B)= (oo- olB- . . . - o,BS).
0 for j
< b; if b
= 0, then uo # 1 unless
=1
345
For the actual computation of forecasts, we consider the general transfer function model
(14.4.9) and rewrite it in the difference equation form
c(B)I; = d(B)Xt-+
+ e(B)a,,
(14,4,20)
where
and
Thus,
and Gt is generated either by Equation (14.4.20) or (14.3.19) or as the one-step forecast

error 1; - i - ~ ( l ) It. is clear that for 1 > (g + r), the forecast i ( l ) of Y,+)depends only on
the past history of Y and the present and past history of X. The required forecast of the input
series can be easiIy obtained from Equation (14.4.10) as discussed in Section 5.3 for the
univariate AIUMA models.
346
14.4.3 An Example
The numerical computation of the minimum mean square error forecasts for stationary input
and output series is simpler than that of nonstationary input and output series. For illustration,
we consider the nonstationary case of the Lydia Pinkham example. Although the causal transfer
function model has been found inadequate in describing the relationship between the Lydia
Pinkham annual advertising budget and sales, the model equations themselves can be used to
illustrate the computational aspect of the forecasting procedure described above.
EXAMPLE 14.2 From (14.3,33), we have that
and
(1 - .07B
where 6:
+ .41B2)(1 - B)X, = a,,
(14.4.24)
45175, and 6: = 52601. To compute the forecast, we rewrite (14.4.23) as
Hence,
The one-step ahead forecast from the forecast origin n = 40 is
where k m ( l ) = Xll if it is h o w ; othenvise, it equals the one-step ahead forecast of Xdl

determined from its own past through Equation (14.4.24). Now, from the data and model fitting,
we have
= - 101.05, Y38= 2518, Y39= 2637, Ya = 2177, X39= 1145, X40= 1012, and
Xjl = 836. Thus,
To obtain the forecast variance, we need to compute the weights uj and t,bj. Note that Equation
(14.4.23) can be written as
14.4 Forecasting Using Transfer Function Modeb
Let
and
a(a)(~)=
e , a,,
where
Thus,
and
Using (14.4.151, we have the following i,bj weights:
To obtain the tij weights, we note that
where
347
348
Hence,
and
np)= 0,
for j 2 4.
Using (14.4.181,we have
and
Thus, by (14.4.19),the uj weights for j = 0, 1, . . 1 - 1 are equal to the coefficient of Bjin

the following expression:
14.5 Bivariate Frequency-Domain Analysis
349
Having obtained the uj and weights, we can now calculate the one-step ahead forecast error
variance using (14.4.8)and get
The forecasts fm(l)and their forecast error variances V ( I ) for other 1 can be calculated
similarly. Reed from Section 14.3.5 that in fitting the data set, the last 14 observations
(1947-1960) were held back to examine the forecasting performance. For the 14 one-step ahead
sates forecasts using the actual one-step ahead advertising value, Helmer and Johansson (1977)
reported a mean squared forecasting error of 8912 for Equation (14.3.33)or (14.4.23) and 9155
for (14,3,34), both of which are considerably smaller than the MSE of 16,080 obtained from
a univariate time series model of sales. Because of a feedback relationship from sales to advertising at lag 1,it does not seem appropriate to assume that the next period's advertising budget is
known with certainty when forecasting the next period's sales. Rather than the actual advertising
levels, if one-step ahead advertising levels from Equation (14.4.24) are used, the corresponding
MSEs of the 14 one-step ahead sales forecasts are 15,177 for (14.3.33)or (14.4.23)and 14,044
for (14.3.34),respectively.
For more examples and discussion on forecasting using transfer function models, see
Liu (1987).

14.5.1 Cross-Covariance Generating Functions
and the Cross-Spectrum
Suppose that x, and y, are jointly stationary with the cross-covariance function y,(k) for
k = 0, & 1, f2, . . . . Define the cross-covariance generating function as
where the coefficient of gk is the kth cross-covariance between x, and y,, This equation is the
generalization of the autocovariance generating function introduced earlier. Specifically, y,,(B) =
yx(B) and yyy(B)= yy(B) are autocovariance generating functions for x, and y , respectively. If
03
the cross-covariancesequence yxy(k)is absolutely sumable, i.e., Zk=-lyxy(k)l < m, then its
Fourier &ansfom exists and is called the moss-spectrum (or cross-spectral density) between x,
aad y, and is given by
350
CHAPTER I4 Transfer Function Models
which is the generalization of the spectrum (or spectral density) for the univariate series as
introduced in Section 12.2. Specifically, fx;(o) = f,(w) andf,,(w) = &(o) are spectrums for
x , and y,, respectively. Note that even when yxy(k) is real for the real processes x, and y,,
f,,(w) will in general be complex because yxy(k) # yx,(-k). Hence, we can write
where cXy(u)and -q,,(u)
are the real and imaginary parts offxY(w).More precisely,
and
q+
,(o.)
l o o
=y J k ) sin ok.
2.rr k
2
= 4
(14.5.5)
The function cxy(@)is called the cospectrum, and q (a)is called the quadrature spectrum,
XY
of x, and y,. These functions, however, are difficult to ~nterpret.Alternatively, we can express
f,,(o) in the polar form
where
and
The functions Axy(o) and +xy(o) are known as the cross-amplitude spectrum and the phase
spectrum, respectively. Two other useful functions are the gain function and the coherence.
The gain function is defined as
which is the ratio of the cross-amplitude spectrum to the input spectrum. The coherence (or
the squared coherency) is defined by
whch is essentially the standardized cross-amplitude specaum.
351
In practice, several cross-spectral functions are needed to describe the relationship between
two series in the frequency domain. In general, the cross-spectrum between two series is more
easily analyzed through the gain function, the phase spectrum, and the coherence. Hence, they are
possibly the three most commonly used cross-spectral functions in kequency domain analysis.
14.5.2 Interpretation of the Cross-Spectral Functions

To see the implications of the above functions associated with the cross-spectrum, we recall
from Section 12.1.4 that any stationary zero mean process can be written in terms of the
spectral or Cramer's representation. That is,
where dU,(o) and dU,,(w) are each orthogonal processes such that for w f A,
where dU:(o) and dU;(w) are the complex conjugates of dU,(w) and dUy(w),respectively. If
x, and y, are also jointly stationary, we have
Now, for the right-hand side to be a function of k only, the integrand in (14.5.14) must be zero
when w # A. Thus, in addition to (14.5.13), we also require cross-orthogonality, namely,
E [ d & ( w ) dU;(A)] = 0,
Therefore, letting A = o in (14.5.14), we obtain
for w # A.
(14.5.15)
352
Assume that x, and y, are both purely nondeterministic and jointly stationary zero-mean
processes with absolutely s u m a b l e autocovariance and cross-covariance functions, i.e.,
By the same argument that was used to derive (12.1.37), we have
and
where
and
E[dU,(w) dU;(A)] = 0,
for o # A.
Now, E [ d U , ( o )d q t o ) ] = Cov(dU,(w),dUy(o)). Thus, from (14.5.19) we see that the

cross-amplitude spectrum,A&) = If,,(w)!, measures the covariance between the w-frequency
component of x, and the o-frequency component of y,. Also,
which implies that the gain function GI,(@) is simply the absolute value of the standard least
squares regression coefficient of the o-frequency of x,. Similarly, we can write
353
Hence, K$(w) is square of the correlation coefficient between the o-frequency component
ofx, and the w-ffequency'component of yy,.clearly, 0 5 K$(o) 5 1. A value of ~ z ~ ( o )
close to one implies that the o-frequency components of the two series are highly linearly
)
zero implies that they are only slightly linearly related.
related, and a value of ~ & ( w near
Like the correlation coefficient between two random variables, the coherence is invariant
under linear transformations. To see, suppose that
and
Then, by (12.3,3), we have
or, equivalently, by f 14,5,17) and (14.5.18),

dU,(o) = a(e-'")
dU' (o)
and
Hence,
fWY(w)= ~l(e-~")~(e~~).A~(m).
It follows that when a(eio) and P ( e i w )are nonzero,
hence, the coherence is invariant under linear transformation.
354
CHAPTER 14 Transfer Function Modets

To learn the interpretation of the phase spectrum, we recall from Section 12.1.4 that for
any o , dUx(w) and dU,(w) are complex valued random variables. Hence, we can write
where A,(o) and +x(o) are the amplitude spectrum and the phase spectrum of the x, series.
Similarly,
and Ay(o) and $(w) are the amplitude spectrum and the phase specirum of the y, series. Thus,
E[Ax(w)Ay( o ) 1E [e'[+1(~)-4~(~)]]
,
where for simplicity we assume that the amplitude spectrum and the phase spectrum are
independent. Consequently, A,,(&) can also be thought as the average value of the product
of amplitudes of the o-frequency components of x, and y,, and the phase spectrum # x y ( ~ )
represents the average phase shift, [+,(w) - #y(w)j, between the w-frequency components
of x, and y,. In terms of a causal model y, = ax,-,
e, or x, = py,-,
a,, where there is no
feedback relationship between x, and y,, the phase spectrum is a measure of the extent to
which each frequency component of one series leads the other. The w-frequency component
of x, leads the o-frequency component of y, if the phase +xy(w)is negative. The w-frequency
component of x, lags the o-frequency component of y, if the phase +xy(w)is positive. For a
given +xy(o), the shift in time units is #~,(w)/w. Hence, the actual time delay of the component of y, at frequency o is equal to
which is not necessarily an integer.

From (14.5.6) and (14.5.8), it is clear that if &,(o) is a solution to (14.5.8), then
+ x y ( ~=
) +iy(w) f 2 r k for any integer k is also a solution of (14.5.8). Thus, the phase
spectrum is defined only mod(2n). In practice, however, if we take the given function to be
nonnegative, the range of +x,(w) is conventionally restricted to the interval from -.rr to w.
The value of #x,(o), which is the argument of fxy(o), i,e.,
is then the angle between the positive half of the c,?(w) axis and the line joining the origin to
the point (cx,(o), -q,,(w)). It follows that +xy(w) and -qXy(w) have the same sign. Furthermore, because +,,(o) is an odd symmetric function that is zero at o = 0, the phase spectrum
is normally graphed only for o over the range 0 5 w 5 ?r,
355
From (14.5.15) the components of x, and y, are independent at different fkquencies, Thus,
we need only examine the rilationship between the corresponding components of xt and y, at each
frequency as emphasized above in the phrase such as "the w-frequency components of x, and y,."
14.5.3 Examples
EXAMPLE 14.3 Assume that x, and y, are jointly stationary and uncorrelated processes.
Then yxy(k) = 0 for all k. Hence, by (14.5.2a), we have
&(o) = 0,
which implies that c,,(w) = 0, q&)

for all w.
for dl w ,
= O,AX@) = 0, &.,y(o) = 0, GX@) = 0, and ~ & ( o )0
EXAMPLE 14.4 Consider the model
where we assume that nl

processes. Then
> 0 and x, and e, are jointly independent zero mean stationary

yV(k) = E ( x f ~ t + k )
=
E[xt(ffxttk-,
= ayx(k - n i ) .
Hence, by (14.5.2a),
Equation (145.33) implies that
+ ett~>]
356
CHAPTER I4 Transfer Function Modeb
FIGURE 14.5 The phase spectrum for ExarnpIe 14.4, with m = 1.
This phase spectrum shown in Figure 14.5is a straight line with slope (-m) and is worthy
of further discussion, Because the phase function is negative for all o , the x, series leads the y,
series at all frequencies. By Equation (14.5.30), the actual time delay of they, series equals m
at all frequencies. In general, the phase spectrum between two series may not be a straight line,
Thus, the time dday between two series may change from frequency to frequency.
In Figure 14.5, if m instead equaled 4, then following the convention of restricting the
phase to the interval ( - T , T ) we would have a graph Like Figure 14.6. In this case, the slope
of each line is (-m).
The gain function for this example is given by
which equals, as expected, the absolute value of the least squares regression coefficient. For the
coherence, we also need f,(w), which, by (12.2.22)and (12,3,3), equals
FIGURE 14,6 The ~ h a s espectrum for Example 14.4, with m = 4.
357
Hence,
KG(@)
If e, = 0,then
= 1 for all frequencies o.This result is also expected because, in this
case, x , and y, satisfy the exact linear relationship y, = ax,-,.
14.5.4 Estimation of the Cross-Spectrum
For a given set of time series data x, and y,, for t = 1, 2, . . , and n, we have learned from
Chapter 13 that the spectrumfJo)for the x, series can be estimated by the smoothed spectrum
where K ( k ) is the sample autocovariance function, WX(k)is the lag window, and M, is the
truncation point. With similar notation, the smoothed spectrum for the y, series is given by
Simifarly, we can estimate the cross-spectrum by
where Txy(k)is the sample cross-covariancefunction and bVx,(k) and Mxy are the corresponding
lag window and the truncation point for the smoothing. Clearly, we can write jXy(m) as
where
358
CEIAPTER 14 Transfer Function Models

and
1
&(o) = 'GV,Y?X,(k)sin wk
2 g k=-hiJy
are the estimated cospecirum and quadrature spectrum, respectively.

The truncation point MXyand the lag window Wx,(k) are chosen in a similar way to those
used in the univariate spectral analysis. Rigorously, the truncation points M,, My, and M,, and
the lag windows W,(k),Wy(k), and Wxy(k)do not necessarily have to be the same and should
be allowed to vary depending on the characteristics of the univatiate as well as the joint series.
In practice, however, because of the simplification of the sampling properties of the spectrum
estimates, they are often chosen to be the same. That is especially true for the fag window.
Having estimated the cospectrum and quadrature spectrum, we may estimate the crossamplitude spectrum, the phase spectrum, the gain function, and the coherence as follows:
and
In the estimated cross-spectrum, the lag window places heavier weight on the sample crosscovariances around the zero fag, Thus, if one series leads another and the cross-covariance
does not peak at zero lag, then the estimated cross-spectrum will be biased. This bias is especially severe for the estimated coherence, Because the coherence is invariant under the linear
transformation, the bias can be reduced by the proper alignment of the series. That is, two
series can be aligned by shifting one of the series by a time fag T so that the peak in the
cross-correlation function of the aligned series occurs at the zero lag. The time lag r can be
estimated by the lag corresponding to the maximum cross-correlation or by Equation
(14.5.30) and the slope of the phase spectrum. It is also worth noting that the estimates of
cross-amplitude, phase, and gain are unreliable when the coherence is small.
We do not discuss the sampling properties of cross-spectral estimates. Interested readers
are referred to the books by Jenkins and Watts (1968), Hannan (19701, Brillinger (1975),
Priestley (1981), and others. Instead, we discuss the relationship between the cross-spectrum
and the transfer function model in the next section.
359

14.6.1 Construction of Transfer Function Models
through Cross-SpectrumAnalysis
Consider the two-sided general transfer function model
=.
03
v(B)x,
+ n,,
(14.6.lb)
where v(B) = Xj=- v j ~such

j
that Zj,Ivj[ < m. The series x, and nt are assumed to
be independent and jointly stationary zero mean processes. Then,
CO
Multiplying both sides of (14.6.2) by Bkand summing, we get
By (14.5.2b), we have
Hence,
for all w such thatf,(o) # 0. The function u(w) is called the frequency response function of
the linear (transfer function) system. Equation (14.6.5) shows that the frequency response
function is the ratio of the cross-spectrum to the input spectrum. The impulse response function vkcan be obtained from
b(o)eiwi
TI
vk =
do.
360

To derive the noise spectrum (or error spectrum)f,(w), we again use the results from
Equations (12.2.22) and (12.3.3) and obtain
Hence,
where we note, from (14,5,19), that f$(o)= &(o). The noise autocovariance function
y,(k) can then be obtained from
By substituting the estimates of &(o),f,(o),

and f,(w) as described in Chapter 13 and
Section 14.5.4, we obtain estimates of the impulse response function v k and the noise autocorrelation function p,(k). These estimates can then be used to identify the transfer functionnoise model as discussed in Section 14.3. In other words, the bansfer function model can
also be constructed through the use of cross-spectrum analysis.
14.6.2 Cross-Spectral Functions of Transfer Function Models

The gain function between the input x, and the output y, can be easily seen from (14.5.9) and
(14.5.5) as
Similarly, becausef,(w) is nonnegative and real-valued, we have, from (14.5.31) and (14.6.5), that
Hence, the phase spectrum between the input series x, and the output series y, depends only
on the phase shift of the frequency response function v(o).
14.7 Multiple-Input Transfer Function Models
361
For the coherence, we note, from (14.6.4) and (14.6.7), that
Thus, the coherence is a measure of the ratio of the noise spectrum to the output spectrum at
frequency w. The coherence K$(W) approaches unity as the ratio f,(o)/f,(w) approaches
zero, Conversely, the coherence approaches zero as the ratio fn(w)/'(w) approaches unity.
From (14.6.7), because the output equals the buried signal plus the noise, we can form the
noise-to-signal ratio
wheref,(o) = Ivfo)12fx(u) represents the signal of the system. The noise-to-signal ratio is
large when the coherence is small, and vice versa. Equivalently, it can be shown that
Hence, the coherence is also a direct measure of the signal-to-noise ratio f,(w)/f;,(o) at
frequency o.The larger the coherence, the higher the signal-to-noise ratio.
The noise-to-signal ratio or, equivalently, the signal-to-noiseratio can be used as a criterion
for model selection. Other things being equal, a model with a uniformly smaller noise-to-signal
ratio and hence a Iarger signal-to-noiseratio is obviously more preferable.
14.7 Multiple-Input Transfer Function Models

More generally, the output series may be influenced by multiple input series, and we have
the multiple-input causal model
362
CHAPTER 14 Transfer. Function Models
or
is the transfer function for the jth input series xjt and a, are
where vj (B) = u~(B)B~~/S~(B)
assumed to be independent of each of the input series xj,, j = 1,2, , . ,k. If the input series
xi, and xi, are uncorrelated for i # j, then all aspects of the time domain and frequency
domain analysis of the single-input transfer function model discussed above can be extended
and applied with no difficulty. For example, we can construct the transfer function vj(B)
relating y, and xj, separately, one at a time. The noise model is then identified through the
study of the generated noise series
Again, if the original series are nonstationary, some transformations must be taken to
produce stationarity.
The model building is much more complicated when the input series xi, and xjt for i # j
are correlated. Liu and Hanssen (1982) suggested that a common filter be chosen from the
AR polynomials of the input series with roots dose to one to reduce their cross-correlations.
We believe, however, that much more research is still needed in the model building of
multiple-input transfer function models when input series are cross-correlated.
The intervention and outlier models introduced in Chapter 10 are special cases of a
transfer function model. Ln other words, some input series xjt could be intervention or outlier
variables. As an example, we refer readers to Pack (1979), who fitted the Lydia Pinkham
monthly data to a transfer function model and incorporated two price increases as isolated
intervening events.
We close this chapter with the following remarks.
1. The reader may notice the close relationship between the transfer function models and
the multiple regression models. In fact, one may think that the multiple regression models are special cases of the transfer function models because the noise processes in the
regression models are often assumed to be uncorrelated white noise, The relationship is
even more apparent if a transfer function model is written in terms of the difference equation form. There is, however, a vital difference between the transfer function and regression models in terms of modeling. The construction of both the transfer function and the
noise models is important in the process. Thus, although the transfer function is linear,
the resulting model is, in general, highly nonlinear in terms of its parameters, unlike the
regression modeling, where linear models are usually the desired end products. In other
words, linearity in parameter is a normal case with regression models but a special case
with transfer function models.
2. One important element in a time series model is the time unit used in the model. To
obtain proper and valid statistical inferences, the choice of an appropriate time unit is
Exercises
363
very important. If the model for a phenomenon under investigation is regraded

appropriate in terms of a certain time unit, say a month, then proper inferences about
the underlying model should be drawn in terms of this same monthly time unit.
Improper use of data in some larger time unit such as quarterly or annual quantities
could lead to a substantial information loss in parameter estimation and forecasting.
More seriously, it could cause a misspecification of lag structure in causality study.
In fact, Tiao and Wei (1976) and Wei (1981) show that a causal relationship is often
destroyed by temporal aggregation. This problem is addressed in more detail later
in Chapter 20.
Calculate and plot the impulse response weights for the following transfer functions:
(a) v ( B ) = B / ( l - .8B),
(b) v(B) = (3 2B)/(1 .6B),
( c ) v ( B ) = (B ~ ' ) / ( 1- .8B - 6 ~ ~ ) .
14.2 Consider the A R M A ( 1 , l ) process
where a, is a white noise series with zero mean and constant variance 1 . CalcuIate
the cross-correlation function between a, and 2,.
= (2 + B)x, + rt,, where x, and a, are unconelated white noise
processes, each with mean 0 and variance 3. Find the CCF between x, and y,.
14.3 Suppose that y,
14.4 Consider the transfer function model
and (1 - 4 B ) x , = a,, where oo = 1, ol = -2, 6 = .5, 4 = .4, 8 = .2,

u: = .53,ui = .35,agg = a100 = 0, and the last few observations on x, and y, are
as follows:
Calculate the forecasts $loo ( I ) and their forecast error variances for 1 = 1, 2, and 3.
364
14.5 Consider the following monthly housing sales X, and housing starts Y, between
January 1965 and December 1975:
Housing starts Y, (read across) in thousands of units.
52.149
47.205
82.150
100.931
98.408
80.876
85.750
72.351
61.198
46.561
84.748
79.828
69.068
69.362
59.404
40.157
40.274
66.592
79.839
87,341
78.194
81.704
69.088
47.026
45.234
86.806
81.424
86.398
82.522
80.078
51.300
47.909
71.941
84.982
91.301
71.504
68.039
55.069
42.827
33.363
74.848
83.007
75.461
77.291
75.961
54.856
58.287
91.584 116.013 115.627
102.149 102.882
92.904
80.362
76.185
135.167 131.870 119.078 131.324 120.491
77.105
73.560 105.136 120.453 131.643
84.504
86.004
70.488
46.767
43.292
96.340
99.318
90.715
79.782
73.443
39.791
39.959
62.498
77.777
92.782
84.51 7
93.826
71.646
55.650
Housing sales XI (read across) in thousands of units.

38 44 53 49 54 57 51 58 48 44
42
29
35
34
34
45
51
54
36
29
43
32
43
40
29
49
56
58
42
33
53
41
46
43
36
62
60
66
53
44
49
44
46
42
42
62
65
63
53
54
49
49
43
43
43
58
64
64
55
56
40
47
41
44
44
59
63
60
48
51
40
46
44
39
44
64
63
53
47
51
36
47
47
40
48
62
72
52
43
53
29
43
41
33
45
50
61
44
39
45
31
45
40
32
44
52
65
40
33
45
42
26
34
32
31
40
50
51
36
30
44
37
23
31
32
28
37
44
47
28
23
38
(a) Plot and examine the two series in the same graph.
(b) Prewhiten the X, series and filter the Y, series.
(c) Compute the sample CCF (for lags from -24 to 24) between the prewhitened
and filtered series. Plot them together with f2 standard errors.
(d) Identify a tentatively transfer function-noise model for the data set.
(e) Estimate the parameters of your tentative model and perform diagnostic checking for model adequacy.
(f) Forecast housing starts and attach their 95% forecasts limits for the next 12 months.
14.6 Suppose that y, = x, + . 8 ~ , - ~ e, and x, = a,, where a, and e, are independent white
noise processes, each with mean zero and varian'ce 5. For the above model, find
(a) The cross-covariance function.
(b) The cross-spectrum.
(c) The amplitude.
Exercises
365
(d) The phase.

(e) The gain function.
(f) The coherence.
14.7 Given
yt = ( x ,
+ x,-I
3- ~
~ 't.- x,-3)
2
4- nt,
where x, and n, are assumed to be independent and jointly stationary zero mean
processes, find the frequency response function, gain function, and phase function.
14.8 For the data set given in Exercise 14.5, calculate, plot, and discuss
(a) The cospectrum.
(b) The phase spectrum,
(c) The gain function.
(d) The coherence.
Time Series Regression

and Garch Models
Regression models are without a doubt the most commonly used statistical models. The
standard assumptions associated with these models, however, are often violated when time
series data are used. In this chapter, we illustrate the use of some t h e series techniques in
regression analysis. In the process, we will introduce the autoregressive conditional heteroscedasticity (ARCH) and generalized autoregressive conditional heteroscedasticity
(GARCH) models which have been found to be useful in many economic and financial
studies.

--
--
In standard regression analysis, we consider the model
or, equivalently,
where Ytis a dependent variable, Xi is a (1 X k) vector of independent variables, is a (k X 1)

vector of parameters, and E, is an error term often assumed to be i.i.d. N(0, u:). Under these
standard assumptions, it is well known that the OLS estimator of P is a minimum variance
unbiased estimator. When Xi is stochastic in (15.1.1), conditional on Xi,the results about the
OLS estimator of P also hold as long as E, and Xi are independent for all s and r.
When Xi is a vector of k lagged values of Y,, i.e,, Xi =
. ,,)&I
=
. . . ,& ) I , and et is white noise, the model in (15. f .l)becomes an AR(k) model
(x-l, . .
367
The OLS estimator of P is asymptotically unbiased and consistent,As shown in Section 7.4,
however, this result no longer holds when the E , are autocorrelated. When that is the case, the
estimator is no longer consistent. This caveat is important, When time series variables are used
in a model, it is the norm rather than the exception that the error terms are autocorrelated. Even
in univariate time series variables analysis when the underlying process is known to be an AR
model as in (15.1.2), the error terns E, could still be autocornelated unless the correct order of
k is chosen. Thus, a residual analysis is an important step in regression analysis when time
series variables are involved in the study.
Many methods can be used to test for autocorrelation. For example, we can either use
the test based on the Durbin-Walson statistic or we can perform the residual autocorrelation
analysis discussed in Chapter 6. The latter method is particularly useful. Based on the residuals from a preliminary analysis of (15.1.1), an underlying model can be identified for the
error term using the identification techniques discussed in Chapter 6. A final analysis can
then be performed on a modified model with a specified model substituted for the error term.
In time series regression analysis, the following model is often proposed to account for the
autoconelated errors:
for t = 1, 2,
. . . ,n, where
and then, are i.i.d. N(0, (r2).

Let
The matrix form of the model in (15.1.3) is
where S follows a multivariate normal distribution N(0, 2).When PI, . . . , qp and a2 are
known in (15.1.4) and the model is stationary, Z can be easily calculated, The diagonal
element of X is the variance of E,, the jth off-diagonal element corresponds to the jth autocovariance of et, and they can be easily computed as shown in Chapter 3. Given Z, the generalized least squares (GLS) estimator
is known to be a minimum variance unbiased estimator.

Normally, we will not know the variance-covariance matrix T: of because even if e,
follows an AR(p) model given in (15.1.41, the u 2 and AR parameters q are usually
unknown. As a remedy, the following iterative GLS is often used:
368
CHAPTER 15 Time Series Regression and Garch Models
1. Calculate OLS residuals 2, from OLS fitting of (15.1.3).

2. Estimate yrj and a2for the AR(p) model in (15.1.4) based on the OLS residuals, it,using
any method discussed in Chapter 7, For example, a simple conditional OLS estimation
can be used.
3. Compute Z from (151.4) using_the values of q and u2obtained in step 2.
4. Compute the GLS estimator, P = (X'B-lX)-l x'Z-IY, using B obtained in step 3.
Compute the residuals E, from the GLS model f~ttingin step 4, and repeat steps 2
through 4 until some convergence criterion (such as the maximum change in the =timates between iterations becoming less than some specified quantity) is reached.
More generally, the error structure can be modified to include an ARMA model. The
above GLS iterative estimation can still be used except that a nonlinear least squares estimation instead of OLS is needed to estimate the parameters in the error model. Alternatively, by substituting the error model in the regression equation, (15.1.31, we can also
use the nonlinear estimation or maximum likelihood estimation discussed in Chapter 7
to jointly estimate the regression and error model parameters
and q ' s as we did in
Chapter 14.
Although the error term, e,, can be autocorrelated in the regression model, it should be
stationary. A nonstationary error structure could produce a spurious regression where a significant regression can be achieved for totally unrelated series as shown in Granger and
Newbold (1986), and Phillips (1986). In such a case, one should properly difference the
series before estimating the regression.

One of the main assumptions of the standard regression analysis and regression models with
autocorrelated errors introduced in Section 15.1 is that the variance, m2, of the errors it, is
constant. In many practical applications, this assumption may not be realistic. For example,
in financial investment, it is generally agreed that stock markets' volatility is rarely constant
over time. Indeed, the study of the market volatility as it relates to time is the main interest
for many researchers and investors.
Such a model incorporating the possibility of a nonconstant error variance is called a
heteroscedasticity model. Many approaches can be used to deal with heterocedasticity. For
example, the weighted regression is often used if the error variance at different times is
known. In practice, however, the error variance is normally unknown; therefore, models to
account for the heteroscedasticity are needed.
Let us first consider the standard regression model with uncorrelated error
where ct = ttt and the n, are uncorrelated but have variances that change over time. Following
Engle (1982), let us assume that the error term can be modeled as
369
where the e, are i.i.d. random variables with mean 0 and variance 1,independent of past realizations of rz,,j, and
Given all the information up to time (f - I), the conditional variance of the nt becomes
which is related to the squares of past errors, and it changes over time. A large error through
n:-i gives rise to the variance, which tends to be followed by another large error. This phenomenon of volatility clustering is common in many financial time series.
Using the forecasting results from Chapter 5, we see that Equation (15.2.4) is simply the
optimal forecast of n f if n? follows the folloiving AR(s) model
where a, is an N ( 0 , a:) white noise process. Thus, Engle (1982) calied the model of the error
term ti, with the variance specitication given in (15.2.2) and (15.2.3) or, equivalently, in
(15.2.5) the autoregressive conditional heteroscedasticity (ARCH) model. More specifically,
it is an ARCH model of order s and is often denoted ARCHCs), The term has also been used
for the model for Y, in (15.2.1) with an error variance structure given in (15.2.2) and (15.2.3).
In financial applications, because the mean return is often expected to be related to the variance, Engle, Lilien, and Robins (1987) generalize (15.2.1) to
and call it the ARCH-in-mean or ARCH-M model.

To test for ARCH, one can first fit the ordinary least squares regression in (15.2.1) for
t = 1,2, , , , n. The fist step is to compute the OLS residuals i, = fit. Then, we can form
the series S: and examine its ACF and PACF to see whether it follows an AR model. Alternatively, we can, following Engle (1982), specify the following ARCS) model to ti::
- ..
for t = s 1, s 2, . . . , n and test Ho : B1 =

= 8, = 0.Under the null hypothesis that
the 11, are i.i.d. N(0, a'), we have Ho : = . = 8, = 0. Thus, if Ho is true we expect that
the estimates of Bj's are close to zero and that the value of the coefficient of determination R~
is small. Because
~2
regression sum of squares

total sum of squares '
370
CHAPTER 15 Time Series Regression and Garch Models
and under the null hypothesis
an ARCH phenomenon exists when the value of (n - S)R* is statistically significant.

A natural extension to the ARCH model is to consider that the conditional variance of
the error process is related not only to the squares of past errors but also to the past conditional variances. Thus, we have the more general error process
where the e, are i.i,d. random variables with mean O and variance 1, independent of past
realizations of n,-i,
and the roots of ( 1 B - . - #, Br) = 0 are outside the unit circle, To guarantee
u: > 0, we assume that Oo > O and that +i and Oj are nonnegative, The model for the error
tern given in (15.2.8) with the property (15.2.9) is known as the generalized autoregressive
condition& heteroscedasticity (GARCH)model of order ( I ; s) and is denoted by GARCH(c s),
This model was first proposed by Bollerslev (1986).Ciearly, when r = 0, the model reduces to
the ARCH(s) process.
One should not be misled by (15.2.9)and conclude that the conditional variance follows
an ARMA(c s) model with r being the AR order and s being the MA order. In fact, the model
in (15.2,9)is not a proper ARMA model. From Chapter 5, we see that in an ARMA process,
the associated error is a white noise process corresponding to the one-step ahead forecast
error. In (152.91, neither n f nor CT? plays this proper role. Let a, = (n: - u:) so that
a; = (nt - a,). We can rewrite (15.2.9)in terms of n! and a, as follows:
( 1 - arlB where nt = max(l; s),
+i
=;
. - a,Bn')n: = Oo -k (1 - rPIB -
- +r Br)ut,
(15,2.1 la)
0 for i > r, Qi = 0 for i > s,
and
We claim that a, is the associated white noise process for the process of n:; and therefore,
(15.2.10)or (15.2.11a) is a proper ARMA model, which follows because ~ , - ~ ( n :=) u f ,cr;
371
is the one-step ahead forecast of r z z , and a, is the corresponding one-step ahead forecast error.
In other words,
and
E (aiaj) = E (it - u3)( n f - uf) = ~ ( u f e f- a:)(crfej - uf)
= E [rrfu!(ef
= 0,
I ) (e2 - 111
(15.2.12)
for i f j,
where we note that the ef are i.i.d. x2 (1). Thus, the GARCH(c s) model in (15.2.8) and
(15.2.9) implies that 1t: follows an ARMA(m, r) model in (15.2.11a) with the AR order being
m = max(r; s).
From (15.2.10) or (15.2,11a), we see that this process will have a unit root if (1 - al
- . . . - am) = 0,i.e., if
= Eyil(4i Bi) =
2j=16j = 1 . In this case,
the model will be called an integrated GARCH process, or simply IGARCH.
From (15.2.3) and (15.2.9), it is clear that in the specification of ARCH, GARCH, or
IGARCH models, the effect of errors on the conditional variance is symmetric, i.e., a positive
error has the same effect as a negative error of the same magnitude. Moreover, because CT;
and n? are always nonnegative in (15.2.9), it is obvious that some restrictions on the coeffiand Bj are needed. To accommodate the asymmetric relation between many
cients Bo, +i,
financial variables and their volatility changes and to relax the restriction on the coefficients
in the model, Nelson (I991) proposed that
2pla;.
xj'_lr$j+
where qo = 1 and a, = a,/a,. The function g is chosen to allow for asymmetric changes,
depending on the sign of a,,in the conditional variance. For example, we may choose
In this case, when 0 < at < m, g(at) is linear in a, with slope (6 a);when -03 < a, < 0,
g(a,) is linear with slope (6 - a)).Thus, g(a,) altows the conditional variance to respond
asymmetrically to the rises and falls of the process. The specification of (15.2.13) implies that
u! will be positive regardless of the sign of the coefficients. The coefficients t,hj are often
assumed to relate to an ARMA specification to the model, i.e.,
a7
where Zj=o@jBj = $(B) = 6(B)/#(B). The polynomial $(B) is such that the roots of
4(B) = 0 are all outside the unit circle and share no common factor with the polynomial
8(B). The model in which the evolution of the conditional variance satisfies (15.2.13) is
called exponential GARCH, or simply EGARCH.
372
CHAPTER 15 Time Series Regression and Gar& Models
More generally, the regression model with autocorrelated error can be combined with
the conditional heteroscedasticity model, i,e.,
where
&I
+ . . . + p p ~ t - 3p. )ti,
= p1~~-l
(15.2.16)
11 = IT, e,,
u: = do
(15,2.15)
+ $luf-l + . 3
+
. ~ , ~ ~ - , + B ~ n ~ ~ ~ + (15.2.17)
~ ~ ~ + B , n ~ ~ ~
and the et are i.i.d. N(0, 1) and independent of past realizations of ?I,+. To test for heieroscedasticity in the error variance, we can perform the following steps:
1.
2.
3.
4.
CalcuIate OLS residuals 2, from the OLS fitting of (15,2.14).

Fit an AR(p) model (15.2.15) to the g,.
Obtain the residuals i tfrom the AR fitting in (15.2.15).
Form the series if
and compute its sample ACF, i.e.,
where
Similarly, we can also compute its PACF, Then, we can cbeck whether these ACF and PACF
follow any pattern. A pattern of these ACF and PACF not only indicates ARCH or GARCH
errors, but it also forms a good basis for their order specification.
To see how the ACF and PACF pattern can help identify a GARCH model, we f i s t
note that as shown in (15.2.11a), a general GAICCH(r, s) model for u! corresponds to an
ARMACnr, r) model for n! with m = max(l; s). Thus, both ACF and PACF of 2; will show
patterns of an exponential decay. The orders r and s can be identified using the similar
techniques of identifying the orders of an ARMA modef. On the other hand, for a pure
ARCH(s) model, its ARMA representation, from (15.2.1 la), reduces to the AR(s) model
Hence, the corresponding PACF will cut off after lag s,
15.3 Estimation of Garch Models
373
Alternatively, we can .also use the following portmanteau Q statistic to test for
= 0, i = 1,2, . . . ,k as suggested by McLeod and Li (1983):
pi(n:)
As in Chapter 7, under the null hypothesis that the ?$are white noise, the Q(k) statistic
approximately follows a x2(k) distribution. In this case, the significance of the Q(k) statistic
occurring only for a small value of k indicates an ARCH model, and a persistent significance
for a large value of k implies a GARCH model.
Recall that to test for an ARCH model, we can also fit model (15.2.6) and check the
significance of the statistic given in (15.2.7).
15.3 Estimation of GARCH Models

Once a model is identified, the next step is to estimate its parameters. Let us consider the
general regression model with autoconelated error and conditional heterocedasticity GARCH
structure for the error variance,
where
et = qqst-1
+- - +Q
~ E ~ -n,,
~
n, = Utei,
and the et are i.i.d. N(0, 1) and independent of past realizations of ~ z , - ~The
. following methods
can be used to estimate the parameters,
15.3.1 Maximum Likelihood Estimation

Note that the above mode1 can be rewritten as
374
..
Let Y = ( Y l , . , Y,), X = (XI, . . , X,), and Yo and & be some proper starting values
required for computing n, for t = I , . n from (15.3.6). Then, similat to the estimation of
transfer function models discussed in Section 14.3.3, we can obtain the maximum likelihood
estimates of the parameters through maximizing the conditional likelihood function
..
or the log-likeljhood function
. . . , cp,), 4 = (41,. . . , #,I,
where u: is given in (15.3.4), n, is given in (15.3.6), p = (ql,

and 0 = (do, dl, , . ,8,).
15.3.2 Iterative Estimation

Alternatively, because the GARCH(,; s) model for of in (15.3.4) is equivalent to an
ARMA(n1, r) model for n; with m = max (r, s), Len,
where the a, are zero mean white noise, we can estimate the parameters using the following steps:
1. Compute GLS estimator of /3 and Q using the iterative GLS method discussed in
Section 15.1. Let g, be the error series obtained from the last iteration of the GLS
estimation.
2. Calculate the sample ACF and PACF of the g, series, and identify and fit its AR(p)
representation,
3. Compute the residuals, k,, from the A R ( p ) fitting of the 2, series in step 2.
4. Fit the identified ARMA (nz,r) model on the squared residuals ,:A i.e.,
zo,
&
Let
&, and be the estimates of the parameters in this ARMA[m, r) model. Then
from (15,2.1la, b), we have
15.4 Computation of Forecast Error Variance
375
15 -4 Computation of Forecast Error Variance

An important use of an ARCH or GARCH model is to find a forecast error variance that
changes over time. We first note that the optimal 1-step ahead forecast of Y,+)given all
information up to time t is the conditional expectation Er(Yr+I),The associated 1-step
ahead forecast error is E,+[ = Y,+/ - Et(Yt+l).To calculate the conditional variance, we
rewrite (15.3.2) as
(l-cplB-...-
pPBP)c, = n,.
(15.4.1)
Thus,
where Q0 = 1 and
Using the result in Section 5.2.1, we see that given all the information up to time t, the I-step
ahead forecast error variance is the conditional variance
where crf+l-j = E,(nf+l-i), which can be calculated from (15.3.4) or, equivalently, (15.3.9)
using the estimated parameters obtained earlier, i.e.,
Clearly, in using (15.3.4) or (15.3.9), we may need a: and 11; for t = -m

where nz = max(r, s). BollersIev (1986) suggests that these terms be chosen as
g: = 11:
= G2,
fort = -m
+ 1 , . . . ,0,
+ 1, . . . , and 0,
(15.4.6)
376
CHAPTER 15 Time Series Regression and Carch Models
where
As a special case, we note that in (15.3.2) that when E, = n,, the model reduces to the
GARCH(c s) model of uncorrelated errors, and we have Var,(~,+~)
= u:+~.
Thus, given
the information up to time ( t - 11, the one-step ahead conditional forecast enor variance
is simply (T:, which changes over time.

EXAMPLE 15.1 In this example, we consider the bivariate series listed as Series W15
in the appendix and plotted in Figure 15.1, which were originally presented in Bryant and
Smith (1995). This series contains 46 monthly observations on Louisiana and Oklahoma
spot prices for natural gas from January 1988 to October 1991, measured in dollars per
million British thermal units ($/MBtu). The spot market for natural gas is the market in
which natural gas is sold under a contract term of no more than one month. It was known
1988
1989
1990
1991
Year
FIGURE 15.1 Spot prices of natural gas in Oklahoma

(solid h e ) and Louisiana (dotted line) between January
1988 and October 1991.
377
that the spot price for Louisiana natural gas was known on or before the first day of trading on the Oklahoma market. To examine whether one can predict the Oklahoma spot
price from the current spot price of Louisiana, we will try a time series regression model.
Preliminary Analysis We begin with a simple regression model with uncorrelated errors.
That is, let 0,and L, be the spot prices for Oklahoma and Louisiana, respectively. We consider
the OLS estimation for the model
ot
PO 4-
PIL, f
(15.5.1)
&I,
which gives R~ = ,9444 and the following estimation result:

Estimate
Parameter
St.E.
..
t-value
P-value
Thus, the OLS regression equation is 0, = -12 .8L,, and we can compute the OLS residuals,
gt = 0, - 6,. The PACF of the g, series shows a significant spike at lag 1. Thus, we propose an
AR(1) model for the error term, i,e.,
The AR(1) fitting of the series g, gives = ,502. To check for heteroscedasticity in error
variance, we compute the residuals, Ci, = 6, - .502i,-1,and calculate the portmanteau Q(k)
statistic on the squared ii? series, with the results summarized in Table 15.1.
TMLE 15.1 Test for heteroscedasticity.
Q(k) Statistic
P-value
378
Construction of a GARCH (r, s) Model At a .= -05, Table 15.1 shows that there is no
evidence of heteroscedasticity. For a = .I, however, these Q(k) statistics clearly indicate an
ARCH(1) or GARCH(0, 1) model. Thus, we will refit the spot prices with the model
where the e, are i.i.d. N(0, I), independent of
and
The estimation results of this model are summarized in Table 15.2. From this table, we see
that the estimate is not significant but that 81 is highly significant. Hence,
a0
or, equivalently, from (15.2.10),

la; =
1.956~~;-~
+ a,,
(15.5.8)
where a, is a zero mean white noise process.
Computation of the Conditional Forecast Error Variance To calculate the conditional variance, we note that from (15.4.2) and (15.4.3) that
with & = 1 t,bj = (*502)jforj = 1, 2,. . . . Thus, given all the information up to time f, by
(15.4.4) the I-step ahead forecast error variance is the conditional variance
TABLE 15.2 Estimation of the GARCHEO, I] model.

Coefficient
Value
St.E.
t-value
P-value
Illustrative Examples
15.5
379
where 2
= ~ , ( n : + ~ -Specifically,
~);
following (15.4,4) and (15.4.5), given the information up to October 1991 that corresponds to t = 46, we can compute the conditional forecast
error variances for t = 47 and t = 48 as follows:
where n~ is the residual at t = 46 obtained from the autoregressive error fitting in (15.5.4).
The other 1-step ahead conditional forecast error variances can be calculated similarly.
Although we introduce the conditional variance through a regression model, the concept
clearly applies to the univariate time series models. In other words, in the general ARMA model
(1 - # l B - . . . -
+ p B P ) z , = %,(
+ (1 - B I B
"
'
- OqB4)a,,
the white noise process a, may follow a GARCH(r s) model, i.e.,
where the e, are i.i.d. random variables with mean 0 and variance 1,independent of past realizations of a,i, and
or an ARCH model if r
= 0. Let
us consider the following example.
EXAMPLE 15.2 We recall Series WS, the yearly US tobacco production from 1871 to
1984, first considered in Chapter 4 with its plot shown in Figure 4.2. The series is more volatile
at some times than at others. Instead of taking a transformation, we now fit an ARIMA(0, 1, 1)
model to the original series and try to examine its residuals. The estimated model is
where for consistency of notations used in this chapter we use n, to denote the white noise
error series. Traditionally, the variance of 12, is assumed to be constant. Examination of ACF
and PACF of n!, however, suggests the following ARCH(2) model:
nf = .17291~f .369811?-~ a,.

(.089)
(.094)
380
CHAPTER 1 5 Time Series Regression and Gar& Models

Given the information up to 1984 that corresponds to t = 114, we can compute the conditional
error variances for t = 1I5 and t = 116 as follows:
which shows that the variances are related to the squares of past errors and they change over time.
The investigation of conditional variance models has been one of the main areas of
study in time series analysis of financial markets. Toward these ends, the GARCH model and
its variations have been applied to many risk and volatility studies. We use the above simple
examples to illustrate the procedure. Other examples include Engle, Ng, and Rothschild
(1990), Day and Lewis (1992), and Lamoureux and Lastrapes (1993). These studies suggest
that the selected model does not have to be complicated. Indeed, a very simple GARCH(c s)
model with r 5 2 and s 5 2 is often sufficient to provide a significant improvement over the
traditional homoscedasticity models.
15.1 Consider the ARCH(1) model n, = Ute,, where the e, are i.i.d, random variable with
mean 0 and variance 1, and cr? =. Oo O1 n;-!_l.Show that the unconditional variance
of rz, is Var(n,j = Oo/(l el).
15.2 Let 11, be the Gaussian white noise process with mean 0 and variance a2.If we fit
the model by OLS
for t = s
1, s 2, . . , N , show that ( N - s
multiple coefficient of determination.
----*
~ ~X2
(s), where R~ is the
15.3 Consider the general GARCHfl; s) model, 11, = Ute,, where the e, are i.i.d. random
variables with mean 0 and variance 1, and
Show that for the unconditional variance of n, to be positive and finite the coefficients
need to satisfy the constraints do > 0, c#+ 2 0, Oj r 0,and
+i
2&,Oj < 1.
Exercises
381
15.4
Consider the U.S. liquor sales between 1970 and 1980 given in Exercise 8.6 of Chapter 8.
(a) Build an appropriate ARIMA-GARCH model for the series.
(b) Compute the next 12-month forecasts and their conditional variances from the
model.
[c) Compare and comment on the fesults obtained from the ARfMA-GARCH model
and the standard ARIMA model.
15.5
Find a financial time series. Fit the series with a univariate model, perform an analysis
on its squared residuals, and construct a GARCH model for the error variance if
needed.
15.6 Find some related time series of your interest.

(a) Fit a regression model using OLS, perform a residual anaiysis, identify the error
process, and construct a GARCH model if needed,
(b) Compute the one-step ahead and two-step ahead conditional forecast error
variances.
Vector Time Series Models

The time series data in many empirical studies consist of observations from several variables. For example, in a sales performance study, the variables might include sales volume,
prices, safes force, and advertising expenditures. In the transfer function model, we study a
specific relationship between variables, i.e., an input-output relationship between one output
variable and one or several input variables. In regression time series models, we also concentrate on relationship between a dependent variable and a set of independent variables. In
many applications, however, transfer function and regression models may not be appropriate.
In this chapter, we introduce a more general class of vector time series models to describe
relationships among several time series variables. After an introduction of some basic concepts for vector processes, stationary and nonstationary vector ARMA models are discussed.
Some useful notions such as partial autoregression and partial lag correlation matrix function
are introduced. Detailed empirical studies are presented. The chapter ends with some
descriptions of spectral properties of vector processes.
1 6.1 Covariance and Correlation Matrix Functions
..
Let 2, .= {ZIrt,Zz,t, . , Z,,,,]', t = 0, fI, f2, . . . , be an rn-dimensional jointly

stationary real-valued vector process so that the mean E(Zi,,) = pi is constant for each
i = 1, 2, . . . , m, and the cross-covariance between Zj,, and Zj,,, for afi i = 1, 2, . , nt
and j = 1, 2, , . . , m, are functions only of the time difference (s - t). Hence, we have
the mean vector
..
and the lag-k cdvariance matrix
16.1 Covariance and Correlation Matrix Functions
383
where
.. .
fork = 0, fl,&2,
, i = 1,2, . . . ,m, and j = 1,2, . . . ,nt. As a function of k, r(k) is called
the covariance matrix function for the vector process Z,. For i j, yii(k)is the autocovariance
function for the ith component process Zi,; for i f j, yij(k) is the cross-covariance function
between Z,i and q,,.The matrix r(0)is easily seen to be the contemporaneous variancecovariance matrix of the process. A jointly stationary process implies that every univariate
component process is stationary. A vector of univariate stationary processes, however, is not
necessarily a jointly stationary vector process.
The correlation matrix function for the vector process is defined by
for i = 1, 2, . . . , nr and j = 1, 2, . . . , m, where D is the diagonal matrix in which the ith

diagonal element is the variance of the ith process; i.e.,
Clearly, the ith diagonal eIement of p(k), pii(k), is the autoconelatian function for the ith
component series ZC whereas the (i, j)th off-diagonal element of p(k),
,.
represents the cross-correlation function between ZL and Zj,

Like the univariate autocovarizince and autocorrelation functions, the covariance and
correlation matrix functions are also positive semidefinitein the sense that
384
and
for any set of time points q, t2, . . . , tn and any set of real vectors al,a2,. . . , and an.The
results follow immediately by evaluating the variance of Ey=iaiZ,, and its standardization.
Note, however, that yv(k) P yU(-k) for i # j and also that r ( k ) # I'(-k). Instead, because
~ i j ( k=
) EI(Zi,t - Pi)(z, - pj)1 = Et(Zj, t+k - pjl(Zi, t - Pi11 = yji (-k), we have
,+,
Sometimes, the covariance and correlation matrix functions are also called the autocovariance
and autocorrelation matrix functions.

of Vector Processes
An m-dimensional stationary vector process 2, is said to be a linear process or a purely
nondeterministic vector process if it can be written as a linear combination of a sequence
of nz-dimensional white noise random vectors, i.e.,
where 'Po= I is the ne X nl identity matrix, the 'Pis are nz X m coefficient matrices, and the
a,'s are m-dimensional white noise random vectors with zero mean and covariance matrix
structure
where X is a ni X m symmetric positive definite matrix. Thus, although the elements of a, at

different times are uncorrelated, they may be contemporaneously correlated. Equivalently,
using the backshift operator BSa, = a,-,, we can write
zZ0
q s B S .The above representation is known as the

where 2, = Z, - p and g ( B ) =
moving average or Wold's representation.
16.2 Moving Average and Autoregressive Representations of Vector Processes
385
Let!P,=[$~,s],i=1,.2,..., m , j = 1 , 2 ,..., m , w h e r e + i i , - , = l i f i = j a n d $ e , a = O
CO
if i # j. We can write Y ( B ) = [$ii(B)], where @ii(B) = Es=Ot,bv,,BS. For the process to be

stationary, we require that the coefficient matrices q,be sqye summable in the sense that each
of the m X 111 sequences $c+ is square summable, i.e., X,=*
$$,$ < 00 for i = 1, 2, . , ttf
and j = 1,2, . . . , m.Here and in the following an infinite sum of random variables is defined as
the limit in quadratic mean of the finite partial sums. Thus, Z, in (16.2.1)or (16.2.3)is dehed
..
such that
E[(i,
5 'U,a,-j)'(it
- 2q j a t j ) ]
-+
j=O
j=O
asn
m. (16.2.4)
Another useful form to express a vector process is the autoregressive representation, in

which we regress the value of Z at time t on its own past values plus a vector of random
shocks, i.e.,
or, in terms of the backshift operator,
where
and the H, are m X m autoregressive coefficient matrices. Let E, = Illv, $1, i = 1, 2, . . . , m,

j = 1,2,..., m . A l s ~ , E ~ ,l i~f =i = j a n d I I ~ , o = O i f i # j j , i . e . , ~ o = I . W e h a v e ~ ( B ) =
[Eij(B)], where nii(B) = I ? ,0 - z ~ ~ I ST ~ ,5 The
.
vector process is said to be invertible if
the autoregressive coefficient matrices TZ, are absolutely summable, i.e., Z~olIIii,sl
< ca
for all i and j.
An invertible process is not necessarily stationary. For a vector process with an invertible autoregressive representation to be stationary, we require that no zeros of the determinant of the autoregressive matrix polynomial, denoted by I ~ ( B ) I , lie on or inside the unit
circle, i.e., III(B>I # 0 for IBI 5 1. Similarly, a stationary process is not necessarily invertible. For a vector process with a stationary moving average representation to be invertible,
no zeros of the determinant of the moving average matrix polynomial should lie on or inside
the unit circle, i.e., I*(B)~ # 0 for IBI 5 1.
386
16.3 The Vector Autoregressive Moving Average Process

A useful class of parsimonious models is the vector autoregressive moving average
ARMA(p, q) process
where
and
are the autoregressive and moving average matrix polynomials of orders p and q, respectively, and Qo and goare nonsingular m X rn matrices. For any nondegenerate case where
the covariance matrix X of a, is positive definite, we assume in the following discussions,
=
= I, the m X nt identity matrix, When p = 0, the
with no loss of generality, that
process becomes a vector MA(q) model
When q = 0, the process becomes a vector AR(p) model

Z,
+ @ p ~ t - p + a,.
(16.3.3)
The process is stationary if the zeros of the determinmtal polynomial J@,(B)Jare outside
the unit circle. In this case, we can write
where
such that the sequence 9,is square summable. The process is invertible if the zeros of the
determinantal polynomial le4(B)1 are outside the unit circle, enabling us to write
16.3 T h e Vector Autoregressive Moving Average Process
387
with
such that the sequence is absolutely summable.

Recall from Section 3.2 that, for univariate time series, a given autocovariance or autocorrelation function may correspond to more than one ARMA(p, q) model. To ensure the
unique representation, we have imposed invertibility conditions in the model selection. Similarly, in the vector ARMA model in (16.3.1), left-multiplying both sides of (16.3.1) by an
arbitrary nonsingular matrix or by a matrix polynomial in B yields a class of equivalent models that ail have identical covariance matrix structures. Thus, without further restrictions, the
model in (16.3.1) is not identifiable in the sense that we cannot uniquely determine the values of p and q and the coefficient matrices aiand Bj from the covariance matrices of 2,.
The problem of identsability for vector ARMA models has been studied by Hannan
(1969, 1970, 1976, 1979). Hannan considers the class of stationary processes satisfying the
following conditions:
A. The only common left divisors of @,(B) and B,(B) are unimodular ones, i.e., if @,(B) =
C(B)H(B) and 8 ( B ) = C(B)K(B), then the determinant [c(B)I is a constant; and
B. The zeros of IQp(~)I(possibly after some proper dEerencing and transformations) are
outside the unit circle and the zeros of IB~(B)Iare on or outside the unit circle.
He then argues that with the above conditions the model in (16.3.1) may be identified by
any one of the following procedures:
1, From each class of equivalent modeh choose those with minimal moving average order q.
Among these choose one with minimal autoregressive order p. The resulting representation will be unique if the rank t@,Bq3 is equal to nt.
2. Represent e p ( B )in lower triangular form. If the order of +ij(B), the (i, j ) element of
@,(B), is less than or equal to the order of +ii(B) for all i = 1, . . . , ni, and j = 1, . . . ,
In, then the model is identifiable.
3. Represent @,(B) in a form such that @,(B) = &(B)I, where &(B) = I - $lB is a univariate AR polynomial of order p, The model is identifiable if +p # 0.
Procedure 1 is simpler and more commonly used than procedures 2 and 3. Other important phenomena on a vector ARMA representation are presented in Chapter 17.
Vector M(1)Models The vector AR(1) model is given by
Z,=
@ti,-I + a,.
388
For nt = 2,
Thus, apart from current shocks, each ii,,

depends not only lagged values of iiit
but also lagged
values of other variables $ For example, if zlg,
and &, represent the sales and the advertising expenditures (measured f?om some level p) of a company at time t, then Equation (16.3.10)
implies that current sales depend not only on the previous sales but also on the advertising
expenditure at the last time period. Moreover, there is a feedback relationship between two
series. Thus, the current advertising expenditure will also be influenced by the sales performance at the previous time period.
,.
Vector ArWIA and Transfer Function Models If 412 0 in (16.3.9), then we have
One should not, however, mistakenly regard (16.3.12) as a causal transfer function model
This is so
relating the input Z1,,to the output Z2, with one period delay with effect
because the input series Zl,, and the noise series a2,,are correlated, which violates a basic
assumption in the transfer function model, To reduce (16.3.12) to a causal transfer function
model, we let
where a is the regression coefficient of a*, on al,,.The error term bz,,is independent of al,,
and hence bl, Thus, we have
,.
389
This result shows that the input series ZI,, actually has an instantaneous effect a and a oneperiod delay effect (#21 - CY#I~).
Let the variance-covariance matrix X of [al,,,
a2,!]' be
It can be easily shown that
Thus, treating the model in (16.3.12) as a transfer function model could be very misleading
if u12# 0,
In summary, if the autoregressive matrix polynomial (P,(B) is triangular, a joint multivariate
vector AR model can be reduced to a causal transfer function model. More generdiy, if
is
block triangular, then a joint multivariate vector model can be reduced to a block uansfer
function model where a feedback relationship could exist among series within each block.
Stationarity and Invertibility Conditians The vector AR(I) process is clearly invertible,
For the process to be stationary, the zeros of the determinantal equation must be outside
the unit circle. Letting A = B-', we have
1 @k~I
390
CHAPTER I6 Vector Time Series Models
11
. ..
Thus, the zeros of are related to the eigenvalues of (Dl. Let hi, A*,
,A, be the
eigenvalues and hl, h2,. . . , h, be the associated eigenvectors of QI, such that @,hi= &hi
for i 1, 2, . . . , rn. Far simplicity, assume that the eigenvectors are linearly independent.
Let A = diag[hl, h2, . ,Am]and H = [hl, h2$. . . , h,].
We have H = HA and
..
Now,
Hence, the zeros of [I - ~ ~ are

3 outside
1
the unit circle if and only if all the eigenvalues hi
are inside the unit circle. This result implies that an equivalent condition for stationarity of a
vector AR(1) process is that all the eigenvalues of
lie inside the unit circle, i.e., lhil < 1
for i = 1,2, , ,M. In such case,
..
lim An = lim diag[Af, A$,
n-+w
n+w
,,
. ,Ah]
and
lim @: = lim HAHI'
n-03
fim H A ~ H - ' = 0
It follows that
and
Thus, we can write
ITAH-'..
- - . HAH-'
n+m
(16.3.22)
39 1
a.
where
= I and the cPlf matrix decreases toward the 0 matrix as s increases. Note,
however, that
where
(B) is the adjoint matrix of a l ( B ) . Hence, we can rewrite (16.3.24a) as
Because the order of the polynomial IB~(B)Iis m and the maximum order of the polynomiaf
@ t ( B ) is (m - I), (16.3.26) implies that each Z i t from a joint multivariate AR(1) model is
marginally a univariate ARMA process of order up to (m,111 - I). Because cancellations may
occur for some common factors between IQ~(B)Iand @$ ( B ) , the (i, j ) element of the matrix
cPf (B), the autoregressive components of the univariate series may not be identical.
are linearly indeRecall that in the above derivation we assume that the eigenvectors of
pendent. In general, the eigenvectors of Qll may not be all linearly independent. In such case,
however, there still exists a nonsingular matrix H (see, e.g,, Gantmacher 119601 and Noble
[1969]) so that
where the adjacent A's may be equal and the 6's are either 0 or 1. The above results still
< I for i = 1,2, , . . , and m.
follow if
16.3.1 Covariance Matrix Function for the Vector AR(1) Model

r(k)
;=
E[z~-~z;]
= E [ Z , - ~ ( @ ~ Z , - ~ a,)']
=
E [ Z , - ~ Z ; - ~ @ ~~ ~ - ~ a : ]
where we note that E ( z , - ~ ~ ;=

) 0 fork 2 I. Fork = 1, we have
r(I) = r(O)@i;hence,
392
and
Given the covariance matrix function r(k), Equations (16.3.29) and (16.3.30) can be used to
calculate Z and a,. Conversely, given cPl and X, we can compute r(0) by noting that
(16.3,30) implies that
Thus,
Solving Equation (16.3.31) for r(O), we let vec(X) be the column vector formed by
stacking the columns of a matrix X. For example, if
then
Using the result (see SearIe E1982, p. 3333),

vec(ABC) = (C' @ A ) vec(B),
(16.3.33)
where 63 denotes the Kronecker product, we have from (16.3.31)
vec(r(0)) = vet(@ ,r(O)@;) vec(Z)
(al63 Ql) vec(r(0)) + vec (2)
393
The use of (16.3.34) is given in Example 16.1. The function T(k) for k 2 1 can then be
obtained from the second part of (16.3.28).
EXAMPLE 16.1 Consider the two-dimensional vector AR(1) process
where
and
From (16.3.34). we have
Specifically,when
are .4 and -9, which are inside the unit circle; hence, the process is
the eigenvalues of
stationary, Because the eigenvalues of (A C
3 3 )are products of eigenvalues of A with those of
394
B, it follows that all eigenvalues of (rPI

(16.3.34) exists. Thus,
Q1)are inside the unit circle; hence, the inverse in
and, by (16.3.281,
16.3.2 VectorAR(p) Models

The general vector AR(p) process
(I-
a l ~ ' -. . . - (
P ~ B P=) a~, ~
(16.3.35a)
is obviously invertible. For the process to be stationary we require that the zeros of
1 1 - GJIB - + * * - gP\lie outside the unit circle or, equivalently, the roots of
be inside the unit circle. In such case, we can write
395
where the
weights are square summable and can be obtained by equating the coefficients
of ~j in the matrix equation
( I - (PIB -
. . - a,Bp)(Z + YIB + * 2 ~ 2 + . - ) = I ,
(16.3.38)
1.e.,
!PI
Because
where @:(B)
IQ~(B)I
[@$ ( B ) ] is the adjoint matrix of
(B), we can write (16.3.37a) as
Now,
is a polynomial in B at most of order nip, and each 4 ( B ) is a polynomial in B
at most of order ( n ~- 1)p. Thus, each individual component series follows a univariate
ARMA model up to a maximum order (mp, (m - 1)p). The order can be much less if there
are common factors between the AR and MA polynomials.
The covariance matrix function is obtained by Ieft-multiplying
on both sides of the
transposed equation of (16.3.35a) and then taking expectations, i.e.,
Thus,
396
where 0 is the m X m zero matrix. More compactly, for k = 1,

generalized Yule-Walker matrix equations:
. . . ,p, we obtain the following
16.3.3 Vector MAGI) Models

The vector MA(1) models are given by
where the a, are m X 1 vector white noise with mean zero and variance-covariance matrix X.
For m = 2, we have
The covariance matrix function of Z, is
Note that f(-1) = r l ( l ) and that r(k) cuts off after lag 1, a behavior that parallels the univariate MA<1) process.
To solve for
and X in terms of r(k)'s, we note that, from (16.3.44), and (16.3.45)
397
Hence,
which is a quadratic matrix equation in QI.'Amethod to solve such an equation can be found
in Gantmacher (1960).
The vector MA(1) process is clearly stationary, For the process to be invertible, the zeros
must be outside the unit circle or, equivalently, the eigenvalues of 8 1 should
of - 81~1
be all less than 1 in absolute value, The proof is exactly identical to that of the stationarity
condition for tbe vector ARf 1)process. In such a case,
IZ
and we can write

Q)
Z,
2 @fZf-s= a,,
s=l
where the Qf matrix decreases toward the 0 matrix as s increases.
16.3.4 Vector MA(q) Models

The general vector MA(q) process is given by
The covariance matrix function is
where e0= I and F(-k) = r 1 ( k ) .Note that r(k) cuts off after lag q. The process is always
stationary. If it is also invertible, then we can write
with
398
CHAPTER I 6 Vector Time Series Models

so that the II, are absolutely summable. Because the elements of the adjoint matrix 8:(8)
are polynomials in B of maximum order (m - l ) q , the only requirement needed for the
model to be invertible is that all the zeros of16,(3)I be outside the unit circle, The II, can be
obtained by equating coefficients of ~j in
ll (B)8,(B)
[I Hence,
I,
-I I ~ B ..*I[
~ 1 - BIB -
I.
(16.3.53)
n, can be calculated recursively from the Qj as
n,= ns-,eli- . + n,-,o,,

where
- E),Bq]
no= -I
s = 1,2,...,
(16.3.54)
and llj = 0 f o r j < 0.
16.3.5 Vector ARMA(1,l) Models

The vector ARMA(1, 1) model is given by
The model is stationary if the zeros of the determinantal polynomial - Q ~ B Iare outside the
unit circle or if all the eigenvalues of
are inside the unit circle. In such a case, we can write
where the
'Psweights are obtained by equating the coefficients of Bj in the matrix equation

( I - a I B ) ( I -t- 'PIB
+ 'P2B2 -I-
-) = (I - BIB),
The pracess is invertible if the zeros of 11 - 8 i ~are( outside the unit circle or if aII the
eigenvalues of
are inside the unit circle. In such a case, we can write
such that
399
and
For the covariance matrix function, we consider
and note that
Hence, we obtain
Using the first two equations in (16.3,60), we have
and
When the parameter matrices

and Z are known, Equation (16.3.61) is of the same form
as Equation (16.3.31). Thus, r(0)can be solved in the same way as illustrated in Example 16.1
and is given by
The other covariance matrices r(k)can then be calculated through (16.3.60),

Conversely, given the covariance matrix function r(k), we can obtain the parameter
matrices as follows. First, from the third equation of (16.3.60), for k = 2, we obtain
Next, using the first two equations in (16.3.60), we get
400
Hence,
Substituting the second equation of (16.3.60) into (16.3.65), we have
where
Equation (16.3.66) is of the same form as Equation (16.3.46) and can be solved similarly for
B1.Having and B1,Z can be calculated from (16.3.64)
16.4 Nonstationary Vector Autoregressive

Moving ~ v e r & eModels
In the analysis of time series, it is very common to observe series that exhibit nonstationary
behavior. One useful way to reduce nonstationq series to stationary series is by dserencing.
For example, in univariate time series, a nonstationrvy series 2, is reduced to a stationary
series (1 - B ) ~ z for
, some d > 0,so we can write
where gS,(B) is a stationary AR operator. A seemingly natural extension of (16.4.1) to the vector
process is
or, equivalently,
This extension, however, implies that all component series are differenced the same number
of times. This restriction is obviously unnecessary and undesirable. To be more flexible, we
assume that even though 2, may be nonstationary, it can be reduced to a stationary vector
series by applying the differencing operator D(B), i.e.,
16.5 Identification of Vector Time Series Models
401
where
and (dl, dz, . . . ,dm)is a set of nonnegative integers. Thus, we have the nonstationary vector
ARMA model for Z,
[B~(B)I
where the zeros of IOp(3)[and

are outside the unit circle.
Differencing on vector time series is much more complicated and should be handled
carefully. OverdiEerencing may lead to compfications in model fitting, One should be particularly careful when the orders of differencing for each component series are the same. Box
and Tiao (1977) show that identical differencing inlposed on a vector process may lead to a
noninvertible representation. Furthermore, in some cases, as discussed in Chapter 17, a linear combination of nonstationary series could become stationary. In this circumstance, a
model purely based on differences may not even exist. In such a case, one can use the foilowing generalization of the vector ARMA model proposed by Tiao and Box (1981):
where the zeros of the determinantal polynomial [ Q ~ ( Bare

) on or outside the unit circle.

In principle, identification of vector time series models is similar to the identification of
univariate time series models discussed in Chapters 6 and 8. For a given observed vector
time series Z1, &, . . . ,Zn, we identify its underlying model from the pattern of its sample
correlation and partial correlation matrices after proper transformations are applied to reduce
a nonstationary series to a stationary series.
16.5.1 Sample Correlation Matrix Function

Given a vector time series of la observations ZI, Z2, :. . ,and Z,, we can calculate the sample
correlation matrix function
402

where the ;ij(k) are the sample cross-conelations for the ith and jth component series
;ij(k)
ZZ!(zipt
(zj,t+k - zj)
- z,) Z,=~(Z~,,
- zj)2]1p
zi)
- 2
n
[x;l(zi,,
zi
and and ndj are the sample means of the corresponding component series. For a stationary vector process, Hannan (1970, p. 228) shows that j(k) is a consistent estimator that is asymptotically normally distributed, The variance and covariance of these sample cross-correlation Fii (k)
obtained by Bartlett (1966) and given in (14.3.4) and (14.3.5) are rather complicated and depend
on the underlying models. Ifpgi(k)= 0 for [ k ]> q for some q,then, from (1 4.3.51, we have
Var[ie(k>]
n - k [ I + 2~
S=I P ~ ~ ( S ) P ~ ~ ( ~ I
)kI] ,> 4.
When the 2, series are white noise, Bartlett's approximation for the asymptotic covariance
simplifies to
Cov [ & j ( k ) , fiii(k + S ) ]
n-k
and
For large samples, (ta - k) is often replaced by n in the above expressions.

The sample correlation matrix function is very useful in identifying a finite-order moving
average modd as the correlation matrices are 0 beyond lag q for the vector MA(q) process.
Unfortunately, the matrices are complex when the dimension of the vector is increased. The
crowded numbers often make recognition of patterns difficult. To alleviate the problem, Tiao
and Box (1981) introduced a convenient method of summarizing the sample correlations. Their
method is to note the symbols -, and in the (i, j ) position of the sample correlation matrix,
where -t- denotes a value greater than 2 times the estimated standard enors, - denotes a value
less than -2 times the estimated standard enors, and - denotes a value within 2 estimated standard errors.
+,
16.5.2 Partial Autoregression Matrices

A useful tool for identifying the order of a univariate AR model is the partial autocorrelation
function. As dehed in Section 2.3, it is the correlation between Z, and Z,+.k after their mutual linear dependency on the intervening variables ZrS1,Zt+2,. . . , and Z,+k-l has been removed, i.e,,
16.5 Identification of Vector Erne Series Models

A
403
where 2, and Zt+Rare the minimum mean squared error linear regression estimators of Zt
and Zt+k based on Zt+f,ZIC2,. . . , and Zt+k-l. AS shown in Section 2.3, the partial autocorrelation between Z, and Zr+kis also equal to the last regression coefficient associated with 2,
when regressing Zt+kon its k lagged variables Zf+R-l,Zt+k-2r. . . , and Z,. The partial autocorrelation is useful for identifying the order of a univariate AR(p) model because +kk is
zero for [kl > p, In this section, we consider a generalization of the concept to a vector time
series process proposed by Tiao and Box (1981).
Tiao and Box (1981) define the partial autoregression matrix at lag s, denoted by $(s),
to be the last matrix coefficient when the data are fitted to a vector autoregressive process
of order s. This definition is a direct extension of the Box and Jenkins (1976, p. 64) definition
of the partial autocorrelation function for univariate time series. That is, B(s) is equal to @s,
in the multivariate linear regression
where.% ,+, is the error term, and the nt X nt matrix coefficients a , k, k = 1, 2,

those that minimize
. . . , s, are
\where /%I2= YV; for the m-dimensional vector V,.

Minimization of (16.5.7) leads to a multivariate generalization of the Yule-Walker
equations in unnormalized form,
The Tiao and Box definition of 9 t s ) leads to solving (16.5.8) for a;,, and determining
the partial autoregression matrix requires solving (16.5.8) for successively higher orders of s.
If, for s 2 2, we let
404
and
then the system (16.5.8) can be written as
Hence,
Equation (16.5.11) implies that
Substituting (16.5.13) into (16.5,12) and solving for @j,, we have

@,:
,= (r(0)
- b' (s)[A(s) 1-'b(s))-l
(r(s)- b' (s)[A(s) ]-lc(s)}.
(16S.14)
The partial autoregression matrix function is therefore defined to be

B(s) =
ri(1)~r(o~~-l,
{rf(s) - cl(s)[A(s)]-lb(s)) (r(0)
bf (s)[A(s)]-lb(s))-I,
Li
= 1,
> 1.
(16.5.15)
The 9 ( s ) so defined has the property that if the model is a vector AR(p), then
Like the partial autocorrelation function for the univariate case, the partial autoregression
matrix function, 9 ( s ) , has the cutoff property for vector AR processes.
Although the autoregression matrix function, 9 ( s ) , is motivated from consideration of
vector GR models, the function is defined in terms of the F(k)'s through (16.5.15) and hence
can be applied to any stationary process. Unllke the univariate partid autocorrelation function, however, the elements of P(s) are not proper correlation coefficients.
16.5 Identification of Vector % m e Series Models
405
Sample Partial Autoregression Matrices Sample estimates of the P ( s ) ca_n be

obtained by replacing the unknown r(s) in (16.5.15) by the sample covariance matrices qs):
- -
wbere z = (Z1, &, . . . , Zm)' is the sample mean vector. Alternatively, estimates of the
P ( s ) can be obtained by fitting vector autoregressive models of successively higher orders
by standard multivariate linear regression models. First, we note that if a process is a stationary vector AR(p), we can write it as
where r' is a constant vector. Thus, given n observations, and fort = p
+ 1, . . . , n, we have
where
When the a, are N(0, Z) Gaussian vector white noise, we have, from the discussion of
multivariate linear models given in Supplement 16.A, the following results.
is joint multivariate normal with the mean and variance-covariancematrix
where @ denotes the Kronecker product.

2. The maximum likelihood estimate of Z is given by
406
tvhereN = n
- p and S ( p ) is the residual sum of squares and cross-products, i,e.,
The estimated variance-covariance matrix of P is
...
The estimated variances and correlations of the elements of such as 41,

,and &p
are provided by many statistical software that contain multivariate Linear models.
3. S(p) follows aAWishartdistribution with (N- pm - I) d.f, and is denoted as WN-pm-I(Z).
Furthermore, P is independent of S(P).
4. The maximum likelihood function is given by
Summary Statistics for Partial Autoregression To test for the significance of the
sample partial autoregression, we can use an overall X2 test. TOdo so, we note that Equation
(16.5.18) can also be written as
where
and
To test the hypothesis
407
we note that under Ho, the estimate of Z is given by
where S ( p - 1)is the residual sum of squares and cross-products under Ho, i.e.,
The corresponding maximum likelihood function under Ho is given by
Thus, the likelihood ratio statistic for testing QP = 0 against QP f 0 is
Following the result given in Supplement 16.A, we see that under Ho, the statistic
asymptotically has a chi-square distribution with m2 degrees of freedom. We reject Ho if U is

small and hence when M(p) is large.
One can also use Akaike's information criterion (AIC) to select a model, Some software programs provide both M ( p ) and AIC. For example, the software SCA computes
AIC as
2
AIC = - - In(maximum likelihood)
II
+ -2n (the number of parameters).
For an m-dimensional vector AR(p) model, it becomes
and one selects a mode1 that gives the smallest AIC value. Ideally, different criteria will
reach the same model. Otherwise, use of good judgment from a model builder may be
required.
408
CHAPTER 16 Vector Time Series Mod&
16.5.3 Partial Lag Correlation Matrix Function

By definition, the partial autocorrelation function between 2, and Zt, for the univariate time
series is the correlation coefficient between Z,and &+, after the hear dependence of each op the
intervening variables Z,,l, . . . , Z,+s-l
is removed (see, for example, Haman [1970, p. 221). As
shown in Section 2.3, this partial autocorrelation between 2, and q+,can also be obtained as the
last regression coefficient associated with Z, when regressing &+, on its s lagged variables Zl+s-I,
Zt+,-z, . . . ,Z,. 'Iiao and Box (1981) extend the latter result to the vector case and intxoduce the
partial autoregression matrix function P(s). As we have noted, however, unlike the univariate
partial autocorrelation function, the elements of 9 ( s ) are not proper correlation coefficients.
Heyse and Wei (1985a, b) extend the definition of univariate partial autocorrelation to
vector time series and derive the correlation matrix between Z, and Z,,, after removing the
linear dependence of each on the intervening vectors ZtLtl,2t+2,
. . . , and ZI+,-l. This correlation matrix is defined as the correlation between the residual vectors
and
The multivariate linear regression coefficient matrices ad-l,k and Ps-l,

k are those that
and E [ ~ V , - ~ , , / ~respectively.
],
Note that (16.5.32) is the residual
minimize E[~U,-,,,+~/~]
from a regression of Z,+, on the carriers Z,+s-i,
. . , Z,+l, and that (16.5.33) is the
residual from a regression of 2, on the same set of carriers.
For s 2 2, we let
and use A(s) and c(s) from (16.5.9) and consider the minimization of
409
Taking the derivative of (165.35) with respect to the elements of lu'(s) and setting the resulting
equations equal to 0 gives (Graham [1981, p. 541)
which are recognized as the multivariate normal equations for the regression of Zt+, on
i.e.,
Zt+s-1, . . - ,
Solving the system (15.5.36)or (16.5.37) gives the familiar multivariate linear regression
coefficient matrices
The Linear combination cu(s)Z,(s) defines the linear projection of Zt+, onto the space
spanned by Z,+,-l, . . . , Z,+ I. Because
we have that u,-l, ,+, and Z,(s) are uncorrelated and that
so
Similarly, letting
410
CHAPTER 16 Vector Time Series Modeh
and using A(s) and b(s) from (16.5.9) and Z,(s) from (16.5.34),we can write the normal
equations for the
of (16.5.33) as
which has the solution
/ I r (s) = [A(s)]-I b(s).

Also, in a similar manner to (16.5.391,
The covariance between v,-~, and us-l, ,+, is
,,
We denote var(us-1, ,+,I by V,(s), var(vs-1, ,) by Vv(s), and cov(vs-I, us-1, ,,I by Vv,(s).
Note that c o v ( ~ , - ~,+, vS-~,,) is equal to Vk(s).
For the case s = 1, it is easily seen from (16.5.32) and (16.5.33) that V,(l) = V,(l) = r(0)
and that Vw(l) = r(1) because there are no intervening vectors between Z, and Z,,l,
Heyse and Wei (1985a, b) define the partiat lag autoconelation matrix at lag s to be
where Dv(s) is a diagonal matrix in which the ith diagonal element is the square root of the ith
diagonal element of Vv(s) and D,(s) is similarly defined for V,(s). The ternpartial lag autocorrelation ntatri~is used because it is the autocorrelation matrix between the elements of Z, and
Z,+$,after the removal of their linear dependence on the vectors at the intervening lags, Z,+l, . . . ,
Z,+,-l. They call P(s), as a function of s, the partial lag autocorrelationmatrix function. P(s), as a
matrix of autocorrelation coefficients, is a vector extension of the partial autocorrelation function
in the same manner as the autocorrelation matrix function is a vector extension of the autocorrelation function. Recognizing that the univariate case corresponds to the special case when nt = 1,
they recommend referring to P(s) as the partial lag autoconelation function even in the univariate
case. In this book, we simply refer to P(k) as the partial lag correlation matrix function.
Note from (16.5.15) that Tiao and Box's partial autoregression matrix at lag s equals
9 ( s ) = v&,
(s) [v.,(s)]-'.
(16S.44)
The following is a summary of the relationships and differences between our partial lag
correlation matrix P(s) and Tiao and Box's partial autoregression matrix B(s)
16.5 Identification of Vector Erne Series Models
41 1
1. From (16.5.43) and (16.5,44), we see that P(s) = P ( s ) = 0 if and only if V,,{s) = 0.
Thus, P(s) and B(s) share the same cut-off property for autoregressive processes.
2. When m = 1, it is easily shown by reversing the order of the equations in either
(16.5.36) or (16.5.40) that
and
Thus,
Hence, the partial lag correlation matrix and Tiao and Box's partial autoregression
matrix are equal in the univariate case.
3. The partial lag correlation matrix P(s) is a correlation coefficient matrix because
each of its elements is a properly normalized correlation coefficient. Here, however,
P ( s ) is not equal to the correlation coefficient matrix between the elements 2, and
Z,+,, allowing for Zt+l, . . . , Zt+,-l, except in the univariate case mentioned in
step 2, This property is one of the reasons that Tiao and Box called 9 ( s ) the partial
autoregression matrix.
4. Even for a general m-dimensional vector process, when s = I, we see from (16.5.43)
that P(1) = p(1). That is, the partial lag correlation matrix at lag I reduces to the
regular correlation matrix at lag 1, which is to be expected because in this case there
are no intervening vectors between 2, and Z,+l. This natural property that also hoIds
for the univariate partial autocorrelation function 41ek,
however, is not shared by 9 ( s )
except in the univariate case when nz = 1.
Recursive Algorithm for Computing P(s) In this section, we discuss a recursive

procedure for computing the partial lag correlation matrices. This method is also mentioned in Ansley and Newbold (1979a) and provides a vector generalization of Durbin's
(1960) recursive computational procedure for univariate partial autocorrelations. Box,
Jenkins, and Reinsel (1994, p. 87) give a nice summary of Durbin's procedure.
Given the regressions from (16.5,32) and (16,5.33),for s 2 2,
412
CHAPTER t 6 Vector Time Series Models
we want to consider the regressions

S
Zt+s+l =
2
k= I
ff,
kZt+s+~-td- Us, t+s+l,
Corresponding to the definition of the partial lag cornelation matrix, our interest is in the correlation between v,, and us, ,+s,l; thus, we require the computation of the multivariate linear
regression coefficient matrices u s , k and p,, k in (16.5.47) and (16.5.48). Let
be the regression of the residual us-1, ,+, from (16.5.45) on the residual v,-1,
(16.5,46), where the multivariate linear regression coefficient is given by
1Yf
, from
COV(U~-I.
t+r, vs-I, t) [ ~ = ( ~ s -t)l 1-I
,
= VL ( s ) [V,(s)]-'
Note from (16.5.44) that a: is, in fact, equal to the partial autoregression matrix at Iag s.
From (16.5.47), however, the partial autoregression matrix at lag s is equal to a,, Therefore, in the regression relation (16.5.49), a1 = a , ,and uf+, = us,,+,+l. Substituting for
us-,, ,+, and vs-I,, from (16.5.45) and (16,546) in the regression (16.5,49) gives
,.
Hence,
Because the multivariate linear regression coefficient matrix that defmes the regression of
Z,+, on the carriers Zt+,-l, . . . , Z, is the same as those that define the regression of Zt+,+l
on the carriers Z,+,, . . . , Zt+r,the coefficients as,in (16.5.47) can be computed from
and
Similarly, the multivariate linear regression coefficients P , k i n (16.5.48) can be found by

considering the regression of vS-l., on us-1, +
,, is.,
413
and
This development leads naturally to the following recursive procedure for computing
are obtained using (16.5.39),(16.5.41),
and (16.5.42),
P(s) in which V,(s), Vv(s),and V,&)

respectively. For s = 1,
s- l
vU(s) = r(0)- k=21ffs- 1, kr(k)

Vv(s) = r(O)-
s-
k=
Ps-l,krf(k)
[from (16.5.39)1,
[from (16,5.41)],
The partial lag correlation matrix at lag s is, from (16.5.43),
where D,fs) is a diagonal matrix whose ith diagonal element is the square root of the ith
diagonal element of Vv(s)and D,(s) is similarly defined for V,(s).
Zn addition, vve note that this algorithm aIso provides 9 ( k ) , the partiat autoregression
matrix. The matrix coefficients for fitting Z, to a vector autoregressive process of order s can
be computed from 8 ,k = as,as in (16.5.50).
Follo~vingQuenouille (1957,p. 40), Ansley and Newbold (1979a) define the multivariate
partial autocorrelation matrix at lag s to be
414
where W,(s) and W,(s) are the symmetric square roots of V,(s) and V,(s), defined such that
[W,(S)]~ = V,(s) and [W,(s)12 = Vv(s). Like 9 ( s ) , however, Q(s) js not equal to the correla. , and
tion coefficient matrix between the elements of Z, and Z,+,, allowing for
Z,+,-l. In fact, its elements are not correlation coefficients except in the univariate case in
which S ( s ) = Q ( s ) = P(s).
..
Sample Partial _Lag Correlation Matrices Estimates o_f the partid lag correlation
matrices, denoted_P(s), are obtained by using r ( j )in place of r ( j )f o r j = 0, 1, . . , s - 1
in P(s). Because P(s) is a proper correlation matrix, the results of sample correlation matrices can be used for its inference. Under the null hypothesis that Z, is an autoregressive
process of order s - 1, the two residuajs u,-r, I+s and v,-1, are uncorrelated white $oise
series. U$ng Quenouille (1957, p. 41) and Hannan (1970, p, 400), the elements of P(s),
denoted I$(s), are independent and asymptotically normally distributed with mean 0 and
variance 1/11. Thus, we can use Tia? and Box's
-, notation introduced at the end of
Section 16.5.1 to indicate values of ej(s) that are larger than 2/1/;;, less than -2/2/;;, or
between -2/\/;; and 2 / G . In addition, n[Z;(s)12 is asymptotically distributed as a X2 with
one degree of freedom, which implies that
+,
is asymptotically distributed as a X2 with m2 degrees of freedom. The term X(s)provides a

diagnostic aid for determining the order of an autoregressive model.

Once a vector ARMA model in (16.3.1) is identified, the methods discussed in Chapter 7
for the univariate case can be generalized to estimate the associated. parameter matrices
= (@I, . . . , a,), 0 = (81, , . , , B,), and X. For example, given Z = (ZI, Z2, . . , Z,)
from a zero mean Gaussian vector ARMA(p, q ) process, the log-likelihood function is
given by
fnL(@, 8,812)
=
nnt
11
-ln27r - -1nIXI
"
,=I
--
a; X-la,
where
a,
2,- cPIZ,-l
- - - . - @,Z,-,
+ - -1- @,a,-,
(16.6.2)
415
and
Alternatively, because 2, fort 5 0 are not available, (16.6.1) can be replaced by the conditional
log-likelihood function
where
..
and the ap+~-,, . ,a, are assumed to be zero. Clearly, maximizing the log-likelihood function
in general leads to a nonlinear estimation problem.
Properties of the maximum likelihood estimators obtained from (16.6.4) have been discussed by Nicholls (1976, 1977) and Anderson (1980). Let q be the vector of all parameters
and be the corresponding maximum likelihood estimator. They have shown that 6 is asymptotically unbiased and consistent and that fi(G - q) has an asymptotic multivariate normal
distribution, i.e.,
where --+denotes convergence in distribution, Z i is the inverse of the information matrix
and the derivatives are evaluated at the true parameter vector q.

If the process Z, has a nonzero mean vector p,it can be estimated by the sample mean, i.e.,
where Z is the vector of sample means. The sample mean is then subtracted from the series
prior to estimating the parameters q.It can be shown that
where
and fi = Z is asymptotically independent of 6. Hence, asymptotically,this result is equivalent

to jointly estimating p with the other parameters by the maximum likelihood method.
41 6
The exact likelihood function for vector time series has been studied by Osborn (1977),
Phadke and Kedern (1978), FLillmer and Tiao (1979), Nicholls and Hall (1979), and Hannan,
Dunsmuir, and Deistler (1980). The estimation algorithms are complicated and in general
very slow, Z a o and Box recommend using the conditional method in the preliminary stages
of model building and later switching to the exact method.
Estimation of a vector time series model often takes multiple stages. Initially, we would
estimate every element of a parameter matrix. Based on the initial unrestricted fit, we might
consider the restricted fit by retaining only the statistically significant coefficients with the
nonsignificant coefficients set to zero.
Once the parameters have been estimated, one should check the adequacy of the fitted
mode1 through a careful diagnostic analysis of the residuals
whey Z,is now used to denote Z, if p = 0 and to denote (2, - f i ) orhenvise and where the &i
and ej are estimates of the parameters $and
For an adequate model, the residuals should be
white noise. Hence, the correlation matrices of it
should be insignificant and have no pattern,
Overall x2 tests based on the sample comlations of the residuals are suggested by Hosking
(1980) and Li and McLeod (1981). A close inspection of the correlation sfructure, however, may
reveal subtle relationships that suggest directions of improvement in the proposed model.
Forecasting for a vector ARMA model may be carried out in exactly the same way as
in the univariate case discussed in Chapter 5. The recursion (5.3.3) for computing forecasts,
the computation of the qjweights (5.2.3) or (5.2.26), the forecast error variance (5.2.9), the
forecast limits of (5.2.10), the autoregressive representation of forecasts (5.4.3) and (5.44,
forecast updating (5,5.5), and the eventual forecast function (5.6.2) are generalized directly
to the vector case simply by replacing scalars by vectors and matrices. For example, the
{-step ahead minimum mean square enor forecast can be recursively calculated by
where Zn(j ) = Z,+j forj 5 0,E(al+j) = an+jfor j

cast error variance-covariancematrix is given by
5 0,
and E(anq) = 0 for j
> 0. The fore-
where the 'Pj matrices are computed using (16.3.5) and qo= I .

-
Recall that the analysis of the Lydia Pinkham data performed in Section 14.3.5 indicates that
there is a feedforward and feedback relationship between advertising and sales. We now apply the
joint vector time series approach introduced in the preceding sections to andyze this data set.
417
Only a few time series software programs are available for modeling vector ARMA models.
These include SCA, MTS, and the newer version of SAS. SCA 1997 implements the partial autoregression matrices proposed by Tiao and Box (1981), and MTS, a multivariate time series program developed by Automatic Forecasting Systems, h c . (1987), implements the partial lag correlation matrices suggested by Heyse and Wei (1985a, b). We use MTS in the followving analysis.
16.7.1 Model Identification
Let Z1,, be the series of annual advertising expenditures and &, be the series of annual sales
for the Lydia Pinkham data between 1907 and 1960 as plotted in Figure 14.3. As shown in
Section 14.3.5, there is a need for differencing the data. Table 16.1 shows and Figure 16.1
plots the sample correlation matrices along with their indicator symbols for the differenced
series of advertising and sales from lag 1 through 10. We note that the (i, j> element of the
lag k matrix, > ( k ) , is the estimate of pii(k) if i 5 j and of p&-k) if i > j. The pattern of
bll(k) indicates a possible low-order AR model for the series (1 - B)ZI,, and the pattern of
c22(k)implies a possible MA(1) model for the series (1 - B ) q
For more clues, we examine the partial lag correlation matrix function together with the
indicator symbol and summary statistics shown in Table 16.2. The result indicates a possible
AR(2) model for the data, i.e.,
,.
( I - Q I B - a2B2)(1
- B ) Z t = a,,
(16.7.1)
16.7.2 Parameter Estimation

Once a tentative model is identified, efficient estimates of the parameter matrices Qi and X are
determined by maximizing the likelihood function. The results of the estimation for Equation
(16.7.1) are shown in Table 16,3. The parameters were estimated in three stages. In the first
TABLE 16.1 Sample correlation matrices for the differenced Lydia Pinkham data.
418
FIGURE 16.1 Sample auto- and cross-correlations for Lydia Pinkham data.
stage, all dements of
and Q2 were estimated with their estimated standard enors. Those
elements of Q2 that {yere less than 1.5 times their estimated standard etrors were set to zero
2) and Q2(2, 2). After the second estimation stage,
for the second stage. T$ey included e2(l,
attention focused on
Because ar(2,1) was less than 1.5 times its estimated standard
error, it too was dropped from the model. The third stage estimates represent our fitted model
419
TABLE 16.2 Sample partial lag correlation matrix function for differenced
Lydia Pinkham data.
6@I
k
1
2
Indicator symbols
[ :;; I:
[ : * I]
[I T]
[: :]
[: :]
[ : -1
[: :]
::;;I
[-.a
-.32
5
6
[-: I::[ 2 I:-
[-:;:I:;;]
[
-.05
-.20]
X(k1
P-value
17.59
,001
25.00
,000
3.61
.479
5.53
.242
7.45
.I15
2.70
.618
TABLE 16.3 Estimation results for Equation (16.7.1).
[I) Full mode1

(.17)
(.I61
(2) Intermediate
(-16) (.17)
-.27
(,16)
(-16)
[:a:
(3) Final model

t.13)
and
43.301
420
TABLE 16.4 Residual correlation matrices for Equation (1 6.7.2).
16.7.3 Diagnostic Checking

To guard agynst modelAmisspec%cation,a detailed diagnostic analysis of the residual series
it = 2, - @lZI-l - @2Z1-2is necessary. Such an analysis includes plots of standardized
residuals against time and analysis of residual correlation matrices. Tabie 16.4 gives the residual
correlation matrices along with their indicator symbols for lags 1 through 4 that indicate that the
restricted AR(2) model (16.7.2) provides an adequate representation of the data.
16.7.4 Forecasting
For illustration, we use Equation (16.7.2) to compute the one-step ahead forecast of Zs5 from
the forecast origin t = 54. First, we note that after multiplying the matrices and rearranging
terms, we can rewrite Equation (16.7.2) as
and
Hence, given Z1, = 608, . . . , Zl, 52 = 639, Z1,53 = 644,Z1,54 = 564, &, 1 = 1016,
&,5.2 = 1390, Z2,53 = 1387, and Z2,54 = 1289, we have
., .,

*
421
&.
54(1) be the one-step ahead forecast

Let el, 1 = Zr,55 - Zl,54(l) and e*,1 = &, errors. Note that el, I = alS55
and,e2,1 = a2,55, and the forecast error variance for [el, 1, ez, is
given by
T%us,Var(el,I ) = 31.32, Var(e2,1) = 43.30, and Cov(el, 1, e2, 1) = 17.05.
16.7.5 Further Remarks

From Equation (16;7.2), we conclude that cPl(l, 2) > 0.This conclusion confirms our suspicions from the cross-correlation functions estimated from the residual series of the transfer
function model, shown in Table 14.9, that a feedback relationship with sales leading advertising existed in the data. This feedback relationship is even more explicit horn the forecast
equation of the advertising given in (16.7.3).
One should not conclude from (16.7.3) and (167.4) that there is no contemporaneous
relationship between sales and advertising. In our form of ARMA models, we choose a.
go= I, but a general form of the variance-covariance matrix X for the innovation a,. Hence,
the contemporaneous relationship between components of vector series is modeled through
the off-diagonal elements of 2. From 2 in (16.7.21.,, we can estimate the correlation
between the residuals irl,, and
which equals $;I,tqt
= 17.05/v(31.32)(43.30) = .46.
This estimate indicates that advertising and sales were also contemporaneously related,
The advantage of a joint vector model over a transfer function model is evident when there
is a feedback relationship between variables. It should be emphasized, however, that the above
results by no means imply that the proposed vector model is the best choice for describing the
advertising and sales phenomena of Lydia Pinkham Medicine Company. Model building is
a delicate process that also requires proper understanding of related subject matters under
study. In this case, these matters include the sales, marketing, and operation of the company.
The above example is used to illustrate a vector time series model and its impfications for the
data set when only the past time series observations are used. For an interesting study of building a model for the Lydia Pinkham data using other external information, the reader is referred
to Palda (1964), among others.
q,,

The spectral results in Sections 12.1 and 14.5 for the univariate and bivariate processes are
readily generalized to the m-dimensional vector process. For a jointly stationary nt-dimensional
vector process Zt = [ZI, Z2,,, . ,Zm,t l l , the spectral representation of 2,is given by
,,
..
422
where dU(w) [dUl(w), dU2(w), . . . ,dU,!w)ll and the dUi(w), i = 1,2, . , , nt, are bothi
orthogonal as well as cross-orthogonal. The spectral representation of the covariance matrix
function is given by
where
[ E( d Ui (o)dU; (w ) )1
[dFi,.(w>],
(16.8.3)
d ~ * ( wis) the transpose of the complex conjugate of dU(w), and F(w) is the spectral distribution matrix function of 2,.The diagonal elements Fiio) are the spectral distribution functions
of the Zi and the off-diagonal elements Fij(o) are the cross-spectral distribution functions
between the Zi, and the Zj,
If the covariance matrix function is absolutely summable in the sense that each of the m X m
sequences yg(k) is absolutely summable, then the spectnun matrix or the spectral density matrix
function exists and is given by
f.
Thus, we can write
and
f ( w ) = 2~
2 T( k ) e-iwk.
The element fii(w) in (16.8.4) is the spectrum or the spectral density of Zi,
,, and the element
fj,(m) in (16.8.4) is the cross-spectrum or the cross-spectrd density of Zi, and Zj, It is easily
seen that the spectral density matrix function f (w) is positive semi-definite,i.e., c'f(w)c 2 0
for any nonzero m-dimensional complex vector c. Also, the matrixf (w) is Hermitian, i.e.,
,.
Hence, h ( w ) = fz(w) for all i and j.

The results of bivariate frequency-domain analysis can be used for the multivariate
case. Specifically, the cospectrum, phase spectrum, gain function, and coherence may now
be defined for any pair of component processes in Z,.
Supplement 16.A Muttivariate Linear Regression Models
423
...
..
Let Yl,
, Y, be the m response (dependent) variables and XI, . , Xr be a common set
of r predictor (independent) variables, Assume that each response follows its own multiple
regression such that
where the error term a = [al, . . . , a,]' has mean E(a) = 0 and variance-covariance
Var(a) = X. Now suppose that there are a total of N trials in the system of (16.A.1) and
that we let Yi = [Yil,
. . . , Yim3' denote the responses for the ith trial, [l, Xil, , X,]
denote the constant term and predictor variables for the ith trial, and ai = [ail, . . , a!,]'
denote the corresponding errors. We can represent the system in the matrix form
...
or, more compactly,
Y =
NXfn
N X ( r + 1) (r+I)Xm
E ,
NXm
where Y, X, p, and E denote the matrices of responses, predictors, parameters, and errors,
respectively. For clarity, we also list their dimensions underneath. If ai and aj are uncorrelated for i # j, then the error specification becomes
and
The model given in (16.A.2) or (16.A.3) is the so-called multivariate linear regression model.
Assume further that rank (X) = (r 1) < Nand ai N(O, Z) for i = 1, . . . , N. We have
the following results:
1. The MLE for P is fi = (XiX)-'X'Y.

fi N[P,3,I:(X'X)-'1.
2.
424
and ~
4.
f :WN-r-l(X), a Wishart distribution with (N - r - 1) degrees of freedom.

and 2 are independent.
e-N1n/2
5.
~ ( b 5)
, = ( 2 n ) ~ 1m51~W' where L (.) is the likelihood function.
In certain applications, it may be of interest to test the null hypothesis that some of the
predictor variables are unnecessary. Let us label these predictors [Xk+t,
Xk+21 . . . ,Xr] and label
the corresponding parameters for the ith response [/3(k+l)i,P(kf2)i,. . . , Pn.],i = l r . ,I l l .
By partitioning the matrices X and P for the N trials presented in (16A.3) accordingly, i.e.,
..
we can rewrite (16.A.3) as
The hypothesis that the predictors in X2are unnecessary is equivalent to testing that Ho : P(21= 0
versus Ha: p(2)# 0. Under the hypothesis we have
Let
c(l)
be the estimate of
becomes
and
(x;x~)-~x;Y
for the reduced model of (16.A.6). The estimate of X in this case now
425
Let fl be the original parameter space for (16.A.3) and nobe the reduced parameter space
under Ho. To test the null hypothesis Ho, we can apply the likelihood ratio test
It is welf known that
which follows approximately a chi-square distribution with the degree of freedom equal to
[dim(St) - dim(&)], i.e., X2((r- k)m). For a better approximation, one can use the modified
statistic due to Bartlett (1947),
It is clear that 0 < A 5 1. We reject the null hypothesis when A is smail or -21n A and
Bartlett's modified statistic is large.
In the vector AR(p) modeI, we have
,,
, ,I1, T = {T,, . . . , T,],'

the mi are m X m coefficient matrices, and
where Z, = [Zl, . . . ,Z
the at are Gaussian vector white noise N(0,X). Thus,
When testing the hypothesis
we have N = ( n - p), r
= pln,
and k = (p - 1)m.
426
CHAPTER I 6 Vector Time Series Models
Determine the stationarity and invertibifity of the following two-dimensional vector

models:
(a) (I - QIB)Zt = a,, where GI=
(c) Zt = (I - 81B)al, where Bl
[:!:36],
18],
12
5
[:4
and z = I
and Z = I.
(d) (I - GIB)Zt = (I - @$)a,, where @I =
3
: 6 ] 3'1
.4
= [.3
.2
,6]'md'
=I.
(e) (I - QIB)Z, = (I - 81B)a, and a, N(0, Z), where
16.2 Consider the following simple demand and supply model:
Q, = ao 3- ale + al,t
Qt = Po + @&-I + a2, t
(demand)
where Q, represents quantity demanded at time t and Pi represents the price at

time t, and at = [ a l , a;?,,I' is a two-dimensional stationary vector process.
(a) Express the model for [Q,, Pt]' as a joint vector process if a, is a vector white noise
process.
(b) Repeat the problem in part fa) if al, = (1 - OllB)bl, and az, = (1 - 822B)b2,t,
where b, = [bl, ,, bz,,]' is a vector white noise process.
,,
16.3 Find the marginal process Zl,, for the models given in Exercise 16.1.
16.4 Consider the models given in Exercise 16.1.

(a) Find the covariance matrix function, r(k),for k -. 0, *l, zk2, and f3.
(b) Find the correlation matrix function, p (k), for k = f1,f2, and f3.
16.5 Find the AR and MA representations for the models given in parts (d) and (e) of
Exercise 16.1.
where 141 < 1, 181 < 1, and a = [al,t, az, t]' N ( 0 , Z ) .

(a) Write the process in a vector form.
(b) Is the process [Z1,,, 5,
,I' stationary7 Invertible?
Exercises
427
(c) Write down the model for the vector of first differences (I - IB)Z,, where
2, -- [Z,, &,,I1. IS the resulting model stationary? Invertible?
,,
16.7 Calcufate the partial autoregression matrices 9 ( k ) with k = 1, 2,
. . , 10, for the
models given in parts (a) and (c) of Exercise 16.1.

Calculate the partial lag correlation matrix function P(k) with k
for the models given in parts (a) and (c) of Exercise 16.1.
1, 2,
. . . , 10,
Calculate the one-step ahead forecast Z,(l) and the forecast error variance for the
model given in part (d) of Exercise 16.1.
Consider the U.3, housing sales and housing starts given in Exercise 14.5.
(a) Calculate the sample correlation matrix function ji(k), with k = 1, 2,
for the original series between 1965 and 1974.
(b) Repeat part (a) for the properly differenced series.
. . . , 15,
Repeat Exercise 1610 for the partial lag correlation matrix functions P(k).
Complete the construction of a joint vector model for the data of U.S. housing sales
and starts between 1965 and 1974 from the information obtained in Exercises 16.10
and 16.11.
Forecast the housing starts for 1975 using the joint vector model and compare the
result with the forecast obtained in Exercise 14.5.
Build a joint vector model for the five series of U.S. hog data given in the appendix
as Series W13.
Let n be the number of observations for the vector ARfp) model given in (16.5.17).
Show that to properly estimate the parameters in the model, it is required that
2
,(
+i)m
+ f.
More on Vector Time Series

Although the basic procedures of model building between univariate time series and vector
time series are the same, some important phenomena are unique in vector time series. In
this chapter, we introduce concepts of cointegrakion, partial processes, and equivalent representations of a vector time series model. Understanding these concepts is important in the
analysis and model building of vector time series.
17.1 Unit Roots and Cointenration in Vector Processes

A univariate nonstationary process or series Zt is said to be integrated of order d, denoted as I(d), if its (d - 1)th difference is nonstationary but the dth difference, Adz, =
(1 - Bld2, is stationary. Thus, Z, is Ifl) if Zt is nonstationary but its first difference,
AZ, =: (1 - B)Z,, is stationary. Under this notion, an I(0) process will be stationary.
Because the difference of a stationary process is stationary, however, to restrict ourselves to a real integrated process we define an I(0) process to be a stationary process
where in terms of its MA representation, 2, = $(B)n, =
t/.~~a,-~,
we have $(I) # 0.
Under this restriction, the differenced series, AZ, = (1 - B)Z,, of a stationary process,
Zt = B(B)a,, where B(B) =
and 2,Fl~9j[
< W, is not I(0) because the resulting MA representation of the differenced series is AZ,
6(B)(1 - B)a, = +(B)a, and
+(I) = 0. Thus, for 2, to be Z(1) we mean that AZ, = (1 - B)Z, is not only stationary but
is actually I ( 0 ) .
As a nonstationary process, an integrated series can wander extensively. Economic and
market forces, however, tend to keep many of these integrated series together and form equilibrium relationships. Examples include short-term and long-term interest rates, income and
consumption, and nominal GNP. This leads to the concept of cointegration.
An (m X 1) vector time series 2, is said to be cointegrated of order (d, b), denoted as
CI(d, b), where 0 < b 5 d, if each of its component series ZL is I(d) but some linear combination of the series j3'Zt is I(d - b) for some nonzero (1 X m) constant vector P'. In such a
case, p' is called a cointegrating vector which is sometimes also called the long-run parameter. The cointegrating vector is clearly not unique because if P'Z, is stationary, then so too is
cp'Z, for any nonzero constant c. Hence, cp' is also a cointegrating vector. The most cornmon case is d = b = 1, and it wiU be used in our discussions.
zFo
z;~B~B~,
17.1 Unit Roots and Cointegration in Vector Processes
429
The dangers of differencing all the components of a vector time series was reported by
Box and Tiao (1977), who showed that nonstationarity of a vector process can be caused by
a small number of nonstationary components. Granger (1986) and Engle and Granger (1987)
further developed these ideas and-proposed the concept of cointegration. This concept has
been widely used in many interesting applications, especially in the area of economics and
finance. See, for example, Kremers (1989).
As an illustration, consider the two-dimensional vector AR(1) process
Clearly, the component ZIT,is a random walk,
For the component &,*,we have 5,= -+Zl,, becomes
+ an, ,.After differencing, this equation
which follows an MA(1) process. Thus, each c.omponent series is an I(1) process. However,
let p' = [# 11, then the linear combination
is clearly I(0). Hence, Zt is cointegrated with a cointegrating vector P' = [+ 11. This interesting phenomenon is shown in Figure 17,1, where Z1,, and Z2, are each nonstationary, but
their linear combination is stationary.
More generally, if the (rn X 1) vector series Z, contains more than two components,
each being I ( l ) , then there may exist k (< m) linearly independent (1 X m) vectors
Pi, Pi, . , Pi such that P'Zt is a stationary (k X 1) vector process, where
..
is a (k X m) cointegrating matrix. Clearly, vectors Pf are not unique, neither is Pi. If for
any other (1 X m) vector b' that is Linearly independent of the rows of P', however, we
have that bJZ,is nonstationary, then the 2, is said to be cointegrated of rank k. The vectors
pi, Pi,. . . , form a basis for the space of the cointegrating vectors known as the cointegration space.
430
FIGURE 17.1 A cointegrated process where Z1,, (a heavy solid

line] and Zz, (a dotted line] are each nonstationary, but their linear
combination Yt fa thin solid line) is stationary.
17.1.1 Representations of Nonstationary Cointegrated Processes

Representation Because each component of 2,is I(l), the implication is that
AZ, = (I - B ) Z , is stationary, Let e0 = E(AZ,). We can write AZ, in the MA form
AZ,
O0
= 'P(B)a,,
(17.1 -4)
where a, is the vector white noise process with mean E(aJ = 0, variance-covariance matrix
E(s,s:) = Z, q ( B ) =
q j B j , q,, = I, and the coefficient matrices qj is absolutely
summable.
Let X, = *(B)a,. Then, AZ, - O0 = X,and
zFo
Z, = Zpl
+ Oo + XI.
Applying a direct substitution for Z, - in (17.1.5) gives
(17.1.5)
431
A straightforward extension of Equations (9.4.5) and (9.4.7) to vector processes gives
where Y , = x;o'P:a,-j
and P; = - E E I ~ +which
i is an absolutely sumable matrix
sequence. Hence, the vector process Y, is stationary. Combining (17.1.6) and (17.1.7), we get
and
P'Z,
@'(Zo - Yo)+ P'eOt + f3'T(l)[al -t- a2
+ . + a,] + P'Y,.
*
(17.1.9)
+ + -+
Clearly, the process br[al a2

. a,] is nonstationary for any nonzero (1 X m)
vector b'; hence, (17.1.9) implies that P'Z, will be stationary if and only if
and
Equation (17.1.10) places a restriction on the nonzero drift across the components, whereas
Equation (17.1.11) says that the determinant I ~ ( B ) \ = 0 at B = 1; hence, Q(B) is noninvertible. By (17.1.4), this implies that we cannot invert the MA form and hence can never represent a cointegrated process with a vector AR form in AZ,. The vector AR representation of a
cointegrated process must be in terms of Z, directly.
EXAMPLE 17.1 It can be easily shown from (17.1.2) and (17.1.3) that the MA representation of the two-dimensional cointegrated process given in (17.1.1) is given by
0
Note that
*(B) =
and
*(l) =
[-4
O]
0
0,
but lq(1)) = 0. Thus, !PCB) is not invertible, and we cannot represent the process in an AR
form in terms of AZ,.
AR Representation Now, suppose that 2, is nonstationary and can be represented as an

AR(p) model
432
such that I@,(B)~= 0 contains some unit roots, where @,(B) = I - a I B each component of Z, is 1(1), then, from (17.1.4),
( 1 - B ) Z , = O0
. .-
BP. I f
+ 'P(B)a,.
Hence,
Multiplying (1 - 3)on both sides of (17.1.12) and comparing with (17.1.141, we have
where we note that (1

implies that
- B)Bo = 0. Because Equation (17.1.15) holds for any a,, however, it
and
for any B. Hence,
Comparing (17.1.10) and (17.1.1 1) with (17.1.16) and (17.1.171, we see that in terms of an AR
representation, the matrix @,(l) must belong to the space spanned by the rows of P'. That is,
for some (m X k) matrix M.

EXAMPLE 17.2 Although an integrated process cannot be represented in an AR form in
terms of AZ,, its AR representation does exist in terms of 2, itself. Recall the two-dimensional
cointegrated process and its AR presentation in terms of Z, given in (17.1.1):
In this case,
433
and the cointegrating matrix
P'
14 11.
Thus, Z, is a cointegrrtted process of rank k,= 1. Note that

%(I)
0
[$
= MB' =
MI4 11,
where
Error-Correction Representation Next, note that in the AR representation, the matrix

polynomial Op(B) can be written as
Qp(B) = I
- QIB -
QPBP = (I - AB)
- ( q B + . . + rup-l~p-') (1 - B),
(17.1.19)
+ --+
where A = @,
.
@, and 9 = -(Qj+1
(17.1.12) can be rewritten as
Z, = O0
Subtracting
+ Qp) f o r j = 1,2, . . . , p - 1. Thus,
+ AZtil + ( Y ~ A Z +~ - ~ + L Y ~ - ~ A Z+~ a,,- ~ + ~ (17.1.20)

a
from both sides of (17.1.20) gives
AZf = e0 + yZ,-$ -t- alAZ,-l

where y
A - I = --@,(I)
+ - . . + ru,-lAZ,-,+I + a,,
(17.1.21)
-MP' by (17.1.18) and (17.1.19). Hence,
for some fm X k) matrix M, where Y, - 1 = P'Z,-* is a (k X $)-dimensionalstationary

process. Equations (17.1.21) and (17.1.22) imply that the differenced series AZ, of a cointegrated process 2, cannot be described using only the values of past lagged differences, AZj,
for j < t. The process must include an "error-correction" term, MY,-l = MP'Z,-l. If we
regard the relation AZ, in terms of its own past lagged values AZj for j < f as a long-run
equilibrium, then the term Ytml = P'Zf-l can be viewed as an error from the equilibrium
and the coefficient M is the required adjustment or correction for this error. Accordingly,
434
CHAPTER 1 7 More on Vector Time Series

Equation (17.1.21) or (17.1.22) is known as the error-correction representation of the cointegrating system. The representation was first proposed by Davidson et al. (1978).
EXAMPLE 17.3 For the cointegrated process given in (17.1.1),
we have p = 1, O0 = 0, Qj = 0 for j 2 2, and hence aj= 0 for j 1 1. From Example 17.2,
M=
[i]
and
&-I
= p'Z,-I =
[4
11[
Z1, t-1
2 2 , r-1
@l,t-l
+ Z2,,-1.
Thus, the error-correction representation of (17.1.1) is given by
where Yt-l
+Z1,
+ 5,t - l
is the stationary error-correction term.
17.1.2 Decomposition of 2,
Assume that nonstationarity of Zt is due to unit roots in the AR matrix polynomial. That is,
in the AR representation
Qp(B)Zt = 00 + at,
(17.1,23)
some of the raots of I @ , ( B ) /

= 0 are equal to 1. More specifically, assume that there are h (5I )
unit roots and that all other roots of J@, ( B ] I
= 0 are outside the unit circle. Thus,
]@p(l)[= 11 - Zf'=l@jl
= 0; the matrix I&l@j
has h eigenvdues equal to 1 and ( ~ n- h)
other eigenvalues less than 1 in absolute value. Let P be an ( m X m) matrix such that
P-l[Z&l@j]~= diag[Ih Ak]= J, where k = (m - h) and J is the Jordan canonical form
of Ef= @j. Then,
r P
Let P = [S R] and (P-')'

ate (nl X k) matrices so that
[H
PI, where S and H are (nt x
h) matrices, and R and
435
Recall that y = -@,(I) in (17.1.21),which can therefore also be written as
AZ,
= Q0
+ C Y ~ ~ ' Z+, alAZt-l

-~
+ - . + ap-1AZ,-p+l + a,,
(17.1.26)
where a = R(AA- 19 = -M. It follows that although 2, is nonstationary, the k (= fnt - h))
linear combinations BIZr are stationary, where k = rank(@,(l)) = rank(Pf). That is, 2, is
cointegrated of rank k.
Let Y1,,
= HrZ,. Because P-' is nonsingular, the rows of H' are linearly independent of
the rows of pi. Thus, Yl,, must be nonstationary. In fact, rank(Hr) = h, which also equals
the number of unit roots. This h-dimensional Yl,process is actually the underlying common
stochastic trend related to the k unit roots, which is aIso known as the common factors in the
process of Z,. It is the common driving force for the nonstationary phenomenon in each
component Zi, of 2,.
Let Y2,, = PfZf.Then,
and
Hence, the cointegrated process 2, is a linear combination of the 11-dimensionalpurely nonstationary component, Y1,,
= H'Z,, and the k-dimensional stationary component, Y2, = plZI,
where k = (m - h). The concepts discussed in this section are closely related to the common
factor analysis studied by Pena and Box (1987), Stock and Watson (1988), and many others.
17.1.3 Testing and Estimating Cointegration

Following the above discussion, we see that the first step in testing cointegration is to test
the null hypothesis of a unit root in each component series Zi individually using the unit
root tests discussed in Chapter 9. If the hypothesis is not rejected, then the next step is to
test cointegration among the components, i.e., to test whether Y, = P'Z, is stationary for
Sometimes the choice of the matrix/vector p' is based on some
some matrix or vector
theoretical consideration. For example, if Z, = [Z1,, Z2,J', where Z1,, represents revenue
and Z2,,represents expenditure, we may want to test whether the revenues and expenditures
are in some long-run equilibrium relation and therefore whether Y, = Zl, - Z2, is stationary. In such a case, we choose pr = [l -11.
Tn testing for cointegration in 2,with a known cointegrating vector P', we can fomulate the null hypothesis to test whether the process Y, = PfZ, contains a unit root so that we
can again use the tests discussed in Chapter 9. We will conclude that Zt is cointegrated if the
null hypothesis is rejected.
When the cointegrating vector is unknown, we can use the following methods to test
and estimate the cointegration.
a'.
436
CHAPTER 17 More on Vector Erne Series
The Regression Method First, note that if the m-dimensional vector process
Z', = [ZI,,, Zz,,, . . . ,Z,,,] is cointegrated, then there is a nonzero 111 X 1 vector p' =
[el, ~ 2 ., . . , c,] such that P'Z, is stationary. With no loss of generality, say cl # 0. Then,
(l/cl)P1 is also a cointegrating vector. Thus, a very natural approach that was also suggested
by Engle and Granger (1987) to test and estimate cointegration is to consider the regression
model for Zl,
and check whether the error series E, is I(1) or IfO). If E, is I(1) and contains a unit root, then
Zt cannot be cointegrated. Othenvise, if E, is I(O), then Z, is cointegrated with a normalizing
cointegrating vector given by P' = [ l ,
. . . , The results remain the same for the
above model with a constant term included.
There are many subtle points for the regression model in (17.1.29). In drawing inferences
using the OLS estimated model, the following should be noted:
.I,$
e
series for nonstationarity, we calculate the OLS estimate,
1. in t e ~ t i n g ~ t herr2r
P' = [ 1, 41, ,#rnJ and use the residual series i, for the test.
2. The estimates 4i do not have asymptotic t-distributions, and the standard t-test for
#i = O is not applicable unless E, is I(0). As indicated in Chapter 15, the regression in
(17.1-29)will produce spurious regression if E, is non~tationary.~
3. If 6, is I@) and Z, is cointegrated, then we can let Y,-l = P'Z,-l in (17.1.22) and
perform the estimation of the error-correction model.
4. We estimate the cointegrating vector by normalizing the first element to be unity.
Clearly, we can normalize any nonzero element ci and regress 4 , on other variables in
estimating the regression. Although most of the time the results are consistent, inconsistent conclusions could sometimes occur, which is a weakness of the approach. Because
of its simplicity, however, the approach is still very commonly used.
. .:
To test whether the error series st is I(l), Engle and Granger (1987) proposed several
test statistics, including the standard Dickey-Fuller test and the augmented Dickey-Fuller
test. For these later tests, recall from Equation (9.4.22), in terms of the error term process in
(17,1,29), that to test for a unit root in the general case we either test Ho : p = 1 against
HI : q < 1 for the model
or, equivalently, test Ho : h = 0 against HI : h < 0 for the model
In testing, note that we use the residuafs i,obtained from the OLS fitting of (17.1.29), but they
are only estimates and not the actual error terns. To adjust for the difference, instead of using
TmLE 17.1 Critical values of T, for Ho : A

Level of significance:
=0
against HI : h
1%
437
< 0.
5%
the standard Dickey-Fuller tables, Engle and Granger (1987) conducted a Monte Carlo experiment on the model
and produced the critical values given in Table f7,l for the t-statistic, T = h/S$., One should
reject the null hypothesis if the T-value is less than the critical value and conclude that cointegration exists among the variables in the vector.
EXAMPLE 17.4 From the discussion of this section, it is clear that in modeling vector
time series, one should not automatically difference the component series even if they are
each nonstationary. In fact, in terms of a vector AR model, the results of this section show
that if the component series are cointegrated, then a stationary vector AR model for the differenced series does not exist. One should either construct a nonstationary AR vector model
for the original series or construct an error-correction mode1 for the differenced series. Let us
now further examine the Lydia Pinkham data discussed in Chapter 16 and test whether the
advertising and sales series are cointegrated.
Let Zl, be the advertising series and Z 2 , , be the sales series. To test whether they are
each I(d), first, we obtain the following separate OLS estimated regression for each series:
where the values in the parentheses under estimates are their standard errors. Thus, T =
(-8251 - f)/.07965 = -2.196 and T = (.9222 - 1)/.05079 = - 1.532, respectively, At
5%, the critical value from Table G of the appendix is -2.93. Thus, the hypothesis of a unit
root is not rejected for either series, and both ZIT,and &, are I(&, with d I1, To find the
order of d for the two series, we now consider the following OLS estimated regressions on
their differences:
438
where the values in the parentheses under estimates are their standard errors. Now, T =
(.(I253 - 1)/.1409 = -6.918, and T = (.429 - I)/. 1278 = -4.468, respectively,At 5%, the
critical value kom Table G is -2.93. Thus, the hypothesis of a unit root is rejected for either
differenced series, and we conclude that both ZIP,and &, are I(1).
Next, we consider the regression model
The OLS estimated equation is
with the estimated regression of (17.1,32) for p
I,
From Table 17.1, because the value of T = -.421/.1177 = -3.58 is less than the critical
value -3.37 at the 5% level, we reject the hypothesis of a unit root. Thus, we conclude that
the residua1 series is I(0) and the advertising and sales are cointegrated.
It will be interesting to see whether the same conclusion will be reached when sales is
regressed on advertising. In this case, we consider the regression model
The OLS estimated equation is
with the following estimated regression of A$ on
From Table 17.1, because the value of T = -.3005/.0972 = -3.09 is not less than the critical values -3.37 at the 5% level, we cannot reject the hypothesis of a unit root. Thus, the
residual series is not I(O), and the advertising and sales series are not cointegrated, which is
different from the earlier conclusion. It should be noted, however, at 1% significance level,
the critical value is -4.07 from Table 17.1. Both cases become insignificant, and we reach
the same conclusion that the advertising and sales series are not cointegrated. This may be
expected. For most companies, especially growth companies, we expect their revenues after
expenditures to grow.
Unit Roots and Cointegration in Vector Processes
17.1
439
The Likelihood Ratio Test Let Z, be a nonstationary (m x I) vector process that can
be represented as a vector AR(p.)form
iPp(B)Z, =
where @,(B) = I - a I B written as
Q0
+ a,,
(17.1.33)
iPpBP. When 2,is cointegrated, by (17.1.21), it can be
If in (17.1.33) lap(B)I = 5 contains li unit roots and k = (rt~- h) roots outside the unit circle,
then, from the discussion of Sections 17,l.l and 17.1.2, it is equivalent to stating that the
process Z, contains k cointegrating relations. Under this null hypothesis Ho, we have
for some (m X k) matrix a , and Y, = PIZt is a (k X 1)-dimensional stationary process.

Thus, only k linear combinations, p'Zt-l, of Ztw1,which are stationary, will actually
appear in (17.1.34).
If the vector white noise process at is Gaussian, i.e., if the a, are i.i.d. N(O, X), then given
a sample of (11 p) observations on Z,, i.e., (Z-p+l, Z-p+2, . . . ,Z1, , . ,Zn), the likelihood
function of {Z1, Z2,, . . ,Z,,] conditional on {Z-p+l, Z-p+2,, , , &) is
The MLE of (00, y, a*,

. . . , ap-l, Z) will be chosen so that the likelihood function is maximized subject to the constraint that y can be written in the form of (17.1.35) or, equivalently,
subject to the reduced-rank restriction on y so that rank(?) = k.
To test the null hypothesis, one can use the following likelihood ratio
Under the null hypothesis, some linear combinations of Z, components are nonstationary, so
the asymptotic distribution of -2 In A is no longer a chi-square distribution. This particular distribution was studied by Johansen (1988,1991) and Reinsel and Ahn (1992). They show that
440
...
where j ? ~ ~ + ~ ,, and 1;,(i12

2 fizz 2 . . 2 Cm2) are the (nl - k) = h smallest sample partial
canonical correlations between the components of AZ,and Z,-l given
. . . , AZt-p+l.
Furthermore, when (17.1.34) does not contain a constant k m ,
where W(x) is a h-dimensional standard Brownian motion process, which depends only on
h and not on the orderp of the AR model. The asymptotic distribution of -2 In A when the
model contains a constant term can be derived similarly, and we refer readers to Reinsel
and Ahn (1992) for details. We reject the null hypothesis when A is small or, equivalently,
when -2 In A is large. The approximate critical values of the asymptotic distribution of
-2 In A for the likelihood ratio test of Ho:rank(y) 5 k against a general alternative, for
both cases without and with a constant term, based on Monte Carlo simulations have been
obtained by Johansen (1988) and Reinsel and Ahn (1992). Some commonly used values
selected from Reinsel and Ahn are shown in Table 17.2. The estimate of the cointegrating
matrix can be obtained from the estimation of the error-correction model in (17.1.34).
Note that from the discussion in Section 17.1.2, it is clear that testing for k cointegratjng
relations is equivalent to testing for h = (nt - k) unit roots in the process. When h = 1, the
asymptotic distribution in (17.1.39) reduces to
TABLE 17.2 Critical values for the likelihood ratio test statistic under
the nu11 hypothesis of rankfy) = k = [trt - h), against the alternative of
rank fy) > k, where h is the number of unit roots.
.
-..
Case 1: Without a Constant
Case 2: With a Constant
Probability of a Smaller Value

.90
.95
.99
441
TABLE 17.3 Critical values for the likelihood ratio test statistic under
the nu11 hypothesis of rankcy) = k = (nt - h) against the alternative of
rankCy) = k + f , where h is the number of unit roots.
h
Case I: Without a Constant

.9
.95
.99
Case 2: With a Constant
which can be seen to be the square of the asymptotic distribution of T statistic given in
(9.3.7) that we have used to test the unit root for a univariate time series model.
We may also consider the likelihood ratio test of the null hypothesis of k cointegrating
relations against the alternative of (k 1) cointegrating relations. The test statistic has been
derived by Johansen (1991) and is given by
where jk+l
is the (k
1)' largest sample partial canonical correlation between the components of AZ, and Z,-r given AZ,-I, . . . , AZt-pf The critical values of the asymptotic distribution for the test statistic have been obtained through simulations by Johansen and Juselius
(1990), and some commonly used values are given in Table 17.3.
EXAMPLE 17,5 For the Lydia Pinkham data, we have shown in Example 17.4 that the
advertising and sales series are each I(1). To illustrate the use of the likelihood ratio test,
following (17.1.34) and the fitted model (16.7.1) discussed in Section 16.7, we now consider
the error-correction model
The squared partial canonical comlations between AZt and Z,-I given
and AZti2 are
obtained as if= .2466 and = .1166. Because nt = 2,the rank (y) to be tested will be either
0 or 1.We begin with testing tbe hypothesis that rank (y) = k = 0 or equivalently, h = 2. Now,
;z
442
CHAPTER 17 More on Vector %me Series

which is larger than the 5% critical value 17.97 from Table 17.2. Thus, we reject the null
hypothesis and conclude that the series of advertising and sales are possibly cointegrated.
Next, we test the null hypothesis that rank (y) = k = 1 or equivalently, h = 1. In this case,
which is less than the 5% critical value 8.16 from Table 17.2. Thus, we don't reject the
null hypothesis and conclude that the series of advertising and sales are cointegrated. In
testing rank ( y ) = k = 1, because In = 2, we can also use h = 1 in Table 17.3. The 5%
critical value is 8.08. The minor difference between Tables 17.2 and Table 17.3 for h = 1
comes from different simulations.
If 1% level of significance is used in testing the hypothesis that rank(y) = k = 0 or
equivalently, h = 2, the value of the test statistic 21-99 is less than the critical value
22.79 from Table 17.2. The hypothesis is not rejected; the testing stops, and we conclude
that advertising and sales series are not cointegrated. Thus, at 1% level of significance,
the regression method and the likelihood ratio test reach the same conclusion.

In Section 16.5.3, we discussed the partial lag correlation matrix at lag s as the correlation
(expressed as an rn X m matrix) between the vectors Z, and Z,,,, after removing the linear
dependence of each on the vectors at intervening lags Z,+I, . . . , Z,+,_l. In this section, we
introduce another form of partial correlation proposed by Heyse and Wei (1984), in which Z,
is partitioned into three subvectors, Z, = (Zi?,,Zh,,, Z5, ,)' of dimensions ml,m2, and m3,
respectively. That is, we consider the correlation (expressed as a matrix of dimension m - 1 4 )
between the subvectors (Zi, ,, Zi,,)' and (Zi, , 25,, + , ) I after removing the linear dependence
of each on the third component series Zj,t. Heyse and Wei (1984) call this the partial process correlation matrix at lag s. These correlations are useful in determining whether an apparent linear
relationship between Z1,,and &,,is due to an intrinsic reIationship of each wjth Z3,,.
17.2.1 Covariance Matrix Generating Function

Consider a zero mean covariance stationary vector time series 2,of dimension nt having moving
average representation
...
) is an m X m matrix polynomial in the backshift

where q(B) = - q l B - * 2 ~ 2 operator B, such that the coefficients, q s , are square matrices satisfying the square summability
condition,
443
The covariance matrix generating function for the process Z, is defined as
Extending a result in Box, Jenkins, and Reinsel (1994, p. 85), it can be shown that
where F = 13-'. Writing GZ(B)= [ y @ ( ~ we

) ] ,note that
is the autocovariance generating function for the ith component process of 2,. ' h e variance of the
process Zi yii(0),is the coefficient of BO = 1, and the autocovariance at lag s, yii(s),is the coefficient of BS in (17.2.4). Similarly, yU(B) =
y i i ( ~ ) i~ =s , 1, . . . , m,j = 1, . . . ,rn,
and i # j, is the cross-covariance generating function between the ith and jth components of 2,.
,,
,:x
17.2.2 Partial Covariance Matrix Generating Function
,, ,,
Suppose that Z, is suitably partitioned into 3 subvectors, 2, = (Zi, 25, Z$,,)', of

dimensions nt,, m2, and m3 with ml m2
1113 = ni. Our interest is in the correlation
matrix at lag s for the process [Zi, Z$,,Ip, after removing the finear dependence on the
process Z3,. Partitioning conformably the *(B) and a, the moving average representation for Z,, (17.2.1), can be written as
+
,,
where the components q!tii(B) are matrix polynomials in B with 1Clii(0)= I,, and
t,bij(0) = Ornix,, for i # j. The lag s covariance matrix can be similarly partitioned as
where r i i ( s ) = E[Zi,,
Zj,,+,]. The covariance matrix generating function (17.2.2) for this
particular partition of Z, can be written as
444
where
Note that because rij(s)= rji(-s), the components of GZ(B)are related as
To find the correlation between (Zi, ,,Zi!)' and (Z ;, ,+, Z 5, ,+,)I after removing the linear
dependence of each on the series &, we let Zl,, and i2,t
be the linear projections of Zl,, and
Z;?,on Z3,1trespectively, and consider the correlation matrix function for the residual process
,,
That is, let
and
where the ai and Pi are chosen to minimize, respectively,
and
Taking the partial derivative of (17.2.7) with respect to acj (Graham [1981, p. 541) and setting
the resulting derivative equal to 0 leads to the normal equations
445
Multiplying the jth equation in (17.2.9) by Bj and summing over j gives
Hence,
from which ive have the solution
The covariance matrix at lag s for the residual process ( Z 1 , - Z
is equal to
The covariance matrix generating function for the residual process can be determined by
multiplying (17.2.10) by BS and summing overs, i.e.,
=
s=-cn
Similarly, E[1&,,,
[rll( s )BS - r t 3 ( s )BSa'(B) -
- @(B)Z3, 12]
(Y
( B ) r 3 * ( s BS
)
+ ~ r ( B ) r ~ B~ S( ~s )' ( B ) ]
is minimized by choosing
The covariance matrix at lag s for the residual process ( Z 2 ,
- 2 2 , t) is equal to
446
CHAPTER f 7 More on Vector Time Series
from which the covariance matrix generating function can be determined as
Also,
and
Thus,
and
Note that from (17.2.11) to (17.2.14) we have the covariance matrix generating function for
the residual process (Z1.3,t , ZGG3,
t)f
If we let
be the covariance matrix for the residual process (z11.3,

t , Z5.3,
at fag s, then r f ~ . ~ (iss )

obtained directly as the coefficient of BS in G ~ ~ . ~ Specifically,
(B).
I'f2,3(0)is the coefficient
of BQ in (17.2.15) and represents the variance-covariance matrix for the residual process.
447
Let D be the diagonal matrix, with its ith diagonal element being the square root of the
ith diagonal element of rr2,3(0);then we can define the correlation matrix at lag s for the
residual process as
,,
Heyse and Wei (1984) call the residual process (Zi.3, Zb,3.,)' in (17.2.6) the partial
process of (Zi,,, Zh,,)', allowing for Z3, t , because it represents the remaining stochastic
process in Zl,, and.&, after the linear relationship of each with the third stochastic compo( S(17,2,15)
)
the partial process covarinent series Z3*has been removed, They call G ~ * . ~ in
ance matrix generating function and &(s) in (17.2.16) the partial process correlation
matrix function for the partial process of (Zi, Z5, ,)', allowing for 5,,.
,,
17.2.3 Partial Process Sample Correlation Matrix Functions
+
,,
The partial process conelation matrix function, P ~ * . ~ ( sfor

) , the (,nl
m2)-dimensional
(Zi, ,, Z5, ,)' component of Z,, after removing the linear dependence on Z3, is the correlation
matrix function for the patial process, E a sample of the partiat process j = [ ~ i , ~~ ,i . ~ ,
were observable, then clealy ~ f 2 . ~could
( s ) be estimated directly using the procedures discussed
in Section 16.5.1. Thus, one intuitively reasonable estimation procedure would be first to compute the residuds &om the regression equations
,,
and
il,
using multivariate least squares estimates for Gk and

where rr is the number of obsexved
values of 2,.
The partial process correlation matrix at lag s, P f 2 . 3 ( ~ ) , can be estimated by the sample
correlation matrix at lag s for the series
where t = Q
+ 1, Q + 2, . . ,n - P, with Q = max(Ql, Q2) and P = max(P1, P2).That is,
~]'
448
CWPTER 1 7 More on Vector Time Series
where
" (n - P - Q ) is the effective sample size, Zt is the vector of means for kf,and b is the
diagonal rn2trix in which the kth diagonal element js the square root of the corresponding
element of I ' ] 2 , 3 ( ~ ) .
Let f;]2,3(s)be the partial process sample covariance matrix at lag s for the unobservable partial process z:; for i = 1, . . . , (nrl nt2), and j = 1, . . . ,A(~trl n13, let yL(s) be
the (i, j)th element of yiz3(s) Also denote the (i, j)th element of ~ J z , ~ by
( s ?);(s), which
we recall was computed from the series of estimated residuals ~ f ( s in
) (17.2.17) and
(17.2.18). Hannan (1970, p. 453) shows that the jk(s) differ from the yh(s) by quantities
that are of the order (no - s)ll and that ~ [ ? J ( S ) - y h ( s ) ] converges in probability to
zero, which implies that -$$(s)
has the same asymptotic distribution as
yi(s). In
other words, for large tt and moderate choices for PI, Q1,
P2,and Q2, elements of the partial
process sample correlation matrix obtained by computing the sample correlation matrix for
the estimated residuals (17.2.17) and (17.2.18)can be interpreted in the usual way.
One problem with this procedure for computing f i f 2 , 3 ( ~ )is with the choices of P I , Q1,
P2,and Q2 for computing the residuals in (17.2.17)and (17.2,18).This problem is common
in building statistical models, and we don't seek to solve it here. It is apparent that the
choice of the P's and Q's involves a trade-off. If they are too small, then some effects of
the third series will remain in fif2.3(s).If they are too large, then the effective sample size
110 will be too small. In dealing with these choices, we must remind ourselves that our interest is not in estimating the linear regression coefficients ak,k = - P I , -(PI - I), . . ,Q1,
and
I = -P2, -(P2 - I), . , ,Q2.Instead, our interest is in removing the linear dependence of Z,, and Z2, on 23, for the purpose of estimating the remaining correlation structure in ( Z i , Z$,,). In practice, diagnostic aids that examine the statistical significance of
the parameters are helpful. Nonetheless, it is generally advisable to choose several values
for the limits to be sure that the resulting estimates of P f 2 . 3 ( ~ do
) not change dramatically.
Note also that because the effective sample size no uses max(P1, P2) and max(Q1, Qz),
one can choose both sets of limits at the maximum in (17.2.17) and (17.2.18) without
reducing ?la.
no
,
,,
17.2.4 An Empirical Example: The U.S. Hog Data

The classical U.S, hog data consist of five series of 82 annual observations for the years
1867 to 1948. For convenience, the data are listed in the appendix as Series W13. The five
series are
HI:Hog numbers recorded on January 1in the Census of Agriculture.

P,: Hog prices in dollars per head on January 1.
Q,: Corn prices in dollars per bushel on December I.
C,: Corn supply in bushels produced during the year.
\VI:Farm wage rate.
449
TMLE 17.4 Estimated cross-correlation for hog numbers H; and hog prices P,+, at lags.
Lag s
$12
(4
Indicator
Lag s
i 1 2 (s)
Indicator
Note: 2 x estimated standard error = .22.

+ indicates cross-corretations greater than .22.
indicates cross-correlations between -.22 and .22.
A description of the data appears in Quenouille (1957), Box and Tiao (1977), and Tiao
and Tsay (1983), who fit the data to a five-dimensional vector autoregressive moving average
model. The data were logged and linearly coded by Quenouifle, and this form of the data
was used for the present analysis.
Our attention focuses on the relationship between hog numbers Ht and hog prices P,.In
terns of our development thus far, we are considering the case where Zf, = Ht, Z2,, = P,,
and 23,t = (Qt, CI, WJt.
The estimated cross-correlations, iI2(s),for hog numbers Ht and hog prices P,+,are
shown in Table 17.4 for lags s = 0, f 1, . . . ,f 12. These values were taken from the (1, 2)
element of the sample correlation matrix 6 (s) for the five series. These cross-correlationsare
persistently high and provide very little infomation regarding the relationship between hog
numbers and prices, except possibly that the series are nearly nonstationq or are commonly
related to a third series.
Following our definition of the partial process and the development in Section 17.2.3,
we first want to compute the residuals
and
450
C W E R 17 More on Vector rime Series
TABLE 17.5 Estimated partial process cross-correlations

for hog numbers H, and hokprices P,,, at lag s.
Lag s
fif2.3
Tndicator
Lag s
$f2.3(~)
Indicator
Note: 2 X estimated standard error .= .23.

-k
indicates cross-correlations greater than .23.
- indicates cross-correlations less than - .23.

a
indicates cross-correlations between -.23 and .23.

A
using ordinary least squares estimates for the Gik and Pn. To observe the effect of
increasing the limits in the regressions (17.2.20) and (17.2.21), we used the symmetric
limits L = P I = Ql = P2 = Q2 = 0, 1, , , , , 8. The partial process sample cor_elajion
matrix function 6L2,3(s) was then computed from the estimated residual series ( ~ f~ , f )
for each of these choices and was used to estimate the cross-correlation between hog
numbers and prices after removing the linear dependence on corn price, corn supply,
and wage rates. These values are shown in Table 17.5 for the choice L = 3 along with
the
-, , symbols indicating whether correlation coefficients exceed two times their
estimated standard errors (1/75)'12.
Interpretation of ij12.3(~)in Table 17.5 indicates that hog numbers and prices were
related in the period under study beyond their common dependence on corn prices, corn
supply, and wage rates. Only four values of bf2.3(s) are larger than twice their estimated
standard errors, however, and of these we consider only j?#f2.3(~),
j312.3(1), and 612.3(-1)
because the meaning of jf2.3(-~1) = -.30 is not clear and may be due to random variation. Here, $12,3(0) = -.31 indicates that hog numbers Hi and hog prices P, at time t
were negatively correlated contemporaneousfy, fifi.S(l) = -.51 shows that hog numbers H, at time t and hog prices
at time t
1 were negatively correlated, and
j12,3(-1) = -23 indicates that hog prices P, at time t were positively correlated with hog
numbers HttI at time t 1.
Next, we examine the effect of increasing the h i t s of the re essions (17.2.20) and
(17.2.21) on the three partial process sample cross-conrelations 5 2.3(- I), b12.3(0), and
jj]2.3(f). In Table 17.6, the three estimates are shown for each of the choices L = 0, 1, . , 8,
The value L =. null represents the null case, i,e.,the sample cross-correlations jj12(- 1), b12(0),
and b12(1).
+,
..
'
451
TABLE 17.6 .Estimated partial process cross-correlations for

increasing values of the limits P = Q in (1 7.2.20) and (1 7.2.21).
Null
0
1
2
3
4
5
6
7
8
'
Note: Null indicates sample cross-correlations.

+ and - indicate partial process cross-correlations which exceed t w o times their
estimated standard errors.
The estimates i 1 2 . 3 ( - l ) r fi[2.3(0), and i j l * . 3 ( f ) are plotted against the limit L in

Figure 17.2(a), (b), and (c), respectively.
The results of this analysis indicate that the effect of increasing the limit L on the
l)
although ij[2,3(~)
does diminish to values
estimates $f2,3(-1) and $ f ~ . ~is(minimal,
closer to zero as the limit increases. Hence, our interpretation of the contemporaneous
cross-correlation between hog numbers H, and hog prices Pt from Table 17.5 may be
questionable. The additional evidence enhances our conclusion that hog numbers H,at
time t and hog prices P,+, at time t
1 were negatively correlated and that hog prices
P,at time t and hog numbers H,+I at time t 1 were positively correlated for the period
under study beyond their common dependence on corn prices, corn supply, and wage
rates. This result certainly corresponds to simple economic theories relating supply and
prices. This phenomenon becomes apparent for this data, however, only through the study
of the partial process and the partial process correlation function.

An ARMA model can often be represented by different forms. For example, with no loss of
generality, consider the zero mean first-order univariate autoregressive (AR(1)) model
where 4(B) = (1 - #B), B is the backshift operator such that BZ, = Z,-l, and a, is a white noise
series with mean zero and constant variance u2.If the zero of the polynomial +(B) lies outside
the unit circle, then the model can also be written as the infinite moving average (MA) f o m
452
FIGURE 17.2 fif2.~(j)

for increasing values of the limits in
the regressions (17.2.20) and (17.2.21).
where $(B) = [ # ( B ) ] - ~ = (1 #B
moving average (MA( 1))model
+ # 2 ~ 2+ - - ). Similarly, the first-order univafiate
17.3 Equivalent Representations of a Vector ARMA Mode1
453
with O(B) = (1 - 6B) and.its zero outside the unit circle, can alternatively be expressed as
the infinite autoregressive (AR) form
rr(B)Z,=a,,
(17.3.4)
). The infinite forms in (17.3.2) and

with r ( B ) = [o(B)]-I = ( 1 -I- 63 4- d 2 ~ 2
(17.3.4) indeed contain an infinite number of terms as long as the parameters 4 and 0 are
not zero. More generally, there are interesting dual representations. That is, for a univariate
time series, a finite-order stationary AR process can be represented by an infinite-order
MA form, and a finite-order invertible MA process can be represented by an infinite-order
AR form. Thus, a finite-order AR process corresponds to an infinite-order MA process,
and a finite-order MA process corresponds to an infinite-order AR process.
These alternative representations are useful. For example, the AR form is useful in
expressing the forecast as a weighted average of previous observations, and the MA form
makes the calculation of the forecast variance especially simple.
Now, suppose that Z, = [Z1,*,.
, Z,Jf is an nt X 1 vector. The generalization of
(17.3.1) is the vector AR(1) process
+
..
where @(B) = (I
- QB),I is an nt X m identity matrix, Q =. [+ij] is a nonzero m X nt coefficient matrix with its (i,13 element t$i j, and a, = [al, . . . ,a,
is an m X 1 vector white noise
process with mean vector 0 and positive definite variance-covariance matrix 2.Because a vector
time series is a generalization of a univariate time series, it is intuitively reasonable to expect
that the above dual representations also hold for a vector process. In other words, we expect
that a finite-order vector AR process wilt correspond to an infinite-order vector MA process
and that a finite-order vector MA process will correspond to an infinite-order vector AR process.
This conjecture is not necessarily true, however. The possibitity of multiple representations for a
vector ARMA model has been known in the literature (see, for example, Hannan and Deistler
[1988]). In Section 17.3.1, we present some simple results showing that under certain conditions
a vector process can be simultaneously represented by a finite-order vector AR model, a finiteorder vector MA model, or a finite-order vector autoregressivemoving average (ARMA)model.
We also establish the maximum order of these representations. In Section 17.3.2, we discuss
some of the implications of these special types of vector time series models.
,,
,I'
17.3.1 Finite-Order Representations of a Vector Time Series Process

For the vector AR model given in (17.3.5), it is known from Section 16.3.1 that when roots
of the determinantaf polynomial I@(B)~ = 0 lie outside the unit circle or, equivalently, when
all the eigenvalues of @ lie inside the unit circle, the model can be written as the MA form
454
where Go = I, Is the MA form in (17.3.6) always an infinite order? To answer the question,
let us consider the three-dimensional vector AR(1) model taken from Newbold (19821,
(I - (PB)Z, = a,,
(17,3.7)
with
Because Q2 # 0 and @ j = 0 for j > 2, the form in (17.36) represents a vector MA(2)
model. In fact, any vector AR(1) model with a nilpotent coefficient matrix can also be written as a finite-order vector MA model. To see why that happens, let I A ~ denote the determinant of a matrix A and adj(A) denote the adjoint matrix of A. We note that in the vector
AR(1) model given in (17.3.5),
Now, @(B) = (I - @B) is a matrix polynomial in B of order 1. The order of the resulting
matrix polynomial in B of its adjoint matrix clearly will be finite. Thus, the inverse of a nondegenerate AR(1) matrix polynomial (i.e., @(B) # I) will be of a finite order if the determinant
\@<B)\is independent of B. For the example given in (17.3.7), we note that the determinant
11 - @BI = 1. To understand this special feature of a vector ARMA model, we investigate in
this section conditions for these seemingly different but equivalent representations.
From the above observations, we have the following results (more detailed proofs can
be found in Shen and Wei [19951):
1. Given the m-dimensional vector AR(1) model
defined in (17,3.5), the follotving conditions are equivalent:

(a) The vector AR(1) process can be expressed as a finite-order vector MA(q) model.
(b) The determinantal polynomial 11 - rP~lisindependent of B. Espedally, with B = 0,
we have11 - @BI = 1.
(c) I@[ = 0 , Z;"=,l@(i)l = O , 21,iq,,[@(i,j)l
= O , ..., Zy='=r+ii=O, where
@(il, . . , ik) is the matrix of with its il, . . . , ik columns and rows removed, and
(Pu denotes the ith diagonal element of a.
(d) The eigenvalues of @ are all 0.
(e) The matrix @ is nilpotent, i.e,, there exists an integer k such that Gkf 0, and @ j = 0
forj > k. In fact,
2, = [ I - @B]-Ia, =
1
adj [I - Q B ] a,.
11 - OBI
17.3
Equivalent Representations of a Vector ARMA Model
455
Because 1 - @BIis a constant, (I - @B) is an m X m matrix with its elements being

polynomiafs in B of order 1, and the elements of the adjoint matrix adj fl - @33 are
polynomials in B of order less than or equal to (m - I), the nr-dimensional vector
AR(1) model can be expressed as a finite m-dimensional vector MA(q) model with
q 5 (ni - 1).
2. Similarly, the m-dimensional vector MA(1) model
where O = [ O i j ] is an nz X nz coefficient matrix with (i, j? element Oimj,can be expressed

as a finite-order vector AR(p) model, with p 5 (m - I), if any one of the following
holds:
(a) The determinantal polynomial 11 - OBI is independent of 3.
(b) 0
6
0 , j
<
j
i
j d i i = O , where
@fir, . . , ik) is the matrix of 8 with its il, . . . , iA columns and rows removed
and Bi,i denotes the ith diagonal element of 6.
(c) The eigenvalues of 6 are all 0.
(d) The matrix 63 is nilpotent.
3. Next, let us consider the na-dimensional vector ARMA(p, q) process
where@,@) = (I - e l B - QpP)and8,(B) = (I - e l B matrix polynomials in B of order p and q, respectively. Because
- epBq) are
the process can be represented as a finite-order vector MA(k) model with k r [(nl - 1) p
q] if the determinantal polynomial I a p ( ~ )is[ independent of B. On the other hand,
because
the process can be represented as a finite-order vector AR(n) model with ti 5 [(nt - 1) q
-t. p] if the determinantal polynomial I O,(B)I is independent of B.
4. More generally, if, in (17.3. ll), @,(B) = Hp,(B)flp-,,(B) such that IH,,(B)~ is independent of B, then the process can be written as the finite-order vector ARMA(p - pl, k)
model withpl r p and k . 5 [(m- 1) pl q]
456
On the other hand, if, in (17.3.11), 0,(B) = V,,(B)r,-,,(B) such that IV, , ( B)[
is independent of B, then the process can be written as the finite-order vector ARMA(n, q - ql)
model with ql r q and n 5 [(m- 1) ql f p]
EXAMPLE 17.6 Consider the two-dimensional vector AFWA(2,2) process
where
= (1 - 1.4B f .45B2), which is not independent

It can be easily shown that 11 - a l B of B. Hence, Model in (17.3.16) cannot be represented as a finite-order vector MA model. It can
be shown, however, that the
(I -
@I B
- @, B
~=
) ( I - m)(I -
ne),
with
Hence, by (17.3.14), (17.3.16) can also be written as the vectorAKMA(1; 3) model
where
Similarly, 11 - elB - e2lI21 = (1 - B + .18~'), which is not independent of B; hence, the

process cannot be represented by a finite-order purely AR vector model. Because, however,
(I - elB -
~ =~
( I)
V B ) (I - r B ) ,
17.3 Equivalent Representations of a Vector A R M Model
457
with
by (17.3.15), (17.3.16) can also be written as the vectorARPVIA(3, 1)model
(I - nlB -
n 2 ~-2lI3Il3)Z, = (I - TB)al,
li].
where
HI
[189
n2
[-.l.,
, ,I,
-.6
and
n3 =
[-.l0
o
-51'
The models in (17.3.16), (17.3.17), and (17.3,18) are equivalent. Tiao and Tsay (1989) also
call these equivalent models exchangeable models.
17.3.2 Some Implications

The results in Section 17.3.1 indicate that for a vector time series, a certain process can be
simultaneously represented as a finite-order vector AR model, a finite-order vector MA model,
or a finite-order vector ARMA model, a phenomenon that cannot exist in the univariate time
series. In this section, we examine some implications of these representations. Some implications are direct consequences of the representations.
Four implications are as follows:
1. A finite-order vector AR or ARMA model is a finite period memory model if it can also
be represented as a finite-order vector MA model. That is, the optimal forecast of the
process after a finite period of time is the mean vector of the process. This property is in
drastic contrast to the univariate time series process, where the mean becomes the optimal forecast for an AR or an ARMA model only as a limit when the forecast period
tends to infinity.
2. In terms of stationarity in the wide sense, an m-dimensional stationary vector process 2,
can be written in an MA form
k] are square summable in the

such that the nl X nr coefficient matrices q k=
sense that each of the nt X nt sequences is square summable, i.e. ~ g ~ <
$ C*3$ for,
458
. . ,111 and j = I, . . , m.Thus, no nonstationary vector process can be represented by both a finite-order AR vec'tor model and a finite-order vector MA model.
An ni-dimensional vector process 2, is said to be invertible if it can be written in an
AR form
i = 1,
such that the nt X nz coefficient matrices Ilk = Inii,kRf are absolutely summable, is.,
~ g ~ <t 03~for~i =,1,~ . l, nt and j = 1, . . . , In. Hence, no noninvertible vector
process can be simultaneously represented by both a finite-order vector AR model and
a finite-order vector MA model. In other words, if a vector time series process can be
simultaneously represented by a finite-order AR model and a finite-order MA model,
then the process must be both stationary and invertible.
3. A nonstationary process 2,is said to be integrated of order d, denoted as I(d), if its
(d - 1)th difference is nonstationary but its dth difference is stationary, As discussed in
Section 17,1, an nr X 1 vector process Z, = (Z1,,, . , 2, 3' is said to be cointegrated if
each of its component series Zi, ,is 1(1),but some linear combination of Z,, i.e., C'Z, for
some k X m matrix C' with k < m,is a stationary k X 1 vector process. The discussion
in item 2 implies that a cointegrated vector process Z, cannot be simultaneously represented by a finite-order vector AR, a finite-order vector MA, or a finite-order vector
ARMA process. If multiple representations occur, then they must be in terms of its
linear combination C'Z, for some matrix C'.
4. If a vector time series process can simultaneously be represented by a finite-order
AR model and a finite-order MA model, then the process can be reduced to an inputoutput type of transfer function model discussed in Chapter 14. We illustrate the
phenomenon through a first-order vector process. The results for other cases are
simif ar.
First, let us consider an m-dimensional vector AR(1) process
..
..
(I - @B)Z, = a,.
(17.3.21)
If the process can also be written as a finite-order vector MA model, then by earlier
results the eigenvalues of cP are all zero. Hence, there exists a nonsingular matrix H
such that H@H-' = J, where 3 is a Jordan matrix whose elements are zero except for
those on the first superdiagonal, which are equal to 0 or 1, i.e.,
17.3 Equivalent Representations of a Vector ARMA Mode1
459
where S = 0 or 1. Multiplying H on both sides of (17.3.21) and letting Y , = HZ, and

e, = Ha,, gives
H(I - @B)H-'RZ, = Ha,
where e, is the m-dimensional white noise process with mean vector 0 and variancecovariance matrix W H ' , (I - JB) is a square matrix whose elements are zero except
for those on the principal diagonal, which are all equal to 1, and those on the first superdiagonal, which are all equal to -6B. Thus, (17.3.23) becomes
Next, consider an nt-dimensional vector MA(1) process
If the process can also be written as a finite-order vector AR model, then there exists an
integerp so that 8p # 0 but @"
0 for n > p, and
Because 8 is nilpotent, there exists a nonsingular matrix R such that ROR-' = J,

where 3 is a Jordan matrix defined in (17.3.22). Multiplying R on both sides of
(17.3.26) and letting W, = RZ, and r, = Ra, gives
R(1 + 8 B
-F
8 2 ~ 2.
+ @PBP)R-~RZ,= Ra,
or
(I
+ RBR-'B + - . + R@PR-'BP)W, = r,.
460
Because RakR-I = Jk,
where r, is the m-dimensional white noise process with mean vector 0 and variancecovariance matrix RXR'. More explicitly, (17.3.27) becomes
These examples show that if a vector. time series process can simultaneously be represented by a finite-order vector AR model and a finite-order vector MA model, then it
can be written in a vector AR form with a triangular coefficient matrix. Thus, the
process can be reduced to an input-output type of transfer function model as discussed
in Section 16.3.1.
17.1 Consider the process, 2, = T(B)a, = (1 - B)a, = a, - a, - 1, where the a, are i.i.d,
random variables with mean 0 and variance 1. Note that *(I) = 0; hence, it is not
I(0). Find the process of its sum, and show that the resulting process is asymptotically stationary and thus is not an Z(1) integrated process.
17.2 Consider the two-dimensional vector AR(1) process
Verify that each component series of Z, is I(1).
17.3 Consider the bivariate process given in Exercise 17.2,

(a) Why is 2, cointegrated? Find its cointegrating vector.
(b) Fiid its MA representation.
(c) Find its AR representation.
(d) Find its error-correctionrepresentation.
Exercises
461
17.4 Find a vector time series with its component series being nonstationary,
(a) BuiId a vector ARMA model for the series, and examine the interrelationship
among the component series.
(b) Perform the unit root test to verify whether each component series is 41).
(c) Test whether the components ate cointegrated.
(d) If the components are cointegrated, find an error-correction model for the series
that describes a long-term relationship among the components.
,,2 2 ,
17.5 Let Z, =. (Z1,
5,t ) l
and
,,
where a, = (ar, a2, ,, a3,J' is a vector of white noise process with mean zero and
variance-covariance matrix I; = I.
(a) Find the covariance matrix generating function GZ (B) of Z,.
(b) Find the covariance matrix function r z ( k ) and correlation matrix function pZ(k)
of Z,.
(c) Find the partial process covariance matrix generating function G ] * , ~ ( Bfor
)
(Zl,,, Z2, ,Ir allowing for 23,r.
(d) Find the partid rocess covariance matrix function 11[2.3(k)and correlation
matrix function p 2.3(k).
(e) Comment on your findings.
17.6 Let 2, = (ZIP,,&,,, Z3, ,Ir and
where a, = (al, t, a*,r, a3.J' is a vector of white noise process with mean zero and
variance-covariance matrix
(a) Find the covariance matrix generating function GZ(B)of Z,.

(b) Find correlation matrix function pI2(k) for (Zl, Z2,
,,
462
(c) Find the partial process covariance matrix generating function G ~ z . ~ ( for
B)
(21,,,Z2, A ' allowing for 23,
(d) Find the partial process correlation matrix function p12.3(k) for fZ1, 22,,)'
allowing for &,t .
(e) Comment on your findings.
,,
17.7 Find a vector time series and perform a partial process analysis as discussed in
Section 17.2.4.
17.8 Let 2, be the m-dimensional vector AR(1) model
(I - cPB)Z, = a,,
where a, is the rn-dimensional white noise vector process with mean 0 and variancecovariance matrix Z. Show that the following conditions are equivalent:
(a) The vector AR(1) process can be expressed as a finite-order vector MA(q)
model.
(b) The determinantal polynomial 11 - @BI is independent of B. Especially, when
~ = 0 , w e h a v e [ 1 - @ B / = 1.
(c) The eigenvalues of Q1 are all 0,
(d) The matrix is nilpotent, i.e., there exists an integer k such that @ # 0, and
@ j = 0 for j > k.
17.9 fa) Find an example of a vector process that has both finite AR and finite MA representations.
(b) Express the model in part (a) in a transfer function model.
State Space Models and

the Kalman Filter
..
Having presented various univariate and multivariate time series models in previous chapters, we now give a brief introduction to the state space model and Kalman filtering, This
very genera1 approach in time series modeling can be used for both univariate and multivariate time series.
18.1 State Space Representation
The state space representation of a system is a fundamental concept in modern control theory.
The state of a system is defined to be a minimum set of information fiom the present and past
such that the future behavior of the system can be completely described by the knowledge
of the present state and the future input. Thus, the state space representation is based on
the Markov property, which implies that given the present state, the future of a system is independent of its past. Consequently, the state space representation of a system is also calfed
the Markovian represenhtion of the system. Let Y1,
and Yube the outputs of a system to the
inputs XI, and Xa, respectively. A system is said to be linear if and only if a tinear combination
of the inputs, ax1, bX2,, produces the same h e a r combination of the outputs, aY1, 4- by2,,
for any constants a and b. A system is said to be time-invariant if the characteristics of a system
do not change with time so that if the input X, produces the output Y,,then the input X,-,,wit1
produce the output Y,-,,. A system is said to be linear time-invariant if it is both linear and
time-invariant. Many physical processes can be modeled by h e a r time-invariant systems that
contain stationary processes discussed in previous chapters as special cases.
For a linear time-invariant system, its state space representation is described by the
state equation
and the output equation
464
CHAPTER X 8 State Space Models and the Kalman Fitter

where Y, is a state vector of dimension k, A is a k X k transition matrix, G is a k X ~z input
matrix, X,is an n X 1 vector of the input to the system, 2, is an nt X 1 vector of the output,
and H is an nt X k output or observation matrix. If both the input X, and the output 2,are
stochastic processes, then the state space representation is given by
AY, + Ga,+1
Z, = HY, + b,,
Y,+1 =
where a,+l = XI+*

-E(XI+IXi, i 5 i ) is the a X 1 vector of one-step ahead forecast error
of the input process X,and b, is an rn X 1 vector of disturbances assumed to be independent
of a,. The vector a,+l is also known as the innovation of the input X, at time (t 1- 1). When
2, = X,,b, vanishes from (18.1,4), the state space representation of a stationary stochastic
process 2, becomes
Thus, the process 2, is the output of a time-invariant linear stochastic system driven by a
white noise input a,. The Y , is known as the state of the process.
The state equation is also called the system equation or the transition equation, and
the output equation is also referred to as the measurement equation or the observation
equation. The state space representation of a system is related to the Kalman filter and was
originally developed by control engineers (see Kalman f19601, Kalman and Bucy {1961],
and Kalman, Falb, and Arbib l19691). The system is also known as a state space model.
Akaike (1974a) seems to have been the first to apply the concept directly to the analysis of
ARMA models.
18.2 The Relationship between State Space and ARMA Models

To see the relationship between state space and ARMA models for either the univariate or
multivariate case, consider the zero mean stationary nl-dimensional vector ARMA(p, q)
model
where @(B) = (I - Q I B - - . - @,BP), 8(B) =(I

- BIB nt-dimensional multivariate zero mean white noise process.
. . . - QpB4),and a, is an
18.2 The Relationship Between State Space and ARMA Models
Rewriting (18.2.la) in a moving average form
where
Let
Then
Now,
Thus,
= I, we have
465
466
CHAPTER 18 State Space Models and the KaIman Filter
We assume, without loss of generality, that p > q by adding

sary. Then, from (I8.2. lb), we have
@i
= 0 in (18.2.lb) if neces-
In fact, it is clear that Z1+p,il,

for i e 0 is a function of Z,, Zl+llr,. , and Z,+p-llt, Hence,
the state vector is {Z,, Z,+Il,, . . , Z,+,-ll,), and the state space representation of the vector
ARMA(p, q) model is given by
and
Note that in the above representation, the first I? components of the state vector are equal
to Z,. Equation (18.2.6) is usually suppressed in the representation of an ARMA model.
Now suppose that a process 2, has a state space representation
where it is assumed that Y, is a nap X 1 vector of the state and a, is the innovation of Z,. If
the characteristic polynomial of A is given by IAI - A( = Xy=O#iA~-i, where tho = 1,
then by the Cayley-Hamilton theorem (see, for example, Noble [I9691 and Searle 119821)
18.2 The Relationship Between State Space and A R M Models
467
By successive substitution in the first equation of (18.2.7), we have, for i > 0,
Now,
Hence, Zt has an AlZMA representation
where H(Ap
+l~i-l
+ #qAp-l + . + # J ~ - . ~+A +pX)Yt = 0 by (18.2.8) and Bi = H(Ai +
+ .., + + i ~ ) ~ .
In the representation (18.2.101, we use the Cayley-Hamilton theorem and obtain the AR
polynomial as Qp(B) = $p(B)I where $(B) = 1 - #lB - . - +,LIP is a univariate
polynomial of order p. Recall from Section 16.3, this is one of possible representations of an
identifiable ARMA model.
Note that in the state space representation (18.2.5) of an ARMA model, because the first
m components of the state vector are equal to Z,, we can also recast the state representation
into an ARMA modef by directly solving the state space system of equations for the first m
components. tVe illustrate this method in the following examples.
468

EXAMPLE 18.1 Consider the univariate ARMA(2, 1) model,
By rewriting (18.2.11) in the moving average form
where !Po = 1, 'PI = c#q - 81, . . . ,we have, from (18.2.5) and (18.2,6), the following state
space representation of (18.2.11):
To recast the state space representation (18.2.12) into an ARMA model for Z,, we note that
(18.2.12) implies that
By (18.2.13), we have
Because Zt+l!t+l= Z,+, and Zl+21t+2= Z,+2, by substituting (f8,2.13) into (18.2.14) and
(18.2.14) into (18,2.15), we obtain
which is theARMA(2, 1) model given in (18.2.11).

For a multivariate vector process, it is possible for some components of Z,+d, to be linearly
combinations of .the remaining components. In such cases, the state vector consists of a subset of
the possible components of {Z,+,!, 1 i = 0, I, . . . , ( p - 1)).Thus, depending on whether the
redundant components, which are linearly dependent on others, have been eliminated, seemingly
different but equivalent structure may arise. We illuseate this point in the following example,
18.2 The Relationship Between State Space and ARMA Models
469
EXAMPLE 18.2 Consider the following two-dimensional vector ARMA(1, 1) model:
By (18.2.5), the state space representation of (18.2.16) is given by
where
1=
I'[
$21
'"]
and 0 2 =
[o0
0] is a zero matrix added to change the ARMA
422
(1, 1) model to an ARMA(2, I) model so that p > q, By the above remarks, the state vector
is equal to fZ[, Z~+llt]'
if all components of this vector are linearly independent and equal to
a subset of [Z:, Z:+llt]r if some components of the vector are linearly dependent. To
examine the linear dependency of the components of the possible state vector [Z:, Z~+llt]',
we note that [ZI, Z:+lltI1 = [Z1,!, 22,f , Zt, t+llfr22,r+~ltl',NOIV,
Hence, the state vector is [Z1,,, &,1, Zl, t+llt]',Because the component &,,+dt is a linear
as shown in (18.2.21), it is not included in the state vector.
combination of Zl,, and
The representation (18.2.17) can now be simplified by representing Z1,
22, t+l, and
and
Z1, fcdt+l in terms of the state vector [Zl, ,, 22, !, Zl, t+llt]'and the input noises at,
a2,f + l . Clearly, from (18.2.18), (18.2.19), and (18.2.20), we have
470
CHAPTER 18 State Space Models and the Kalrnan Filter
Combining (18.2.22), (18.2.23), and (18,2.24), we obtain the reduced state space representation
We may recast the state space form (18.2.25) into ARMA form by expressing the system of equations in (18.2.25)in tems of Zl,, and &, Using (18.2.22), we can write the conditional expectations in (18.2.24) as
,.
Hence,
Combining (18.2.27) and (18.2.23), we get the vector ARMA(1, 1) model given in (18.2.16).
Thus, theoretically there is no distinction between the state space and the ARMA representations of a stationary process. The results obtained fiom one representation can be used
for the other representation. Unless all components of a state vector are linexly independent,
however, when converting a state space form into an ARMA form, there will be nontrivial
common factors in the autoregressive and moving average polynomials. Hence, the resulting
ARMA model may have superficially inflated the orders o f p and q.
18.3 State Space Model Fitting and Canonical

Correlation Analysis
The state space.representation given in Sections 18.1 and 18.2 is clearly not unique. For
example, given (18.1.5), we can form a new state vector V, = MY, for any nonsingular
matrix M and obtain a new state space representation
471
K+I = A
I +~Glat+l
(18.3.1)
and
where Al = MAMA1,GI = MG, and HI = EM-'. Through a canonical representation as

shown in Akaike (1976), however, we can obtain a unique solution. In the canonical correlation representation, the state vector is uniquely determined through the canonical correlation analysis between the set of current and past observations (Z,, ZnV1;. . . ) and the set
of current and future values (Z,, Zn+lln,. . . ). In terms of an AR(p) model, because the
eventual forecast function is determined by the AR polynomial and (Z,, Zn-l, , , Z,-p)
contains essentially all the information relevant for the future values of the process, the
canonical correlation analysis is simply performed between the data space
..
and the predictor space
Consider tire following block Hankel matrix of the covariance between Dn= (Zh, Zk-l,.
Zb-p)' and F,,
= (Zk, Zh+lln,. . , ZA+pln)'defined by
. .,
where we use a property of conditional expectations and hence that COV(Z,-~,Z,+iln) =

COV(Z,-~,Zn+j). Akaike f1974a, 1976) shows that under the nonsingularity assumption of
r(O), for general vector ARMA models, the rank of I' is equal to the dimension of the
state vector and thus is also equal to the number of nonzero canonical correlations between
D,, and F,.
When the model is unknown, the choice of the order p is obtained from the optimal AR
fitting of the data, which is often .based on AIC discussed in Chapter 7. For the vector
process, AIC is defined as
472
CHAPTER 18 State Space Models and the K a h a n Filter
where
11
IX,~
= the number of observations,
= determinant of the covariance matrix for the
innovations or the white noise series in the AR(p) fitting, and
m = the dimensions of the vector process 2,.

The optimal AR order p is chosen so that AIC is minimum. Thus, the canonical correlation
analysis will be based on the block Hankel matrix of sample covariances, i.e.,
where r(j),
j
0,1 , . .., 2 p , are the sample covariance matrices as defined in (16.5.16).
As discussed in Example 18.2, because some components of a prediction vector Zn++,
may be Linearly combinations of remaining components, the canonical correlation analysis is
performed between dl the components of the data space
and the components of the predictor space
Because the state vector is known to be a subset of the predictor space, a sequence of
potential state vectors, Yi,is determined through a canonical correlation analysis ketween
a sequence, F;,of subset? of Fn,
and the data space D,, based on the submatrix FJ forming from the columns of r, which correspond to the components of 'D, and .Fi.
More specifically, because the canonical correlation between 2, = [ZIP., Z2, n , . . ,
Zm,,If and 'D, are 1, . . . , 1, which are clearly nonzero, the current state vector is set to
Zn and the first subset in the sequent:,
is set to [Z1,, &,,,, . . . , 2%n, Z1, n+llnl'. If
the smallest canonical correlation of F' is judged to be zero, then a linear combination of
F,!,
is uncorrelated with the data space 'D,. Thus, the component Z1,ntlln and any Zl, n+qn
are excluded from further consideration for components in the state vector. If the smallest
canonical correlation is nonzero, then Zl, n+lin is added to the current state vector. The
sequence F;is now generalized by adding to the current state vector the next component
of Fnthat does not correspond to the component that has previously failed to be included
FA,
473
in the state vector. The smallest canonical correlation of fj is computed and tested for its
significance. If it is significaly different from zero, then the component is added to the
state vector. Otherwise, the component is excluded from the current state vector and also
from any further consideration. The state vector selection is complete when no elements
of Fnare left to be added to the current state vector.
For each step in the sequence of canonical correlation analysis, the significance of the
smallest canonical correlation, denoted by F ~ , ,is based on the following information criterion from Akaike (1976):
where q is the dimension of F/,

at the current step, If C 5 0, p i , is taken to be zero; othenvise,
it is taken to be nonzero. To test the significance of the canonical correlation &
;,
one can also
foilow Supplement 18A and use the approximate ,y2 test given by Bartlett (1941), who shows
that under the null hypothesis of zero canonical correlation, the statistic
has an approximate chi-square distribution with [m(p 1) - q

11 degrees of freedom,
i.e., x2(m(p 1) - q 1).
Once the state vector is identified, we have the canonical representation of a state space
model
where a, is a Gaussian vector white noise series with mean 0 and variance-covariance matrix
2, i.e., N(0, X), and H = [I,, Oj where I, is the m X ni identity matrix. Clearly, from the
results of Section 18.2, estimates for the matrices A, G , and I:can be obtained from the parameter estimates of the optimal AR model fitting. More natural estimates of the elements
for the transition matrix A, however, are obtained from the canonical correlation analysis.
For example, let k be the number of components of the final state vector Y,, hence, A is
a k X k transition matrix, From (18.3.9), it is known that rn 5 k 5 m(p + I). We now
illustrate how the estimates in the lirst row of A are derived, which are related to the first
step of the sequence of canonical correlation analyses when Zl, ,,++ is added to the vector
Z, to form the first subset Fj;in deciding whether Zl, n+lln should be included in the state
vector. When the smaflest canonical correlation between F,!and
, Dnis judged to be nonzero,
Zl, n+lln
becomes the (?n 4- 1)th component of the state vector. Thus, the first row of A will
have 1 in the (ni + 11th column and 0 elsewhere. When the smallest canonical correlation is
judged to be zero, a linear combination of 3: is uncorrelated with the-data space V,,,
and
Zl, n+lln is excluded from the state vector, Because the determinant of r(0)is nonzero, we
can take the coefficient of Z1,n+lln in this linear combination to be unity. Thus, we have the
474
relationship Z1,n+rl,, = or'Z,; the coefficients of the vector a are used as estimates of the first
m columns of the first row of the transition matrix A, and the remaining (k - jn) columns of
the first row are zero. The estimates in the other rows of A are obtained similarly.
Alternatively, once a state space model (18.3.12) is identified, one can use the maximum
likelihood procedure b obtain more efficient estimates of A, G , and 2. For a given sequence
of n observations Z,, Z2, . . . ,and Z,, because
we have
and
a, = [H(I - AB)-lG]-'Z,.
Thus, the log-likelihood function, subject to additive constants, becomes
In L(A, G , X
. . . , Z,)
cx
1z
- -In
1
121 - -trZ-'S(A,G),
2
(18.3.16)
where
The standard maximum likelihood estimation, as discussed in Section 16.6, can now be used
to derive estimates of A, G , and 2.As pointed out in Section 16.6, the maximum likelihood
estimation involves a highly nonlinear iterative procedure. The estimates obtained from the
canonical correlations analysis may serve as initial estimates in this more efficient estimation
procedure. In the process of estimation, one can set certain elements of A and G to some
constants such as zero or one.
1 8.4 Empirical Examples

We now give two examples of the state space modeling. The fust one is a univariate time
series, and the second is a multivariate vector process. SAS~ETSof the SAS Institute
(1999) is one of a few computer software that can be used to perform state space
modeling. Its STATESPACE procedure is used in the analysis of both examples. We
note, however, in Equation (18.3.11), the STATESPACE procedure uses a multiplicative
475
TABLE 18.1 AIC of AR models for the blowfly data.
AIC
333.93
335.56
337.37
338.93
340.45
constant, -(n - 1 / 2 [ m ( p 1) - q
I]), instead of -(n - 1/2fm(p f 1) -t- q
I]),
although in a large sample, the difference in using these two different multiplicative
constants will not affect the conclusion in most cases.
EXAMPLE 18.3 The univariate time series used in this illustration is the square root transformed blowfly data of 82 observations (Series W3),which we analyzed in Chapters 6 and 7.
The first step in the state space modeling is to find an optimal AR fitting of the data.
From Table 18.1, the AIC is minimum at p = 1. Hence, the optimal AR order p is chosen to
be 1, which corresponds to the same model we identified for the series in Chapter 6 using
the sample ACF and PACF.
Withp = 1, we apply the canonical correlation analysis between the data space {Z,, Z,-l}
and the predictor space {Z,, Z,,Ib}. The minimum canonical conelation coefficient equals
.0891, and the information criterion equals
which is negative. Hence, Z,+lb is not included in the state vector, This result is also supported by Bartlett's chi-square test
which is not significant as x2 = .63 < X $ 5 ( l ) = 3.04. The above canonical correlation
data analysis is summarized in Table 18.2.
TABLE 18.2 Canonical correlation analysis for blowfly data.

State vector
Correlations
Infor. (C)
Chi-square
DF
476
Thus, the state vector consists only of 2, and the canonical representation of the state
space model for the data is given by
which is the AR(1) model we fitted in Chapter 7. The estimate of the transition matrix A is
.73, which is a scalar and is equal to ;I. Because the AR(1) is a Markov process, this result
is certainly expected.
EXAMPLE 18.4 As a second illustration, consider the bivariate series of a leading indicator X, and sales Y, taken from Series M in Box, Jenkins, and Reinsel (1994). They fitted the
data with the transfer function model
where x, = (1 - B)X,, y, = (1 - B)Y,, Gz = .0676, and 6: = ,0484. We now apply the state
space procedure to construct a state space model for the series 2, = [x,, y,]' for f = 1,2, . . . ,149.
Again, in constructing a state space model for the data, we f i s t search for the optimal
order in the AR fitting of the data. From Table 18.3 because AIC is minimum at p = 5, the
optimal AR order is chosen to be 5.
Based on p = 5, we next apply the canonical correlation analysis between the data
space {Z,, 2,- . . . , Z,-$) and the predictor space {Z,, Zn+lb, . . . , Z,+51n). As mentioned earlier, however, that Zn,ll, through Zn+51, are potential elements of the state vector
does not mean that all their components ~ ~ +yn+iln,
~ lfor~ i ,= 1, . . . , 5 are needed in the final
state vector, Thus, the canonical correlation analysis is performed between
and
TABLE 18.3 AIC of AR models for the differences of series M

in Box, Jenkins, and Reinsel
AIC
-230.24
-271.62
-308.88
-608.97
-665.00
18.4
Empirical Examples
477
TABLE 18.4 Canonical correlation analysis for the differences of Series M

in Box, Jenkins, and Reinsel.
State vector
Correlation
Infor.(C)
Chi-square
DF
as shown in Table 18.4. Because the C value equals 12.63, which is negative when the
component xn+lln is considered, xn+in for i 2 1 are excluded from the state vector. The C
values are positive when the components y,,+lln and Yn+nln are considered but negative when
y , , + . ~is
~ considered. Hence, the final state vector, from Table 18.4,is
The resulting state space model is
where [ar, a2,,+,]' is an N(0, X) vector white noise process.

The estimates of A, GIand X are given by
and
478
CHAPTER 18 State Space Modeb and the Kalman Filter

where the value in parentheses below an estimate is the standard error of the estimate.
Because the t-value of the (4, 1) element of the input matrix is much less than 1 (t =
.0011,097 = .M),
we set it to zero and reestimate the restricted model. The final fitted state
space model becomes
and
Diagnostic tests indicate that the residuals are not different from white noise series, and
we conclude that the model is adequate. It would be interesting to compare the state space
model (18.4.7) and the transfer function model (18.4.4) in terms of variable relationships
and forecasting, We leave it as an exercise,

Consider a general state space model of a stochastic system given by (18.1.3) and
(18.1.4), i.e,,
= AX + Ga,+1
Z, = HY, b,,
Y,+1
where A, G, and H are fixed matrices of order k X k, k X n, and rn X k, respectively. The

vector a, is the system noise, which is an i.i.d. Gaussian white noise N ( 0 , Z ) series, and btis
the measurement noise, which is an i.i.d. Gaussian white noise N(0, a)series. Furthermore,
it is assumed that the two noise series a, and bt are independent of each other. More precisely, a, and b, are assumed to have a joint multivariate normal distribution with mean 0 and
a block diagonal covariance matrix,denoted by
18:5 The Kalman Filter and fts Applications
479
Once a state space model is constructed, the I-step ahead forecasts from the forecast origin
time t can be calculated as follows:
Hence,
where
Clearly, from (18.5.4), the accuracy of the forecasts k,(l)depends on the quality of the estimate
Y, of the state vector Y,,which summarizes the information fiom the past that is needed to forecast the future. To improve forecasts, when a new observation becomes available, it should be
used to update the state vector and hence to update the forecasts. In this section, we introduce the
Kalman filter, which is a recursive procedure used to make inferences about the state vector Y,.
To specify the distribution of the state vectors Y, for t = 1, 2, . . . , inn(18.5.1), we
assume that the distributionp(Yo) of the state vector Yo at time zero has mean Yo and covariance Vo, i s . ,
This equation, together with the system equation in (18.5.1), determines the distribution of
the state vectors p(YJ, for t = 1 , 2 , , . . . These distributions present our belief prior to having observed the data Z;* and are referred to as the prior distributions. After observing the
data, we revise our prior distribution and obtain the posterior distribution of the state vector,
which is the conditional distributionp(Y, ZF), where
ZT = ( ~ ~ 51 1
t).
(is.5.6)
480
By Bayes's theorem,
Hence, a posterior distribution p(Y,+, I 2:) at an earlier' time becomes a prior distribution for
calculating a new posterior di~tributionp(Y,+~
I ZT+l)when a new observation Zr+lbecomes
available,
Now, suppose that the posterior distribution, p(Y, 1 Zf), of the state vector Y tat time t is
a normal distribution with a mean Y, and a covariance matrix V,, i.e.,
At time (r + I), when a new observation Ztcl becomes available, we would like to update
the state vector and derive the new posterior distribution P(Y,+~I ZFtl). Because
knowing Z,+l is equivalent to knowing the one-step ahead forecast error e , + ~Thus,
.
to
derive the posterior distribution P(Y,+~I ZT+l) = P ( Y , + ~I Z,+l,ZT), we need only to find
) el+l,.ZT)of (Y,+lett1, ZT). The conditional distribution
the posterior distributi~np(Y~+~
of (Y,+l I e,+l, ZT), however, follows immediately if we can find the joint normal distribution of IfY,+l, e,+l)' I GI.
Note that at time (t I), having observed Zf but prior to observing
we have, via
the system equation in (18.5. l), the prior distribution
where
R,+l = AV,Af
+ GXG'.
(18.5.11)
After observing Zt+l, from (18.5.91, we have the posterior or conditional distribution
(er+l
I T + I23
~ - N(H(Y,+I - A$,
0).
(18.5.12)
481
Recall that for any two random vectors X1 and X2,
In terms of a joint multivariate normal distribution, we have
if and only if
Now, let X2correspond to Y,+l

and X1 correspond to e,+l.Then, from (18.5.10) and (18.5.14),
and by (18.5.12), (18.5.15), and (18.5.16), we have
Hence,
Also, by the same equations,
we get
482
Thus, by (18.5.13),
It follows, from (18.5.15), that
(X+l le,+1, Zt)
~ ( i ; + l ,VI+1)7
where
with
Equations (18.5.21) and (18.5.22) are the basic recursive formula used to update the
mean and the covariance matrix and hence the distribution of the sta? vector, Y,+l, after the
new observation, Z,+l, has become available. The updated estimates Y,+I of the state is, from
(18.5.21), the sum of the projected es~imateusing observations up to time t, and the one-step
is called the Kalman gain, which
ahead forecast error e , + ~= Z,+l - Zt(l). The matrix
determines the weight given to the forecast error. In fact, it can be easily seen from (18.5.21)
that the Kalman gain is simply the coefficient of the least squares regression of Yt+l on the
forecast error e,+l conditional on Z,. Thus, Kalman filtering is a recursive updating procedure that consists of forming a preliminary estimate of the state and then revising the estimate by adding a correction to this preliminary estimate. The magnitude of the correction is
determined by how t e l l the preliminary estimate predicted the new observation. The Kalman
filter will tend toward a steady state when the Kalman gain approaches a limit.
Clearly, for the given state space model (18.5.1), the rehcursive equations to compute
in (18.5.21) and V,,l in (18.5.22) require initial values Yo and Vo specified in (18.5.5).
These initial values can be the values from the previous studies. Otherwise, some guess values from experience can be used. In that case, following a principle of uninformative prior,
the Vo is often chosen as a diagonal matrix with large elements. Thus, for a large n, the
results will be dominated by the information from the data.
483
The methods of Kalman filtering were originalfy developed by Kalman (1960) within
the context of Iinear systems. Because of the ease of implementation of the algorithm on digital computers, however, these methods have now become well known and widely used in
many areas of applications. For example, consider the simple state space model
Z, = BY, b,
Y, = A x u 1 + a,,
where
The commonly used multiple regression model, Z, = XP b,, can be easily seen as the
state space model with H = X (a I X nt vector of independent variables), A = I, Y,= P, (an
m X 1 vector of nt coefficents), and X = 0. The time-varying regression model
is the above state space model with H = X,Y, = P,, and Z J- 0, The advantage of this formulation is that one can estimate the time-varying coefficients, P,, using Kalman filtering.
Other applications include time series forecasting (Mehra [1979]), Bayesian forecasting (Harrison and Stevens [1976]), the analysis of time series models with timevarying coefficients (Young [1974]; Bohlin [1976]), regression analysis (O'Hagan
[1978]), missing observations (Kohn and Ansley 1198311, quality control (Phadke
[1981]), temporal aggregation (Schmidt and Gamerman 11997]), and data disaggregation
(Gudmundsson [1999]).
The recursive updating equations can be derived using different approaches. For example, Priestley and Subba Rao (1975) derive the K a h a n filter algorithm through the statistical
technique of factor analysis. An expository article is given by Meinhold and Singpunvalla
(1983). For further references, see K a h a n (1960), Kalman and Bucy (1961), Jazwinski
(1970), and others,

Let X be a q X 1 vector and Y be an r X 1 vector with the variance-covariance matrix Z given by
Wlth no loss of generality, assume that q 5 r. In many applications, we are interested in

finding the correlation between the Iinear combination of the variables in X and the linear
484
combination of variables in Y, i.e., the correlation between atXand P'Y for some nonzero
q X I vector IX and r X 1vector p.
Let U1 = aiX and Vl = P$Y.The first canonical correlation, p(l), is defined to be the
maximum correlation between U1and Vl over all possible arl and 01, i.e.,
such that Var (ruiX) = 1 and Var (PiY) = 1. Thus, we need to maximize
such that
IX~Zxal
= PiXYS1= 1. Using Lagrange multipliers, we
maximize the function:
Taking partial derivatives with respect to al,PI,+I, and 42and setting them to 0, we get
Multiplying the first equation in (18.A.5) by ~ i the

, second equation by pi, and using the
constraints in the last two equations, the system to solve for a1 and pl in (18.A.5) becomes
and p = 241 = 242 = Corr(Ul, Vl)= ~ l l j 2 ~For

~ fa1
i ~and
.
implication is that
PI not identically zero, the
485
Note that if lZyl # 0, from the results of partitioned matrices, we have
and therefore the equivalent implication
Clearly, there are q solutions for p2 in (18.A.8); we choose p(1) to be the positive square
root of the maximum solution. The corresponding vectors oil and P1 are obtained
through (18.A.6). The resulting U1 = atjX and Vl = PiY are called the first canonical
variates.
Let Ui = a : X , and Vi = P:Y.The ith canonical correlation, pti), is defined to be the
maximum correlation between Ui and Vi so that Var(a1X) = 1 and Var(PiY) = 1, and U i
and V iare uncorrelated with all previous (i - 1) canonical variates. If we then let the q solutions of (18.A.8) be ordered so that pf 2 p$ r . . 2 pi, it can be shown that p(i) is the
positive square root of the ith largest solution, p;.
Let
be the sample variance-covariance matrix computed from a sample of size n, i.e.,
where xj and yj are q X 1 and r X 1 vector observations of X and Y.

The sample canonical correlations are ;(i), i = I , . . . , q, where fif r fij 2
are the solutions of
...
Assume that X and Y follow a joint multivariate normal distribution N ( p , 2).To test that
X and Y are uncorrelated, i.e., XXy # 0, and hence they are independent, it is equivalent to
486
CHAPTER 18 State Space Models and the Kalrnan Fiiter
test the null hypothesis HI : (p(l), . . . , p(q)) are jointly zero against the alternative that they
are not jointly zero. It can be shown that the likelihood ratio statistic is
# 0, and
From (18.A.9), when tix(
(gY[# 0,we have
hence,
It is clear from (18.A.10) that 62 is a characteristic root of 2gi%xy$~2ux.

Thus, it follows
that
Under the null hypothesis, from the large sample result of Wilks (19381, we have
To improve the X2 approximation, Bartlett (1941) suggests replacing the multiplicative factorn by the factor (rt -(r
q 1)/2),ie.,
+ +
If H I is rejected, we next test that Hz : (p(2),

statistic
. . , , p(q)) are jointly
zero. We use the test
Exercises
487
Under the nuU hypothesis H2, we have
+ q + 1)
In A2 +X 2 ( ( r - 1)(q - I)).
(18.A.16)
I f H l is rejected and Hz is not rejected, then p(f) is the only significant canonical correlation. If
Hz is rejected, however, we can continue the process. At the kth step, we test that Hk:(p(k), , , ,
p(q)) are jointly zero, using the test statistic
Under the null hypothesis we have
In the canonical correlation analysis of state space modeling, we have r = (p 4- 1)1n and k = q.
Canonical correlation analysis is used in testing the independence of two sets of variables. It is an important data-reduction method discussed in most books on multivariate
analysis. See, for example, Johnson and Wichern (1998).
18.1 Find the state space representation for the following models:
(a)
(b)
(c)
(d)
ZI = (1 - OIB (1 - 41B - &B~)z,

= (1 - OIB - 0 ~ 3 ~ ) a , .
(1 - &B)(1 - B)Zt = (1 - BIB)al.
Z, = C$~Z,-, a, - Bla,-l - f12at-z.
18.2 (a) Find the state space representation for the model given in Exercise 16.1, part (c).
(b) Is the representation obtained in part (a) unique? Why?
18.3 Consider the so-called autoregressive signal plus noise model in which we observe
2, =
+ b,
and the unobserved process Y,follows an autoregressive process, say, an AR(1) process
488
CHAPTER I8 State Space Models and the Kalman Filter
where a, and b, are independent zero mean Gaussian white noise processes with
variances cr: and mg,respectively.
(a) Express the model in the state space form.
(b) Find the steady state of the Kalman filter,
18.4 In Exercise 18.3, if 4
= 1, the autoregressive signal plus noise model becomes the

random walk plus noise model
2, =
r: + b,,
where the unobservable process Y, follows a random walk given by
(a) Express the model in the state space form.

(b) Show that the random walk plus noise model is in fact anAHMA(0, 1, I) model.
18.5 Compare the forecasts obtained from the transfer function model (18.4.4) and the
state space model (18.4.71,
18.6 Consider the corn supply data in series W13.

(a) Find a proper univariate ARMA model for the series.
(b) Find a state space model for the series.
(c) Compare your results in parts (a) and (b).
18.7 Use canonical correlation analysis to construct a state space model for the Lydia
Pinkham data Series W12.
Long Memory and

Nonlinear Processes
In earlier chapters, we discussed various methods for analyzing both univariate and multivariate
time series. All time series variabla (differencing if necessary) discussed so far, however, are
short memory processes with autocorrelation functions that decrease fairly fast, and all time
series models presented there are linear. To fit these linear models, we sometimes apply transformations so that the transformed series can fit these models well. Many of these commonly
used transfornations such as the logarithmic transformation and square root transformation are
nonlinear. The transformed series folloiving a linear model clearly indicates that the original
series may follow a nonlinear model. In fact, many time series exhibit chmacteristics such as
sudden bursts of large amplitude at irregular time epochs that are commonly found in seismological data as well as some financial series, and these finear models cannot properly describe
them. In this chapter, we introduce long memory processes and some nollliwar time series models that are in describing these long memory and nonlinear phenomena.
19.1 Long Memory Processes and Fractional Differencing

19.1.1 Fractionally Integrated ARMA Models and Their ACF
The ARIMA(p, d, q) process plays a dominant role in all chapters we have studied. In this
process, we have assumed that the value of d is an integer. In other words, the original 2,
process is nonstationary but its dth differences for some integer d is stationary. Clearly, for a
process to be stationary, the value of d has to be less than 1. We would, though, like to know
more exactly the range of d for the process
where #,(B)B,(B) # 0 for

5 1&d a, is a white noise process with mean zero and constant
variance u:, to be stationary and invertible. With no loss of generality, to answer the question,
we consider the simple case
(1 - B ) ~ =
z ,a,,
(19.1.2)
490
If the process in (19.1.2) is stationary, then we should be able to write it as
such that ( G I ) is square summable. From Taylor series expansion, we have the general
binomial formula
where
and r(.)is the gamma function. Using Sfizling's formula
we have
Clearly, (I)~)is square summable if and only if 2(1 - d) > 1, i.e., d < .5. Similarly, we see
that the process is invertible if and only if -.5 < d. Thus, the process in (19.1.2) or more
generally the process in (19.1.1) is stationary and invertible if and only if -.5 < d < .5.
The process in (19.1.1) with -.5 < d < .5 is therefore called the autoregressive fractionally
integrated moving average rnodeI, often denoted the ARFIMA(p, d, q ) model. The process
in (19.1.2) with -.5 < d < .5 is called the fractionally integrated (or differenced) noise,
Clearly, the spectrum of the fractionally differenced noise exists and, from (11.2.5) and
(l2.2.8b), i t equals
although it diverges at the zero frequency. Now, fiom (12,1.4),
where we use the following integral result (see Gradshteyn and Ryzhik (t196.51)
Thus,
Using Stirling's formula again, we have
where c is a constant with respect to k.
491
492

A stationary process is said to be a short memory process if its ACF is geometricaIly
bounded, i.e., if
where c is a positive constant and 0 < r < 1. It is known that the stationary ARMA models that we studied in earlier chapters are short memory processes. On the other hand,
a stationary process is said to be a long memory process if its ACF follows an asymptotic
hyperbolic decay, i.e., if
where a < 0. A long memory process with an ACF which is not absolutely summable, i.e.
03
IPk[ = 00, is further said to be persistent.
Because d is known to be less than .5, it is clear that the ACF given in (19.1.8) follows an asymptotic hyperbolic decay. Hence, the fractionally differenced noise process
and thus the ARFIMA(p, d, q) process is a Iong memory process. Furthermore, for
the ACF, pk, in (19,1,8), 2g..+m
lPkl converges only if (2d - 1) < -1, or d < 0. The
fractionally differenced noise process and thus the AFSIMA(p, d, q) process is in fact
persistent when 0 < d < -5.
19.1.2 Practical Implications of the ARFIMA Processes

An AlWMA process contains several characteristics that, in finite sampIes, are similar to those
of a nonstationary process, For example, the ACP of a persistent AREZMA(p, d, q) model

decays very slo\vly, a phenomenon simitar to the sample ACF of a nanstationary time series discussed in Chapter 4. Also, realizations of both persistent ARFlMA models and nonstationary
ARIMA models have periodograms that diverge at the zero frequency. These similarities often
lead to model misspecification, For instance, a stationary ARFLRlA model could be misspecified
as a nonstationary ARbL.4 model. The consequence of this overdifferencing has some undesirable effectson parameter estimation and forecasting.
In t e r m of forecasting, we showed in Chapter 5 that the forecast based on a stationary
process converges to the mean value of the process. Thus, the forecast of a long memory process
should converge to the mean value of the process, although it will converge at a much slower
rate. A misspecified nonstationary model wiU therefore produce a bias and a forecast error with
an inllated variance. For more details, see Crato and Ray (1996).
19.1.3 Estimation of the Fractional Difference

For the ARFIMA model given in (19.1. I), let W, = (1 - B ) Zt,
~ and let fW(m)and fz(w) be the
spectral density fbnction of {W,) and {Z,), respectively. Then,
493
where
is the spectral density of a regularARMA(p, q) model. Note thatti(@) +OI as w -+0.

Taking logarithms on both sides of (19.1.1 I), we get
Replacing w by the Fourier frequencies wj = 2njlr1, j = 1,

the periodogram of {Z,), to both sides, gives
For near zero, i.e., for j = 1, . . . , m

I n ( f ~ ( ~ ~ ) / ' f ~ t0.
O Thus,
>)~
. . . , [n/2]and adding In IZ(oj),
<< (142)such that m/n -+ 0 as n
-t
m,
we have
where kj = h b(q),
c
In fw(0), 4 = lnl 1 T eiujI-2,and ej = In (Zz ( o j ) / f Z(wj)).Here we
recall from Chapter 13 that the sequence [Iz(oj)/fi(oj)] and therefore ej are approximately
i.i.d. random variables. For computation, because
we have
The least square estimator of d is given by
494
Geweke and Porter-Hudak (1983) showed that
The 95% corhdence interval for d is obtained by
In practice, we often take ni = [6]

so that the regression (19.1.13) is performed over the
Fourier frequencies oj= 2 ~ j / nj , = 1, . . . ,[fi],
where [.I is the greatest integer function.
Once d is estimated, we can estimate the parameters, +j and Bj, from the fractionally differenced data Wt = (1 - B ) Zt
~ using the methods introduced in Chapter 7. The estimated
model can then be used to forecast the future values with the results discussed in Chapter 5.
One should beware, however, that because time series aggregates are often used in data
analysis and modeling, a generating long memory ARFLMA process could be misidentified as
a short memory ARhilA process due to the consequence of aggregation as shown in Teles, Wei,
and Crato (1999). This misspecification may lead to a very undesitable result in forecasting.
19,2 Nonlinear Processes

With no loss of generality, we now consider a zero mean process. It is known that a nondeterministic linear time serie-s process Z, can always be written in the form
where t/io = 1, a, is a white noise process with mean 0 and variance a:, and
03
$I ( B ) =
(4BiIf zjzo
$; 4 m, then the process is stationary, and fiom, section 12.2.2,
the spectrum exists and equals
zpo
The equation given in (19.2.1) is the general form of a linear process. More generally,
a time series process can be written as
495
The process, Z,, in (14-2.3) is linear if it contains only the first term. The process is nonlinear
if it contains more than the first term. Clearly, second-order properties such as autocorrelation functions and spectra are no longer sufficient to describe the genera1 nonlinear process
given in (19.2.3). The higher-order moments are called for even with a weakly stationary
process. In addition, for the models to be presented in this section, we also require that (a,}
in (19.2.3) be a process of iqi.d.(0, cr;) random variables.
19.2.1 Cumulants, Polyspectrum, and Tests for Linearity and Normality

Other than regular moments, many higher-order moments are available for describing a
process. Because of their many desirable properties, we choose cumulants. Assume that the
process 'in (19.2.3) is an 11th order weakly stationary process. Then, from Chapter 2, all its
joint regular moments up to order rl exist and are time-invariant. In such a case, the 11th-order
t, in
cumulant, C(kl, k2, . . , kn-,), exists and is defined to be the coefficient of int1t2
the Taylor expansion about the origin of
q(tl,f2,. . .,t,) = ln{E[exp(itlZt 4- it2Zl+k,+
1).
-i- itnZt+k,,-l)
(14.2.4)
..
As a result, the function, rp(tl, tz, . , t,,),in (19.2.4) is also known as the cumulant generating function. Clearly, because of the nth-order stationarity, the cumulant generating function
is independent of time origin.
Following the simiIar results from Chapter 12, if the nth-order cumulants, C(kl, k2, . . . ,
kn-,), are absolutely summable, then its Fourier transform exists and equals
where -T 5 wi 5 a for i = 1, . . . , a - I. The function, f (wl, 12, . . . , w,,-~),is often

known as the nth-order cumulant spectrum or the nth-order polyspectrum. Specifically, the
second-order polyspectntm is
and the third-order polyspectmm becomes

- i ( 4 +wk2),
-T 5
I,,
12
w,
(19.2.7)
kl=-co k2=-w
which is also known as a bispectrum.
Note that E[exp (itZ)] is the characteristic function of Z. For a Gaussian random variable
with mean p and variance a2, it is known that its characteristic function is given by exp[it
(it)2 u2/2j. It also follotvs that its cumulants of order greater than two are zero. In addition, if
496
Y and Z are independent, for any constants a and b we have: E[exp(itlaY + it2bZ)] =
E[exp(itlaY)] E[exp(it2bZ)]. Because a multivariate normal can be defined as a vector of
linear combinations of independent univariate normal variables (see, for example, Rao t1965,
p. 440]), it follows that all cumulants of order greater than two are also zero; hence, the bispectrum and all higher-order polyspectra of a Gaussian process are identically zero. One can
use these higher-order polyspecka to measure the departure of the process from normality.
To illustrate the procedure of deriving cumufants, let us consider the second-order
cumulant, C(kl), which is the coefficient of i2t1t2in the Taylor expansion about the origin of
From the Taylor series for a function of several variables, we have
Now,
a ~([exp(itlZ,+ it2Zt+lrllliZ,+k15
at,
where p
E[exp(it1Zt
+ it2~t+q)
(o,o)
= E(Zt). Hence,
C(kl)
= E(Zt - w)(Zt+kl - P ) =
Ykl)
and the second-order polyspectrum is simply the spectrum
where
-T
5 u1k 'IT.Similarly, the
third-order cumulant equals
(19.2.9)
497
Let
It can be seen that T ( q , w2) can be used to test whether a process is linear because under the
-
M
-
null hypothesis the process is linear, we have 2, = z @ j a , - j ,and

j=O
where ru = ~ ( a ;and
) +j = 0 for j
< 0, Hence,
Thus, under the null hypothesis that a process is linear, the function
is constant. Testing for linearity can therefore be based on checking the constancy of the following sample statistic over a grid of frequencies
498
where
M is the truncation point, W(k) is the lag window, which was discussed in Chapter 13,
m = max(0, kl, k2), and
If the process is Gaussian, then (Y = ~ ( a ? =

) 0. Both the bispectrurnf(ol, u2)and the
function T ( o l ,w2) equal zero at all frequencies. Thus,;esting for normality can also be based
on checking whether the sample statistic f(ml, m2) or T(wl, 0 2 ) is statistically zero over a grid
of frequencies. For more details of this and the previous tests, we see Subba Rao and Gabr
(1980, 1984). A non-Gaussian process could also have a zero bispectrum. From the earlier
discussion, we see that, for a comprehensive test, one should really be testing all the higherorder polyspectra.
The principal advantage of the above test is that it can be used for any type of nonlinear
processes. As a result, however, the test suffers in terms of its power. Additionaliy, as pointed
out in Chan and Tong [1986), the performance of the test is also affected by the choice of the
lag windows, the truncation points, and the placing of the grids of frequencies. For these reasons, great skill is required for its successful use.
19.2.2 Some Nonlinear Time Series Models

The general form of the process given in (19.2.3) contains an infinite number of parameters. Once nonlinearity in the process is c o n h e d , the next task is to build a suitable
finite number parameter nonlinear model. Many nonlinear models have been proposed in
the literature with the objective of describing different characteristics exhibited by time
series encountered in practice that cannot be appropriately accounted for by linear
processes. In addition to the finite order form of (19.2.3) that has been called nonlinear
moving average (NLMA) models, we give below some other general finite parametric
nonlinear models without details. Interested readers should see excellent references in
499
Subba Rao and Gabr (1984), Tong (1983, 1990), Priestley (1988), and Fan and Yao
(2003). The models are as follows:
1, The bilinear model of order ( p , q, 7, 8s)( W p , q, 7, $1model) is
2, The nonlinear autoregressive W A R ) model of order p (NLAR(p) model) is
wheref (.) is a nonlinear function.

3. The threshold autoregressive PAR)mode1 is
where d is the delay parameter, T and S are the location and scale parameters, respectively, and I(.) is a smooth function, such as logistic and exponential, or an indicator
function.
4. The exponential autoregressive model of order (p, d ) (EXPAR(p, d ) model) is
Z, =
2 (+;
i+ a ; e - p ~ Z f - d ) ~ , - ~ a,,
i= l
where pi 2 0 for i = 1, . . . ,p and Zt+ is a model-dependent variable.

The series a, in above models are all i,i.d. (0, ui) random variables.
These models are in the form of general orders. In practice, just as in linear time series
modeling, the orders that have been commonly chosen are relatively small. For example,
with respect to the NLAR model, NLAR(1) and NLAR(2) models are the most commonly
used. When the function I(.) in model 3 above is an indicator function, the model becomes
a simple two regime threshold autoregressive model. Because of its many interesting properties and close connection with the models studied in earlier chapters, we discuss the general k regime threshold autoregressive model further in the next section.

Threshold autoregressive models were originally proposed by Tong (1978, 1983, 1990)
and Tong and Lim (1980). They are piecewise linear models where the linear relationship changes according to the values of the process. Specifically, let us partition the real
500

k
Ri,where RI = (-m, rl), Ri = fq-1, ri) for
line R into k intervals or regimes, R =
i = 2, . , k - 1, Rk = Irk, w), and - rn < rl < ..- < r k - 1 < 03 are the thresholds.
A time series process 2, is a self-exciting threshold autoregressive (TAR) process if it
follows the model
..
2, = &I
+ $#jj)z,-, + a!",
i= 1
where j = 1, . . . ,k, d is a positive integer and is known as the delay parameter, and a(jl is a
sequence of independently identically distributed random noise with mean zero and variance
C T ~For
.
convenience, we will refer to (19.3.1) as a TAR(k; pl, . . . ,pk; d ) model, where k is
the number of regimes, separated by (k - 1) thresholds ris, and pi denotes the order of the
autoregressive model in the jth regime. The process is a linear autoregressive model in each
regime and is nonlinear when there are at least two regimes with different linear models. The
model contains many interesting features, such as limit cycles, amplitude dependent frequencies, and jump phenomena, which cannot be described by a linear time series process,
Moreover, it reduces to a linear model when k = 1. It becomes a nonhomogeneous linear AR
model when only the noise variances differ for different regimes; it becomes a random level
differ for different regimes.
shift model when only the constant terms
EXAMPLE 19.1 Letk = 2, R, = I-m, O),R, = [O,m),p, = p z = 1 , a n d d

following simple TAR(2; 1, 1; 1) model,
= 1 forthe
where a!'),i = 1,2, is a sequence of i.i.d. (0, c$) random variables. The series exhibits an
asymmetric pattern of increasing and decreasing. At time (t - I), if the value of Z,-l is negative, then the value of 2, at the next period tends to be positive. If the value of 2,- 1 is positive, however, it tends to remain positive for a few periods before it changes to negative. As a
consequence, the series has large upward jumps in the negative regime (regime I), the number of observations in the negative regime is less than that in the positive regime (regime 2),
and the variance, (T?,of the error term in the negative regime is normally larger than the variance, uj, of the error term in the positive regime. The mean of the series, p, = E(Z,), is a
weighted average of the conditional means of the two regimes, which is nonzero, and the
weight for each regime is equal to the probability of 2, in that regime. Many financial time
series exhibit phenomena illustrated in this example.
19.3.1 Tests for TAR Models

Let p = maxIpl, . . . ,pk). With the understanding that
(19.3,l)as
.
4jj)
= 0 for i > pi,we can rewrite
501
and simply call it the TAR(k; p; d ) model. Petruccelli and Davies (1986) proposed a test for
threshold nonlinearity using the concept of ordered autoregression. Tsay (1989) further
developed the method into a modeling procedure for TAR models. For simplicity, but with
no loss of generality, we illustrate the test using the TAR(2; p; d) model.
For any given A R ( p ) regression with p observations, Z, = $o
~ ~ = ' = l + i ~ ,a,,
- i for
t = (p I), , , , n, let us call (Z,, 1,
. . . , Zr-d, . . . , ZI-p) a case of data for the AR(p)
model. Suppose that the threshold variable Zt-d assume values {Zh, . . . ,Zn-d), where h = rnax
(1,p
1 - d ) , Let (t> be the time index of the ith smallest observation of (Zh, . . . , Z,,
We can obtain the folloiving arranged autoregression with cases rearranged according to the
ordered values of the delay variable ZtPd:
+
+
With a proper choice of r, the above-rearranged autoregression can be fitted iteratively. For
example, Tsay (1989) suggested an iterative fitting with r = r-, r1, . . . ,(11 - d - h I),
where rd, = (n/lO) p. After each iterative AR fitting of the model in (19.3.4),we can compute
the one-step ahead standardized forecast error,
Under assulnption of linearity, the least
squares estimates of an AR model are consistent and the one-step ahead standardized forecast
errors 2;s are known to be asymptoticallyindependent and identically distributed with zero mean
and unit variance. Thus, we can form the cumulative sum
S, =
i=r-+I
Zi,
for r = rd, -t- 1,
. . . , ( n - d - h -k 1)
and compute the test statistic
T=
max
Is,l
v(Tl
- -
r&~r~(tt-d-/[+I)
+ 1 - rhn)'
From the invariance principle (Feller, 1971, p. 342), we can compute the asymptotic p-value
of the test statistic T by
The linearity is rejected if P(T < x) is less than the sign5cance level a.
It is we11 known that when the mode1 is linear, the one-step ahead standardized forecast
errors Zj, ( r ~ , I) 5 i 5 (n - d - h I), are not only asymptotically independent and
502
CHAPTER 19 Long Memory and NonIinear Processes
identically distributed but are also orthogonal to the regressors {Z(o+d-lr

Thus, Tsay (1989) suggested the least squares regression
for (rd, -t- 1) S i 5 (n - d - h
. . . , Zco+d-p).
+ I) and the associated F statistic
where the summations are over all the observations in (19.3.7) and giis the least squares residual of (19.3.7).Under the 19earity assumption, i.e., when Z, follows a TAR(k; p; 6)model with
k = 1 , the sample statistic F(p, d) is asymptotically distributed as an F distribution with @ 1)
y d (11 - d - h - r- - p) degrees of freedom. The linearity assumption is rejected if
F ( ~ , > Fa,(p+l, r ~ - d - h - r ~ - p ) -
19.3.2 Modeling TAR Models

The AIC Approach Tong and Lim (1980) proposed a TAR modeling procedure based
on Akaike's information criterion. We illustrate the procedure for a TAR (2; pl, pz; d ) model,
The other cases follow similarly.
Step 1. Search for the orders of the piecewise AR models,
Given any fixed values d and r, let P be the maximum order to be entertained for the
two piecewise linear AR models. The choice of P is subjective. Tong (1990) suggested P =
1
na with a < 2.
Based on the given r, construct an arranged autoregression for each regime.
The orders pi for j = 1,2 are chosen so that
where nj is the number of observations in the arranged autoregression of the jth regime,
gj@i) is the vector of residuals from an AR (4)fitting, and \]XI[ = (x', x)lI2 is the norm of a
vector x,Let
Step 2. Search for the threshold value r.

Let d be fixed and allow r to vary over a set of potential candidates R = { r l , r2,
and minimize the AIC over this set. Thus, we choose the value "rf r so that
AIC(d, ?)
min(AIC(d, r)}.
reR
. . . ,r,)
(19.3.11)
503
Because r can be any value in an interval, this step may take several iterations based on
refined grids of potential candidates. To initiate the iteration, Tong (1990) suggested using
the sample percentiles as initial candidates of r.
Step 3. Search for the delay parameter d.

Let D = {I, 2, , , , , T} be the set of iotential candidates of the delay parameter d. We
now let d vary and choose its value 2 by
where lid = max(d, P). Because the effective number of observations in the airanged autoregression varies as the value of d changes, we should normalize the AIC by the effective number of observations (n - nd).
EXAMPLE 19.2 Consider the series of annual sunspot numbers between 1700 and 1983
that is a subseries of Series W2 discussed in earlier chapters. In this example, however, we
use the original numbers instead of their square roots, Let P = 11, R = {rZ0,r30, . . , raO},
where r, is the qth percentile of the data and D = {1,2,
,6). The AIC approach leads to
a TAR(2; 4, 11; 3) model. The estimation resufts are
...
The numbers of observations are 129 and 144, and the residual variances are 258.66 and
71.44, respectively,
In addition to the formal AIC approach, Tong (1983, 1990) suggested performing a preliminary analysis of a time series using graphical methods such as reverse data plots,
directed scatter diagrams, and bivariate histograms to search for evidence of nonlinear time
series characteristics such as time irreversibility, cyclical behavior (limit cycles), and nonnormality. For details, we refer readers to Tong's books.
The Arranged Autoregression Approach We now illustrate the arranged autoregression approach proposed by Tsay (1989).
Step 1. Tentatively select the AR order.

Select the maximum order, P, to be entertained for the two piecewise 1inearAR models. This
tentative selection of the AR order can be based on the PACF or on other criteria such as AIC.
Step 2. Search for the delay parameter d and hence the threshold variable ZtVd.
Let D be the set of possible delay lags. Often, we let D = (1, 2, , , P). Then fit the
arranged autoregressions for the given P and every element d of D, and perform the threshold nonlinearity test. An estimate of the delay parameter, d, is chosen such that
..
i ( ~2),= ~nax{;(p,
d))
dED
504
or, more precisely,

pvalue(2) = min{p-value[fi(~,d ) ] 1,
deD
(19.3.15)
The argument of the above selection is that if a TAR model is called for, then a reasonable
starting point for the delay parameter is to begin with the one that gives the most significant
result in the threshold nonlinearity test.
Step 3. Search for the threshold values.
Let b,j be the recursive estimate of the lag-j AR parameter in the arranged autoregression. Under linearity assumption, $j is a consistent estimate of $j, and the t-ratios of +j will
behave exactly as those of a linear AR model before the recursion reaches
the threshold
*
value r. Once the recursion reaches the threshold value r, the estimate #j begins to change
and the t-ratios start to deviate. Thus, the threshold values can be identified through the
change points in the convergence pattern of the t-ratios from a scatterplot of the recursive
t ratios of versus the values of the threshold variable Z,+.
6
Gj
Step 4. Refine the results.

Refine the AR order and threshold values in each regime using tbe available linear
autoregression techniques, if necessary.
EXAMPLE 19.3 We now analyze the same subseries of 284 annual sunspot numbers
using the arranged autoregression method. To be comparable with the AIC approach, we
also choose P = 11; hence D = 11, 2, . , 11). The results of threshold nonlinearity
F tests based on the given P and D are summarized in Table 19.1. The value d = 2 gives
the minimum p-value. Thus, we select ZlW2as our threshold variable. To search for the
..
TABLE 19.1 The p-values of

F tests for the annuat sunspot
numbers (Series W2).
D
p-value
505
TIGURE 19.1 Scatterplot of recursive t-ratios of the lag-2 AR coefficient

+z versus ordered 2t-2for the sunspot numbers.
h
threshold value, we construct a scatterplot of the t-ratios of #2 versus the ordered values
of Zfd2in Figure 19.1, From the plot, we see that the t-ratio changes its direction twice,
once at Zt-2 = 35 and again at around Z1-2 = 70, suggesting that rl = 35 and i-2 = 70.
We then use AIC to refine the AR orders in each regime. The results lead to the foflowing
TAR(3; 11, 10, 10; 2) model:
The numbers of observations are f 18, 89, and 66, and the residua1 variances are 174.23,
125.68, and 89.11, respectively.
506
Chan (1989) gave a detailed analysis of annual sunspot numbers including TAR models
in (19.3.13) and (19.3.16). There are many studies on nonlinear models and their applications.
Because their ability to capture the asymmetric phenomena, the models or the hybrids of
these models such as TAR-GARCH models are found useful in many economic and financial
studies. For more examples, we refer readers to Subba Rao and Gabr (1984), Tsay (1989,
1991), Chan (1990), Tong (1990), and Pena, Tiao, and Tsay (2001) among others,
~ = (1
19.1 Consider the AIZFIMA(0, .4, 1) model, (1 - B ) . Zt
i.i.d. N(O, 1) random variables.
(a) Find and plot its spectrum.
(b) Find and plot its autocorrelation function.
(c) Comment your hdings in part (a) and (b).
.8B)a,, where the a, are
19.2 Let Z, be a stationary Gaussian process. Show that its bispectrurn is identically
zero.
19.3 Show that the bilinear model, 2, = qbIZ,-l 3- $zZ,-la,-l 3- a,, can be expressed in
the form of (19.2.3).
z:=~(+~
19.4 Consider the EXPAR(2, 1) model, Z, =

-F + i e + b l z : - l ) ~ t - i a,. Discuss
the different phenomena of the model when [ ~ , - ~ 1is large and when I Z , - ~ ~ is small,
and show that the EXPAR model behaves rather Like the TAR model where the
coefficients change smoothly between two extreme values.
19.5 Consider Series W2 of the yearly sunspot numbers from 1700 to 2001 given in the
appendix.
(a) Test the linearity of the underlying process using the bispectrurn test.
(b) Build a TAR model for the series. Compare and comment your TAR model with
the standard AR models obtained in Example 6.2.
19.6 Consider Series W7 of the yearly number of lynx pelts sold by the Hudson's Bay
Company in Canada from 1857 to 1911 given in the appendix.
(a) Perform the normality test on the series.
(b) Perform the linearity test on the series.
(c) Build a TAR model for the series. Compare and comment your TAR model with
the standard AR models obtained in Example 6.7.
Aggregation and
Systematic Sampling
in Time Series
In any scientific investigation, after making a conjecture or a proposition about the underlying
model of a phenomenon, a scientist must test his or her proposition against observable data.
Suppose that, based on theoretical grounds, a scientist proposes an underlying model for a
time series process in terms of a basic time unit t. In some fields, a scientist can test his or her
proposition by analyzing the data from a designed experiment in terms of the same time unit t
used in his or her proposition. In some cases, however, controlled experiments may not be
possible. Although time series observations may be available, the investigator often cannot
choose the time interval. Thus, the assumed time unit in the model may not be the same as the
time unit for the observed data. In many studies, data often are available only through aggregation or systematic sampling.
Generally speaking, a time series variable is either a flow variable or a stock variable. A flow variable such as industrial production exists only through aggregation over a
time interval. A stock variable such as the price of a given commodity exists at every
time point. The values of a flow variable are usually obtained through aggregation over
equal time intervals, whereas the values of a stock variable are often obtained through
systematic sampling. In this chapter, we study the consequences of aggregation and systematic sampling on model structure, parameter estimation, forecasting, and various time
series tests.
20.1 Temporal Aggregation of the ARIMA Process

Let zt be the equally spaced basic series whose dth difference ivf = (1
covariance stationary process with mean 0 and autocovariance function
- B ) ~ zfollows
,
a
508
Assume that the observed time series ZT is the m-period nonoverlapping aggregates of z,
defined as
where T is the aggregate time unit and m is fixed and is called the order of aggregation.
Thus, for example, if t represents the time unit of month and m is 3, then the ZT are quarterly
sums of the monthly series 2,. We call ZT an aggregate series, and we refer to z, as a basic or
nonaggregate series.
20.1.1 The Relationship of Autocovariances between

the Nonaggregate and Aggregate Series
To derive the relationship between the autocovariances of the nonaggregate series z, and its
aggregate series ZT, we first define the following nt-period overlapping sum:
and note that
Let
be the backshift operator on the aggregate time unit Tsuch that L ~ Z T= &--I. Not+,'
Thus, by letting
we have
509
Hence, the dth difFerenced,aggregate series, UT = (1 - B ) d ~ Tis, covariance stationary as it

is a finite moving average of a stationary process w,.Stram and Wei (1986b) show that the
autocovariance function y,(k) for ~ v and
,
the autocovariance function yU(k) for UT are
related as follows:
where B now operates on the index of yw(J3 such that By,&? = y,,(j - 1). Thus,yLl(k)is the
linear transformation of the autocovariances y W O from j mk - (d 4- l)(m - 1) 10j = rnk
(d l)(m - 1). The coefficients in the Iinear transformation are found by expanding the polynomial (1 f B 4- . . ~ m - ~ ) % ~ +We
l ) . em write this transformation in the follo~ving
matrix form:
The coefficient matrix A is equal to
+
+
+ + . +
where 0,,is a 1 X ,t vector of zeros and C is a 1 X [2(d

l)(m - 1) I] vector of Ci,
which is the coefficient of B' in the polynomial (1
B
B ~ - ~ ) ~ (Because
~+~).
y,(-k) = yw(k) for all k, we can, in the above matrix, delete the first (d
l)[ni - 1)
columns corresponding to yw[-(d
I)(m - I)], . . . , and y,(-1), by adding them to the
columns corresponding to y,E(d + l)(az - I)], . . . ,and yw(l), respectively. Thus,
+
where A$, is the resufting matrix formed by this deletion of the first (d -t l)(nt
and adding them to the proper remai&ng columns of the matrix A.
1) columns
EXAMPLE 20.1 To clarify the construction of the matrices A and A;, we consider an
MA(2) model,
510
+ +
where y,(O) = (1 Of B$)ai, y,(f) = (-el 4- 8162)u,2, yz(2) = - 8 2 d , and Y2(3= 0,

[jl > 2. To find the model for its third order aggregate, & = (1 + B + B * ) z ~we
~ , have d = 0,
nt = 3, and from equation (20.1.7)
Hence, by (20.1.8),
By using (20.1.9), equation (20.1.1 1) is reduced to
Thus, the third aggregate of an MA(2) model is an MA(1) model,
where yA0) = (I + a 2 ) a 3 = 3[1 81 + O$](T; + 4[-01 + f1192]~; ~[-o~]cT$,and

y d l ) = -0u3 = 2[-02]u: + [-dl + d182]u2. The parameters, O and oi,of the aggregate model for ZT can be found to be functions of 01,B2, and (TI
through 'the relations,
20.1 Temporal Aggregation of the ARlMA Process
511
and
20.1.2 Temporal Aggregation of the IMA(d, q) Process

Suppose that the nonaggregate series zt folfows an IMA(d, q) model
where the a, are white noise series with mean 0 and variance a:. We would like to know the
corresponding model for the ntth-order aggregate series & defined in (20.1.1). Let
UT = (1 - LjjdzT.
From (20.1.7), we see that yu(l) is a weighted sum of those y,(k) for
which ml - (d -I
1)(nz - 1) Ik 5 mE + (d
lf(rn - 1). Because y,(k) = 0 for lkl > q,
yv(2) will be identically zero if I > q* = [d 1 + (q - d - l)/m]. ?%us,the only nonzero
autocovariances of UT are at most yu(0), yu(l), . . . , yufq*), which implies that the
n~th-orderaggregate series &-= (1 B -IB'"l)~mTfollo~san IMA(d, No) process
+
- .- +
where the AT is a white noise series with mean 0 and variance ui,
and [XI
is used to denote the integer part of x. The parameters Pi's and ui are functions of 8;s
and a:. They can be derived from solving the system of equations in (20.1.9), which reduces to
a much simpler form for an IMA(d, q) model. In fact, equations (20.1.13) and (20.1.15) impfy
that in shrdying temporal aggregation for the IMA(4 q) model, one need only consider the first
(q* 1) rows and (q 1) columns of the coefficient matrix A$. Letting A $ ( ~ be
) this submatrix consisting of these first (q* t 1) rows and (q 1)columns of A$, we have
It should be noted that q* in (2011.15) is only an upper bound. To see that, let us consider
the case when m = 3 and the model for z, is an MA(2) process, z, = (1 - OIB - J2~')a,.
h this case, q* = 1,Also, by (20.1.7),
512
Thus, from (20,1.8), (20.1.9), and (20.1.16), we have that
and
where ~ v =
, z, and UT = ZT,
Now,
so that
Thus, if %2 = 01/(01 - 2), then yufl) = 0 and the aggregates become an uncorrelated white
noise series.
20.1.3 Temporal Aggregation of the AR(p) Process

Suppose that the nonaggregate series z, follows a stationary AR@) process
(l-$lIB-...-
@P)z,
= a,.
(20.1.20)
Let &(B) = (1 - #lB - , * - # I ~ B P )and 6 ~ for

' i = 1, . . . ,p* be the distinct roots of &(B),
each with multiplicity q such that
si = p. For my given value of m, let b equal the number
of distinct values Sin for i = 1, . ,p*. Furthermore, partition the numbers si, i = 1, . . ,p*,
into b distinct sets Ai such that sk and sj E Ai if and only if 6f: = Sjm. Then Stram and Wei
(1986b) show that the nzth-order aggregate series, &- = (1 B
Bm-')zmT,folfows an
ARMA(M, N l ) model
~71,
..
+ + .. +
a
20.1
Temporal Aggregation of the ARlMA Process
513
where
maxAi = the largest element in ini,
the AT are white noise with mean 0 and variance -3, and ai, pj, and u$are functions of +;s
and ui.
EXAMPLE 20.2 Assume that the nonaggregate series z, follows the AR(2) model
with
= -(.5)'j3,
= -(.5)2fl,
and CT:
~n = 3, we first find the roots of #p(B)
= (1
1. To derive the aggregate model for ZT when
- 413 - # 2 ~ 2 ) , which are
and
1
They are distinct, each with multiplicity 1. Thus, p* =: 2, sl = q = 1. But, 61 = S$ = 5,
whichimplies that b = 1 and$II = ( 1 ) . HenceM = 1 andNl = [l f 1 - (2 + 1)/3] = 1,
and the third-order aggregate series, ZT, follows anARMA(1, I) model.
# Sj but ST = 6y. Clearly,

the process given in (20.1.24) has a hidden periodicity of order 3. It is worth noting that
A model is said to have hidden periodicity of order m if
the reduction of the AR order on aggregation in the above example is due to the hidden periodicity, For a more detailed discussion, see Stram and Wei (1986b).
20.1.4 Temporal Aggregation of the ARIMA(p, d, q) Process

Now assume that the nonaggregate series follows a mixed ARTMA(p, d, q ) model
514
where
and the a, are white noise. The polynomials $(B) and O,(B) are assumed to have their roots
outside the unit circle and to have no roots in common. Additionally, we also assume that the
model has no hidden periodicity of order m in the sense that if a;', i = 1, . . . ,p, are the
roots of +p (B), then S? = Sf' if and only if ai = Si. Letting &(B) = llf==I(l- SjB) and
multiplying
on both sides of (20.1.26), we get
Let
It is easily seen that
for K
> N2,where
Thus, the aggregate series ZT = ZmTfollows an ARIMAf p, d, N2) model
where the AT are white noise with mean 0 and variance o j , and the parameter Pi's and a$
are functions of &'s, %;s, and 02.Note that in this case, the AR order is unchanged by
aggregation and that the roots of the AR polynomial of the aggregate series model are the
mth powers of the roots of the nonaggregate series AR polynomial.
If the ARIMA(p, d, q) model of the nonaggregate series z, has hidden periodicity of
order ni in the AR polynomial, then by the results of Section 20,1.3,the mth-order aggregate
20.1 Temporaf Aggregation of the ARIMA Process
515
series, &-, will follow a mixed ARIMA(M, d, N) model. The AR order M is determined by
(20.1,22), and the MA order N is given by
More generally, aggregation of seasonal ARIMA models was studied by Wei

(1978b). Assume that the nonaggregate series z, follows a general multiplicative seasonal
ARIMA(p, d, q) X (E: D,Q), model with a seasonal period s,
where
have roots outside of the unit circle and the a, is a white noise process with mean 0 and variance a:. Using the results from previous sections, it is easily seen that the mth-order
aggregate series, &-,when s = mS for some integer S and nz < S, will follow a multiplicative
seasonal m A ( M , d, N) X
D, QIs model with a seasonal period S, i.e.,
(e
where
which all have roots outside of the unit circle. M is determined by (20.1.22), N by (20.1.31),
and the AT is a white noise process with mean 0 and variance ~ 1The
. parameters a i s are
functions of #j's; the parameters Pi's and mi are functions of 4;s, Oils, and cr;. When ni r S,
aggregation reduces a seasonal model to a regular AIUMA model.
20.1.5 The Limiting Behavior of Time Series Aggregates

To see the implications of time series aggregation, it is interesting to study the limiting
behavior of the time series aggregates. Suppose that z, follows an ARIMA(p, d, q) model in
(20,1,26).When m 4 m, Tiao (1972) shows that the limiting model for the aggregates, ZT,
516
exists and equals the IMA(d, d ) process, which is independent of p and q. When z, follows a
seasonal ARWIA(p, d, q) X
D,Q), model, Wei (1978b) shows that the limiting aggregate
mode1 of ZT becomes an IMA (D d, D + d ) process. Wei and Stram (1988) show that the
results hold also for norinvertible models and that if 1 is a root of the moving average polynomial for the zt process, then 1 is also a root of the moving average polynomial for the Zjprocess. Two interesting consequences of the results are the following:
(e
1. The Limiting model of the time series aggregates from a stationary model is a white
noise process.
2. The limiting model is unique and is independent of the stationary portions of the nonaggregate model.
As a result, to study the limiting model of a time series aggregate, with no loss of generality, one needs only consider the case where the underlying nonaggregate series z, follows an
IMA(d, d ) model. In this case, the autocovariances yu(k) for UT = (I - B)'ZT and the
autocovariances yw(j?for rv, = (1 - 3)d are related, by (20.1.16),as
where A$(d) becomes a (d -t. 1) X (d 1) square matrix. Wei and Stram (1988) call the
matrix ~ f , ( d the
) aggregation matrix for the linear transformation that maps the autocovariances, yw(j)'s, of the underlying IMA(d, d ) model for z, to the autocovariances, yutj)'s, of
the IMA(d, d) aggregate model for G.
From the construction, it is easily seen that the matrix
Ad,(d)is nonsingular, Upon normalization, we have
where
and a\ is the first row of ~ $ ( d )Note

. that
where y, = (y,(O), yw(l),

by (20.1.34),
517
. , . ,y,(d))'. Now, y,(O) is the variance of ~ v =, ( 1 - B)~z,,and
which is the variance of UT = (1 - BldzT,Hence, sip, > 0 and the aggregation transformation defined in (20.1.35)is continuous.
Let fm be the mth-order aggregation transformation given in (20.1.35).If
then
Hence, the autocorrelation vector of a limiting model is unaffected by aggregation. That is, if
we let p r ) be the autoconelation vector of the limiting model, then
Thus, the limiting autocorrelation vector p r j is simply the eigen-vector of the aggregation
matrix A$(d) with the associated eigenvalue a i p r ) ,
Let the invertible limiting model of aggregates be
where 8iw)(B) = (1 - 0jpO)f3- . ; - B & ~ ) L ?the

~ , roots of 8$?')(23) = 0 are all outside the
unit circle, and the AT are white noise with mean 0 and variance ui. Wei and Stram (1988)
prove that the autocorrelation vector p y ) for the invertible limiting aggregate mode1 in
(20.1.40)is given by the eigenvector associated with the largest eigenvdue of ~ $ , ( d where
),
the
eigenvector is scaled so that its first element equals 1. Wei and Stram give an interesting note
518
TABLE 20.1 Sample autocorrelations &(k) of (z,).
TABLE 20.2 Sample autocorrelations j?~,(k) of {[I - B ) t , )
{w,).
that because the linear transformation given i n (20.1.35) is continuous, when deriving the
aggregate model, the nt in ~ d , ( dcan
) be any integer larger than or equd to 2. The limit can
then be taken through n$ as k -+ a.Specifically, we can choose m = 2, which greatly simplifies the calculation of the coefficients of ~j in the polynomial (1 f 3 f
and
hence simplifies the construction of A$d) as illustrated in the following example.
--+
EXAMPLE 20.3 To illustrate the above results, let us consider the number of monthly
unemployed young women per 1000 persons between the ages of 16 and 19 in the United
States from January 1961 to August 2002, which was examined in Chapters 5 and 7. For
convenience, Tables 20.1 and 20.2 repeat the sample autocorrelation functions of the original
series and its first differences. It is evident that the original series z, is nonstationary and that
the series of its first differences (1 - B)z, is stationary following an MA(1) model.
To examine the limiting behavior of the autocorrelation function of the aggregates, we
compute the sample autocorrelations of for nt = 6 and rn = 12 corresponding to the semiannual and annual total of unemployed young women. The autocorrelations for &- show
nonstationai-ity. Table 20.3 shows the autocorrelations of the differenced aggregates, i.e.,
TABLE 20.3 Sample autocorrelations $o(k) of ((I
- B)ZT] = {UT)
20.1
Temporal Aggregation of the ARIMA Process
519
Is the above phenomenon expected? To answer this question, let us derive the invertibfe autoconelations &)(o) and p r ) ( l ) of the limiting aggregate model
(1 - B)zT = o [ ~ ) ( B ) A ~ .
Take nz = 2 and consider the Iimit of mk as k + 03. For m = 2 and d = 1, (20.1.7) gives
y,,(k) = (1 ~ ) ~ ~ , ( 22),
k and (20.1.8) and (20.1.9) imply that
Hence, by (20.1.16),
The characteristic equation, det(Ai(1) - AI) = 0, is
and the eigenvalues are easily seen to be 2 and 8. Thus, A,

= 8 and its corresponding
eigenvector is the solution of the system of equations, ( ~ 4 ( 1 )- 8Z)x = 0, i.e.,
Because rank (AI(1) - 81)

an associated eigenvector is
=.
1, there is one free variable, and it can easily be shown that
Thus, the autocorrefation structure of the Limiting aggregate model becomes
Taus, the result shown in Table 20.3 is expected from the limiting behavior of temporal
aggregates.
This result may not be evident purely based on empirical values. For example, with 500
observations of 6,Table 20.2 clearly implies an IMA(1, 1)model for 2,. The result with nr = 6, as
shown in Table 203, also implies an IMA(I, 1) model for the aggregates &. When m = 12, the
520
standard deviation of iU(k)is approximately equal to -2; hence, as shown in Table 20.3, the sample autocorrelation for this case indicates a.white noise phenomenon, Even though the value
of i w ( l )is -.41 for $v, = (1 - B)z,, however, the value of i U ( l )is .34 for Z& = (1 - B)ZT
when nl = 6 and becomes .25 when nr = 12. This reduction in fiu(l) is a direct result of temporal
aggregation.
In summary, temporal aggregation may complicate model structure. A simple AR process
becomes a mixed ARMA model upon aggregation. As the order of aggregation becomes large,
however, it also tends to simplify the model fonn.
20.2 The Effects of Aggregation on Forecasting and

20.2.1 Hilbert Space
A linear vector space L is a nonernpty
set that is closed under vector addition and scalar

multiplication such that for any elements x, y, z in C and any real numbers a and b, the operations satisfy the following properties:
I. The operation of addition satisfies

l.x+y-y+x.
2. (x + y) + 2 = x 3. (y + 2).
3. There exists a unique element 0 in C such that 0 + x = x.
4. For each x in C,there exists a unique - x in C such that x + ( - x ) = 0.
11. The operation of scalar multiplication satisfies
1. a(bx) = (ab)x.
2. lx = x.
m. The operation of addition and scalar multiplication satisfies
1, a ( x + y ) = a x + a y .
2. (a + b)x = ax -t- bx.
An inner product is a real-valued function on a space C denoted by (x, y), such that the
following are true:
1. ( x , ~ =
) @,,TI.
2. ( x t- y, 2 ) = (x, 2 ) + (y, 2).
3. (ax, y) = a(x, y ) for any real number a.
4. (x, x ) 2 0 and (x, x) = 0 if and only if x = 0.
Two elements x and y are said to be orthogonal if (x, y) = 0. From an inner product, we
define
as a norm of x, often denoted by [x[. It satisfies the following conditions:
Then [x - y [ is defined as the distance between x and y and is called the metric of the
space, It is easy to see that the finite linear combination of zero mean random variables is a
linear vector space and E(XY) is an inner product.
20.2 The Effects of Aggregation on Forecasting and Parameter Estimation
52 1
A sequence x,, is said to be a Cauchy sequence if
A Hilbert space, X,is a complete inner product linear vector space in the sense that the
limit of every Cauchy sequence is also an element in the space so that the space is closed.
Suppose that 'If1 and 'If2 are subspaces of a Hilbert space 7.1. Then the sum of H I and 'Hz
is defined as the set
If 'HI and Z2are orthogonal subspaces in the sense that every element x E 'HI is orthogonal
to every element y E 'Hz, then HI 71z is closed and hence is a Hilbert space. In this case,
we will write the sum as
@ ?f2.We have the following useful theorem [for the proof see
Halmos [1951]f:
Projection Theorem If ?
islo
a subspace of a Hilbert space X,then 71 = 7fo @ H & , where
7-ld is rhe orthogonal complement of 540.
Every element x 6 7-l is uniquely expressed as
where p(x) E 7&, [x - ptxfl E Xd and Lx - p(x) [ = rnin,,% [x - y [. The point p(x) is
called the orthogonal projection of x on ?lo, and [x - p(x)] is the residual of x from 'Ho.
20.2.2 The Application of Hilbert Space in Forecasting

A linear manifold is a nonempty subset of a Hilbert space such that if x and y are in the
subset, then ax
by is also in the subset for any real numbers a and b. Let z, be a zero
mean stationary process. Under the inner product (I,, x,) = E(z,x,), it is known that the
closed linear manifold H,denoted by Ff = L(z,:t = 0,*l, k2, . f , is a Hilbert space
(see Parzen f1961al Anderson [1971, Section 7.61, or Koopmans 11971, Section 7.41).
E(zt - xJ2 is the distance norm between the elements z, and x, in the space. In time series
analysis, we are interested in forecasting ZN + 1, for some 1 2 1, based on the knowledge of
past history (z, : t 5 N ) . Consider the closed linear manifold L{z,:t 5 N j . It is clear that
C{zt:t 5 N } is a subspace of 71. Thus, finding the best linear forecast of z ~ +which
~ , is an
element in H
' , based on its past history, is equivalent to finding the element ? N ( I ) in
LIZ,:r 5 N ) such that the mean squared error
..
By the above Projection Theorem, the best linear forecast of Z N + ~is simply its orthogonal projection on the subspace L(z,:t 5 N } . Under the minimum mean square error criterion deduced from the inner product E(z+,), this orthogonal projection is the same as the
522
conditional expectation E [ Z ~ + ~5 ~NZ] . ~Also,

: ~ by the same theorem, the forecast error
(z~+
- ~&(l)f is in the space C L { z t:t 3 N ) and hence is orthogonal to the subspace
C { z , : ts N j .
20.2.3 The Effect of Temporal Aggregation on Forecasting

Suppose that we are interested in forecasting a future aggregate ZM1 at time N. If both
nonaggregate and aggregate series are available, then this forecast can be derived from tbe
nonaggregate model of z, or from the aggregate model of ZT. It is of interest to examine the
relative efficiency between the two forecasts. Let
7&
= C ( z t : t = 0, f l , f2,., .)
be the closed linear manifold spanned by the nonaggregate series z, and
be the closed linear manifold spanned by the mth-order aggregate series ZT. Let
be the dosed linear manifold spanned by the past history of the nonaggregate series { z , ~ ,
. . .I, and
ZrnN-1,
be the closed linear manifoId spanned by the past history of the aggregate series {ZN, ZN- . . .).
It is easy to see that X,is a subspace of 'Hd and that H$ is a subspace of a#.
Thus, the best linear forecasts of ZN+[based on the nonaggregate process zl and the aggreg$e s e d a Q are simply
the projections of ZNfl on the two subspaces
and 'FI,P.Thus, letting Zi(1) and ZN(i) be the
best linear forecasts of ZN+h E 2 1, based on the nonaggregate process z, and the aggregate
process &,respectively, we have
Thus, forecasting from an aggregate model is in general less efficient than forecasting from a
nonaggregate model.
Suppose that the nonaggregate series z, follows a general multiplicative seasonal ARIMA
model in (20.1.32) and hence the aggregate series ZI. follows the model in (20.1.33). The
20.2
The Effects of Agsregation on Forecasting and Parameter Estimation
523
variances of the forecast errors based on the two models can be explicitly calculated. Specifically, the variance of the foiwasterror based on Lhe aggregate model is given by
where !Po = 1 and the qj's can be calculated from the aggregate model (20.1.33) using the
method discussed in Section 5.2. To calculate the variance of the forecast error from the
nonaggregate model, we first note that
where
$ro =
1 and the *'s are obtained similarly but based on the nonaggregate model
(20.1.32). Let
It was shown in Wei (1978b) that the variance of the forecast error based on the nonaggregate
model is
where
m- 1
Caj
2
Gj-ii=O
The relative efficiency of forecasting future aggregates using the aggregate model can
be measured by the variance ratio
where m is used to indicate the order of aggregation. Because

and &,(I) are obviously
unbiased forecasts for
(20.2.5) implies that 0 5 l(m, I ) 5 1. In fact, it was shown by
Wei (1978b) that ((m,1) has the following properties:
1. 0 5 c(m, I) 5 1 for all m and I.

2. [ ( l ) = hm
J,
,
(n~,
I)= 1 for all I, if d = 0.
3. [ ( I ) << I, and Y(1) -+ 1 only when 1 + 07,if d > 0.
4. b(nl, 1) = 1for all m and 1 if p = d q = 0 and s = mS for some integer S.
524
h summary, the above results imply that insofar as forecasting the future aggregates is concerned, the loss in efficiency through aggregation can be substantial if the nonseasonal component of the model is nonstationary. The loss in efficiency is less serious for long-termforecasting,
particularly when the nonseasonal component is stationary. There is no loss in efficiency due to
aggregation if the basic model is a purely seasonal process, and the seasonal periods s and S for
the basic and aggregate models are related by s = 111s
where m is the order of aggregation.
20.2.4 Information Loss Due to Aggregation in Parameter Estimation

For a given time series model, let r ) be the k x 1 vector of all the k parameters. It often
happens that over the relevant region of the parameter space, the fog-likelihood function
is approximately quadratic in the elements of r) so that
where the partial derivatives
are constant. For large samples, the variance-covariance matrix

likelihood estimator ;j is given by
V(4) for the maximum
where I(q) = -E[lii] is the information matrix for the parameter q. Using (7.2.10), we have
Hence,
and
where we use that, in the quadratic approximation (20.2.10), the partial derivatives lii are
constant over the region of approximation.
20.2 The Effects of Aggregation on Forecasting and Parameter Estimation
525
Let 6 and 5 be the maximum likelihood estimators of q based on the nonaggregate

model and the aggregate model, respectively. Then
and
where I d ( q ) and I,(?) are the corresponding information matrices based on the nonaggregate
and the aggregate models, respectively. Define
7(nt) =
det
V(6)
- -det V ( G )
Z=-
1-
det Ia(rl)
det Id(?) '
We can use ~ ( m to
) measure the information loss in estimation due to aggregation.
Unfortunately, the relationship between the parameters in the aggregate and nonaggregate models is extremely complicated; hence, the derivation of ~ ( mfor
) a general model
is difficult. As shown in Section 20.1.4, however, if the nonaggregate model of z, is a
purely seasonal model
with the a, being i.i.d. N(0, a:), then the aggregate model for the mth-order aggregate ZT
with s = nzS for some integer S becomes
where the AT's are i.i.d. N(0, mu:). It follows that
and hence
To see the implication of (20.2.19),assume that P = 1, Q = 0, and the nonaggregate series is a

2
5
monthly series with s -- 12. We have ~ ( 2 )= ~ ( 3 =
) 3,
~ ( 6=) 5,
and ~ ( 1 2 )= $. Thus,
when estimating the parameters in the underlying model, temporal aggregation leads to a
tremendous loss in information.In fact, r(m) in (20.2.19)is an increasing function ofm and the
number of parameters in the model. The higher the order of aggregation and the more parameters in the model, the more serious is the information loss in estimation.
a,
526

In the previous sections, we discussed temporal aggregation of a flow variable. We now
study the phenomenon of a stock variable. The value of a stock variable such as the price of
a given commodity exists at every time point. In practice, however, it is rarely the ease that
every possible value of a stock variable is observed and recorded. Instead, the observed
series is often only a subseries obtained by a scheme of systematic sampling. For example,
in the stock market, the often-reported series is the daily closing price. In quality control,
only every lath item is inspected. We examine the relationship between the underlying
process zt and the model based on the sampled subseries yn where y~ = z , ~ and rn is the
systematic sampling interval.
With no loss of generality, assume that the underlying process zt follows theARIMA(p, 4 q)
process
where 4pfB) and 8&B) are stationary and invertible AR and MA polynomials, respectively,
and the a,'s are white noise series with mean 0 and variance ~ 2 Let
. al, a2, . . . , and 8, be
the reciprocal of the roots of #p(B). Then, (20.3.1) can be written as
Multiplying both sides of the above equation by
we get
Let t; = flfX1(1 - 6 P m ) ( l - B ~ ) ~ zwhich

,,
is now a moving average MA[(p i- d)nr
( q - p - d)l process. Let
where B = Bm. Note that ByT = BmzmT= Z,,,T-~ = Z,,,(T-~)

= y ~ - ~Hence,
.
I3 is the
backshift operator on y ~ Now,
.
E(CTCT-j) = E(VnlTVmT-mj),
which is nonzero only when
n z j S { ( p d ) m ( q - p. - d ) } or j 5 [ ( p + d )
follows an ARIMA(p, d, r) process
+ +
527
+ [q - p - d)/nl]. Thus, the series y~
where r 5 [ { p d ) (q - p - d)/nz], the erYsare white noise with mean zero and variance
ma,the a:s are functions of +;s, and the Pj's and the
are functions of +is, ej's, and &.
W e n d = 0, (20.3.2) reduces to a result obtained by Brewer (1973). That is, the subseries
systematically sampled from a stationary ARMA(p, q) process becomes an W ( p , r )
process with r 5 [ p (q - p)/mj. To study the limiting behavior of the sampled subseries, we
consider first the case in which the underlying process zt follows an integrated moving average
IMA(d, q) process
G-a
Let ~ v =
, (1
-~
) ~ The
z , . autocovariance generating function, g,(B), of jv, is given by
where yw(j] is the jth autocovariance of the IV, series. Let
We obtain the autocovariance generating function of v, as
where
(:)
= ,z!/(r!(n -
r)l).Now let
=
Then,
where
(:)
= 0 for n
< I-.
V , ~ T=
(2
- B m ) d ~ m T(1
B)dyl,
528
C W T E R 20 Aggregation and Systematic Sampling in Time Series
It is easy to show that for ~ t >

t q
I:...,.
v f2
.,d-n-l
10
for it
d.
Thus, as m increases, the subseries yr approaches an IMA(d, d - 1) process.

Because an ARIMA(p, d, q) process can be approximated with any desired accuracy
by an IMA process, the above limiting result holds also for any homogeneous autoregressive integrated moving average process, As expected, when d = 0, the limiting model from
a stationary process becomes white noise.
In summary, a subseries obtained by systematically taking every nith item from an
ARTMA(p, d, q) process follows an ARIMA(p, d, r) model where r is less than or equal
to the integer part of ( ( p d ) (q - p - d)/nt).When rn becomes large, it approaches
an IMA(d, d - 1) process. In particular, the sampled subseries from an ARIMA(p, 1, q)
process approaches a simple random walk model. The random walk model has been widely
used to describe the behavior of stock prices. It would be interesting to investigate whether
the random walk model is indeed the underlying model for the stock prices or simply a consequence of systematic sampling.
Systematic sampling of a stock variable is an important special case of a more general
problem known as missing observations in the literature, The results presented in this
section are taken from Wei (1981) and mainly deal with the effect of systematic sampling
on model structure. Other references on this topic include Quenouille (1958), Brewer
(19731, Werner (1982), and Weiss (1984).Once the resulting model of a sampled subseries
is known, the effects of systematic sampling on parameter estimation and forecasting can
be examined analogously to the effects of temporal aggregation discussed in Section 20.2;
we do not repeat them here. For reference, we refer readers to Zellner (1966) and Jones
(1980) for the effect on parameter estimation of missing observations from ARMA models,
and to Wei and Tiao (1975) and Liitkepohl (1987) for the effect of systematic sampling
on forecasting.
+ +
20.4 The Effects of Systematic Sampling and Temporal

Aggregation on Causality
20.4.1 Decomposition of Linear Relationship between Two Time Series
The study of causal relationships among a set of time series variables has been one of the most
important problems in the literature. In an interesting paper, Geweke (1982) derived a decomposition of linegr relationships behveen two time series variables xt and y,. By denoting FX,,as
the linear relationship between x, and y,, F, ,as the linear causality from xt to y,, Fy +,, as the
linear causality horn y, to x,,and F, . y as the instantaneous linear causality between x, and y,, he
showed that F,,, = F,,, + F,,,
F,.,.
20.4
The Effects of Systematic Sampling and Temporal Aggregation on Causality
529
Consider a bivariate time series x, and y, with the invertible moving average representation
where [a,, b,]' is a bivariate white noise series with mean 0 and variance-covariance matrix
2=
["a
L+ub
u$].
V b
Now, let us consider the linear projections of y, on the following different subspaces:
1. The linear projection of y, on L{yi:j < t ) :
where i d l , is a white noise series with mean 0 and variance af.Note that the marginal
series y, derived from (20,4.1) is
for some a(%) such that z61t is white noise. The projection (20.4.3) is obtained by inverting
(20.4.4) into an autoregressive form,
2. The linear projection of y, on L{ { y j :j < t ) U { x j :j < t ) ) :
where 1 1 is~ a~ white noise series with mean 0 and variance a$.By inverting (20.4.1) into
an autoregressive representation
where
the representation (20.4.5) is obtained from the first component of the joint model in
(20.4.6). Hence, uzt = a, and a$ = IT:.
530
CHAPTER 20 Aggregation and Systematic Sampling in Erne Series
3. The linear projection ofy, on L { ( y j : j < t } U { x j :j r t ) ) :
where ug, is a white noise series with mean 0 and variance o$.By premultipfying (20.4.6)
by the matrix, which diagonalizes the variance-covariancematrix given in (20.4.2),that is,
by premultiplying the matrix
the projection (20.4.7) is obtained from the first component of the resulting system.
Hence, u g , = at - (uab/ug)bp
4. Theprojectionofy,onL{{yj:j < f ] U { x j : - - m < j < w:,)):
where u4, is a white noise series with mean 0 and variance a$.Let
be the cross-covariance generating function of [y,, x,]' given in (20.4.1). In terms of its
linear projection on the space (xi : -a 4j 4 a},y, can be written as
where r t j , is a white noise series with mean 0 and variance ui,and
The projection (20.4.8)is obtained by multiplying (20.4.9) by $-' (B). Define
20.4
The Effects of Systematic Sampling and Temporal Aggregation on CausaIity
53 1
It can be easity seen that
20.4.2 An Illustrative Underlying Model

For the purpose of illustration and discussion, suppose that on some theoretical grounds the
output variable y, is related to the input variable x, in terms of the time unit t by
where a, and bt are zero mean white noise processes with variances cr; and erg, respectively,
and they are independent of each other. The model can be written in the invertible bivariate
moving average form
where [a,, b,]' is white noise with mean 0 and variance-covariance matrix
Using (20.4.4), we have
where (Y is obtained by solving the equation
subject to -1 < a < 1, which gives
The one-step ahead forecast error variances for the other projections can be easily shown to
be crj = cr? = u$ = a:. Hence, Fy+x = FV = 0 and
532
C W T E R 20 Aggregation and Systematic Sampling in Time Series
TMLE 20.4 Linear causality for the

underlying model in Equation (20.4.12).
That is, the linear relationship between x and y as represented in (20.4.12) contains only the
one-sided linear causality from x, toy,. For numericat illustration, letting v = 1, crz/ui = 1,
we obtain Table 20,4showing causality for the underlying model.
20.4.3 The Effects of Systematic Sampling and Temporal

Aggregation on Causality
The Effect of Systematic Sampling on Causality Suppose that both y, and x, in
(20.4.12)are stock variables and that only the subseries YT and XTare obtained, where YT = ymT
and XT = xm=, and m is ihe systematic sampling interval. It can be easily shown hat for the
model given in (20.4.12), the corresponding model for (YnXTf is
where AT and Erare two independent white noise series with mean 0 and variances
and
respectively. The corresponding bivariate moving average representation is
For v = 1, o:/u$ = I, and rn = 3, Table 20.5 gives the measures showing the effect of systematic sampling on causality.
Thus, the linear relationship between XT and YT still contains only the one-sided linear
causality from XT to YT.Systematic sampling preserves the direction of the causal relationship among the variables. It weakens the relationship as shown by the magnitude of the coefficient of Xr- given in (20.4.14), however.
20.4 The Effects of Systematic Sampling and Temporal Aggregation on Causality
533
TABLE 20.5 Effect of systematic

sampling on causality.
The Effect of Temporal Aggregation on Causality Now suppose that both y, and x,
in (20.4.12) are flow variables and the observed series are mth-order aggregates
and
For the given one-sided causal model in (20.4.12) or (20.4.13),the corresponding model, in terms
of a moving average representation, for ZT = [Yz XTl1,
was shown in llao and Wei (1976) to be
where q = +m, CT is a bivariate white noise process with mean 0 and variance-covariance
matrix X, and H and 2 are related such that
with S =
$)2(1
- (P2) and the 2 X 2 symmetric matrix G(B) having elements
534
TABLE 20.6 Effect of temporal aggregation

on causality.
and
= 1, and nl = 3, we obtain the linear relationship and causafity

For v = 1, CT:/(T~
between XT and YT upon temporal aggregation for 4 = 0 and # = .5 as shown in
Table 20.6,
Thus, as discussed in Tiao and Wei (19761, temporal aggregation turns the one-sided
causal relationship from x, to y, into a pseudo two-sided feedback system. In fact,
from Table 20.6 and Table 20.4, it is clear that upon aggregation the instantaneous linear
causality between XT and YT becomes a dominant force in the linear relationship
between XT and YT. This result is true even for the case where the input is white noise,
i.e., when 4 = 0.
As pointed out by Wei (1982), the magnitude of the measures F, , F,,, F,
and F,,
cannot be used for comparison unless the time units used are the same. For example, for
the model given in (20.4.12), it was shown in Table 20.4 that F*,
In (2.13) for 4 = .5.
The corresponding model (20.4.14) based on sampled variables YT and XT for nl = 3 gives
F,, = ln(2.33) for = .5 as shown in Table 20.5. The latter magnitude of FPy is larger. The
causal effect rom XT to YT in the fatter case, however, is easily seen fiom (20.4.14) to be
weaker because 141< 1.
20.5 The Effects of Aggregation on Testing for Linearity

and Normality
In this section, we study the effects of the use of time series aggregates in performing linearity and normality tests.
20.5.1 Testing for Linearity and Normality

One widely used linearity and normality test is the bispectral test. As described in
Section 19.2.2, this test was proposed by Subba Rao and Gabr (1980, 1984). It was later
modified by Hinich (1982). In this discussion, we use Hinich's modified bispectral test.
20.5 The Effects of Aggregation on Testing for Linearity and Normality
535
Let 2, be a third-order stationary process. Recall from Section 19.2.2 that the bispectral
density function is
where C(kl, k2) is the third-order cumulant of Z,. The principal domain of f ( w l , u 2 ) ,
deduced from the properties of C ( k l , k2) is the triangular set 0 = {O 5 olI V , oz I ol,
2u1 o25 2w), and it is sufficient to consider only the frequencies in this region.
To estimate the bispectral density function based on a third-order stationary time series
Z1, . . , Z,, we let wh = 2~rIt/11for h = 0, 1 ,
, 11 and for each pair of integers j and s
define the set D = {O < j 5 11/2,O < s 5 j, 2 j s 5 n ) . Define also the lattice of points
L = ((2u - 1 ) ~ / 2(2v
, - I)M/2 : v = 1 , 2 , , , 1' and rc I nl(2M) -v
in the set D.
The estimator of the bispectral density function is
...
..
+ i)
where
..
is the finite Fourier transform of Zl, , ,Z, 2 = Z:= l~,/~z, and * denotes complex conjugate.
Equation (20.5.2) shows that j ( q , w2) is constructed by averaging the F(j, s) in a square of
M~ points with the centers of the squares being defined by the lattice of points L. The estimator
f(mI, w2)is consistent if M = nC is rounded to the nearest integer with < c < 1 ,
Let
where 8, = 2 ~ ( 2 x- 1)M/(2n), f ( w ) denotes the estimator of the spectral density function

constructed by averaging M adjacent periodogram ordinates, and R,,, is the number of ( j , s)
in the square that are in D but not on the boundaries j = s or (2j s) = 11 plus twice the
n u ~ b e on
r these boundaries. If the square is in D,then R , , = M2. Hinich (1982) shotvs that
2 1 ~ , , ~is1approximately
~
distributed.as x2(2, hKv),i.e., as a noncentral chi-square variable
with two degrea of freedom and noncentrality parameter
-+
536
The bispectral test is based on the variables 2[Gt,"l2*Because these variables are independent,
the statistic
is approximately x 2 ( 2 ~A), for large n, where h

in L, is approximately equal to ti2/(12@).
~I,,YELAU,vand Q, the number of
(14,
v)
Test of Linearity A zero mean stationary linear process zt can be written in the form
where a, is a sequence of independent and identically distributed random variables with

w
zero mean and variance Var(a,) = r+i and ~(a:) = p3,. Here, $(x) = Zj,-$jxj,
00
Zj=-lqjdljl
< 00, and B is the backshift operator, Bz, = zt-,. From Sections 12.2.2 and
19.2.1, we have
and
f (o)=
:I*(e-")I2.
2,
ff
We see from equations (20.5.7) and (20.5.8) that for a linear process
parameter (20.5.4) is constant for all squares in D, i.e.,
z,, the noncentrality
where we note that R,, = hf2 and M = nc. Because 21k,v12 is independent and
= 1 + h,,,/2, the parameter All can be consistently estimated by
E ( I ~ , v12)
Consequently, the distribution
x2(2, io)
converges to a x2 (2, AO) as n+a.
20.5 The Effects of Aggregation on Testing for Linearity and Normality
537
Thus, under the null hypothesis of linearity, the noncentrality parameter of the asymptotic
distribution of the 21k3,12is constant for alI u, v in L and squares wholly in D. On the other
hand, if the null hypothesis is false, then the noncentrafity parameter will be different for different values of Ir and v. As a result, 'the sample dispersion of the 2/C,.l2 wilf be larger than that
expected under the null hypothesis. Ashley et al. (1986) proposed using the 80% quantile of the
2(tu,.I2 to measure their dispersion. The sample 80% quantile 2.8 has an asymptotic distribution
N(&,8, ug)with ug = .8(1 - .8)/[&[g((.g)]2, where t . 8 is the 80% quantile of the ,y2(2, io)
distribution with density function g and QD denotes the number of squares wholly in D.Linearity is rejected
is significantly larger than (&
ifl.*
Test of Normality To test a time series for normality, we first note that in (20.5.6), if a,
is Gaussian, then p30= 0. Thus, its bispectral density function f(ol,
w2) is identically zero
for all oland wz. Thus, testing a linear process for normality can be achieved by checking
whether its bispectral density function is zero at d l frequencies.
Under the null hypothesis, the statistic defined in (20.5.5) is approximately X 2 ( 2 ~0),
where a is the
for large n, i.e., a central chi-square and the hypothesis is rejected if >
significance level of the test. Rejection of the null hypothesis implies that the bispectral density function of 2, is not identically zero. Thus, Z, cannot be a linear Gaussian process.
X2,
20.5.2 The Effects of Temporal Aggregation on Testing

for Linearity and Normality
+. - +
Let ZT be the nonoverjapping aggregates of z,, i.e., ZT = (1 + B

Bm-')zmn where T
is the aggregate time unit and rn is the order of aggregation. The time series ZTis the aggregate time series, and z, is the unobserved nonaggregate time series.
For the given stationary linear Gaussian process z, in (20.5.61, it is known from Tiao and
Wei (1976) and Wei (1978b) that ZT is also a stationary linear Gaussian process with the
representation
where ET is a sequence of i.i.d. random variables with zero mean and variance Var(eT) = a:
and E(E$) = p p Here, * ( x ) = 2Fo*j.~j,
Z ~ o ~<' P
W ,~and B is the backshift operator,
3% = &--I. The parameters qj and uz are functions of the parameters
and L T ~ of the
nonaggregate process ia (20.5.6). Therefore, the tests based on the Z7. series follow also the
same lines as for the z, series.
Some Simple Nonlinear Time Series Models As introduced in Section 19.2.2, many
nonlinear time series models have been proposed in the literature. To study the effects of aggregation on the linearity test, we will consider the folIowing simple models:
1, BiIinear BL (I, 0, 1, 1) model,
538
2. NLAR(1) model,
z,
( , 0 0 4 ~ , --t
~ a,)(.55zt-1) 4- a,.
3. NLMA(2) model,
4. Threshold TAR(2; 2,3; 3) model,
In the above models, a, is a sequence of i.i.d, standard normal variables.
Aggregation Effects on the Test of Linearity To analyze the effects of aggregation

on the test of linearity, Teles and Wei (2000) reported a simulation experiment in which 1000
time series of 12,000 observations each were generated from the above nonlinear models
and were temporally aggregated with orders of aggregation ni = 2,3,6, 12.
Recall that the noncentrality parameter A , , is constant far all n, v (for squares wholly
in the principal domain) for linear models; therefore, the test is based on the constancy of
this parameter, Because nonlinear models do not generally share this property, we fist
analyze how aggregation affects the dispersion of this parameter in the lattice in the principal domain. To this end, denoting the (sample) standard deviation of the estimates of the
noncentrality parameters computed from the simulation by SL)and S& for the nonaggregates and the aggregates, respectively, we show the values of ~ ~ ~ l ( 2 n Tin-Table
l ) 20,7
and note that SDA= SD when m = I. This ratio measures the impact of aggregation while
taking into account the decrease in sample size. We use the value c = 0.55 for the smoothing parameter.
The reslllts dearly show that the dispersion of the values of the noncentrality parameters decreases as a consequence of aggregation and that this effect generally increases
with the order of aggregation. Thus, A,, tends to be constant as m increases. The effect of
aggregation on the test statistic is severe even for low values of m. The reduction of the
TABLE 20.7 Aggregation effects on the dispersion of the noncentrality parameter.

Order of aggregation
Model
12
BL
NLAR
NLMA
TAR
18.020
3.1 62
1.626
3.306
8.692
2.606
0.734
1.601
4.793
2.249
0.577
0.990
4.835
2.008
0.556
0.988
2.708
1.749
0.563
0.863
2.144
1.623
0.569
0.858
1.593
1.409
0.568
0.899
20.5
The Effects of Aggregation on Testing for Linearity and Normality
539
TABLE 20.8 Aggregation effects on the power of the bispectral test.

Model
BL
NLAR
0.990
0.657
0.747
0.798
0.281
0.456
0.005
0.416
NLMA
TAR
12
0.148
0.249
0.001
0.01 1
0.095
0.110
0.002
0.013
0.027
0.044
0.003
0.000
0.019
0.024
0.000
0.001
0.008
0.013
0.000
0.000
variability of the valtes of the noncentrality parameter implies that the sample dispersion
of the estimators 2l1r,,,l2 decreases as a consequence of aggregation and therefore the
hypothesis of linearity will be less rejected, Teles and Wei (2000) also investigated the
power of the test with the results shown in Table 20.8 (nz = 1 represents the situation of no
aggregation, i.e., the test based on the nonaggregate time series).
We can draw two conclusions:
I. Aggregation causes the power of the test to decrease, and the higher the order of aggregation, the greater the loss.
2. The most striking conclusion is the abrupt loss of power that occurs for aggregate data.
Even for the lowest order of aggregation, i.e., nz = 2, the power is very low, reflecting
the strong decrease in the variability of the noncentrality parameter noted above. Thus,
the test of linearity is very sensitive to aggregation.
A similar study was conducted by Granger and Lee (1989) on some time domain tests. The
effects of temporal aggregation we have obtained are much more severe than the results reported
in their study.
Aggregation Effects on the Test of Normality Under the null hypothesis of Gaussianity, folfo\ving (20.5.5), we denote. the test statistic based on the aggregate series as &, which is
asymptotically X 2 ( 2 ~ A0)
, for large n~ = rdm, where QA F;. n i / ( 1 2 ~ 3 =
) rn2{'-') n / ( 1 2 ~ * )
w ln2Cc-')Q and MA = n i . Because m 2 2 and < c < 1, it implies that QA< Q. The null hypothesis is rejected for large values of iA.
Recall that if the process z, is non-Gaussian, t_he distribution of the test statistic S is
,
asymptotically X 2 (2Q, A). Similarly, the statistic SA will be asymptotically x 2 ( 2 ~ AA*).
Thus, because QA < Q, aggregation leads to a decrease in the degrees of freedom of the
asymptotic distribution of the test statistic as a consequence of the smaller number of observations in the aggregate time series. More important, it causes the noncentrality parameter to
change. We now analyze how this parameter is affected.
For a h e a r non-Gaussian process z,, we know that the noncenttality parameter of the
asymptotic distribution of 21q,,v)2 is constant as shown in expression (20.5.9), i.e.,
A U,Y = 2nk-I pjo/u,6= ho for all u, v and for all squares in D, where p20 5t 0.Consequently,
A =: E,.,Xo = QAo, assuming for simplicity that all squares Ee entirely in in.
540
Because ZTis also a linear process, the corresponding noncentrality parameter is also
constant:
where pjs = E(EQ and oi = Var(q); therefore, hA = QahA,,.Bearing in mind that m 2 2

and < c < 1, the above expression shows that the relationship between hA,and & depends on
the relationships between pjcand E.CJ. and between v, and a,. For example, if & is the mth-order
aggregate series from the series z, that follows a stationary and invertibleAKMA(1, 1) model,
&
with [#I < 1 and 101 < 1, Teles and Wei (2002) show that the relationship between the noncentrality parameters A. and hA,is given by
Because $ < c < 1 and m r 2, expression (20.5.13) shows that the noncentrality parameter of the asymptotic distribution of 21t,,,12 decreases at a rate of m2 as the order of aggregation increases. As a consequence of the reduction of A&, the noncenkality parameter of the
distribution of the test statistic iA
also decreases, Thus, AA < A, and the value of hA decreases at the rate of the square of the
aggregation order. The decrease of the noncentrality parameter implies that temporal aggregation brings the approximate distribution of the test statistic closer to that valid under the null
hypothesis, i.e., a ceneal chi-square, for which the noncentrality parameter is zero. It also
means that aggregation induces normality. Consequently, the null hypothesis will be less
rejected when tested with the aggregate time series than with the basic series. The higher the
order of aggregation, the stronger this effect will be, i.e., fewer rejections will happen.

To test for a unit root, we consider the AR(1) model
and test the null hypothesis Ho : 4 = 1 against the alternative HI : # < 1, where a, is assumed to
be a sequence of independent and identically distributed variables with zero mean and variance
a
:
, and Q I= 0.
The Effects of Aggregation on Testing for a Unit Root
20.6
Recall from Chapter 9.that the test statistic is n($

as the nomalized unit root estimator, where
541
- I), which is also commonly known
The limiting distribution of this statistic under the null hypothesis is given by
where W(u) represents the standard Wiener process. The Studentized statistic of
conventional regression t statistic, is, under the null hypothesis,
4, i.e., its
where S; is the estimated standard deviation of $ and 8: is the OLS estimator of o:,i.e.,
&;
"
-2
1
11
A?
"
2
(z, -2
1
12
,=2
&zt-*)2.
The limiting distribution of Z$ under the null hypothesis is given by
Because of the process of data collection or the practice of researchers, time series are
frequently obtained through temporal aggregation. For example, the series commonly used
in analysis and modeling are quarterly or annual totals. As a result, the series used in testing
for a unit root are often time series aggregates. It is important to kno-iv the consequences of
unit root testing when nonaggregate series are not available and aggregate data are used.
20.6.1 The Model of Aggregate Series

Under the nuU hypothesis Ho : # = 1, model (20.6.1) becomes a random walk, (1 - B)z, = a,.
The corresponding model for the aggregate series is shown by Teles and Wei (1999) as
542
CHAPTER 20 Aggregation and Systematic Sampling in %me Series
where the s~ are independent and identically distributed variables with zero mean and variance uz and the parameters O and gz are determined as follows:
1. Ifnz = 1, then@ = 0 a n d u 2 =
2. If ni 2 2, then
(~2,
These results imply that the aggregate series folloivs an ARIMA(0, I , 1) model, which
means that the unit root remains after temporal aggregation, i s , , the aggregate time series is
still nonstationary. Moreover, as shown by (20.6.71,the moving average parmeter @ of the
aggregate model is negative and it depends exclusively on the order of aggregation m. It is
easy to see from that O is a decreasing function of ni. Because O < 0, we conclude that,
when IV increases, O increases in absolute value.
20.6.2 The Effects of Aggregation on the Distribution

of the Test Statistics
To examine whether the aggregate series ZTcontains a unit root, from the results (20.6.6) and
(20.6.7),we should test the null hypothesis Ho: @ = 1 against the alternative HI:@ < 1 from
the model
where lTfo1lows the MA(1) model YT = ET - @ E ~ - ~ with

,
the ET being i.i.d. with zero mean
and variance u:. The OLS estimator of Q, is
and, because under the null hypothesis & =
.zr -
O ET-I,we have
Therefore, we obtain the following expression for the normalized unit root estimator:
543
The Studentized statistic ,Of the OLS estimator 6 for (20.6.6) is, under the null hypothesis,
where we applied (20.6.10).

The limiting distributions of the test statistics under the null hypothesis are again shown
by TeIes and Wei (1999) as
Expressions (20.6.12) and (20.6.13) show that the limiting distributions of the test statistics for the aggregate time series depend on the order of aggregation m. Because nt 2 2, the
distributions of the aggregate statistics are shifted to the right relatively to those of the basic
statistics given in (20.6.3) and (20.6.51, and the shift increases with the order of aggregation.
20.6.3 The Effects of Aggregation on the Significance Level

and the Power of the Test
The shifi to the right shown by the distributions of the test statistics implies that the percentiles tabulated by Dickey (1976, pp. 52-53) and by Fuller (1996, pp. 641-642) may lead
to improper conclusions of the test when aggregate series are used to test for a unit root. In
this section, we investigate more explicitly the impact of the use of aggregate time series on
the test in terms of its significance level and power.
We start by studying the effects on the empirical significance levels of the test. In fact,
because the rejection region is on the left tail of the distribution, i.e., the null hypothesis of
a unit root is rejected when the values of the test statistics are smaller than the appropriate
critical points, the distribution shift to the right caused by aggregation will lead to fewer
rejections than expected under the n d l hypothesis. Therefore, the empirical significance
levels will be lower than the nominal level.
To assess this effect, we conducted a simulation experiment as follows. In the finite
sample case, 50,000 time series of n
600 observations each were generated from a
random walk model z, = z,-1
a,, where zl = 0 and a, is a sequence of independent and
identically distributed N(0, 1) random variables, The generated time series were aggregated
with orders of aggregation nt = 2 , 3 , 4 , 6 , 8 , 12. Concerning the Limiting case, 100,000 values of each test statistic were generated from the appropriate limiting distributions, as given
in equations (20.6.3) and (20.6.5) for the basic series and in equations (20.6.12) and
(20.6.13) for the aggregate case with Tn = 2,3,4,6,8, 12, The observed significance levels
544
TABLE 20.9 Aggregation effects on the empirical significance level of the test.
3
4
12
0.0153
0.0154
0.01 50
0.0150
0.0157
0.0154
Finite sample
NURE
Stdt. Stat.
0.0509
0.0500
0.021 1
0,0213
0.0174
0.0173
0.0165
0.0161
Limiting distribution
based on the critical points provided by Dickey (1976, pp, 52-53) and by Fuller (1996,
pp. 641-642) obtained from the simulation are shown in Table 20.9, where nn = 1 represents the basic case when there is no aggregation. The nominal significance level used is
5%. In the table, NLJRE and Stdt. Stat. represent the normalized unit root estimator and the
Studentized statistic, respectively.
The simulation results clearly show that, for the basic series (m = I), the critical points
provided by Dickey and Fuller are appropriate. In fact, the empirical significance levels are
practically coincident with the nominal level of 5%. The observed significance levels
decrease when the order of aggregation increases, however, and the impact is very strong.
Moreover, we observe that a serious impact occurs even for nt = 2. It is clear that the effect
of aggregation on the test is severe for any order of aggregation.
These conclusions are also supported by several simulation studies like those reported
by Schwert (1989), Pantula (1991), Agiaklogfou and Newbold (1992), and Cheung and Lai
(1995) showing that the Dickey-Fuller test is severely affected by the presence of a moving
average component in the model, even for large sample sizes. For negative values of the
moving average parameter, which is the situation we have, the observed significance levels
of the test in their simulations were also lower than the nominal levels.
The distribution shift of the test statistics also has important implications on the power of the
test. Because the null hypothesis is rejected when the test statistics are lower than the critical
points, the use of the unadjusted percentiles as critical points for the test based on aggregate series
will lead to smaller rejection regions than implied by the signi6cance level adopted for the test.
As a result, there will be fewer rejections of the null hypothesis than expected under the alternative hypothesis of no unit roots, i.e.,the power of the t a t will decrease and will be lower than that
obtained for basic time series. To avoid the problem, the tables of percentiles provided by Dickey
(1975, pp. 52-53) andFuller (1996, pp. 641-642) need to be adjusted when aggregate time series
are used in testing for a unit root, Thus, we constructedTables J and K in the appendix, which are
suitable for thg distributions of the test statistics for the most commonly used orders of aggregation m -- 1,2,3,4,6, and 12, where m = 1 represents no aggregation with values close to those
in Tables F and G except for rounding. To obtain the corrected percentiles, we followed the procedure of Dickey (1976), which is based on Monte Carlo simulations. We adapted the procedure
545
and computed the test statistics for aggregate series. The sample sizes shown in the tables are the
sizes of aggregate series. For example, the 5% percentile of the normalized unit root estimator
based on an aggregate time series of size r t =
~ 100 resulting from an order of aggregation nt = 2
is -5.82. Tables J and K are for models wvithout a constant term. We refer readers to Teles (1999)
for more detailed tables including tables for models with a constant term.
20.6.4 Examples
To illustrate the effects of aggregation on the tests of a unit root, we analyze two examples:
a simulated series and the time series of the U.S, new one-family houses sold.
EXAMPLE 20.4 In this example of a simulated Time Series, we generated a time series
4- a,, where the a, are i,i.d. N(0, 1). This
of 240 observations from the model z, = 0 . 9 5 ~ , - ~
series was aggregated with rn = 3.
First, let us test the basic series z, (240 observations) for a unit root. Without assuming a
known model, we examine its sample autocorrelation function and partial autocorrelation
function, both suggest an AR(1) model. The least squares estimation leads to the following
result [standard error in parenthesis):
To_test the hypothesis of a unit root with this series, the values of the test statistics are
n(# - 1) = 240(0.9603 - 1) = -9.53 and = (0.9603 - 1)/0.0182 = -2.18. From the
tables provided by Dickey (1976, pp. 52-53) and by Fuller (1996, pp. 641-642), for n = 240
and a 5% significance level, the critical point for the normalized unit root estimator is behveen
-8.0 and -7.9, and for the Studentized statistic it is -1.95. The conclusion of the test is to
reject the hypothesis of a unit root, and we conclude that z, is stationary. The result is certainly
consistent with the underlying simulated model.
Let us now consider the aggregate series ZT for m = 3 (80 observations) and use it to test
for a unit root. To be realistic with actual data analysis, however, we will not assume a known
model and will proceed with a standard model building procedure to identify a best model for
the series. The sample autocorrelations and partial autocorrelations (table not shown) suggest
an AR(1) model. An overfitting with an ARhllA(1, 1) model gives a nonsignificant MA term;
hence, we will retain the AR(1) model. The least squares estimation leads to the following
model (standard error in parenthesis):
The values of the test statistics are rtA(6 - 1) = SO(0.205 - 1) = -6.36 and
(0.9205 - 1)/0.0448 = -1.77. Because the test is based on an aggregate time series
with m = 3 and n~ = 80, we will use the adjusted critical points with nn = 3 from Tables J
and K. At a 5% significance level, the critical point for the normalized unit root estimator is
behveen -5.45 and -5.40, and for the Studentized statistic it is -1.60. Consequently, we
reject the null hypothesis of a unit root and conclude that the aggregate series ZT is stationary.
Z&
546
If we derived our conclusion of this test from the unadjusted percentiles given by Dickey
(1976, pp. 52-53) and by Fuller (1996, pp.'641442), then the critical points for n = 80 at a
5% significance level would be between -7.9 and -7.7 for the normalized unit root estimator and - 1,95 for the Studentized statistic. We would then retain the hypothesis of a unit root
and conclude that the series ZT is nonstationary, This wrong conclusion would lead us to
overdifference ZT and therefore distort its analysis.
There have been extensive studies on the aggregation of stationary processes, including
those by Amemiya and Wu (19721, Brewer (1973), Wei (1 978a,b), Weiss (1984), Stram and
Wei (1986b), and Lutkepohl(1987). From these studies, it is known that the aggregate series
of a stationary time series is stationary. Thus, this example shows that using the unadjusted
critical points to test for a unit root with an aggregate series may lead to the conclusion that
a stationary process contains a unit root and to overdifferencing. To avoid this problem, the
adjusted percentiles should be used when the test is based on an aggregate series.
EXAMPLE 20.5 We now consider the time series of the monthly U.S. new one-family
houses sold in the United States from January 1968 through December 1982 (180 observations) listed as W15 in the appendix. The source of the data is the Current Construction
Reports published by the Bureau of the Census, U.S. Department of Commerce. The data
have been seasonally adjusted, i.e., the series has been adjusted by subtracting the corresponding monthly average seasonal values.
Suppose first that we are interested in modeling the quarterly U.S.new one-family
house sales, &-(60 observations). To test & for a unit root, we first note that as a seasonally
adjusted series, it has a zero mean. Next, the sample autocarrelation and partial autocorrelation functions of ZTgiven in the following table lead to the identification of an AR(1) model:
Sample autocorrelations of ZT
Sample partial autocorrefationof ZT
An overfitting with an ARMA(1, 1) model gives the nonsignificant MA term
and we will retain the AR(1) model. The least squares estimation (standard error in parenthesis) gives .
547
The values of the test statistics are nA(@ - 1) = 60(.8915 - 1) = -6.51 and
T&= (3915 - 1)/.059 = -1.84. Because the test is based on an aggregate time series
with m = 3 and IZA = 60, we should use the adjusted critical points from Tables J and K.
At a 5% significance level, the 'critical point for the normalized unit root estimator is
between -5.45 and -5.40, and for the Studentized statistic it is -1.60. Consequently,
we reject the null hypothesis of a unit root, i,e., we conclude that the aggregate series ZT
is stationary.
If we based the conclusion of this test on the unadjusted percentiles given by Dickey
(1976, pp. 52-53) and by Fuller (1996, pp. 641-642), then the critical points for n = 60 at a
5% significance level would be between -7.9 and -7.7 for the normalized unit root estimator and -1.95 for the Studentized statistic. As in Example 20.4, we would then retain the
hypothesis of a unit root and conclude that the series ZT is nonstationary.
We note that the data were collected monthly. In fact, we can consider the month as the
appropriate time unit for the series of the new one-family houses sold because the demand for
new homes depends on the capacity of home buyers to pay for the houses they buy. The overwhelming majority of home buyers rely on mortgages to finance their homes, Therefore, the
decision of buying a home is strongly dependent on mortgage payment plans. In the period
we are studying, most payment plans are monthly, and they reflect the way the demand for
new homes is formed.
Consequentiy, we may take the month as the basic time unit and therefore the monthly
series zt as the basic series. We now test I , for a unit root. Based on its sample autocorrelation
function and partial autocorrelation function (table not shown), the series z, is identified as an
AR(1) model. We then fit an AR(1) model to z, with least squares. The estimation results are
(standard error in parenthesis)
Ton test the hypothesis of a unit root with this series, we compute the test statistics
nf+ - 1) = 180(.9412 - 1) =. -10.58 and 7$ = (.9412 - 1)/.0252 = -2.33. Because z,
is the basic series, we will use the tables provided by Dickey and Fuller, The critical point
for the normalized unit root estimator is between -8.0 and -7.9, and for the Studentized
statistic it is -1,95 for 11 = 180 and a 5% significance level. Therefore, we reject the hypothesis of a unit root and conclude that z, is stationary. Consequently, as in the previous example, concluding that ZT is nonstationary, as implied by the use of the unadjusted critical
points, clearly leads to a wrong conclusion.
20.6.5 General Cases and Concluding Remarks

We analyzed how the use of aggregate t h e series affects testing for a unit root. The presentation is based on the model originally proposed in Dickey and Fuller (1979). For a nonzero
mean process, we can simply compute the mean adjusted series before applying the testing
procedure.
For a general process, we consider the AR (p) approximation of the aggregate series, (1 Q ~ B - . , . - QipBP)&- ET. To test for a unit root, we assume that (B) = q,p,_l(B)(l - B)
548
- ~P - ~has
) roots lying outside the unit circle. Thus,
where%-I (B) = (1 - B - . . - ' ~ p B
testing for a unit root is equivalent to testing-@= 1 in the model
where
= (ZT-j - Zw-l). Following the results in Section 9.4, the normalized unit
root estimator becomes
where h ( B ) = l/rp(B), and Q(B) = (1

t statistic is
- cprB - . . - r P p - l ~ ~ - iand
) , the corresponding
The limiting distributions of these test statistics in (20.6.19) and (20.6.20) are the same as
those in (20.6.12) and (20.6.13). Aggregation causes the distributions shift to the right, leads
to empirical significance levels lower than the nominal level, and significantly reduces the
power of the tests. Similar to the results in Section 9.4, however, when aggregate data are
used, we can use the same adjusted Table J and Table K for the normalized unit root estimator in (20.6.19) and the r statistic in (20.6.20), respectively. For details and examples, see
Teles (1999).
To correct these problems, we constnrcted adjusted tables of critical points suitable for
testing for a unit root based on aggregate time series. Both simulation and empirical examples clearly show that the use of the adjusted tables is necessary for a proper test whenever
aggregate time series are used.
A similar study was reported by Pierse and Snell (1995). Because their study does not
provide the explicit distribution of the test statistic used in testing aggregate series, one
could not examine the underlying causes of the effects as a result of order of aggregation
illustrated in this paper. One may argue that many tests are available for a series with correlated errors, for example, the augmented Dickey-Fuller test, the Phillips (f987a,b) test,
and the Phillips and Perron (1988) test. Although aggregation induces correlation, however,
the induced correlation is often not strong enough to show significance in the process of
empirical model fitting, which is clear from both the simulated and real-life examples in
Section 20.6.4. As a result, the standard Dickey-Fuller test instead of other desirable tests
may still be used to make the inference. The essence of the results in this study is that
although the standard Dickey-Fuller test can still be used, for a proper inference, the
adjusted tables based on the order of aggregation should be used as long as an aggregate
time series is used in the analysis. The Dickey-Fuller test is possibly one of tbe most
widely used tests in time series analysis, so it is hoped that the results presented in this
study will be useful to data analysts.
549

-
Temporal aggregation and systematic sampling pose important problems in time series
analysis. In working with time series data, one must first decide on the observation time unit
to be used in the analysis, If the model for a phenomenon under shrdy is regarded appropriate in terms of a certain time unit f, then the proper inferences about the underlying model
should be drawn from the analysis of data in terms of this time unit. As shown in the preceding sections, improper use of data in some larger time unit could lead to a substantial information loss in parameter estimation and forecasting. More important, the improper use of
aggregate data could distort results in terms of lag structure and causality and hence may
lead to very serious consequences in terms of final policy and decision making.
Most time series procedures have been established under the assumptions such as linearity, normality, and stationarity. Many time series encountered in practice are, however,
often nonlinear, nonnormal, or nonstationary. To check whether an observed series confirms
the assumption, tests have been introduced to test linearity, normality, and nonstationarity.
Because of the process of data collection or the practices of researchers, time series used
in analysis and modeling are frequently obtained through temporal aggregation. As a result,
the series used in testing are often time series aggregates. In this chapter, we study the
effects of the use of aggregate time series in performing various tests. Through the study
of its impacts on sampling distributions of the test statistics, we show that the use of time
series aggregates results in serious Ioss of power of the tests and hence often leads to erroneous conclusions. To avoid missing the nature of the underlying process, special care is
required in interpreting the testing results when time series aggregates are used in the tests.
We presented only some results which are directly related to the materials covered in
this book. References on the effect of temporal aggregation on model structure include
Engle and Liu (1972), Tiao (1972), Brewer (1973), Tiao and Wei (1976), Terasvirta (1977),
Wei (1978b), Harvey (1981), Werner (1982), Engle (1984), Liitkepohl(1984), Niemi (1984),
Weiss (1984), Stram and Wei (1986b), and Wei and Stram (1988). The consequence of temporal aggregation on parameter estimation has been studied by Zellner and Mont-marquette
(1971), Engte and Liu (1972), Tiao and Wei (1976), Wei (1978a,b), Wei and Mehta (1980),
and AhsanulIah and Wei (1984b). The effect of temporal aggregation on forecasting has been
investigated by Amemiya and Wu (1972), Tiao and Wei (1976), Wei (1978b), Abraham
(19821, Ahsanullah and Wei (1984a), and Liitkepohl (1987). The problems on aggregation
and seasonal adjustment are also studied by Lee and Wei (1979, 1980).
A related problem that has not been discussed in this chapter is the contemporaneous
aggregation of time series. It deals with aggregation of several component series in a system.
For example, the total money supply is the aggregate of a demand deposit component and a
currency component. National housing starts is the aggregate of component housing starts in the
northeast, north central, south, and west regions. An interested reader is referred to the work
by Rose (1977), %ao and Guman (1980), Wei and Abraham (1981), Kohn (1982), Liitkepohl
(19841, Engel (1984), and Liitkepohl (19871, among others. The book by Liitkepohl (1987)
deals with temporal aggregation of vector ARMA processes and contains extensive references.
To recover information Ioss, several disaggregation procedures have been introduced.
Due to space limitations, we will not discuss them here and refer interested readers to
Boot, Feibes, and Lisman (1967), Chow and Lin (1971), Cohen, Miiller, and Padberg
(1971), Denton (19711, Stram and Wei (1986a), Al-Osh (1989), Guerrero (19901, Wei and
550
Stram (1990), Chan (1993), Gudmundsson (1999), Hotta and Vasconcellos (1999),
Hodgess and Wei (1996,20OOa,b), and Di Fonzo (2003), among others.
20.1 Consider the nonaggregate model z, = (1 - ,1B - . 2 ~ ~ ) awhere

,,
a, is a zero mean
Gaussian process with a: = 2.
(a) Let ZT = (1 iB + B2)z3T. Find the autocovariance function, yz(k), of ZT for
k = 0, 1 , 2 , and 3, and find the model for ZT.
(b) Let ZT = Z ~ T Find
.
the autocovariance function, yZ(k),of & for k = 0, 1, 2, and
3, and the model for &,
- .9B)z, = a,, where a, is a Gaussian N(0, 1)

white noise process,
(a) Let ZT = (1 B ~ ~ )Find
z the
~ autocovariance
~ .
function, yz(k), of ZT for
k = 0, 1,2, and 3, and the model for ZT.
(b) Let ZT = z 3 ~Find
. the autocovariance function, yZ(k), of &-for k = 0, 1,2, and
3, and the model for ZT.
20.2 Consider the nonaggregate model, (1
+ +
20.3 Consider Series W4 of the monthly unemployed young women between ages 16
and 19 in the United States from January 1961 to August 2002.
(a) Build a monthly model using the data from January 1961 to December 2001,
Forecast the next nine-month data for unemployed young women, and obtain the
corresponding quarterly forecasts for quarters 1,2, and 3 of 2002.
(b) Build a quarterly model for the quarterly unemployed data from the first quarter
of 1961 to the fourth quarter of 2001. Forecast the next three quarters.
(c) Derive the implied quarterly aggregate model of the monthly model obtained in
part (a), and compute forecasts for the next three quarters.
(d) Compare and comment on your findings in parts (a), (b), and (c).
m-1
20.4
(a) Let H,
=:
Bj)x,. Prove that

j=O
(b) In part (a), consider x, = (1 - 9B)at, where a, is a white noise process with
mean 0 and variance IT:. Show that
E(H,Ht-
0=
t = 0,
[m(L - 13)~+ 26]u,2,
( m - {)(I - 19)~u;, 4 = 1, , , , , (m - I),
-8of,
e = rn,
10.
otherwise.
Exercises
55 1
(c> Consider the one-sided causal model,
yr = ax,-l
+ e,,
where x, = (1 - 8B)a,, et and at are independent white noise processes with

mean 0 and variance c ~ ;and a:, respectively. Show that the inh order temporal
aggregation turns the one-sided causal relationship into a two-sided feedback
system,
whereF =
XT is an MA(1) process with the autocovarince generating
function (AGF) given by yx(B) = oi{[m(l - 6 ) 2 291 - 6(B F)}, and
the UTis an error process independent of XT with the AGF given by
a20:{[m(l
el2 + 281 - B(B + F ) -
q 4 [ ( r n - 1 ) + B ] [ ( n t - I) + F ]
+ nta:.
[nt(1 - 0l2 + 26 - 6 ( B + F ) ]
(1 -
20.5 Let ZT be the m* order aggregate of z, where zt = a,, and a, is a sequence of i.i.d.
random variable with mean 0 and variance (~2.
Let A. and hA,be the noncentrality
parameter for z, and ZT,respectively. Show that hA, = h o / d c , where < c < 1.
20.6 (a) Perform the unit root test on Series W4.

( b ) Perform the unit root test on the third-order aggregates of Series W4.
(c) Compare and comment on your findings in parts (a) and (b).
(d) Repeat the above for Series W14.
References
Abraham, 3, (1982). Temporal aggregation and time series, Intematio!ial Statistical Raiew,
50,285-291.
Abraham, B., and Box, G . E. P. (1979). Bayesian analysis of some outlier problems in time series,
Biornetrika, 66,229-236.
Abraham, B., and Chuang, A. (1989). Outlier detection and time series modeling, Tecl~rzotnetrics,
31,241-248.
Abraham, B., and Ledolter, J. (1983). Statistical Method for Forecastittg, fohn Wiley, New York.
Abraham, B., and Ledolter, J. (1984). A note on inverse autocorrelations, Bionletrika, 71, 609-614.
Abraham, B., and Wei, W. W, S. (1984). Inferences about the parameters of a time series model with
changing variance, Metrih, 31,183-194.
Agiakloglou, C., and Newbold, P. (1992), Empirical evidence on Dickey-Fuller-type tests, Joanial of
Ktne Series AnalysysB, 13,471-483.
Ahsanullah, M., and Wei, W. W. S. (1984a). Effects of temporal aggregation on forecasting in an
ARMA(1, 1) process. ASA Proceedings of Bzcsiness and Econoaiic Statistics Sectiott, 297-302.
Ahsanullah, M., and Wei, IV, W. S. (1984b). The effects of time aggregation on the AR(1) process,
Corrrputatio~ralStatistics Qirarterly, 1, No. 4,343-352.
Akaike, H. (1969). Power spectrum estimation through autoregressive model fitting, Ann. Inst. Sfatist.
Math., 21,407-419.
Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle, Proc.
2nd btfernational Sytiipositmr oti Infontzatio~zTheory (33s. B. N . Petrov and F. Csaki), 267-281,
Akademiai Kiado, Budapest.
Akaike, W. (1974a). Matkovian representation of stochastic processes and its application to the analysis of autoregressive moving average processes, Anrr. Inst. Statist. Math., 26,363-387.
Akaike, H . (1974b). A new look at the statistical model identification,IEEE Transactiorts ott Ailto?natic
Control, AC-19,716-723.
Akaike, H. (1976). Canonical correlations analysis of time series and the use of an information criterion, in Systet~tIderttificatiotl: Advances atld Case Studies (Eds. R. Mehra and D. G. M o t i s ) ,
Academic Press, New York and London.
Akaike, H. (1978). A Bayesian analysis of the minimum AIC procedure, k m . Itjst. Statist. Math.,
30A, 9-14,
Akaike, N. (1979). A Bayesian extension of the minimum AIC procedure of autoregressive model
fitting, Biomem.kz, 66,237-242.
Al-Osh, M. (1989). A dynamic linear model approach for disaggregating time series data, Jounlal of
Forecasting, 8,85-96.
Ali, M. M. (1977). Analysis of ARMA models--estimation and prediction, Bio~netrikn,64, 1-11.
Arnemiya, T., and Wu, R. Y. (1972). The effect of aggregation on prediction in the autoregressive
model, J. Anel: Statist Assoc. No. 339,628432.
Anderson, T.W, (1971). The StatisficaIAnalysis of Etne Series, John Wiley, New York.
553
554
References
Anderson, T.FV, (1980). Maximum likelihood estimation for vector autoregressive moving aterage
models, in Directions in Time Series (Eds. D. R. Brillinger and G. C . Tiao), Institute of Mathematical Statistics,49-59.
Andrew, P. E., and Herzberg, A. M. (1985). The Data: A Collectiotl of Problems front Statistics,
Springer-Verlag, Berlin.
Ansley, C. Fa(1979). An algorithm for the exact likelihood of a mixed autoregressive moving average
process, Biornetn'h, 66,59455.
Ansley, C. F., and Newbold, P. (1979a). Multivariate partial autocorrelations,ASA Proceedings of Business at~dEconomicStatistics Section, 349-353.
Ansley, C. F., and Newbold, P. (1979b). On the finite sample distribution of residual autocorrelations in
autoregressive-moving average models, Bionletrika, 66,547-554.
Asllley, R. A,, Patterson, D. M., and Hinich, M. 1. (1986). A diagnostic test for nonlinear serial dependence in time-series fitting errors, Joimtal of 7itne Series Analysis, 7,165-178.
Automatic Forecasting Systems, Inc. (1987). MTS User's Guide, Hatboro, PA.
Bartlett, M. S. (1941). The statistical significance of canonical correlation, Biotnetrika, 32,29-38.
Bartlett, M . S. (19461. On the theoretical specification of sampling properties of autocorrelated time
series, J. Royal Stat. Soc., B8,2741.
Bartlett, M. S. (1947). Multivariate analysis, JournaE of the Royal Statistical Socieq Srpplement, 3 9 ,
176-197.
Bartlett, M. S. (1950). Periodogram analysis and continuous spectra, Biorttetrifka,37, 1-16.
Bartlett, M. S. (1955). Stocltastic Processes, Cambridge University Press, Cambridge.
Bartlett, M. S. (1966). An htroduction to Stochastic Processes with Reference to Merhods aj~dApplication, 2nd ed., Cambridge University Press, London.
Beguin, J. M.,Gourieroux, C., and Monfort, A. (1980). identification of a mixed autoregressivemoving average process: The comer method, fine Series (Ed. 0.D. Anderson), 423-436, NorthHolland, Amsterdam.
Bell, W. R. (1983). A computer program for detecting outliers in time series, ASA Proceedings of
Business artd Econottlic Statistics Section, 634-639.
Bell, W. R., and Hillmer, S. C. (1984). Issues involved with the seasonal adjustment of economic time
series, Journal of Bicsiness and Eco~iomicStatistics, 2,291-320.
Blackman, R. B., and Tukey, J. iV, (1959). The Meamrement of Power Spectnimfrom the Poi~tto f W w
of Con~ntrrrlicationsE~zgirteentig,Dover, New York (Originally published in Bell Systenis Tech.
Joicmal, 37, 185-282 and 485-569[1958]).
Bloomfield, P. (2000). Fourier Analysis of Erne Series: An Introduction, 2nd ed., Wiley Interscience,
New York.
B o b , T. (1976). Four cases of identification of changing systems, in Systent Identijcation: Advances
nnd Cases Strldies (Eds. R. Mehra and D. G. Ladotis]. Academic Press, New York and London.
Bollerslev, T.(1986). Generalized autoregressive conditional heteroskedasticity,Journal of Econottletr i a , 31,307-327.
Boot, J. C. G., Feibes, W,, and Lisman, J. H. C , (1967). Further methods of derivation of quarterly
figures from annual data, Allied Statistics, 1665-67.
Box, G. E. P., and Cox, D. R. (1964). An analysis of transformations,J. Roy Stat. Soc. Ser. B. 26,211-252.
Box, G . E. I?, and Jenkins, G. M. (1976). Time Series Analysis Forecasting a~tdCo~ztrol,2nd ed.,
Holden-Day, San Franscisco.
Box, G. E. P., Jenkins, G. M., and Reinsel, G . C. (1994). Time Series Analysis, Forecasting and Control,
3rd ed., Prentice Hall, Englewood Cliffs, NJ,
Box, G. E. F., and Pierce, D. A. (1970). Distribution of residual autocorselations in autoregressiveintegrated moving average time series models, J. Amer. Statist. Assoc., 65, 1509-1526.
Box, G. E. P., and llao, G. C. (1965). A change in Ievel of a non-stationary time series, Biottietrika,
52.181-192.
References
555
Box, G. E. P., and Tiao, G. C. (1975). Intervention analysis with applications to economic and environmental problems, J. Antes Starist. Assoc., 70,70-79.
Box, G. E. P., and Tiao, G. C. (1977). A canonical analysis of multiple time series, Bio~netrika,
64,355-365.
Brewer, K, B. W. (1973). Some consequences of temporal aggregation and systematic sampling for
ARMA and ARMAX models, Joitn~alof Econorjtetrics, I, 133-154.
Brillinger, D. R. (1975). Rote Series: Data Ar~alysisand Tlteory, Holt, Rinehart and Winston,
New York.
Btillinger, I).R., Guckenheimer, J., Guttorp, P. E., and Oster, G.(1980). Empirical modeling of population time series data: The case of age and density dependent vital rates, Lecfirreson Matkenzatics in the Life Science, (Amer. Math. Soc.) 13,65-90.
Brillinger, D. R., and Rosenblatt, M. (1967). Asymptotic theory of kth order spectra, Spectral Analysis
of Zi~neSeries (Ed. B. Harris), 153-188, John Wiley, New York.
Bryant, P. G., and Smith, M . A. (1995). Practical Data Analysis: Case Studies in Business Sratistics,
Richard hvin Inc., Burr Ridge, Illinois.
Burr,I. W. (1976). Statistical Qllality Coritrol Metl~ods,Marcel Dekker, New York.
Buys-Ballot, C. H. D. (1847).LRo CIraentenfsP2riodiqrres de Te~rtpBrature,
Kernink et Fils, Utrecht.
Campbell, M. J., and Walker, A. M. (1977). A survey of statistical work on the MacKenzie River series
of annual Canadian lynx trappings for the years 1821-1934, and a new analysis, J. Roy. Statist.
Soc., Ser. A, 140,411431; Discussion 448-468.
Chan, W. S . (1989). Sortte Robirst Methods for Etne Series Modeling, unpublished Ph.D. Dissertation,
Temple University.
Chan, W. S. (1990). On tests for no linearity in Hong Kong stock returns, Ho~tgKong Jotrntal of
Btcsiness Martoger?tettt,8, 1-11.
Chan, N. H., and Wei, C. 2.(1988). Limiting distribution of least squares estimates of unstable autoregressive processes, Annals of Statistics, 16,367401.
Chan, W. S., and Tong, H. (1986). On tests for non-linearity in time series, Jo~cntalof Forecnsti~ig,
5,217-228.
Chan,W. S., and Wei, W, '8. S. (1992). A comparison of some estimators of time series autocorrelations. Comnptttational Statistics and Data Altalysis, 14, 149-163.
Chan, W.S. (1993). Disaggregation of annual time series data to quarterly figures: a comparative study,
Jotrnzal of Forecasting, 12.677488.
Chang, I., and Tiao, G. C. (1983). Estimation of time series parameters in the presence of outliers,
Technical Report 8,University of Chicago, Statistics Research Center.
Chang, I., Tiao, G. C., and Chen, C. (1988). Estimation of time series parameters in the presence of
outliers, Tecl~nonternks,
30, No. 2,193-204.
Chen, C., and Liu, L. M. (1991). Forecasting time series with outliers, Journal of Forecmti~tg,
12,13-35.
Chen, C., and t i u , L. M. (1993). Joint estimation of model parameters and outliers effect in time
series. Jozrntal of the Aaerica~rStatistical Association, 88,284-297.
Cheung, Y. W., and Lai, K. S. (1995). Estimating finite sample critical values for unit root tests using
pure random walk processes: a note, Jountal of Etne Series Analysis, 16,493498.
Chow, G. C., and Lin, A. (1971). Best linear unbiased interpolation, distribution and extrapolation of
time series by related series, Review ofEcoilomic Statistics, 53,372-375,
Clarke, D. G . (1976). Econometric measurement of the duration of advertising effect on sales, Jortrnal
of Marketing Reseadt, 13, 345-357.
Cleveland, W. S. (1972). The inverse autocorrelations of a time series and their applications, Teclznontetrics, 14, 277-293.
Cohen, K. J., Miiller, W., and Padberg, M. W. (1971). Autoregressive approaches to disaggregation of
time series data, Applied Stahtics, 20, 119-129.
556
References
Cooley, J. W., and Tbkey, J. IF', (1965). An algorithm, for the machine calculation of complex Fourier
series, Matft. Conzp., 19, 197-301.
Crato, N., and Ray, B. K,(1996). Model selection and forecasting for long-range dependent processes,
Jourtlal of Forecastirzg, 15, 107-125.
Cupingood, L. A., and Wei, \V. W.S. (1986). Seasonal adjustment of time series using one-sided filters,
Jo~mralof Business and Econotnic Statistics, 4,473-484.
Dagum, E. B. (1980). The X- 11 -ARIMA seasonal adjustment method, Catalogue 12-564, Statistics
Canada, Ottawa, Ontario.
Daniell, P. J. (1946). Discussion on symposium on autoconelation in time sedes, J. Roy. Statist. Soc.,
Suppl. 8,88-90.
Davidson, J. E. H., Henry, D. F., Srba, F., and Yea, S. (1978). Econometric modeling of the aggregate
time series relationship between consumers' expenditure and income in the United Kingdom,
Ecotloniic Jountal, 88,661492.
Day, T,E., and Lewis, C. M.(1992). Stock market volatility and the information content of stock index
options, Jorrmal of Eco~zol?retrics,
52,267-287.
Denton, F. (1971). Adjustment of monthly or quarterly series to annud totals: an approach based 'on
quadratic minimization, J. Amer. Statist. Assoc., 66,99-101.
Di Fonzo, T. (2003). Temporal disaggregation of economic time series: towards rt dynamic extension,
Working Paper #I2 of Eurostate, Office for Official Publications of the European Communities,
Luxembourg, 1 4 1 .
Dickey, D. A. (1976). Estirnatio~~
and Hypothesis Testit~gin No~zstatiorralyTirne Series, unpublished
PhD.dissertation, Iowa State University.
Dickey, D. A., Bell, B., and Miller, R. (1986). Unit Roots in time series models: Tests and implications,
Tlte American Statistician, 40, No. 1, 12-26.
Dickey, D. A,, and Fuller, W. A. (1979). Distribution of the estimates for autoregressive time series
with a unit root, 3. hnex Statist. Assoc., 74,427-431,
Dickey, D. A., Hasza, D. P., and Fuller, W. A. (1984). Testing for unit roots in seasonal lime series,
J. Anler: Statist. Assoc., 79,355-367.
Draper, N.,and Smith, H. (1981). Applied Regression Analysis, 2nd ed., Wiley-Interscience,
New York.
Dunsmuir, \V., and Robinson, R. M. (19815. Asymptotic theory for time series containing missing and
amplitude modulated observations, Sankhya, A, 26C-281.
Durbin, J. (1960). The fitting of time series models, Reviav of the Institute of Intenratiotral Statistics,
28,233-244.
Engei, E. M. R, A. (1984). A unified approach to the study of sums, products, time-aggregation and
other functions of ARMA processes, Journal of Erne Se~iesAnalysis, 5, No. 3, 159-171.
Engle, R. F. (1982). Autoregressive conditional heteroscedasticity with estimates of the variance of
United Kingdom Inflation, Ecotrotrzetrica, 50,987-1007.
Engle, R. F., and Granger, C . W. 1. (1987). Co-integration and error correction: representation,
estimation, and testing, Eco~to~netrica,
55,251-276.
Engle, R. F.,Lilien, D. M., and Robins, R. P. (1987). Estimating time varying risk premia in the term
structure: the ARCH-M model, Econo~ne~ca,
55,391-407.
Engle, R. F., and Liu, T. C . (1972). Effects of aggregation over time on dynamic characteristics of an
econometric model, in Econometric Models of Cyclical Behaviors, Vol. 2 (Ed. B. G. Hickman),
673-737, Columbia University Press, New York.
Engle, R. E, Ng, V, K., and Rothschild, M. (1990). Asset pricing with a factor-arch covariance
structure: empirical estimates for Treasury bills, Jo~~rnal
of Econometrics, 45,213-237.
Fan, J., and Yaa, Q. (2003). Nonlinear Erne Series, Springer-Verlag, New York.
Feller, W. (1971). An Introdziction to Probability Theory and Its Appiications, Vol. 2, 2nd ed., John
Wiley, New York.
References
557
Findley, D. F. (1985). On the .unbiased property of AIC for exact or approximating linear stochastic
time series models, Jortnaal of i7t1teSeries Analysis, 6 , 229-252.
Fisher, R. A. (1929). Tests of significancein harmonic analysis, Proc. Roy. Soc,, Ser. A. 125,54-59.
Fox, A. J. (1972). Outliers in time series, J. Roy. Statist. Soc., Ser. l3,43,350-363.
Fuller, W. A. (1996). Ztttrodrrction to ~tatisficalKtne Series, 2nd ed., John Wiley, New York.
Gantmacher, F. R. (19Hl). The Theory of Matrices, Chelsea, New York.
Getveke, J. M. (1982). Measurement of linear dependence and feedback between multiple time series,
J. Arne,: Statist. Assoc., 77,304-3 13.
Geweke, J., and Porter-Hudak, S . (1983). The estimation and application of long-memory time series
models, Jounlalof Erne Series Analysis, 4,221-238.
Gnedenko, B. V. (1962). The TIzeoly of Probability (Trans. B. D. Seckler), Chelsea, NewYork.
Godfrey,M. D. (1974). Computational methods for time series, Bitll bist. Math Appl., 10,224-227.
Good, I. 1. (1958). The interaction algorithm and partial Fourier series, J. Roy. Statist. Soc., Ser. B, 20,
361-372. Addendum (1960), 322,372-375.
Gradshteyn, I. S., and Ryzhik, 1. M. (1965). Tabks of hztegrals, Series and Pmd~rcts,4th ed., Academic
Pfess, New York.
Graham, A. (1981). Kronecker Products and Matrix Calclrlus: Wtlt Applications, John Wiley,
New York.
Granger, C. W. J. (1986). Developments in the study of co-integrated economic variables, Oxford
Brrlleti~tof Ecot1oniics and Statistics, 48,213-228.
Granger C . W. J., and Lee, T. H. (1989). The effects of aggregation on nonlinearity, tech. report,
(University of California, San Diego); also in R. Mariano (Ed.), Advances ill Statistical Analysis
and Statistical Co~nplrting,JAI. Press, Greenwich, CT.
Granger, C. W. J., and Newbold, P. (1 986). Forecast Ecoiiotnic Eine Series, 2nd ed., Academic Press,
New York.
Gray, K.L., Kelley, G. D., and Mclntire, D. D. (1978). A new approach to AFWA modeling, Contrn~tnicatiotls ka Statistics, 87,1-77.
Gudmundsson, G. (1999). Disaggregation of annual flow data with multiplicative trends, Jorrnial of
Forecasting, 18,33-37.
Guerrero, V. M. (1990). Temporai disaggregation of time series: an ARlMA based approach, b~tematiottal
Staiistical Review, 5 8 , 2 9 4 .
Halmos, P. R. (1951). bltroduction to Hilberf Space, Chelsea, New York.
Hamilton, J. D. (1994). Kitre Serieshzalysis, Princeton University Press, Princeton, NJ.
Hannan, E. J. (1969). The identification of vector mixed autoregressive moving average system, Biontetrikn, 56,223-225.
Hannan, E. J. (1970). Multiple Kttte Series, John Wiey, New York.
Hannan, E. J. (1976). Review of multiple time series, S M Reviaus, 18,132.
Hannan, E. J. (1979). The statistical theory of linear systems, h Developn~etttsiti Statistics (Ed. P. R.
~rishnaiah),Vol. 2,81-121, Academic Press, NewYork and London.
Hannan, E. J. (1980). The estimation of the order of an ARMA process,A~tn.Statist, 8, 1071-1081.
Hannan, E. J., and Deistlet, M. (1988). T)re Statistical Theory of Littear Systetns, John W~ley& Sons,
New York.
Hannan, E. I., Dunsmuir, W. T,M.,and Deistfer, M. (1980). Estimation of vector ARMAX models,
Jorrrnal of M~/ilrltivariate
Analysis, 10,275-295.
Hannan, E. J., and Quinn, B. G. (1979). The determination of the order of an autoregression, J. Roy.
Statist. Sac., Ser. B, 41, 190-195.
Harrison, P. f., and Stevens, C. E (1976). Bayesian forecasting (with discussion), J. Roy. Statist. Soc.,
Ser. B,38,205-247.
Hartley, H. 0.(1949). Tests of significancein harmonic analysis, Biometrika, 36,194-201.
Harvey, A. C , (1981). Enre Series Models, Halsted Press, New York.
558
References
Helmer, R. M., and Johansson, J. K. (1977). An exposition of the Box-Jenkins transfer function
analysis with an application to the advertising-sates relationship, Jolrnial of Marketing Research,
14,227-239.
Heyse, J. E, and Wei, W. W. S. (1984). Partial process autocorrelation for vector time series, tech.
report # 30, Department of Statistics, Temple University.
Heyse, J. E, and Wei, W. IV. S. (1985a). Inverse and partial lag autocorrelation for vector time series,
ASA Pruceedirigs of Biisiness and Economic Statistics Section, 233-237.
Heyse, J. F,, and Wei, W. ?V. S. (1985b). The partial lag autocorrelation function, tech. report # 32,
Department of Statistics, Temple University.
Heyse, J. F., and Wei, W. W. S. (1986). Modelling the advertising-sale relationship through use of
multiple time series techniques, Joimtal of Forecasfitig,4, 165-183.
Hillrner, S. C., Bell, ?V. R,, and Tiao, G. C. (1983). ModeUing considerations in the seasonal adjustment of economic time series, in Applied Time Series Analysis ofEcotromic Data (Ed. A. Zeher),
74-100, U.S. Bureau of the Census, Washington, DC.
Hillmer, S. C., and 'Iiao, G. C. (1979). Likelihood function of stationary multiple autoregressive
moving average models, J. Amel: Statist. Assoc., 74,652-660.
HUmer, S . C., and Tiao, G. C. (1982). An approach to seasonal adjustment, J. Antel: Statist. Assoc.,
77,63-70.
Hinich, M. J. (1982). Testing for Gaussianity and linearity of a stationary time series, Jounial of Time
Series Analysis, 3, 169-176.
Hodgess, E. M., and Wei, W. W. S. (1996). Temporal disaggregation of time series, Applied Sratistical
Science I, 33-44.
Hodgess, E. M., and Wei, W. W. S. (2000a). An autoregressive disaggregation method, Applied Statistical Science V, 221-235.
Hodgess, E. M., andlvei, W. W. S. (2000b). Temporal disaggregation of stationary bivariate time series,
Linear Afgebra arid Its Applications, 321, 175-196.
Hosking, J. R. M. (1980). The multivariate portmanteau statistic, 3. an re^ Statist. Assoc., 75,602607.
Hotta, L. K., and Vasconcellos, R. L. (1999). Aggregation and disaggregation of stmctural time series
models, Journal of Tirne Series Aaalysis, 20, 155-171.
Izenman, A. J. (1985). J. R. Wolf and the Zurich sunspot relative numbers, The Mathematical Ititelligeltcer, 7 , No. 1,27-33.
Izenman, A. J., and Zabell, S. L. (1981). Babies and the blackout: The genesis of a misconception,
Social Sciertce Research, 10,282-299.
Jazwinski, A. H. (1970). Stochcntic Processes and Filtering Theory, Academic Press, New York and
London.
Jenkins, G. M., and Watts, D.G. (1968). Spectral Analysis and Its Applications, Holden-Day, San
Francisco.
Johansen, S. (1988). Statistical analysis of cointegration vectors, Journal ofEco~io?nic
Dy~iamicsarzd
Control, 12,231-254.
Johansen, S. (1991). Estimation and hypothesis testing of cointegration vector in Gaussian vector
autoregressive models, Economerrica, 59, 1551-1580.
Johansen, S., and Juselius, K. (1990). Maximum likelihood estimation and inference on cointegration
with applications to the demand for money Oxford Bulleiitz of Econoniics and Statistics, 52,
169-210.
Johnson, R. A., and Wichern, D. W. (1998). Applied Mrdtivariate Statistical Analysis, 4th ed., Prentice
Hall, Upper Saddle River, New Jersey.
Jones, R. H. (15180). Maximum likelihood filtering of ARMA models to time series with missing obser22, 389-395.
vations, Teclu~oitietrics,
Kalrnan, R. E. (1960). A new approach to linear Htering and prediction problems, Trans. ASME J. Bark
Engrg., Ser. D, 82,3545,
References
J,
559
Kalman, R. E., and Bucy, R. S.(1961). New results in linear filtering and prediction problems, i?at~s.
ASME. 3. Basic Eltgrg., Series D, 83, 95-108.
Kalman, R.E., Falb, P. L., and Arbib, M. A. (1969). Topics in Mather~rnfical
Systern Theory,McGraw-Kill,
New York.
Kohn, R. (1982). When is an aggregate of a time series efficient forecast by its past? Jotoaal of
Ecorrometrics, 18,337-349.
Kohn, R., and Ansley, C. E (1983). Exact likelihood of vector autoregressive moving average process
with missing or aggregated data, Bior~letrika,70,225-278.
Kolmogorov, A. (1939). Sur l'interpolation et l'extrapolation des suites statiomaires, C. R. Acad. Sci.
Paris, 208,2043-2045.
Kolmogorov, A. (1941). interpolation und extrapolation von stationtiren Z u w g e n Folgen, Bull. Acad.
Sci. (Nauk), USSR, Ser. Math., 5,3-14.
Koopmans, L. H. (1974). The Spectral Analysis of 7I11teSeries, Academic Press, New York and London.
Kremers, J. M. (1989). U.S. federal indebtedness and the conduct of fiscal policy, Jozrn~alof Monetary
Econoinics, 23,219-23 8.
Lamoureux, C. G., and Lastrapes, W. D. (1993). Forecasting stock return variance: toward an understanding of stochastic implied volatitities, Review of Fi~rurtcialStudies, 5,293-326.
Ledolter, J. (1990). Outlier diagnostics in time series models, Journal of T h e Series Analysis,
11,317-324.
Lee, J. H., and Wei, W. W. S. (1995). A model-independent outlier detection procedure, Jorrmnl of
Applied Statistical Science, 2, No. 4,345-359.
Lee, R. M., and Wei, W. W. S. (1979). The Census X-11 program and quartedy seasonal adjustments,
ASA Proceeditrgs of Business and Econotltic Section, 366-370.
Lee, R. M., and Wei, W. W. S. (1980). Model based seasonal adjustment and temporal aggregation,
ASA Proceedings of Brlsitiess and Ecor~or~ric
Section, 427-43 1 .
Li, LV,K. (2003). Diagnostic Checks i~aTinze Series, Chapman and HWCRC, Lundort.
Li, W. K., and McLeod, A. I. (1981). Distribution of the residual autocorrelations in multivariate
ARMA times series models, J. Roy. Statist. Soc., Ser. B,43,231-239.
Lin, K. S. (1987). A comparative study of various univariate time series models for Canadian lynx
data, Journal of Enie Series Analysis, 8, No. 2, 161-176.
Liu, L. M. (1987). Sales forecasting using multi-equation transfer function models, Jorrmal of Forecasting, 6,223-238.
Liu, L. M., and Hanssen, D. M. (1982). Identification of multiple input transfer function models,
Cot~trnrttticationk Statistics, 11, 297-314.
Ljung, G, (1993). On outlier detection in time series, 3, Roy. Stat. SOC.,B55,559-567.
Ljung, G. M., and Box, G.B. P. (1978). On a measure of lack of fit in time series models, Biometriku,
65,297-303.
Ljung, G. M., and Box, G. E. P. (1979). The likelihood function of stationary autoregressive moving
average models, Bionretn'ka, 66,265-270.
Liitkepohl, H. (1984). Forecasting contemporaneously aggregated vector ARMA processes, Jonnral of
Busitzess and Economic Stafistics,2,201-214.
Llltkepohl, H.(1987). Forecasting Aggregated VectorARMA Processes, Springer-Verlag, Berlin.
Marquardt, P. W. (1963). An algorithm for least squares estimation of non-linear parameters, Soc.
Ind. Appl. Math., 2,431441.
Martin, R. D. (1980). Robust estimation of autoregressive models, in Directions irr Tirne Series (Ms.
D. R. Brillinger and G. C . Tiao), 254-288, institute of Mathematical Statistics, Hagwood, CA.
Masarotto, G. (1987). Robust identification of autoregressive moving average models, Applied
Statistics, 36,214-220.
Mcleod, A. I., and Li, W. K.(1983). Diagnosdc checking ARMA time series models using squaredresidual autocorrelations,Jorrmal of lime Series Analysis, 4,269-273.
560
References
Mehra, R. K. (1979). Kalman filters and their applications to forecasting, in Forecasting W s .
S . Makridakis and S. C. Wheelwright), TIMS Studies in the Marzagernent Sciertce, 12, 74-894,
North-Holland, Amsterdam.
Meinhold, R. J,, and Shgpurwalla, N. D. (1983), Understanding he Kalman filter, ~men'cmStatistician,
57,123-127.
Miller, R., and Wichern, D. W. (1977). fittermediate Busir~essStatistics, Holt, Rinehart and Winston,
New York.
Montgomery, D. C., and Weatherby, G. (1980). Modeling and forecast time series using transfer
function and intervention methods, AlIE Traasactions, 289-307.
Mliller, D., and Wei, W. W. S. (1997). Iterative least squares estimation and identification of the
transfer function model, Jottntol of Enre Series Atialysis, 18,579-592.
Nelson, D. B. (1991). Conditional heteroskedasticity in asset return: a new approach, Econo~netricn,
59,347-370.
Newbold, P. (1974). The exact likelihood function for a mixed autoregressive moving average process,
Bionzetrika, 61,423-426.
Newbold, P. (1982). Causality testing in economics, E~neSeries A~ialysis:Tl~eotyand Practice 1 (Ed.
0. D. Anderson), 701-716, North Holland, Amsterdam.
Newton, H. J., and Pagano, M. (1984). Simultaneous confidence bands for autoregressive spectra,
Bio~tzetrika,71, 197-202.
Nicholls, D. E (1975). The effrcient estimation of vector h e a r time series models, Biometrtka,
63,381-390.
Nicholls, D. F. (1977). A comparison of estimation methods for vector linear time series models,
Biornetrika, 64, 85-90.
Nicholls, D. E, and Hall, A. D. (1979). The exact likelihood function of multivariate autoregressive
moving average models, Bio~>ietrika,
66,259-264.
Nicholson, A. J. (1950). Population oscillations caused by competition for food, Nature, London,
165,476477.
Niemi, H. (1984). The invertibility of sampled and aggregated ARMA models, Metrib, 31,43-50.
Noble, 3. (1969). AppEied LinearAlgebra, Prentice Hall, Englewood Cliffs, NJ.
O'Hagan, A. (1978). Curve fitting and optimal design for prediction, J. Roy. Statist. Soc., Ser. B,
40, 1 4 2 .
Qsborn, R. R. (1977). Exact and approximate maximum likelihood estimators for vector moving
average processes, 3. Roy. Statirt. Soc., Ser. B,39, 114-1 18.
Pack, D. J. (1979). Forecasting time series affected by identifiable isolated events and an explanatory variable, tech, report, Computer Science Division, Oak Ridge National Laboratory, Oak
Ridge, TN.
Palda, K. S. (1964). The Measitrenlent of Cumulative Advehing Effects, Prentice Hall, Englewood
Cliffs, NJ.
Pantula, S. G. (1991). Asymptotic distributions of unit-root tests when the process is nearly stationary,
Joronol of Business atsd Econornic Statistics, 9,63-7 1 .
Parzen, E. (1961a). An approach to time series analysis, Ann. Math Statist., 32,951-989.
Parzen, E. (1961b). Mathematical considerations in the estimation of spectra, Techiornetrics,3,167-190.
Pamen, E. (1963). Notes on Fourier analysis and spectral windows, in Etlte Series Analysis Papers
(Ed. E. Parzen), Holden-Day, San Francisco.
Parzen, E. (1974). Some recent advances in time series modelling, ZEEE Trans. Arrto~naricConrrol,
AC-19,723-729.
Parzen, E. (1 977). Multiple time series modeling: Determining the order of approximating autoregressive
schemes, in Mz~ltiva~iutehzalysk
N (Ed. l? Krishnaiah), 283-295, Noah-Holland, Amsterdam.
Pena, D., and Box, G . E. P. (1987). Identifying a simplifying structure in time series, ofAmer. Statist
Assoc., 82,836843.
References
561
Pena, D., Tiao, G. C., and Tsay, R. S. (2001). A Colrrse i71 llitne Series kralysis, John Wiley, New York.
Petruccelli, J, D., and Davies, N. (1986). A portmanteau test for self-exciting threshold autoregressivetype nonlinearity in time series, Bio~ttetrika,73,687-694.
Phadke, M. S. (1981). Quality audit using adaptive Kalman filtering, ASQC Qrraliry Congress
Trarisactions-Sa?zFrancisco, 1045-1052.
Phadke, M. S., and Kedem, G. (1978). Computation of the exact likelihood function of multivariate
moving average models, Bio~netrika,65,511-519.
Phillips, P. C. B. (1986). Understanding spurious regressions in econometrics, Jolrr-rralof Eco~~onlerrics,
33,311-340.
P M p s , P. C . 3.(1987a). %me series regression with a unit root, Ecoilotttefrica,55,277-301.
Phillips, P. C . B. (1987b). Towards a unified asymptotic theory for autoregression, Biontetrika,
74,535-547.
Phillips, P. C . B., and Perron, P. (1988). Testing for a unit root in time series regression, Biotnetrika,
75,335-346.
Pierce, D. A. (1968). Distribution of residual correlations in dynamiclstochastic time series models,
tech. report 173, Dept. of Statistics, Univ, of ?Visconsin-Madison.
Pierce, D. A. (1980). A survey of recent developments in seasonal adjustment, The h ~ e r i c a Siaiisti~t
cia~z,34, 125-1 34.
Pierse, R. G., and Snell, A. 5. (1995). Temporal aggregation and the power of tests for a unit root,
Jozcmal of Econonretrics, 65,333-345.
Priestley, M. B. (1981). Spectral Analysis nnd Enre Series, Vols. I and 11, Academic Press, London.
Priestley, M. El. (1 988). Norr-Linear and Non-stationary Etne Series Analysis, Academic Press, London.
Priestley, M. B., and Subba Rao, T. (1975). The estimation of factor scores and Kalmm filtering for
discrete parameter stationary processes, Int, J. Control. 21,971-975.
Quenouille, M. H. (1949). Approximate tests of correlation in time series, J. Roy. Stat. Soc., B11,
68-84.
Quenouille, M. H. (1957). The Analysis of Mrdtiple E ~ n eSeries, Griffin,London.
Quenouille, M. H. (1958). Discrete autoregressive schemes with varying time-intervals, Me~ika,
1,21-27.
Rao, C. R. (1965). Littear Statistical I~lferenceand Its Applications, John Wiley, New York.
Reinsel, G . C., and Ahn, S. K.(1992). Vector autoregressive models with unit roots and reduced rank
structure: estimation, likelihood ratio test, and forecasting, Jotinla1 of Ti,ne Series Analysis,
13,353-375.
Rose, D. E. (1977). Forecasting aggregates of independent Al2IMA process, Jol~rnalof Econo~tzefrics,
5,323-346.
Said, S., and Dickey, D. A. (1985). Hypothesis testing in A R M @ , 1, q) models, J. htre,: Statist. Assoc.,
80,369-374.
SAS Institute, Inc. (1999). S A S m User's Guide,Version 8, Cary, NC.
SCA Corp. (1992). Forecasfitzg and Elite Series Analysis Using the SCA Statistical System, Vol. 1,
DeKalb, IL.
SCA Corp. (1997). Forecastitig and Tiltie Series Analysis Usktg the SCA Statistical System, Vol. 2,
DeKdb, a.
Schmidt, A. M., and Gameman, D. (1997). Temporal aggregation in dynamic hear models, Joztrnal
of Forecasting, 16,293-310.
Schuster, A. (1898). On the investigation of hidden periodicities with application to a supposed 26-day
period of meteorological phenomena, Terx Mag. Abnos. Elect., 3, 13-41.
Schwartz, G. (1978).Estimating the dimension of a model, Ann. Statist., 6,461464.
Schwert, G. W. (1989). Tests for unit roots: a Monte CarIo investigation, Jartntal of Business atid
Ecorzontic Statistics, 7, 147-159.
Searle, S. R. (1982). MatrixAlgebra Usefrilfor Stadistics,John Wiley, New York.
562
References
Shaman, P. (1983). Properties of estimates of the mean square error of prediction in autoregressive
models, in Sttidies itt Eco~lometlics,2Trint.e Series, and M~tltivariateStatistics Rds. S. Karlin,
T. Amemiya, and L. A. Goodman), 331-342, Academic Press, New York and London.
Shen, S. Y.,and Wei, W. W. S. (1995). A note on the representation of a vector M A model, Jountal
of Applied Statistical Science, 2 , 31 1-3 18.
Shibata, R. (1976). Selection of the order of an autoregressivemodel by Akaike's information criterion,
Biotrteti-ika,63, 117-126.
Slutzky, E. (1927). The summation of random causes as the source of cyclic processes, Economefrica,
5, f 05-146 (1937). Translated from the earlier paper of the same title in Problems of Ecorrott~ic
Conditiotrs (Ed. Conjuncture Institute Moscow).
Stock, J. H., and Watson, M. W. (1988). Testing for common trends, J. Amer: Statist. Assoc.,
83,1097-1 107.
Stone, M. (1979). Comments on model selection criteria of Akaike and Schwartz, J. Roy. Statist. Soc.,
Ser. B, 41,276-278.
Stram, D. O., and Wei, W. W, S. (1986a). A methodological note on the disaggregation of time series
totals, Jolrrnal of721neSeries Alzalysis, 7 ,No. 4,293-302.
Stram, D. O., and Wei, W. W. S. (1986b). Temporal aggregation in the ARIMA process, Journal of
Time Series Analysis, 7 , No. 4,279-292.
Subba Rao, T,, and Gabr, M. M. (1980). A test for linearity of stationary time series, Joun~alof Tinte
Series Atmlysis, 1, 145-158.
Subba Rao, T., and Gabr, M. M. (1984). An bitmdrrction to Bispectral Analysis atrd Bilitlear Erfze
Series Models, Springer-Verlag,Berlin.
Teles, P. (1999). The Effects of Teittporal Aggregation otr Erne Series Tests,unpublished Ph.D. dissertation, Temple University.
Teles, P., and Wei, W. W. S. (1999). The use of aggregate series in time series testing, 1999IWEWNSF
31neSeries Col~erenceProceedings IX & X , 1-29, Academia Sinica, Taipai, Taiwan.
Teles, P., and Wei, W. W. S. (2000). The effects of temporal aggregation on tests of Linearity of a time
series, Cot~rprttationalStatisrics arid Data Analysis, 34,91-103.
Teles, P., and Wei, \V.W. S. (2002). The use of aggregate time series in testing for Gaussianity, Journal
of Tirne Series Analysis, 23,95-116.
Teles, P., Wei, W, W. S., and Crato, N. (1999). The use of aggregate time series in testing for long
memory, Bulletin of the International Statistical Institute, 3,341-342.
TerSisvirta, T. (1977). The invertibility of sums of discrete MA and ARMA processes, Scandirzaviatr
Joirnral of Statistics, 4 , 165-170.
Tiao, G. C. (1972). Asymptotic behavior of temporal aggregates of time series, Biotnetrika,
59,525-53 1.
Tiao, G. C., and AIi, M. M. (1971). Analysis of correlated random effects: linear model with two random components, Biortretrika, 58, 37-51.
liao, G. C., and Box, G. E. P. (1981). Modeling multiple time series with applications, J. h e r : Sfah'st.
Assoc., 76,802-816.
Tiao, G. C., and Guttman, I. (1980). Forecasting contemporal aggregates of multiple time series,
Jolrrnal of Econometrics, 12,219-230.
Tiao, G. C., and Tsay, R.S. (1983). Multiple time series modeling and extended sample cross-correlations, Joitmal of Bruitless and Ecoliornic Statistics, 1,No. 1,43-56.
Tiao, G . C., and Tsay, R. S. (1989). Model specification in multivariate time series (with discussion), J.
Roy. Statist. Soc., B51,157-213.
Tiao, G. C., and .Wei, W. W. S . (1976). Effect of temporal aggregation on the dynamic relationship of
two time series variables, Bioinetrika, 63,513-523.
Tong, H. (1977). Some comments on the Canadian lynx data with discussion,J. Roy. Statist. Soc., Ser. A,
140,448468.
References
563
Tong, H. (1978). On a threshold model, in Pattent Recognition and Signal Ptacessing (Ed. C . N.Chen),
Sijthoff and Noordhoff, Amsterdam.
Tong, H. (1 983). TttrestroldModels in Nora-linear Etne Series Analysis, Springer-Verlag,New York.
Tong, H. (1990). Non-linear Eine Series; A Dytmnic System Appmaclt, 0.dord University Press, New Yo&
Tong, H., and Lim, K. S. (1980). Threshold autoregression, limit cycles, and cyclical data (with discus'
sion), J. Roy. Statist. Soc., B42,245-292.
Tsay, R. S. (1986). %me series model specification in the presence of outliers, J. A~necStatist. Assoc.,
81,132-141.
Tsay, R. S. (1488). Outliers, level shifts, and variance changes in time series. Jozrrtlal of Forecasting,
7, 1-22.
Tsay, R. S. (1989). Testing and modeling threshold autoregressive processes, J. Anel: Statist. ASSOC.,
84,231-240.
Tsay, R. S. (1991). Detecting and modeling no linearity in univariate time series analysis, Statistics
Sinica, 1,431451.
Tsay, R. S., and Xao, G. C. (1984). Consistent estimates of autoregressive parameters and extended
sample autocorrelation function for stationary and non-stationary A
M models, J. Amer; Statist.
Assoc., 79,84-96.
Tukey, J. W. (1967). An introduction to the calcuIations of numerical spectrum analysis, in Advanced
Se~ninaron Spectral [email protected]. Harris), 2546, John Wiley, New York.
Wegman, E. J. (1986). Another look at Box-Jenkins forecasting procedure, Coaittiztn. Statist. Si~aula.,
lS{Z), 523-530.
Wei, S., and IVei, W. W. S . (1998). Comparison of some estimates of autoregressive parameters, Jattnlal
of Applied Statistical Science, 8,51-58.
Wei, W. W. S. (1978a). The effect of temporal aggregation of parameters estimation in distributed lag
model, Jolmial of Ecotzotnetrics, 8,237-246.
Wei, W. W. S. (1978b). Some consequences of temporal aggregation in seasonal time series models, in
Seasonal Analysis of Ecorzoinic Zt?re Series (Ed. A. Zellner), 433-444, U.S. Department of
Commerce, Bureau of the Census, Washington, DC.
Wei, W. W. S. (1981). Effect of systematic sampling on ARlMA models, Cortimrmicatio~tsin StafisticsTheox and Math., A10(23), 2389-2398.
Wei, W. W. S. (1982). The effect of systematic sampling and temporal aggregation on causality-A cautionary note, J. Anrel: Statist. Assoc., 77,316-319.
Wei, W.IV. S., and Abraham, B. (1981). Forecasting contemporal time series agmgates, Co~nmirtnications
in Statistics-Tl~eoc mid Meth., A10(13), 1335-1344.
Wei, W. W. S., and Mehta, J. (1980). Temporal aggregation and information loss in a distributed lag
model, in Analyzitlg n n ~ eSeries (Ed. 0. D. Anderson), 391400, North-Holland, Amsterdam.
Wei, IV. IV. S., and Stram, D. 0. (1990). Disaggregation of time series models, J. Ray. Statist. Soc.,
B52,453467.
Wei, \V. ?V. S,, and Tiao, G. C. (1975). The effect of temporal aggregation on forecasting in the
dynamic model, ASA Proceedings of Btcsitzess and Ecorro~nicSection, 613417.
Wei, \V.W. S., and Stram, D. 0. (1988). An eigenvdue approach to the limiting behavior of time series
aggregates, Anti. Inst. Statist. Matlr, 40, 101-1 10.
Weiss, A. A. (1984). Systematic sampling and temporal aggregation in time series models, Jounial of
Eco~rometrics,26,255-278.
Werner, H . J. (1982). On the temporal aggregation in discrete dynamical systems, in Systent Modelitzg
and Optitnization (Eds. R. F. Drenick and F. Kozin), 819-825, Springer, New York.
Whittle, I? (1952). The simultaneous estimation of a time series' harmonic covariance structure,
Trabajos. Estadist., 3,43-57.
Whittle, P. (1954). A statistical investigation of sunspot observations with special reference to
H. Alfven's sunspot model, The Astrophs. J , 120,251-260.
564
References
Whittle, P. (1983). Prediction and Regulatiott Lty Liriear hast-Square Methods, 2nd ed., Oxford,
Blackwell.
Wichern, D. W,,
MMir, R. B., and Hsu, D. A. (1976). Changes af variance in first-order autoregressive
time series models with an application. Appl. Stat., 25 (3), 248-256.
Wiener, N. (1949). The Extrapolatio~i,Irtterpolatiot~and St~rootlairtgof Statio~raryE I HSeries
~
wifh
Eltgineeri~rgApplicafions,John Wiley, New York.
Wilk, S. S. (1932). Certain generalization in the analysis of variance, Bio~~ietrika,
24,471-494.
Wold, H . 0.A. (1938).A Stzdy in the Analysis of Stafiortury7i11leSeries, Ahqvist and Wiksell, Uppsala.
Yaglom, A. M. (1962). An bttmduction to the Themy of Stationary Rat~dotnAatctiotzs, Prentice Hall,
Engleivood Cliffs, NJ.
Young, P. C. (1 974). Recursive approaches to time series analysis, Bull. Inst. Math. Appl., 10,209-224.
Yule, G . U. (1927). On a method of investigating periodicities in disturbed series with special reference
to Wolfer's sunspot numbers, Plzilos. Trans. Roy. Soc. Lotzdotl, Ser. A, 226,267-298.
Zellner, A. (1966). On the analysis of first order autoregressive models with incompiete data,
Intenmtioaal Econornic Reviews, 7,72-76.
Zellner, A. (1978). Semo~zalA~zniysisof Econo~)zicE~neSeries, @id.A. Zellner), U.S. Department of
Commerce, Bureau of the Census, Washington DC.
Zellner, A., and Montmarquette, C. (1971). A study of some aspects of temporal aggregation problems
in econometric analyses, Review of Ecotrot~iicsand Statistics, 53,335-342.
Appendix
Series W l
Series W2
Series W3
Series W4
Daily Average Number of Truck Manufacturing Defects

Wolf Yearly Sunspot Numbers from 1700 to 2001
Blowfly Data
Monthly Unemployed Young Women between Ages 16 and 19 in the United
States from January 1961 toAugust 2002
Series W5 yearly Cancer (All Forms) Death Rate (per 100,000) for Pennsylvania
between 1930 and 2000
Yearly U.S, Tobacco Production from 1871 to 1984
Series W6
Yearly Number of Lynx Pelts Sold by the Hudson's Bay Company in Canada
Series W7
from 1857 to 1911
Series W8
Simulated Seasonal Time Series
Series W9
Monthly Employment Figures for Males between Ages 16 and 19 in the
United States from January 1971 to December 1981
Series W10 Quarterly U.S. Beer Production from the First Quarter of 1975 to the Fourth
Quarter of 1982
Series W11 Daily Closing Stock Price of Duke Energy Corporation between January 3,
2002 and August 31,2002
Series W12 Lydia Pinkham Annual Advertising and Sales Data fiom 1907 to 1960
Series W13 The Five Series: U.S. Hog Data from 1867 to 1948
Series W14 Number of Monthly Airline Passengers in the United States from January
1995 to March 2002
Series W15 Oklahoma and Louisiana Spot Prices for Natural Gas between January 1988
and October 1991
Series W16 U.S. New One-Family Houses Sold (thousands) from January 1968 to
December 1982
Table A
Table B
Table C
Table D
Table E
Table F
Tail Areas of the Standard Normal Distribution

Critical Values of the &Distribution
Critical Values of the Chi-Square (x2)Distribution
Critical Values of the F-Distribution for F0,05
(vl, v2)
Critical Values of the F-Distribution for FO,ol(vl, u2)
Empirical Cumulative Distribution of n ( 4 - 1) for #
566
Appendix
Table G
Table H
Table I
Table J
Table K
Empirical Cumulative Distribution of Tfor q5 = 1

Percentiles of tl(6 - 1) for the Zero Mean Seasonal Models
Percentiles of the Studentized T Statistic for the Zero Mean Seasonal
Models
Empirical Percentiles of the Normalized Unit R2ot Estimator for Aggregate
Time Series: AR(1) Model without Intercept nA(@ - 1)
Empirical Percentiles of the Studentize! Statistic for Aggregate Time Series:
AR(1) Model without Intercept T4 = (0- 1)/S&
Series W1 Daily Average Number of Truck Manufacturing Defects (Read Across)
Series W2 Wolf Yearly Sunspot Numbers from 1700 to 2001
1700
1701
1702
1703
1704
1705
1706
1707
1708
1709
1710
1711
1712
1713
1714
1715
1716
1717
5.00
11.00
16.00
23.00
36.00
58.00
29.06)
20.00
10.00
8.00
3.00
0.00
0.00
2.00
11.00
27.00
47.00
63.00
1718
1719
1720
1721
1722
1723
1724
1725
1726
1727
1728
1729
1730
1731
1732
1733
1734
1735
60.00
39.00
28.00
26.00
22.00
11.00
21.00
40.00
78.00
122.00
103.00
73.00
47,OO
35.00
11.00
5.00
16.00
34.00
1736
1737
1738
1739
1740
1741
1742
1743
1744
1745
1746
1747
1748
1749
1750
1751
1752
1753
70.00
81.00
111.00
101.00
73.00
40.00
20.00
16.00
5.00
11.00
22.00
40.00
60.00
80.90
83.40
47.70
47.80
30.70
1754
1755
1756
1757
1758
1759
1760
1761
1762
1763
1764
1765
1766
1767
1768
1769
1770
1771
12.20
9.60
10.20
32.40
47.60
54.00
62.90
85.90
61.20
45.10
36.40
20.90
11.40
37.80
69.80
106.10
100.80
81.60
(Corrtinued)
Erne Series Data Used for Illustrations
567
Series W2 (continued)
(Corttintted)
568
Appendix
569
Series W3 Blowfly Data (Read Across)

1676
3050
2175
4369
4651
8579
2330
3075
3772
1628
3394
4243
7533
3123
3815
3517
2388
3869
4620
6884
3955
4639
3350
3677
2922
4849
4127
4494
4424
3018
3156
1843
3664
5546
4780
2784 5860
2625 2412
4272 ,3771
2837 4690
3016 2881
6316 6650
5753 5555
5781
2221
4955
5119
3821
6304
5712
4897
2619
5584
5838
4300
4842
4786
3920
3203
3891
5389
4168
4352
4066
3835
2706
3501
4993
5448
3215
3618
2717
4436
4446
5477
2652
Series W4 Monthly Unemployed Young Women between Ages 16 and 19 in the United
States from January 1961 to August 2002 (in Thousands)
Year
Jan,
Feb.
Mar.
Apr.
May
June
July
Aug.
Sept.
Oct.
Nov.
Dec.
(Continired )
570
Appendix
Series W4 (coritinued)
Year
Jan.
Feb.
Mar.
Apr.
May
June
July
Aug.
Sept.
Oct.
Nov.
Dec.
Series W5 Yearly Cancer (All Forms) Death Rate (per 100,000) for Pennsylvania between
1930 and 2000
1930
1941
1952
1963
1974
1985
1996
100.4
131.7
157.4
177.8
196.4
235.2
251.1
105.1
134.8
161.9
179.9
199.1
235.8
250.1
109.4
136.1
165.0
180.0
203.8
241.0
247.6
110.5
137.8
164.0
182.0
205.7
235.9
251.3
116.9 118.2 122.9 126.5 125.8

142.8 144.5 144.4 150.5 152.2
168.2 170.8 169.9 169.7 173.3
184.2 187.7 187.4 186.5 188.3
215.9 216.3 219.5 219.6 222.8
241.4 249.6 248.6 251.4 250.2
244.2
130.7
155.2
173.8
190,6
226.3
249.5
131.7
157.2
175.8
190.8
230.8
248.7
Series W6 Yearly U.S.Tobacco Production from 1871 to 1984 (in Millions of Pounds)
Series W7 Yearly Number of Lynx Pelts Sold by the Hudson's Bay Company
in Canada from 1857 to 1911
571
572
Appendix
Series W8 Simulated Seasonaf T h e Series (Read Across)
Series W9 Monthly Employment Figures for Males between Ages 16 and 19 in the
United States from January 1971 to December 1981 (in Thousands)
Year Jan. Feb, Mar. Apr. May June July Aug. Sept. - Oct. Nov. Dec.
Series WlO Quarterly U.S. Beer

Production from the First Quarter
of 1975 to the Fourth Quarter of
1982 (Millions of Barrels)
Quarter
Year
IIT
TV
Series W l 1 Daily Closing Stock Price of Duke Energy Corporation between

January 3,2002 and August 31,2002 (Read Across)
573
574
Appendix
Series W12 Lydia Pinkham Annual Advertising and Sales Data from 1907 to 1960
Year
Advertising
(in $1000)
Sales
(in $1000)
Advertising
Year
(in $1000)
Sales
(in $1000)
Xme Series Data Used for Illustrations
575
Series W13 The Five Series: U.S. Hog Data from 1857 to 1948
Year
Hog Numbers
Hog Prices
Corn Prices
Corn Supply
Wage Rates
(Continued)
576
Appendix

Year
Hog Numbers
Hog Prices
Corn Prices
Corn Supply
Wage Rates
Series W W
(continued)
Year
577
Hog Numbers
--
Hog Prices
Corn Prices
Corn Supply
Wage Rates
Series W14 Number of Monthly Airline Passengers in the United States from January 1995 to Marc
Year
Jan.
Feb.
Mar.
June
Aug.
1995
1996
1997
1998
1999
2000
2001
2002
40878
41689
44850
44705
45972
46242
49390
42947
38746
43390
43133
43742
45101
48 160
4795 1
42727
47103
51410
53305
53050
55402
58459
58824
53553
48561
51317
52925
54702
56457
60787
59515
51443
54377
55868
5 6546
58424
6 1098
62654
Oc
579
%me Series Data Used for Illustrations
Series W15 Oklahoma and Louisiana Spot Prices for Natural Gas between
January 1988 and OCtober 1991
-
Year
Month
Oklahoma Spot Price
Louisiana Spot Price
580
Appendix
Series W1S (continued)

-
Year
Month
Oklahoma Spot Price
Louisiana Spot Price
1990
1990
1991
1991
1991
1991
1991
1991
1991
1991
1991
1991
11
12
1
2
3
4
1.807
1,903
1.627
1.223
1.208
1.208
1,205
1.165
1,020
1.065
1.287
1.613
2.052
2.237
1.830
1.340
1.318
1.320
5
6
7
8
9
10
1.298
1.258
1.117
1.137
1,368
1.732
Series W16 U.S.New One-Family Houses Sold (Thousands) from January 1968
to December 1982
Year
Jan.
Feb.
Mar.
Apr.
May
June
July
Aug,
Sept.
Oct.
Nov.
Dec.
Statistical Tables
Table A Tail Areas of the Standard Normal Distribution
581
582
Appendix
Table A (continued)
Table B Critical Values of the t-Distribution
(Continued)
Statistical Tables
Table B (continued)
583
584
Appendix
Table B (contir~ued)
.25
.I0
.05
.025
.01
,005
Statistical Tables
Table B (continued)
D. B.Owen, Hartdbook of Statistical Tabh, O 1962, Addison-Wesley Publishing Co. Copyright renewal
O 1990. Reprinted by permission of Pearson Education, Inc. Publishing as Pearson Addison Wesley.
TabIe C Critical Values of the Chi-square (,y2)Distribution
585
586
Appendix
Table C (coatinued)
(Continued)
Statistical Tables
Table C (conti~zlued)
,995
-99
.975
-95
For large values of v, one can use approximation ,yz =
587
-05
.025
(m+ ~,)'/2,
.01
.a05
where N, is the a% upper tail
standardized normal deviate given in Table A. For example, when v = 100,

x i s (d2(100) - 1 + 1,645)'/2 = 124.059.
Table D Critical Values of the F-Distribution for F 0 . 0 5 ( ~v2)

l,
588
Appendix
Table D (continued)
c
10
11
12
13
14
4.96
4.10
3.98
3.89
3.81
3.74
3,71
3.59
3.49
3.41
3.34
3.48
3,36
3.26
3.18
3.11
3.33
3,22
3.20
3.09
3.11
3.03
2-96
3.00
2.92
2.85
3.14
3.01
2.91
2.83
2.76
4.84
4.75
4.67
4.60
3.07
3.02
2.95
2.90
2.85
2.77
2.70
2.80
2.71
2.65
Statistical Tables
Table D (corrtinued)
589
Reproduced from Table 18 of Bioinetrikn Tablesfor Statisticiarls,Vol. I, 1958, by permission of E. S. Pearson

and the Biometrika T i t e e s .
590
Appendix
Table E Critical Values of the F-Distribution for FO.OI(~l,

v2)
1
Statistical Tables
59 1
Reproduced from Table 18 of Biometrika Tables for Statisticians, Vol. 1, 1958, by permission of E. S. Pearson and
the Biornetrika Trustees.
592
Appendix
Table I? Empirical Cumulative Distribution of n(4
- 1) for + = 1
Probability of a Smaller Vafue
-01
,025
.05
-10
-11.8
-12.8
-13.3
-13.6
-13.7
-13.7
-9.3
-9.9
-10.2
-10.4
-10.4
-10.5
-7.3
-7.7
-7.9
-8.0
-8.0
-8.1
-5.3
-5.5
-5.6
-5.7
-5.7
-5.7
(a)
25
50
100
250
500
w
.SO
.9 0
-32
1.01
-97
.95
.94
-93
-93
.95
,975
.99
i
-.84
d.85
-.86
-.86
-.86
1.41
1.34
1,31
1.29
1.29
1.28
1.78
1.69
1.65
1.62
1.61
1.60
2.28
2.16
2.09
2.05
2.04
2.03
Reproduced with minor notation adjustment from Table 10.A.1 in Fuller (1996). Copyright O I996 by
John W e y & Sons, Inc. This material is used by permission of John Wdey & Sons, Inc.
Statistical Tables
593
Table G Empirical Cumulative Distribution of T for I$ = 1

Reproduced with minor notation adjustment from Table I0.A.2 in FulIer (1996). Copyright Q 1996 by John
Wiley & Sons, Inc.This material is used by permission of John Wiley & Sons, Inc.
594
Appendix
Table H Percentiles of n(&
1) for the Zero Mean Seasonal Models

11
= Ins
Reproduced with Minor Notation Adjustment from Table 2 in Dickey, Ham, and Fuller [1984),
by permission of the American Statistical Association. Copyright 1984 by the American Statistical
Association. All rights reserved.
Statistical Tables
595
Table I Percentiles of the Studentized T Statistic for the Zero Mean Seasonal Models
Reproduced with Minor Notation Adjustment from Table 3 in Dickey. Hasza, and Fuller [1984), by permission
of the American Statistical Association. Copyright 1984 by the American Statistical Association. All rights reserved.
596
Appendix
Table 3 Empirical Percentiles of the Normalize! Unit Root Estimator for Aggregate
Time Series: AR(1) Modef without Intercept n,(@
1)
Order of aggregationm = 1
'#A
25
50
100
250
500
750
m
.O 1
-11.87
-12.82
-13.30
-13.59
-13.69
-13.72
-13.78
,025
.05
.lo
-9.35
-9.91
-10.19
-10.36
-10.42
-10.44
-10.48
-7.32
-7.69
-7.88
-7.99
-8.03
-8.04
-8.07
-5.32
-5.52
-5.61
-5.67
-5.69
-5.70
-5.71
-90
1.01
0.97
0.95
0.93
0.93
0,93
0.93
Order of aggregation rn = 2
Order of aggregation III = 3
.95
1.40
1.34
1.31
1,29
1.29
1.28
1.28
.975
1.78
1.69
1.65
1.62
1.61
6 1
1.60
-99
2.28
2.15
2.08
2.04
2.03
2.03
2.02
Statistical Tables
Table J (continued)
597
Order of aggregation 112 = 4
n~
-01
,025
.05
-10
.90
.95
,975
1.75
1.66
1.62
1.60
2.18
2.07
2.01
1.97
1.96
1.96
1.95
.99
Order of aggregation m = 6
-
25
50
100
250
500
750
cn
-8.44
-8.76
-8.92
-9.02
-9.05
-9.06
-9.08
-6.51
-6.68
-6.76
-6.81
-6.83
-6.83
-6.85
-5.02
-5.12
-5.17
-5.19
-5.20
-5.21
-5.21
-3.55
-3.60
-3.62
-3.64
-3.64
-3.64
-3.65
1.31
1.25
1.22
1.21
1.20
1.20
1.19
1.59
1.59
1.58
This table was constructed by Pado J. F. C. Teles using the Monte Cado method. DetaiIs are given in
Teles (1 999).
2.77
261
2.52
2.48
2.46
2.45
2,44
598
Appendix
Table K Empirical Percentiles of the SJudentized Statistic for Aggregate Time Series:
AR(1) Mode1 without Intercept T& = (@ - 1)/S&
Order of aggregation nt = 1
Order of aggregation rn = 3
Statistical Tables
599
Table K (continued)
---.
. - - .-
Order of aggregation r?t = 4
Order of aggregationm = 6
n~
-01
,025
.05
.10
-90
.95
,975
Order of aggregation tn = 12
This table was constructed by Paulo J. E C. Teles using the Monte CarIo method. Details are given in
Teles (1999).
.99
Author Index
Abraham, B., 76,128,223,235,549,553,563

Agiaktoglou, C., 544,553
Ahn, S. K., 439,440,561
Ahsanufiah, M., 549,553
Akaike, H., 133, 156, 157,319,464,471,473,553
A1-Osh, 549,553
Ali, M. M., 145,553,562
Arnemiya, T., 546,549,553
Anderson, T.W., 415,521,553,554
Andrew, P. F., 121,554
Ansley, C. E, 145,153,411,413,483,554,559
Arbib, M. A., 464,559
Ashley, R. A., 537,554
Bartlett, M. S., 19,20,112,308,330,402,
425,473,486,554
Beguin, J. M., 133,554
Bell, W. R., 109, 164,223,554,556,558
Blackman, R. B., 305,310,311,554
Bloomfield, P., 261,554
BoMin, T., 483,554
Bollerslev, T., 370,375,554
Boot, T. C. G., 549,554
Box,G. E. P., 25,26,70,85,114, 129,132, 143,
145, 153,212,218,219,223,322,401,402,
403,408,4IO,411,417,429,435,443,449,
476,553,554,555,559,560,562
Brewer, K. R. W., 527,528,546,549,555
Brjllinger, D. R., 112,115,261,358,555
Bryant, P. G., 376,555
Bucy, R. S., 464,483,559
Burr, I. W., 112,555
Buys-Ball~t,C. H. D., 160,555
Campbell, M. J., f 21,555
Chan, N. H., 186,555
Chan, W. S., 233,235,498,506,550,555
Chang, I., 223,226,555

Chen, C., 223,230,235,555
Cheung, Y. W., 544,555
Chow, G. C., 549,555
Chuang, A., 235,553
Clarke, D. G., 336,555
Cleveland, W, S., 128,275,555
Cohen, K.J., 549,555
Cooley, J. W., 259, 556
Cox, D. R., 85,554
Crato, N., 492,494,556,562
Cupingood, L. A., 164,556
Dagum, E. B., 164,556
Daniell, P. J., 314, 556
Davidson, J. E. H., 434,556
Davies, N., 501,561
Day, T. E., 380,556
DeistIer, M., 416,453,557
Denton, F., 549,556
Dickey, D. A., 109,190,191,193,194,195,
201,206,207,208,209,543,544,545,546,
547,556,561
Di Fonzo, 550,556
Draper, N.,148,556
Dunsmuir, W. T. M.,234,416,556,557
Durbh, J., 41 1,556
Engel, E. M. R. A., 549,556

EngIe, R. E, 368,369,380,429,436,437,549,556
Falb, P, L., 464,559
Fan, J., 499,556
Feibes, W., 549,554
Felier, W,, 501,556
Findley, D. F,, 157,557
Fisher, R. A., 294,557
602
Author lndex
Fox, A. J., 223,557

FuUer, W. A,, 20, 109, 186,190, 191,193,194,
195,201,207,208,209,543,544,545,546,
547,556,557
Gabr, M. M., 498,499,506,534,562
Gamerman, D., 483,561
Gantmacher, F. R., 391,397,557
Geweke, J. M., 494,528,557
Gnedenko,B. V., 20,557
Godfrey, M. D., 261,557
Good, I. J., 259,557
Gourieroux, C., 133,554,557
Gradshteyn, 491,557
Graham, A., 409,444,557
Granger, C. W. J., 332,368,429,436,437,
539,556,557
Gray, H. L., 133,557
Cuckenheimer, J., 115,555
Gudmundsson, G., 483,550,557
Guerrero, 549,557
Guttman, I., 549,562
Guttorp, P. E., 115,555
Hall, A. D., 145,416,560
Walmos, P. R., 521,557
Hamitton, J. D., 201,557
Hannan, E. J., 20,157,358,387,402,408,414,
416,448,453,557
Hanssen, D. M., 362,559
Harrison, I? J., 483
Hartley, H. O., 295,557
Harvey, A. C., 549,557
Hasza, D. P., 207,208,209,556
Helmer, R. M., 336,339, 349,558
Henry, D. F., 434,556
Herzberg,A. M., 121,554
Heyse, J. F,, 336,408,410,417,442,447,558
Hillmer, S. C., 145,164,223,416,554,558
Hinich, M. J., 534,535,537,554,558
Hodgess, E. M., 550,558
Hoskng, J. R. M., 416,558
Hotta, 550,558
Hsu, 235,563
Izenman, A. J., 112,212,221,222,558

Jazwinski, A. H., 483,558
Jenkins, G. M., 25,26,70,114,132,143,313,322,
358,411,443,476,554,558
Johansen, S., 439,440,441,558

Sohansson, I. K.,336,339,349,558
Johnson, R. A., 487,558
Jones, R. H., 528,558
Juselius, K., 441,558
Kalman, R. E., 88,464,483,558,559
Kedem, G., 416,561
Keliey, G. D., 133,557
Kohn, R., 483,549,559
Kolmogorov,A., 88,559
Koopmans, L. H., 521,559
Kremers, J. M., 429,559
Lai, K. S., 544,555

Lamoureux, 380,559
Lastrapes, W. D., 380,559
Ledolter, J., 76, 128,230,553,554
Lee, J. H., 230,559
Lee, R. M., 549,559
Lee, T.H., 539,557
Lewis, C. M., 380,556
ti,W. K., 235,373,416,559
Liiien, D.M. 369,556
Lim, K.S., 499,502,563
Lin, A,, 549,555
Lin, K. S., 121,559
Lisman, J. H. C., 549,554
Liu, L. M., 230,235,349,362,
555,559
Liu, T. C., 549,556
Ljung, G. M., 145,153,230,559
LUtkepohl, H., 528,546,549,559
Marquardt, P. W., 148,559
Maain, R. D., 223,559
Masarotto, 235,559
McIntire, D. D., 133,557
McLeod, A. I., 373,416,559
Mehra, R. K., 483,560
Mehta, J., 549,563
Meinhold, R. J., 483,560
Miller, R., 109, 146,235,556,
560,563
Monforl, A., 133,554
Montgomery, D. C,, 212,220,560
Montmarquette,C., 549,563
Miiler, W., 549,555
Muller, D., 333,560
Author Index
Nelson, D. B., 371,560
Newbold, P., 145,153,332,368,411,413,454,
544,553,554,557,560
Newton, H. J., 319,560
Ng, V. K., 380,556
Nicholls, D. F., 145,415,416,560
Nicholson, A. J., 115,560
Niemi, H., 549,560
Noble, B., 391,466,560
O'Nagan, A., 483,560
Osborn, R. R., 416,560
Oster, G., 115, 555
Pack, D. J., 362,560

Padberg, M. W,, 549,555
Pagano, M., 319,560
Pdda, K. S., 336,421,560
Pantula, S. G., 544,560
Panen,E., 19,157,311,319,521,560
Patterson, D. M., 537,554
Pena, D., 435,506,560,561
Penon, P., 200,201,206,548,561
Petruccelii, 501,561
Phadke, M. S., 416,483,561
Phillips, P, C. B., 200,201,206,368,
548,561
Pierce, D. A., 153, 164,554,561
Pierse, R. G., 548,561
Porter-Hudak, S., 494,557
Priestley, M. B., 121,312,358,483,499,561
Quenouilte,M. H., 22,235,413,414,
449,528,561
Quim, 3.G.,157,557
Rao, C. R., 496,561
Ray, B. K.,492,556
Reinsel, G. C., 114,132,143,322,411,439,
440,443,476,554,561
Robins, 369,556
Robinson, R. M., 234,556
Rose, D. E., 549,561
Rosenblatt, M., 112,555
Rothschild, M., 380,556
RyzWc, 491,557
Said, S., 206,561
Schmidt, A. M., 483,561
Schuster, A., 290,561
Schwartz, G., 157, 561

Schwert, G.W., 544,561
Seacle, S. R., 392,466,561
Shaman, P., 92,562
Shen, S. Y.,454,562
Shibata, R., 156,562
Singpurwalla, N. D., 483,560
Slutzky, E., 47,562
Smith, H., 148,556
Smith, M. A., 376,555
Snell, A. J., 548,561
Srba, E, 434,556
Stevens, C. F., 483,557
Stock, J. H., 435,562
Stone, M., 157,562
Stram,D. Q., 509,512,513,516,517,546,
549,550,562,563
Subba Rao, T., 483,498,499,506,534,
561,562
Teles, P., 494,538,539,540,541,543,
548,562
Terhvirta, T., 549,562
Trao, G. C., 129, 130,132,145, 164,212,
218,219,223,226,363,401,402,403,
408,410,416,417,429,449,457,506,
515,528,533,534,537,549,554,555,
558,561,562,563
Tong, H., 115,121,498,499,502,503,
506,555,562,563
Tsay, R. S., 130, 132,223,230,235,449,457,
501,502,503,506,561,562,563
Tukey, J, W., 26,259,261,305,310,311,
554,556,563
Walker,A. M., 121,555

Watson, M. JV,, 435, 562
Watts, D. G., 313,358,558
Wegman (1986), E. J., 91,563
Weatherby, G., 212,220,560
Wei, C. Z., 186,555
Wei, S., 235,563
Wei, W. W. S., 164,230,233,235,333,336,
363,408,410,417,442,447,454,494,
509,512,513,515,516,517,523,528,
533,534,537,538,539,540,541,543,
546,549,550,553,555,556,558,559,
562,563
603
604
Author Index
Weiss, A. A., 528,546,549,563

Werner, H. J,, 528,549,563
Whittle, P., 88, 112,295, 563
Wichem, D.\V,, 146,235,487,558,560,563
Wiener, N.,88,563
Wiks, S. S., 486,563
Wold, H. 0.A., 24,26, 160,271,563
Wu, R. Y., 546,549,553
Yaglom, A. M., 88,563

Yao, G., 499,556
Yeo, S., 434,556
Young, P. C . , 483,563
Yule, G. U., 33,42, 112,563
Zabell, S. L., 212,222,222,558

Dllner, A., 164,528,549,563
Subject Index
Additive outlier, 223

Aggregate Series, 508
limiting behavior, 515
Aggregation, 507
of t h e m process, 507
of seasonal model, 515
matrix, 516
order of, 508,523
(see aIso effect of aggregation, aggregate series,
and temporal aggregation)
Ilkaike's information criterion (AIC), 156,407
Aliasing, 285,286
Amplitude, 70
Amplitude, spectrum, 250,354
Analysis of variance for periodogram, 290
ARCH models, 366,368
ARFlMA model, 490,492
ARMA (See Autoregressive moving average), 57
ARTMA (See Autoregressive htegrated moving
average), 71
ARMAX model, 322
Autocorrelation function (ACF), 10
covariance of sample, 20
for stationary processes, 109
generating, 25
inverse, 122
matrix, 384
of AR[l) process, 34
of AR(2) process, 41
of AR@) process, 45
of MA(1) process, 48
of MA(q) process, 52
of ARMA(1, 1) process, 60,64
of ARMA(p, q) process, 58
partial, 11
properties of, 10
sample, 20
a-trimmed sample, 233
standard error of sample, 21,35
Autocorrelation matrix function, 384
Autocovariance function, 10
definition of, 10,272
generating, 25
sample, 18
spectral representation of, 267
Autocovariance generating function, 25
of ARMA(p, q) model, 275
inverse, 123
Autocovariance matrix function, 384
Autoregressive integrated moving average
(ARMA) processes, 71
ARIMA model, 72,271
ARIMA(O,l, 1) process, 74
forecasting of, 91
IMA(1, 1) process, 74
seasonal, 164
Autoregressive moving average processes, 26,57
AR representation, 23
AR(1) process, 34
AR(2) process, 39
A R ( p ) process, 45
ARMA(1, 1) process, 59
ARMA(p, q) process, 57
estimation of, 136
exact likelihood function, 143
MA presentation, 23
MA(1) process, 48
MA(2) process, 51
MA(q) process, 52
spectrum of, 274
state space representation of, 466
variance-covariance matrix of estimates, 148
vector ARMA process, 386
606
Subject Index
Autoregressive processes, 33
first-order,34
pth-order, 45
second-order, 39
stationarity of, 33,38
vector, 384
Autoregressive representation, 23
Autoregressive signal plus noise model, 487
Autoregressive spectral estimation, 319
Backcasting method, 140,141
Backshift operator, 23,27,384
Bandwidth, of window, 302,307
Bayesian information criterion (BTC), 156
Bilinear model, 499
BL(p, q, K s) model, 499
BL(l,O, 1 , 1) model, 537
Bispectrum, 495
Blackman-Tukey window, 310
Box-Cox's transformation, 85,109
Brownian motion, 187 ,440
Buys-Ballot table, 160,161, 165
CAT criterion, 157
Canonical correlations, 483
Causal model, 323,340,354
Causality, 340,363
linear, 528
Cayley-Hamilon theorem, 466,467
Central limit theorem for stationary
processes, 199
Coherence, 350,361
estimation of, 358
Cointegrated processes, 428
AR representation of, 431
decomposition of, 434
error-correction representation of, 433
MA representation of, 430
representation of, 430
Cointegrating vectors, 428
Cointegration, 428
testing and estimating, 435
Complex conjugate, 238,272,351
Complex exponentials, 240
Conditional sum of squares function, 139
Confidence interval, 3 13
for spectral ordinates, 313
Continuous time series, 1
Convolution, 262
Correlation function, 7
Correlation matrix function, 382

partial lag, 408
sample, 401
Correlogram, 10
sample, 20
Cospectrurn, 350
Covariance matrix function, 382,383,391
Covariance matrix generating function, 442
Cramer's representation, 273,351
Cross-conelogram, 326
Cross-correlation function (CCF), 325,363
and the transfer function, 325
sample, 329
Cross-covariance function, 326,383
Cross-covariancegenerating function, 349
Cross-spectrum, 349,359
and transfer function models, 359
estimation of, 357
interpretation of, 351
Cumulants, 495
Cyclical model, 268
Daniell's estimator, 315
Data, appendix, 565
blowfiy data, 2, 115,223, 153,475,569
contaimated blowfly data, 2,223
Colorado ski resort data, 183
daily average number of truck manufacturing
defects, 2, 111, 153,228,566
daily closing stock price of Duke Energy
Corporation, 216, 573
Lydia Pinkham annual advertising and sales
data, 335,416,437,441,574
Monthly airline passengers in the U.S.,
23 1,578
monthly average of hourly readings of ozone
in downtown Los Angela, 218
monthly U.S. electricity consumption, 220
monthly employment figures for young man,
172,185,572
monthly unemployed young women,
68,116,153,161,518,569
monthly U.S. housing sales and housing
starts, 364
monthly U.S. new one-family houses sold,
546,580
Oklahoma and Louisiana spot prices
for natural gas, 376,579
Quarterly U.S. beer production,
2,178,228,573
Subject Index
Data Continlred
series C of Box, Jenkins, and Reinsel, 132
series M of Box, Jenkins, and Reinsel, 476
U.S. hog data, 448,575
simulated ARCI) series, 34,36,543
simulated AR(2) series, 43
simulated ARMAtf, 1) series, 34,36,545
simulated ARIMA series, 134, 135
simulated cointegrated series, 430
simulated MA(1) series, 50
simulated MA(2) series, 52
simulated series with outliers, 236
simulated random walk series, 78
simulated seasonal time series, 153, 170,572
weekly births in New York City, 222
Wolf yearly sunspot numbers, 112, 153,566
yearly cancer death rate for Pennsylvania,
117,119,153,570
yeady number of lynx pelts sold, 121,124,
153,296,317,320,571
yearly U S , personal consumption of gasoline
and oil, 220
yearly U.S. tobacco production, 2,69,
120,153,570
yearly U.S.lumber production, 159
Data space, 471
Delta function, 263
Deterministic trends, 69
Deterministic trend term, 72, I10
test for, 110
Diagnostic checking, 152,172
of ARMA models, 153
of transfer function models, 334
of vector time series models, 416
Difference equations, 26
auxiliary equation, 27
general solution of, 27
homogeneous, 27
nonhomogeneous, 27
particular solution of, 27
Differencing, 70, 109
overdifferencing, f 09
Dirac's delta function, 263
Didchlet conditions, 255
for periodic funclions, 255
for nonperiodic functions, 258
Disaggregation, 483
Discrete time series, I
Dual relationship, between AR(p) and MA(q)
processes, 54
607
Effect (s)
of aggregation, on forecasting and parameter
estimation, 520
of aggregation, on testing for a unit root, 540
of aggregation, on testing for linearity and
normality, 534,537
of differencing, 284
of moving average, 283
of systematic s a m p h g and temporal
aggregation on causality, 528
of systematic sampling on causality, 532
of temporal aggregation on causality, 533
of temporal aggregation, on forecasting, 522
Efficiency, relative, for forecasting future
aggregates, 523
Energy spectrum, 250
Equivalent degrees, of freedom for smoothed
spectrum, 316
EquivaIent representation, of a vector
ARMA model, 387,451
Ergodicity, 19,20
Ensemble, 6
average, 18
Estimation
ARMA spectral, 3 18
of autocorrelations, 16
of autocovariances, 16
autoregressive spectral, 319
conditional maximum likelihood, 138
of cross-spectrum, 357
of GARCH models, 373
least squares, 139, 148, 151
maximum likelihood, 140
of mean, 16
nonlinear, 145
parameter, 136
of spectrum, 289
'unconditional maximum likelihood, 140
Euler relation, 238
Even function, 266
Eventual forecast functions, 100
Exact likelihood function, 143
Exponential autoregressive model, EXPAR(p, d)
model, 499
Exponential smoothing, 97
Extended sample autocorrelation function
(FSACF), 128,130
asymptotic variance of, 130
608
Subject Index
(Fm,
Fast Fourier transform

258,259
Filter, 281
Kaiman, 463
linear, 281
moving average, 283
transfer function, 283
Filtered output series, 329,331
Fisher's test, 294
Aow variable, 507
Forcing function, 27
Forecast, 88
computation of, 94
error, 88, 89, 181,343,375,416
eventual functions of, 100
limits of, 90
minimum mean square error, 88,
342,416
updating, 99
variance of, 90,181,343,375,416
Forecasting, 25, 88
application of Hilbert space in, 521
ARIMA models, 91
effects of temporal aggregation on, 522
seasonal models, 170,175
using transfer function models, 341
vectorARM.4 models, 414
Fourier
analysis, 237
coefficients, 241,242,289
frequencies, 241,290
inverse transform, 257
series, 241
Fourier representation, 241,289
of finite sequences, 241
of periodic sequences, 242
of nonperiodic sequences
(discrete-timeFourier transform), 247
of periodic functions, 254
of continuous-timefunctions
(continuous-timeFourier transform),
254,256
Fourier series, of the sequence, 241
Fourier transform, 247
discrete-time, 247
continuous-time,256
integral, 258
inverse, 249,257
.
pair@),249, 258
Fractional difference, 489,492
estimation of, 492
Frequency, 70,286
fundamental, 244
spacing of, sample, 248
Frequency domain analysis, 3,237
bivariate, 349
Frequency response function, of linear system. 359
Gain function, 352,358
GARCH models, 366,368
estimation of, 373
computation of forecast error variance, 375
Gaussian process, 10,16
General AEUMA models, 168
Generalized function, 263
Generalized least squares (GLS), 367
Harmonic components, 244
Heteroscedasticity, 368
Hidden periodic components, 292
Hidden periodicity, of order nl, 513
High pass filter, 284
Hilbert space, 520
application in forecasting, 521
Homogeneous equation, 27
Identifiabitity, for vector ARMA models, 387
Identification
of ARTMA models, 108
of noise model, 331
of seasonal models, 172
of Series W1-W7,111
Impulse response function, 324
Impulse response functions, typical, 324
Impulse response weights, 324-325
Information criterion (AIC), 133,156
Information loss, 524
due to aggregation in parameter estimation, 524
Inner product, 520
Innovation, of input, 464
Innovational outlier, 223
Input matrix, 464
Input series
prewhitened, 329
nonstationary, 343
Interpretation, of the cross-spectral functions, 351
Intervention analysis, 212
intervention analysis, examples of, 215
Intervention models, 212
Subject Index
Interventions, 212
Inverse autocorrelation function (MCF'), 122,275
Inverse autocorrelation generating function, 125
Inverse autocovariancefunction, 275
Inverse Fourier transform (inverse Fourier
integral), 257
Inverse process, 126
Tnvertibility, 25, 33,385
of AR models, 33
of ARMA models, 47
of MA models, 47
of vector ARMA models, 386
Kalman filter, 463,478
Kalman gain, 482
Kernel, 302
Kronecker product, 392
Lag window, 304,305,316,358,498
Leakage, 310
Least squares estimation, 139
conditional, 139
nonlinear, 146
ordinary, 151
unconditionat, 140
Likelihood ratio test, 425,439
Linear difference equations, 26
Linear manifold, 521
Linear time-invariant system, 463
Linear vector space, 520
Linearity test, 495
Long memory processes, 489
Low pass filter, 284
Matrix,
absolutely summable, 385
Coefficient, 384
Correlation, 383
Covariance, 382
equation, 397
polynomial, 385
square summable, 385
Maximum likelihood estimation, 138
GARCH models, 373
ARMA models, 138
conditional, 138
unconditional, 140
vector ARMA models, 415
m-dimensional white noise random
vectors, 384
609
Mean function, 7
sample, 17
variance of sample, 18
Mean absolute error, 182
Mean absolute percentage error, 182
Mean percentage error, 181
Mean square error, 19,88,91, 181
Measurement equation, 464
Method of moments, 136
Metric, of the space, 520
Minimum mean square error forecasts, 89
for ARMA models, 89
for ARTMA models, 91
for stationary input and output series, 342
for nonstationary input and output
series, 343
for vector time series models, 416
Model identification, 108,230,331,401,417
Model selection, 156
Akaike's AZC, 156
Akaike's BIC, 156
based on forecast errors, 181
method of, 156
Parzen's CAT, 157
Schwartz's SBC, 157
Models
autoregressivemoving average (ARMA), 57
autoregressiveintegrated moving average
(ARiMA), 69,7 1
construction of transfer function, 329
deterministic trend, 69
GARCH, 366,368
general ARIMA, 168
intervention, 2f 2
multiple-input transfer function, 361
nonlinear time series, 498
nonstationary time series, 69
nonstationary vector autoregressivemoving
average, 400
realizable, 323
seasonal, 160, 164, 169
state space, 463
stationary time series, 33
threshold autoregressive, 499
time series regression, 366
transfer function, 322
vector, 382,387,394,396,397,398
Moving average (MA)
and autoregressive representations of time series
process, 23
610
Subject Index
Moving average (MA) Cotrtinued

and autoregressive representations of vector
processes, 384
representation, 24,89
Moving average processes, 47
autocorrelation function of, 52
invertibility of, 47
MA(1) process, 48
MA(2) process, 51
MA(q) process, 52
stationarity of, 47
vector MA processes, 397
Multiple-input transfer function models, 361
Multiplicative model, 166
Multivariate linear regression models, 423
Multivariate partial autocorrelation matrix, 413
Noise model, 331
Noise spectrum, 360
Noise-to-signal ratio, 361
Nonaggregate series, 508
Nonhomogeneous equation, 27
Nonlinear autoregressive model, 499
NLAR(p) model, 499
NLAR(1) model, 499
Nonlinear moving average (NLMA) model, 498
Nonlinear processes, 489,494
Nonlinear time series models, 498
Normality test, 495
Nonperiodic sequence, 247
Nonstationarity
homogeneous, 71
in the mean, 2,69
in the variance and the autocovariance, 2,80
Nonstationary time series models, 68
Nonstationary vector autoregressive moving
average models, 400
nth order weakly stationary, 8
Nyquist frequency, 286
Observation equation, 464
Order of aggregation, 508,523
Ordinary least squares (OLS)
estimation, 129,151,189,366
Orthogonal basis, 241
Orthogonal functions, 238
Orthogonal subspaces, 522.
Outliers, 212,223
additive (AO), 223
detection of, 226
.innovational, 223
level shift, 230
time series, 223
temporary change, 230
Outlier model, 224
Output equation, 464
Output (or observation) matrix, 464
Overdifferencing, 109,401
PACP, IACF, and ESACF, for seasonal models, 169
Parseval's relation, 244,250,258,290
Parsimony, 27
Partial autoconelation function ( P A W , 11, 15
for seasonal models, 169
of AR(p) process, 47
of ARMA(1, 1) process, 59
of ARMA(p, q) protess, 59
of MA(q) process, 54
sample, 22
standard error of sample, 35
Partial autoregression matrices, 402
sample, 405
Partid autoregression matrix function, 404
Partial covariance matrix generating
function, 443
Partid lag autocorrelation matrix, 408,410
Partial lag correlation matrix, 413
sample, 414
Partid lag correlation matrix function, 408
Partial process, 442,447
correlation matrix, 442,447
co~~elation
matrix finction, 447
covariance matrix generating function, 443
sample correlation matrix function, 447
Parzen's CAT criterion, 157
Period, la,242
fundamental, 242
seasonal, 160
Periodic components, hidden, 292
Periodicity,
hidden, 513
test for, 292
Periodogram, 289
analysis, 289
sampling properties of, 290
Phase, 70
Subject Index
Phase spectrum, 354,358
Polyspect~um,495,496
Portmanteau test, 334
lack of fit test, 153
Positive semidefinite, 10
Power, of sequence, 245
Power spectrum, 245
Power transformation, 85
Prediction, 88
Predictor space, 471
Prewhitened input series, 329
Pretvhitening, 331
double, 332
Process
completely stationary, 7
stationary, spectral theory of, 264
strictly stationary, 7
strongly stationary, 7
Projection theorem, 521
Pulse function. 213
Quadrature spectrum, 350
Random walk model, 72,189
with drift, 73
Random walk plus noise model, 488
Rational spectrum, 275
Realizable model, 323
Realization, 6
Real-valued process, 7
Rectangular window, 306
Recursive algorithm, for computing P(s), 41 1
Relationship
between cross-correlation and transfer
functions, 328
of autocovariances between nonaggregate and
aggregate series, 508
between state space and ARMA models, 464
Sample
autocorrelation function, 20
autocovariance function, 18
correlation matrix function, 401
conelogram, 20
cross-correlation function, 329
cross-covariance function, 329
inverse autocorrelation function, 126
mean, 17
partial autocorrelation function, 22
partial autoregression matrices, 405
spacing of frequencies, 248

spectrum, 298
Sample function, 6
Sampling properties, of periodogram, 290
SBC criterjon, f 57
Schwartz's Bayesian criterion, 157
Seasonal adjusted series, 164
Seasonal models, 160
ARIMA, 164
forecasting, 170
IACF for, 169
model building for, 170
PACF for, 169
testing for unit root, 206
Seasonal dummy,162
Seasonal period, 160
Seasonal time series, 3, 160,206
moving average method, 163
regression method, 162
seasonal adjustment, 164
seasonal ARIMA models, 164
Second order stationary, i n distribution, 7
Sequence
absolutely summable, 250
. finite, 242
nonperiodic, 247
periodic, 242
power of, 245
square surnmable, 250
Short memory process, 492
Signal, of the system, 361
Signal-to-noise ratio, 361
Single-input transfer function models, 322
Smoothed spectrum, 301,357
Smoothing
in frequency domain, 301
in time domain, 304
Space
MIbert, application of in forecasting, 521
predictor, 47 1
Space representation, 464
Spectral density, 271
Spectral density matrix function, 422
Spectral distribution function, 267
Spectral distribution matrix function, 422
Spectral estimation, ARMA, 318
Spectral representation
of covariance matrix function, 422
of autocovariance functions, 267,268
of stationary processes, 272
61 1
612
Subject Index
Spectral window, 301,305

Spectrum, 264,274. See also Cospectrum;
Cross spectrum
amplitude, 250
and autocovariance generating hnction, 274
discrete, 269
energy, 250
estimation of, 289,298
of ARMA models, 274
of linear filters, 281
line, 270
matrix, 422
noise, 361
phase, 354,358
power, 245
properties of, 244
quadrature, 358
rational, 275
sample, 298
of seasonal models, 279
smoothed, 301,315,357
of sum of two independent processes, 278
of white noise process, 276
Spectrum matrix, 422
Squared coherency, 350
Square summable, 250,385
State, of the process, 464
State equation, 464
State space model, 463
canonical correlation analysis and, 470
data space, 47 1
fitting, 470
input equation, 464
measurement equation, 464
output equation, 464
predictor space, 471
system equation, 464
transition equation, 464
State space representation of ARMA models, 464
State vector, 464
Stationarity, 7
of autoregressive moving average
processes, 57
of autoregressive processes, 33,38
complete, 7
covariance, 8
in distribution, 7
of vector ARMA processes, 386
strict, 7
strong, 7
weak, 8
wide sense, 8
Stationary process
short memory process, 492
spectral theory of, 264
Stationary time series models, 33
Steady state, 482
Step function, 213
Stochastic integral, 272
Stochastic process, 6
Stochastic trend, 71
Stochastic trend models, and differencing, 70
Stock variable, 507
Strongly stationary, 7
Subspaces, orthogonal, 521
Summary statistics, for partial autoregression, 406
Sunspot numbers, 112,113,504,506
System, 463
linear, 463
System equation, 464
Systematic sampling
in time series, 507
of AUIMA process, 526
effect of, on causality, 528
Systematic sampling interval, 526
Tables, statistical, 581-599
analysis of variance table for periodogram
analysis, 290
Buys-Ballot table for seasonal data, 165
chi-square distribution, 585
characteristics of ACF and PACF for stationary
processes, 109
cointegration and unit root test on residuals, 437
equivalent degrees of freedom for various
windows, 3 17
F-distribution, 587
Fisher's test for the largest periodogam
ordinate, 295
likelihood ratio test for cointegration rank,
440,441
some typical impulse response functions,
324-326
standard normal distribution, 581
summary of models for series Wf -W7,155
t-distribution, 582
unit root test
for aggregate series, 596-599
for nonaggregate series, 592-593
for seasonal series, 594-595
Subject Index
TAR-GARCH models, 506
Temporal aggregation, 507
of AR(p) process, 512
of W
A process, 507
of IMA(d, q), process, 511
of ARIMA(p, d, q) process, 5 13
Threshold autoregressive models, 499
modeling TAR models, 502
self-excitingTAR models, 500
TAR(k;pl, . . . ,p& d) model, 500
TAR(2; I , 1; 1) model, 500
TAR(2; 4, 1I; 3) model, 503
TAR(3; 11, 10,lO; 2) model, 505
test for, 500
Time delay, 354
Xme domain analysis, 3,237
Time series, 1
Time series, homogeneous, nonstationary, 71
Time series aggregates, limiting behavior
of, 515
T i e series models
nonlinear, 498
nonstationary, 68
seasonal, 160
stationary, 33
vectors, 382,401
Time series outliers, 223
'rime series regression, 162,366
Time-invariant system, 463
Time-varying regression model, 483
Traditional methods, f 62
Transfer function, 283,322
Transfer function model, 322,331
and cross-spectrum, 359
and vector ARMA model, 388
construction of, 329
diagnostic checks of, 334
estimation of, 332
forecasting of, 341
identification of, 331
multiple input, 361
single input, 322
stable, 322
Transition equation, 464
Transition matrix, 464
Trend
deterministic, 69
stochastic, 70
Trend-cycle, 162
Triangular window, 308
Trigonometric functions, 238

orthogonality of, 240
Trigonometric identities, 240
Truncation point, 313,357,498
Tukey window, 311
Tukey-Hamming window, 31 1
Qpicd impulse response functions, 324
Unconditional least squares estimators, 140
Unconditional maximum likelhood
estimation, 140
Unconditional sum of squares, 140
Unit root, 186
AR(1) model, 189
AR(p) model, 201
a general model, 196
seasonal model, 206
testing for, 186, 196,201,540
tables, 592-599
Updating
forecasts, 99
procedure for a state vector, 482
Variance
of forecast errors, 90
of sample autocorrelations, 21
of sample autocovariances, 19,20
of sample cross-correlations, 330
of sample mean, 17
of sample partid autocorrelations,22
of sample spectrum, 300
of smoothed spectrum, 315
Variance function, 7
Variance stabilizing transformations, 83
Variance-covariance matrix, 383
Vector time series models, 382
AR(1) models, 387
ARMA(1, 1)models, 398
ARMA and transfer function models, 388
AR representation of, 384
AR(p) models, 394
autoregressive moving average process, 386
estimation of, 414
forecasting of, 414
identifiability of, 387
identification of, 401
invertibility of, 385, 389
MA(1) models, 396
MA(q) modeb, 397
MA representation of, 384
613
61 4
Subject Index
Vector time series models Continlced

at-dimensional white noise random, 384
nonstationarity, 400
stationarity of, 385,389
unit roots and cointegration in, 428
Vector processes
autoregressive presentations of, 384
spectral properties of, 421
Weakly stationary, 8
Weighting function, 304,305
White noise processes, 15, 152,384
spectrum of, 276
Wiener process, 187
Window carpentry, 313
Window closing, 3 13
Window
bandwidth of, 302,307
Bartlett's, 308
Blackman-Tbkey, 310
hamming, 3 11
hanning, 3 1 1
lag, 304,305,358
leakage, 3 10
Parzen, 3 11
rectangular, 306
spectral, 301,305,311
triangular, 308
lbkey, 31 1
Tukey-Hamming, 3 11
Wold's decomposition, of stationary
process, 271
Wold's representation, 24,384
X-12 method, 164

Yule process, 42
Yule-Walker equations, 136,403
matrix, generalized, 396
Yule-Walker estimators, 137

Time Series Analysis - Univariate and Multivariate Methods by William Wei PDF

Uploaded by

Copyright:

Available Formats

Time Series Analysis - Univariate and Multivariate Methods by William Wei PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Time Series Analysis - Univariate and Multivariate Methods by William Wei PDF

Uploaded by

Copyright:

Available Formats

Univariate and Multivariate Methods

Boston San Francisco New York

Library of Congress Cataloging-in-PublicaHonData

About the Author

Stationary Time Series Models

The Dual Relationship Between AR(p) and

Nonstationary Time Series Models

6.4 Extended Sample Autocorrelation Function and Other

6.4.1 The Extended Sample Autocorrelation Function (ESACF)

Parameter Estimation, Diagnostic Checking,

The Method of Moments

Seasonal Time Series Models

CHAPTER 9 Testing for a Unit Root

Testing for a Unit Root in Seasonal Time Series Models

CHAPTER 10 Intervention Analysis and Outlier Detection

CHAPTER 1 I Fourier Analysis

CHAPTER 12 Spectral Theory of Stationary Processes

12,2 The Spectrum of Some Common Processes

CHAPTER 13 Estimation of the Spectrum

The Spectral Window

CHAPTER 14 Transfer Function Models

14.3.4 Diagnostic Checking of Transfer Function Models

CHAPTER 15 Time Series Regression and GARCI-I Models

CHAPTER 16 Vector Time Series Models

16.4 Nonstationary Vector Autoregressive Moving Average Models

CHAPTER 17 More on Vector Time Series

CHAPTER 18 State Space Models and the Kalman Filter

CHAPTER 19 Long Memory and Nonlinear Processes

CHAPTER 20 Aggregation and Systematic Sampling in Time Series

20.6 The Effects of Aggregation on Testing for a Unit Root

Preface to the Second Edition

from the Web site: http://www.sbm,temple.edu/-wwei/.

Preface to the First Edition

1.2 Examples and Scope of This Book

@) Yearly U.S. tobacco

(c) Quarterly U.S. beer productian

FIGURE 1.1 Examples of some time series.

Examples and Scope of This Book

FIGURE 1.2 The Lydia Pinkham advertising and sales data,

1.2 Examples and Scope of This Book

Finally, we address a nontrivial, important problem in time series analysis. In working

2.1 Stochastic Processes

FG,,Gz(xl,x2) = F & , + , , Z ~ + ~x2)

+ k, and tz + k; and nth-order

the variance function of the process

the covariance function between Z, and Z,

and the correlation function between Z,, and Z,

CHAPTER 2 Fundamental Concepts

Letting ti = t - k and t2 = t, we get

Z(w, t) where t belongs

2.1 Stochastic Processes

EXAMPLE 2.2 Consider the time sequence

CHAPTER 2 Fundamental Concepts

2.2 The Autocovariance and Autocorrelation Functions

Cov(Z,, Zt+k) = EVt - ~)(Zt+k- EL),

and the correlation between 2, and Zr+k as

The Partial Autocorreiation Function

FIGURE 2.1 Example of autocorrelation function (ACF).