Forecasting - Theory and Practice

International Journal of Forecasting 38 (2022) 705–871
Contents lists available at ScienceDirect
International Journal of Forecasting

journal homepage: www.elsevier.com/locate/ijforecast
Review
Forecasting: theory and practice

∗
Fotios Petropoulos 1 , , Daniele Apiletti 2 , Vassilios Assimakopoulos 3 ,
Mohamed Zied Babai 4 , Devon K. Barrow 5 , Souhaib Ben Taieb 6 ,
Christoph Bergmeir 7 , Ricardo J. Bessa 8 , Jakub Bijak 9 , John E. Boylan 10 ,
Jethro Browell 11 , Claudio Carnevale 12 , Jennifer L. Castle 13 , Pasquale Cirillo 14 ,
Michael P. Clements 15 , Clara Cordeiro 16,17 , Fernando Luiz Cyrino Oliveira 18 ,
Shari De Baets 19 , Alexander Dokumentov 20 , Joanne Ellison 9 , Piotr Fiszeder 21 ,
Philip Hans Franses 22 , David T. Frazier 23 , Michael Gilliland 24 ,
M. Sinan Gönül 25 , Paul Goodwin 1 , Luigi Grossi 26 , Yael Grushka-Cockayne 27 ,
Mariangela Guidolin 26 , Massimo Guidolin 28 , Ulrich Gunter 29 , Xiaojia Guo 30 ,
Renato Guseo 26 , Nigel Harvey 31 , David F. Hendry 32 , Ross Hollyman 1 ,
Tim Januschowski 33 , Jooyoung Jeon 34 , Victor Richmond R. Jose 35 ,
Yanfei Kang 36 , Anne B. Koehler 37 , Stephan Kolassa 38,10 ,
Nikolaos Kourentzes 39,10 , Sonia Leva 40 , Feng Li 41 , Konstantia Litsiou 42 ,
Spyros Makridakis 43 , Gael M. Martin 23 , Andrew B. Martinez 44,45 ,
Sheik Meeran 1 , Theodore Modis 46 , Konstantinos Nikolopoulos 47 ,
Dilek Önkal 25 , Alessia Paccagnini 48,49 , Anastasios Panagiotelis 50 ,
Ioannis Panapakidis 51 , Jose M. Pavía 52 , Manuela Pedio 53,54 , Diego J. Pedregal 55 ,
Pierre Pinson 56 , Patrícia Ramos 57 , David E. Rapach 58 , J. James Reade 59 ,
Bahman Rostami-Tabar 60 , Michał Rubaszek 61 , Georgios Sermpinis 62 ,
Han Lin Shang 63 , Evangelos Spiliotis 3 , Aris A. Syntetos 60 , Priyanga
Dilini Talagala 64 , Thiyanga S. Talagala 65 , Len Tashman 66 ,
Dimitrios Thomakos 67 , Thordis Thorarinsdottir 68 , Ezio Todini 69,70 , Juan
Ramón Trapero Arenas 55 , Xiaoqian Wang 36 , Robert L. Winkler 71 ,
Alisa Yusupova 10 , Florian Ziel 72
1
School of Management, University of Bath, UK
2
Politecnico di Torino, Turin, Italy
3
Forecasting and Strategy Unit, School of Electrical and Computer Engineering, National Technical University of Athens, Greece
4
Kedge Business School, France
5
Department of Management, Birmingham Business School, University of Birmingham, UK
6
Big Data and Machine Learning Lab, Université de Mons (UMONS), Belgium
7
Faculty of Information Technology, Monash University, Melbourne, Australia
8
INESC TEC – Institute for Systems and Computer Engineering, Technology and Science, Porto, Portugal
9
Department of Social Statistics and Demography, University of Southampton, UK
10
Centre for Marketing Analytics and Forecasting, Lancaster University Management School, Lancaster University, UK
11
School of Mathematics and Statistics, University of Glasgow, UK
12
Department of Mechanical and Industrial Engineering, University of Brescia, Italy
13
Magdalen College, University of Oxford, UK
14
ZHAW School of Management and Law, Zurich University of Applied Sciences, Switzerland
∗ Corresponding author.
E-mail addresses: [email protected], [email protected] (F. Petropoulos).
https://doi.org/10.1016/j.ijforecast.2021.11.001
0169-2070/© 2021 The Author(s). Published by Elsevier B.V. on behalf of International Institute of Forecasters. This is an open access article under
the CC BY license (http://creativecommons.org/licenses/by/4.0/).
F. Petropoulos, D. Apiletti, V. Assimakopoulos et al. International Journal of Forecasting 38 (2022) 705–871
15
ICMA Centre, Henley Business School, University of Reading, UK
16
Faculdade de Ciências e Tecnologia, Universidade do Algarve, Portugal
17
CEAUL, Faculdade de Ciências, Universidade de Lisboa, Portugal
18
Pontifical Catholic University of Rio de Janeiro (PUC-Rio), Brazil
19
Department of Business Informatics and Operations Management, Faculty of Economics and Business Administration, Universiteit
Gent, Belgium
20
Let’s Forecast, Australia
21
Faculty of Economic Sciences and Management, Nicolaus Copernicus University in Torun, Poland
22
Econometric Institute, Erasmus School of Economics, Rotterdam, The Netherlands
23
Department of Econometrics and Business Statistics, Monash University, Melbourne, Australia
24
SAS, USA
25
Newcastle Business School, Northumbria University, Newcastle upon Tyne, UK
26
Department of Statistical Sciences, University of Padua, Italy
27
Darden School of Business, University of Virginia, USA
28
Finance Department, Bocconi University and Baffi-CAREFIN Centre, Milan, Italy
29
Department of Tourism and Service Management, MODUL University Vienna, Austria
30
Robert H. Smith School of Business, University of Maryland, USA
31
Department of Experimental Psychology, University College London, UK
32
Nuffield College and Institute for New Economic Thinking at the Oxford Martin School, University of Oxford, UK
33
Amazon Research, Germany
34
Korea Advanced Institute of Science and Technology, South Korea
35
McDonough School of Business, Georgetown University, USA
36
School of Economics and Management, Beihang University, Beijing, China
37
Miami University, OH, USA
38
SAP, Switzerland
39
Skövde Artificial Intelligence Lab, School of Informatics, University of Skövde, Sweden
40
Department of Energy, Politecnico di Milano, Italy
41
School of Statistics and Mathematics, Central University of Finance and Economics, Beijing, China
42
Manchester Metropolitan University Business School, UK
43
M Open Forecasting Center & Institute for the Future, University of Nicosia, Nicosia, Cyprus
44
Office of Macroeconomic Analysis, US Department of the Treasury, Washington DC, USA
45
GWU Research Program on Forecasting, Washington DC, USA
46
Growth Dynamics, Lugano, Switzerland
47
Durham University Business School, Durham University, UK
48
Michael Smurfit Business School, University College Dublin, Ireland
49
Centre for Applied Macroeconomic Analysis, Australia
50
Discipline of Business Analytics, The University of Sydney Business School, Australia
51
Department of Electrical and Computer Engineering, University of Thessaly, Volos, Greece
52
GIPEyOP, UMMICS, Department of Applied Economics, Universitat de Valencia, Spain
53
School of Accounting and Finance, University of Bristol, UK
54
Baffi-CAREFIN Centre, Bocconi University, Italy
55
ETSI Industrial, Universidad de Castilla-La Mancha, Ciudad Real, Spain
56
Department of Technology, Management and Economics, Technical University of Denmark, Denmark
57
Porto Accounting and Business School, Polytechnic of Porto, Portugal
58
Department of Economics, Saint Louis University, USA
59
Department of Economics, School of Politics, Economics and International Relations, University of Reading, UK
60
Cardiff Business School, Cardiff University, UK
61
SGH Warsaw School of Economics, Collegium of Economic Analysis, Poland
62
Adam Smith Business School, University of Glasgow, UK
63
Department of Actuarial Studies and Business Analytics, Macquarie University, Australia
64
Department of Computational Mathematics, University of Moratuwa, Sri Lanka
65
Department of Statistics, Faculty of Applied Sciences, University of Sri Jayewardenepura, Sri Lanka
66
Foresight, International Institute of Forecasters, USA
67
School of Economics and Political Science, National and Kapodistrian University of Athens, Greece
68
Norwegian Computing Center, Oslo, Norway
69
University of Bologna, Italy
70
Italian Hydrological Society, Bologna, Italy
71
Fuqua School of Business, Duke University, Durham, USA
72
House of Energy Markets and Finance, University of Duisburg–Essen, Germany
article info a b s t r a c t
Keywords: Forecasting has always been at the forefront of decision making and planning. The
Review uncertainty that surrounds the future is both exciting and challenging, with individuals
Encyclopedia and organisations seeking to minimise risks and maximise utilities. The large number of
Methods
forecasting applications calls for a diverse set of forecasting methods to tackle real-life
Applications
challenges. This article provides a non-systematic review of the theory and the practice
Principles
Time series of forecasting. We provide an overview of a wide range of theoretical, state-of-the-art
Prediction models, methods, principles, and approaches to prepare, produce, organise, and evaluate
706
forecasts. We then demonstrate how such theoretical concepts are applied in a variety
of real-life contexts.
We do not claim that this review is an exhaustive list of methods and applications.
However, we wish that our encyclopedic presentation will offer a point of reference for
the rich work that has been undertaken over the last decades, with some key insights
for the future of forecasting theory and practice. Given its encyclopedic nature, the
intended mode of reading is non-linear. We offer cross-references to allow the readers
to navigate through the various topics. We complement the theoretical concepts and
applications covered by large lists of free or open-source software implementations and
publicly-available databases.
© 2021 The Author(s). Published by Elsevier B.V. on behalf of International Institute of
Forecasters. This is an open access article under the CC BY license
(http://creativecommons.org/licenses/by/4.0/).
Contents
1. Introduction............................................................................................................................................................................................. 709
2. Theory ...................................................................................................................................................................................................... 710
2.1. Introduction to forecasting theory........................................................................................................................................... 710
2.2. Pre-processing data ................................................................................................................................................................... 711
2.2.1. Box–Cox transformations .......................................................................................................................................... 711
2.2.2. Time series decomposition ....................................................................................................................................... 712
2.2.3. Anomaly detection and time series forecasting ..................................................................................................... 713
2.2.4. Robust handling of outliers in time series forecasting.......................................................................................... 713
2.2.5. Exogenous variables and feature engineering ........................................................................................................ 714
2.3. Statistical and econometric models......................................................................................................................................... 715
2.3.1. Exponential smoothing models ................................................................................................................................ 715
2.3.2. Time-series regression models ................................................................................................................................. 715
2.3.3. Theta method and models ........................................................................................................................................ 716
2.3.4. Autoregressive integrated moving average (ARIMA) models ............................................................................... 717
2.3.5. Forecasting for multiple seasonal cycles ................................................................................................................. 718
2.3.6. State-space models .................................................................................................................................................... 718
2.3.7. Models for population processes.............................................................................................................................. 719
2.3.8. Forecasting count time series ................................................................................................................................... 720
2.3.9. Forecasting with many variables.............................................................................................................................. 721
2.3.10. Functional time series models.................................................................................................................................. 722
2.3.11. ARCH/GARCH models................................................................................................................................................. 723
2.3.12. Markov switching models ......................................................................................................................................... 723
2.3.13. Threshold models ....................................................................................................................................................... 724
2.3.14. Low and high prices in volatility models................................................................................................................ 725
2.3.15. Forecasting with DSGE models................................................................................................................................. 725
2.3.16. Robust equilibrium-correction forecasting devices ................................................................................................ 726
2.3.17. Forecasting with data subject to revision ............................................................................................................... 727
2.3.18. Innovation diffusion models ..................................................................................................................................... 727
2.3.19. The natural law of growth in competition ............................................................................................................. 728
2.3.20. Synchronic and diachronic competition .................................................................................................................. 729
2.3.21. Estimation and representation of uncertainty........................................................................................................ 730
2.3.22. Forecasting under fat tails ........................................................................................................................................ 730
2.4. Bayesian forecasting .................................................................................................................................................................. 731
2.4.1. Foundations of Bayesian forecasting........................................................................................................................ 731
2.4.2. Implementation of Bayesian forecasting ................................................................................................................. 732
2.4.3. Bayesian forecasting with copulas ........................................................................................................................... 732
2.5. Variable and model selection ................................................................................................................................................... 733
2.5.1. Leading indicators and Granger causality ............................................................................................................... 733
2.5.2. Model complexity ...................................................................................................................................................... 734
2.5.3. Variable selection ....................................................................................................................................................... 734
2.5.4. Model selection .......................................................................................................................................................... 735
2.5.5. Cross-validation for time-series data....................................................................................................................... 735
2.6. Combining forecasts .................................................................................................................................................................. 736
2.6.1. Forecast combination: a brief review of statistical approaches ........................................................................... 736
2.6.2. Density forecast combinations.................................................................................................................................. 737
2.6.3. Ensembles and predictive probability post processors.......................................................................................... 737
2.6.4. The wisdom of crowds .............................................................................................................................................. 738
2.7. Data-driven methods................................................................................................................................................................. 739
2.7.1. Forecasting with big data.......................................................................................................................................... 739
707
2.7.2. Forecasting on distributed systems ......................................................................................................................... 739

2.7.3. Agent-based models................................................................................................................................................... 740
2.7.4. Feature-based time series forecasting ..................................................................................................................... 741
2.7.5. Forecasting with bootstrap ....................................................................................................................................... 742
2.7.6. Bagging for time series forecasting.......................................................................................................................... 742
2.7.7. Multi-step ahead forecasting .................................................................................................................................... 743
2.7.8. Neural networks ......................................................................................................................................................... 744
2.7.9. Deep probabilistic forecasting models..................................................................................................................... 744
2.7.10. Machine learning........................................................................................................................................................ 745
2.7.11. Machine learning with (very) noisy data ................................................................................................................ 746
2.7.12. Clustering-based forecasting ..................................................................................................................................... 746
2.7.13. Hybrid methods.......................................................................................................................................................... 747
2.8. Methods for intermittent demand........................................................................................................................................... 748
2.8.1. Parametric methods for intermittent demand forecasting ................................................................................... 748
2.8.2. Non-parametric intermittent demand methods ..................................................................................................... 748
2.8.3. Classification methods ............................................................................................................................................... 749
2.8.4. Peak over the theshold.............................................................................................................................................. 749
2.9. Reasoning and mining............................................................................................................................................................... 750
2.9.1. Fuzzy logic .................................................................................................................................................................. 750
2.9.2. Association rule mining............................................................................................................................................. 750
2.9.3. Forecasting with text information ........................................................................................................................... 751
2.10. Forecasting by aggregation ....................................................................................................................................................... 752
2.10.1. Cross-sectional hierarchical forecasting .................................................................................................................. 752
2.10.2. Temporal aggregation ................................................................................................................................................ 753
2.10.3. Cross-temporal hierarchies ....................................................................................................................................... 753
2.10.4. Ecological inference forecasting ............................................................................................................................... 754
2.11. Forecasting with judgment....................................................................................................................................................... 754
2.11.1. Judgmental forecasting .............................................................................................................................................. 754
2.11.2. Judgmental adjustments of computer-based forecasts.......................................................................................... 755
2.11.3. Judgmental model selection...................................................................................................................................... 756
2.11.4. Panels of experts ........................................................................................................................................................ 756
2.11.5. Scenarios and judgmental forecasting ..................................................................................................................... 757
2.11.6. Trusting model and expert forecasts ....................................................................................................................... 758
2.12. Evaluation, validation, and calibration .................................................................................................................................... 758
2.12.1. Benchmarking ............................................................................................................................................................. 758
2.12.2. Point, interval, and pHDR forecast error measures................................................................................................ 759
2.12.3. Scoring expert forecasts ............................................................................................................................................ 760
2.12.4. Evaluating probabilistic forecasts ............................................................................................................................. 760
2.12.5. Assessing the reliability of probabilistic forecasts ................................................................................................. 761
2.12.6. Statistical tests of forecast performance ................................................................................................................. 762
2.12.7. Forecasting competitions........................................................................................................................................... 762
2.13. The future of forecasting theory .............................................................................................................................................. 763
3. Practice .................................................................................................................................................................................................... 764
3.1. Introduction to forecasting practice ........................................................................................................................................ 764
3.2. Operations and supply chain management ............................................................................................................................ 765
3.2.1. Demand management................................................................................................................................................ 765
3.2.2. Forecasting in the supply chain ............................................................................................................................... 766
3.2.3. Forecasting for inventories........................................................................................................................................ 767
3.2.4. Forecasting in retail ................................................................................................................................................... 767
3.2.5. Promotional forecasting............................................................................................................................................. 768
3.2.6. New product forecasting ........................................................................................................................................... 769
3.2.7. Spare parts forecasting .............................................................................................................................................. 770
3.2.8. Predictive maintenance ............................................................................................................................................. 770
3.2.9. Reverse logistics ......................................................................................................................................................... 771
3.3. Economics and finance.............................................................................................................................................................. 771
3.3.1. Macroeconomic survey expectations ....................................................................................................................... 771
3.3.2. Forecasting GDP and inflation .................................................................................................................................. 772
3.3.3. Forecasting unemployment....................................................................................................................................... 773
3.3.4. Forecasting productivity ............................................................................................................................................ 773
3.3.5. Fiscal forecasting for government budget surveillance ......................................................................................... 774
3.3.6. Interest rate prediction.............................................................................................................................................. 775
3.3.7. House price forecasting ............................................................................................................................................. 776
3.3.8. Exchange rate forecasting ......................................................................................................................................... 777
3.3.9. Financial time series forecasting with range-based volatility models................................................................. 777
3.3.10. Copula forecasting with multivariate dependent financial times series ............................................................. 778
708
3.3.11. Financial forecasting with neural networks............................................................................................................ 778

3.3.12. Forecasting returns to investment style.................................................................................................................. 779
3.3.13. Forecasting stock returns .......................................................................................................................................... 780
3.3.14. Forecasting crashes in stock markets ...................................................................................................................... 780
3.4. Energy ......................................................................................................................................................................................... 781
3.4.1. Building energy consumption forecasting and optimisation ................................................................................ 781
3.4.2. Electricity price forecasting....................................................................................................................................... 782
3.4.3. Load forecasting ......................................................................................................................................................... 785
3.4.4. Crude oil price forecasting ........................................................................................................................................ 785
3.4.5. Forecasting renewable energy technologies ........................................................................................................... 786
3.4.6. Wind power forecasting ............................................................................................................................................ 786
3.4.7. Wave forecasting ........................................................................................................................................................ 787
3.4.8. Solar power forecasting............................................................................................................................................. 788
3.4.9. Long-term simulation for large electrical power systems .................................................................................... 788
3.4.10. Collaborative forecasting in the energy sector ....................................................................................................... 790
3.5. Environmental applications ...................................................................................................................................................... 791
3.5.1. Forecasting two aspects of climate change............................................................................................................. 791
3.5.2. Weather forecasting................................................................................................................................................... 791
3.5.3. Air quality forecasting ............................................................................................................................................... 793
3.5.4. Forecasting and decision making for floods and water resources management ............................................... 793
3.6. Social good and demographic forecasting .............................................................................................................................. 794
3.6.1. Healthcare ................................................................................................................................................................... 794
3.6.2. Epidemics and pandemics ......................................................................................................................................... 795
3.6.3. Forecasting mortality ................................................................................................................................................ 796
3.6.4. Forecasting fertility .................................................................................................................................................... 797
3.6.5. Forecasting migration ................................................................................................................................................ 798
3.6.6. Forecasting risk for violence and wars.................................................................................................................... 798
3.7. Systems and humans................................................................................................................................................................. 799
3.7.1. Support systems ......................................................................................................................................................... 799
3.7.2. Cloud resource capacity forecasting ........................................................................................................................ 799
3.7.3. Judgmental forecasting in practice .......................................................................................................................... 800
3.7.4. Trust in forecasts........................................................................................................................................................ 802
3.7.5. Communicating forecast uncertainty ....................................................................................................................... 802
3.8. Other applications...................................................................................................................................................................... 803
3.8.1. Tourism demand forecasting .................................................................................................................................... 803
3.8.2. Forecasting for aviation ............................................................................................................................................. 804
3.8.3. Traffic flow forecasting .............................................................................................................................................. 804
3.8.4. Call arrival forecasting ............................................................................................................................................... 805
3.8.5. Elections forecasting .................................................................................................................................................. 807
3.8.6. Sports forecasting....................................................................................................................................................... 807
3.8.7. Forecasting for megaprojects .................................................................................................................................... 808
3.8.8. Competing products................................................................................................................................................... 809
3.8.9. Forecasting under data integrity attacks................................................................................................................. 809
3.8.10. The forecastability of agricultural time series ........................................................................................................ 810
3.8.11. Forecasting in the food and beverage industry ...................................................................................................... 810
3.8.12. Dealing with logistic forecasts in practice .............................................................................................................. 811
3.9. The future of forecasting practice............................................................................................................................................ 812
4. Forecasting: benefits, practices, value, and limitations ..................................................................................................................... 814
Declaration of competing interest........................................................................................................................................................ 815
Acknowledgments .................................................................................................................................................................................. 815
Appendix A. List of acronyms ............................................................................................................................................................... 815
Appendix B. Software ............................................................................................................................................................................ 827
Appendix C. Data sets ............................................................................................................................................................................ 827
References ............................................................................................................................................................................................... 830
1. Introduction1
Forecasting has come a long way since early humans

‘‘In theory, there is no difference between theory and looked at the sky to see if the weather would be suitable
for hunting, and even since hunters could get a forecast
practice. But, in practice, there is.’’
such as ‘‘a high of 40 with a chance of rain’’. Now a
[Benjamin Brewster (1882)] 1 This subsection was written by Robert L. Winkler.
709
hunter can look at a smartphone to instantly get hour-by- process has spurred an explosion in trying to gather in-
hour forecasts of temperatures and probabilities of rain at formation on the internet.
multiple locations as well as videos of maps showing fore- Risk is an important consideration in decision making,
casted weather patterns over the coming hours. Tailored and probability forecasts can quantify such risks. The-
forecasts of increasing sophistication can be generated to oretical work in probability forecasting has been active
inform important decisions of many different types by for some time, and decision makers in many areas of
managers, public officials, investors, and other decision practice have embraced the use of probability forecasts.
makers. In the Bayesian approach, inferences and forecasts are
In the 15 years since the excellent review paper by De probabilistic in nature, and probability forecasts can be
Gooijer and Hyndman (2006), the field of forecasting has generated in many other ways too.
seen amazing growth in both theory and practice. Thus, The U.S. National Weather Service began issuing prob-
this review is both timely and broad, ranging from the abilities of precipitation to the public in the 1960s. Yet
highly theoretical to the very applied. extensive widespread use and dissemination of probabil-
Rapid advances in computing have enabled the analy- ities has only developed since the turn of the century.
sis of larger and more complex data sets and stimulated Now probability forecasts are increasingly communicated
interest in analytics and data science. As a result, the to the public and used as inputs in decision making.
forecaster’s toolbox of methods has grown in size and Nate Silver’s FiveThirtyEight.com report gives probability
sophistication. Computer science has led the way with forecasts for elections, medicine and science, sporting
methods such as neural networks and other types of ma- events, economic measures, and many other areas, often
chine learning, which are getting a great deal of attention looking at multiple forecasting models individually and
from forecasters and decision makers. Other methods, also combining them.
including statistical methods such as Bayesian forecast- It is natural for people to desire certainty. When prob-
ing and complex regression models, have also benefited ability forecasts of precipitation were first disseminated
from advances in computing. And improvements have widely, many were very sceptical about them, with some
not been limited to those based on computing advances. accusing the forecasters of hedging and saying ‘‘Don’t give
For example, the literature on judgmental forecasting has me a probability. I want to know if it’s going to rain or
expanded considerably, driven largely by the ‘‘wisdom of not’’. Of course, point forecasts often are given along with
crowds’’ notion. probability forecasts. The current frequent exposure to
The combining, or aggregation, of forecasts, which is probabilities helps the general public better understand,
not a new idea, has received increased attention in the appreciate, and feel more comfortable with them. And the
forecasting community recently and has been shown to current situation in the world with COVID-19, increases
perform well. For example, the top-performing entries in in huge fires, big storms, political polarisation, interna-
the M4 Competition run by Spyros Makridakis combined tional conflicts, etc., should help them realise that we are
forecasts from multiple methods. Many models have been living in an age with huge uncertainties, and forecasts
developed to forecast the number of deaths that will be that quantify these uncertainties can be important. Where
caused by COVID-19, and combining the forecasts makes possible, visualisation can help, as indicated by the saying
sense because it is hard to know which one will be the that a picture is worth a thousand words. Examples are
most accurate. It is consistent with Bayesian ideas since it the cones of uncertainty on maps in forecasts of the speed,
can be viewed as updating, with each individual forecast severity, and future path of hurricanes, and the time line
added to the combined forecast (also called an ensemble) of the probability of a team winning a game, updated
contributing some new information. quickly after each play.
Despite the excitement surrounding these new devel- Put simply, this is an exciting time for the field of
opments, older methods such as ARIMA and exponential forecasting with all of the new theoretical developments
smoothing are still valuable too. Exponential smoothing, and forecasting applications in practice. Forecasting is so
along with other simple approaches, are quite robust and ubiquitous that it’s not possible to cover all of these devel-
not as prone to overfitting as more complex methods. In opments in a single article. This article manages to cover
that sense, they are useful not only on their own merits, quite a few, and a good variety. Using short presentations
but as part of an ensemble that also includes more sophis- for each one from an expert ‘‘close to the ground’’ on that
ticated methods. Combined forecasts are more valuable if theoretical topic or field of practice works well to provide
the forecasts come from methods that are diverse so that a picture of the current state of the art in forecasting
their forecast errors are not highly correlated. theory and practice.
The conditions leading to larger, more sophisticated
toolboxes for forecasters have also led to larger data sets 2. Theory
with denser grids and improved models in areas of appli-
cation. This has happened with models of the atmosphere, 2.1. Introduction to forecasting theory2
which are important in formulating improved weather
forecasts. More detailed information about customers and The theory of forecasting is based on the premise that
their preferences allows the development of improved current and past knowledge can be used to make pre-
models of customer behaviour for managers. In turn, fore- dictions about the future. In particular for time series,
casting methods that can handle all of that information
quickly are valuable for decision-making purposes. This 2 This subsection was written by Anne B. Koehler.
710
there is the belief that it is possible to identify patterns choice of a forecasting method. In fact, there may be no
in the historical values and successfully implement them historical observations at all for the item of interest, when
in the process of predicting future values. However, the judgmental methods must be used (e.g., time taken to
exact prediction of futures values is not expected. In- complete construction of a new airport). The nature of the
stead, among the many options for a forecast of a single data may well require the development of a new forecast-
time series at a future time period are an expected value ing method. The frequency of observations can include all
(known as a point forecast), a prediction interval, a per- sorts of variations, such as every minute, hourly, weekly,
centile and an entire prediction distribution. This set of monthly, and yearly (e.g., the electricity industry needs
results collectively could be considered to be ‘‘the fore- to forecast demand loads at hourly intervals as well as
cast’’. There are numerous other potential outcomes of a long term demand for ten or more years ahead). The data
forecasting process. The objective may be to forecast an could be composed of everything from a single important
event, such as equipment failure, and time series may play time series to billions of time series. Economic analysis
only a small role in the forecasting process. Forecasting often includes multiple variables, many of which affect
procedures are best when they relate to a problem to be one another. Time series for businesses are likely to be
solved in practice. The theory can then be developed by important at many different levels (e.g., stock keeping
understanding the essential features of the problem. In unit, common ingredients, or common size container)
turn, the theoretical results can lead to improved practice. and, consequently, form a hierarchy of time series. Some
In this introduction, it is assumed that forecasting the- or many of the values might be zero; making the time
ories are developed as forecasting methods and models. A series intermittent. The list of forms for data is almost
forecasting method is defined here to be a predetermined endless.
Prior to applying a forecasting method, the data may
sequence of steps that produces forecasts at future time
require pre-processing. There are basic details, such as
periods. Many forecasting methods, but definitely not all,
checking for accuracy and missing values. Other matters
have corresponding stochastic models that produce the
might precede the application of the forecasting method
same point forecasts. A stochastic model provides a data
or be incorporated into the methods/models themselves.
generating process that can be used to produce prediction
The treatment of seasonality is such a case. Some fore-
intervals and entire prediction distributions in addition
casting method/models require de-seasonalised time se-
to point forecasts. Every stochastic model makes assump-
ries, while others address seasonality within the meth-
tions about the process and the associated probability
ods/models. Making it less clear when seasonality is con-
distributions. Even when a forecasting method has an
sidered relative to a forecasting method/model, some gov-
underlying stochastic model, the model is not necessarily ernmental statistical agencies produce forecasts to extend
unique. For example, the simple exponential smoothing time series into the future in the midst of estimating
method has multiple stochastic models, including state seasonal factors (i.e., X-12 ARIMA).
space models that may or may not be homoscedastic Finally, it is extremely important to evaluate the effec-
(i.e., possess constant variance). The combining of fore- tiveness of a forecasting method. The ultimate application
casts from different methods has been shown to be a very of the forecasts provides guidance in how to measure
successful forecasting method. The combination of the their accuracy. The focus is frequently on the difference
corresponding stochastic models, if they exist, is itself a between the actual value and a point forecast for the
model. Forecasts can be produced by a process that incor- value. Many loss functions have been proposed to capture
porates new and/or existing forecasting methods/models. the ‘‘average’’ of these differences. Prediction intervals
Of course, these more complex processes would also be and percentiles can be used to judge the value of a point
forecasting methods/models. forecast as part of the forecast. On the other hand, the
Consideration of the nature of the variables and their quality of prediction intervals and prediction distributions
involvement in the forecasting process is essential. In can themselves be evaluated by procedures and formulas
univariate forecasting, the forecasts are developed for a that have been developed (e.g., ones based on scoring
single time series by using the information from the his- rules). Another assessment tool is judging the forecasts by
torical values of the time series itself. While in multi- metrics relevant to their usage (e.g., total costs or service
variate forecasting, other time series variables are in- levels).
volved in producing the forecasts, as in time series regres- In the remaining subsections of Section 2, forecast-
sion. Both univariate and multivariate forecasting may ing theory encompasses both stochastic modelling and
allow for interventions (e.g., special promotions, extreme forecasting methods along with related aspects.
weather). Relationships among variables and other types
of input could be linear or involve nonlinear structures 2.2. Pre-processing data
(e.g., market penetration of a new technology). When an
explicit functional form is not available, methodologies 2.2.1. Box–Cox transformations3
such as simulation or artificial neural networks might A common practice in forecasting models is to trans-
be employed. Theories from fields, such as economics, form the variable of interest y using the transformation
epidemiology, and meteorology, can be an important part initially proposed by Box and Cox (1964) as
of developing these relationships. Multivariate forecasting (yλ − 1)/λ λ ̸ = 0
{
could also mean forecasting multiple variables simultane- y(λ) = .
log(y) λ=0
ously (e.g., econometric models).
The data or observed values for time series come in
many different forms that may limit or determine the 3 This subsection was written by Anastasios Panagiotelis.
711
The range of the transformation will be restricted in a 2.2.2. Time series decomposition4
way that depends on the sign of λ, therefore Bickel and Time series decomposition is an important building
Doksum (1981) propose the following modification block for various forecasting approaches (see, for exam-
ple, Sections 2.3.3, 2.7.6 and 3.8.3) and a crucial tools
(|y|λ sign(yi ) − 1)/λ λ ̸ = 0
{
y(λ) = , for statistical agencies. Seasonal decomposition is a way
log(y) λ=0 to present a time series as a function of other time se-
ries, called components. Commonly used decompositions
which has a range from (−∞, ∞) for any value of λ.
are additive and multiplicative, where such functions are
For a recent review of the Box–Cox (and other similar)
summation and multiplication correspondingly. If logs can
transformations see Atkinson, Riani, and Corbellini (2021). be applied to time series, any additive decomposition
The initial motivation for the Box–Cox transforma- method can serve as multiplicative after applying log
tion was to ensure data conformed to assumptions of transformation to the data.
normality and constant error variance that are required The simplest additive decomposition of a time se-
for inference in many statistical models. The transforma- ries with single seasonality comprises three components:
tion nests the log transformation when λ = 0 and the trend, seasonal component, and the ‘‘remainder’’. It is
case of no transformation (up to an additive constant) assumed that the seasonal component has a repeating
when λ = 1. Additive models for log(y) correspond to pattern (thus sub-series corresponding to every season
multiplicative models on the original scale of y. Choices are smooth or even constant), the trend component de-
of λ between 0 and 1 therefore provide a natural con- scribes the smooth underlying mean and the remainder
tinuum between multiplicative and additive models. For component is small and contains noise.
examples of forecasting models that use either a log or The first attempt to decompose time series into trend
Box–Cox transformation see Sections 2.3.5 and 2.3.6 and and seasonality is dated to 1847 when Buys-Ballot (1847)
for applications see Sections 3.2.5, 3.6.2 and 3.8.4. performed decomposition between trend and seasonal-
The literature on choosing λ is extensive and dates ity, modelling the trend by a polynomial and the sea-
back to the original Box and Cox (1964) paper - for a re- sonality by dummy variables. Then, in 1884 Poynting
view see Sakia (1992). In a forecasting context, a popular (1884) proposed price averaging as a tool for eliminat-
method for finding λ is given by Guerrero (1993). The ing trend and seasonal fluctuations. Later, his approach
method splits the data into blocks, computes the coeffi- was extended by Anderson and Nochmals (1914), Hooker
cient of variation within each block and then computes (1901) and Spencer (1904). Copeland (1915) was the
the coefficent of variation again between these blocks. The first who attempted to extract the seasonal component,
λ that minimises this quantity is chosen. and Macaulay (1931) proposed a method which is cur-
Since the transformations considered here are mono- rently considered ‘‘classical’’.
The main idea of this method comes from the obser-
tonic, the forecast quantiles of the transformed data will,
vation that averaging a time series with window size of
when back-transformed, result in the correct forecast
the time series seasonal period leaves the trend almost
quantiles in terms of the original data. As a result finding
intact, while effectively removes seasonal and random
prediction intervals in terms of the original data only
components. At the next step, subtracting the estimated
requires inverting the transformation. It should be noted
trend from the data and averaging the result for every
though, that prediction intervals that are symmetric in season gives the seasonal component. The rest becomes
terms of the transformed data will not be symmetric the remainder.
in terms of the original data. In a similar vein, back- Classical decomposition led to a series of more com-
transformation of the forecast median of the transformed plex decomposition methods such as X-11
data returns the forecast median in terms of the orig- (Shishkin, Young, & Musgrave, 1967), X-11-ARIMA (Dagum,
inal data. For more on using the median forecast see 1988; Ladiray & Quenneville, 2001), X-12-ARIMA (Find-
Section 2.12.2 and references therein. ley, Monsell, Bell, Otto, & Chen, 1998), and X-13-ARIMA-
The convenient properties that apply to forecast quan- SEATS (Findley, 2005); see also Section 2.3.4.
tiles, do not apply to the forecast mean, something recog- Seasonal trend decomposition using Loess (STL: Cleve-
nised at least since the work of Granger and Newbold land, Cleveland, McRae, & Terpenning, 1990) takes iter-
(1976). Back-transformation of the forecast mean of the ative approach and uses smoothing to obtain a better
transformed data does not yield the forecast mean of the estimate of the trend and seasonal component at every
original data, due to the non-linearity of the transforma- iteration. Thus, starting with an estimate of the trend
tion. Consequently forecasts on the original scale of the component, the trend component is subtracted from the
data will be biased unless a correction is used. For some data, the result is smoothed along sub-series correspond-
examples of bias correction methods see Granger and ing to every season to obtain a ‘‘rough’’ estimate of the
Newbold (1976), Guerrero (1993), Pankratz and Dudley seasonal component. Since it might contain some trend,
(1987) and Taylor (1986b) and references therein. it is averaged to extract this remaining trend, which is
The issues of choosing λ and bias correcting are ac- then subtracted to get a detrended seasonal component.
counted for in popular forecasting software packages. No- This detrended seasonal component is subtracted from
tably, the method of Guerrero (1993) both for finding the data and the result is smoothed again to obtain a
λ and bias correcting is implemented in the R packages
forecast and fable (see Appendix B). 4 This subsection was written by Alexander Dokumentov.
712
better estimate of the trend. This cycle repeats a certain approaches compare the predicted values with the origi-
number of times. nal data. If the deviations are beyond a certain threshold,
Another big set of methods use a single underlining the corresponding observations are treated as anoma-
statistical model to perform decomposition. The model lies (Luo, Hong, & Fang, 2018a; Luo, Hong & Yue, 2018;
allows computation of confidence and prediction inter- Sobhani, Hong, & Martin, 2020). Contextual anomalies and
vals naturally, which is not common for iterative and anomalous sub-sequences are vastly covered by model-
methods involving multiple models. The list of such meth- based approaches. Limitations in the detectability of
ods includes TRAMO/SEATS procedure (Monsell, Aston, & anomalous events depend on the input effects of external
Koopman, 2003), the BATS and TBATS models (De Livera, time series. Examples of such effects are included in
Hyndman, & Snyder, 2011), various structural time se- SARIMAX models for polynomial approaches (see also
ries model approaches (Commandeur, Koopman, & Ooms, Section 2.3.4). In nonlinear contexts an example is the
2011; Harvey, 1990), and the recently developed seasonal- generalised Bass model (Bass, Krishnan, & Jain, 1994)
trend decomposition based on regression (STR: Doku- for special life cycle time series with external control
mentov, 2017; Dokumentov & Hyndman, 2018); see also processes (see Section 2.3.18). SARMAX with nonlinear
Section 2.3.2. The last mentioned is one of the most perturbed mean trajectory as input variable may help
generic decomposition methods allowing presence of separating the mean process under control effects from
missing values and outliers, multiple seasonal and cyclic anomalies in the residual process. Feature-based
components, exogenous variables with constant, varying, approaches, on the other hand, do not rely on predictive
seasonal or cyclic influences, arbitrary complex models. Instead, they are based on the time series fea-
seasonal schedules. By extending time series with a se- tures measured using different statistical operations (see
quence of missing values the method allows forecasting. Section 2.7.4) that differentiate anomalous instances from
typical behaviours (Fulcher & Jones, 2014). Feature-based
2.2.3. Anomaly detection and time series forecasting5 approaches are commonly used for detecting anomalous
time series within a large collection of time series. Under
Temporal data are often subject to uncontrolled, unex-
this approach, it first forecasts an anomalous threshold
pected interventions, from which various types of anoma-
for the systems typical behaviour and new observations
lous observations are produced. Owing to the complex
are identified as anomalies when they fall outside the
nature of domain specific problems, it is difficult to find a
bounds of the established anomalous threshold (Tala-
unified definition for an anomaly and mostly application-
gala, Hyndman, Leigh, Mengersen, & Smith-Miles, 2019;
specific (Unwin, 2019). In time series and forecasting
Talagala, Hyndman, Smith-Miles & Sevvandi et al., 2020).
literature, an anomaly is mostly defined with respect to
Most of the existing algorithms involve a manual anoma-
a specific context or its relation to past behaviours. The
lous threshold. In contrast, Burridge and Robert Taylor
idea of a context is induced by the structure of the input
(2006) and Talagala, Hyndman, Smith-Miles and Sev-
data and the problem formulation (Chandola, Banerjee, &
vandi et al. (2020) use extreme value theory based data-
Kumar, 2007, 2009; Hand, 2009). Further, anomaly detec-
driven anomalous thresholds. Approaches to the problem
tion in forecasting literature has two main focuses, which
of anomaly detection for temporal data can also be di-
are conflicting in nature: one demands special attention
vided into two main scenarios: (i) batch processing and
be paid to anomalies as they can be the main carriers
(ii) data streams. The data stream scenario poses many
of significant and often critical information such as fraud additional challenges, due to nonstationarity, large vol-
activities, disease outbreak, natural disasters, while the ume, high velocity, noisy signals, incomplete events and
other down-grades the value of anomalies as it reflects online support (Luo, Hong, Yue, 2018; Talagala, Hyndman,
data quality issues such as missing values, corrupted data, Smith-Miles & Sevvandi et al., 2020).
data entry errors, extremes, duplicates and unreliable val- The performance evaluation of the anomaly detec-
ues (Talagala, Hyndman, & Smith-Miles, 2020). tion frameworks is typically done using confusion matri-
In the time series forecasting context, anomaly detec- ces (Luo, Hong, Yue, 2018; Sobhani et al., 2020). However,
tion problems can be identified under three major um- these measures are not enough to evaluate the perfor-
brella themes: detection of (i) contextual anomalies (point mance of the classifiers in the presence of imbalanced
anomalies, additive anomalies) within a given series, (i) data (Hossin & Sulaiman, 2015). Following Ranawana and
anomalous sub-sequences within a given series, and (iii) Palade (2006) and Talagala et al. (2019), Leigh et al. (2019)
anomalous series within a collection of series (Gupta, have used some additional measures such as negative
Gao, Aggarwal, & Han, 2013; Talagala, Hyndman, Smith- predictive value, positive predictive value and optimised
Miles, Sevvandi & Muñoz, 2020). According to previous precision to evaluate the performance of their detection
studies forecast intervals are quite sensitive to contextual algorithms.
anomalies and the greatest impact on forecast are from
anomalies occurring at the forecast origin (Chen & Liu, 2.2.4. Robust handling of outliers in time series forecasting6
1993a). Estimators of time series processes can be dramat-
The anomaly detection methods in forecasting applica- ically affected by the presence of few aberrant obser-
tions can be categorised into two groups: (i) model-based vations which are called differently in the time series
approaches and (ii) feature-based approaches. Model-based literature: outliers, spikes, jumps, extreme observations
5 This subsection was written by Priyanga Dilini Talagala. 6 This subsection was written by Luigi Grossi.
713
(see Section 2.2.3). If their presence is neglected, coeffi- robust forecasting procedure based on weighted likeli-
cients could be biasedly estimated. Biased estimates of hood estimators to improve point and interval forecasts
ARIMA processes will decrease the efficiency of predic- in functional time series contaminated by the presence of
tions (Bianco, García Ben, Martínez, & Yohai, 2001). More- outliers.
over, as the optimal predictor of ARIMA models (see Sec-
tion 2.3.4) is a linear combination of observed units, the 2.2.5. Exogenous variables and feature engineering7
largest coefficients correspond to observations near the Exogenous variables are those included in a forecast-
forecast origin and the presence of outliers among these ing system because they add value but are not being
units can severely affect the forecasts. Proper prelimi- predicted themselves, and are sometimes called ‘features’
nary analysis of possible extreme observations is an un- (see Section 2.7.4). For example, a forecast of county’s
avoidable step, which should be carried out before any energy demand may be based on the recent history of
time series modelling and forecasting exercise (see Sec- demand (an endogenous variable), but also weather fore-
tion 2.3.9). The issue was first raised in the seminal paper casts, which are exogenous variables. Many time series
by Fox (1972), who suggests a classification of outliers in methods have extensions that facilitate exogenous vari-
time series, separating additive outliers (AO) from innova- ables, such as autoregression with exogenous variables
tion outliers (IO). The influence of different types of out- (ARX). However, it is often necessary to prepare exoge-
liers on the prediction errors in conditional mean mod- nous data before use, for example so that it matches the
temporal resolution of the variable being forecast (hourly,
els (ARIMA models) is studied by Chen and Liu (1993a,
daily, and so on).
1993b) and Ledolter (1989, 1991), while the GARCH con-
Exogenous variables may be numeric or categorical,
text (see also Section 2.3.11) is explored by Catalán and
and may be numerous. Different types of predictor present
Trívez (2007) and Franses and Ghijsels (1999). Abraham
different issues depending on the predictive model being
and Box (1979) propose a Bayesian model which reflects
used. For instance, models based on the variable’s abso-
the presence of outliers in time series and allows to mit-
lute value can be sensitive to extreme values or skewness,
igate their effects on estimated parameters and, conse-
whereas models based on the variable value’s rank, such
quently, improve the prediction ability. The main idea is
as tree-based models, are not. Exogenous variables that
to use a probabilistic framework allowing for the presence
are correlated with one another also poses a challenge for
of a small group of discrepant units. some models, and techniques such as regularisation and
A procedure for the correct specification of models, partial leased squares have been developed to mitigate
accounting for the presence of outliers, is introduced by this.
Tsay (1986) relying on iterative identification-detection- Interactions between exogenous variables my also be
removal of cycles in the observed time series contami- important when making predictions. For example, crop
nated by outliers. The same issue is tackled by Abraham yields depend on both rainfall and sunlight: one without
and Chuang (1989): in this work non-influential outliers the other or both in excess will result in low yields, but
are separated from influential outliers which are observa- the right combination will result in high yields. Inter-
tions with high residuals affecting parameter estimation. actions may be included in linear models by including
Tsay’s procedure has been later modified (Balke, 1993) to product of the two interacting exogenous as a feature in
effectively detect time series level shifts. The impulse- and the model. This is an example of feature engineering, the
step-indicator saturation approach is used by Marczak process of creating new features based on domain knowl-
and Proietti (2016) for detecting additive outliers and edge or exploratory analysis of available data. In machine
level shifts estimating structural models in the framework learning (see Section 2.7.10), many features may be cre-
of nonstationary seasonal series. They find that timely ated by combining exogenous variables speculatively and
detection of level shifts located towards the end of the passed to a selection algorithm to identify those with
series can improve the prediction accuracy. predictive power. Combinations are not limited to prod-
All these works are important because outlier and ucts, or only two interacting variables, and where many
influential observations detection is crucial for improv- exogenous variables are available, could include summary
ing the forecasting performance of models. The robust statistics (mean, standard deviation, range, quantiles...) of
estimation of model parameters is another way to im- groups of variables.
prove predictive accuracy without correcting or removing Where exogenous variables are numerous dimension
outliers (see Section 3.4.2, for the application on energy reduction may be applied to reduce the number of fea-
data). Sakata and White (1998) introduce a new two-stage tures in a forecasting model (see also Section 2.5.3). Di-
estimation strategy for the conditional variance based on mension reduction transforms multivariate data into a
Hampel estimators and S-estimators. Park (2002) pro- lower dimensional representation while retaining mean-
poses a robust GARCH model, called RGARCH exploiting ingful information about the original data. Principal com-
the idea of least absolute deviation estimation. The robust ponent analysis (PCA) is a widely used method for linear
approach is also followed for conditional mean models dimension reduction, and non-linear alternatives are also
by Gelper, Fried, and Croux (2009) who introduce a robust available. PCA is useful when the number of candidate
version of the exponential and Holt-Winters smoothing predictors is greater than the number of time series ob-
technique for prediction purposes and by Cheng and Yang servations, as is often the case in macroeconomic fore-
(2015) who propose an outlier resistant algorithm devel- casting (Stock & Watson, 2002). It is routinely applied in
oped starting from a new synthetic loss function. Very
recently, Beyaztas and Shang (2019) have introduced a 7 This subsection was written by Jethro Browell.
714
applications from weather to sales forecasting. In retail To that end, Hyndman, Koehler, Snyder, and Grose (2002)
forecasting, for example, past sales of thousands of prod- extended exponential smoothing methods under State
ucts may be recorded but including them all as exogenous Space models using a single source of error (see Sec-
variables in the forecasting model for an individual prod- tion 2.3.6) to equip them with a statistical framework
uct may be impractical. Dimension reduction offers an capable of providing future probability distributions. For
alternative to only using a subset of the available features. example, SES can be expressed in the State Space as a local
Preparation of data for forecasting tasks is increasingly level model:
important as the volume of available data is increasing
in many application areas. Further details and practical yt = ℓt −1 + ϵt ,
examples can be found in Albon (2018) and Kuhn and ℓt = ℓt −1 + αϵt .
Johnson (2019) among other texts in this area. For deeper
where the state is the level (ℓ) and ϵ is the Gaussian
technical discussion of a range of non-linear dimension
noise. Note the difference between traditional exponen-
reduction algorithms, see Hastie, Tibshirani, and Friedman
tial smoothing methods and exponential smoothing mod-
(2009).
els (under the state space approach). The former only
provide point forecasts, meanwhile the latter also offers
2.3. Statistical and econometric models
probabilistic forecasts, which obviously includes predic-
2.3.1. Exponential smoothing models8 tion intervals. In addition, some exponential smoothing
Exponential smoothing is one of the workhorses of models can be expressed an ARIMA models (see also
business forecasting. Despite the many advances in the Section 2.3.4).
field, it is always a tough benchmark to bear in mind. So far, we have introduced the main exponential
The development of exponential smoothing dates back to smoothing using SES, however real time series can include
1944, where Robert G. Brown through a mechanical com- other components as trends, seasonal patterns, cycles,
puting device estimated key variables for fire-control on and the irregular (error) component. In this sense, the
the location of submarines (Gardner, 1985). More details exponential smoothing version capable of handling local
about the state of the art of exponential smoothing can be trends is commonly known as Holt’s method (Holt, 2004,
found in Gardner (2006). originally published in 1957) and, if it also models a
The idea behind exponential smoothing relies on the seasonality component, which can be incorporated in an
weighted average of past observations, where that weight additive or multiplicative fashion, it is called Holt-Winters
decreases exponentially as one moves away from the method (Winters, 1960). Exponential smoothing models
present observations. The appropriate exponential have been also extended to handle multiple seasonal
smoothing method depends on the components that ap- cycles; see Section 2.3.5.
pear in the time series. For instance, in case that no clear Fortunately, for various combinations of time series
trend or seasonal pattern is present, the simplest form patterns (level only, trended, seasonal, trended and sea-
of exponential smoothing methods known as Simple (or sonal) a particular exponential smoothing can be cho-
Single) Exponential Smoothing (SES) is adequate, such as: sen. Pegels (1969) proposed a first classification of ex-
ponential smoothing methods, later extended by Gardner
ft +1 = α yt + (1 − α )ft (1985) and Taylor (2003a). The state space framework
In some references, is also known as Exponentially mentioned above, developed by Hyndman et al. (2002),
Weighted Moving Average (Harvey, 1990). The formula allowed to compute the likelihood for each exponential
for SES can be obtained from minimising the discounted smoothing model and, thus, model selection criteria such
least squares error function and expressing the result- as AIC could be used to automatically identify the ap-
ing equation in a recursive form (Harvey, 1990). If ob- propriate exponential smoothing model. Note that the
servations do not have the same weight, the ordinary equivalent state space formulation was derived by us-
least squares cannot be applied. On the other hand, the ing a single source of error instead of a multiple source
recursive form is very well-suited for saving data storage. of error (Harvey, 1990). Hyndman et al. (2008) utilised
In order to use SES, we need to estimate the initial the notation (E,T,S) to classify the exponential smooth-
forecast (f1 ) and the exponential smoothing parameter ing models, where those letters refer to the following
(α ). Traditionally, the initialisation was done by using components: Error, Trend, and Seasonality. This notation
either ad hoc values or a heuristic scheme (Hyndman, has gained popularity because the widely-used forecast
Koehler, Ord, & Snyder, 2008), however nowadays it is package (Hyndman et al., 2020), recently updated to the
standard to estimate both the initial forecast and the fable package, for R statistical software, and nowadays
optimal smoothing parameter by minimising the sum of exponential smoothing is frequently called ETS.
squares of the one-step ahead forecast errors. The esti-
mation of the smoothing parameter usually is restricted 2.3.2. Time-series regression models9
to values between 0 and 1. Once SES is defined, the The key idea of linear regression models is that a target
method only provides point forecasts, i.e., forecasts of (or dependent, forecast, explained, regress) variable, y,
the mean. Nonetheless, it is of vital importance for many i.e., a time series of interest, can be forecast through
applications to provide density (probabilistic) forecasts. other regressor (or independent, predictor, explanatory)
8 This subsection was written by Juan Ramón Trapero Arenas. 9 This subsection was written by Vassilios Assimakopoulos.
715
variables, x, i.e., time series or features (see Section 2.2.5), when computing prediction intervals. If these assump-
assuming that a linear relationship exists between them, tions are violated, that may mean that part of the variance
as follows of the target variable has not been explained by the model
and, therefore, that other or more regressors are needed.
yt = β0 + β1 x1t + β2 x2t + · · · + βk xkt + et ,
In case of non-linear dependencies between the target
where et is the residual error of the model at time t, β0 and the regressor variables, data power transformations
is a constant, and coefficient βi is the effect of regressor (see Section 2.2.1) or machine learning approaches can be
xi after taking into account the effects of all k regressors considered (see Section 2.7.10).
involved in the model. For example, daily product sales Apart from time series regressors, regression models
may be forecast using information related with past sales, can also exploit categorical (dummy or indicator) vari-
prices, advertising, promotions, special days, and holidays ables (Hyndman & Athanasopoulos, 2018) which may
(see also Section 3.2.4). e.g., inform the model about promotions, special events,
In order to estimate the model, forecasters typically and holidays (binary variables), the day of the week or
minimise the sum of the squared∑ errors (ordinary least month of the year (seasonal dummy variables provided as
n 2
squares estimation, OLS), SSE = t =1 et , using the ob- one-hot encoded vectors), trends and structural changes,
servations available for fitting the model to the data (Ord, and the number of trading/working days included in
Fildes, & Kourentzes, 2017) and setting the gradient ∂∂β SSE
the examined period. In cases where the target series
i
equal to zero. If the model is simple, consisting of a single is long and displays complex seasonal patterns, addi-
regressor, then two coefficients are computed, which are tional regressors such as Fourier series and lagged values
the slope (coefficient of the regressor) and the intercept of both the target and the regressor variables may be-
(constant). When more regressor variables are considered, come useful. Moreover, when the number of the potential
the model is characterised as a multiple regression one regressor variables is significant compared to the obser-
and additional coefficients are estimated. vations available for estimating the respective coefficients
A common way to evaluate how well a linear regres- (see Section 2.7.1), step-wise regression (James et al.,
sion model fits the target series, reporting an average 2013) or dimension reduction and shrinkage estimation
value of ∑ ȳ, is through the coefficient of determination, methods (see Section 2.5.3) can be considered to facilitate
n
(ft −ȳ)2
R2 = ∑nt =1 , indicating the proportion of varia- training and avoid overfitting. Finally, mixed data sam-
2
t =1 (yt −ȳ) pling (MIDAS) regression models are a way of allowing
tion in the dependent variable explained by the model.
different degrees of temporal aggregation for the regres-
Values close to one indicate sufficient goodness-of-fit,
sors and predictand (see also Section 2.10.2 for further
while values close to zero insufficient fitting. However,
goodness-of-fit should not be confused with forecastabil- discussions on forecasting with temporal aggregation).
ity (Harrell, 2015). When the complexity of the model is
increased, i.e., more regressors are considered, the value 2.3.3. Theta method and models10
of the coefficient will also rise, even if such additions lead In the age of vast computing power and computational
to overfitting (see Section 2.5.2). Thus, regression models intelligence, the contribution of simple forecasting meth-
should be evaluated using cross-validation approaches ods is possibly not en vogue; the implementation of com-
(see Section 2.5.5), approximating the post-sample accu- plicated forecasting systems becomes not only expedient
racy of the model, or measures that account for model but possibly desirable. Nevertheless forecasting, being a
complexity, such as information criteria (e.g., AIC, AICc, tricky business, does not always favour the complicated
and BIC) and the adjusted coefficient of determination, or the computationally intensive. Enter the theta method.
R̄2 = 1−(1−R2 ) n− n−1
(James, Witten, Hastie, & Tibshirani, From its beginnings 20 years back in Assimakopoulos and
k−1
2013). Other diagnostics are the standard deviation of Nikolopoulos (2000) to recent advances in the monograph
the residuals and the t-values of Nikolopoulos and Thomakos (2019), to other work in-
√ of the regressors. Resid-
between and recently too, the theta method has emerged
∑n
(yt −ft )2
ual standard error, σe = t =1
, summarises the
n−k−1 as not only a powerfully simple but also enduring method
average error produced by the model given the number in modern time series forecasting. The reader will benefit
of regressors used, thus accounting for overfitting. The t-
by reviewing Sections 2.3.1, 2.3.4 and 2.3.9 for useful
values measure the impact of excluding regressors from
background information.
the model in terms of error, given the variation in the
The original idea has been now fully explained and
data, thus highlighting the importance of the regressors.
understood and, as Nikolopoulos and Thomakos (2019)
To make sure that the produced forecasts are reli-
have shown, even the revered AR(1) model forecast is
able, the correlation between the residuals and the ob-
indeed a theta forecast – and it has already been shown
servations of the regressors must be zero, with the for-
by Hyndman and Billah (2003) that the theta method can
mer displaying also insignificant autocorrelation. Other
represent SES (with a drift) forecasts as well. In its sim-
assumptions suggest that the residuals should be nor-
plest form the method generates a forecast from a linear
mally distributed with an average value of zero and that
combination of the last observation and some form of
their variability should be equal across time (no het-
‘‘trend’’ function, be that a constant, a linear trend, a non-
eroscedasticity present). Nevertheless, in practice, it is
parametric trend or a non-linear trend. In summary, and
rarely necessary for residuals to be normally distributed in
order for the model to produce accurate results, while the
homoscedasticity assumption becomes relevant mostly 10 This subsection was written by Dimitrios Thomakos.
716
under the conditions outlined extensively in Nikolopou- Given the simplicity of its application, the freely avail-
los and Thomakos (2019), the theta forecasts can be ex- able libraries of its computation, its scalability and perfor-
pressed as functions of the ‘‘theta line’’: mance, the theta method should be considered as a crit-
ical benchmark henceforth in the literature – no amount
Qt (θ ) = θ yt + (1 − θ )Tt +1 of complexity is worth its weight if it cannot beat a single
where Tt +1 is the trend function, variously defined de- Greek letter!
pending on the modelling approach and type of trend one
2.3.4. Autoregressive integrated moving average (ARIMA)
is considering in applications. It can be shown that the,
models11
univariate, theta forecasts can given either as
Time series models that are often used for forecast-
ft +1|t = yt + ∆Qt (θ ) ing are of the autoregressive integrated moving average
class (ARIMA: Box, George, Jenkins, & Gwilym, 1976). The
when the trend function is defined as Tt +1 = µt and as notation of an ARIMA(p, d, q) model for a time series y is
ft +1|t = Qt (θ ) + θ ∆E(Tt +1 ) (1−φ1 L−· · ·−φp Lp )(1−L)d yt = c +(1+θ1 L+· · ·+θq Lq )+ϵt ,
when the trend function is left otherwise unspecified. The where the lag operator L is defined by Lk yt = yt −k .
choice of the weight parameter θ on the linear combi- The ϵt is a zero-mean uncorrelated process with common
nation of the theta line, the choice and number of trend variance σϵ2 . Some exponential smoothing models (see
functions and their nature and other aspects on expanding Section 2.3.1) can also be written in ARIMA format, where
the method have been recently researched extensively. some ETS models assume that d = 1 or d = 2. For
The main literature has two strands. The first one example, SES is equivalent to ARIMA(0,1,1) when θ1 =
details the probabilistic background of the method and α − 1.
derives certain theoretical properties, as in Hyndman and The parameters in the ARIMA model can be estimated
Billah (2003), Thomakos and Nikolopoulos (2012, 2015) using Maximum Likelihood, whereas for the ARIMA(p,
and a number of new theoretical results in Nikolopou- d, 0) Ordinary Least Squares can be used. The iterative
los and Thomakos (2019). The work of Thomakos and model-building process (Franses, van Dijk, & Opschoor,
Nikolopoulos provided a complete analysis of the theta 2014) requires the determination of the values of p, d, and
method under the unit root data generating process, ex- q. Data features as the empirical autocorrelation function
plained its success in the M3 competition (Makridakis & and the empirical partial autocorrelation function can be
used to identify the values of p and q, in case of low
Hibon, 2000), introduced the multivariate theta method
values of p and q. Otherwise, in practice one relies on
and related it to cointegration and provided a number
the well-known information criteria like AIC and BIC (see
of other analytical results for different trend functions
Section 2.5.4). The function auto.arima of the forecast
and multivariate forecasting. The second strand of the
package (Hyndman et al., 2020) for R statistical software
literature expands and details various implementation
compares models using information criteria, and has been
(including hybrid approaches) of the method, as in the
found to be very effective and increasingly being used in
theta approach in supply chain planning of Nikolopou-
ARIMA modelling.
los, Assimakopoulos, Bougioukos, Litsa, and Petropou- Forecasts from ARIMA models are easy to make. And,
los (2012), the optimised theta models and their rela- at the same time, prediction intervals can be easily com-
tionship with state space models in Fioruci, Pellegrini, puted. Take for example the ARIMA(1,0,1) model:
Louzada, and Petropoulos (2015) and Fiorucci, Pellegrini,
Louzada, Petropoulos, and Koehler (2016), hybrid ap- yt = c + φ1 yt −1 + ϵt + θ1 ϵt −1 .
proaches as in Theodosiou (2011) and Spiliotis, Assi- The one-step-ahead forecast from forecast origin n is
makopoulos and Nikolopoulos (2019), to the very latest fn+1|n = c +φ1 yn +θ1 ϵn as the expected value E(ϵn+1 ) = 0.
generalised theta method of Spiliotis, Assimakopoulos The forecast error is yn+1 − fn+1|n = ϵn+1 and, hence,
and Makridakis (2020). These are major methodological the forecast error variance is σϵ2 . The two-steps-ahead
references in the field, in addition to many others of pure forecast from n is fn+2|n = c + φ1 fn+1|n with the forecast
application of the method. error equal to ϵn+2 +φ1 ϵn+1 and the forecast error variance
The theta method is also part of the family of adaptive (1 + φ12 )σϵ2 . These expressions show that the creation
models/methods, and a simple example illustrates the of forecasts and forecast errors straightforwardly follow
point: the AR(1) forecast or the SES forecast are both from the model expressions, and hence can be automated
theta forecasts but they are also both adaptive learning if necessary.
forecasts, as in the definitions of the recent work by Kyr- An important decision when using an ARIMA model is
iazi, Thomakos, and Guerard (2019). As such, the theta the choice for the value of d. When d = 0, the model is
forecasts contain the basic building blocks of successful created for the levels of the time series, that is, yt . When
forecasts: simplicity, theoretical foundations, adaptability d = 1, there is a model for (1 − L)yt , and the data need
and performance enhancements. Further research on the to be differenced prior to fitting an ARMA model. In some
usage of the theta method within the context of adaptive specific but rare cases, d = 2. The decision on the value of
learning appears to be a natural next step. In the context
of this section, see also Section 2.3.16 on equilibrium 11 This subsection was written by Philip Hans Franses & Sheik
correcting models and forecasts. Meeran.
717
d is usually based on so-called tests for unit roots (Dickey and the computational time is high. Svetunkov and Boy-
& Fuller, 1979; Dickey & Pantula, 1987). Under the null lan (2020) introduced the Several Seasonalities ARIMA
hypothesis that d = 1, the data are non-stationary, and (SSARIMA) model which constructs ARIMA in a state-
the test involves non-standard statistical theory (Phillips, space form with several seasonalities. While SSARIMA is
1987). One can also choose to make d = 0 as the null flexible and allows for exogenous regressors, it is compu-
hypothesis (Hobijn, Franses, & Ooms, 2004; Kwiatkowski, tationally expensive, especially for high frequency series.
Phillips, Schmidt, & Shin, 1992). The power of unit root Taylor (2003b) introduced Double Seasonal
tests is not large, and in practice one often finds signals Holt-Winters (DSHW) to extend ETS for modelling daily
to consider d = 1 (Nelson & Plosser, 1982). and weekly seasonal cycles. Following that, Taylor (2010)
For seasonal data, like quarterly and monthly time proposed a triple seasonal model to consider the intraday,
series, the ARIMA model can be extended to Seasonal intraweek and intrayear seasonalities. Gould et al. (2008)
ARIMA (SARIMA) models represented by ARIMA(p, d, q)(P, and Taylor and Snyder (2012) instead proposed an ap-
D, Q )s , where P, D, and Q are the seasonal parameters proach that combines a parsimonious representation of
and the s is the periodicity. When D = 1, the data are the seasonal states up to a weekly period in an innova-
transformed as (1 − Ls )yt . It can also be that D = 0 and tion state space model. With these models, components
d = 1, and then one can replace c by c1 D1,t + c2 D2,t + can change, and decomposition is possible. However, the
· · ·+ cs Ds,t where the Di,t with i = 1, 2, . . . , s are seasonal implementation is not flexible, the use of exogenous re-
dummies. The choice of D is based on tests for so-called gressors is not supported, and the computational time
seasonal unit roots (Franses, 1991; Ghysels, Lee, & Noh, could be high.
1994; Hylleberg, Engle, Granger, & Yoo, 1990). An alternative approach for forecasting series with
Another popular extension to ARIMA models is called MSC is TBATS (De Livera et al., 2011, see also Section 2.2.2).
ARIMAX, implemented by incorporating additional ex- TBATS uses a combination of Fourier terms with an ex-
ogenous variables (regressors) that are external to and ponential smoothing state space model and a Box–Cox
different from the forecast variable. An alternative to ARI- transformation (see Section 2.2.1), in an entirely auto-
MAX is the use of regression models (see Section 2.3.2) mated manner. It allows for terms to evolve over time and
with ARMA errors. produce accurate forecasts. Some drawbacks of TBATS,
however, are that it is not flexible, can be slow, and does
2.3.5. Forecasting for multiple seasonal cycles12 not allow for covariates.
With the advances in digital data technologies, data In response to shortcomings in current models, Fore-
is recorded more frequently in many sectors such as casting with Additive Switching of Seasonality, Trend
energy (Wang, Liu, & Hong, 2016, and Section 3.4), health- and Exogenous Regressors (FASSTER) has been proposed
care (Whitt & Zhang, 2019, and 3.6.1), transportation
by O’Hara-Wild and Hyndman (2020). FASSTER is fast,
(Gould, Koehler, Ord, Snyder, Hyndman, & Vahid-Araghi,
flexible and support the use of exogenous regressors into
2008), and telecommunication (Meade & Islam, 2015a).
a state space model. It extends state space models such
This often results in time series that exhibit multiple sea-
as TBATS by introducing a switching component to the
sonal cycles (MSC) of different lengths. Forecasting prob-
measurement equation which captures groups of irregular
lems involving such series have been increasingly drawing
multiple seasonality by switching between states.
the attention of both researchers and practitioners leading
In recent years, Machine Learning (ML; see
to the development of several approaches.
Section 2.7.10) approaches have also been recommended
Multiple Linear Regression (MLR) is a common ap-
for forecasting time series with MSC. MultiLayer Percep-
proach to model series with MSC (Kamisan, Lee, Suhartono,
tron (MLP: Dudek, 2013; Zhang & Qi, 2005), Recurrent
Hussin, & Zubairi, 2018; Rostami-Tabar & Ziel, 2020); for
Neural Networks (RNN: Lai, Chang, Yang, & Liu, 2018),
an introduction on time-series regression models, see Sec-
Generalised Regression Neural Network (GRNN: Dudek,
tion 2.3.2. While MLR is fast, flexible, and uses exogenous
2015), and Long Short-Term Memory Networks (LSTM:
regressors, it does not allow to decompose components
Zheng, Xu, Zhang, & Li, 2017) have been applied on real
and change them over time. Building on the foundation
data (Bandara, Bergmeir, & Hewamalage, 2020a; Xie &
of the regression, Facebook introduced Prophet (Taylor &
Ding, 2020) with promising results. These approaches
Letham, 2018), an automated approach that utilises the
Generalised Additive Model (Hastie & Tibshirani, 1990). are flexible, allow for any exogenous regressor and suit-
Although the implementation of Prophet may be less able when non-linearity exists in series, however in-
flexible, it is easy to use, robust to missing values and terpretability might be an issue for users (Makridakis,
structural changes, and can handles outliers well. Spiliotis, & Assimakopoulos, 2018).
Some studies have extended the classical ARIMA (see
Section 2.3.4) and Exponential Smoothing (ETS; see 2.3.6. State-space models13
Section 2.3.1) methods to accommodate MSC. State Space (SS) systems are a very powerful and useful
Multiple/multiplicative Seasonal ARIMA (MSARIMA) model framework for time series and econometric modelling
is an extension of ARIMA for the case of MSC (Taylor, and forecasting. Such systems were initially developed
2003b). MSARIMA allows for exogenous regressors and by engineers, but have been widely adopted and devel-
terms can evolve over time, however, it is not flexible, oped in Economics as well (Durbin & Koopman, 2012;
12 This subsection was written by Bahman Rostami-Tabar. 13 This subsection was written by Diego J. Pedregal.
718
Harvey, 1990). The main distinguishing feature of SS sys- treated in other sections of this paper. The following is
tems is that the model is formulated in terms of states a list of possibilities, not at all exhaustive:
(αt ), which are a set of variables usually unobserved, but
which have some meaning. Typical examples are trends, • Univariate models with or without inputs: regres-
seasonal components or time varying parameters. sion (Section 2.3.2), ARIMAx (Section 2.3.4), trans-
A SS system is built as the combination of two sets of fer functions, exponential smoothing (Section 2.3.1),
equations: (i) state or transition equations which describe structural unobserved components, Hodrick–Prescott
the dynamic law governing the states between two adja- filter, spline smoothing.
cent points in time; and (ii) observation equations which • Fully multivariate: natural extensions of the pre-
specify the relation between observed data (both inputs vious ones plus echelon-form VARIMAx, Structural
and outputs) and the unobserved states. A linear version VAR, VECM, Dynamic Factor models, panel data (Sec-
of such a system is shown in Eq. (1). tion 2.3.9).
• Non-linear and non-Gaussian: TAR, ARCH, GARCH
αt +1 = Tt αt + Γ t + Rt ηt , ηt ∼ N(0, Qt ) (Section 2.3.11), Stochastic Volatility (Durbin & Koop-
yt = Zt αt + Dt + Ct ϵt , ϵt ∼ N(0, Ht ) (1) man, 2012), Dynamic Conditional Score
α1 ∼ N(a1 , P1 ) (Harvey, 2013), Generalised Autoregressive Score
In this equations ηt and ϵt are the state and observa- (Creal, Koopman, & Lucas, 2013), multiplicative un-
tional vectors of zero mean Gaussian noises with covari- observed components.
ance St . Tt , Γ t , Rt , Qt , Zt , Dt , Ct , Ht and St are the so-called • Other: periodic cubic splines, periodic unobserved
(time-varying) system matrices, and a1 and P1 are the ini- components models, state dependent models, Gegen-
tial state and state covariance matrix, respectively. Note bauer long memory processes (Dissanayake, Peiris, &
that Dt and Γ t may be parameterised to include some Proietti, 2018).
input variables as linear or non-linear relations to the
Once any researcher or practitioner becomes
output variables yt .
acquainted to a certain degree with the SS technology,
The model in Eq. (1) is a multiple error SS model. A
some important advanced issues in time series forecasting
different formulation is the single error SS model or the
may be comfortably addressed (Casals et al., 2016). It is
innovations SS model. This latter is similar to (1), but
the case, for example, of systems block concatenation,
replacing Rt ηt and Ct ϵt by Kt et and et , respectively. Then,
systems nesting in errors or in variables, treating errors
naturally, the innovations form may be seen as a re-
in variables, continuous time models, time irregularly
stricted version of model (1), but, conversely, under weak
spaced data, mixed frequency models, time varying pa-
assumptions, (1) may also be written as an observation-
rameters, time aggregation, hierarchical and group fore-
ally equivalent innovations form (see, for example, Casals,
casting (Villegas & Pedregal, 2018) (time, longitudinal or
Garcia-Hiernaux, Jerez, Sotoca, & Trindade, 2016, pp. 12–
both), homogeneity of multivariate models (proportional
17).
covariance structure among perturbations), etc.
Once a SS system is fully specified, the core problem is
All in all, the SS systems offer a framework capable
to provide optimal estimates of states and their covari-
of handling many modelling and forecasting techniques
ance matrix over time. This can be done in two ways,
available nowadays in a single environment. Once the ini-
either by looking back in time using the well-known
tial barriers are overcome, a wide panorama of modelling
Kalman filter (useful for online applications) or taking
opportunities opens up.
into account the whole sample provided by smoothing
algorithms (typical of offline applications) (Anderson &
2.3.7. Models for population processes14
Moore, 1979).
Over the past two centuries, formal demography has
Given any set of data and a specific model, the system
established its own, discipline-specific body of meth-
is not fully specified in most cases because it usually
ods for predicting (or projecting 15 ) populations. Popula-
depends on unknown parameters scattered throughout
tion sciences, since their 17th century beginnings, have
the system matrices that define the SS equations. Esti-
been traditionally very empirically focused, with strong
mation of such parameters is normally carried out by
links with probability theory (Courgeau, 2012). Given
Maximum Likelihood defined by prediction error decom-
the observed regularities in population dynamics, and
position (Harvey, 1990).
that populations are somewhat better predictable than
Non-linear and non-Gaussian models are also possible,
many other socio-economic processes, with reasonable
but at the cost of a higher computational burden because
horizons possibly up to one generation ahead (Keyfitz,
more sophisticated recursive algorithms have to be ap-
plied, like the extended Kalman filters and smoothers of 1972, 1981), demographic forecasts have become a best-
different orders, particle filters (Doucet & Gordon, 2001), selling product of the discipline (Xie, 2000). Since the 20th
Unscented Kalman filter and smoother (Julier & Uhlmann,
1997), or simulation of many kinds, like Monte Carlo, 14 This subsection was written by Jakub Bijak.
bootstrapping or importance sampling (Durbin & Koop- 15 The demographic literature sometimes makes a distinction be-
man, 2012). tween unconditional forecasts (or predictions) and projections, condi-
tional on their underlying assumptions. In this section, we use the
The paramount advantage of SS systems is that they
former term to refer to statements about the future, and the latter
are not a particular model or family of models strictly to the result of a numerical exercise of combining assumptions on
speaking, but a container in which many very differ- fertility, mortality and migration in a deterministic model of population
ent model families may be implemented, indeed many renewal.
719
century, methodological developments in human demog- Lutz, Butz, & Samir, 2017), or extrapolated by using prob-
raphy have been augmented by the work carried out in abilistic methods, either for the components or for past
mathematical biology and population ecology (Caswell, errors of prediction (Alho & Spencer, 1985, 2005; De Beer,
2019a). 2008). An impetus to the use of stochastic methods has
The theoretical strength of demography also lies al- been given by the developments in the UN World Popula-
most exclusively in the formal mathematical description tion Prospects (Azose, Ševčíková, & Raftery, 2016; Gerland
of population processes (Burch, 2018), typically growth et al., 2014). Parallel, theoretical advancements included a
functions and structural changes. Historically, such at- stochastic version of the stable population theory (Keiding
tempts started from formulating the logistic model of & Hoem, 1976), as well as coupling of demographic un-
population dynamics, inspired by the Malthusian the- certainty with economic models (Alho, Hougaard Jensen,
ory (Pearl & Reed, 1920; Verhulst, 1845). Lotka’s (1907) & Lassila, 2008).
work laid the foundations of the stable population theory Since its original formulation, the cohort-component
with asymptotic stability under constant vital rates, sub- model has been subject to several extensions (see, for ex-
sequently extended to modelling of interacting popula- ample, Stillwell & Clarke, 2011). The multiregional model
tions by using differential equations (Lotka, 1925; Volterra, (Rogers, 1975) describes the dynamics of multiple re-
1926b). By the middle of the 20th century, the potential gional populations at the same time, with regions linked
and limitations of demographic forecasting methods were through migration. The multistate model (Schoen, 1987)
already well recognised in the literature (Brass, 1974; generalises the multiregional analysis to any arbitrary set
Hajnal, 1955). of states (such as educational, marital, health, or employ-
In the state-of-the-art demographic forecasting, the ment statuses, and so on; see also state-space models in
core engine is provided by matrix algebra. The most com- Section 2.3.6). The multiregional model can be in turn
mon approach relies on the cohort-component models, generalised to include multiple geographic levels of analy-
which combine the assumptions on fertility, mortality and sis in a coherent way (Kupiszewski & Kupiszewska, 2011).
migration, in order to produce future population by age, Recent developments include multifocal analysis, with
sex, and other characteristics. In such models, the deter- an algebraic description of kinship networks (Caswell,
ministic mechanism of population renewal is known, and 2019b, 2020). For all these extensions, however, data
requirements are very high: such models require detailed
results from the following demographic accounting iden-
information on transitions between regions or states in
tity (population balancing equation, see Bryant & Zhang,
a range of different breakdowns. For pragmatic reasons,
2018; Rees & Wilson, 1973):
microsimulation-based methods offer an appealing alter-
P [x + 1, t + 1] = P [x, t ] − D[(x, x + 1), (t , t + 1)] native, typically including large-sample Monte Carlo sim-
+ I [(x, x + 1), (t , t + 1)] − E [(x, x + 1), (t , t + 1)] ulations of population trajectories based on available tran-
sition rates (Bélanger & Sabourin, 2017; Zaidi, Harding, &
where P [x, t ] denotes population aged x at time t, Williamson, 2009).
D[(x, x + 1), (t , t + 1)] refer to deaths between ages x Aside of a few extensions listed above, the current
and x + 1 in the time interval t to t + 1, with I and methodological developments in the forecasting of
E respectively denoting immigration (and other entries) human populations are mainly concentrated on the ap-
and emigration (and other exits). In addition, for the proaches for predicting individual demographic compo-
youngest age group, births B[(t , t + 1)] need to be added. nents (see Sections 3.6.3–3.6.5), rather than the
The equation above can be written up in the general description of the population renewal mechanism. Still,
algebraic form: Pt +1 = GPt , where Pt is the population the continuing developments in population ecology, for
vector structured by age (and other characteristics), and example on the algebraic description of transient and
G is an appropriately chosen growth matrix (Leslie ma- asymptotic population growth (Nicol-Harper, Dooley, Pack-
trix), closely linked with the life table while reflecting man, Mueller, Bijak, Hodgson, Townley, & Ezard, 2018),
the relationship above, expressed in terms of rates rather bear substantial promise of further advancements in this
than events (Caswell, 2019a; Leslie, 1945, 1948; Preston, area, which can be additionally helped by strengthened
Heuveline, & Guillot, 2000). collaboration between modellers and forecasters work-
In the cohort-component approach, even though the ing across the disciplinary boundaries on the formal de-
mechanism of population change is known, the individ- scriptions of the dynamics of human, as well as other
ual components still need forecasting. The three main populations.
drivers of population dynamics — fertility, mortality, and
migration — differ in terms of their predictability (Na- 2.3.8. Forecasting count time series16
tional Research Council, 2000): mortality, which is mainly Probabilistic forecasts based on predictive mass func-
a biological process moderated by medical technology, tions are the most natural way of framing predictions of
is the most predictable; migration, which is purely a a variable that enumerates the occurrences of an event
social and behavioural process is the least; while the pre- over time; i.e. the most natural way of predicting a time
dictability of fertility — part-biological, part-behavioural – series of counts. Such forecasts are both coherent, in the
is in the middle (for component forecasting methods, see sense of being consistent with the discrete support of the
Sections 3.6.3–3.6.5). In practical applications, the com- variable, and capture all distributional – including tail –
ponents can be either projected deterministically, follow-
ing judgment-based or expert assumptions (for example, 16 This subsection was written by Gael M. Martin.
720
information. In contrast, point forecasts based on sum- to unity properties of the probabilities (see also Harris,
mary measures of central location (e.g., a (conditional) Martin, Perera, & Poskitt, 2019, for related, albeit non-
mean, median or mode), convey no such distributional count data work). Model uncertainty is catered for in a
information, and potentially also lack coherence as, for variety of ways: via nonparametric (McCabe et al., 2011)
example, when the mean forecast of the integer-valued or bootstrapping (Bisaglia & Gerolimetto, 2019) methods;
count variable assumes non-integer values. These com- via (frequentist) model averaging (Sun et al., 2021); or
ments are even more pertinent for low count time series, via an informal comparison of predictive results across
in which the number of rare events is recorded, and for alternative models (Jung & Tremayne, 2006). Methods de-
which the cardinality of the support is small. In this case, signed explicitly for calibrating predictive mass functions
point forecasts of any sort can be misleading, and contin- to observed count data – whether those functions be
uous (e.g., Gaussian) approximations (sometimes adopted produced using frequentist or Bayesian methods – can be
for high count time series) are particularly inappropriate. found in Czado, Gneiting, and Held (2009) and Wei and
These points were first elucidated in Freeland and Mc- Held (2014); see also Sections 2.12.4 and 2.12.5.
Cabe (2004), and their subsequent acceptance in the liter- Finally, whilst full probabilistic forecasts are increas-
ature is evidenced by the numerous count data types for ingly common, point, interval and quantile forecasts are
which discrete predictive distributions are now produced; certainly still used. The need for such summaries to be
including counts of: insurance claims (McCabe & Martin, coherent with the discrete nature of the count variable
2005), medical injury deaths (Bu & McCabe, 2008), web- appears to be now well-accepted, with recent work em-
site visits (Bisaglia & Canale, 2016), disease cases (Bisaglia phasising the importance of this property (Bu & McCabe,
& Gerolimetto, 2019; Mukhopadhyay & Sathish, 2019; 2008; Homburg, Weiß, Alwan, Frahm, & Göb, 2019; Hom-
Rao & McCabe, 2016), banking crises (Dungey, Martin, burg et al., 2020; Mukhopadhyay & Sathish, 2019).
Tang, & Tremayne, 2020), company liquidations (Hom-
burg, Weiß, Alwan, Frahm, & Göb, 2020), hospital emer- 2.3.9. Forecasting with many variables17
gency admissions (Sun, Sun, Zhang, & McCabe, 2021), Multivariate models – regression models with multi-
work stoppages (Weiß, Homburg, Alwan, Frahm, & Göb, ple explanatory variables – are often based on available
2021), and the intermittent product demand described in theories regarding the determination of the variable to
Section 2.8 (Berry & West, 2020; Kolassa, 2016; Snyder, be forecast, and are often referred to as structural models.
Ord, & Beaumont, 2012). In a stationary world without structural change, then it
The nature of the predictive model for the count vari- would be anticipated that the best structural model would
able, together with the paradigm adopted (Bayesian or provide the best forecasts, since it would provide the
frequentist), determine the form of the probabilistic fore- conditional mean of the data process (see, for example,
cast, including the way in which it does, or does not, Clements & Hendry, 1998). In a non-stationary world of
accommodate parameter and model uncertainty. As high- unit roots and structural breaks, however, this need not
lighted in Sections 2.4.1 and 2.4.2, the Bayesian approach be the case. In such situations, often simple forecast mod-
to forecasting is automatically probabilistic, no matter els can outperform structural models, especially at short
what the data type. It also factors parameter uncertainty forecast horizons (see, for example, Hendry & Clements,
into the predictive distribution, plus model uncertainty 2001). Multivariate forecast models require that explana-
if Bayesian model averaging is adopted, producing a dis- tory variables also be forecast – or at least, scenarios be
tribution whose location, shape and degree of dispersion set out for them. These may be simplistic scenarios, for
reflect all such uncertainty as a consequence. See Berry example all explanatory variables take their mean values.
and West (2020), Bisaglia and Canale (2016), Frazier, Such scenarios can play a useful role in formulating policy
Maneesoonthorn, Martin, and McCabe (2019), Lu (2021), making since they illustrate in some sense the outcomes
McCabe and Martin (2005) and Neal and Kypraios (2015), of different policy choices.
for examples of Bayesian probabilistic forecasts of counts. Since the 1980s and Sims (1980), vector autoregres-
In contrast, frequentist probabilistic forecasts of counts sive (VAR) models have become ubiquitous in macroeco-
typically adopt a ‘plug-in’ approach, with the predictive nomics, and common in finance (see, for example, Has-
distribution conditioned on estimates of the unknown brouck, 1995). A VAR model is a set of linear regression
parameters of a given count model. Sampling variation in equations (see also Section 2.3.2) describing the evolution
the estimated predictive (if acknowledged) is quantified of a set of endogenous variables. Each equation casts each
in a variety of ways. Freeland and McCabe (2004), for variable as a function of lagged values of all the vari-
instance, produce confidence intervals for the true (point- ables in the system. Contemporaneous values of system
wise) predictive probabilities, exploiting the asymptotic variables are not included in VAR models for identifi-
distribution of the (MLE-based) estimates of those prob- cation purposes; some set of identifying restrictions are
abilities. Bu and McCabe (2008) extend this idea to (cor- required, usually based on economic theory, and when
related) estimates of sequential probabilities, whilst Jung imposed the resulting model is known as a structural VAR
and Tremayne (2006) and Weißet al. (2021) exploit boot- model. VAR models introduce significantly greater levels
strap techniques to capture point-wise sampling variation of parameterisation of relationships, which increases the
in the forecast distribution. McCabe, Martin, and Harris level of estimation uncertainty. At the same time VAR
(2011), on the other hand, use subsampling methods to models afford the forecaster a straightforward way to
capture sampling fluctuations in the full predictive dis-
tribution, retaining the non-negativity and summation 17 This subsection was written by J. James Reade.
721
generate forecasts of a range of variables, a problem when time series can arise from measurements obtained by sep-
forecasting with many variables. As with autoregressive arating an almost continuous time record into consecutive
methods, VAR models can capture a significant amount intervals (e.g., days or years, see Horváth & Kokoszka,
of variation in data series that are autocorrelated, and 2012). We refer to such data structure as sliced functional
hence VAR methods can be useful as baseline forecasting time series, examples of which include daily precipitation
devices. VAR-based forecasts are often used as a bench- data (Gromenko, Kokoszka, & Reimherr, 2017). On the
mark for complex models in macroeconomics like DSGE other hand, when the continuum is not a time variable,
models (see, for example, Del Negro & Schorfheide, 2006). functional time series can also arise when observations
The curse of dimensionality in VAR models is particu- over a period are considered as finite-dimensional reali-
larly important and has led to developments in factor- sations of an underlying continuous function (e.g., yearly
augmented VAR models, with practitioners often reducing age-specific mortality rates, see Li, Robinson, & Shang,
down hundreds of variables into factors using principal 2020e).
component analysis (see, for example, Bernanke, Boivin, & Thanks to recent advances in computing storage, func-
Eliasz, 2005). Bayesian estimation is often combined with tional time series in the form of curves, images or shapes
factor-augmented VAR models. is common. As a result, functional time series analysis has
Often, significant numbers of outliers and structural received increasing attention. For instance, Bosq (2000)
breaks require many indicator variables to be used to and Bosq and Blanke (2007) proposed the functional au-
model series (see also Sections 2.2.3 and 2.2.4). Indicator toregressive of order 1 (FAR(1)) and derived one-step-
saturation is a method of detecting outliers and structural ahead forecasts that are based on a regularised Yule–
breaks by saturating a model with different types of indi- Walker equations. FAR(1) was later extended to FAR(p),
cator, or deterministic variables (Castle, Doornik, Hendry, under which the order p can be determined via Kokoszka
& Pretis, 2015c; Johansen & Nielsen, 2009). The flexibility and Reimherr’s (2013) hypothesis testing. Horváth, Liu,
of the approach is such that it has been applied in a Rice, and Wang (2020) compared the forecasting per-
wide variety of contexts, from volcanic eruptions (Pretis, formance between FAR(1), FAR(p), and functional sea-
Schneider, & Smerdon, 2016) to prediction markets and sonal autoregressive models of Chen, Marron, and Zhang
social media trends (Vaughan Williams & Reade, 2016). (2019b).
A particularly important and ever-expanding area of To overcome the curse of dimensionality (see also
empirical analysis involves the use of panel data sets Sections 2.2.5, 2.5.2 and 2.5.3), a dimension reduction
with long time dimensions: panel time series (Eberhardt, technique, such as functional principal component anal-
2012). The many variables are then extended across many ysis (FPCA), is often used. Aue, Norinho, and Hörmann
cross sectional units, and a central concern is the depen- (2015) showed asymptotic equivalence between a FAR
dence between these units, be they countries, firms, or in- and a VAR model (for a discussion of VAR models, see
dividuals. At the country level one approach to modelling Section 2.3.9). Via an FPCA, Aue et al. (2015) proposed a
this dependence has been the Global VAR approach of, for forecasting method based on the VAR forecasts of prin-
example, Dees, Mauro, Pesaran, and Smith (2007). In more cipal component scores. This approach can be viewed as
general panels, the mean groups estimator has been pro- an extension of Hyndman and Shang (2009), in which
posed to account for cross-section dependence (Pesaran, principal component scores are forecast via a univari-
Shin, & Smith, 1999). ate time series forecasting method. With the purpose
Outliers, structural breaks, and split trends undoubt- of forecasting, Kargin and Onatski (2008) proposed to
edly also exist in panel time series. The potential to test estimate the FAR(1) model by using the method of pre-
for common outliers and structural changes across cross dictive factors. Klepsch and Klüppelberg (2017) proposed
sectional units would be useful, as would the ability to a functional moving average process and introduced an
allow individual units to vary individually, e.g., time- innovations algorithm to obtain the best linear predic-
varying fixed effects. Nymoen and Sparrman (2015) is tor. Klepsch, Klüppelberg, and Wei (2017) extended the
the first application of indicator saturation methods in VAR model to the vector autoregressive moving average
a panel context, looking at equilibrium unemployment model and proposed the functional autoregressive moving
dynamics in a panel of OECD countries, but applications average model. The functional autoregressive moving av-
into the panel context are somewhat constrained by com- erage model can be seen as an extension of autoregressive
puter software packages designed for indicator saturation integrated moving average model in the univariate time
series literature (see Section 2.3.4).
(Section 3.3.3 discusses further the case of forecasting
Extending short-memory to long-memory functional
unemployment). The gets R package of Pretis, Reade, and
time series analysis, Li et al. (2020e) and Li, Robinson,
Sucarrat (2017, 2018) can be used with panel data.
and Shang (2021) considered local Whittle and rescale-
range estimators in a functional autoregressive fraction-
2.3.10. Functional time series models18
ally integrated moving average model. The models men-
Functional time series consist of random functions ob-
tioned above require stationarity, which is often rejected.
served at regular time intervals. Functional time series can
Horváth, Kokoszka, and Rice (2014) proposed a functional
be classified into two categories depending on if the con-
KPSS test for stationarity. Chang, Kim, and Park (2016)
tinuum is also a time variable. On the one hand, functional
studied nonstationarity of the time series of state densi-
ties, while Beare, Seo, and Seo (2017) considered a coin-
18 This subsection was written by Han Lin Shang. tegrated linear process in Hilbert space. Nielsen, Seo, and
722
Seong (2019) proposed a variance ratio-type test to de- The GARCH model has been extended in the diverse
termine the dimension of the nonstationary subspace in aspects of non-linearity, asymmetry and long memory.
a functional time series. Li, Robinson, and Shang (2020d) Among many such extensions, the Exponential GARCH
studied the estimation of the long-memory parameter in (EGARCH) model by Nelson (1991) uses log transforma-
a functional fractionally integrated time series, covering tion to prevent negative variance; the Threshold GARCH
the functional unit root. (TGARCH) model by Zakoian (1994) allows for different
From a nonparametric perspective, Besse, Cardot, and responses on positive and negative shocks. A small piece
Stephenson (2000) proposed a functional kernel regres- of information can have more impact when the time se-
sion method to model temporal dependence via a simi- ries is under stress than under a stable time series (Engle,
larity measure characterised by semi-metric, bandwidth 2004). Another pattern often observed in the volatility
and kernel function. Aneiros-Pérez and Vieu (2008) in- time series is slowly decaying autocorrelation, also known
troduced a semi-functional partial linear model that com- as a long memory pattern, which Baillie, Bollerslev, and
bines linear and nonlinear covariates. Apart from condi- Mikkelsen (1996) capture using a slow hyperbolic rate
tional mean estimation, Hörmann, Horváth, and Reeder of decay for the ARCH terms in the fractionally inte-
(2013) considered a functional autoregressive conditional grated GARCH (FIGARCH) model. Separately, in a fur-
heteroscedasticity model for estimating conditional vari- ther approach to directly estimating long term volatility,
ance. Rice, Wirjanto, and Zhao (2020) proposed a condi- the GARCH-MIDAS (Mixed Data Sampling) model pro-
tional heteroscedasticity test for functional data. posed by Engle, Ghysels, and Sohn (2013) decomposes
Kokoszka, Rice, and Shang (2017) proposed a portman- the conditional volatility into the short-term volatility,
teau test for testing autocorrelation under a functional as captured by the traditional GARCH, and the long-term
generalised autoregressive conditional heteroscedasticity volatility represented by the realised volatilities. The Het-
model. erogeneous Autoregressive (HAR) model by Corsi (2009)
considers the log-realised volatility as a linear function
of the log-realised volatility of yesterday, last week and
2.3.11. ARCH/GARCH models19
last month to reflect traders’ preferences on different
Volatility has been recognised as a primary measure
horizons in the past. This model is extended by Wilms,
of risks and uncertainties (Gneiting, 2011a; Markowitz,
Rombouts, and Croux (2021) to incorporate information
1952; Sharpe, 1964; Taylor, McSharry, & Buizza, 2009);
about future stock market volatility by further including
for further discussion on uncertainty estimation, see Sec-
option-implied volatility. A different approach to volatility
tion 2.3.21. Estimating future volatility for measuring the
modelling, discussed in Section 2.3.14, is the use of low
uncertainty of forecasts is imperative for probabilistic
and high prices in the range-based volatility models.
forecasting. Yet, the right period in which to estimate
The univariate GARCH models surveyed so far have
future volatility has been controversial as volatility based
been exended to multivariate versions, in order to model
on too long a period will make irrelevant the forecast
changes in the conditional covariance in multiple time
horizon of our interests, whereas too short a period re-
series, resulting in such examples as the VEC (Bollerslev,
sults in too much noise (Engle, 2004). An alternative to
1987) and BEKK (Engle & Kroner, 1995), an acronym de-
this issue is the dynamic volatility estimated through the
rived from Baba, Engle, Kraft, and Kroner. The VEC model,
autoregressive conditional heteroscedasticity (ARCH) pro-
a direct generalisation of the univariate GARCH, requires
posed by Engle (1982), and the generalised autoregressive more parameters in the covariane matrices and provides
conditional heteroscedasticity (GARCH) model proposed better fitness at the expense of higher estimation costs
by Bollerslev (1987). The ARCH model uses the weighted than the BEKK. The VEC model has to ensure the posi-
average of the past squared forecast error whereas the tivity of the covariance matrix with further constraints,
GARCH model generalises the ARCH model by further whereas the BEKK model and its specific forms, e.g., factor
adopting past squared conditional volatilities. The GARCH models, avoid this positivity issue directly at the model
model is the combination of (i) a constant volatility, specification stage. In an effort to further reduce the num-
which estimates the long-run average, (ii) the volatility ber of parameters to be estimated, the linear and non-
forecast(s) in the last steps, and (iii) the new informa- linear combinations of the univariate GARCH models, such
tion collected in the last steps. The weightings on these as the constant conditional correlation model of Boller-
components are typically estimated with maximum likeli- slev (1990) and the dynamic conditional correlation mod-
hood. The models assume a residual distribution allowing els of Tse and Tsui (2002) and of Engle (2002), were
for producing density forecasts. One of the benefits of the investigated.
GARCH model is that it can model heteroscedasticity, the
volatility clustering characteristics of time series (Mandel- 2.3.12. Markov switching models20
brot, 1963), a phenomenon common to many time series Since the late 1980s, especially in macroeconomics and
where uncertainties are predominant. Volatility clustering finance, the applications of dynamic econometric mod-
comes about as new information tends to arrive time elling and forecasting techniques have increasingly relied
clustered and a certain time interval is required for the on a special class of models that accommodate regime
time series to be stabilised as the new information is shifts, Markov switching (MS) models. The idea of MS is
initially recognised as a shock. to relate the parameters of otherwise standard dynamic
19 This subsection was written by Jooyoung Jeon. 20 This subsection was written by Massimo Guidolin.
723
econometric frameworks (such as systems of regressions, 2.3.13. Threshold models21

vector autoregressions, and vector error corrections) to It is a well-known fact that financial and economic
one or more unobserved state variables (see Section 2.3.6 time series often display non-linear patterns, such as
for a definition), say, St , that can take K values and capture structural instability, which may appear in the form of
the notion of systems going through phases or ‘‘regimes’’, recurrent regimes in model parameters. In the latter case,
which follow a simple, discrete stochastic process and are such instability is stochastic, it displays structure, and
independent of the shocks of the model. as such, it can be predicted. Accordingly, modelling eco-
For instance, an otherwise standard AR(1) model can nomic and financial instability has become an essential
be extended to yt = φ0,St + φ1,St y(t −1) + σSt ϵt , where goal for econometricians since the 1970s.
all the parameters in the conditional mean as well as the One of the first and most popular models is the thresh-
variance of the shocks may be assumed to take different, old autoregressive (TAR) model developed by Tong (1978).
estimable values as a function of St . Similarly, in a K - A TAR model is an autoregressive model for the time
regime MS VAR(p), the vector of intercepts and the p series yt in which the parameters are driven by a state
autoregressive matrices may be assumed to depend on variable St (see Section 2.3.6 for a definition), which is
St . Moreover, the covariance matrix of the system shocks itself a random variable taking K distinct integer values
may be assumed to depend on some state variable, either (i.e., St = k, k = 1, . . . , K ). In turn, the value assumed
the same as the mean parameters (St ) or an additional, by St depends on the value of the threshold variable qt
specific one (Vt ), which may depend on lags of St . When when compared to K − 1 threshold levels, q∗k . For instance,
a MS VAR model is extended to include exogenous regres- if only two regimes exists, it is St = 1 if qt ≤ q∗1
sors, we face a MS VARX, of which MS regressions are just and St = 2 otherwise. The threshold variable qt can be
a special case. exogenous or can be a lagged value of yt . In the latter
Even though multivariate MS models may suffer from case, we speak of self-exciting threshold autoregressive
issues of over-parameterisations that must be kept in (SETAR) models. Other choices of qt include linear (Chen,
check, their power of fitting complex non-linearities is Chiang, & So, 2003; Chen & So, 2006; Gerlach, Chen, Lin,
unquestioned because, as discussed by Marron and Wand
& Huang, 2006) or non-linear (Chen, 1995; Wu & Chen,
(1992), mixtures of normal distributions provide a flexible
2007) combinations of the lagged dependent variable or
family that can be used to approximate many distribu-
of exogenous variables.
tions. Moreover, MS models are known (Timmermann,
The TAR model has also been extended to account for
2000) to capture key features of many time series. For
different specifications of the conditional mean function
instance, differences in conditional means across regimes
leading to the development of the threshold moving aver-
enter the higher moments such as variance, skewness,
age (TMA – see, for example, De Gooijer, 1998; Ling, Tong,
and kurtosis; differences in means in addition to differ-
& Li, 2007; Tong, 1990) and the threshold autoregressive
ences in variances across regimes may generate persis-
moving average (TARMA – see, for example, Amendola,
tence in levels and squared values of the series.
Niglio, & Vitale, 2006; Ling, 1999) models. Those models
The mainstream literature (see, e.g., Hamilton 1990,
are similar to the ones described in Section 2.3.4, but their
or the textbook treatments by Kim, Charles, and Nel-
son 1999, and Guidolin and Pedio 2018) initially focused parameters depend on the regime K .
on time-homogeneous Markov chains (where the prob- A criticism of TAR models is that they imply a con-
abilities of the state transitions are constant). However, ditional moment function that fails to be continuous. To
the finance and business cycles literatures (Gray, 1996) address this issue, Chan and Tong (1986) proposed the
has moved towards time-heterogeneous MS models, in smooth transition autoregressive (STAR) model. The main
which the transition matrix of the regimes may change difference between TAR and STAR models is that, while
over time, reacting to lagged values of the endogenous a TAR imposes an abrupt shift from one regime to the
variables, to lagged exogenous variables, or to the lagged others at any time that the threshold variable crosses
values of the state (in a self-exciting fashion). above (or below) a certain level, a STAR model allows for
MS models may be estimated by maximum likelihood, gradual changes among regimes.
although other estimation methods cannot be ruled out, In its simplest form, a STAR is a two-regime model
like GMM (Lux, 2008). Typically, estimation and infer- where the dependent variable yt is determined as the
ence are based on the Expectation-Maximisation algo- weighted average of two autoregressive (AR) models, i.e.,
rithm proposed by Dempster, Laird, and Rubin (1977), a ∑
yt = +j = 1p φj,1 yt −j P(St = 1; g(xt ))
filter that allows the iterative calculation of the one-step
ahead forecast of the state vector given the information p
∑
set and a simplified construction of the log-likelihood + φj,2 yt −j P(St = 2; g(xt )) + ϵt ,
of the data. However, there is significant evidence of j=1
considerable advantages offered by Bayesian approaches
where xt is the transition variable and g is some transfor-
based on Monte Carlo Markov chain techniques to esti-
mation of the transition variable xt . Regime probabilities
mating multivariate MS models (see, for example, Hahn,
are assigned through the transition function
Frühwirth-Schnatter, & Sass, 2010).
F (k; g(xt )), with F being a cumulative density function
Notably, MS models have been recently generalised
of choice. The transition variable xt can be the lagged
in a number of directions, such as including regimes in
conditional variance functions, for example of a GARCH
or DCC type (see Pelletier, 2006, and Section 2.3.11). 21 This subsection was written by Manuela Pedio.
724
endogenous variable, yt −d for d ≥ 1 (Teräsvirta, 1994), a Gerlach, & Lin, 2008), CARGPR (Chan, Lam, Yu, Choy, &
(possibly non-linear) function of it, or an exogenous vari- Chen, 2012), STARR (Lin, Chen, & Gerlach, 2012), and
able. The transition variable can also be a linear time trend MSRB (Miao, Wu, & Su, 2013). The last group is method-
(xt = t), which generates a model with smoothly chang- ologically different because the estimation of model pa-
ing parameters (Lin & Teräsvirta, 1994). Popular choices rameters is based on the sets of three prices, i.e., low,
for the transition function F are the logistic function high and closing. This approach comprises the GARCH
(which gives rise to the LSTAR model) and the exponen- models (Fiszeder & Perczak, 2016; Lildholdt, 2002; Venter,
tial function (ESTAR). Notably, the simple STAR model De Jongh, & Griebenow, 2005) and the SV model (Horst,
we have described can be generalised to have multiple Rodriguez, Gzyl, & Molina, 2012).
regimes (Van Dijk, Franses, & Lucas, 1999). The development of multivariate models with LH prices
Threshold models are also applied to modelling and has taken place in the last few years. They can be divided
forecasting volatility; for instance, the GJR-GARCH model into three groups. The first one includes models, used tra-
of Glosten, Jagannathan, and Runkle (1993) can be inter- ditionally to describe returns or prices, but they are based
preted as a special case of a threshold model. A few multi- on the price range or RB variance estimators. They com-
variate extensions of threshold models also exist, such as prise such models like multivariate EWMA, VAR, HVAR,
vector autoregressive threshold models, threshold error and vector error correction (VEC). It is a simple approach,
correction models (Balke & Fomby, 1997), and smooth however most models omit modelling the covariance of
transition vector error correction models (Granger & Swan- returns. The second group is formed by the multivariate
son, 1996). RB volatility models like RB-DCC (Chou, Wu, & Liu, 2009),
DSTCC-CARR (Chou & Cai, 2009), RR-HGADCC (Asai, 2013),
2.3.14. Low and high prices in volatility models22 RB-MS-DCC (Su & Wu, 2014), DCC-RGARCH (Fiszeder,
Volatility models of financial instruments are largely Fałdziński, & Molnár, 2019), RB-copula (Chiang & Wang,
based solely on closing prices (see Section 2.3.11); mean- 2011; Wu & Liang, 2011). The third group includes the
while, daily low and high (LH) prices significantly in- multivariate co-range volatility models like multivariate
crease the amount of information about the variation CARR (Fernandes, de Sá Mota, & Rocha, 2005), BEKK-
of returns during a day. LH prices were used for the HL (Fiszeder, 2018) and co-range DCC (Fiszeder & Fałdz-
construction of highly efficient estimators of variance, so iński, 2019). These models apply LH prices directly not
called the range-based (RB) estimators (e.g., Fiszeder & only for the construction of variances of returns but also
Perczak, 2013; Garman & Klass, 1980; Magdon-Ismail & for covariances. Section 3.3.9 discusses the use of the
Atiya, 2003; Parkinson, 1980; Rogers & Satchell, 1991; range-based volatility models in financial time series fore-
Yang & Zhang, 2000). Recently, Riedel (2021) analysed casting.
how much additional information about LH reduces the
time averaged variance in comparison to knowing only 2.3.15. Forecasting with DSGE models23
open and close. RB variance estimators, however, have Dynamic Stochastic General Equilibrium (DSGE) mod-
a fundamental drawback, as they neglect the temporal els are the workhorse of modern macroeconomics em-
dependence of returns (like conditional heteroscedastic- ployed by monetary and fiscal authorities to explain and
ity) and do not allow for the calculation of multi-period forecast comovements of aggregate time series over the
dynamic volatility forecasts. business cycle and to perform quantitative policy anal-
In the last dozen or so years, numerous univariate ysis. These models are studied in both academia and
dynamic volatility models have been constructed based policy-making institutions (for details, see: Christiano,
on LH prices. Some of them were presented in the re- Eichenbaum, & Trabandt, 2018; Del Negro & Schorfheide,
view paper of Chou, Chou, and Liu (2015). These models 2013; Paccagnini, 2017). For example, the European Cen-
can be divided into four groups. The first one comprises tral Bank uses the New Area-Wide Model introduced
simple models, used traditionally to describe returns, but by Warne, Coenen, and Christoffel (2010) and the Fed-
they are based on the price range or on the mentioned eral Reserve Board has created the Estimated, Dynamic,
earlier RB variance estimators. They include such models Optimisation-based model (FRB/EDO) as discussed in
as random walk, moving average, exponentially weighted Chung, Kiley, and Laforte (2010). For an application on
moving average (EWMA), autoregressive (AR), autore- forecasting GDP and inflation, see Section 3.3.2. Devel-
gressive moving average (ARMA; see Section 2.3.4), and oped as a response to Lucas (1976) critique of structural
heterogeneous autoregressive (HAR). The second group macroeconometrics models, DSGEs introduced microfun-
contains models which describe the conditional variance dations to describe business cycle fluctuations.
(or standard deviation) of returns. It comprises models Initially calibrated, estimated DSGEs have been employed
like GARCH-PARK-R (Mapa, 2003), GARCH-TR (Fiszeder, in shocks identification and forecasting horseraces for
2005), REGARCH (Brandt & Jones, 2006), RGARCH (Molnár, the last 15 years. Estimation became possible thanks to
2016). The third group includes models which describe computational progress and adoption of Bayesian tech-
the conditional mean of the price range. It means that in niques (for technical details, see: An & Schorfheide, 2007;
order to forecast variance of returns the results have to be Fernández-Villaverde & Guerrón-Quintana, 2020; Herbst
scaled. This group contains models like RB SV (Alizadeh, & Schorfheide, 2016). Bayesian estimation allows for at-
Brandt, & Diebold, 2002), CARR (Chou, 2005), TARR (Chen, tributing prior distributions, instead of calibrating, and
22 This subsection was written by Piotr Fiszeder. 23 This subsection was written by Alessia Paccagnini.
725
computing the posterior distribution for selected model commonly includes models with explicit equilibrium-
parameters as well as drawing from predictive density. correction mechanisms such as vector equili-
The Smets and Wouters (2007) DSGE is the most pop- brium-correction models (VEqCM) as well as models with
ular framework referred to in both research and policy implicit equilibrium-correction (or long-run mean rever-
literature. Proposed for the US economy, this medium- sion) mechanisms such as vector auto-regressions (VARs;
scale model is a closed economy composed of households, see Section 2.3.9), dynamic factor models (DFMs), dy-
labor unions, a productive sector, and a monetary policy namic stochastic general-equilibrium (DSGE) models (see
authority that sets the short-term interest rate according Section 2.3.15), most models of the variance (see Sec-
to a Taylor rule. These ingredients are mathematically tion 2.3.11), and almost all regression equations (see
represented by a system of linear rational expectation Sections 2.3.2 and 2.3.4). This class of forecast model
equations. Using a solution algorithm (for example, Blan- is prevalent across most disciplines. For example, Pretis
chard & Kahn, 1980; Sims, 2002), researchers can write (2020) illustrates that there is an equivalence between
the model using the state-space representation composed physical energy balance models, which are used to ex-
by the transition equation and the measurement equa- plain and predict the evolution of climate systems, and
tion. The latter matches the observed data (in the Smets VEqCMs.
and Wouters: output growth rate, consumption, invest- Despite their wide-spread use in economic modeling
ment, wages, worked hours, inflation, and short-term and forecasting, equilibrium-correction models often pro-
interest rate) with the model latent variables. The solved duce forecasts that exhibit large and systematic forecast
model is employed for quantitative policy analysis and to errors. Clements and Hendry (1998, 1999) showed that
predict and explain the behavior of macroeconomic and
forecasts from equilibrium-correction models are not ro-
financial indicators.
bust to abrupt changes in the equilibrium. These types of
DSGE models forecasting performance is investigated
regime changes are very common in macroeconomic time
along two dimensions: point forecast and density forecast
series (see Hamilton, 2016, as well as Section 2.3.12) and
(see Sections 2.12.2 and 2.12.4 for discussions on their
can cause the forecasts from many models to go off track.
evaluation).
Therefore, if for example, there is a change in the equilib-
The point forecast is implemented by conducting both
rium towards the end of the estimation sample, forecasts
static and dynamic analysis, as described in
from this class of models will continue to converge back
Cardani, Paccagnini, and Villa (2019). If the static analysis
provides a unique forecast value, the dynamic analysis to the previous equilibrium.
describes the evolution of the prediction along the time In general, the forecasts from equilibrium-correction
dimension to investigate possible time-varying effects. models can be robustified by estimating all model pa-
Usually, point predictions are compared using the Diebold rameters over smaller or more flexible sub-samples. Sev-
and Mariano (1995) and the Clark and West (2006) tests eral studies have proposed general procedures that al-
that compare predictions from two competing models low for time-variation in the parameters; see, for ex-
The accuracy of the static analysis is based mainly on ample, Giraitis, Kapetanios, and Price (2013), Inoue, Jin,
Mean Absolute Error (MAE) and Root Mean Square Error and Rossi (2017) and Pesaran, Pick, and Pranovich (2013).
(RMSE). MAE and RMSE are used to provide a relative This allows for an optimal or more adaptive selection of
forecasting evaluation compared to other competitors. model estimation windows in order generate forecasts
Following Clements and Hendry (1998), Kolasa, Rubaszek, after structural breaks have occurred.
and Skrzypczynśki (2012) apply the standard forecast An alternative approach for robustifying forecasts from
unbiased test to assess if DSGEs are good forecasters in equilibrium-correction models is to focus on the formu-
the absolute sense. The accuracy of the dynamic anal- lation and estimation of the equilibrium. Hendry (2006)
ysis is based on the Fluctuation Test (for some DSGE shows that differencing the equilibrium-correction mech-
applications, see: Boneva, Fawcett, Masolo, & Waldron, anism can improve the forecasts by removing aspects
2019; Cardani et al., 2019; Giacomini & Rossi, 2016). which are susceptible to shifts. However, differencing the
This test is based on the calculation of RMSEs that are equilibrium also induces additional forecast-error vari-
assessed to investigate if the forecasting performance can ance. Castle, Fawcett, and Hendry (2010) show that it is
be influenced by instabilities in the model parameters. beneficial to update the equilibrium or to incorporate the
The density forecast is based on the uncertainty de- underlying structural break process. Alternatively, Castle,
rived by the Bayesian estimation and it is commonly Clements, and Hendry (2015a) show that there can be
evaluated using the probability integral transform and the large forecast gains from smoothing over estimates of
log predictive density scores (as main references, Kolasa the transformed equilibrium. Building on this, Martinez,
& Rubaszek, 2015a; Wolters, 2015). The statistical signifi- Castle, and Hendry (2021) show that there are many pos-
cance of these predictions is evaluated using the Amisano sible transformations of the equilibrium that can improve
and Giacomini (2007) test that compares log predictive the forecasts. Several of these transformations imply that
density scores from two competing models. the equilibrium-correction model collapses to different
naive forecast devices whose forecasts are often difficult
2.3.16. Robust equilibrium-correction forecasting devices24 to beat. By replacing the equilibrium with smooth esti-
The use of equilibrium-correction models is ubiqui- mates of these transformations, it is possible to outper-
tous in forecasting. Hendry (2010) notes that this class form the naive forecasts at both short and long forecast
horizons while retaining the underlying economic the-
24 This subsection was written by Andrew B. Martinez. ory embedded within the equilibrium-correction model.
726
Thus, it is possible to dramatically improve forecasts from Other approaches seek to model the data revisions pro-
equilibrium-correction models using targeted transforma- cess along with the fully-revised true values of the data, as
tions of the estimated equilibrium so that it is less sus- in Cunningham, Eklund, Jeffery, Kapetanios, and Labhard
ceptible to the shifts which are so damaging to the model (2009), Jacobs and van Norden (2011) and Kishor and
forecasts. Koenig (2012). Reduced form models that avoid the ne-
cessity of estimating unobserved components have
2.3.17. Forecasting with data subject to revision25 adapted the vector autoregression (VAR; see also Sec-
When a forecast is made today of the future value of tion 2.3.9) of Sims (1980) to jointly model different ob-
a variable, the forecast is necessarily ‘real time’ – only served vintages of data. Following Patterson (1995), Gar-
information available at the time the forecast is made ratt, Lee, Mise, and Shields (2008) work in terms of the
can be used. The forecasting ability of a model can be level of the log of(the variable, Ytt +1 , and model the vector )′
evaluated by mimicking this setup – generating forecasts given by Zt +1 = Ytt +1 − Ytt−1 , Ytt−+11 − Ytt−1 , Ytt−+21 − Ytt−2 .
over some past period (so outcomes known) only using Carriero, Clements, and Galvão (2015) and Clements and
data known at each forecast origin. As noted by Clements Galvão (2012, 2013a) minimise the effects of benchmark
and Hendry (2005), out-of-sample forecast performance is revisions and re-basing by(modelling ‘same-vintage-growth
the gold standard. Sometimes the analysis is pseudo real )′
rates’, namely Zt +1 = ytt +1 , ytt + −1 , . . . , yt −q+1 , where
1 t +1
time. At a given forecast origin t, forecasts are constructed
only using data up to the period t, but the data are ytt +1 = Ytt +1 − Ytt−+11 , and q denotes the greatest data
taken from the latest-available vintage at the time the maturity.
study is undertaken. Using revised data to estimate the Galvão (2017) shows how forecasts of fully-revised
forecasting model – instead of the data available at the data can be generated for dynamic stochastic general
time the forecast was made – may exaggerate forecast equilibrium (DSGE; Section 2.3.15) models (for example,
performance, and present a misleading picture of how Del Negro & Schorfheide, 2013), by applying the approach
well the model might perform in real time. The improved of Kishor and Koenig (2012). Clements (2017) argues that
availability of real-time databases has facilitated proper improvements in forecast accuracy might be expected to
real-time studies.26 At time t the data are taken from be greater for interval or density forecasts than point
the vintage available at time t. Data revisions are often forecasts, and this is further explored by Clements and
important, and occur because statistical agencies strive to Galvão (2017).
provide timely estimates which are based on incomplete Surveys on data revisions and real-time analysis, in-
source data (see, for example, Fixler & Grimm, 2005, 2008; cluding forecasting, are provided by Clements and Galvão
Zwijnenburg, 2015). (2019) and Croushore (2006, 2011a, 2011b); see also Sec-
There are a number of possible real-time approaches. tion 3.3.1.
The conventional approach is to estimate the forecasting
model using the latest vintage of data available at time t. 2.3.18. Innovation diffusion models27
Suppose the vintage-t contains data for time periods up Forecasting the diffusion of innovations is a broad field
to t − 1, denoted . . . , ytt −3 , ytt −2 , ytt −1 . The observation for of research, and influential reviews on the topic have
time t − 1 is a first estimate, for t − 2 a second estimate, highlighted its importance in many disciplines for strate-
and so on, such that data for earlier periods will have been gic or anticipative reasons (Mahajan, Muller, & Bass, 1990;
revised many times. Hence the model will be estimated Meade & Islam, 2006; Peres, Muller, & Mahajan, 2010).
on data of different maturities, much of which will have Large-scale and fast diffusion processes of different na-
been revised many times. But the forecast will typically ture, ranging from the spread of new epidemics to the
be generated by feeding into the model ‘lightly-revised’ adoption of new technologies and products, from the
data for the most recent time periods. The accuracy of the fast diffusion of news to the wide acceptance of new
resulting forecasts can be improved upon (in principle) trends and social norms, are demanding a strong effort in
by taking into account data revisions (see, for example, terms of forecasting and control, in order to manage their
Clements & Galvão, 2013b; Kishor & Koenig, 2012; Koenig, impact into socio-economic, technological and ecological
Dolmas, & Piger, 2003). In the following two paragraphs, systems.
we consider alternative real-time approaches which solve The formal representation of diffusion processes is of-
the problem of estimating the model on mainly revised ten based on epidemic models, under the hypothesis that
data, and feeding in mainly unrevised forecast origin data. an innovation spreads in a social system through com-
Koenig et al. (2003) suggest using real-time-vintage munication among people just like an epidemics does
(RTV) data to estimate the model. The idea is to use early through contagion. The simplest example is represented
estimates of the data to estimate the model, so that the by the (cumulative) logistic equation that describes a pure
model is estimated on ‘lightly-revised’ data that matches epidemic process in a homogeneously mixing popula-
the maturity of the forecast-origin data that the forecast tion (Verhulst, 1838). The most famous and employed
is conditioned on. evolution of the logistic equation is the Bass model (Bass,
1969), developed in the field of quantitative marketing
25 This subsection was written by Michael P. Clements. and soon become a major reference, due to its simple and
26 For example, the Federal Reserve Bank of Philadelphia main- powerful structure.
tain a real-time data set covering a number of US macro-
variables, at: https://www.philadelphiafed.org/research-and-data/real-
time-center/real-time-data/, and see Croushore and Stark (2001). 27 This subsection was written by Mariangela Guidolin.
727
The Bass model (BM) describes the life-cycle of an

innovation, depicting its characterising phases of launch,
growth/maturity, and decline, as result of the purchase
decisions of a given cohort of potential adopters. Math-
ematically, the model is represented by a first order
differential equation, describing a diffusion process by
means of three parameters: the maximum market poten-
tial, m, assumed to be constant along the whole diffusion
process, and parameters p and q, referring respectively
to two distinct categories of consumers, the innovators,
identified with parameter p, adopting for first, and the
imitators, adopting at a later stage by imitating others’
behaviour and thus responsible for word-of-mouth dy-
namics. In strategic terms, crucial forecasts are referred
to the point of maximum growth of the life cycle, the
peak, and the point of market saturation. For a general
description of new product forecasting please refer to
Section 3.2.6.
Innovation diffusion models may be also used for post-
Fig. 1. Typical attributes of a growth cycle’s ‘‘seasons’’.
hoc explanations, helping understand the evolution of
Source: Adopted from Modis (1998) with the permission
a specific market and its response to different external from the author.
factors. Indeed, one of the most appealing characteristics
of this class of models is the possibility to give a simple
and nice interpretation to all the parameters involved. In
Sections 2.3.18 and 3.4.5. In fact, logistic growth can be
this perspective, a valuable generalisation of the BM was
detected whenever there is growth in competition, and
proposed in Bass et al. (1994) with the Generalised Bass
competition can be generalised to a high level of abstrac-
Model (GBM). The GBM enlarges the BM by multiplying its
tion, e.g. diseases competing for victims and all possible
hazard rate by a very general intervention function x(t),
accidents competing for the chance to be materialised.
assumed to be non-negative, which may account for ex-
S-curves enter as modular components in many in-
ogenous shocks able to change the temporal dynamics of
the diffusion process, like marketing strategies, incentive tricate natural patterns. One may find S curves inside S
mechanisms, change in prices and policy measures. curves because logistics portray a fractal aspect; a large
Another generalisation of the BM and the GBM, relax- S curve can be decomposed in a cascade of smaller ones
ing the assumption of a constant market potential was (Modis, 1994). One may also find chaos by rendering
proposed in Guseo and Guidolin (2009) with the GGM. the logistic equation discrete (Modis & Debecker, 1992).
This model postulates a time-dependent market poten- Finally, logistics sit in the heart of the Lotka–Volterra
tial, m(t), which is function of the spread of knowledge equations, which describe the predator–prey relations
about the innovation, and thus assumes that a diffusion and other forms of competition. In its full generality, the
process is characterised by two separate phases, infor- logistic equation, in a discrete form, with cross terms to
mation and adoption. The GGM allows a significant im- account for all interrelations between competing species,
provement in forecasting over the simpler BM, especially would give a complete picture in which growth in compe-
through a more efficient description of the first part of the tition, chaos, self-organisation, complex adaptive systems,
time series, often characterised by a slowdown pattern, as autopoiesis, and other such academic formulations, all
noticed by Guseo and Guidolin (2011). ensue as special cases (Modis, 1997).
Other generalisations of innovation diffusion models, Each S curve has its own life cycle, undergoing good
considering competition between products, are treated in and bad ‘‘seasons’’ (see Fig. 1). A large set of behaviours
Section 2.3.20. Applications of innovation diffusion mod- have been tabulated, each one best suited for a particular
els are presented in Sections 3.2.6 and 3.4.5. season (Modis, 1998). Becoming conservative – seeking
no change – is appropriate in the summer when things
2.3.19. The natural law of growth in competition28 work well. But excellence drops in second place during
As early as in 1925 Alfred J. Lotka demonstrated that the difficult times of winter – characterised by chaotic
manmade products diffuse in society along S-shaped pat- fluctuations – when fundamental change must take place.
terns similar to those of the populations of biological Learning and investing are appropriate for spring, but
organisms (Lotka, 1925). Since then S curve logistic de- teaching, tightening the belt, and sowing the seeds for the
scriptions have made their appearance in a wide range next season’s crop belong in the fall.
of applications from biology, epidemiology and ecology to Focusing on what to do is appropriate in spring,
industry, competitive substitutions, art, personal achieve- whereas in fall the emphasis shifts to the how. For ex-
ment and others (Fisher & Pry, 1971; Marchetti, 1983; ample, the evolution of classical music followed a large-
Meade, 1984; Modis, 1992). The reader is also referred to timeframe S curve beginning in the fifteenth century and
reaching a ceiling in the twentieth century; see Fig. 2
28 This subsection was written by Theodore Modis. (Modis, 2013). In Bach’s time composers were concerned
728
Fig. 3. An upward excursion at t1 reaches the same level as the logistic

curve at t2 and can be considered as a ‘‘natural’’ deviation. The same-
size excursion at time t3 has no corresponding point on the curve. The
grey life cycle delimits the position and size of all ‘‘natural’’ deviations.
Source: Adapted from Modis (2007) with permission from the author.
Fig. 2. The evolution of classical music. The vertical axis could

be something like ‘‘importance’’, ‘‘public appreciation’’, or ‘‘public where multiple species struggle with each other to sur-
preoccupation with music’’ (always cumulative). vive, or in socio-economic contexts where products, tech-
Source: Adopted from Modis (2013) with the permission from the
nologies and ideas concur to be finally accepted within a
author.
market and compete for the same market potential. These
competition dynamics are reflected in separate time se-
ries – one for each concurrent – characterised by a locally
with what to say. The value of their music is in its archi-
oscillatory behaviour with nonlinear trends, unexpected
tecture and as a consequence it can be interpreted by any
deviations and saturating effects.
instrument, even by simple whistling. But two hundred
The analytic representation of competition has fol-
years later composers such as Debussy wrote music that
lowed different approaches. A first approach has been
depends crucially on the interpretation, the how. Classical
music was still ‘‘young’’ in Bach’s time but was getting based on complex systems analysis (Boccara, 2004), which
‘‘old’’ by Debussy’s time. No wonder Chopin is more pop- refers to a class of agents (see also Section 2.7.3) that
ular than Bartók. Chopin composed during the ‘‘summer’’ interact, through local transition rules, and produce com-
of music’s S curve when public preoccupation with music petition as an emergent behaviour. This approach may be
grew fastest. Around that time composers were rewarded frequently reduced to a system of differential equations,
more handsomely than today. The innovations they made with suitable mean field approximations. A second ap-
in music – excursions above the curve – were assimilated proach, systems analysis, has been based on systems of
by the public within a short period of time because the ordinary differential equations (ODE).
curve rose steeply and would rapidly catch up with each In this domain, competition may be a synchronic pro-
excursion/innovation. But today the curve has flattened cess, if competitors are present in the environment at the
and composers are given limited space. If they make an same time; for example, two products may enter the mar-
innovation and find themselves above the curve, there ket in the same period. Instead, it is diachronic if competi-
won’t be any time in the future when the public will tors come at a later stage; for example, one product enters
appreciate their work; see Fig. 3 (Modis, 2007). On the a market in a given period and just subsequently other
other hand, if they don’t innovate, they will not be saying similar products enter the same market and start to com-
anything new. In either case today’s composers will not pete. Pioneering contributions of this competition mod-
be credited with an achievement. elling are due to Lotka (1920) and Volterra (1926a), who
S curves constructed only qualitatively can be accu- independently obtained a class of synchronic predator–
rate, informative, and insightful. Practical challenges of prey models; see also Section 2.3.19. A generalised ver-
applying S curves are discussed in Section 3.8.12. sion of the Lotka–Volterra (LV) model has been provided
by Abramson and Zanette (1998).
2.3.20. Synchronic and diachronic competition29 Morris and Pratt (2003) proposed an extended LV
Synchronic and diachronic competition models ac- model for a duopolistic situation, by making explicit the
count for critical life cycle perturbations due to inter- role of carrying capacities or market potentials, and the
actions, not captured by univariate innovation diffusion inhibiting strength of the competitors in accessing the
models (see Section 2.3.18) or other time series models, residual resource. LV models typically do not have closed
such as ARIMA and VAR. This is important especially in form solutions. In such cases, a staked form of equa-
medium and long-term prediction. tions allows a first-stage inference based on nonlinear
Competition in a natural or socio-economic system least squares (NLS), with no strong assumptions on the
generally refers to the presence of antagonists that con- stochastic distributions of error component. Short-term
tend for the same resource. This typically occurs in nature, refining may be grounded on a Seasonal Autoregres-
sive Moving Average with exogenous input (SARMAX)
29 This subsection was written by Renato Guseo. representation. Outstanding forecasts may be obtained
729
including the estimated first-stage solution as ‘exogenous’ differ from MFI since take into account the temporal inter-
input (see Sections 2.2.3 and 2.2.5). dependency of forecast errors and are constructed to have
A different synchronic model, termed Givon-Bass (GB) the observed temporal trajectory of the response variable
model, extending the univariate innovation diffusion mod- fully contained inside the forecast intervals during all
els described in Section 2.3.18 (Bass, 1969; Givon, Ma- lead-times of the time horizon. The number of works
hajan, & Müller, 1995), has been presented in Bonaldo that cover SFI is lower when compared to the MFI, but
(1991), introducing parametric components of global in- some examples are: methods based on Bonferroni- and
teraction. In this model, the residual market (or carrying product-type inequalities applied time series forecasting
capacity) is completely accessible to all models likes ARIMA and Holt-Winters (Ravishanker, yen
competitors, and the rate equations introduce distributed Wu, & Glaz, 1991); combination of bootstrap replications
seeding effects. The GB model has a closed form solu-
and an heuristic optimisation procedure to find an en-
tion that was independently published by Krishnan, Bass,
velope of the temporal trajectories closest to the de-
and Kummar (2000). The more general model by Savin
terministic forecast (Staszewska-Bystrova, 2011, see also
and Terwiesch (2005) and related advances by Guseo
Section 2.7.5); sampling forecast errors at different hori-
and Mortarino (2010) were extended to the diachronic
zons and estimate the SFI with the empirical Mahalanobis
case in Guseo and Mortarino (2012), defining a com-
petition and regime change diachronic (CRCD) model distance (Jordá, Knüppelc, & Marcellino, 2013).
with a closed form solution. A relevant improvement of Advances in Operations Research for decision-making
CRCD has been proposed in Guseo and Mortarino (2014), problems under uncertainty imposed new requirements
by introducing within-brand and cross-brand word-of- in forecast uncertainty estimation and representation. On
mouth effects, not present in standard LV models. The one hand, stochastic optimisation requires a scenario rep-
unrestricted unbalanced diachronic competition (unre- resentation for forecast uncertainty (Powell, 2019). This
stricted UCRCD) model, is defined with these new factors. motivated research in methods that generate uncertainty
The model assumes, among other specifications, a con- forecasts represented by random vectors (term used in
stant market potential. In Guseo and Mortarino (2015) statistics) or path forecasts (term used in econometrics),
this assumption is relaxed, by introducing a dynamic such as parametric copula combined with MFI (Pinson,
market potential (Guseo & Guidolin, 2009, 2011). Some Madsen, Nielsen, Papaefthymiou, & Klöckl, 2009), para-
applications are summarised in Section 3.8.8. metric dynamic stochastic model (Li & Chan, 2011) or
epi-spline basis functions (Rios, Wets, & Woodruff, 2015).
2.3.21. Estimation and representation of uncertainty30 On the other hand, robust optimisation does not make
Forecasting uncertainty consists in estimating the pos- specific assumptions on probability distributions and the
sible range of forecast errors (or true values) in the fu- uncertain parameters are assumed to belong to a deter-
ture and the most widely adopted representation is a ministic uncertainty set. Hence, some authors proposed
forecast interval (Patel, 1989). The forecast interval in- new methods to shape forecast uncertainty as polyhedral
dicates a range of values and the respective probabil- or ellipsoidal regions to enable a direct integration of
ity, which is likely to contain the true value (which is forecasts in this type of optimisation problem (Bertsimas
yet to be observed) of the response variable. Since for
& Pachamanova, 2008; Golestaneh, Pinson, & Gooi, 2019).
a specific lead-time the forecast interval only encom-
Finally, communication of forecast uncertainty
passes information from a marginal distribution, it can
(e.g., MFI, SFI, random vectors) to decision-makers re-
also be named marginal forecast interval (MFI). A MFI
quires further attention since it remains as a major bot-
can be obtained from: parametric distribution, such as
tleneck for a wide adoption by industry, particularly in
a Gaussian distribution with conditional mean and vari-
ance estimated with a Generalised ARCH model (Baillie uses cases with multivariate time series (Akram, Bin-
& Bollerslev, 1992); non-parametric distribution, e.g., ob- ning, & Maih, 2015) and adverse weather events (Ramos,
tained with conditional kernel density estimation (Hynd- Mathevet, Thielen, & Pappenberger, 2010). Please also see
man, Bashtannyk, & Grunwald, 1996); directly estimated Section 3.7.5.
with statistical learning methods, such as quantile regres-
sion (Taylor & Bunn, 1999) or bootstrapping (Masarotto, 2.3.22. Forecasting under fat tails31
1990), or with machine learning algorithms like quantile A non-negative continuous random variable X is fat-
random forests (Meinshausen, 2006). For a combination of tailed, if its survival function S(x) = P(X ≥ x) is regularly
density forecasts from different models, see Section 2.6.2. varying, that is to say if S(x) = L(x)x−α , where L(x) is
For multi-step ahead forecasting problems (see also L(tx)
a slowly varying function, for which limx→∞ L(x) = 1
Section 2.7.7), in particular when information about fore-
for t > 0 (Embrechts, Klüppelberg, & Mikosch, 2013).
cast uncertainty is integrated in multi-period stochas-
The parameter α is known as the tail parameter, and it
tic optimisation (Dantzig & Infanger, 1993), information
governs the thickness of the tail – the smaller α the fatter
about the temporal dependency structure of forecast er-
the tail – and the existence of moments, so that E [X p ] <
rors (or uncertainty) is a fundamental requirement. In this
case, the concept of simultaneous forecast intervals (SFI)
∞ if and only if α > p. Often α is re-parametrised as
can be found in the statistical literature (Chew, 1968). SFI ξ = 1/α .
30 This subsection was written by Ricardo Bessa. 31 This subsection was written by Pasquale Cirillo.
730
Fat tails are omnipresent in nature, from earthquakes 2.4. Bayesian forecasting
to floods, and they are particularly common in human-
related phenomena like financial markets, insurance, pan- 2.4.1. Foundations of Bayesian forecasting32
demics, and wars (see, for example, Mandelbrot, 1983; The Bayesian approach to forecasting produces, by de-
Taleb, 2020, and references therein). fault, a probabilistic forecast (see also Section 2.6.2 and
Forecasting fat-tailed random variables is therefore Section 2.12.4) describing the uncertainty about future
pivotal in many fields of life and science. However, while values of the phenomenon of interest, conditional on all
a basic coverage of the topic is available in most time known quantities; and with uncertainty regarding all un-
known quantities having been integrated out. In order to
series and risk management manuals (e.g., McNeil, Frey, &
produce such forecasts, the Bayesian approach requires (i)
Embrechts, 2015; Shumway & Stoffer, 2017), the profound
a predictive model for the future value of the relevant
implications of fat tails are rarely taken into consideration,
phenomenon, conditional on the observed data and all
and this can generate substantial errors in forecasts. model unknowns; (ii) a model for the observed data;
As observed in Taleb, Bar-Yam, and Cirillo (2020), any and (iii) a distribution describing the (subjective or objec-
statistical forecasting activity about the mean – or an- tive) prior beliefs regarding the model unknowns. Using
other quantity – of a phenomenon needs the law of large these quantities, the standard calculus of probability dis-
numbers (LLN), which guarantees the convergence of the tributions, and Bayes’ theorem, then yield the Bayesian
sample mean at a given known rate, when the number of predictive (equivalently, forecast) density function (where
observations n grows. density is used without loss of generality).
Fat-tailed phenomena with tail parameter α ≤ 1 are Stated more formally, given observed data up to time
trivially not predictable. Since their theoretical mean is n, y = (y1 , . . . , yn )′ , and denoting the model unknowns
not defined, the LLN does not work, for there is nothing by θ , Bayesian forecasting describes the behaviour of the
the sample mean can converge to. This also applies to future random variable Yn+1 via the predictive density:
apparently infinite-mean phenomena, like pandemics and
∫
wars, i.e., extremely tricky objects of study, as discussed p(yn+1 |y) = p(yn+1 |θ, y)p(θ|y)dθ, (2)
in Cirillo and Taleb (2016a). In similar situations, one
where yn+1 denotes a value in the support of Yn+1 and
can rely on extreme value theory to understand tail risk
p(yn+1 |θ, y) is the predictive model for Yn+1 conditional
properties, but should refrain from standard forecasting.
on y and the model unknowns θ . Critically, and in contrast
For random variables with 1 < α ≤ 2, the LLN can
with frequentist approaches to forecasting, parameter un-
be extremely slow, and an often unavailable number of certainty has been factored into p(yn+1 |y) via the process
observations is needed to produce reliable forecasts. Even of integration with respect to the posterior probability
for a well-behaved and non-erratic phenomenon, we all density function (pdf) for θ , p(θ|y). The posterior pdf is
agree that a claim about the fitness or non-fitness of given, by Bayes’ theorem, as p(θ |y) ∝ p(y|θ ) × p(θ ), where
a forecasting approach, just on the basis of one single p(y|θ ) defines the assumed model for y (equivalently,
observation (n = 1), would be considered unscientific. the likelihood function), and the prior pdf p(θ ) captures
The fact is that with fat-tailed variables that ‘‘n = 1’’ prior beliefs about θ . Moreover, uncertainty about the
problem can be made with n = 106 observations (Em- assumed predictive model itself can be easily accommo-
brechts et al., 2013; Taleb, 2020). In the case of events dated using Bayesian model averaging, which involves
like operational losses, even a larger n → ∞ can still be taking a weighted average of model-specific predictives,
just anecdotal (Cirillo & Taleb, 2016a). with posterior model probabilities (also obtained via by
According to Taleb (2020), owing to preasymptotics, a Bayes’ theorem) serving as the weights. See Greenberg
conservative heuristic is to manage variables with α ≤ (2008), Koop (2003) and O’Hagan and Forster (2004) for
2.5 as practically unpredictable (for example, see Sec- textbook illustrations of all of these steps.
tions 3.6.2 and 3.6.6). Their sample average is indeed too No matter what the data type, the form of predictive
model, or the dimension of the unknowns, the basic man-
unstable and needs too many observations for forecasts
ner in which all Bayesian forecast problems are framed is
to be reliable in a reasonable period of time. For α > 2.5,
the same. What differs however, from problem to prob-
conversely, forecasting can take place, and the higher α
lem, is the way in which the forecasting problem is solved.
the better. In any case, more research is strongly needed To understand why, it is sufficient to recognise that in
and desirable (see also Section 4). order to obtain the predictive density p(yn+1 |y) we must
Observe that even discussing the optimality of any be able to (somehow) perform the integration that de-
alarm system (see, for example, Svensson, Holst, Lindquist, fines this quantity. In almost any practical setting, this
& Lindgren, 1996; Turkman & Turkman, 1990) based on integration is infeasible analytically and we must rely
average forecasts would prove meaningless under ex- on computational methods to access the predictive den-
tremely fat tails (α ≤ 2), when the LLN works very slowly sity. Therefore, the evolution of the practice of Bayesian
or does not work. In fact, even when the expected value forecasting has gone hand in hand with developments in
is well-defined (i.e., 1 < α < 2), the non-existence of Bayesian computation (Martin, Frazier, & Robert, 2020).
the variance would affect all the relevant quantities for Through the lens of computation, in Section 2.4.2 we
the verification of optimality (De Mare, 1980), like for briefly describe the methods of implementing Bayesian
instance the chance of undetected events. For all these forecasting.
quantities, the simple sample estimates commonly used
would indeed be misleading. 32 This subsection was written by David T. Frazier & Gael M. Martin.
731
2.4.2. Implementation of Bayesian forecasting33 by user-specified measures of predictive loss, new ap-
If the posterior is accessible via methods of exact proaches to Bayesian prediction have recently been pro-
simulation – e.g., Monte Carlo simulation, importance posed, and we briefly discuss two such classes of methods.
sampling, Markov chain Monte Carlo (MCMC) sampling, First are methods for combining predictives in which
pseudo-marginal MCMC (Andrieu, Doucet, & Holenstein, the weights are not equated to the posterior model prob-
2011; Andrieu & Roberts, 2009) – an estimate of the abilities (as in standard Bayesian model averaging) but,
predictive density p(yn+1 |y) in (2) can be produced us- rather, are updated via problem-specific predictive cri-
ing draws of the unknown θ from the posterior pdf, teria, or via predictive calibration (Dawid, 1982, 1985;
p(θ|y). In most cases, this simulation-based estimate of Gneiting, Balabdaoui, & Raftery, 2007); see Bassetti, Casarin,
p(yn+1 |y) can be rendered arbitrarily accurate by choos- and Ravazzolo (2018), Basturk, Borowska, Grassi, Hooger-
ing a very large number of posterior draws; hence the heide, and van Dijk (2019), Billio, Casarin, Ravazzolo, and
use of the term ‘exact predictive’ to reference this es- van Dijk (2013), Casarin, Leisen, Molina, and ter Horst
timate. See Geweke and Whiteman (2006) for an early (2015), McAlinn, Aastveit, Nakajima, and West (2020),
review of Bayesian forecasting implemented using exact McAlinn and West (2019), and Pettenuzzo and Ravazzolo
simulation, Sections 2.3.15, 2.4.3, 2.5.3, 3.3.2 and 3.3.10 (2016), for a selection of approaches, and Section 2.6.2
for further discussions and a range of relevant applica- for related discussion. Importantly, these methods do not
tions, and Chapters 3, 7, and 9 in Geweke, Koop, and assume that the true model is spanned by the constituent
van Dijk (2011) for applications in (general) state space, model set. Second are methods in which the standard
macroeconometric and finance settings respectively. In Bayesian posterior, which is itself based on a potentially
addition, a 2008 special issue of International Journal of misspecified model, is replaced by a generalised version
Forecasting on Bayesian Forecasting in Economics pro- that is designed for the specific predictive task at hand
vides coverage of forecasting applications (and methods (e.g., accurate prediction of extreme values); with the
of computation) that exploit exact simulation methods, as overall goal then being to produce predictions that are
do selected chapters in Brooks, Gelman, Jones, and Meng accurate according to the particular measure of interest.
(2011) and O’Hagan and West (2010). See Frazier, Loaiza-Maya, Martin, and Koo (2021), Loaiza-
In cases where the posterior is not readily accessible, Maya, Martin and Frazier (2020) and Syring and Martin
due to either the intractability of the likelihood func- (2020), for specific examples of this methodology, as well
tion or the high dimension of the model unknowns, or as proofs of its theoretical validity.
both, methods of approximation are required. Frazier et al.
(2019), for instance, produce an ‘approximate predic- 2.4.3. Bayesian forecasting with copulas34
tive’ by replacing the exact posterior in (2), p(θ|y), with Copulas provide intrinsic multivariate distribution
an ‘approximate posterior’ constructed using approximate structure that allows for modelling multivariate depen-
Bayesian computation (ABC – Sisson, Fan, & Beaumont, dence with marginal distributions as the input, making
2019). In large samples, this approximate predictive is it possible to forecast dependent time series and time
shown to be equivalent to the exact predictive. The ap- series dependence. This review focuses on the Bayesian
proximate and exact predictives are also shown to be approach for copula forecasting. For rigorous topics on
numerically indistinguishable in finite samples, in the copula introduction (Joe, 1997; Nelsen, 2006; Trivedi &
cases investigated; see also Canale and Ruggiero (2016), Zimmer, 2007), copula modelling techniques (Durante &
and Kon Kam King, Canale, and Ruggiero (2019). Related Sempi, 2015), vine copulas (Joe, 2014), review on fre-
work produces an approximate predictive by exploiting quentist approaches for copula-based forecasting (Pat-
variational Bayes (Blei, Kucukelbir, & McAuliffe, 2017) ton, 2013), see the aforementioned references and the
approximations of the posterior (Chan & Yu, 2020; Koop references therein.
& Korobilis, 2018; Loaiza-Maya, Smith, Nott & Danaher, The advantages of the Bayesian copula approach com-
2020; Quiroz, Nott, & Kohn, 2018; Tran, Nott, & Kohn, pared to the frequentist treatments are (i) the Bayesian
2017). The flavour of this work is broadly similar to that approach allows for jointly modelling the marginal mod-
of Frazier et al. (2019); that is, computing the predictive els and the copula parameters, which improves the fore-
p(yn+1 |y) via an approximation to the posterior does not casting efficiency (Joe, 2005; Li & Kang, 2018), (ii) proba-
significantly reduce predictive accuracy. bilistic forecasting is naturally implied with the Bayesian
The Bayesian paradigm thus provides a very natural predictive density, and (iii) experts information can be
and coherent approach to prediction that can be imple- seamlessly integrated into forecasting via the priors’ set-
mented successfully via one of any number of computing.
tational methods. Inherent to the approach, however, is Forecasting with copulas involves selecting between
the assumption that the model we are using to make a different class of copulas. Common approaches include
predictions is an accurate description of the data gen- Bayesian hypothesis testing (see Section 2.4.1) where cop-
erating process (DGP) that has generated the observed ula parameters are treated as nuisance variables (Huard,
data; or if we are averaging across models using Bayesian Évin, & Favre, 2006), or parsimonious modelling of co-
model averaging, we must assume that the assumed set of variance structures using Bayesian selection and model
models spans the DGP for the observed data. In response averaging (Min & Czado, 2011; Pitt, Chan, & Kohn, 2006;
to this limitation, allied with the desire to drive prediction Smith, 2010).
33 This subsection was written by David T. Frazier & Gael M. Martin. 34 This subsection was written by Feng Li.
732
One particular interest in copulas is to forecast the participants, policy makers, the general public, etc. are
dynamic dependencies between multivariate time series. interested in which direction the economy of a country is
Time-varying copula construction is possible via (i) an developing ahead of the publication date of the respective
autoregressive or GARCH form (see Section 2.3.11) of de- quarterly GDP figures (see also Section 3.3.2), for which
pendence parameters (Lucas, Schwaab, & Zhang, 2014; these leading indicators play a crucial role. This is of
Patton, 2006), (ii) factor copula construction (Oh & Patton, particular relevance for short-term forecasting and now-
2018; Tan, Panagiotelis, & Athanasopoulos, 2019) that casting of such a (low-frequency) variable characterised
simplifies the computation, (iii) the stochastic copula au- by a publication lag.
toregressive model (Almeida & Czado, 2012) that depen- For some leading indicators, single variables are com-
dence is modelled by a real-valued latent variable, and bined into a composite index such as The Conference
(iv) covariate-dependent copulas approach by parame- Board Leading Economic Index published by The Confer-
terising the dependence as a function of possible time- ence Board for the United States. It consists of averages of
dependent covariates (Li & Kang, 2018) that also improves ten variables, including average weekly hours in manufac-
the forecasting interpretability. ARMA-like and GARCH- turing, manufacturers’ new orders, private housing build-
like dependences in the tail can be considered as special ing permits, and average consumer expectations (The
cases of Li and Kang (2018). Conference Board, 2020). Other leading indicators consist
In multivariate time series forecasting (see also Sec- of one variable only and are purely survey-based, such
tion 2.3.9), unequal length of data is a common issue. One as the monthly ifo Business Climate Index published by
possible approach is to partition the copula parameters the ifo Institute for Germany, which covers the appraisal
into elements relating only to the marginal distributions of current and future expected business of executives of
and elements only relating to the copula (Patton, 2006a). approximately 9,000 German firms across industries (ifo
For mixed frequency data, it is possible to decompose Institute, 2020).
the copula dependence structure into linear and nonlinear If a specific leading indicator is able to capture certain
components. Then the high and low frequency data is economic changes before they happen, such a leading in-
used to model the linear and nonlinear dependencies, dicator xt is said to Granger-cause (or predictively cause)
respectively (Oh & Patton, 2016). Bayesian data augmen- a forecast variable yt (Granger, 1969), implying that a
tation is also used to forecast multivariate time series ‘‘cause’’ cannot happen after an effect. In (theoretical)
with mixed discrete and continuous margins (Smith & econometric terms and following the notation of Lütke-
Khaled, 2012). For other treatments for discrete and con- pohl (2005), yt is Granger-caused by xt if for at least one
tinuous time series (see, for example, Panagiotelis, Czado, forecast horizon h = 1, 2, . . . the following inequality
& Joe, 2012; Panagiotelis, Czado, Joe, & Stöber, 2017). holds:
The Bayesian approach for lower dimensional copula ∑ ∑
forecasting (d < 10) is straightforward with traditional (h | Ωi ) < (h | Ωi \ {xt | t ≤ i}).
Gaussian copulas, Student’s-t copulas, Archimedean cop- y y
ulas, or pair copula combinations. In higher dimensional In other words, the mean square error of the optimal
settings, special considerations are required to save the h-step-ahead predictor of the forecast variable, which in-
computational burden, such as low rank approximation cludes the information contained in the leading indicator
of covariates matrix (Salinas, Michael, Laurent, Roberto & in the set of all relevant information to the forecaster
Jan, 2019) or factor copula models with stochastic load-
y (h | Ωi ), must be
∑
available at the forecast origin
ings (Creal & Tsay, 2015). smaller than the mean square error of that h-step-ahead
In the Bayesian setup, forecasting model performance predictor of the forecast variable without
is typically evaluated based on a K -fold out-of-sample log
∑ the information
contained in said leading indicator y (h | Ωi \ {xt | t ≤
predictive score (LPS: Geweke & Amisano, 2010), and out- i}) (Lütkepohl, 2005).
of-sample Value-at-Risk (VaR) or Expected Shortfall (ES) Nonetheless, the concept of Granger causality also has
are particularly used in financial applications. The LPS is its limitations. The notion that a ‘‘cause’’ cannot happen
an overall forecasting evaluation tool based on predictive after an effect is only a necessary, yet not a sufficient
densities, serving out-of-sample probabilistic forecasting. condition for causality, hence the reason why ‘‘cause’’
LPS is ideal for decision makers (Geweke, 2001; Geweke has been written with quotation marks in this section. If
& Amisano, 2010). The VaR gives the percentile of the one ignored this fact and equated causality with Granger
conditional distribution, and the corresponding ES is the causality, they would commit an informal logical fallacy
expected value of response variable conditional on it lying called Post hoc ergo propter hoc (i.e., after this, therefore
below its VaR. because of this; Walton, Reed, & Macagno, 2008). A bold
example of this fallacy would be, ‘‘Because the birds mi-
2.5. Variable and model selection grate, winter is coming’’. This is a fallacy, as winter would
come at about the same time every year, no matter if
2.5.1. Leading indicators and Granger causality35 there were migrating birds or not. Moreover, hardly any
Leading (economic) indicators are variables that try economic activity is monocausal.
to capture changes in the development of economic ac- There are also different types of formal statistical
tivity before those changes materialise. Typically, market Granger causality tests available for different data struc-
tures that are implemented in typical statistics/
35 This subsection was written by Ulrich Gunter. econometrics software packages. For the simple case of
733
two variables (Granger, 1969), say the forecast variable Kolasa, & Rubaszek, 2017) or oil prices (Rubaszek, 2020)
and the leading indicator, the null hypothesis of a bivari- compared to a number of complex competitors.
ate Granger causality test is that the leading indicator Wide support to the view that model simplicity im-
does not Granger-cause the forecast variable. Under this proves forecasting performance is presented by Green
null hypothesis, the F -test statistic on the joint impact and Armstrong (2015), in an introductory article to the
of the coefficients of the past realisations of the leading special issue of Journal of Business Research ‘‘Simple versus
indicator employed as explanatory variables in a multiple complex forecasting’’, as well as the results of M1 and
linear regression with the forecast variable as dependent M2 competitions (Makridakis et al., 1982, 1993). Stock
variable and its past realisations as additional explanatory and Watson (1998) also show that for most US monthly
variables will not be statistically significantly different series complex non-linear autoregressive models deliver
from zero. The maximum lag for past realisations would less accurate forecasts than their linear counterparts. On
be optimally determined, for instance, by some infor- the contrary, the results of M4 competition tend to favour
mation criterion (e.g., AIC, BIC; see also Section 2.5.4). more complex models (Makridakis, Spiliotis & Assimakopou-
Practical applications of Granger causality and leading los, 2020, and Section 2.12.7).
indicators in tourism demand forecasting can be found, Why are then complex models preferred to simple
for instance, in Section 3.8.1. ones, if the latter deliver more accurate forecasts? Brighton
and Gigerenzer (2015) claim that there is a tendency to
2.5.2. Model complexity36 overemphasize the bias component and downplay the
A simple model must be easily understood by decision- role of variance. This behaviour is leading to an implicit
makers. On the contrary, relationships in a complex model preference towards more complex forecasting methods,
are opaque for its users (see also Section 3.7.4). In this which is called by the authors as ‘‘bias bias’’. To avoid
context, complexity is not measured solely by the number it, one can follow the golden rules of forecasting, which
of parameters, but also by the functional form of the says: be conservative in the choice of over-ambitious
relationships among variables. models and be wary of the difficulty to forecast with
Complex models are commonly believed to deliver complex methods (Armstrong, Green, & Graefe, 2015). The
better forecasts, as they well describe sophisticated eco- alternative is to use methods, which explicitly account
nomic structures and offer good fit the data. Consequently, for the bias–variance trade-off, e.g. machine learning (see
they are favoured by researchers, decision-makers and Section 2.7.10).
academic journals. However, empirical studies provide
2.5.3. Variable selection37
little evidence about their forecasting superiority over
References to ‘big data’ have become somewhat ubiq-
simple models. This can be explained using the bias–
uitous both in the media and in academic literature in
variance trade-off framework, in which the mean squared
recent years (see Section 2.7.1 but also Section 2.2.5).
error can be decomposed into
Whilst in some disciplines (for example Finance) it has
MSE = noise + v ariance + bias2 . become possible to observe time series data at ever higher
frequency, it is in the cross section that the amount of
Noise is driven by the random term in the model. It cannot data available to analysts has seen exponential growth.
be avoided, even if we know the true DGP. Variance is Ordinary Least Squares (OLS) is the standard tool for
caused by the need to estimate the model parameters, data analysis and prediction, but is well known to per-
hence its value increases with model complexity and de- form poorly when there many potential explanatory vari-
clines with the sample size. Bias is predominantly related ables; Bishop (2006) sets out clearly why this is so. In
to model mis-specification, which is most likely to occur situations where there is no obvious underlying model
for simple methods. The implications of the above frame- to suggest which of a potentially large set of candidate
work are twofold: (i) the relationship between model variables to focus on, the researcher needs to add new
complexity and MSE is U-shaped and (ii) the optimal tools to the tool kit.
model complexity increases with the sample size. There are two principal sets of approaches to this
The illustration of the bias–variance trade-off for a problem. The first seeks to reduce the model dimensions
simple autoregressive model (see Section 2.3.4) is pro- by summarising the predictors in to a much smaller num-
vided by Ca’ Zorzi, Muck, and Rubaszek (2016). They ber of aggregated indices or factors. The second is to
present analytical proof that for any persistent DGP of the employ a regularisation strategy to reduce the effective
form yt = c + φ yt −1 + ϵt characterised by half-life of dimension of the problem by shrinking the size of the
over two years, the accuracy of forecasts from the random regression coefficients. Some strategies reduce a subset of
walk or the AR(1) model with parameter φ fixed at an these coefficients to zero, removing some variables from
arbitrary value consistent with half-life of five years tend the model entirely and these can hence be described as
to be higher than that from the estimated AR(1) model. variable selection procedures.
This result explains why in numerous studies the random Such procedures may be applicable either when a
walk is a tough benchmark as well as why a simple, problem is believed to be truly sparse, with only a small
calibrated AR(1) model can be successful in forecasting number of variables having an effect, or alternatively
inflation (Faust & Wright, 2013), exchange rates (Ca’ Zorzi, when a sparse model can provide effective forecasts, even
36 This subsection was written by Michał Rubaszek. 37 This subsection was written by Ross Hollyman.
734
though the underling system is in fact more complex (see methods (Castle, Doornik, & Hendry, 2020b; Hendry &
also Section 2.5.2). Doornik, 2014, see also Section 3.6.2). Autometrics, a
In the frequentist framework, the Least Absolute multiple-path block-search algorithm (see Doornik, 2018),
Shrinkage and Selection Operator (LASSO) procedure of Tib- uses impulse (IIS: Hendry, Johansen, & Santos, 2008a
shirani (1996) has proven effective in many applications. and Johansen & Nielsen, 2009) and step (SIS: Castle, Doornik,
The LASSO requires choice of an additional regularisation Hendry, & Pretis, 2015b) indicator saturation for dis-
parameter (usually selected by some statistical criteria covering outliers and breaks, as well as selecting over
or via a cross validation process). Various refinements other variables. Although knowing the in-sample DGP
to the original LASSO procedure have been developed, need not suffice for successful forecasting after out-of-
see in particular Rapach, Strauss, Tu, and Zhou (2019) sample shifts, like financial crises and pandemics, ‘robust
for a recent forecasting application. An alternative fre- variants’ of selected models can then avoid systematic
quentist approach to variable selection and shrinkage is mis-forecasts (see, for example, Doornik, Castle, & Hendry,
the Complete Subset Regression (CSR) model of Elliott 2020a; Martinez et al., 2021, and Section 3.3.4).
(2015), where separate OLS regressions are run on all Saturation estimation has approximately K = k +
possible combinations of potential regressors, with fore- 2n candidates for k regressors, with 2K possible models,
casts generated by averaging across the entire set of requiring selection with more candidate variables, K , than
models. Kotchoni, Leroux, and Stevanovic (2019) combine observations, n (also see Section 2.7 and Section 2.3.9).
CSR with LASSO in a comprehensive empirical economic Selection criteria like AIC (Akaike, 1973), BIC (Schwarz,
forecasting exercise. 1978), and HQ (Hannan & Quinn, 1979) are insufficient
Where the underlying process is not properly sparse in this setting. For saturation estimation, we select at a
(see Giannone & Primiceri, 2017, for a discussion), it is tight significance level, α = 1/K , retaining subject-theory
perhaps more natural to work in a Bayesian framework variables and other regressors. When forecasting is the
where samples can be drawn from the variable selec- aim, analyses and simulations suggest loose significance
tion part of model reflecting the estimated probability for then selecting other regressors, close to the 10% to
of inclusion of each variable. The appropriate degree of 16% implied significance level of AIC, regardless of loca-
regularisation can also be selected in a similar way. Fore- tion shifts at or near the forecast origin (Castle, Doornik,
casts are then constructed as weighted averages over & Hendry, 2018a). At loose levels, Autometrics can find
several sparse models. This approach has proven to be multiple undominated terminal models across paths, and
very effective in practice, for example in fitting Bayesian averaging over these, a univariate method and a robust
Vector Auto Regressions to forecast economic time se-
forecasting device can be beneficial, matching commonly
ries. Early examples include George and McCulloch (1993)
found empirical outcomes. The approach applies to sys-
and Mitchell and Beauchamp (1988), which use binary
tems, whence selection significance of both indicators
indicators to select variables in to the model. More recent
and variables is judged at the system level. Capturing in-
approaches use continuous random variables to achieve a
sample location shifts remains essential (Doornik, Castle,
similar effect, making computation more tractable. Exam-
& Hendry, 2020b).
ples include the Horeshoe Prior (Carvalho, Polson, & Scott,
There are costs and benefits of selection for forecasting
2010; Piironen & Vehtari, 2017) and the LN-CASS prior
(also see Sections 2.5.3 and 2.11.3). Selection at a loose
of Thomson, Jabbari, Taylor, Arlt, and Smith (2019). Cross
significance level implies excluding fewer relevant vari-
(2020) is a recent example of an economic forecasting
ables that contribute to forecast accuracy, but retaining
exercise using several such models.
more irrelevant variables that are adventitiously signifi-
cant, although fewer than by simply averaging over all
2.5.4. Model selection38
sub-models (Hoeting, Madigan, Raftery, & Volinsky, 1999).
Taxonomies of all possible sources of forecast errors
Retaining irrelevant variables that are subject to location
from estimated models that do not coincide with their
data generating process (DGP) have revealed that two shifts worsens forecast performance, but their coefficient
mistakes determine forecast failures (i.e., systematic de- estimates are driven towards zero when updating es-
partures between forecasts and later outcomes), namely timates as the forecast origin moves forward. Lacking
mis-measured forecast origins and unanticipated location omniscience about future values of regressors that then
shifts (Clements & Hendry, 1999; Hendry & Mizon, 2012). need to be forecast, not knowing the DGP need not be
The former can be addressed by nowcasting designed costly relative to selecting from a general model that
to handle breaks (Bańbura, Giannone, & Reichlin, 2011; nests it (Castle et al., 2018a). Overall, the costs of model
Castle & Hendry, 2010; Castle, Hendry, & Kitov, 2018). It is selection for forecasting are small compared to the more
crucial to capture the latter as failure to do so will distort fundamental benefit of finding location shifts that would
estimation and forecasts, so must be a focus when select- otherwise induce systematic forecast failure.
ing models for forecasting facing an unknown number of
in-sample contaminating outliers and multiple breaks at 2.5.5. Cross-validation for time-series data39
unknown times. When building a predictive model, its purpose is usu-
Consequently, selection must be jointly over both ob- ally not to predict well the already known samples, but to
servations and variables, requiring computer learning obtain a model that will generalise well to new, unseen
38 This subsection was written by David F. Hendry. 39 This subsection was written by Christoph Bergmeir.
735
data. To assess the out-of-sample performance of a pre- serial correlation, e.g., with a Ljung–Box test (Ljung &
dictive model we use a test set that consists of data not Box, 1978). This procedure is implemented in the forecast
used to estimate the model (for a discussion of different package (Hyndman et al., 2020) in R (R Core Team, 2020),
error measures used in this context see Section 2.12.2). in the function CVar.
However, as we are now using only parts of the data for
model building, and other parts of the data for model 2.6. Combining forecasts
evaluation, we are not making the best possible use of
our data set, which is a problem especially if the amount 2.6.1. Forecast combination: a brief review of statistical ap-
of data available is limited. Cross-validation (CV), first proaches40
introduced by Stone (1974), is a widely used standard Given N forecasts of the same event, forecast com-
technique to overcome this problem (Hastie et al., 2009) bination involves estimation of so called combination
by using all available data points for both model building weights assigned to each forecast, such that the accuracy
and testing, therewith enabling a more precise estimation of the combined forecast generally outperforms the accu-
of the generalisation error and allowing for better model racy of the forecasts included. Early statistical approaches
selection. The main idea of (k-fold) cross-validation is to adopted a range of strategies to estimate combination
partition the data set randomly into k subsets, and then weights including (i) minimising in-sample forecast error
use each of the k subsets to evaluate a model that has variance among forecast candidates (Bates & Granger,
been estimated on the remaining subsets. An excellent 1969; Min & Zellner, 1993; Newbold & Granger, 1974),
overview of different cross-validation techniques is given (ii) formulation and estimation via ordinary least squares
by Arlot and Celisse (2010). regression (Granger & Ramanathan, 1984; MacDonald &
Despite its popularity in many fields, the use of CV in Marsh, 1994), (iii) use of approaches based on Bayesian
time series forecasting is not straightforward. Time series probability theory (Bordley, 1982; Bunn, 1975; Clemen
are often non-stationary and have serial dependencies & Winkler, 1986; Diebold & Pauly, 1990; Raftery, 1993,
(see also Section 2.3.4). Also, many forecasting techniques and Section 2.4), (iv) and the use of regime switching and
iterate through the time series and therewith have diffi- time varying weights recognising that weights can change
culties dealing with missing values (withheld for testing). over time (Diebold & Pauly, 1987; Elliott, Timmermann, &
Finally, using future values to predict data from the past is Komunjer, 2005; Lütkepohl, 2011; Tian & Anderson, 2014,
not in accordance with the normal use case and therewith and Section 2.3.12). Barrow and Kourentzes (2016) con-
seems intuitively problematic. Thus, practitioners often tains a very good documentation and empirical evaluation
resort to out-of-sample evaluation, using a subset from of a range of these early approaches, while De Menezes,
the very end of a series exclusively for evaluation, and Bunn, and Taylor (2000) and Armstrong (2001a) contain
therewith falling back to a situation where the data are guidelines on their use.
not used optimally. Recent statistical approaches use a variety of tech-
To overcome these problems, the so-called time-series niques to generate forecasts and/or derive weights. Ko-
cross-validation (Hyndman & Athanasopoulos, 2018) ex- lassa (2011) apply so called Akaike weights based on
tends the out-of-sample approach from a fixed origin to Akaike Information Criterion (Sakamoto, Ishiguro, & Kita-
a rolling origin evaluation (Tashman, 2000). Data is sub- gawa, 1986), while bootstrapping has been used to gen-
sequently moved from the out-of-sample block from the erate and combine forecast from exponential smooth-
end of the series to the training set. Then, the model can ing (Barrow, Kourentzes, Sandberg, & Niklewski, 2020;
be used (with or without parameter re-estimation) with Bergmeir, Hyndman, & Benítez, 2016; Cordeiro & Neves,
the newly available data. The model re-estimation can be 2009, but also Section 2.7.5 and Section 2.7.6), artificial
done on sliding windows with fixed window sizes or on neural networks (Barrow & Crone, 2016a, 2016b, and Sec-
expanding windows that always start at the beginning of tion 2.7.8), and other forecast methods (Athanasopoulos,
the series (Bell & Smyl, 2018). Song, & Sun, 2018; Hillebrand & Medeiros, 2010; Inoue
However, these approaches extending the & Kilian, 2008). Barrow and Crone (2016b) developed
out-of-sample procedure make again not optimal use of cross-validation and aggregating (Crogging) using cross-
the data and may not be applicable when only small validation to generate and average multiple forecasts,
amounts of data are available. Adapting the original CV while more recently, combinations of forecasts gener-
procedure, to overcome problems with serial correlations, ated from multiple temporal levels has become popu-
blocked CV approaches have been proposed in the lit- lar (Athanasopoulos, Hyndman, Kourentzes, & Petropou-
erature, where the folds are chosen in blocks (Bergmeir los, 2017; Kourentzes & Athanasopoulos, 2019; Kourentzes,
& Benítez, 2012; Racine, 2000) and/or data around the Petropoulos, & Trapero, 2014, and Section 2.10.2). These
points used for testing are omitted (Burman, Chow, & newer approaches recognise the importance of forecast
Nolan, 1994; Racine, 2000). Finally, it has been shown generation in terms of uncertainty reduction (Petropou-
that with purely autoregressive models, CV can be used los, Hyndman, & Bergmeir, 2018a), the creation of diverse
without modifications, i.e., with randomly choosing the forecasts (Brown, Wyatt, Harris, & Yao, 2005b; Lemke &
folds (Bergmeir, Hyndman, & Koo, 2018). Here, CV esti- Gabrys, 2010), and the pooling of forecasts (Kourentzes,
mates the generalisation error accurately, as long as the Barrow, & Petropoulos, 2019; Lichtendahl Jr & Winkler,
model errors from the in-sample fit are uncorrelated. This 2020).
especially holds when models are overfitting. Underfit-
ting can be easily detected by checking the residuals for 40 This subsection was written by Devon K. Barrow.
736
Now nearly 50 years on from the seminal work of Bates areas of macroeconomics and finance. The improved per-
and Granger (1969), the evidence that statistical combi- formance of combined density forecasts stems from the
nations of forecasts improves forecasting accuracy is near fact that pooling of forecasts allows to mitigate potential
unanimous, including evidence from competitions (Makri- misspecifications of the individual densities when the
dakis et al., 1982; Makridakis & Hibon, 2000; Makridakis, true population density is non-normal. Combining nor-
Spiliotis et al., 2020, and Section 2.12.7), and empiri- mal densities yields a flexible mixture of normals density
cal studies (Andrawis, Atiya, & El-Shishiny, 2011; Elliott which can accommodate heavier tails (and hence skew-
et al., 2005; Jose & Winkler, 2008; Kourentzes et al., ness and kurtosis), as well as approximate non-linear
2019). Still, researchers have tried to understand why specifications (Hall & Mitchell, 2007; Jore, Mitchell, &
and when combinations improve forecast accuracy (Atiya, Vahey, 2010).
2020; Palm & Zellner, 1992; Petropoulos et al., 2018a; The predictive density combination schemes vary
Timmermann, 2006), and the popularity of the simple av- across studies and range from simple averaging of indi-
erage (Chan & Pauwels, 2018; Claeskens, Magnus, Vasnev, vidual density forecasts to complex approaches that allow
& Wang, 2016; Smith & Wallis, 2009a). Others have inves- for time-variation in the weights of prediction models,
tigated properties of the distribution of the forecast error
also called experts (see Aastveit, Mitchell, Ravazzolo, &
beyond accuracy considering issues such as normality,
van Dijk, 2019, for a comprehensive survey of density
variance, and in out-of-sample performance of relevance
forecast combination methods). A popular approach is to
to decision making (Barrow & Kourentzes, 2016; Chan,
combine density forecasts using a convex combination of
Kingsman, & Wong, 1999; Makridakis & Winkler, 1989).
experts’ predictions, so called ‘linear opinion pools’ (see,
Looking forward, evidence suggests that the future lies
in the combination of statistical and machine learning for example, Geweke & Amisano, 2011; Hall & Mitchell,
generated forecasts (Makridakis, Spiliotis et al., 2020), 2007; Kascha & Ravazzolo, 2010). In order to determine
and in the inclusion of human judgment (Gupta, 1994; the optimal combination weights this method relies on
Petropoulos, Kourentzes, Nikolopoulos, & Siemsen, 2018b; minimising Kullback–Leibler divergence of the true den-
Wang & Petropoulos, 2016, but also Section 2.11.1). Addi- sity from the combined density forecast. These linear
tionally, there is need to investigate such issues as decom- approaches have been extended by Gneiting and Ran-
posing combination accuracy gains, constructing predic- jan (2013) to allow non-linear transformations of the
tion intervals (Grushka-Cockayne & Jose, 2020; Koenker, aggregation scheme and by Kapetanios, Mitchell, Price,
2005), and generating combined probability forecasts and Fawcett (2015), whose ‘generalised pools’ allow the
(Clements & Harvey, 2011; Hall & Mitchell, 2007; Raftery, combination weights to depend on the (forecast) value of
Madigan, & Hoeting, 1997; Ranjan & Gneiting, 2010, and the variable of interest.
Section 2.6.2). Finally, there is need for the results of com- Billio et al. (2013) developed a combination scheme
bined forecasts to be more interpretable and suitable for that allows the weights associated with each predic-
decision making (Barrow & Kourentzes, 2016; Bordignon, tive density to be time-varying, and propose a general
Bunn, Lisi, & Nan, 2013; Graefe, Armstrong, Jones Jr, & state space representation of predictive densities and
Cuzán, 2014; Todini, 2018). combination schemes. The constraint that the combina-
tion weights must be non-negative and sum to unity
2.6.2. Density forecast combinations41 implies that linear and Gaussian state-space models can-
Density forecasts provide an estimate of the future not be used for inference and instead Sequential Monte
probability distribution of a random variable of interest. Carlo methods (particle filters) are required. More re-
Unlike point forecasts (and point forecasts supplemented cently, McAlinn and West (2019) developed a formal
by prediction intervals) density forecasts provide a com- Bayesian framework for forecast combination (Bayesian
plete measure of forecast uncertainty. This is particularly predictive synthesis) which generalises existing methods.
important as it allows decision makers to have full in- Specifying a dynamic linear model for the synthesis func-
formation about the risks of relying on the forecasts of
tion, they develop a time-varying (non-convex/nonlinear)
a specific model. Policy makers like the Bank of England,
synthesis of predictive densities, which forms a dynamic
the European Central Bank and Federal Reserve Banks
latent (agent) factor model.
in the US routinely publish density forecasts of different
For a discussion on methods for evaluating probabilis-
macroeconomic variables such as inflation, unemploy-
tic forecasts, see Sections 2.12.4 and 2.12.5.
ment rate, and GDP. In finance, density forecasts find
application, in particular, in the areas of financial risk
management and forecasting of stock returns (see, for ex- 2.6.3. Ensembles and predictive probability post processors42
ample, Berkowitz, 2001; Guidolin & Timmermann, 2006; Improved rational decisions are the final objective
Shackleton, Taylor, & Yu, 2010; Tay & Wallis, 2000, inter of modelling and forecasting. Relatively easy decisions
alia). The reader is referred to Section 3.3 for a discussion among a number of alternatives with predefined and
of relevant applications. known outcomes become hard when they are conditioned
Initial work on forecast combination focused on the by future unknown events. This is why one resorts to
combination of point forecasts (see Section 2.6.1). In re- modelling and forecasting, but this is insufficient. To be
cent years attention has shifted towards evaluation, com- successful, one must account for the future conditioning
parison and combination of density forecasts with em- event uncertainty to be incorporated into the decision-
pirical applications that are mostly encountered in the making process using appropriate Bayesian approaches
41 This subsection was written by Alisa Yusupova. 42 This subsection was written by Ezio Todini.
737
(see also Section 2.4), as described by the decision theory likely occur?’’ can be answered using a multi-temporal
literature (Berger, 1985; Bernardo, 1994; DeGroot, 2004). approach (Coccia, 2011; Krzysztofowicz, 2014, see also
The reason is that taking a decision purely based on Section 2.10.2) and results of its applications were pre-
model forecasts is equivalent to assuming the future event sented in Barbetta, Coccia, Moramarco, Brocca, and Todini
(very unlikely, as we know) to equal the forecasted value. (2017), Coccia (2011), and Todini (2017).
Therefore, the estimation of the predictive probability
density is the essential prerequisite to estimating the 2.6.4. The wisdom of crowds43
expected value of benefits (or of losses) to be compared Multiple experts’ forecasts are collected in a wide va-
and traded-off in the decision-making process (Draper & riety of situations: medical diagnostics, weather predic-
Krnjajić, 2013). This highly increases the expected ad- tion, forecasting the path of a hurricane, predicting the
vantages together with the likelihood of success and the outcome of an election, macroeconomic forecasting, and
robustness of the decision (Todini, 2017, 2018). more. One of the central findings from the forecasting
In the past, the assessment of the prediction uncer- literature is that there is tremendous power in combining
tainty was limited to the evaluation of the confidence lim- such experts’ forecasts into a single forecast. The simple
its meant to describe the quality of the forecast. This was average, or what Surowiecki refers to as ‘the wisdom
done using continuous predictive densities, as in the case of crowds’ (Surowiecki, 2005), has been shown to be a
of the linear regression, or more frequently in the form surprisingly robust combined forecast in the case of point
of predictive ensembles. These probabilistic predictions, forecasting (Armstrong, 2001b; Clemen, 1989; Clemen &
describing the uncertainty of the model forecasts given Winkler, 1986, and Section 2.6.1). The average forecast is
(knowing) the observations can be used within the his- more accurate than choosing a forecast from the crowd at
torical period to assess the quality of the models (Todini, random and is sometimes even more accurate than nearly
2016). When predicting into the future, observations are all individuals (Mannes, Larrick, & Soll, 2012). The average
no more available and what we look for (known as predic- point forecast also often outperforms more complicated
tive probability) is the probability of occurrence of the un- point aggregation schemes, such as weighted combina-
known ‘‘future observations’’ given (knowing) the model tions (Smith & Wallis, 2009b; Soule, Grushka-Cockayne,
forecasts. This can be obtained via Bayesian inversion, & Merrick, 2020).
which is the basis of several uncertainty post-processors Mannes et al. (2012) highlight two crucial factors that
used in economy (Diebold, Gunther, & Tay, 1998, and influence the quality of the average point forecast: indi-
Section 3.3), hydrology (Krzysztofowicz, 1999; Schwanen- vidual expertise and the crowd’s diversity. Of the two:
berg, Fan, Naumann, Kuwajima, Montero, & Assis dos ‘‘The benefits of diversity are so strong that one can com-
Reis, 2015; Todini, 1999, 2008, and Section 3.5.4), mete- bine the judgments from individuals who differ a great
orology (Economou, Stephenson, Rougier, Neal, & Mylne, deal in their individual accuracy and still gain from aver-
2016; Granger & Pesaran, 2000; Katz & Lazo, 2011; Reg- aging’’ (Mannes et al., 2012, page 234).
giani & Boyko, 2019, see also Section 3.5.2), etc. Accord- Larrick and Soll (2006) define the idea of ‘bracketing’:
ingly, one can derive a predictive probability from a single In the case of averaging, two experts can either bracket
model forecast to estimate the expected value of a deci- the realisation (the truth) or not. When their estimates
sion by integrating over the entire domain of existence all bracket, the forecast generated by taking their average
the possible future outcomes and theirs effects, weighted performs better than choosing one of the two experts
with their appropriate probability of occurrence. at random; when the estimates do not bracket, aver-
When several forecasts are available to a decision aging performs equally as well as the average expert.
maker, the problem of deciding on which of them one Thus, averaging can do no worse than the average ex-
should rely upon becomes significant. It is generally hard pert, and with some bracketing, it can do much better.
to choose among several forecasts because one model Modern machine learning algorithms such as the random
could be best under certain circumstances but rather forest exploit this property by averaging forecasts from
poor under others. Accordingly, to improve the avail- hundreds of diverse experts (here, each ‘‘expert’’ is a
able knowledge on a future unknown event, predictive regression tree; Grushka-Cockayne, Jose & Lichtendahl,
densities are extended to multiple forecasts to provide 2017).
decision makers with a single predictive probability, con- Only when the crowd of forecasts being combined has
ditional upon several model’s forecasts (Coccia & Todini, a high degree of dispersion in expertise, some individuals
2011; Raftery et al., 1997, see also Section 2.6.1 and in the crowd might stand out, and in such cases, there
Section 2.6.2). could be some benefits to chasing a single expert fore-
A number of available uncertainty post processors caster instead of relying on the entire crowd. Mannes,
can cope with multi-model approaches, such as Bayesian Soll, and Larrick (2014) suggest that combining a small
model averaging (Raftery, 1993; Raftery et al., 1997), crowd can be especially powerful in practice, offering
model output statistics (Glahn & Lowry, 1972; Wilkd, some diversity among a subset of forecasters with an
2005), ensemble model output statistics (Gneiting, Raftery, minimum level of expertise.
Westveld, & Goldman, 2005), and model conditional pro- When working with probabilistic forecasting (see also
cessor (Coccia & Todini, 2011; Todini, 2008). Sections 2.6.2, 2.6.3 and 2.12.4), averaging probabilities
Finally, important questions such as: (i) ‘‘what is the is the most widely used probability combination method
probability that an event will happen within the next
x hours?’’ and (ii) ‘‘at which time interval it will most 43 This subsection was written by Yael Grushka-Cockayne.
738
(Clemen, 2008; Cooke, 1991; Hora, 2004). Stone (1961) interpretable, and so they are mostly used for forecasting
labelled such an average the linear opinion pool. O’Hagan, rather than for policy analysis.
Buck, Daneshkhah, Richard Eiser, Garthwaite, Jenkinson, Big data has been effectively used in nowcasting, where
Oakley, and Rakow (2006) claimed that the linear opinion improved estimates of the forecast origin lead to better
pool is: ‘‘hard to beat in practice’’. forecasts, absent any later shifts. Nowcasting can benefit
Although diversity benefits the average point fore- from large data sets as the events have happened and
cast, it can negatively impact the average probability the information is available, see Castle et al. (2018) for
forecast. As the crowd’s diversity increases, the aver- a nowcasting application, and Section 2.5.1 on leading
age probability forecast becomes more spread out, or indicators. However, the benefits of big data are not as
more underconfident (Dawid, DeGroot, Mortera, Cooke, evident in a forecasting context where the future values
French, Genest, et al., 1995; Hora, 2004; Ranjan & Gneit- of all added variables also need to be forecast and are as
ing, 2010). Averaging quantiles, instead of probabilities, uncertain as the variable(s) of interest.
can offer sharper and better calibrated forecasts (Lich- Macroeconomic time series data are highly
tendahl, Grushka-Cockayne, & Winkler, 2013). Trimmed non-stationary, with stochastic trends and structural breaks.
opinion pools can be applied to probability forecasts, also The methods of cross-validation and hold-back, frequently
resulting in better calibrated forecasts (Jose, Grushka- used to handle bid data, often assume that the data
Cockayne, & Lichtendahl, 2014, see also Section 2.12.5). generating process does not change over time. Forecasting
The ubiquity of data and the increased sophistication models that assume the data are drawn from a stationary
of forecasting methods results in more use of probabilistic distribution (even after differencing) do not forecast well
forecasts. While probabilities are more complex to elicit, ex ante. So while there seems to be lots of mileage in
evaluate, and aggregate compared to point estimates, they improving forecasts using big data, as they allow for more
do contain richer information about the uncertainty of flexible models that nest wider information sets, more
interest. The wisdom of combining probabilities, however, general dynamics and many forms of non-linearity, the
utilises diversity and expertise differently than combining statistical problems facing ‘small’ data forecasting mod-
point forecasts. When relying on a crowd, eliciting point els do not disappear (Doornik & Hendry, 2015; Harford,
forecasts versus eliciting probabilities can significantly 2014). Castle, Doornik, and Hendry (2020a) do not find
influence the type of aggregation one might choose to use. improvements in forecasting from big data sets over small
models. It is essential to keep in mind the classical sta-
2.7. Data-driven methods tistical problems of mistaking correlation for causation,
ignoring sampling biases, finding excess numbers of false
2.7.1. Forecasting with big data44 positives and not handling structural breaks and non-
The last two decades have seen a proliferation of lit- constancies both in- and out-of-sample, in order to guard
erature on forecasting using big data (Hassani & Silva, against these issues in a data abundant environment.
2015; Swanson & Xiong, 2018; Varian, 2014) but the
evidence is still uncertain as to whether the promised 2.7.2. Forecasting on distributed systems45
improvements in forecast accuracy can systematically be Big data is normally accompanied by the nature that
realised for macroeconomic phenomena. In this section observations are indexed by timestamps, giving rise to
we question whether big data will significantly increase big data time series characterised by high frequency and
the forecast accuracy of macroeconomic forecasts. Athey long-time span. Processing big data time series is ob-
(2018) argues that machine learning methods are seen structed by a wide variety of complications, such as sig-
as an efficient approach to dealing with big data sets, nificant storage requirements, algorithms’ complexity and
and we present these methods before questioning their high computational cost (Galicia, Torres, Martínez-Álvarez,
success at handling non-stationary macroeconomic data & Troncoso, 2018; L’heureux, Grolinger, Elyamany, & Capretz,
that are subject to shifts. Section 2.7.2 discusses big data 2017; Wang, Kang, Hyndman, & Li, 2020; Wang, Yang,
in the context of distributed systems, and Section 2.7.11 Du, & Niu, 2018). These limitations accelerate the great
evaluates a range of machine learning methods frequently demand for scalable algorithms. Nowadays, increasing
applied to big data. attention has been paid to developing data mining tech-
The tools used to analyse big data focus on regulariza- niques on distributed systems for handling big data time
tion techniques to achieve dimension reduction, see Kim series, including but not limited to processing (Mirko &
and Swanson (2014) for a summary of the literature. Kantelhardt, 2013), decomposition (Bendre & Manthalkar,
This can be achieved through selection (such as Automet- 2019), clustering (Ding, Wang, Dang, Fu, Zhang, & Zhang,
rics, Doornik, 2018, but also see Section 2.5.3 and Sec- 2015), classification (Triguero, Peralta, Bacardit, García, &
tion 2.5.4), shrinkage (including Ridge Regression, LASSO, Herrera, 2015), and forecasting (Galicia et al., 2018). For
and Elastic Nets, see Section 2.7.11 but also Section 3.3.13 forecasting problems based on big data sets and/or large
for an applied example), variable combination (such as sets of predictors, please refer to Sections 2.7.1 and 3.3.13.
Principal Components Analysis and Partial Least Squares), Distributed systems, initially designed for independent
and machine learning methods (including Artificial Neu- jobs, do not support to deal with dependencies among
ral Networks, see Section 2.7.8). Many of these methods observations, which is a critical obstacle in time series
are ‘black boxes’ where the algorithms are not easily processing (Li, Noorian, Moss, & Leong, 2014; Wang, Kang,
44 This subsection was written by Jennifer L. Castle. 45 This subsection was written by Xiaoqian Wang.
739
Hyndman, & Li, 2020). Various databases (e.g., InfluxDB,46 & Macready, 1997), model selection (see Section 2.5.4)
OpenTSDB,47 RRDtool,48 and Timely49 ) can function as and model combination (see Section 2.6) are involved in
storage platforms for time series data. However, none finalisation of algorithms for forecasting on distributed
of these databases supports advanced analysis, such as systems (e.g., Bendre & Manthalkar, 2019; Galicia et al.,
modelling, machine learning algorithms and forecasting. 2019; Li et al., 2014; Xu et al., 2020).
Additional considerations are therefore required in fur-
ther processing time series on distributed systems. Mirko 2.7.3. Agent-based models50
and Kantelhardt (2013) developed the Hadoop.TS library Time series forecasting involves use of historical data
for processing large-scale time series by creating a time to predict values for a specific period time in future. This
series bucket. Li et al. (2014) designed an index pool approach assumes that recent and historical patterns in
serving as a data structure for assigning index keys to time the data will continue in the future. This assumption is
series entries, allowing time series data to be sequen- overly ingenuous. However, this is not reliable in some
tially stored on HDFS (Hadoop Distributed File System: situations. For example, (i) forecasting COVID-19 cases
Shvachko, Kuang, Radia, & Chansler, 2010) for MapRe- (see also Section 3.6.2) where, due to interventions and
duce (Dean & Ghemawat, 2008) jobs. Chen, Li, Rong, Bilal, control measures taken by the governments and due to
Li, and Philip (2019a) proposed a data compression and the change in personal behaviour, the disease transmis-
abstraction method for large-scale time series to facilitate sion pattern changes rapidly, and (ii) forecasting sales of a
the periodicity-based time series prediction in a parallel new product (see also Section 3.2.6): external factors such
manner. as advertisement, promotions (see Section 3.2.5), social
The evolution of the algorithms for efficiently fore- learning, and imitation of other individuals change the
casting big data time series on distributed systems is system behaviour.
largely motivated by a wide range of applications in- In such circumstances to make reliable forecasts it
cluding meteorology, energy, finance, transportation and is important to take into account all information that
farming (Chen et al., 2019a; Galicia et al., 2018; Hong & might influence the variable that is being forecast. This
Pinson, 2019; Sommer, Pinson, Messner, & Obst, 2020). information includes a variety of environmental-level and
Researchers have made several attempts to make ma- individual-level factors. An agent-based modelling is a
chine learning techniques available for big data time se- powerful tool to explore such complex systems. Agent-
ries forecasting on distributed systems (Galicia, Talavera- based modelling approach is useful when, (i) data avail-
Llames, Troncoso, Koprinska, & Martínez-Álvarez, 2019; ability is limited, (ii) uncertainty of various interventions
Li et al., 2014; Talavera-Llames, Pérez-Chacón, Martínez- in place and a rapidly changing social environment, and
Ballesteros, Troncoso, & Martínez-Álvarez, 2016; Xu, Liu, (iii) limited understanding of the dynamics of the variable
& Long, 2020). Talavera-Llames et al. (2016) presented of interest.
a nearest neighbours-based algorithm implemented for Agent-based modelling disaggregates systems into in-
Apache Spark (Zaharia, Xin, Wendell, Das, Armbrust, Dave, dividual level and explores the aggregate impact of in-
Meng, Rosen, Venkataraman, Franklin, Ghodsi, Gonzalez, dividual behavioural changes on the system as a whole.
Shenker, & Stoica, 2016) and achieved satisfactory fore- In other words, the key feature of agent-based modelling
casting performance. Galicia et al. (2018) proposed a scal- is the bottom-up approach to understand how a system’s
able methodology which enables Spark’s MLlib (Meng complexity arises, starting with individual level (see also
et al., 2016) library to conduct multi-step forecasting by Section 2.10.1). As opposed to this, the conventional time
splitting the multi-step forecasting problem into h subseries forecasting approaches are considered top-down
problems (h is the forecast horizon). approaches.
Another strand of the literature on forecasting big Agent-based models have two main components: (i)
data time series is to improve time-consuming estimation Agents, and (ii) Rules, sometimes referred as procedures
methods using a MapReduce framework. Sheng, Zhao, and interactions. Agents are individuals with autonomous
Leung, and Wang (2013) learned the parameters of echo behaviour. Agents are heterogeneous. Each agent individ-
state networks for time series forecasting by design- ually assesses on the basis of a set of rules. An agent-
ing a parallelised extended Kalman filter involving two based modelling approach simulates how heterogeneous
MapReduce procedures. Recently, Sommer et al. (2020) agents interact and behave to assess the role of differ-
accurately estimated coefficients of a high-dimensional ent activities on the target variable. According to Farmer
ARX model by designing two online distributed learning and Foley (2009), ‘‘An agent-based model is a comput-
algorithms. Wang, Kang, Hyndman, and Li (2020) resolved erised simulation of a number of decision-makers (agents)
challenges associated with forecasting ultra-long time and institutions, which interact through prescribed rules’’.
series from a new perspective that global estimators are Their paper highlights the importance of adopting agent-
approximated by combining the local estimators obtained based models as a better way to help guide financial
from subseries by minimising a global loss function. Be- policies.
sides, inspired by the no-free-lunch theorem (Wolpert A general framework for agent-based modelling in-
volves three main stages (See Fig. 4): (i) setup envi-
46 Available at https://www.influxdata.com/time-series-database/. ronments and agents, (ii) agent-based modelling, and
47 Available at http://opentsdb.net/ (iii) calibration and validation. The first two steps are
48 Available at https://oss.oetiker.ch/rrdtool/.
49 Available at https://code.nsa.gov/timely/. 50 This subsection was written by Thiyanga S. Talagala.
740
Fig. 4. Framework for Agent-based modelling.
self-explanatory. The final step involves calibration of the but not limited to time series clustering (Kang, Belušić,
model with empirical data and then evaluates whether & Smith-Miles, 2014, 2015; Wang, Smith-Miles, & Hynd-
the agent-based model mirrors the real-world man, 2006, and Section 2.7.12), classification (Fulcher, Lit-
system/target. The validation step involves testing the tle, & Jones, 2013; Nanopoulos, Alcock, & Manolopoulos,
significance of the difference between agent-based model 2001), anomaly detection (Kang, 2012; Talagala, Hynd-
results and real data collected about the target. One of man, & Smith-Miles, 2020, and Section 2.2.3), and
the main challenges in designing an agent-based model forecasting (Kang, Hyndman, & Smith-Miles, 2017; Montero-
is finding a balance between model simplicity and model Manso, Athanasopoulos, Hyndman, & Talagala, 2020, see
realism (see also Section 2.5.2). The KISS principle (keep also Section 2.2.5). The choice of features depends on
it simple, stupid), introduced by Axelrod (1997) is often the nature of the data and the application context. The
cited as an effective strategy in agent-based modelling. state-of-the-art time series feature representation meth-
A high level of expertise in the area of the subject is ods quantify a wide range of time series characteristics,
necessary when developing an agent-based model. including simple summary statistics, stationarity (Montero-
Despite these limitations and challenges, agent-based Manso et al., 2020; Wang, Kang, Petropoulos, & Li, 2021),
modelling has been used extensively to model infec- model fits (Christ, Braun, Neuffer, & Kempa-Liehr, 2018;
tious disease transmission and forecasting (Tracy, Cerdá, Fulcher & Jones, 2014), time series imaging (Li, Kang & Li,
& Keyes, 2018; Venkatramanan, Lewis, Chen, Higdon, Vul- 2020), and others. In the forecasting community, two lines
likanti, & Marathe, 2018). Agent-based modelling of forecasting approaches have been developed using time
approaches have been widely used in early phases of series features, namely feature-based model selection and
the COVID-19 outbreak, to assess the impact of differ- feature-based model combination. The motivation behind
ent interventions on disease spread and forecasts (Wal- them is no single model always performs the best for
lentin, Kaziyeva, & Reibersdorfer-Adelsberger, 2020). In all time series. Instead of choosing one model for all the
a review paper, Weron (2014) states some applications data, features can be used to obtain the most appropriate
of agent-based models for electricity demand forecast- model or the optimal combination of candidate models,
ing. Xiao and Han (2016) use agent-based models to per series.
forecast new product diffusion. Furthermore, thinking As early as in 1972, Reid (1972) argues that time series
the other way around, Hassan, Arroyo, Galán Ordax, An- characteristics provide valuable information in forecast
tunes, and Pavón Mestras (2013) explain how forecasting model selection, which is further echoed by
principles can be applied in agent-based modelling. Makridakis and Hibon (1979). One way to forecast an
extensive collection of time series is to select the most
appropriate method per series according to its features.
2.7.4. Feature-based time series forecasting51
Pioneer studies focus on rule-based methods (for exam-
A time series feature can be any statistical represen-
ple, Arinze, 1994; Wang, Smith-Miles, & Hyndman, 2009)
tation of time series characteristics. A vast majority of
to recommend the ‘‘best’’ forecasting model per series
time series mining tasks are based on similarity quan-
based on its features. Another line of approaches apply
tification using their feature representations, including
regression to study how useful features are in predicting
which forecasting method performs best (for example,
51 This subsection was written by Yanfei Kang. Meade, 2000; Petropoulos, Makridakis, Assimakopoulos,
741
& Nikolopoulos, 2014). With the advancement of ma- 1994). But, if the time series process is driven from iid
chine learning (see also Section 2.7.10), more recent lit- innovations, another way of resampling can be used.
erature uses ‘‘meta-learning’’ to describe the process of The iid Bootstrap can then be easily extended to a
automatically acquiring knowledge for forecast model dependent setup. That was the spirit of sieve bootstrap
selection. The first such study is by Prudêncio and Luder- proposed by Bühlmann (1997). This method is based on
mir (2004), who apply decision trees for forecast model the idea of fitting parametric models first and resampling
selection. Lemke and Gabrys (2010) compare different from the residuals. Such models include, for example,
meta-learning approaches to investigate which model the linear regression (Freedman, 1981) and autoregressive
works best in which situation. Kang et al. (2017) propose time series (Efron & Tibshirani, 1986). This approach is
using feature spaces to visualise the strengths and weak- different from the previous bootstrap methods for de-
nesses of different forecasting methods. Other algorithms pendent data; the sample bootstrap is (conditionally) sta-
such as neural networks (see also Section 2.7.8) and tionary and does not present a structure of dependence.
random forecasts are also applied to forecast model se- Another different feature is that the sieve bootstrap sam-
lection (Kück, Crone, & Freitag, 2016; Talagala, Hyndman, ple is not a subsample from the original data, as in the
& Athanasopoulos, 2018). previous methods. Observe that even if the sieve boot-
One of the pioneering studies in feature-based forecast strap is based on a parametric model, it is nonparametric
combination is the rule-based approach by Collopy and in its spirit. The AR model (see Section 2.3.4) here is just
Armstrong (1992), who develop 99 rules for forecast com- used to filter the residuals series.
bination based on 18 features. Recently, Kang, Hyndman A few years ago, the sieve bootstrap was used for es-
and Li (2020) use 26 features to predict the performances timating forecast intervals (Andrés, Peña, & Romo, 2002;
of nine forecasting methods with nonlinear regression Zagdański, 2001). Motivated by these works, Cordeiro
models, and obtain the combination weights as a tailored and Neves (2006, 2009, 2010) developed a procedure to
softmax function of the predicted forecasting errors. The estimate point forecasts. The idea of these authors was
feature-based forecast model averaging (FFORMA) frame- to fit an exponential smoothing model (see Section 2.3.1)
work proposed by Montero-Manso et al. (2020) employ to the time series, extract the residuals and then apply
42 features to estimate the optimal combination weights the sieve bootstrap to the residuals. Further developments
of nine forecasting methods based on extreme gradient of this procedure include the estimation of forecast in-
boosting (XGBoost, Chen & Guestrin, 2016). Li, Kang et al. tervals (Cordeiro & Neves, 2014) and also the detection,
(2020) first transform time series into images, and use estimation and imputation of missing data (Cordeiro &
features extracted from images to estimate the optimal Neves, 2013). In a recent work (Bergmeir et al., 2016),
combination weights. For feature-based interval forecast- a similar approach was also consider, the residuals were
ing, Wang, Kang et al. (2021) investigate how time series extracted and resampled using moving block bootstrap
features affect the relative performances of prediction (see Section 2.7.6 for further discussion).
intervals from different methods, and propose a general Bickel and Freedman (1981) and later in Angus (1992)
feature-based interval forecasting framework to provide showed that in extreme value theory, the bootstrap ver-
reliable forecasts and their uncertainty estimation. sion for the maximum (or minimum) does not converge
to the extremal limit laws. Zelterman (1993) pointed out
2.7.5. Forecasting with bootstrap52 that ‘‘to resample the data for approximating the dis-
The bootstrap methodology has been widely applied tribution of the k largest observations would not work
in many areas of research, including time series analysis. because the ‘pseudo-samples’ would never have values
The bootstrap procedure (Efron, 1979) is a very popular greater than Xn:n ’’53 . A method considering to resample a
methodology for independent data because of its sim- smaller size than the original sample was proposed in Hall
plicity and nice properties. It is a computer-intensive (1990). Recently, Neves and Cordeiro (2020) used this
method that presents solutions in situations where the idea and developed a preliminary work in modelling and
traditional methods fail or are very difficult to apply. forecasting extremes in time series.
However, Efron’s bootstrap (iid bootstrap) has revealed
itself inefficient in the context of dependent data, such as 2.7.6. Bagging for time series forecasting54
in the case of time series, where the dependence struc- The term bagging was proposed by Breiman (1996) to
ture arrangement has to be kept during the resampling describe the generation of several versions of a predictor,
scheme. via Bootstrap procedures (introduced in Section 2.7.5),
Most of the resampling for dependent data consider with a final aggregation stage. Thus, ‘‘bootstrap
segments of the data to define blocks, such that the de- aggregating’’ was established as bagging. The main idea
pendence structure within each block can be kept. Differ- is to improve predictors’ accuracy once the data sets,
ent versions of blocking differ in the way as blocks are draw randomly with replacement, will approximating
constructed: the nonoverlapping block bootstrap (Carl- the original distribution. The author argues that bagging
stein, 1990), the moving block bootstrap (Künsch, 1989), works for unstable procedures, but it was not tested for
the circular block bootstrap (Politis & Romano, 1992), time series. Years after, Kilian and Inoue (2004) suggested
and the stationary block bootstrap (Politis & Romano,
53 max(X , . . . , X ).
1 n
52 This subsection was written by Clara Cordeiro. 54 This subsection was written by Fernando Luiz Cyrino Oliveira.
742
the first attempts for temporal dependent data. For data- 2.7.7. Multi-step ahead forecasting55
driven methods, to forecasting and simulation time series Given a univariate time series comprising n observa-
and deal with predictors ensembles, bagging has shown tions, y1 , y2 , . . . , yn , multi-step ahead point forecasting
as a powerful tool. involves producing point estimates of the H future val-
A general framework for ensemble forecasting meth- ues yn+1 , yn+2 , . . . , yn+H , where H > 1, is the forecast
ods involves four main stages: (i) data treatment, (ii) re- horizon (Ben Taieb, 2014).
sampling, (iii) forecasting, and (iv) aggregation. However, The (naive) recursive strategy estimates a one-step-
for time series, bootstrap should be done carefully, as the ahead autoregressive model to predict yt +1 from
serial dependence and non-stationarity must be consid- yt , yt −1 , . . . , by minimising the one-step-ahead forecast
ered. errors. Each forecast is then obtained dynamically by iter-
As mentioned in Section 2.7.5, this led Bergmeir et al. ating the model H times, and by plugging in the missing
(2016) to propose a bagging version for exponential lagged values with their respective forecasts. The direct
smoothing, the Bagged ETS. As pre-treatment, after a strategy builds separate h-step-ahead models to predict
Box–Cox transformation, the series is decomposed into yt +h from yt , yt −1 , . . . for h = 1, 2, . . . , H, by minimising
trend, seasonal, and remainder components via STL de- h-step-ahead forecast errors, and forecasts are computed
composition (Cleveland et al., 1990). The resampling stage directly by the estimated models.
uses moving block bootstrap (MBB: Lahiri & Lahiri, 2003), In theory, with linear models, model misspecification
applied to the remainder. There are several discussions plays an important role in the relative performance be-
in the literature about this procedure, mainly regarding tween the recursive and direct strategy (Chevillon, 2007).
the size of the blocks. MBB resampling the collection of If the model is correctly specified, the recursive strategy
overlapping (consecutive) blocks of observations. The idea benefits from more efficient parameter estimates, while
is to keep the structure still present in the remainder. The the direct strategy is more robust to model misspeci-
forecasts are obtained via ETS methods (see Section 2.3.1) fication. With nonlinear models, recursive forecasts are
and, for the final aggregation, the authors adopted the known to be asymptotically biased (Fan & Yao, 2005; Lin
median. Their method is evaluated on the M3 data set & Granger, 1994; Teräsvirta, Tjostheim, & Granger, 2010),
and outperformed the original benchmarks. The work and the direct strategy is often preferred over the recur-
of Bergmeir et al. (2016) inspired many others: Dantas, sive strategy since it avoids the accumulation of forecast
Cyrino Oliveira, and Varela Repolho (2017) applied the errors. In practice, the results are mixed (Atiya, El-shoura,
idea for air transport demand data and de Oliveira and Shaheen, & El-sherif, 1999; Kline, 2004; Marcellino, Stock,
Cyrino Oliveira (2018) for energy consumption, proposing & Watson, 2006; Pesaran, Pick, & Timmermann, 2011;
the so-called remainder sieve bootstrap (RSB). Sorjamaa, Hao, Reyhani, Ji, & Lendasse, 2007), and depend
Dantas and Cyrino Oliveira (2018) proposed an ex- on many interacting factors including the model complex-
tension to the Bagged ETS where bagging and exponen- ity (see also Section 2.5.2), the (unknown) underlying data
tial smoothing are combined with clustering methods generating process, the number of observations, and the
(clustering-based forecasting methods are discussed in forecast horizon (see also Section 2.7.4).
Section 2.7.12). The approach aims to consider and re- Hybrids and variants of both recursive and direct strate-
duce the covariance effects among the ensemble time gies have been proposed in the literature. For example,
series, creating clusters of similar forecasts – since it could one of the hybrid strategies (Sorjamaa & Lendasse, 2006;
impact the variance of the group. A variety of forecasts Zhang & Hutchinson, 1994; Zhang, Zhou, Chang, Yang,
are selected from each cluster, producing groups with & Li, 2013) first produce recursive forecasts, then adjust
reduced variance. these forecasts by modelling the multi-step forecast er-
In light of the aforementioned, there are several possi- rors using a direct strategy (Ben Taieb & Hyndman, 2014).
bilities for each stage of the mentioned framework. In this Variants of the recursive strategy match the model esti-
context, to investigate the reasons why bagging works mation and forecasting loss functions by minimising the
well for time series forecasting, Petropoulos et al. (2018a) implied h-step-ahead recursive forecast errors (Bhansali
explored three sources of uncertainty: model form, data, & Kokoszka, 2002; Bontempi, Birattari, & Bersini, 1999;
and parameter. While arguably bagging can handle all McNames, 1998). Variants of the direct strategy exploit
of them, Petropoulos et al. (2018a) showed that sim- the fact that the errors of different models are serially cor-
ply tackling model uncertainty is enough for achieving a related (Chen, Yang, & Hafner, 2004; Franses & Legerstee,
superior performance, leading to the proposal of a Boot- 2009c; Lee & Billings, 2003; Pesaran et al., 2011). The
strap Model Combination (BMC) approach, where differ- idea is to reduce the forecast variance of independently
ent model forms are identified in the ensemble and fitted selected models by exploiting the relatedness between
to the original data. the forecasting tasks, as in multi-task learning (Caruana,
Finally, Meira, Cyrino Oliveira, and Jeon (2020) pro- 1997). For example, a multi-horizon strategy will measure
posed ‘‘treating and pruning’’ strategies to improve the forecast accuracy (see Section 2.12.2) by averaging the
performance of prediction intervals for both model se- forecast errors over multiple forecast horizons (Bontempi
lection and forecast combinations. Testing over a large & Ben Taieb, 2011; Kline, 2004). Different multi-horizon
set of real time series from the M forecasting competi- strategies can be specified, with different formulation of
tions (see also Section 2.12.7), their results highlighted the objective function (Ben Taieb, Sorjamaa, & Bontempi,
the importance of analysing the prediction intervals of the
ensemble series before the final aggregation. 55 This subsection was written by Souhaib Ben Taieb.
743
2010). One particular case is the multi-output strategy adjusting the weights that connect the nodes in a way that
which estimates a single model for all horizons by min- the network maps the input value of the training data to
imising the average forecast error over the entire forecast the corresponding output value. This mapping is based on
horizon (Bontempi & Ben Taieb, 2011). a loss function, the choice of which depends on the nature
Forecasting strategies are often model-dependent, es- of the forecasting problem. The most common NN pro-
pecially with machine learning models (see Section 2.7.10). cedure, is the back-propagation of errors (for additional
Furthermore, model architecture and parameters are of- details on training, see Section 2.7.11).
ten trained by taking into account the chosen forecasting The simpler and most common NN topology, is the
strategy. For example, we can naturally implement and Multilayer Forward Perceptron (MLP). In MLP, the hidden
train recursive forecasting models using recurrent neural nodes contain the sigmoid function and the information
networks (see also Section 2.7.8). Also, different specifi- moves in forward direction (from the inputs to the output
cations of the decoder in sequence-to-sequence models nodes). An another NN topology where the information
will induce different forecasting strategies, including vari- moves also only in a forward direction is the Radial Basis
ants of direct and multi-horizon strategies. For more Function NN (RBF). Now the hidden neurons compute
details, we refer the reader to Hewamalage, Bergmeir, and the Euclidean distance of the test case from the neuron’s
Bandara (2021) and Section 4.2 in Benidis et al. (2020). centre point and then applies the Gaussian function to
Which forecasting strategy is best is an empirical ques- this distance using the spread values. Recurrent Neural
tion since it involves a tradeoff between forecast bias and Networks (RNNs) are NN topologies that allow previous
variance (Ben Taieb & Atiya, 2015; Ben Taieb, Bontempi, outputs to be used as inputs while having hidden states.
Atiya, & Sorjamaa, 2012). Therefore, the forecasting strat- The information moves both forwards and backwards.
egy should be part of the design choices and the model RNNs have short-term memory and inputs are taken po-
selection procedure of any multi-step-ahead forecasting tentially from all previous values. MLPs, RBFs and RNNs
model. are universal function approximators (Hornik, 1991; Park
& Sandberg, 1991; Schäfer & Zimmermann, 2006). How-
2.7.8. Neural networks56 ever, the amount of NN complexity in terms of hidden
Neural Networks (NNs) or Artificial Neural Networks layers and nodes to reach this property, might make the
(ANNs) are mathematical formulations inspired by the NN topology computationally unfeasible to train (see also
work and functioning of biological neurons. They are char- the discussion in Section 2.7.11). For the interaction of
acterized by their ability to model non-stationary, nonlin- NNs with the probability theory, we refer the reader to
ear and high complex datasets. This property along with last part of Section 2.7.9.
the increased computational power have put NNs in the
frontline of research in most fields of science (De Gooijer 2.7.9. Deep probabilistic forecasting models57
& Hyndman, 2006; Zhang, Eddy Patuwo, & Y. Hu, 1998). Neural networks (Section 2.7.8) can be equipped to
A typical NN topology is consisted by three types of provide not only a single-valued forecast, but rather the
layers (input, hidden and output) and each layer is con- entire range of values possible in a number of ways (see
sisted by nodes. The first layer in every NN, is the input also Sections 2.6.2 and 2.6.3). We will discuss three se-
layer and the number of its nodes corresponds to the lected approaches in the following, but remark that this is
number of explanatory variables (inputs). The last layer is a subjective selection and is by far not comprehensive.58
the output layer and the number of nodes corresponds to
the number of response variables (forecasts). Between the 1. Analogously to linear regression and Generalised
input and the output layer, there is one or more hidden Linear Models, obtaining probabilistic forecasts can
layers where the nodes define the amount of complexity be achieved by the neural network outputting not
the model is capable of fitting. Most NN topologies in the the forecasted value itself but rather parameters of
input and the first hidden layer contain an extra node, probability distribution or density (Bishop, 2006).
called the bias node. The bias node has a fixed value In forecasting, a prominent example is the DeepAR
of one and serves a function similar to the intercept in model (Salinas, Valentin, Jan & Januschowski, 2019),
traditional regression models. Each node in one layer has which uses a recurrent neural network architecture
connections (weights) with all or a subset (for example, and assumes the probability distribution to be from
for the convolutional neural network topology) of the a standard probability density function (e.g., nega-
nodes of the next layer. tive binomial or Student’s t). Variations are possi-
NNs process the information as follows: the input ble, with either non-standard output distributions
nodes contain the explanatory variables. These variables in forecasting such as the multinomial distribu-
are weighted by the connections between the input and tion (Rabanser, Januschowski, Flunkert, Salinas, &
the first hidden nodes, and the information reaches to the Gasthaus, 2020) or via representing the probability
hidden nodes as a weighted sum of the inputs. In the hid-
den nodes, there is usually a non-linear function (such as 57 This subsection was written by Tim Januschowski.
the sigmoid or the RelU) which transform the information 58 For example, contemporary topics in machine learning such as
received. This process is repeated until the information generative adversarial networks can be naturally lifted to forecasting
reaches the output layer as forecasts. NNs are trained by and similarly, more traditional probabilistic machine learning ap-
proaches such as Gaussian Processes (Maddix, Wang, & Smola, 2018).
We ignore the important area of Bayesian deep learning (see Wang &
56 This subsection was written by Georgios Sermpinis. Yeung, 2016, for a survey) entirely here for lack of space.
744
density as cumulative distribution function (Sali- 2.7.10. Machine learning59

nas, Michael et al., 2019) or the quantile func- Categorising forecasting methods into statistical and
tion (Gasthaus et al., 2019). machine learning (ML) is not trivial, as various criteria
2. An alternative approach is to apply concepts for can be considered for performing this task (Januschowski
quantile regression (Koenker, 2005) to neural net- et al., 2020). Nevertheless, more often than not, fore-
works, e.g., by making the neural network pro- casting methods are categorised as ML when they do
duce values for selected quantiles directly (Wen, not prescribe the data-generating process, e.g., through
Torkkola, Narayanaswamy, & Madeka, 2017). a set of equations, thus allowing for data relationships
3. It is possible to combine neural networks with to be automatically learned (Barker, 2020). In this re-
existing probabilistic models. For example, neu- spect, methods that build on unstructured, non-linear
regression algorithms (see also Section 2.3.2), such as
ral networks can parametrise state space mod-
Neural Networks (NN), Decision Trees, Support Vector
els (Durbin & Koopman, 2012) as an example for
Machines (SVM), and Gaussian Processes, are considered
another class of approaches (Rangapuram, Seeger,
as ML (Makridakis et al., 2018).
Gasthaus, Stella, Wang, & Januschowski, 2018), dy-
Since ML methods are data-driven, they are more
namic factor models (Geweke, 1977) with neural
generic and easier to be adapted to forecast series of dif-
networks (Wang, Smola, Maddix, Gasthaus, Fos- ferent characteristics (Spiliotis, Kouloumos, Assimakopou-
ter, & Januschowski, 2019) or deep temporal point los & Makridakis, 2020). However, ML methods also dis-
processes (Turkmen, Wang, & Januschowski, 2019). play some limitations. First, in order for ML methods to
take full advantage of their capacity, sufficient data are
The appeals of using neural networks for point fore- required. Thus, when series are relatively short and dis-
casts carry over to probabilistic forecasts, so we will only play complex patterns, such as seasonality and trend, ML
comment on the elegance of modern neural network pro- methods are expected to provide sub-optimal forecasts
gramming frameworks. To the forecasting model builder, if the data are not properly pre-processed (Makridakis
the availability of auto gradient computation, the integra- et al., 2018; Zhang et al., 1998). On the other hand, when
tion with highly-tuned optimisation algorithms and scal- dealing with long, high-frequency series, typically found
ability considerations built into the frameworks, means in energy (Chae, Horesh, Hwang, & Lee, 2016, but also
that the time from model idea to experimental evalua- Section 3.4), stock market (Moghaddam, Moghaddam, &
tion has never been shorter. In the examples above, we Esfandyari, 2016, and Section 3.3), and demand (Carmo &
brushed over the need to have loss functions with which Rodrigues, 2004, but also Section 3.2) related applications,
we estimate the parameters of the neural networks. Stan- ML methods can be applied with success. Second, com-
dard negative log-likelihood based approaches are easily putational intensity may become relevant (Makridakis,
expressible in code as are approaches based on non- Spiliotis et al., 2020), especially when forecasting numer-
standard losses such as the continuous ranked probability ous series at the weekly and daily frequency (Seaman,
score (Gneiting et al., 2007, and Section 2.12.4). With 2018) or long-term accuracy improvements over tradi-
open source proliferating in the deep learning commu- tional methods are insignificant (Nikolopoulos & Petropou-
los, 2018). Third, given that the effective implementation
nity, most of the above examples for obtaining probabilis-
of ML methods strongly depends on optimally deter-
tic forecasts can readily be test-driven (see, for example,
mining the values of several hyper-parameters, related
Alexandrov et al., 2019).
both with the forecasting method itself and the training
For the future, we see a number of open challenges.
process, considerable complexity is introduced, significant
Most of the approaches mentioned above are univari- resources are required to set up the methods, and high
ate, in the following sense. If we are interested in fore- experience and a strong background in other fields than
casting values for all time series in a panel, we may forecasting, such as programming and optimisation, are
be interested in modelling the relationship among these needed.
time series. The aforementioned approaches mostly as- In order to deal with these limitations, ML meth-
sume independence of the time series. In recent years, ods can be applied in a cross-learning (CL) fashion in-
a number of multivariate probabilistic forecasting mod- stead of a series-by-series one (Makridakis, Spiliotis et al.,
els have been proposed (Rangapuram, de Bezenac, Beni- 2020), i.e., allow the methods to learn from multiple
dis, Stella, & Januschowski, 2020; Salinas, Michael et al., series how to accurately forecast the individual ones
2019), but much work remains to obtain a better un- (see also Section 2.12.7). The key principle behind CL is
derstanding. Another counter-intuitive challenge for neu- that, although series may differ, common patterns may
ral networks is to scale them down. Neural networks occur among them, especially when data are structured
are highly parametrised, so in order to estimate param- in a hierarchical way and additional information, such
eters correctly, panels with lots of time series are needed. as categorical attributes and exogenous/explanatory vari-
However, a large part of the forecasting problem land- ables (see Section 2.2.5), is provided as input (Fry &
scape (Januschowski & Kolassa, 2019) consists of fore- Brundage, 2020). The CL approach has several advantages.
casting problems with only a few time series. Obtaining First, computational time can be significantly reduced
good uncertainty estimates with neural networks in these
settings is an open problem. 59 This subsection was written by Evangelos Spiliotis.
745
as a single model can be used to forecast multiple se- data environments; see Filippou, Rapach, Taylor, and Zhou
ries simultaneously (Semenoglou, Spiliotis, Makridakis, (2020) for a recent empirical application.
& Assimakopoulos, 2021). Second, methods trained in A second reason for the popularity of machine learning
a particular dataset can be effectively used to provide in the era of big data is the existence of powerful tools
forecasts for series of different datasets that display sim- for accommodating complex predictive relationships. In
ilar characteristics (transfer-learning), thus allowing the many contexts, a linear specification appears overly re-
development of generalised forecasting methods (Ore- strictive, as it may neglect important nonlinearities in the
shkin, Carpov, Chapados, & Bengio, 2020a). Third, data data that can potentially be exploited to improve forecast-
limitations are mitigated and valuable information can be ing performance. Neural networks (NNs; see Section 2.7.8)
exploited at global level, thus allowing for patterns shared are perhaps the most popular machine-learning device
among the series, such as seasonal cycles (Dekker, van for modelling nonlinear predictive relationships with a
Donselaar, & Ouwehand, 2004) and special events (Huber large number of features. Under a reasonable set of as-
& Stuckenschmidt, 2020), to be effectively captured. sumptions, a sufficiently complex NN can approximate
Based on the above, CL is currently considered the any smooth function (for example, Barron, 1994; Cybenko,
most effective way of applying ML for batch time series 1989; Funahashi, 1989; Hornik, Stinchcombe, & White,
forecasting. Some state-of-the-art implementations of CL 1989).
include long short-term memory NNs (Smyl, 2020), deep By design, NNs are extremely flexible, and this flexibil-
NNs based on backward and forward residual links (Ore- ity means that a large number of parameters (or weights)
shkin, Carpov, Chapados, & Bengio, 2020b), feature-based need to be estimated, which again raises concerns about
XGBoost (Montero-Manso et al., 2020), and gradient overfitting, especially with very noisy data. The weights of
boosted decision trees (Bojer & Meldgaard, 2020). a NN are typically estimated via a stochastic gradient de-
scent (SGD) algorithm, such as Adam (Kingma & Ba, 2015).
2.7.11. Machine learning with (very) noisy data60 The SGD algorithm itself has some regularising properties,
With the advent of big data, machine learning now which can be strengthened by adjusting the algorithm’s
plays a leading role in forecasting.61 There are two pri- hyperparameters. We can further guard against overfit-
mary reasons for this. First, conventional ordinary least ting by shrinking the weights via LASSO or ENet penalty
squares (OLS) estimation is highly susceptible to overfit- terms, as well as imposing a dropout rate (Hinton, Srivas-
ting in the presence of a large number of regressors (or tava, Krizhevsky, Sutskever, & Salakhutdinov, 2012; Sri-
features); see also Sections 2.5.2 and 2.5.3. OLS maximises vastava, Hinton, Krizhevsky, Sutskever, & Salakhutdinov,
the fit of the model over the estimation (or training) sam- 2014).
ple, which can lead to poor out-of-sample performance; Perhaps the quintessential example of a noisy data en-
in essence, OLS over-responds to noise in the data, and vironment is forecasting asset returns, especially at short
the problem becomes magnified as the number of features horizons (e.g., monthly). Because many asset markets are
grows. A class of machine-learning techniques, which in- reasonably efficient, most of the fluctuations in returns
cludes the popular least absolute shrinkage and selection are inherently unpredictable – they reflect the arrival of
operator (LASSO: Tibshirani, 1996) and elastic net (ENet: new information, which, by definition, is unpredictable.
Zou & Hastie, 2005), employs penalised regression to im- This does not mean that we should not bother trying
prove out-of-sample performance with large numbers of to forecast returns, as even a seemingly small degree of
features. The LASSO and ENet guard against overfitting by return predictability can be economically significant (e.g.,
shrinking the parameter estimates toward zero. Campbell & Thompson, 2008). Instead, it means that we
Very noisy data – data with a very low signal-to-noise need to be particularly mindful of overfitting when fore-
ratio – exacerbate the overfitting problem. In such an casting returns in the era of big data. Section 3.3.13 dis-
environment, it is vital to induce adequate shrinkage to cusses applications of machine-learning techniques for
guard against overfitting and more reliably uncover the stock return forecasting.
predictive signal amidst all the noise. For LASSO and ENet
estimation, a promising strategy is to employ a stringent 2.7.12. Clustering-based forecasting62
information criterion, such as the Bayesian information The robustness of the forecasting process depends
criterion (BIC, Schwarz, 1978), to select (or tune) the reg- mainly on the characteristics of the target variable. In
ularisation parameter governing the degree of shrinkage cases of high nonlinear and volatile time series, a forecast-
(often denoted by λ). Fan and Tang (2013) and Wang, Li, ing model may not be able to fully capture and simulate
and Leng (2009) modify the BIC penalty to account for the special characteristics, a fact that may lead to poor
a diverging number of features, while Hui, Warton, and forecasting accuracy (Pradeepkumar & Ravi, 2017). Con-
Foster (2015) refine the BIC penalty to include the value of temporary research has proposed some approaches to
λ. These BIC variants induce a greater degree of shrinkage increase the forecasting performance (Sardinha-Lourenço,
by strengthening the BIC’s penalty term, making them Andrade-Campos, Antunes, & Oliveira, 2018). Clustering-
useful for implementing the LASSO and ENet in noisy based forecasting refers to the application of unsupervised
machine learning in forecasting tasks. The scope is to
increase the performance by employing the information
60 This subsection was written by David E. Rapach.
of data structure and of the existing similarities among
61 We do not make a sharp distinction between statistical learning
and machine learning. For brevity, we use the latter throughout this
subsection. 62 This subsection was written by Ioannis Panapakidis.
746
the data entries (Goia, May, & Fusai, 2010); see also 2.7.13. Hybrid methods63
Sections 2.7.4 and 2.7.10. Clustering is a proven method Hybrid approaches combine two or more of the above-
in pattern recognition and data science for deriving the mentioned advanced methods. In general, when methods
level of similarity of data points within a set. The outputs based on AI-based techniques, physical, and statistical
of a clustering algorithm are the centroids of the clusters approaches are combined together, the result is often im-
and the cluster labels, i.e., integer numbers that denote proved forecasting accuracy as a benefit from the inherent
the number of cluster that a specific data entry belongs integration of the single methods. The idea is to mix di-
to (Xu & Wunsch, 2005). verse methods with unique features to address the limita-
There are two approaches in clustering-based forecast- tions of individual techniques, thus enhancing the forecast
ing: (i) Combination of clustering and supervised machine performance (Mandal, Madhira, Haque, Meng, & Pineda,
learning, and (ii) solely application of clustering. In the
2012; Nespoli, Ogliari, Leva, Massi Pavan, Mellit, Lughi, &
first case, a clustering algorithm is used to split the train-
Dolara, 2019, see also Section 2.6). The performance of the
ing set into smaller sub-training sets. These sets contain
hybrid methods depends on the performance of the single
patterns with high similarity. Then for each cluster a
methods, and these single methods should be specifically
dedicated forecaster is applied (Chaouch, 2014; Fan, Chen,
& Lee, 2008). Thus, the number of forecasting algorithms selected for the problem that has to be addressed.
is equal to the number of clusters. This approach en- Hybrid methods can be categorised based on the con-
ables to train forecasters with more similar patterns and stituent methods, but also considering that these base
eventually achieve better training process. The forecasting methods may not necessarily act only on the forecasting
systems that involve clustering are reported to result in stage but also on data treatment and parameters iden-
lower errors (Fan, Mao, & Chen, 2006; Mori & Yuihara, tification stages. In data pre-processing combining ap-
2001). The combination of clustering and forecasting has proaches (see also Section 2.2), different methods can be
been presented in the literature earlier than the sole used for decomposing the time series into subseries (Son,
application of clustering. One of the first articles in the Yang, & Na, 2019) or the signal into different frequen-
literature on combining clustering and forecasting sets up cies (Zang, Cheng, Ding, Cheung, Liang, Wei, & Sun, 2018),
the respective theoretical framework (Kehagias & Petridis, and for classifying the historical data (Huang, Chen, Yang
1997). & Kuo, 2015). An advantage of such hybrid methods is ro-
In the second case, a clustering algorithm is used to bustness against sudden changes in the values of the main
both cluster the load data set and perform the fore- parameters. However, they require additional knowledge
casting (López, Valero, Senabre, Aparicio, & Gabaldon, and understanding of the base methods, and have the
2012). In the sole clustering applications, either the cen- disadvantage of slow response time to new data.
troids of the clusters can be utilised or the labels. Pattern The purpose of the parameter selection stage is to
sequence-based forecasting is an approach that employs optimise the parameters of the model, in terms of extract-
the cluster labels. In this approach, a clustering of all ing nonlinear features and invariant structures (Behera,
days prior to the test day is held and this results in
Majumder, & Nayak, 2018; Ogliari, Niccolai, Leva, & Zich,
sequences of labels of a certain length. Next, the similarity
2018) but also in terms of estimation of the parame-
of the predicted day sequence with the historical data
ter adopted for the prediction; for example, meteoro-
sequences is examined. The load curve of the predicted
logical factors such as temperature, humidity, precipi-
day is the average of the curves of the days following the
same sequences (Kang, Spiliotis, Petropoulos, Athiniotis, tation, snowfall, cloud, sunshine, wind speed, and wind
Li & Assimakopoulos, 2020; Martinez Alvarez, Troncoso, direction (Qu, Kang, Zhang, Jiang, & Ma, 2016). Hybrid
Riquelme, & Aguilar Ruiz, 2011). methods feature straightforward determination of the pa-
There is variety of clustering algorithms that have been rameters with relatively basic structures. However, the
proposed in forecasting such as the k-means, fuzzy C- implementation is sometimes challenging, and depends
means (FCM), self-organising map (SOM), deterministic on the knowledge and expertise of the designer.
annealing (DA), ant colony clustering ACC, and others. Finally, the data post-processing hybrid approaches
Apart from the clustering effectiveness, a selection crite- forecast the residual errors resulted from the forecasting
rion for an algorithm is the complexity. k-means and FCM model. Since these hybrid methods consider residual er-
are less complex compared to the SOM that needs a large rors from the model, they aim in further improving the
number of variables to be calibrated prior to its applica- predictions of the base methods by applying corrections
tion. Meta-heuristics algorithms, such as ACC, strongly de- in the forecasts. However, a disadvantage of these hy-
pend on the initialisation conditions and the swarm size. brid methods is the increased calculation time, as the
Therefore, a comparison of clustering algorithms should residual errors must also be estimated. Also, such hybrid
take place to define the most suitable one for the problem methods are not general and will depend on the field
under study (Elangasinghe, Singhal, Dirks, Salmond, & of application. In many cases, hybrids approaches out-
Samarasinghe, 2014; Li, Han, & Li, 2008; Mori & Yuihara, perform other (single) approaches such as kNN, NN, and
2001; Wang, Pedrycz, & Liu, 2015). ARIMA-based models (Mellit, Massi Pavan, Ogliari, Leva,
The assessment of clustering-based forecasting is held & Lughi, 2020). A great example is the hybrid method
via common evaluation metrics for forecasting tasks (see
by Smyl (2020), which achieved the best performance in
also Section 2.12). The optimal number of clusters, which
the M4 forecasting competition (see also Section 2.12.7).
is a crucial parameter of a clustering application, is se-
In particular, in energy applications (see Section 3.4), a
lected via trial-and-error, i.e., the optimal number corre-
sponds to the lowest forecasting error (Nagi, Yap, Nagi,
Tiong, & Ahmed, 2011). 63 This subsection was written by Sonia Leva.
747
combination of physical and AI-based techniques can lead of bias (inversion bias) and the same researchers (Syn-
to improved forecasting performance. Furthermore, ma- tetos & Boylan, 2005) proposed a modification to his
chine learning methods (see Section 2.7.10) based on method that leads to approximately unbiased estimates.
historical data of meteorological variables combined with This method is known in the literature as the Syntetos-
an optimal learning algorithm and weather classification Boylan Approximation (SBA). It has been found repeatedly
can further improve the forecasting accuracy of single to account for considerable empirical inventory forecast-
methods. However, in general, the weak point of such ing improvements (Eaves & Kingsman, 2004; Gutierrez,
hybrid approaches is that they underperform when mete- Solis, & Mukhopadhyay, 2008; Nikolopoulos, Babai, & Bo-
orological conditions are unstable (Chicco, Cocina, Di Leo, zos, 2016; van Wingerden, Basten, Dekker, & Rustenburg,
Spertino, & Massi Pavan, 2015). 2014) and, at the time of writing, it constitutes the bench-
mark against which other (new) proposed methodolo-
2.8. Methods for intermittent demand gies in the area of intermittent demand forecasting are
assessed.
2.8.1. Parametric methods for intermittent demand forecast- Croston’s method is based upon the assumption of a
ing64 Bernoulli demand arrival process. Alternatively, demand
Demand forecasting is the basis for most planning may be assumed to arrive according to a Poisson pro-
and control activities in any organisation. Demand will cess. It is also possible to adapt Croston’s method so
typically be accumulated in some pre-defined ‘time buck- that sizes and intervals are updated based on a simple
ets’ (periods), such as a day, a week or a month. On moving average (SMA) procedure instead of SES. Boylan
many occasions, demand may be observed in every time and Syntetos (2003), Shale, Boylan, and Johnston (2006),
period, resulting in what is sometimes referred to as ‘non- and Syntetos, Babai, and Luo (2015) presented correction
factors to overcome the bias associated with Croston’s
intermittent demand’. Alternatively, demand may appear
approach under a Poisson demand arrival process and/or
sporadically, with no demand at all in some periods, lead-
estimation of demand sizes and intervals using an SMA.
ing to an intermittent appearance of demand occurrences.
For a detailed review of developments in intermit-
Intermittent demand items monopolise the stock bases in
tent demand forecasting interested readers are referred
the after sales industry and are prevalent in many other
to Boylan and Syntetos (2021).
industries, including the automotive, IT, and electronics
sectors. Their inventory implications are dramatic and
2.8.2. Non-parametric intermittent demand methods65
forecasting their requirements is a very challenging task.
Two main non-parametric forecasting approaches have
Methods to forecast intermittent demand may broadly
dominated the intermittent demand literature: the boot-
be classified as parametric and non-parametric. The for-
strapping approach and the Overlapping/Non-Overlapping
mer suppose that future demand can be well represented
aggregation Blocks approach (Boylan & Syntetos, 2021).
by a statistical distribution (say Poisson or Negative Bino-
The bootstrapping approach relies upon a resampling
mial) which has parameters that are unknown but may be
(with or without replacement) of the historical demand
forecasted using past data. These methods are discussed
data to build the empirical distribution of the demand
in this sub-section. In the latter, the data are not assumed
over a specified interval. As discussed in Section 2.7.5, this
to follow any standard probability distribution. Instead, approach was initially introduced by Efron (1979). Since
direct methods are used to assess the distributions re- then, it has been developed by Willemain, Smart, and
quired for inventory management (see also Section 3.2.3). Schwarz (2004) and Zhou and Viswanathan (2011) to deal
Such methods are discussed in Section 2.8.2. with intermittent demand items (Babai, Tsadiras, & Pa-
Simple Exponential Smoothing (SES; see Section 2.3.1) padopoulos, 2020). Willemain et al. (2004) have proposed
is often used in practice to forecast intermittent demand a method that resamples demand data by using a Markov
series. However, SES fails to recognise that intermittent chain to switch between no demand and demand periods.
demand is built from two constituent elements: (i) the The empirical outperformance of this method has been
inter-demand intervals, which relate to the probability shown when compared to Simple Exponential Smoothing
of demand occurring, and (ii) the demand sizes, when (SES) and Croston’s method (see also Section 2.8.1). How-
demand occurs. The former indicates the degree of in- ever, the findings of Willemain et al. (2004)’s work have
termittence, whereas the latter relates to the behaviour been challenged by Gardner and Koehler (2005) and some
of the positive demands. Croston (1972) showed that limitations have been addressed by Syntetos, Zied Babai
this inherent limitation leads to SES being (positively) and Gardner (2015). Zhou and Viswanathan (2011) have
biased after a demand occurring period; this is some- developed an alternative bootstrapping method. Their
times referred to as an ‘issue point’ bias. Subsequently, method samples separately demand intervals and de-
he proposed a method that forecasts separately the sizes mand sizes and it has been shown to be associated with
of demand, when demand occurs, and the inter-demand a good performance for long lead-times. Teunter and
intervals. Both forecasts are produced using SES, and the Duncan (2009) and Hasni, Aguir, Babai, and Jemai (2019a)
ratio of the former over the latter gives a forecast of the have developed adjustments of the bootstrapping meth-
mean demand per period. Croston’s method was shown ods, where the lead-time demand forecast is adjusted by
by Syntetos and Boylan (2001) to suffer from another type assuming that the first period in the lead-time bucket
64 This subsection was written by Aris A. Syntetos. 65 This subsection was written by Mohamed Zied Babai.
748
corresponds to a non-zero demand. They have demon- robust to inexact specification of cut-off values. Moon,
strated the outperformance of the adjusted bootstrapping Simpson, and Hicks (2013) used logistic regression to
methods in a periodic order-up-to-level inventory control classify time series of demand for spare parts in the South
system. A review of the bootstrapping methods in the Korean Navy. The classification was designed to identify
context of intermittent demand is provided by Hasni, superior performance (accuracy and inventory costs) of
Aguir, Babai, and Jemai (2019b). direct and hierarchical forecasting methods, based on the
Porras and Dekker (2008) were the first to consider serial correlation of demands, the coefficient of variation
aggregation with overlapping and non-overlapping blocks of demand volume of spare parts (see also Section 3.2.7),
(OB and NOB) approach in forecasting the demand of and the functionality of the naval equipment.
spare parts. In the NOB approach, a demand series is Bartezzaghi, Verganti, and Zotteri (1999) identified five
divided into consecutive non-overlapping blocks of time, factors that contribute towards intermittence and ‘lumpi-
whereas in OB, at each period the oldest observation ness’ (intermittence with highly variable demand sizes):
is dropped and the newest is included (Rostami-Tabar, number of potential customers, frequency of customers’
Babai, Syntetos, & Ducq, 2013). Boylan and Babai (2016) requests, heterogeneity of customers, variety of individual
have compared the statistical and inventory performance customer’s requests, and correlations between customers’
of the OB and NOB methods. They found that, unless the requests. These may contribute towards useful classifi-
demand history is short, there is a clear advantage of cations, for example by the number of customers for an
using OB instead of NOB. More recently, based on extreme item. When this number is low and some of the cus-
value theory (EVT), Zhu, Dekker, van Jaarsveld, Renjie, tomers are large, then direct communication with the
and Koning (2017) have proposed an improvement of customers can inform judgmental forecasts. Similarly, if
the OB method that models better the tail of lead-time a customer’s requests are highly variable, then ‘advance
demand. They have shown that the empirical-EVT method demand information’ from customers can help to improve
leads to higher achieved target cycle service levels when judgmental estimates. These strategies can be very use-
compared to the original method proposed by Porras and ful in a business-to-business environment, where such
Dekker (2008). Temporal aggregation is further discussed strategies are feasible.
in Section 2.10.2. An alternative approach to classification is combina-
tion of forecasts (see Section 2.6 for a review, and Sec-
2.8.3. Classification methods66 tion 2.6.1 in particular). Petropoulos and Kourentzes (2015)
In many application areas, forecasts are required across investigated combining standard forecasting methods for
a wide collection of products, services or locations. In this
intermittent demand (e.g., SES, Croston, Syntetos-Boylan
situation, it is convenient to introduce classification rules
Approximation; see also Section 2.8.1). They did not find
that allow subsets of time series to be forecasted using the
this to improve accuracy directly, but obtained good re-
same approaches and methods. Categorisation rules, such
sults from the use of combinations of forecasts at different
as the ABC inventory classification, serve the forecasting
temporal frequencies, using methods selected from the
function only coincidentally. They do not necessarily align
extended SBC classification scheme.
to the selection of the best forecasting method.
Within certain modelling frameworks, classification of
2.8.4. Peak over the theshold67
time series is well established. For example, within an
ARIMA framework (Box, Jenkins, & Reinsel, 2008, and In the forecasting literature, Nikolopoulos (2020) ar-
Section 2.3.4), or within a state-space framework for ex- gues that great attention has been given to modelling
ponential smoothing (Hyndman et al., 2002, and Sec- fast-moving time series with or without cues of infor-
tion 2.3.1), series may be classified, for example based mation available (Nikolopoulos, Goodwin, Patelis, & As-
on the AIC (Akaike, 1973). It is more challenging to clas- simakopoulos, 2007). Less attention has been given to
sify series according to their recommended forecasting intermittent/count series (see Sections 2.3.8, 2.8.1 and
method if some of the candidate methods, such as Cros- 2.8.2), which are more difficult to forecast given the pres-
ton’s method (see Section 2.8.1), lack a fully satisfactory ence of two sources of uncertainty: demand volume, and
model base. In the field of intermittent demand fore- timing.
casting, Syntetos, Boylan, and Croston (2005) proposed Historically there have been few forecasting methods
the SBC classification scheme, enabling time series to be developed specifically for such data (Syntetos, Zied Babai
classified according to their length of average demand in- & Gardner, 2015). We believe that through a time series
terval and coefficient of variation of demand sizes (when decomposition approach á la Leadbetter (1991) we can
demand occurs). These rules were based on assumptions isolate ‘peaks over threshold’ (POT) data points, and create
of independent and identically distributed (iid) demand, new intermittent series from any time series of interest.
and a comparison of expected mean square error between The derived series present almost identical characteristics
methods. The scheme has been extended by Kostenko with the series that Croston (1972) analysed. In essence
and Hyndman (2006) and by Petropoulos and Kourentzes one could use such decomposition forecasting techniques
(2015). In an empirical case-study, Boylan, Syntetos, and to tackle much more difficult phenomena and problems
Karakostas (2008) examined series not necessarily con- coming from finance, politics, healthcare, humanitarian
forming to iid assumptions and found the rules to be logistics, business, economics, and social sciences.
66 This subsection was written by John E. Boylan. 67 This subsection was written by Konstantinos Nikolopoulos.
749
Any time series can be decomposed into two sub- Fuzzy logic tries to overcome this issue by admitting that
series: one containing the baseline (white swans) and one a statement/variable could be partially true or partially
containing the extreme values over an arbitrary-set or false. Mathematically, the fuzzy logic framework is based
rule-based-set threshold (grey and black swans) as pro- on the work of Zadeh (1965) who introduced the theory
posed by Taleb (2008); see also Section 2.3.22. Unfor- of fuzzy sets. The main point of this theory is the definition
tunately, major decision-related risks and most of the of two kinds of sets:
underlying uncertainty lay with these extremes. So, it
1. Crisp sets are the ‘‘classical’’ sets in the Boolean
is very important to be able to effectively model and
logic. An element can belong (or not) to a certain
forecast them.
It is unlikely that any forecasting approach will accu- set.
rately give the exact timing of the forthcoming extreme 2. Fuzzy sets, where an element can belong to the sets
event, but it will instead provide a satisfactory cumulative with a certain membership grade, with a value that
forecast over a long period of time. The question still varies in the interval [0, 1].
stands what can one do with this forecast? For earthquake The definition of the fuzzy sets allows the framework
data, although even if we know that a major earthquake to take into account the uncertainty and vagueness of
is going to hit a region, it is almost impossible to decide information. An extension of this approach is related to
to evacuate cities, but still we can influence and legislate the fact that a certain variable can assume a crisp value
the way structures are built and increase the awareness, (classical theory) or can belong to different fuzzy sets with
training, preparedness and readiness of the public; and different membership grade. For example, in a system
also ensure enough capital on hold to cope with the implemented to forecast the daily pollutant concentra-
aftermath of the major event. For epidemics/pandemics tion in atmosphere, one of the inputs could relate to
(see Section 3.6.2) there are clear implications, as we have the weather conditions, such as the wind speed. In the
evidenced with COVID-19, on how proactively we can classical approach, the system must have as an input the
source and secure human resources, medical supplies, etc. value of the wind speed at a certain day. In the fuzzy
What is the current doctrine when forecasting in such approach, the input of the system could be the member-
a context: advanced probabilistic models. These meth- ship grade of the input variable to three different fuzzy
ods typically require a lot of data and reconstruct the sets: (i) ‘‘Not windy’’, (ii) ‘‘average windy’’, and (iii) ‘‘strong
distributions of the underlying phenomena. These come windy’’. On the other hand, the user of the forecasting
with common successes and a plethora of constraints: system may be only interested in a classification of the
big data sets needed for training the models, high math- output variable instead of the crisp value. In this case, the
ematical complexity, and invisibility to practitioners how fuzzy approach is applied to the pollutant concentration
these methods do actually work and thus less acceptance which could belong with a certain degree to the fuzzy
in practice. Yet again, forecasting accuracy is the name of sets (i) ‘‘not polluted day’’, (ii) ‘‘medium polluted day’’,
the game and thus these forecasting methods are serious (iii) ‘‘high polluted day’’, and (iv) ‘‘critical polluted day’’.
contenders for the task in hand. In fuzzy theory, each fuzzy set is characterised by
Extreme Value Theory (EVT) analyses extreme devi- a (generally nonlinear) function, called the membership
ations from statistical measures of central location to function, linking crisp values to the membership of the
estimate the probability of events that are more extreme different sets. The association of a crisp value to its mem-
than anything observed in the time series. This is usually bership for a set is called fuzzyfication, while the inverse
done in the following two ways (Nikolopoulos, 2020): (i) operation (from a membership value to a crisp value) is
deriving maxima and/or minima series as a first step and called defuzzification. As with the logic theory, the infer-
then having the Generalised Extreme Value Distribution ence system assumes a key role in the fuzzy theory. A
fitted (often the number of extreme events is limited), Fuzzy Inference System (FIS) allows the interpretation of
and (ii) isolating the values that exceed a threshold (point the membership grades of the input variable(s) and, given
over threshold) that can also lead to only a few instances some sets of fuzzy rules, assigns the corresponding values
extracted – so a very intermittent series in nature. The to the output variable(s). In the literature, two main fuzzy
analysis involves fitting a Poisson distribution for the inference systems are presented:
number of events in a basic time period and a second
distribution – usually a Generalised Pareto Distribution – 1. Mamdani system (Mamdani & Assilian, 1975), where
for the size of the resulting POT values. both the input and output of the inference system
are membership functions.
2.9. Reasoning and mining 2. Sugeno system (Sugeno, 1985), where the output
of the inference system is a crisp value, usually
2.9.1. Fuzzy logic68 obtained by applying a linear function to the de-
The ‘‘classical’’ Boolean logic is not able to handle for fuzzified value of the input.
uncertainties and/or vagueness that are necessary when
dealing with many real world problems. This is in part 2.9.2. Association rule mining69
due to the fact that the Boolean logic is based on only Association rule mining is an exploratory data-driven
two values (i.e., a statement can only be true or false). approach which is able to automatically and exhaustively
68 This subsection was written by Claudio Carnevale. 69 This subsection was written by Daniele Apiletti.
750
extract all existing correlations in a data set of categorical support higher than MinSup, pairs A and B = F − A
features. It is a powerful but computationally intensive are derived so that the confidence of A → B is higher
technique, successfully applied in different forecasting than MinConf . The first step of the process is the most
contexts (Acquaviva et al., 2015; Apiletti & Pastor, 2020; computationally expensive. Thus, several algorithms have
Di Corso, Cerquitelli, & Apiletti, 2018). Its results are in a been proposed to solve the problem of frequent itemset
human-readable form. extraction (Zaki, 2000), some specifically addressing high-
The data set must be in the form of transactions, i.e., a dimensionality issues (Apiletti, Baralis, Cerquitelli, Garza,
collection of events, each described by categorical fea- Michiardi, & Pulvirenti, 2015; Apiletti, Baralis, Cerquitelli,
tures. If the phenomena under analysis are modelled by Garza, Pulvirenti, & Michiardi, 2017). Despite being com-
continuous-valued variables, discretisation can be applied putationally demanding, association rule mining is an
to obtain a suitable data set. exhaustive approach, i.e., all and only statistically rele-
Association rule mining core task is the frequent item- vant correlations are extracted. Section 3.8.11 offers an
set extraction, which consists in finding frequently- example of applying association rule mining to forecast
occurring relationships among items in a data set (Han, the quality of beverages.
Pei, & Kamber, 2011). Given a data set of records char-
acterised by several attributes, an item refers to a pair of 2.9.3. Forecasting with text information70
(attribute = value), while a set of items is called itemset. Text data, such as social media posts, scholar arti-
The support count of an itemset is the number of records cles and company reports, often contains valuable in-
r containing that itemset. The support of an itemset is formation that can be used as predictors in forecasting
the percentage of records containing it with respect to models (Aggarwal & Zhai, 2012). Before extracting useful
the total number of records in the data set. An itemset features from the data, a text document needs to be
is frequent when its support is greater than or equal to a cleaned and normalised for further processing. The first
minimum support threshold. step of preparing the data is to filter out stop words –
An association rule is an implication in the form A → the words that do not add much meaning to a sentence,
B, where A and B are disjoint itemsets (i.e., A ∩ B = e.g., ‘‘a’’, ‘‘is’’, and ‘‘me’’. For grammatical reasons, it is
∅) (Tan, Steinbach, & Kumar, 2005). A is called rule body necessary for documents to use different forms of words.
or antecedent and B rule head or consequent. Stemming and lemmatisation can be applied to reduce
To evaluate the quality of an association rule, the inflectional forms or relate different forms of a word to
support, confidence, and lift metrics are commonly ex- a common base form. Stemming often chops off the end
ploited (Han et al., 2011). Rule support is the fraction of of a word, while lemmatisation uses vocabularies and
records containing both A and B, indicating the probability morphological analysis to return the base form (or the
that a record contains every item in these itemsets. The lemma) of a word (Lovins, 1968; Manning, Schütze, &
support of the rule is computed as the support of the Raghavan, 2008). For example, the word ‘‘industries" will
union of A and B. be turned into ‘‘industri’’ or ‘‘industry’’ if stemming or
Rule confidence represents the strength of the implica- lemmatisation is applied.
tion, and is the conditional probability that a transaction To model and analyse text data, we need to trans-
containing A also contains B, P(B|A), i.e., the proportion form it into numerical representations so the forecast-
of records that contain A with respect to those that also ing models can process them as predictors. One way
contain B. of transforming the data is through sentiment analy-
Finally, the lift of a rule measures the correlation be- sis. Sentiment analysis is often applied to detect po-
tween antecedent and consequent. It is defined as the larity within customer materials such as reviews and
ratio between the rule A → B confidence and the sup- social media posts (Archak, Ghose, & Ipeirotis, 2011; Das
port of B. A lift ratio equal to 1.0 implies that itemsets & Chen, 2007). An easy way to obtain the sentiment
A and B are not correlated. A lift higher than 1.0 indi- score of a word is to look it up in a happiness dictio-
cates a positive correlation, meaning that the occurrence nary (for example, the hedonometer dictionary, Hedo-
of A likely leads to the occurrence of B with the given nometer, 2020). Another common way of representing the
confidence. The greater the lift, the stronger the associ- sentiment is to use a vector of numeric values that de-
ation. Finally, a lift lower than 1.0 indicates a negative note the word’s positivity, negativity and neutrality based
correlation between A and B. on existing lexical databases such as the WordNet (Bac-
The problem of association rule mining consists in the cianella, Esuli, & Sebastiani, 2010; Godbole, Srinivasaiah,
extraction of all the association rules having rule support & Skiena, 2007). Once the sentiment of each word is calcu-
and confidence greater than the respective support and lated, we can apply an aggregation algorithm (e.g., simple
confidence thresholds, MinConf and MinSup, defined as average) to measure the sentiment of an entire sentence
parameters of the mining process (Tan et al., 2005). These or paragraph.
thresholds allow to control the statistical relevance of the In scholar articles and company reports, context fea-
extracted rules. tures might be more important than sentiments. The bag-
The process of rule mining consists of two steps. The of-words model and word embeddings are often applied
first step is the computation of frequent itemsets, i.e., item- to generate numeric representations of such text. A bag-
sets with support greater or equal to MinSup. The second of-words model simply returns a matrix that describes the
step is the extraction of association rules from frequent
itemsets. Let be F a frequent itemset, hence having a 70 This subsection was written by Xiaojia Guo.
751
occurrence of words within a document (Goldberg, 2017). The most common approaches to hierarchical fore-
When we use this matrix as input to a forecasting model, casting are bottom-up and top-down. In the bottom-up
each word count can be considered as a feature. The approach forecasts for each time series at the bottom-
Word2Vec method is a widely used embedding method level are first produced and then these are added up to
that is built based on the context of a word. Specifi- obtain forecasts for all other series at the hierarchy (Dunn,
cally, it trains a two-layer neural network that takes each Williams, & Dechaine, 1976). Since forecasts are obtained
word as an input to predict its surrounding words (see at the bottom-level no information is lost due to ag-
Section 2.7.8 for a discussion of neural networks for fore- gregation. In the top-down approach forecasts for the
casting). The weights from the input layer to the hidden top-level series are first generated and then these are
layer are then utilised as the numerical representation for disaggregated generally using historical proportions to
the input word (Le & Mikolov, 2014). Once the text is obtain forecasts for the bottom-level series, which are
turned into numeric representations, they can be used as then aggregated (Gross & Sohl, 1990). Hyndman, Ahmed,
predictors in any forecasting models. The most challeng- Athanasopoulos, and Shang (2011) claim that this ap-
ing part in this process is to find the right technique to proach introduces bias to the forecasts, however Holly-
extract features from the text. man et al. (2021) showed that it is possible to calculate
In terms of software implementation, the Natural Lan- unbiased top-down forecasts.
guage Toolkit (NLTK) and SpaCy library in Python can be Recent research on hierarchical forecasting tackles the
applied to remove stop words and stem or lemmatise problem using a two-stage approach. Forecasts for the
text (Honnibal, 2015; Loper & Bird, 2002). The bag-of- series at all levels of the hierarchy are first obtained
words technique is also available in NLTK. A particu- independently without considering any aggregation con-
lar implementation of the Word2Vec model is available strains (we refer to these as base forecasts). Then, base
on Google code (2013). Moreover, a public data set of forecasts are adjusted so that they become coherent (we
movie reviews that is commonly studied in literature is refer to these as reconciled forecasts). This adjustment
available from the Stanford NLP Group (2013). is achieved by a matrix that maps the base forecasts
into new bottom-level forecasts which are then added
2.10. Forecasting by aggregation up (Hyndman et al., 2011).
Wickramasuriya, Athanasopoulos, and Hyndman (2019)
2.10.1. Cross-sectional hierarchical forecasting71 found the optimal solution for this matrix, which min-
In many applications time series can be aggregated imises the trace of the covariance matrix of the reconciled
at several levels of aggregation based on geographic or forecast errors (hence MinT reconciliation). This optimal
logical reasons to form hierarchical structures. These are solution is based on the covariance matrix of the base
called hierarchical time series. In the retail industry (see forecast errors which incorporates the correlation struc-
also Section 3.2.4), for example, individual sales of prod- ture of the hierarchy. Wickramasuriya et al. (2019) pre-
ucts at the bottom-level of the hierarchy can be grouped sented several alternative estimators for this covariance
in categories and families of related products at increasing matrix: (i) proportional to the identity which is opti-
aggregation levels, with the total sales of the shop or mal only when base forecast errors are uncorrelated and
distribution centre at the top level (Oliveira & Ramos, equivariant (referred to as OLS), (ii) proportional to the
2019; Pennings & van Dalen, 2017; Villegas & Pedregal, sample covariance estimator of the in-sample one-step-
2018). Similarly, cross-sectional hierarchies can be used ahead base forecast errors with off-diagonal elements
for spatial aggregation to help model housing prices or null accounts for the differences in scale between the
traffic in transportation networks, or otherwise formed levels of the hierarchy (referred to as WLS), (iii) pro-
geographical demarcations (for example, Athanasopou- portional to the previous estimator unrestricted also ac-
los, Ahmed, & Hyndman, 2009; Kourentzes & Athana- counts for the relationships between the series (referred
sopoulos, 2019). The forecasts of hierarchical time se- to as MinT-Sample), and (iv) proportional to a shrink-
ries produced independently of the hierarchical structure age estimator based on the two previous estimators, pa-
generally will not add up according to the aggregation rameterising the shrinkage in terms of variances and
constrains of the hierarchy, i.e., they are not coherent. correlations, accounts for the correlation across levels (re-
Therefore, hierarchical forecasting methods that generate ferred as MinT-Shrink). Other researchers focus on simple
coherent forecasts should be considered to allow appro- (equal-weighted) combinations of the forecasts produced
priate decision-making at the different levels. Actually, by at different hierarchical levels (Abouarghoub, Nomikos,
taking advantage of the relationships between the series & Petropoulos, 2018; Hollyman et al., 2021). Pritularga,
across all levels these methods have shown to improve Svetunkov, and Kourentzes (2021) showed that more
forecast accuracy (Athanasopoulos et al., 2009; Shang &
complex reconciliation schemes result in more variability
Hyndman, 2017; Yagli, Yang, & Srinivasan, 2019). One of
in the forecasts, due to the estimation of the elements
the main reasons behind this improved performance is
in the covariance matrix, or the implicit combination
that forecast reconciliation is effectively a special case of
weights. They provide approximations for the covariance
forecast combinations (Hollyman, Petropoulos, & Tipping,
matrix that balance this estimation uncertainty with the
2021); see also Section 2.6.
benefits of more finely tuned weights.
More recently these techniques were extended to prob-
71 This subsection was written by Patrícia Ramos. abilistic forecasting (Ben Taieb, Taylor, & Hyndman, 2020).
752
When base forecasts are probabilistic forecasts charac- so it is intuitive to consider multiple temporal levels
terised by elliptical distributions, Panagiotelis, Gamaku- and benefit from the subsequent forecast combination
mara, Athanasopoulos, and Hyndman (2021) showed that across frequencies. Kourentzes and Petropoulos (2016)
reconciled probabilistic forecasts also elliptical can be showed how multiple temporal aggregation can be ex-
obtained analytically. When it is not reasonable to assume tended to incorporate exogenous variables (see also Sec-
elliptical distributions, a non-parametric approach based tion 2.2.5). However, forecasting at a single level of ag-
on bootstrapping in-sample errors can be used. gregation can still result in better performance when
the seasonal pattern is strong (Spiliotis, Petropoulos &
2.10.2. Temporal aggregation72 Assimakopoulos, 2019; Spiliotis, Petropoulos, Kourentzes
Temporal aggregation is the transformation of a time & Assimakopoulos, 2020).
series from one frequency to another of lower frequency. Athanasopoulos et al. (2017) expressed multiple tem-
As an example, a time series of length n that is originally poral aggregation within the hierarchical forecasting
sampled at a monthly frequency can be transformed to framework (see Section 2.10.1) using the term ‘‘temporal
a quarterly series of length n/3 by using equally-sized hierarchies’’. Temporal hierarchies allow for the applica-
time buckets of three periods each. It is usually applied tion of established hierarchical reconciliation approaches
in an non-overlapping manner, but overlapping aggre- directly to the temporal dimension. Jeon, Panagiotelis,
gation can also be considered. The latter is preferred and Petropoulos (2019) show how temporal hierarchies
in the case when the original series is short, but has can be used to obtain reconciled probabilistic forecasts,
the disadvantage of applying lower weights on the few while (Spiliotis, Petropoulos et al., 2019) explored em-
first and last observations of the series and introducing pirical bias-adjustment strategies and a strategy to avoid
autocorrelations (Boylan & Babai, 2016, and Section 2.8.2). excessive seasonal shrinkage. Nystrup, Lindström, Pin-
Temporal aggregation is appealing as it allows to in- son, and Madsen (2020) proposed estimators for tempo-
vestigate the original series through different lenses. By ral hierarchies suitable to account for autocorrelation in
changing the original frequency of the data, the apparent the data. Finally, Kourentzes and Athanasopoulos (2020)
series characteristics also change. In the case of slow- applied temporal hierarchies on intermittent data, and
moving series, temporal aggregation leads to decrease of showed that higher aggregation levels may offer struc-
intermittence (Nikolopoulos, Syntetos, Boylan, Petropou- tural information which can improve the quality of the
los, & Assimakopoulos, 2011, see also Section 2.8). In forecasts.
the case of fast-moving series, higher levels of aggrega-
tion (i.e., lower frequencies) allow for better modelling of 2.10.3. Cross-temporal hierarchies73
trend patterns, while lower aggregation levels (i.e., higher In the last two subsections (Sections 2.10.1 and 2.10.2),
frequencies) are more suitable for capturing seasonal pat- we saw two complimentary hierarchical structures, cross-
terns (Kourentzes et al., 2014; Spithourakis, Petropoulos, sectional and temporal. Although the machinery behind
Nikolopoulos, & Assimakopoulos, 2014). both approaches is similar, often relying on the hierarchi-
Research has found evidence of improved forecasting cal framework by Athanasopoulos et al. (2009) and Hynd-
performance with temporal aggregation for both slow man et al. (2011), and work that followed from these (par-
(Nikolopoulos et al., 2011) and fast (Spithourakis, Petropou- ticularly Wickramasuriya et al., 2019), they address dif-
los, Babai, Nikolopoulos, & Assimakopoulos, 2011) moving ferent forecasting problems. Cross-sectional hierarchies
time series. This led to characterise temporal aggregation change the unit of analysis but are fixed in the period of
as a ‘‘self-improving mechanism’’. The good performance analysis. For example, a manufacturer may operate using
of temporal aggregation was reconfirmed by Babai, Ali, a hierarchy across products. The different nodes in the
and Nikolopoulos (2012), who focused on its utility per- hierarchy will correspond to different products, product
formance rather than the forecast error. However, one groups, super-groups, and so on, but will all refer to
challenge with single levels of aggregation is the choice the same period, for example, a specific week. Temporal
of a suitable aggregation level for each series (Kourentzes, hierarchies do the opposite, where the unit of analysis is
Rostami-Tabar, & Barrow, 2017). fixed, but the period is not. For example, we may look at
Instead of focusing on a single aggregation level, An- the sales of a specific Stock Keeping Unit (SKU) at a daily,
drawis et al. (2011), Kourentzes et al. (2014), Petropoulos weekly, monthly, quarterly and annual levels. However,
and Kourentzes (2014), and Petropoulos and Kourentzes one can argue that having annual forecasts at the SKU
(2015) suggested the use of multiple levels of aggregation, level may not be useful. Similarly, having aggregate sales
usually abbreviated as MTA (multiple temporal aggrega- across an organisation at a weekly frequency is also of
tion). This not only tackles the need to select a single little value.
aggregation level, but also partly addresses the issue of In connecting these to organisational decisions, we can
model uncertainty, instead of relying on model selec- observe that there is only a minority of problems that ei-
tion and parametrisation at a single aggregation level. ther cross-sectional or temporal hierarchies are natural, as
Using this property, Kourentzes et al. (2017) showed typically decisions can differ across both the unit and the
that MTA will typically lead to more accurate forecasts, period (planning horizon) of analysis. In the latter case,
even if in theory suboptimal. Different frequencies al- both hierarchical approaches are more akin to statistical
low for better identification of different series patterns, devices that can improve forecast accuracy through the
72 This subsection was written by Fotios Petropoulos. 73 This subsection was written by Nikolaos Kourentzes.
753
use of forecast combinations, rather than satisfy the mo- of inconsistencies in correlations and association mea-
tivating argument behind hierarchical forecasting that is sures across different levels of aggregation. This is closely
to provide coherent predictions for decisions at different related to the well-known Simpson’s Paradox (Simpson,
levels of the hierarchy. 1951). In this setting, the ecological fallacy is manifested
Cross-temporal hierarchies attempt to overcome this through aggregation bias (Wakefield, 2004) due to con-
limitation, providing coherent forecasts across all units textual effects and/or spatial autocorrelation (Achen &
and periods of analysis, and therefore a common out- Phillips Shively, 1995). This has led many authors to
look for the future across decision-makers at different disqualify ecological inference forecasts (see, for example,
functions and levels within an organisation. The literature Anselin & Tam Cho, 2002; Freedman, Klein, Ostland, &
remains sparse on how to construct cross-temporal fore- Roberts, 1998; Herron & Shotts, 2004; Tam Cho, 1998)
casts, as the size of the hierarchy can easily become prob- and many others to study under which circumstances
lematic. Kourentzes and Athanasopoulos (2019) propose a ecological inference predictions would be reliable (Fire-
heuristic approach to overcome the ensuing estimation is- baugh, 1978; Forcina & Pellegrino, 2019; Gelman, Park,
sues. The approach works by compartmentalising the esti- Ansolabehere, Price, & Minnite, 2001; Guseo, 2010). De-
spite the criticisms, many algorithms for solving the EIF
mation. First, they obtain estimates of the cross-sectional
problem can be found in the literature, mainly from
reconciliation weights for each temporal level of the hier-
the ecological regression and mathematical programming
archy. Then, these are combined across temporal levels, to
frameworks (some of them available in functions of the R
a unique set that satisfies all coherency constraints. Using
statistical software).
these combined weights, they obtain the reconciled bot-
The ecological regression literature has been prolific
tom level forecasts, which can be aggregated as needed.
since the seminal papers of Duncan and Davis (1953)
Although they recognise that their approach can result
and Goodman (1953, 1959) and is undergoing a renais-
in suboptimal results in terms of reconciliation errors, it sance after King (1997): new methods generalised from
guarantees coherent forecasts. Cross-temporal forecasts 2 × 2 tables to R × C tables have been proposed (King,
are more accurate than either temporal or cross-sectional Rosen, & Tanner, 1999; Rosen, Jiang, King, & Tanner,
hierarchical forecasts and provide a holistic view of the 2001), the geographical dimension of the data is being
future across all planning levels and demarcations. Spili- explicitly considered (Calvo & Escolar, 2003; Puig & Gine-
otis, Petropoulos et al. (2020) also identify the problem, bra, 2015), and new procedures combining aggregated
however, they do not focus on the coherency of forecasts and individual level data, including exit polls (see also
and propose a sequential reconciliation across the two Section 3.8.5), are introduced (Glynn & Wakefield, 2010;
dimensions. This is shown to again be beneficial, but it Greiner & Quinn, 2010; Klima, Schlesinger, Thurner, &
does not achieve coherency. Arguably one can adapt the Küchenhoff, 2019). See King, Tanner, and Rosen (2004) for
iterative correction algorithm by Kourentzes and Athana- a wide survey and Klima, Thurner, Molnar, Schlesinger,
sopoulos (2020) to enforce coherency in this approach as and Küchenhoff (2016) and Plescia and De Sio (2018)
well. for an extensive evaluation of procedures. In mathemat-
ical programming exact and inequality constraints for
2.10.4. Ecological inference forecasting74 the inner-cell values are incorporated in a natural way.
Ecological inference forecasting (EIF) aims to predict Hence, this approach has shown itself to be a proper
the inner-cells values of a set of contingency tables when framework for generating ecological inference forecasts.
only the margins are known. It defines a fundamental The proposals from this approach can be traced back
problem in disciplines such as political science, sociology to Hawkes (1969) and Irwin and Meeter (1969). After
and epidemiology (Salway & Wakefield, 2004). Cleave, them, some key references include Corominas, Lusa, and
Brown, and Payne (1995), Greiner (2007) and Pavia, Cabrer, Dolors Calvet (2015), McCarthy and Ryan (1977), Pavía
and Sala (2009) describe other areas of application. The and Romero (2021), Romero, Pavía, Martín, and Romero
fundamental difficulty of EIF lies in the fact that this is a (2020), and Tziafetas (1986). Solutions based on other
problem with more unknowns than observations, giving strategies, for instance, entropy maximization, have been
also suggested (see, for example, Bernardini Papalia &
rise to concerns over identifiability and indeterminacy:
Fernandez Vazquez, 2020; Johnston & Pattie, 2000).
many sets of substantively different internal cell counts
are consistent with a given marginal table. To overcome
2.11. Forecasting with judgment
this issue, a similarity hypothesis (and, sometimes, the
use of covariates) is routinely assumed. The basic hypoth-
2.11.1. Judgmental forecasting75
esis considers that either conditional row (underlying) People may use judgment alone to make forecasts or
probabilities or fractions are similar (related) among con- they may use it in combination with statistical meth-
tingency tables (Greiner & Quinn, 2010). The covariations ods. Here the focus is on pure judgmental forecasting
among row and column margins of the different tables are (for judgmental adjustments, see Section 2.11.2). Different
then used to learn about the internal cells. types of judgment heuristic (mental ‘rules of thumb’)
The above hypothesis is not a cure-all to the main can be used to make forecasts. The heuristic used de-
drawback of this approach. EIF is exposed to the so- pends on the nature of the information available to the
called ecological fallacy (Robinson, 1950): the presence forecaster (Harvey, 2007).
74 This subsection was written by Jose M. Pavía. 75 This subsection was written by Nigel Harvey.
754
Consider cases where the only relevant information is & Siemsen, 2020; McNees, 1990; Perera, Hurley, Fahimnia,
held in the forecaster’s memory. For example, someone & Reisi, 2019). However, cognitive factors (see, for ex-
might be asked whether Manchester United or Burnley ample, Section 2.11.1), and motivational biases (see Sec-
will win next week’s match. Here one memory-based tion 3.2.2), can lead to the inefficient use of informa-
heuristic that might be applicable is the recognition heuris- tion (Fildes, Goodwin, & Önkal, 2019a), unwarranted ad-
tic: if one recognises one object but not the other, then justments and reductions in accuracy (Fildes, Goodwin,
one should infer that the recognised object has higher Lawrence, & Nikolopoulos, 2009; Franses & Legerstee,
value (Goldstein & Gigerenzer, 2002). In the above ex- 2009a).
ample, most people who recognise just one of the teams People may ‘egocentrically discount’ a computer’s fore-
would be likely to make a correct forecast that Manch- casts when its rationale is less clear than their own
ester United will win (Ayton, Önkal, & McReynolds, 2011). reasoning (Bonaccio & Dalal, 2006). They can also be
The availability heuristic is another memory-based heuris-
less tolerant of errors made by algorithms than those
tic that may be applicable: objects that are most easily
made by humans (Dietvorst, Simmons, & Massey, 2015;
brought to mind are those which are more likely. Thus,
Önkal, Goodwin, Thomson, Gönül, & Pollock, 2009, and
if we are asked which team is likely to come top of the
Section 3.7.4). The random errors associated with algo-
premier league, we would say Manchester United if that
rithmic forecasts, and the salience of rare large errors, can
is the one that most easily comes to mind. The availability
heuristic is often effective because more likely events are therefore lead to an unjustified loss of trust in computer
encountered more often and more recently and are hence forecasts. Adjustments may also give forecasters a sense
better remembered. However, it can be disrupted by, for of ownership of forecasts or be used to justify their
example, greater media coverage of more unlikely (and role (Önkal & Gönül, 2005).
hence more interesting) events. Computer-based forecasts are designed to filter ran-
Consider next cases in which forecasters possess infor- domness from time-series. In contrast, humans tend to
mation about values of one or more variables correlated perceive non-existent systematic patterns in random move-
with the variable to be forecast. For example, teachers ments (O’Connor, Remus, & Griggs, 1993; Reimers & Har-
may wish to forecast the grades of their students in a final vey, 2011, and Section 3.7.3) and apply adjustments to
examination on the basis of past records of various other reflect them. This can be exacerbated by the narrative fal-
measures. Kahneman and Tversky (1973) suggested that lacy (Taleb, 2008), where people invent stories to explain
people use the representativeness heuristic to deal with these random movements, and hindsight bias (Fischhoff,
this type of situation. Forecasters first select a variable 2007), where they believe, in retrospect, that these move-
that they think is able to represent the one that must ments were predictable. Recent random movements, and
be predicted. For example, a teacher may consider that events, are particularly likely to attract undue attention,
frequency in attending voluntary revision classes repre- so long-run patterns identified by the computer are given
sents a student’s ability in the final examination. Thus, if insufficient weight (Bolger & Harvey, 1993). Damaging
a student attended 15 of the 20 revision classes, they are interventions are also probable when they result from
likely to obtain 75% in the final examination. political interference (Oliva & Watson, 2009) or optimism
Finally, consider situations in which people forecast bias (Fildes et al., 2009), or when they reflect information
future values of a variable on the basis of a record of pre- already factored into the computer’s forecast, leading to
vious values of that variable. There is some evidence that, double counting (Van den Broeke, De Baets, Vereecke,
when forecasting from time series, people use anchor-
Baecke, & Vanderheyden, 2019).
and-adjustment heuristics (Hogarth & Makridakis, 1981;
How can interventions be limited to occasions when
Lawrence & O’Connor, 1992). For example, (i) when fore-
they are likely to improve accuracy? Requiring people to
casting from an upward trended series, they anchor on
document reasons justifying adjustments can reduce gra-
the last data point and then make an upward adjustment
tuitous interventions (Goodwin, 2000b). Explaining the
to take the trend into account and (ii) when forecast-
ing from an untrended series containing autocorrelation, rationale underlying statistical forecasts also improved
they anchor on the last data point and make an adjust- adjustment behaviour when series had a simple under-
ment towards the mean to take the autocorrelation into lying pattern in a study by Goodwin and Fildes (1999).
account. However, providing guidance on when to adjust was in-
Kahneman (2011) and others have divided cognitive effective in an experiment conducted by Goodwin, Fildes,
processes into those which are intuitive (System 1) and Lawrence, and Stephens (2011), as was a restriction pre-
those which are deliberative (System 2). We have dis- venting people from making small adjustments.
cussed only intuitive processes underlying judgmental When determining the size of adjustments required,
forecasting (Gigerenzer, 2007). However, they can be sup- decomposing the judgment into a set of easier tasks im-
plemented by deliberative (System 2) processes (Theocharis proved accuracy in a study by Webby, O’Connor, and Ed-
& Harvey, 2019) in some circumstances. mundson (2005). Providing a database of past outcomes
that occurred in circumstances analogous to those ex-
2.11.2. Judgmental adjustments of computer-based forecasts76 pected in the forecast period also improved adjustments
Judgmental adjustments to algorithmic computer-based in a study by Lee, Goodwin, Fildes, Nikolopoulos, and
forecasts can enhance accuracy by incorporating impor- Lawrence (2007). Outcome feedback, where the forecaster
tant extra information into forecasts (Fahimnia, Sanders, is informed of the most recent outcome is unlikely to be
useful since it contains noise and exacerbates the ten-
76 This subsection was written by Paul Goodwin. dency to over-focus on recent events (Goodwin & Fildes,
755
1999; Petropoulos, Fildes, & Goodwin, 2016). However, were asked to select the best performing model out of
feedback on biases in adjustments over several recent a choice of two different qualities (accuracies) of models
periods may improve judgments (Petropoulos, Goodwin, (different combinations of good versus medium versus
& Fildes, 2017). Feedback will be less useful where in- bad). People’s choice outperformed forecasts made by
terventions are rare so there is insufficient data to assess averaging the model outputs, lending credence to the
performance. views of Fifić and Gigerenzer (2014). The performance
The evidence for this section is largely based on of the participants improved with a larger difference in
laboratory-based studies of adjustment behaviour. Sec- quality between models and a lower level of noise in the
tion 3.7.3 gives details of research into forecast adjust- data series. In a second experiment, De Baets and Harvey
ment in practice and discusses the role of forecasting sup- (2020) found that participants adjusted more towards the
port systems in improving the effectiveness of judgmental advice of what they perceived to be a good quality model
adjustments. than a medium or bad quality one.
Importantly, in selecting an algorithm and seeing it
2.11.3. Judgmental model selection77 err, people are quick to abandon it. This phenomenon is
Forecasters – practitioners and researchers alike – use known as ‘algorithm aversion’ (Dietvorst et al., 2015, see
Forecast Support Systems (FSS) in order to perform their also Section 2.11.6) and is due to a ‘perfection schema’ we
forecasting tasks. Usually, such an FSS allows the fore- have in our heads where algorithms are concerned (Mad-
caster to load their historical data and they can then havan & Wiegmann, 2007). We do not expect them to ‘fail’
apply many different types of forecasting techniques to and thus react strongly when they do. While a model may
the selected data. The idea is that the forecaster selects not perform as it should for a particular dataset and may
the method which leads to highest forecast accuracy. Yet, thus elicit algorithm aversion for that particular method,
there is no universally ‘best’ method, as it depends on the one should not abandon it for all datasets and future
data that is being forecasted (see also Section 2.5.4). Thus, forecasts.
selection is important in achieving high accuracy. But how
does this selection occur?
2.11.4. Panels of experts78
Research on judgmental selection of statistical fore-
Panels of experts are often used in practice to produce
casts is limited in quantity. Lawrence, Goodwin, and Fildes
judgmental forecasts (see, for example, Sections 3.2.6 and
(2002) found that participants were not very adept at
3.8.5). This is especially true in cases with limited avail-
selecting good forecasting algorithms from a range of-
able quantitative data and with the level of uncertainty
fered to them by an FSS and had higher error than those
being very high. In this section, three methods for elicit-
who were presented with the optimal algorithm by an
ing judgmental forecasts from panels of experts are pre-
FSS. Petropoulos et al. (2018b) compared judgmental se-
sented: the Delphi method, interaction groups (IG), and
lection of forecasting algorithms with automatic selection
structured analogies (SA).
based on predetermined information criteria. They found
The Delphi method is centred around organising and
that judgmental selection was better than automatic se-
structuring group communication (Rao, Anderson, Suku-
lection at avoiding the ‘worst’ models, but that automatic
mar, Beauchesne, Stein, & Frankel, 2010), which aims
selection was superior at choosing the ‘best’ ones. In the
to achieve a convergence of opinion on a specific real-
end, overall accuracy of judgmental selection was better
than that of algorithmic selection. If their experiment world issue. It is a multiple-round survey in which experts
had included more variation of the data (trends, fractals, participate anonymously to provide their forecasts and
different autoregressive factors) and variation of proposed feedback (Rowe & Wright, 2001). At the end of each
models, this could possibly have led to better algorith- round, the facilitator collects and prepares statistical sum-
mic than judgmental performance (Harvey, 2019). Time maries of the panel of experts’ forecasts. These summaries
series that are more complex will place a higher cognitive are presented as feedback to the group, and may be used
load on judgmental selection. This was confirmed in a towards revising their forecasts. This loop continues un-
study by Han, Wang, Petropoulos, and Wang (2019), who til a consensus is reached, or the experts in the panel
used an electroencephalogram (EEG) for the comparison are not willing to revise their forecasts further. In some
of judgmental forecast selection versus (judgmental) pat- implementations of the Delphi method, justification of ex-
tern identification. They found that pattern identification treme positions (forecasts) is also part of the (anonymous)
outperformed forecast selection, as the latter required feedback process. The Delphi method results in a more
a higher cognitive load, which in turn led to a lower accurate outcome in the decision-making process (Dalkey,
forecasting accuracy. 1969; Steurer, 2011). Rowe and Wright (2001) mentioned
It is likely that, in practice, judgmental selection is that, by adopting the Delphi method, groups of individuals
much more common than automatic selection. This pref- can produce more accurate forecasts than simply using
erence for human judgment over advice from an algo- unstructured methods. A drawback of the Delphi method
rithm has been shown in an experiment by Önkal et al. is the additional cost associated with the need to run
(2009). But how apt are forecasters in distinguishing multiple rounds, extending the forecasting process as well
‘good’ models from ‘bad’ models? This was investigated as increasing the potential drop-out rates. On the other
by De Baets and Harvey (2020) in an experiment. People hand, the anonymity in the Delphi method eliminates
77 This subsection was written by Shari De Baets. 78 This subsection was written by Konstantia Litsiou.
756
issues such as groupthink and the ‘dominant personalities’ The recent research has used behavioural experimen-
effects (Van de Ven & Delbeco, 1971). tation to examine various features of scenarios and their
The IG method suggests that the members of the panel interactions with judgmental adjustments (see
of experts actively interact and debate their points to the Section 2.11.2) of model-based forecasts. Önkal et al.
extent they have to reach an agreement on a common (2013) explored the ‘content’ effects of scenarios where
forecast (Litsiou, Polychronakis, Karami, & Nikolopoulos, through the narration either a bleak/negative future (a
2019). Sniezek and Henry (1989) found that members pessimistic scenario) or a bright/positive future (an opti-
of interacting groups provide more accurate judgments mistic scenario) was portrayed. On a demand forecasting
compared to individuals. However, there is mixed evi- context for mobile phones, the participants first received
dence about the forecasting potential of IG time-series data, model-based forecasts and then asked to
(Boje & Murnighan, 1982; Graefe & Armstrong, 2011; generate point and interval forecasts as well as provide a
Scott Armstrong, 2006). Besides, the need for arrang- confidence measure. With respect to the existence of sce-
ing and facilitating meetings for the IG makes it a less narios, there were four conditions where the participants
attractive option. may receive: (i) no scenarios, (ii) optimistic scenarios,
Another popular approach to judgmental forecasting (iii) pessimistic scenarios, and (iv) both scenarios. Find-
using panels of experts is SA, which refers to the recol- ings indicate that decision makers respond differently to
lection of past experiences and the use analogies (Green optimistic and pessimistic scenarios. Receiving optimistic
& Armstrong, 2007). In the SA method, the facilitator scenarios resulted in making larger adjustments to the
assembles a panel of experts. The experts are asked to model-based forecasts. At the same time, led to an in-
recall and provide descriptions, forecasts, and similari- creased confidence of the participants in their predictions.
ties/differences for cases analogous to the target situation, On the other hand, participants who received negative
as well as a similarity ranking for each of these analogous scenarios tend to lower their predictions the most among
cases. The facilitator gathers the lists of the analogies the four groups. An intriguing finding was the balancing
provided by the experts, and prepares summaries, usu- effect of scenarios on the interval forecast symmetry. The
ally using weighted averages of the recalled cases based lower interval bounds were adjusted upwards the most
on their similarity to the target situation (see also Sec- towards the centre-point of the interval (i.e., model-based
tion 2.6.4). Semi-structured analogies (sSA) have also been predictions) when optimistic scenarios were received.
proposed in the literature, where the experts are asked Similarly, the upper bounds were adjusted downwards
to provide a final forecasts based on the analogous cases the most towards the centre-point of the interval in the
they recalled, which essentially reduces the load for the presence of pessimistic scenarios.
facilitator (Nikolopoulos, Litsa, Petropoulos, Bougioukos, The prospects of receiving a single scenario versus
& Khammash, 2015). Nikolopoulos et al. (2015) supported multiple scenarios were further explored in Goodwin
that the use of SA and IG could result to forecasts that are et al. (2019b). The researchers investigated whether as-
50% more accurate compared to unstructured methods similation or contrast effects will occur when decision
(such as unaided judgment). One common caveat of using makers see optimistic (pessimistic) forecasts followed by
panels of experts is the difficulty to identify who a real pessimistic (optimistic) ones compared against receiving
expert is. Engaging experts with high level of experi- a single scenario in solitude. In case of assimilation, a sce-
ence, and encouraging the interaction of experts are also nario presenting an opposing world view with the initial
supported by Armstrong and Green (2018). one would cause adjustments in the opposite direction
creating an offset effect. On the other hand, in case of
2.11.5. Scenarios and judgmental forecasting79 contrast, the forecasts generated after the initial scenarios
Scenarios provide exhilarating narratives about con- would be adjusted to more extremes when an opposing
ceivable futures that are likely to occur. Through such de- scenario is seen. In two experiments conducted in dif-
pictions they broaden the perspectives of decision makers ferent contexts the researchers found resilient evidence
and act as mental stimulants to think about for contrast effects taking place. Interestingly, seeing an
alternatives. Scenarios enhance information sharing and opposing scenario also increased the confidence of the
provide capable tools for communication within organi- forecasters in their initial predictions.
sations. By virtue of these merits, they have been widely In terms of the effects of scenario presence on the fore-
used in corporate planning and strategy setting since casting performance, however, the experimental evidence
1960’s (Godet, 1982; Goodwin & Wright, 2010; Schoe- indicates the benefits are only circumstantial. Goodwin,
maker, 1991; Wright & Goodwin, 1999, 2009). Even though Gönül, and Önkal (2019a) found that providing scenar-
utilisation of scenarios as decision advice to judgmen- ios worsened forecast accuracy and shifted the resultant
tal forecasting has been proposed earlier (Bunn & Salo,
production order decisions further away from optimal-
1993; Schnaars & Topol, 1987), the research within this
ity. Despite this performance controversy, the decision
domain remained limited until recently when the interest
makers express their fondness in receiving scenarios and
in the subject has rekindled (Goodwin, Gönül, Önkal,
belief in their merits (Goodwin et al., 2019b; Önkal et al.,
Kocabıyıkoğlu, & Göğüş, 2019b; Önkal, Sayım, & Gönül,
2013). Therefore, we need more tailored research on sce-
2013; Wicke, Dhami, Önkal, & Belton, 2019).
narios and judgmental forecasting to reveal the conditions
when scenarios can provide significant improvements to
79 This subsection was written by M. Sinan Gönül. the forecasting accuracy.
757
2.11.6. Trusting model and expert forecasts80 and trust (Cross & Sproull, 2004). Such ‘experienced’ cred-
Defined as ‘‘firm belief in the reliability, truth, and abil- ibility appears to be more influential on users’ accep-
ity of someone/something’’ (Oxford English Dictionary), tance of given forecasts as opposed to ‘presumed’ credibil-
trust entails accepting vulnerability and risk (Rousseau, ity (Önkal, Gönül, Goodwin, Thomson, & Öz, 2017). Source
Sitkin, Burt, & Camerer, 1998). Given that forecasts are credibility can be revised when forecast (in)accuracy is
altered or even discarded when distrusted by users, exam- encountered repetitively (Jiang, Muhanna, & Pick, 1996),
ining trust is a central theme for both forecasting theory with forecaster and user confidence playing key roles (Sah,
and practice. Moore, & MacCoun, 2013).
Studies examining individual’s trust in model versus Trust is critical for forecasting efforts to be translated
expert forecasts show that individuals often distrust algo- into sound decisions (Choi, Özer, & Zheng, 2020; Özer,
Zheng, & Chen, 2011). Further work on fostering trust in
rithms (Burton, Stein, & Jensen, 2020; Meehl, 2013) and
individual/collaborative forecasting will benefit from how
place higher trust on human advice (Diab, Pui, Yankele-
trusted experts and models are selected and combined to
vich, & Highhouse, 2011; Eastwood, Snook, & Luther,
enhance decision-making.
2012, but also Sections 2.11.2, 2.11.3 and 3.7.4). We live
in an era where we are bombarded with news about how 2.12. Evaluation, validation, and calibration
algorithms get it wrong, ranging from COVID-19 fore-
casts affecting lockdown decisions to algorithmic grade 2.12.1. Benchmarking81
predictions affecting university admissions. Individuals When a new forecasting model or methodology is
appear to prefer forecasts from humans over those from proposed, it is common for its performance to be bench-
statistical algorithms even when those forecasts are iden- marked according to some measure of forecast accuracy
tical (Önkal et al., 2009). Furthermore, they lose trust against other forecasting methods using a sub-sample
in algorithms quicker when they see forecast errors (Di- of some particular time series. In this process, there is
etvorst et al., 2015; Prahl & Van Swol, 2017). Such ‘al- the risk that either the measures of accuracy, competing
gorithm aversion’ and error intolerance is reduced when forecasting methods or test data, are chosen in a way that
users have opportunity to adjust the forecasting outcome, exaggerates the benefits of a new method. This possibility
irrespective of the extent of modification allowed (Di- is only exacerbated by the phenomenon of publication
etvorst, Simmons, & Massey, 2018). Feedback appears to bias (Dickersin, 1990).
improve trust, with individuals placing higher trust in A rigorous approach to benchmarking new forecasting
algorithms if they can understand them (Seong & Bisantz, methods should follow the following principles:
2008). Overuse of technical language may reduce un- 1. New methods should always be compared to a
derstanding of the forecast/advice, in turn affecting per- larger number of suitable benchmark methods.
ceptions of expertise and trustworthiness (Joiner, Leve- These should at a minimum include naïve meth-
son, & Langfield-Smith, 2002). Explanations can be help- ods such as a random walk and also popular gen-
ful (Goodwin, Gonul & Onkal, 2013), with their packaging eral purpose forecasting algorithms such as ARIMA
affecting judgments of trustworthiness (Elsbach & Elof- models, Exponential Smoothing, Holt Winters and
son, 2000). Algorithmic appreciation appears to easily the Theta method (see Section 2.3 and references
fade with forecasting expertise (Logg, Minson, & Moore, therein).
2019), emphasising the importance of debiasing against 2. Forecasts should be evaluated using a diverse set
overconfidence and anchoring on one’s own predictions. of error metrics for point, interval and probabilistic
Trusting experts also presents challenges (Hendriks, forecasts (see Section 2.12.2). Where the forecast-
Kienhues, & Bromme, 2015; Hertzum, 2014; Maister, Gal- ing problem at hand should be tailored to a specific
ford, & Green, 2012). Expert forecasts are typically seen problem, then appropriate measures of forecast ac-
as predisposed to group-based preconceptions (Brennan, curacy must be used. As an example, the literature
2020; Vermue, Seger, & Sanfey, 2018), along with con- on Value at Risk forecasting has developed a num-
textual and motivational biases (Burgman, 2016). Mis- ber of backtesting measures for evaluating the qual-
informed expectations, distorted exposures to ‘forecast ity of quantile forecasts (see Zhang & Nadarajah,
failures’, and over-reliance on one’s own judgments may 2018, and references therein).
all contribute to distrusting experts as well as algorithms. 3. Testing should be carried out to discern whether
Credibility of forecast source is an important determi- differences between forecasting methods are statis-
tically significant. For discussion see Section 2.12.6.
nant in gaining trust (Önkal, Gönül, & De Baets, 2019).
However, there should also be a greater awareness
Studies show that the perceived credibility of system
of the debate around the use of hypothesis testing
forecasts affects expert forecasters’ behaviours and trust
both in forecasting (Armstrong, 2007) and more
(Alvarado-Valencia & Barrero, 2014), while providing in-
generally in statistics (Wasserstein & Lazar, 2016).
formation on limitations of such algorithmic forecasts
4. Sample sizes for rolling windows should be chosen
may reduce biases (Alvarado-Valencia, Barrero, Önkal, &
with reference to the latest literature on rolling
Dennerlein, 2017). Previous experience with the source window choice (see Inoue et al., 2017, and refer-
appears to be key to assessing credibility (Hertzum, 2002) ences therein).
80 This subsection was written by Dilek Önkal. 81 This subsection was written by Anastasios Panagiotelis.
758
5. All code used to implement and benchmark new forecasting horizon H with observations yn+1 , . . . , yn+H
forecasting methods should, where possible, be writ- and point forecasts fn+1 , . . . , fn+H , we have:
ten in open source programming languages (such n+H
as C, Python and R). This is to ensure replicability
∑
MSE = (yt − ft )2 ,
of results (for more on the replicablity crisis in
t =n+1
research see Peng, 2015, and references therein)
n+H ∑n+H
6. Methods should be applied to appropriate bench- ∑ t =n+1 |yt − ft |
MAE = |yt − ft |, MASE = ∑n
mark datasets. |yt − yt −1 |
t =n+1 t =2
Regarding the last of these points there are some ex- n+H n+H
∑ |yt − ft | ∑ |yt − ft |
amples in specific fields, of datasets that already play a de MAPE = , sMAPE = 1
facto role as benchmarks. In macroeconomic forecasting, yt (y + ft )
t =n+1 t =n+1 2 t
the U.S. dataset of Stock and Watson (2012, see §2.7.1) is n+H
often used to evaluate forecasting methods that exploit a
∑
Qα = (1 − α )(ft − yt )1yt <ft + α (yt − ft )1yt ≥ft .
large number of predictors, with Forni, Hallin, Lippi, and
t =n+1
Reichlin (2003) and Panagiotelis, Athanasopoulos, Hynd-
man, Jiang, and Vahid (2019) having constructed similar Let us take a step back. Assume we have a full den-
datasets for the EU and Australia respectively. In the field sity forecast and wish to ‘‘condense’’ it to a point fore-
of energy, the GEFCom data (Hong, Pinson, Fan, Zareipour, cast that will minimise some PFEM in expectation. The
Troccoli, & Hyndman, 2016) discussed in Section 3.4.3 and key observation is that different PFEMs will be minimised
the IEEE 118 Bus Test Case data (Peña, Martinez-Anido, & by different point forecasts derived from the same density
Hodge, 2018) are often used as benchmarks. Finally, the forecast (Kolassa, 2020b).
success of the M Forecasting competitions (Makridakis, • The MSE is minimised by the expectation.
Spiliotis et al., 2020) provide a benchmark dataset for • The MAE and MASE are minimised by the median
general forecasting methodologies (see Section 2.12.7 and (Hanley, Joseph, Platt, Chung, & Belisle, 2001).
references therein). • The MAPE is minimised by the (−1)-median (Gneit-
A recent trend that has great future potential is the ing, 2011a, p. 752 with β = −1).
publication of websites that demonstrate the efficacy of • The sMAPE is minimised by an unnamed functional
different forecasting methods on real data. The Covid- that would need to be minimised numerically
19 Forecast Hub82 and the Business Forecast Lab83 pro- (Gonçalves, 2015).
vide notable examples in the fields of epidemiology and • The hinge/tick/pinball Qp loss is minimised by the
macroeconomics and business respectively. appropriate p-quantile (Gneiting, 2011b).
• In general, there is no loss function that is minimised
2.12.2. Point, interval, and pHDR forecast error measures84 by the mode (Heinrich, 2014).
Point forecasts are single number forecasts for an un-
known future quantity also given by a single number. We note that intermittent demand (see Section 2.8)
Interval forecasts take the form of two point forecasts, poses specific challenges. On the one hand, the MAPE is
an upper and a lower limit. Finally, a less common type undefined if there are zeros in the actuals. On the other
of forecast would be a predictive Highest Density Region hand, the point forecasts minimising different PFEMs will
(pHDR), i.e., an HDR (Hyndman, 1996) for the condi- be very different. For instance, the conditional median
tional density of the future observable. pHDRs would be (minimising the MAE) may well be a flat zero, while
interesting for multimodal (possibly implicit) predictive the conditional mean (minimising the MSE) will usually
densities, e.g., in scenario planning. Once we have ob- be nonzero. Our forecasting algorithm may not output
served the corresponding realisation, we can evaluate our an explicit density forecast. It is nevertheless imperative
point, interval and pHDR forecasts. to think about which functional of the implicit density
There are many common point forecast error measures we want to elicit (Gneiting, 2011a), and tailor our error
(PFEMs), e.g., the mean squared error (MSE), mean ab- measure – and forecasting algorithm! – to it. It usually
solute error (MAE), mean absolute scaled error (MASE), makes no sense to evaluate a point forecast with multiple
mean absolute percentage error (MAPE) or many oth- PFEMs (Kolassa, 2020b).
ers (see Section 3.4 in Hyndman & Athanasopoulos, 2018). Interval forecasts can be specified in multiple ways.
Which one is most appropriate for our situation, or should We can start with a probability coverage and require two
we even use multiple different PFEMs? appropriate quantiles – e.g., we could require a 2.5% and
There are many common point forecast error measures a 97.5% quantile forecast, yielding a symmetric or equal-
(PFEMs), e.g., the MSE, MAE, MASE, (s)MAPE, the quantile tailed 95% interval forecast. Interval forecasts (ℓt , ut ) of
score or pinball loss or many others (e.g., sections 5.8 this form can be evaluated by the interval score (Brehmer
and 5.9 in Hyndman & Athanasopoulos, 2021). Assuming & Gneiting, 2020; Winkler, 1972), a proper scoring rule
(section 6.2 in Gneiting & Raftery, 2007):
n historical periods with observations y1 , . . . , yn and a
n+H
∑ 2 2
82 https://viz.covid19forecasthub.org/. ISα = (ut − ℓt ) + (ℓt − yt )1yt <ℓt + (yt − ut )1yt >ut .
83 https://business-forecast-lab.com/. α α
t =n+1
84 This subsection was written by Stephan Kolassa.
(3)
759
We can also use the hinge loss to evaluate the quantile The scoring rules are attractive to managers in practice
forecasts separately. since they are considered proper. Proper scoring rules (see
Alternatively, we can require a shortest interval sub- also Section 2.12.4) incentivise honest forecasts from the
ject to a specified coverage. This interval is not elic- experts, even prior to knowing the realisation of an uncer-
itable relative to practically relevant classes of distribu- tainty, since ex ante the expected score is maximised only
tions (Brehmer & Gneiting, 2020; Fissler, Frongillo, Hlavi- when reported probabilities equals true beliefs (Bickel,
nová, & Rudloff, 2020). 2007; Gneiting & Raftery, 2007; Merkle & Steyvers, 2013;
Yet another possibility is to maximise the interval fore- O’Hagan et al., 2006; Winkler, Muñoz, Cervera, Bernardo,
cast’s probability coverage, subject to a maximum length Blattenberger, Kadane, Lindley, Murphy, Oliver, & Ríos-
ℓ. This modal interval forecast (ft , ft +ℓ) is elicitable by an Insua, 1996). Examples of a scoring rule that is not proper
appropriate ℓ-zero–one-loss (Brehmer & Gneiting, 2020): yet still commonly used are the linear score, which simply
n+H equals the reported probability or density for the actual
outcome, or the skill score, which is the percentage im-
∑
Lℓ = 1ft <yt <ft +ℓ
provement of the Brier score for the forecast relative to
t =n+1
the Brier score of some base line naive forecast (Winkler
= # t ∈ n + 1, . . . , n + H ⏐ ft < yt < ft + ℓ .
{ ⏐ }
(4) et al., 2019).
The pHDR is not elicitable even for unimodal densi- For forecasting continuous quantities, forecasts could
ties (Brehmer & Gneiting, 2020). In the multimodal case, be elicited by asking for an expert’s quantile (or fractile)
the analysis is likely difficult. Nevertheless, a variation of forecast rather than a probability forecast. For instance,
the Winkler score has been proposed to evaluate pHDRs the 0.05, 0.25, 0.50, 0.75 and 0.95 quantiles are often
on an ad hoc basis (Hyndman, 2020). One could also elicited in practice, and in some cases every 0.01 quantile,
compare the achieved to the nominal coverage, e.g., using between 0–1 are elicited (e.g., the 2014 Global Energy
a binomial test – which disregards the volume of the Forecasting Competition, Hong et al., 2016). Proper scor-
pHDR (Kolassa, 2020a). ing rules for quantiles are developed in Jose and Winkler
In conclusion, there is a bewildering array of PFEMs, (2009).
which require more thought in choosing among than is When forecasts are used for decision-making, it is
obvious at first glance. The difficulties involved in eval- beneficial if the scoring rule used relates in some manner
uating interval and pHDR forecasts motivate a stronger to the decision problem itself. In certain settings, the
emphasis on full density forecasts (cf. Askanazi, Diebold, connection of the scoring rule to the decision context
Schorfheide, & Shin, 2018, and Section 2.12.4). is straight forward. For example, Jose, Nau, and Win-
kler (2008) develop scoring rules that can be mapped to
2.12.3. Scoring expert forecasts85 decision problems based on the decision maker’s utility
Evaluating forecasting capabilities can be a difficult function. Johnstone, Jose, and Winkler (2011) develop tai-
task. One prominent way to evaluate an expert’s forecast lored scoring rules aligning the interest of the forecaster
is to score the forecast once the realisation of the un- and the decision maker. Grushka-Cockayne, Lichtendahl,
certainty is known. Scoring forecasts using the outcome’s Jose and Winkler (2017) link quantile scoring rules to
realisations over multiple forecasts offers insights into business profit-sharing situations.
an individual’s expertise. Experts can also use scoring
information to identify ways to improve future forecasts. 2.12.4. Evaluating probabilistic forecasts86
In addition, scoring rules and evaluation measures can Probabilistic forecasting is a term that is not strictly
be designed to match decision-making problems, incen- defined, but usually refers to everything beyond point
tivising forecasts that are most useful in a specific sit- forecasting (Gneiting, 2011a). However, in this section we
uation (Winkler, Grushka-Cockayne, Lichtendahl, & Jose, consider only the evaluation of full predictive distribu-
2019). tions or equivalent characterisations. For the evaluation of
Scoring rules were first suggested for evaluate me- prediction of quantiles, intervals and related objects, see
teorological forecasts in work by Brier (1950). Scoring Section 2.12.2.
rules have since been used in a wide variety of settings, One crucial point for evaluating probabilistic forecasts
such as business and other applications. When forecasting is the reporting, which is highly influenced from meteoro-
a discrete uncertainty with only two possible outcomes logic communities. From the theoretical point of view, we
(e.g., a loan with be defaulted on or not, a customer will should always report the predicted cumulative distribu-
click on an ad or not), the Brier score assigns a score of tion function F̂ of our prediction target F . Alternatively for
−(1 − p)2 , where p is the probability forecast reported that continuous data, reporting the probability density func-
the event will occurs. The greater the probability reported tion is a popular choice. For univariate prediction prob-
for an event that occurs, the higher the score the fore- lems a common alternative is to report quantile forecast
cast receives. Over multiple forecasts, better forecasters on a dense grid of probabilities, as it approximates the full
will tend to have higher average Brier scores. For dis- distribution (Hong et al., 2016). For multivariate forecasts,
crete events with more than two outcomes, a logarithmic it seems to become standard to report a large ensemble
scoring rule can be used. (a set of simulated trajectories/paths) of the full predictive
85 This subsection was written by Yael Grushka-Cockayne. 86 This subsection was written by Florian Ziel.
760
distribution. The reason is that the reporting of a multi- forecast is calibrated if the observation cannot be dis-
variate distribution (or an equivalent characterisation) of tinguished from a random draw from the predictive dis-
sophisticated prediction models is often not feasible or tribution or, in the case of ensemble forecasts, if the
practicable, especially for non-parametric or copula-based observation and the ensemble members look like ran-
forecasting methods. dom draws from the same distribution. Additionally, to
In general, suitable tools for forecasting evaluation ensure their utility in decision making, forecasts should
are proper scoring rules as they address calibration and be sharp, or specific, see Sections 2.12.2 and 2.12.4 as well
sharpness simultaneously (Gneiting & Katzfuss, 2014; Gneit- as (Gneiting et al., 2007).
In the univariate setting, several alternative notions of
ing & Raftery, 2007). Preferably, we consider strictly proper
calibration exist for both a single forecast (Gneiting et al.,
scoring rules which can identify the true predicted distri-
2007; Tsyplakov, 2013) and a group of forecasts (Strähl
bution among a set of forecast candidates that contains
& Ziegel, 2017). The notion most commonly used in ap-
the true model. plications is probabilistic calibration (Dawid, 1984); the
In the univariate case the theory is pretty much settled forecast system is probabilistically calibrated if the prob-
and there is quite some consensus about the evaluation ability integral transform (PIT) of a random observation,
of probabilistic forecasts (Gneiting & Katzfuss, 2014). The that is, the value of the predictive cumulative distribution
continuous ranked probability score (CRPS) and logarith- function in the observation, is uniformly distributed. If
mic scores (log-score) are popular strictly proper scoring the predictive distribution has a discrete component, a
rules, while the quadratic and pseudospherical score re- randomised version of the PIT should be used (Gneiting
main strictly proper alternatives. The CRPS can be well & Ranjan, 2013).
approximated by averaging across quantile forecasts on Probabilistic calibration is assessed visually by plotting
an equidistant grid of probabilities (Nowotarski & Weron, the histogram of the PIT values over a test set. A cali-
2018). brated forecast system will return a uniform histogram,
For multivariate forecast evaluation the situation is a ∩-shape indicates overdispersion and a ∪-shape in-
more complicated and many questions remain open dicates underdispersion, while a systematic bias results
(Gneiting & Raftery, 2007; Meng, Taylor, Ben Taieb, & in a biased histogram (e.g. Thorarinsdottir & Schuhen,
2018). The discrete equivalent of the PIT histogram, which
Li, 2020). The multivariate version of the log-score is a
applies to ensemble forecasts, is the verification rank
strictly proper scoring rule, but it requires the availability
histogram (Anderson, 1996; Hamill & Colucci, 1997). It
of a multivariate density forecast. This makes it impracti-
shows the distribution of the ranks of the observations
cable for many applications. Gneiting and Raftery (2007) within the corresponding ensembles and has the same
discuss the energy score, a multivariate generalisation of interpretation as the PIT histogram.
the CRPS, that is strictly proper. Still, it took the energy For small test sets, the bin number of a PIT/rank his-
score more than a decade to increase its popularity in togram must be chosen with care. With very few bins,
forecasting. A potential reason is the limited simulation the plot may obscure miscalibration while with many
study of Pinson and Tastu (2013) that concludes that bins, even perfectly calibrated forecasts can yield non-
the energy score can not discriminate well differences in uniformly appearing histograms (Heinrich, 2020; Tho-
the dependency structure. In consequence other scoring rarinsdottir & Schuhen, 2018). The bin number should be
rules were proposed in literature, e.g., the variogram chosen based on the size of the test set, with the bin num-
score (Scheuerer & Hamill, 2015) which is not strictly ber increasing linearly with the size of the test set (Hein-
proper. Ziel and Berk (2019) consider a strictly proper rich, 2020). More specifically, the uniformity of PIT/rank
scoring method for continuous variables using copula values can be assessed with statistical tests (Delle Monache,
techniques. In contrast to Pinson and Tastu (2013), recent Hacker, Zhou, Deng, & Stull, 2006; Taillardat, Mestre,
studies (Lerch et al., 2020; Ziel & Berk, 2019) show that Zamo, & Naveau, 2016; Wilks, 2019), where the test
statistics can be interpreted as a distance between the
the energy score discriminates well when used together
observed and a flat histogram (Heinrich, 2020; Wilks,
with significance tests like the Diebold–Mariano (DM)
2019). Testing predictive performance is further discussed
test. In general, we recommended scoring be applied with
in Section 2.12.6.
reliability evaluation (see Section 2.12.5) and significance Calibration assessment of multivariate forecasts is
tests (see Section 2.12.6). Additionally, if we want to complicated by the lack of a unique ordering in higher
learn about the performance of our forecasts it is highly dimensions and the many ways in which the forecasts
recommended to consider multiple scoring rules and eval- can be miscalibrated (Wilks, 2019). Gneiting, Stanberry,
uate on lower-dimensional subspaces. For multivariate Grimit, Held, and Johnson (2008) propose a general two-
problems, this holds particularly for the evaluation of step approach where an ensemble forecast and the corre-
univariate and bivariate marginal distributions. sponding observation are first mapped to a single value by
a pre-rank function. Subsequently, the pre-rank function
2.12.5. Assessing the reliability of probabilistic forecasts87 values are ranked in a standard manner. The challenge
Probabilistic forecasts in the form of predictive distri- here is to find a pre-rank function that yields informa-
butions are central in risk-based decision making where tive and discriminative ranking (Gneiting et al., 2008;
reliability, or calibration, is a necessary condition for the Thorarinsdottir, Scheuerer, & Heinz, 2016; Wilks, 2004);
optimal use and value of the forecast. A probabilistic see Thorarinsdottir et al. (2016) and Wilks (2019) for
comparative studies. Alternatively, Ziegel and Gneiting
(2014) propose a direct multivariate extension of the
87 This subsection was written by Thordis Thorarinsdottir. univariate setting based on copulas.
761
2.12.6. Statistical tests of forecast performance88 different approach for studying performance of multiple
A natural consequence of growth in forecasting method- forecasting models is through the use of multiple compar-
ologies was the development of statistical tests for predic- ison tests such as multiple comparison with a control and
tive ability in the last thirty years. These tests provided multiple comparison with the best (Edwards & Hsu, 1983;
forecasters some formal reassurance that the predictive Horrace & Schmidt, 2000; Hsu, 1981). These tests often
superiority of a leading forecast is statistically significant are based on jointly estimated confidence intervals that
and is not merely due to random chance. measure the difference between two parameters of inter-
One of the early papers that undoubtedly sparked est such as the forecast accuracies of a model and a bench-
growth in this field was Diebold and Mariano (1995, mark. Koning, Franses, Hibon, and Stekler (2005) illus-
DM hereafter). In their seminal paper, DM provided a trates how they can be ex post used to analyse forecasting
simple yet general approach for testing equal predic- performance in the M3 forecasting competition (Makri-
tive ability, i.e., if two forecasting sources (f1,t and f2,t , dakis & Hibon, 2000) using model ranking instead of
t = 1, . . . , h) are equally accurate on average. Math- forecast accuracy scores as its primitives. The multiple
ematically, if we denote the error ei,t = yt − fi,t for comparison of the best was used in the analysis of the
i = 1, 2 and t = 1, . . . , h, the hypotheses for this DM subsequent M4 and M5 Competitions (Makridakis, Spili-
test is H0 : E [L(−e1,t ) − L(−e2,t )] = 0 for all t versus otis et al., 2020; Makridakis, Spiliotis & Assimakopoulos,
H1 : E [L(−e1,t ) − L(−e2,t )] ̸ = 0 under a loss function 2021, and Section 2.12.7).
L. Their population-level predictive ability test has very
few assumptions (e.g., covariance stationary loss differ- 2.12.7. Forecasting competitions89
ential) and is applicable to a wide range of loss functions, Forecasting competitions provide a ‘‘playground’’ for
multi-period settings, and wide class of forecast errors academics, data scientists, students, practitioners, and
(e.g., non-Gaussian, serially and/or contemporaneously software developers to compare the forecasting perfor-
correlated). This test though not originally intended for mance of their methods and approaches against others.
models has been widely used by others to test forecasting Organisers of forecasting competitions test the perfor-
models’ accuracy (Diebold, 2015). mance of the participants’ submissions against some hid-
Modifications were later introduced by Harvey, Ley- den data, usually the last window of observations for
bourne, and Newbold (1998) to improve small sample each series. The benefits from forecasting competitions
properties of the test. Generalisations and extensions have are multifold. Forecasting competitions (i) motivate the
emerged to address issues that DM tests encountered in development of innovative forecasting solutions, (ii) pro-
practice such as nested models (Clark & McCracken, 2001, vide a deeper understanding of the conditions that some
2009), parameter estimation error (West, 1996), cointe- methods work and others fail, (iii) promote knowledge
grated variables (Corradi, Swanson, & Olivetti, 2001), high dissemination, (iv) provide a much-needed, explicit link
persistence (Rossi, 2005), and panel data (Timmermann between theory and practice, and (v) leave as a legacy
& Zhu, 2019). Finite-sample predictive ability tests also usable and well-defined data sets. Participation in fore-
emerged from the observation that models may have casting competitions is sometimes incentivised by mone-
equal predictive ability in finite samples, which generated tary prizes. However, the stakes are usually much higher,
a class called conditional predictive accuracy tests (Clark including reputational benefits.
& McCracken, 2013; Giacomini & White, 2006). The most famous forecasting competitions are the ones
An alternative approach to comparing forecast ac- organised by Spyros Makridakis. Initially, the research
curacy is through the notion of forecast encompassing, question focused on the relative performance of simple
which examines if a forecast encompasses all useful infor- versus complex forecast. M and M3 competitions (Makri-
mation from another with respect to predictions (Chong dakis et al., 1982; Makridakis & Hibon, 2000) empiri-
& Hendry, 1986; Clark & McCracken, 2001; Harvey et al., cally showed that simple methods (such as exponential
1998). Though it has a few more assumptions, forecast en- smoothing; see Section 2.3.1) are equally good compared
compassing tests in certain contexts might be preferable to other more complex methods and models (such as
to the mean square prediction error tests à la Diebold– ARIMA and neural networks; see Section 2.3.4 and Sec-
Mariano (Busetti & Marcucci, 2013). tion 2.7.8 respectively) in point-forecast accuracy – if
Another stream of available statistical tests looks at not better. Moreover, the early Makridakis competitions
multiple forecasts simultaneously instead of pairs. Ad- showed the importance of forecast combinations in in-
dressing a need for a reality check on ‘‘data snooping’’, creasing predictive accuracy. For example, the winner
White (2000) later modified by Hansen (2005) developed of the M3 competition was the Theta method (see Sec-
a multiple model test that uses a null hypothesis of ‘‘su- tion 2.3.3), a simple statistical method that involved the
perior predictive ability’’ instead of the equal predictive combination of linear regression and simple exponen-
ability used in DM tests. These have also been generalised tial smoothing forecasts (Assimakopoulos & Nikolopoulos,
to deal with issues such as cointegrated variables (Cor- 2000).
radi et al., 2001) and multi-horizon forecasts (Quaedvlieg, The M4 competition (Makridakis, Spiliotis et al., 2020)
2019). Recently, Li, Liao and Quaedvlieg (2020) proposed challenged researchers and practitioners alike with a task
a conditional superior predictive ability test similar to Gi- of producing point forecasts and prediction intervals for
acomini and White (2006)’s innovation to the DM test. A 100 thousand time series of varied frequencies. This time,
88 This subsection was written by Victor Richmond R. Jose. 89 This subsection was written by Fotios Petropoulos.
762
the main hypothesis focused on the ability of machine economics, meteorology and climate sciences, as well as
learning and neural network approaches in the task of management science among others. In this section, many
time series forecasting. Machine learning approaches (see aspects of the theory of forecasting were covered, with
Section 2.7.10) that focused on each series independently aspects related to data, modelling and reasoning, forecast
performed poorly against statistical benchmarks, such as verification. Now, the fact that forecasting is mature does
Theta, Damped exponential smoothing or simple averages not mean that all has been done – we aim here at giving
of exponential smoothing models. However, the best two a few pointers at current and future challenges.
performing submissions in the M4 competition (Montero- First of all, it is of utmost importance to remember
Manso et al., 2020; Smyl, 2020) used neural network that forecasting is a process that involves both quantita-
and machine learning algorithms towards utilising cross- tive aspects (based on data and models) and humans, at
learning. So, the main learning outcome from the M4 various levels, i.e., from the generation of forecasts to their
competition is that, if utilised properly, machine learning use in decision-making. A first consequence is that we
can increase the forecasting performance. Similarly to always need to find, depending on the problem at hand,
previous competitions, M4 demonstrated again the use- an optimal trade-off between data-driven approaches and
fulness of combining across forecasts, with five out of the the use of expert judgment. In parallel, forecasting is to be
top six submissions offering a different implementation of thought of in a probabilistic framework in a systematic
forecast combinations. manner (Gneiting & Katzfuss, 2014). This allows us to
Several other forecasting competitions focused on spe- naturally convey uncertainty about the future, while pro-
cific contexts and applications. For example, M2 compe- viding the right basis to make optimal decisions in view of
tition (Makridakis et al., 1993) suggested that the ben- the characteristics of the decision problem, as well as the
efits from additional information (domain expertise) are loss (or utility) function and risk aversion of the decision
limited; see also Section 2.11.4. The tourism forecast- maker. Another consequence is that using forecasts as
ing competition (Athanasopoulos, Hyndman, Song, & Wu, input to decision-making often affects the outcome to be
2011) also showed that exogenous variables do not add predicted itself – a problem known as self-negating fore-
value, while naive forecasts perform very well on a yearly casts (possibly also self-fulfilling) or the prophet dilemma.
frequency (for a discussion on tourism forecasting appli- With advances in the science of dynamic systems and
cations, see Section 3.8.1). The NN3 competition (Crone, game theory, we should invest in modelling those systems
Hibon, & Nikolopoulos, 2011) confirmed the superior per- as a whole (i.e., forecasting and decision-making) in order
formance of statistical methods, but noted that neural to predict the full range of possible outcomes, based on
network approaches are closing the distance. Tao Hong’s the decisions that could be made.
series of energy competitions (Hong, Pinson, & Fan, 2014; In parallel, it is clear that today, the amount of data
Hong et al., 2016; Hong, Xie, & Black, 2019) demonstrated being collected and possibly available for forecasting is
best practices for load, price, solar, and wind forecasting, growing at an astounding pace. This requires re-thinking
with extensions to probabilistic and hierarchical fore- our approaches to forecasting towards high-dimensional
casts (for energy forecasting applications, see Section 3.4). models, online learning, etc. Importantly, the data being
Finally, many companies have hosted forecasting chal- collected is distributed in terms of ownership. And, due to
lenges through the Kaggle platform. Bojer and Meldgaard privacy concerns and competitive interests, some may not
(2020) reviewed the Kaggle competitions over the last be ready to share their data. Novel frameworks to learning
five years, and concluded that access to hierarchical infor- and forecasting ought to be developed with that context
mation, cross-learning, feature engineering, and combina- in mind, for instance focusing on distributed and privacy-
tions (ensembles) can lead to increased forecasting per- preserving learning – an example among many others is
formance, outperforming traditional statistical methods. that of Google pushing forward federated learning (Abadi
These insights were a forerunner to the results of the M5 et al., 2016), an approach to deep learning where the
competition, which focused on hierarchically organised learning process is distributed and with a privacy layer.
retail data (Makridakis, Spiliotis, Assimakopoulos, 2021; Eventually the access and use of data, as well as the
Makridakis, Spiliotis, Assimakopoulos, Chen & Winkler, contribution to distributed learning (and collaborative an-
2021). alytics, more generally), may be monetised, bringing a
Makridakis, Fry, Petropoulos and Spiliotis (2021) pro- mechanism design component to the future theory of
vide a list of design attributes for forecasting competitions forecasting. A simple and pragmatic example is that of
and propose principles for future competitions. forecast reconciliation: if asking various agents to modify
their forecasts to make them coherent within a hierarchy,
2.13. The future of forecasting theory90 such modifications could be monetised to compensate for
accuracy loss.
The theory of forecasting appears mature today, based A large part of today’s modelling and forecasting ap-
on dedicated developments at the interface among a proaches uses a wealth of data to identify and fit models,
number of disciplines, e.g., mathematics and statistics, to be eventually used to forecast based on new data and
computer sciences, psychology, etc. A wealth of these under new conditions. Different approaches have been
theoretical developments have originated from specific proposed to maximise the generalisation ability of those
needs and challenges in different application areas, e.g., in models, to somewhat maximise chances to do well out-
of-sample. At the root of this problem is the effort to go
90 This subsection was written by Pierre Pinson. beyond correlation only, and to identify causality (see,
763
e.g., Pearl (2009) for a recent extensive coverage). While as evangelical forecasting), unencumbered by what the
causality has been a key topic of interest to forecasters for data show. More commonly, the process begins with a
a long time already, new approaches and concepts are be- statistical forecast (generated by forecasting software),
ing pushed forward for identification of and inference in which is then subject to review and adjustment, as illus-
causal models (Peters, Janzing, & Schölkopf, 2017), which trated in Fig. 5.
may have a significant impact on the theory of forecasting. In concept, such an elaborate multi-stage process al-
Eventually, the key question of what a good forecast lows ‘‘management intelligence’’ to improve forecast qual-
is will continue to steer new developments in the the- ity, incorporating information not accounted for in the
ory of forecasting in the foreseeable future. The nature statistical model. In reality, however, benefits are not
of goodness of forecasts (seen from the meteorological assured. Lawrence, Goodwin, O’Connor, and Önkal (2006)
application angle) was theorised a few decades ago al- reviewed more than 200 studies, concluding that human
ready (Murphy, 1993), based on consistency, quality and judgment can be of significant benefit but is also subject
value. We still see the need to work further on that ques- to significant biases. Among the many papers on this
tion – possibly considering these 3 pillars, but possibly subject, there is general agreement on the need to track
also finding other ways to define desirable properties and review overrides, and the need to better understand
of forecasts. This will, in all cases, translates to further the psychological issues around judgmental adjustments.
developing frameworks for forecast verification, focusing The underlying problem is that each human touch
on the interplay between forecast quality and value, but point subjects the forecast to the interests of the review-
also better linking to psychology and behavioural eco- ers – and these interests may not align with creating an
nomics. In terms of forecast verification, some of the accurate, unbiased forecast. To identify where such prob-
most pressing areas most likely relate to (multivariate) lems are occurring, Forecast Value Added (FVA) analysis
probabilistic forecasting and to the forecasting of extreme is an increasingly popular approach among practitioners.
events. When it comes to forecast quality and value, we FVA is defined as the change in a forecasting perfor-
need to go beyond the simple plugging of forecasts into mance metric that can be attributed to a particular step
decision problems to assess whether this yields better or participant in the forecasting process (Gilliland, 2002).
decisions, or not. Instead, we ought to propose suitable Any activity that fails to deliver positive FVA (i.e., fails to
theoretical frameworks that allow assessing whether cer- improve forecast quality) is considered process waste.
tain forecasts are fundamentally better (than others) for Starting with a naive forecast, FVA analysis seeks to
given classes of decision problems. Finally, the link to determine whether each subsequent step in the process
psychology and behavioural economics should ensure a improves upon the prior steps. The ‘‘stairstep report’’ of
better appraisal of how forecasts are to be communicated, Table 1 is a familiar way of summarising results, as in this
how they are perceived and acted upon. example from Newell Rubbermaid (Schubert & Rickard,
Most of the advances in the science of forecasting 2011).
have come from the complementarity between theoret- Here, averaged across all products, naive (random walk)
ical developments and applications. We can then only be achieved forecast accuracy of 60%. The company’s statis-
optimistic for the future since more and more application tical forecast delivered five percentage points of improve-
areas are relying heavily on forecasting. Their specific ment, but management review and adjustment delivered
needs and challenges will continue fuelling upcoming negative value. Such findings – not uncommon – urge
developments in the theory of forecasting. further investigation into causes and possible process
corrections (such as training reviewers or limiting adjust-
3. Practice ments). Alternatively, the management review step could
be eliminated, providing the dual benefits of freeing up
3.1. Introduction to forecasting practice91 management time spent on forecasting and, on average,
more accurate forecasts.
The purpose of forecasting is to improve decision mak- Morlidge (2014c) expanded upon FVA analysis to
ing in the face of uncertainty. To achieve this, forecasts present a strategy for prioritising judgmental adjustments,
should provide an unbiased guess at what is most likely finding the greatest opportunity for error reduction in
to happen (the point forecast), along with a measure of products with high volume and high relative absolute
uncertainty, such as a prediction interval (PI). Such infor- error. Chase (2021) described a machine learning (ML)
mation will facilitate appropriate decisions and actions. method to guide forecast review, identifying which fore-
Forecasting should be an objective, dispassionate ex- casts are most likely to benefit from adjustment along
ercise, one that is built upon facts, sound reasoning, and with a suggested adjustment range. Baker (2021) used
sound methods. But since forecasts are created in social ML classification models to identify characteristics of non-
settings, they are influenced by organisational politics and value adding overrides, proposing the behavioural eco-
personal agendas. As a consequence, forecasts will often nomics notion of a ‘‘nudge’’ to prompt desired forecaster
reflect aspirations rather than unbiased projections. behaviour. Further, Goodwin, Petropoulos, and Hyndman
In organisations, forecasts are created through pro- (2017) derived upper bounds for FVA relative to naive
cesses that can involve multiple steps and participants. forecasts. And de Kok (2017) created a Stochastic Value
The process can be as simple as executive fiat (also known Added (SVA) metric to assess the difference between
actual and forecasted distributions, knowledge of which
91 This subsection was written by Michael Gilliland. is valuable for inventory management.
764
Fig. 5. Multi-stage forecasting process.
Table 1
Stairstep report showing FVA results.
Process step Forecast accuracy FVA vs. Naive FVA vs. statistical
(100%–MAPE)
Naive forecast 60%
Statistical forecast 65% 5%
Adjusted forecast 62% 2% −3%
Including an indication of uncertainty around the point on detecting where their process is severely suboptimal
forecast remains an uncommon practice. Prediction in- and taking measures to redress the problem’’. This is
tervals in software generally underestimate uncertainty, where FVA can help.
often dramatically, leading to unrealistic confidence in For now, the challenge for researchers remains: To
the forecast. And even when provided, PIs largely go prompt practitioners to adopt sound methods based on
unused by practitioners. Goodwin (2014) summarised the the objective assessment of available information, and
psychological issues, noting that the generally poor cali- avoid the ‘‘worst practices’’ that squander resources and
bration of the PIs may not explain the reluctance to utilise fail to improve the forecast.
them. Rather, ‘‘an interval forecast may accurately reflect
the uncertainty, but it is likely to be spurned by decision 3.2. Operations and supply chain management
makers if it is too wide and judged to be uninforma-
tive’’ (Goodwin, 2014, page 5). 3.2.1. Demand management92
It has long been recognised (Chatfield, 1986; Lawrence, Demand management is one of the dominant com-
2000) that the practice of forecasting falls well short of ponents of supply chain management (Fildes, Goodwin,
the potential exhibited in academic research, and revealed & Lawrence, 2006). Accurate demand estimate of the
by the M forecasting competitions. In the M4, a simple present and future is a first vital step for almost all
benchmark combination method (the average of Single, aspects of supply chain optimisation, such as inventory
Holt, and Damped exponential smoothing) reduced the management, vehicle scheduling, workforce planning, and
overall weighted average (OWA) error by 17.9% compared distribution and marketing strategies (Kolassa & Siemsen,
to naive. The top six performing methods in M4 further 2016). Simply speaking, better demand forecasts can yield
reduced OWA by over 5% compared to the combination significantly better supply chain management, including
benchmark (Makridakis, Spiliotis et al., 2020). But in fore- improved inventory management and increased service
casting practice, just bettering the accuracy of naive has levels. Classic demand forecasts mainly rely on qualitative
proven to be a surprising challenge. Morlidge’s (2014b) techniques, based on expert judgment and past experi-
study of eight consumer and industrial businesses found ence (e.g., Weaver, 1971), and quantitative techniques,
52% of their forecasts failed to do so. And, as shown, based on statistical and machine learning modelling (e.g.,
Newel Rubbermaid beat naive by just two percentage Bacha & Meyer, 1992; Taylor, 2003b). A combination of
points after management adjustments. qualitative and quantitative methods is also popular and
Ultimately, forecast accuracy is limited by the nature of proven to be beneficial in practice by, e.g., judgmental ad-
the behaviour being forecast. But even a highly accurate justments (Önkal & Gönül, 2005; Syntetos, Kholidasari &
forecast is of little consequence if overridden by man- Naim, 2016; Turner, 1990, and Section 2.11.2), judgmental
agement and not used to enhance decision making and forecast model selection (Han et al., 2019; Petropoulos
improve organisational performance. et al., 2018b, and Section 2.11.3), and other advanced
Practitioners need to recognise limits to forecastability forecasting support systems (Arvan, Fahimnia, Reisi, &
and be willing to consider alternative (non-forecasting) Siemsen, 2019; Baecke, De Baets, & Vanderheyden, 2017,
approaches when the desired level of accuracy is not see also Section 3.7.1).
achievable (Gilliland, 2010). Alternatives include supply The key challenges that demand forecasting faces vary
chain re-engineering – to better react to unforeseen vari- from domain to domain. They include:
ations in demand, and demand smoothing – leverag-
ing pricing and promotional practices to shape more 1. The existence of intermittent demands, e.g., irregu-
favourable demand patterns. lar demand patterns of fashion products. According
Despite measurable advances in our statistical fore- to Nikolopoulos (2020), limited literature has fo-
casting capabilities (Makridakis, Hyndman & Petropoulos, cused on intermittent demand. The seminal work
2020), it is questionable whether forecasting practice has by Croston (1972) was followed by other represen-
similarly progressed. The solution, perhaps, is what Mor- tative methods such as the SBA method by Syntetos
lidge (2014a, page 39) suggests that ‘‘users should focus
less on trying to optimise their forecasting process than 92 This subsection was written by Yanfei Kang.
765
and Boylan (2001), the aggregate–disaggregate in- management, logistics and human resources (also see Sec-
termittent demand approach (ADIDA) by Nikolopou- tion 3.2.1). Typically, forecasts are based on an amalgam
los et al. (2011), the multiple temporal aggrega- of statistical methods and management judgment (Fildes
tion by Petropoulos and Kourentzes (2015), and & Goodwin, 2007). Hofmann and Rutschmann (2018) have
the k nearest neighbour (kNN) based approach by investigated the potential for using big data analytics in
Nikolopoulos et al. (2016). See Section 2.8 for more supply chain forecasting but indicate more research is
details on intermittent demand forecasting and Sec- needed to establish its usefulness.
tion 2.10.2 for a discussion on temporal aggrega- In many organisations forecasts are a crucial element
tion. of Sales and Operations Planning (S&OP), a tool that brings
2. The emergence of new products. Recent studies together different business plans, such as those relating
on new product demand forecasting are based on to sales, marketing, manufacturing and finance, into one
finding analogies (Hu, Acimovic, Erize, Thomas, & integrated set of plans (Thomé, Scavarda, Fernandez, &
Van Mieghem, 2019; Wright & Stern, 2015), lever- Scavarda, 2012). The purposes of S&OP are to balance sup-
aging comparable products (Baardman, Levin, Per- ply and demand and to link an organisation’s operational
akis, & Singhvi, 2018), and using external informa- and strategic plans. This requires collaboration between
tion like web search trends (Kulkarni, Kannan, & individuals and functional areas at different levels be-
Moe, 2012). See Section 3.2.6 for more details on cause it involves data sharing and achieving a consensus
new product demand forecasting. on forecasts and common objectives (Mello, 2010). Suc-
3. The existence of short-life-cycle products, e.g., cessful implementations of S&OP are therefore associated
smartphone demand (e.g., Chung, Niu, & Sriskan- with forecasts that are both aligned with an organisa-
darajah, 2012; Shi, Yin, Cai, Cichocki, Yokota, Chen, tion’s needs and able to draw on information from across
Yuan, & Zeng, 2020; Szozda, 2010). the organisation. This can be contrasted with the ‘silo
4. The hierarchical structure of the data such as the culture’ identified in a survey of companies by Moon,
electricity demand mapped to a geographical hi- Mentzer, and Smith (2003) where separate forecasts were
erarchy (e.g., Athanasopoulos et al., 2009; Hong prepared by different departments in ‘islands of analysis’.
et al., 2019; Hyndman et al., 2011, but also Sec- Methods for reconciling forecasts at different levels in
tion 2.10.1). both cross-sectional hierarchies (e.g., national, regional
and local forecasts) and temporal hierarchies (e.g., annual,
With the advent of the big data era, a couple of co- monthly and daily forecasts) are also emerging as an ap-
existing new challenges have drawn the attention of re- proach to break through information silos in organisations
searchers and practitioners in the forecasting community: (see Section 2.10.1, Section 2.10.2, and Section 2.10.3).
the need to forecast a large volume of related time se- Cross-temporal reconciliation provides a data-driven ap-
ries (e.g., thousands or millions of products from one large proach that allows information to be drawn from dif-
retailer: Salinas, Michael et al., 2019), and the increas- ferent sources and levels of the hierarchy and enables
ing number of external variables that have significant this to be blended into coherent forecasts (Kourentzes &
influence on future demand (e.g., massive amounts of Athanasopoulos, 2019).
keyword search indices that could impact future tourism In some supply chains, companies have agreed to share
demand (Law, Li, Fong, & Han, 2019)). Recently, to deal data and jointly manage planning processes in an initia-
with these new challenges, numerous empirical studies tive known as Collaborative Planning, Forecasting, and Re-
have identified the potentials of deep learning based plenishment (CPFR) (Seifert, 2003, also see Section 3.2.3).
global models, in both point and probabilistic demand CPFR involves pooling information on inventory levels and
forecasting (e.g., Bandara, Bergmeir, & Smyl, 2020b; Ran- on forthcoming events, like sales promotions. Demand
gapuram et al., 2018; Salinas, Michael et al., 2019; Wen forecasts can be shared, in real time via the Internet,
et al., 2017). With the merits of cross-learning, global and discrepancies between them reconciled. In theory,
models have been shown to be able to learn long memory information sharing should reduce forecast errors. This
patterns and related effects (Montero-Manso & Hyndman, should mitigate the ‘bullwhip effect’ where forecast errors
2020), latent correlation across multiple series (Smyl, at the retail-end of supply chains cause upstream sup-
2020), handle complex real-world forecasting situations pliers to experience increasingly volatile demand, forcing
such as data sparsity and cold-starts (Chen, Kang, Chen, & them to hold high safety stock levels (Lee et al., 2007).
Wang, 2020), include exogenous covariates such as pro- Much research demonstrating the benefits of collabora-
motional information and keyword search indices (Law tion has involved simulated supply chains (Fildes, 2017).
et al., 2019), and allow for different choices of distribu- Studies of real companies have also found improved per-
tional assumptions (Salinas, Michael et al., 2019). formance through collaboration (e.g., Boone & Ganeshan,
2008; Eksoz, Mansouri, Bourlakis, & Önkal, 2019; Hill,
3.2.2. Forecasting in the supply chain93 Zhang, & Miller, 2018), but case study evidence is still
A supply chain is ‘a network of stakeholders (e.g., re- scarce (Syntetos, Babai, Boylan, Kolassa, & Nikolopoulos,
tailers, manufacturers, suppliers) who collaborate to sat- 2016). The implementation of collaborative schemes has
isfy customer demand’ (Perera et al., 2019). Forecasts been slow with many not progressing beyond the pilot
inform many supply chain decisions, including those re- stage (Galbreth, Kurtuluş, & Shor, 2015; Panahifar, Byrne,
lating to inventory control, production planning, cash flow & Heavey, 2015). Barriers to successful implementation
include a lack of trust between organisations, reward sys-
93 This subsection was written by Paul Goodwin. tems that foster a silo mentality, fragmented forecasting
766
systems within companies, incompatible systems, a lack the greater stability of data at the lower frequency level,
of relevant training and the absence of top management with no need for disaggregation.
support (Fliedner, 2003; Thomé, Hollmann, & Scavarda do The variance of forecast errors over lead time is re-
Carmo, 2014). quired to determine safety stock requirements for con-
Initiatives to improve supply chain forecasting can be tinuous review systems. The conventional approach is to
undermined by political manipulation of forecasts and take the variance of one-step-ahead errors and multiply
gaming. Examples include ‘enforcing’: requiring inflated it by the lead time. However, this estimator is unsound,
forecasts to align them with sales or financial goals, ‘sand- even if demand is independent and identically distributed,
bagging’: underestimating sales so staff are rewarded for as explained by Prak, Teunter, and Syntetos (2017). A
exceeding forecasts, and ‘spinning’: manipulating fore- more direct approach is to smooth the mean square errors
casts to garner favourable reactions from colleagues over the lead time (Syntetos & Boylan, 2006).
(Mello, 2009). Pennings, van Dalen, and Rook (2019) dis- Strijbosch and Moors (2005) showed that unbiased
cuss schemes for correcting such intentional biases. forecasts will not necessarily lead to achievement, on
For a discussion of the forecasting of returned items in average, of target cycle service levels or fill rates. Wall-
supply chains, see Section 3.2.9, while Section 3.9 offers ström and Segerstedt (2010) proposed a ‘Periods in Stock’
a discussion of possible future developments in supply measure, which may be interpreted, based on a ‘fictitious
chain forecasting. stock’, as the number of periods a unit of the forecasted
item has been in stock or out of stock. Such measures
3.2.3. Forecasting for inventories94 may be complemented by a detailed examination of error-
Three aspects of the interaction between forecasting implication metrics (Boylan & Syntetos, 2006). For in-
and inventory management have been studied in some ventory management, these metrics will typically include
depth and are the subject of this review: the bullwhip ef- inventory holdings and service level implications (e.g., cy-
fect, forecast aggregation, and performance measurement. cle service level, fill rate). Comparisons may be based on
The ‘bullwhip effect’ occurs whenever there is ampli- total costs or via ‘exchange curves’, showing the trade-offs
fication of demand variability through the supply chain between service and inventory holding costs. Compar-
(Lee, Padmanabhan, & Whang, 2004), leading to excess in- isons such as these are now regarded as standard in the
ventories. This can be addressed by supply chain members literature on forecasting for inventories and align well
sharing downstream demand information, at stock keep- with practice in industry.
ing unit level, to take advantage of less noisy data. Analyt-
ical results on the translation of ARIMA (see Section 2.3.4) 3.2.4. Forecasting in retail95
demand processes have been established for order-up- Retail companies depend crucially on accurate demand
to inventory systems (Gilbert, 2005). There would be no forecasting to manage their supply chain and make de-
value in information sharing if the wholesaler can use cisions concerning planning, marketing, purchasing, dis-
such relationships to deduce the retailer’s demand pro- tribution and labour force. Inaccurate forecasts lead to
cess from their orders (see, for example, Graves, 1999). unnecessary costs and poor customer satisfaction. Inven-
Such deductions assume that the retailer’s demand pro- tories should be neither too high (to avoid waste and extra
cess and demand parameters are common knowledge to costs of storage and labour force), nor too low (to prevent
supply chain members. Ali and Boylan (2011) showed stock-outs and lost sales, Ma & Fildes, 2017).
that, if such common knowledge is lacking, there is value Forecasting retail demand happens in a three-
in sharing the demand data itself and Ali, Boylan, and dimensional space (Syntetos et al., 2016): the position
Syntetos (2012) established relationships between accu- in the supply chain hierarchy (store, distribution cen-
racy gains and inventory savings. Analytical research has tre, or chain), the level in the product hierarchy (SKU,
tended to assume that demand parameters are known. Pa- brand, category, or total) and the time granularity (day,
store, Alfieri, Zotteri, and Boylan (2020) investigated the week, month, quarter, or year). In general, the higher is
impact of demand parameter uncertainty, showing how it the position in the supply chain, the lower is the time
exacerbates the bullwhip effect. granularity required, e.g., retailers need daily forecasts for
Forecasting approaches have been developed that are store replenishment and weekly forecasts for DC distribu-
particularly suitable in an inventory context, even if not tion/logistics activities at the SKU level (Fildes, Ma, & Ko-
originally proposed to support inventory decisions. For lassa, 2019b). Hierarchical forecasting (see Section 2.10.1)
example, Nikolopoulos et al. (2011) proposed that fore- is a promising tool to generate coherent demand forecasts
casts could be improved by aggregating higher frequency on multiple levels over different dimensions (Oliveira &
data into lower frequency data (see also Section 2.10.2; Ramos, 2019).
other approaches are reviewed in Section 3.2.1). Follow- Several factors affect retail sales, which often increase
ing this approach, Forecasts are generated at the lower substantially during holidays, festivals, and other spe-
frequency level and then disaggregated, if required, to cial events. Price reductions and promotions on own and
the higher frequency level. For inventory replenishment competitors’ products, as well as weather conditions or
decisions, the level of aggregation may conveniently be pandemics, can also change sales considerably (Huang,
chosen to be the lead time, thereby taking advantage of Fildes, & Soopramanien, 2019).
94 This subsection was written by John E. Boylan. 95 This subsection was written by Stephan Kolassa & Patrícia Ramos.
767
Zero sales due to stock-outs or low demand occur very Finally, there are differences in the forecasting process:
often at the SKU × store level, both at weekly and daily
granularity. The most appropriate forecasting approaches
• B&M retailers decouple pricing/promotion decisions
and optimisation from the customer interaction, and
for intermittent demand are Croston’s method (Croston,
therefore from forecasting. Online, this is not possi-
1972), the Syntetos-Boylan approximation (SBA; Syntetos
ble, because the customer has total transparency to
& Boylan, 2005), and the TSB method (Teunter, Syntetos,
competitors’ offerings. Thus, online pricing needs to
& Zied Babai, 2011), all introduced in Section 2.8.1. These
react much more quickly to competitive pressures –
methods have been used to forecast sales of spare parts
faster than the forecasting cycle.
in automotive and aerospace industries but have not yet
been evaluated in the retail context.
• Thus, the specific value of predictors is often not
Univariate forecasting models are the most basic meth- known at the time of forecasting: we don’t know
ods retailers may use to forecast demand. They range yet which customer will log on, so we don’t know
from simple methods such as simple moving averages or yet how many people will see a particular product
exponential smoothing to ARIMA and ETS models (dis- displayed on their personalised landing page. (Nor
cussed in Section 2.3). These are particularly appropriate do we know today what remaining stock will be dis-
to forecast demand at higher aggregation levels (Ramos played.) Thus, changes in drivers need to be ‘‘baked
& Oliveira, 2016; Ramos, Santos, & Rebelo, 2015). The into’’ the forecasting algorithm.
main advantage of linear causal methods such as multiple • Feedback loops between forecasting and other pro-
linear regression is to allow the inclusion of external cesses are thus even more important online: yester-
effects discussed above. There is no clear evidence yet that day’s forecasts drive today’s stock position, driving
nonlinear models and novel machine learning methods today’s personalised recommendations, driving de-
can improve forecast accuracy (Fildes et al., 2019b). mand, driving today’s forecasts for tomorrow. Over-
To be effective, point estimates should be combined all, online retail forecasting needs to be more ag-
with quantile predictions or prediction intervals for deter- ile and responsive to the latest interactional de-
mining safety stock amounts needed for replenishment. cisions taken in the web store, and more tightly
However, to the best of our knowledge this is an under- integrated into the retailer’s interactional tactics and
investigated aspect of retail forecasting (Kolassa, 2016; omnichannel strategy.
Taylor, 2007). Systematic research on demand forecasting in an on-
The online channel accounts for an ever-increasing line or omnnichannel context is only starting to appear (e.g.
proportion of retail sales and poses unique challenges to Omar, Klibi, Babai, & Ducq, 2021, who use basket data
forecasting, beyond the characteristics of brick and mortar from online sales to improve omnichannel retail fore-
(B&M) retail stores. First, there are multiple drivers or casts).
predictors of demand that could be leveraged in online
retail, but not in B&M: 3.2.5. Promotional forecasting96
• Online retailers can fine-tune customer interactions, Promotional forecasting is central for retailing (see
e.g., through the landing page, product recommen- Section 3.2.4), but also relevant for many manufacturers,
dations, or personalised promotions, leveraging the particularly of Fast Moving Consumer Goods (FMCG). In
customer’s purchasing, browsing or returns history, principle, the objective is to forecast sales, as in most busi-
current shopping cart contents, or the retailer’s stock ness forecasting cases. However, what sets promotional
position, in order to tailor the message to one spe- forecasting apart is that we also make use of information
cific customer in a way that is impossible in B&M. about promotional plans, pricing, and sales of comple-
• Conversely, product reviews are a type of interaction mentary and substitute products (Bandyopadhyay, 2009;
between the customer and the retailer and other Zhang, Chen, & Lee, 2008). Other relevant variables may
customers which drives future demand. include store location and format, variables that capture
the presentation and location of a product in a store, prox-
Next, there are differences in forecast use: ies that characterise the competition, and so on (Andrews,
Currim, Leeflang, & Lim, 2008; Van Heerde, Leeflang, &
• Forecast use strongly depends on the retailer’s om-
Wittink, 2002).
nichannel strategy (Armstrong, 2017; Melacini, Per-
Three modelling considerations guide us in the choice
otti, Rasini, & Tappia, 2018; Sopadjieva, Dholakia, &
of models. First, promotional (and associated effects) are
Benjamin, 2017): e.g., for ‘‘order online, pick up in
proportional. For instance, we do not want to model the
store’’ or ‘‘ship from store’’ fulfillment, we need sep-
increase in sales as an absolute number of units, but in-
arate but related forecasts for both total online de-
stead, as a percentage uplift. We do this to not only make
mand and for the demand fulfilled at each separate
the model applicable to both smaller and larger applica-
store.
tions, for example, small and large stores in a retailing
• Online retailers, especially in fashion, have a much
chain, but also to gain a clearer insight into the behaviour
bigger problem with product returns. They may need
of our customers. Second, it is common that there are
to forecast how many products are returned over-
synergy effects. For example, a promotion for a product
all (e.g., Shang, McKie, Ferguson, & Galbreth, 2020),
or whether a specific customer will return a specific
product. 96 This subsection was written by Nikolaos Kourentzes.
768
may be offset by promotions for substitute products. Both & Meeran, 2013) namely management judgment, con-
these considerations are easily resolved if we use multi- sumer judgment and diffusion/formal mathematical mod-
plicative regression models. However, instead of working els. In general, the hybrid methods combining different
with the multiplicative models, we rely on the logarithmic approaches have been found to be more useful (Hynd-
transformation of the data (see Section 2.2.1) and proceed man & Athanasopoulos, 2018; Peres et al., 2010). Most
to construct the promotional model using the less cum- of the attempts in New product Forecasting (NPF) have
bersome additive formulation (see Section 2.3.2). Third, been about forecasting ‘adoption’ (i.e., enumerating the
the objective of promotional models does not end with customers who bought at least one time) rather than
providing accurate predictions. We are also interested in ‘sales’, which accounts for repeat purchases also. In gen-
the effect of the various predictors: their elasticity. This eral, these attempts dealt with point forecast although
can in turn provide the users with valuable information there have been some attempts in interval and density
about the customers, but also be an input for constructing forecasting (Meade & Islam, 2001).
optimal promotional and pricing strategies (Zhang et al., Out of the three approaches in NPF, management judg-
2008). ment is the most used approach (Gartner & Thomas,
Promotional models have been widely used on brand- 1993; Kahn, 2002; Lynn, Schnaars, & Skov, 1999) which
level data (for example, Divakar, Ratchford, & Shankar, is reported to have been carried out by either individual
2005). However, they are increasingly used on Stock Keep- managers or group of them. Ozer (2011) and Surowiecki
ing Unit (SKU) level data (Ma, Fildes, & Huang, 2016; (2005) articulated their contrasting benefits and deficits.
Trapero, Kourentzes, & Fildes, 2015), given advances in The Delphi method (see Section 2.11.4) has combined
modelling techniques. Especially at that level, limited the benefits of these two modes of operation (Rowe &
sales history and potentially non-existing examples of Wright, 1999) which has been effective in NPF. Prediction
past promotions can be a challenge. Trapero et al. (2015) markets in the recent past offered an alternative way to
consider this problem and propose using a promotional aggregate forecasts from a group of Managers (Meeran,
model that has two parts that are jointly estimated. The Dyussekeneva, & Goodwin, 2013; Wolfers & Zitzewitz,
first part focuses on the time series dynamics and is 2004) and some successful application of prediction mar-
modelled locally for each SKU. The second part tackles kets for NPF have been reported by Karniouchina (2011)
the promotional part, which pools examples of promo- and Plott and Chen (2002).
tions across SKUs to enable providing reasonable es- In the second category, customer surveys, among other
timates of uplifts even for new SKUs. To ensure the methods, are used to directly ask the customers the like-
expected heterogeneity in the promotional effects, the lihood of them purchasing the product. Such surveys
model is provided with product group information. An- are found to be not very reliable (Morwitz, 1997). An
other recent innovation is looking at modelling promo- alternative method to avoid implicit bias associated with
tional effects both at the aggregate brand or total sales such surveys in extracting inherent customer preference
level, and disaggregate SKU level, relying on temporal is conjoint analysis, which makes implicit trade off cus-
aggregation (Kourentzes & Petropoulos, 2016, and Sec- tomers make between features explicit by analysing the
tion 2.10.2). Ma et al. (2016) concern themselves with the customers’ preference for different variants of the prod-
intra-and inter-category promotional information. The chal- uct. One analysis technique that attempts to mirror real
lenge now is the number of variables to be considered life experience more is Choice Based Conjoint analysis
for the promotional model, which they address by us- (CBC) in which customers choose the most preferred
ing sequential LASSO (see also Section 2.5.3). Although product among available choices. Such CBC models used
together with the analysis tools such as Logit (McFad-
the aforementioned models have shown very promis-
den, 1977) have been successful in different NPF applica-
ing results, one has to recognise that in practice pro-
tions (Meeran, Jahanbin, Goodwin, & Quariguasi Frota Neto,
motions are often forecasted using judgmental adjust-
2017).
ments, with inconsistent performance (Trapero, Pedregal,
In the third approach, mathematical/formal models
Fildes, & Kourentzes, 2013); see also Section 2.11.2 and
known as growth or diffusion curves (see Section 2.3.18
Section 3.7.3.
and Section 2.3.19) have been used successfully to do
NPF (Hu et al., 2019). The non-availability of past data is
3.2.6. New product forecasting97
mitigated by growth curves by capturing the generic pat-
Forecasting the demand for a new product accurately
tern of the demand growth of a class of products, which
has even more consequence with regards to well-being
could be defined by a limited number of parameters such
of the companies than that for a product already in the
as saturation level, inflexion point, etc. For a new product
market. However, this task is one of the most difficult
a growth curve can be constituted from well-estimated
tasks managers must deal with simply because of non-
parameters using analogous products, market intelligence
availability of past data (Wind, 1981). Much work has
or regression methods. Most extensively used family of
been going on for the last five decades in this field.
growth curves for NPF has started with Bass model (Bass,
Despite his Herculean attempt to collate the methods re-
1969) that has been extended extensively (Bass, Gor-
ported, Assmus (1984) could not list all even at that time.
don, Ferguson, & Githens, 2001; Easingwood, Mahajan, &
The methods used before and since could be categorised
Muller, 1983; Islam & Meade, 2000; Peres et al., 2010;
into three broad approaches (Goodwin, Dyussekeneva
Simon & Sebastian, 1987). A recent applications of NPF
focused on consumer electronic goods using analogous
97 This subsection was written by Sheik Meeran. products (Goodwin, Dyussekeneva et al., 2013).
769
3.2.7. Spare parts forecasting98 forecasting spare parts demand include the Hyperbolic-
Spare parts are ubiquitous in modern societies. Their Exponential Smoothing method proposed by Prestwich,
demand arises whenever a component fails or requires Tarim, Rossi, and Hnich (2014) and the modified Croston’s
replacement. Demand for spare parts is typically inter- method developed by Babai, Dallery, Boubaker, and Kalai
mittent, which means that it can be forecasted using (2019).
the plethora of parametric and non-parametric methods
presented in Section 2.8. In addition to the intermittence 3.2.8. Predictive maintenance99
of demand, spare parts have two additional character- A common classification of industrial maintenance in-
istics that make them different from Work-In-Progress cludes three types of maintenance (Montero Jimenez,
and final products, namely: (i) they are generated by Schwartz, Vingerhoeds, Grabot, & Salaün, 2020). Correc-
maintenance policies and part breakdowns, and (ii) they tive maintenance refers to maintenance actions that occur
are subject to obsolescence (Bacchetti & Saccani, 2012; after the failure of a component. Preventive maintenance
Kennedy, Wayne Patterson, & Fredendall, 2002). consists of maintenance actions that are triggered after a
The majority of forecasting methods do not link the scheduled number of units as cycles, kilometers, flights,
demand to the generating factors, which are often related etc. To schedule the fixed time between two preventive
to maintenance activities. The demand for spare parts maintenance actions, the Weibull distribution is com-
originates from the replacement of parts in the installed monly used (Baptista, Sankararaman, de Medeiros, Nasci-
base of machines (i.e., the location and number of prod- mento, Prendinger, & Henriques, 2018). The drawbacks of
ucts in use), either preventively or upon breakdown of the preventive maintenance are related to the replacement
part (Kim, Dekker, & Heij, 2017). Fortuin (1984) claims of components that still have a remaining useful life;
that using installed base information to forecast the spare therefore, early interventions imply a waste of resources
part demand can lead to stock reductions of up to 25%. An and too late actions could imply catastrophic failures.
overview of the literature that deals with spare parts fore- Additionally, the preventive intervention itself could be
casting with installed base information is given by Van a source of failures too. Finally, predictive maintenance
der Auweraer and Boute (2019). Spare parts demand can (PdM) complements the previous ones and, essentially,
be driven by the result of maintenance inspections and, in uses predictive tools to determine when actions are nec-
this case, a maintenance-based forecasting model should essary (Carvalho, Soares, Vita, Francisco, Basto, & Alcalá,
then be considered to deal with this issue. Such forecast- 2019). Within this predictive maintenance group, other
ing models include the Delay Time (DT) model analysed terms are usually found in the literature as Condition-
in Wang and Syntetos (2011). Using the fitted values of Based Maintenance and Prognostic and Health Manage-
the distribution parameters of a data set related to a ment, (Montero Jimenez et al., 2020).
hospital pumps, Wang and Syntetos (2011) have shown The role of forecasting in industrial maintenance is
that when the failure and fault arriving characteristics of of paramount importance. One application is to forecast
the items can be captured, it is recommended to use the spare parts (see Section 3.2.7), whose demands are typi-
DT model to forecast the spare part demand with a higher cally intermittent, usually required to carry out corrective
forecast accuracy. However, when such information is and preventive maintenances (Van der Auweraer & Boute,
not available, then time series forecasting methods, such 2019; Wang & Syntetos, 2011). On the other hand, it is
as those presented in Section 2.8.1, are recommended. crucial for PdM the forecast of the remaining useful time,
The maintenance based forecasting is further discussed in which is the useful life left on an asset at a particular time
Section 3.2.8. of operation (Si, Wang, Hu, & Zhou, 2011). This work will
Given the life cycle of products, spare parts are as- be focused on the latter, which is usually found under the
sociated with a risk of obsolescence. Molenaers, Baets, prognostic stage (Jardine, Lin, & Banjevic, 2006).
Pintelon, and Waeyenbergh (2012) discussed a case study The typology of forecasting techniques employed is
where 54% of the parts stocked at a large petrochemical very ample. Montero Jimenez et al. (2020) classify them
company had seen no demand for the last 5 years. Hinton in three groups: physics-based models, knowledge-based
(1999) reported that the US Department of Defence was models, and data-driven models. Physics-based models
holding 60% excess of spare parts, with 18% of the parts require high skills on the underlying physics of the ap-
(with a total value of $1.5 billion) having no demand plication. Knowledge-based models are based on facts or
at all. To take into account the issue of obsolescence cases collected over the years of operation and main-
in spare parts demand forecasting, Teunter et al. (2011) tenance. Although, they are useful for diagnostics and
have proposed the TSB method, which deals with linearly provide explicative results, its performance on prognos-
decreasing demand and sudden obsolescence cases. By tics is more limited. In this sense, data-driven models
means of an empirical investigation based on the indi- are gaining popularity for the development of compu-
vidual demand histories of 8000 spare parts SKUs from tational power, data acquisition, and big data platforms.
the automotive industry and the Royal Air Force (RAF, In this case, data coming from vibration analysis, lubri-
UK), Babai, Syntetos, and Teunter (2014) have demon- cant analysis, thermography, ultrasound, etc. are usu-
strated the high forecast accuracy and inventory perfor- ally employed. Here, well-known forecasting models as
mance of the TSB method. Other variants of the Croston’s VARIMAX/GARCH (see also Section 2.3) are successfully
method developed to deal with the risk of obsolescence in used (Baptista et al., 2018; Cheng, Yu, & Chen, 2012;
98 This subsection was written by Mohamed Zied Babai. 99 This subsection was written by Juan Ramón Trapero Arenas.
770
García, Pedregal, & Roberts, 2010; Gomez Munoz, De la items. Past sales can then be used in conjunction with this
Hermosa Gonzalez-Carrato, Trapero Arenas, & Garcia Mar- distribution to create forecasts of returns. Serial number
quez, 2014). State Space models based on the Kalman Fil- level (SL) information, is more detailed and consists of
ter are also employed (Pedregal & Carmen Carnero, 2006; the time matching of an individual unit item’s issues and
Pedregal, García, & Roberts, 2009, and Section 2.3.6). Re- returns and thus exactly the time each individual unit, on
cently, given the irruption of the Industry 4.0, physi- a serial number basis, spent with the customer. Seriali-
cal and digital systems are getting more integrated and sation allows for a complete characterisation of the time
Machine Learning/Artificial Intelligence are drawing the to return distribution. Very importantly, it also enables
attention of practitioners and academics alike (Carvalho tracking exactly how many items previously sold remain
et al., 2019). In that same reference, it is found that with customers, providing time series of unreturned past
the most frequently used Machine Learning methods in sales. Unreturned past sales can then be extrapolated
PdM applications were Random Forest, Artificial Neural — along with a time to return distribution — to create
Networks, Support Vector Machines and K-means. forecasts of returns.
Goltsos, Syntetos, and van der Laan (2019) offered
3.2.9. Reverse logistics100 empirical evidence in the area of returns forecasting by
As logistics and supply chain operations rely upon analysing a serialised data set from a remanufacturing
accurate demand forecasts (see also Section 3.2.2), reverse company in North Wales. They found the Beta probabil-
logistics and closed loop supply chain operations rely ity distribution to best fit times-to-return. Their research
upon accurate forecasts of returned items. Such items suggests that serialisation is something worthwhile pur-
(usually referred as cores) can be anything from reusable suing for low volume products, especially if they are ex-
shipping or product containers to used laptops, mobile pensive. This makes a lot of sense from an investment per-
phones or car engines. If some (re)manufacturing activ- spective, since the relevant serial numbers are very few.
ity is involved in supply chains, it is both demand and However, they also provided evidence that such benefits
returned items forecasts that are needed since it is net expand in the case of high volume items. Importantly,
demand requirements (demand – returns) that drive re- the benefits of serialisation not only enable the imple-
manufacturing operations. mentation of the more complex SL method, but also the
Forecasting methods that are known to work well accurate characterisation of the returns process, thus also
when applied to demand forecasting, such as SES for benefiting the PL method (which has been shown to be
example (see Section 2.3.1), do not perform well when very robust).
applied to time-series of returns because they assume
returns to be a process independent of sales. There are 3.3. Economics and finance
some cases when this independence might hold, such as
when a recycler receives items sold by various companies 3.3.1. Macroeconomic survey expectations101
and supply chains (Goltsos & Syntetos, 2020). In these Macroeconomic survey expectations allow tests of the-
cases, simple methods like SES applied on the time se- ories of how agents form their expectations. Expecta-
ries of returns might prove sufficient. Typically though, tions play a central role in modern macroeconomic re-
returns are strongly correlated with past sales and the search (Gali, 2008). Survey expectations have been used
installed base (number of products with customers). After to test theories of expectations formation for the last
all, there cannot be a product return if a product has 50 years. Initially the Livingston survey data on infla-
not first been sold. This lagged relationship between sales tionary expectations was used to test extrapolative or
and returns is key to the effective characterisation of the adaptive hypothesis, but the focus soon turned to testing
returns process. whether expectations are formed rationally (see Turnovsky
Despite the increasing importance of circular econ- and Wachter, 1972, for an early contribution). Accord-
omy and research on closed loop supply chains, returns ing to Muth (1961, p. 316), rational expectations is the
forecasting has not received sufficient attention in the hypothesis that: ‘expectations, since they are informed
academic literature (notable contributions in this area predictions of future events, are essentially the same as
include Clottey, Benton Jr., & Srivastava, 2012; de Brito the predictions of the relevant economic theory.’ This
& van der Laan, 2009; Goh & Varaprasad, 1986; Toktay, assumes all agents have access to all relevant information.
2003; Toktay, Wein, & Zenios, 2000). The seminal work Instead, one can test whether agents make efficient use of
by Kelle and Silver (1989) offers a useful framework to the information they possess. This is the notion of forecast
forecasting that is based on the degree of available inefficiency (Mincer & Zarnowitz, 1969), and can be tested
formation about the relationship between demand and by regressing the outturns on a constant and the forecasts
returns. Product level (PL) information consists of the time of those outturns. Under forecast efficiency, the constant
series of sales and returns, alongside information on the should be zero and the coefficient on the forecasts should
time each product spends with a customer. The question be one. When the slope coefficient is not equal to one, the
then is how to derive this time to return distribution. This forecast errors will be systematically related to informa-
can be done through managerial experience, by investi- tion available at the forecast origin, namely, the forecasts,
gating the correlation of the demand and the returns time and cannot be optimal. The exchange between Figlewski
series, or by serialising and tracking a subset (sample) of and Wachtel (1981, 1983) and Dietrich and Joines (1983)
100 This subsection was written by Aris A. Syntetos. 101 This subsection was written by Michael P. Clements.
771
clarifies the role of partial information in testing fore- 3.3.2. Forecasting GDP and inflation103
cast efficiency (that is, full information is not necessary), As soon as Bayesian estimation of DSGEs became pop-
and shows that the use of the aggregate or consensus ular, these models have been employed in forecasting
forecast in the individual realisation-forecast regression horseraces to predict the key macro variables, for ex-
outlined above will give rise to a slope parameter less ample, Gross Domestic Product (GDP) and inflation, as
discussed in Del Negro and Schorfheide (2013). The fore-
than one when forecasters are efficient but possess partial
casting performance is evaluated using rolling or recur-
information. Zarnowitz (1985), Keane and Runkle (1990)
sive (expanded) prediction windows (for a discussion,
and Bonham and Cohen (2001) consider pooling across see Cardani, Paccagnini, & Villa, 2015). DSGEs are usually
individuals in the realisation-forecast regression, and the estimated using revised data, but several studies pro-
role of correlated shocks across individuals. pose better results estimating the models using real-time
Recently, researchers considered why forecasters might data (see, for example, Cardani et al., 2019; Del Negro
not possess full-information, stressing informational rigidi- & Schorfheide, 2013; Kolasa & Rubaszek, 2015b; Wolters,
ties: sticky information (see, inter alia, Mankiw & Reis, 2015).
2002; Mankiw, Reis, & Wolfers, 2003), and noisy informa- The current DSGE model forecasting compares DSGE
tion (see, inter alia, Sims, 2003; Woodford, 2002). Coibion models to competitors (see Section 2.3.15 for an intro-
duction to DSGE models). Among them, we can include
and Gorodnichenko (2012, 2015) test these models using
the Random Walk (the naive model which assumes a
aggregate quantities, such as mean errors and revisions.
stochastic trend), the Bayesian VAR models (Minnesota
Forecaster behaviour can be characterised by the re- Prior à la Doan, Litterman, and Sims, 1984; and Large
sponse to new information (see also Section 2.11.1). Over Bayesian VAR à la Bańbura, Giannone, and Reichlin, 2010),
or under-reaction would constitute inefficiency. Broer and the Hybrid-Models (the DSGE-VAR à la Del Negro and
Kohlhas (2018) and Bordalo, Gennaioli, Ma, and Shleifer Schorfheide, 2004; and the DSGE-Factor Augmented VAR
(2018) find that agents over-react, generating a nega- à la Consolo, Favero, and Paccagnini, 2009), and the insti-
tive correlation between their forecast revision and er- tutional forecasts (Greenbook, Survey Professional Fore-
ror. The forecast is revised by more than is warranted casts, and the Blue Chip, as illustrated in Edge and Gürkay-
by the new information (over-confidence regarding the nak, 2010).
Table 2 summarises the current DSGE forecasting lit-
value of the new information). Bordalo et al. (2018) ex-
erature mainly for the US and Euro Area and provided
plain the over-reaction with a model of ‘diagnostic’ ex-
by estimating medium-scale models. As general findings,
pectations, whereas Fuhrer (2018) finds ‘intrinsic inflation DSGEs can outperform other competitors, with the ex-
persistence’: individuals under-react to new information, ception for the Hybrid-Models, in the medium and long-
smoothing their responses to news. run to forecast GDP and inflation. In particular, Smets
The empirical evidence is often equivocal, and might and Wouters (2007) was the first empirical evidence
reflect: the vintage of data assumed for the outturns; of how DSGEs can be competitive with forecasts from
whether allowance is made for ‘instabilities’ such as alter- Bayesian VARs, convincing researchers and policymakers
nating over- and under-prediction (Rossi & Sekhposyan, in adopting DSGEs for prediction evaluations. As discussed
2016) and the assumption of squared-error loss (see, for in Del Negro and Schorfheide (2013), the accuracy of
DSGE forecasts depends on how the model is able to
example, Clements, 2014b; Patton & Timmermann, 2007).
capture low-frequency trends in the data. To explain the
Research has also focused on the histogram forecasts
macro-finance linkages during the Great Recession, the
produced by a number of macro-surveys. Density fore- Smets and Wouters model was also compared to other
cast evaluation techniques such as the probability integral DSGE specifications including the financial sector. For
transform102 have been applied to histogram forecasts, example, Cardani et al. (2019), Del Negro and Schorfheide
and survey histograms have been compared to bench- (2013), Galvão, Giraitis, Kapetanios, and Petrova (2016),
mark forecasts (see, for example, Bao, Lee, & Saltoglu, and Kolasa and Rubaszek (2015a) provide forecasting per-
2007; Clements, 2018; Hall & Mitchell, 2009). Research formance for DSGEs with financial frictions. This strand
has also considered uncertainty measures based on the of the literature shows how this feature can improve the
histograms (Clements, 2014a). Sections 2.12.4 and 2.12.5 baseline Smets and Wouters predictions for the business
cycle, in particular during the recent Great Recession.
also discuss the evaluation and reliability of probabilistic
However, the Hybrid-Models always outperform the
forecasts.
DSGEs thanks to the combination of the theory-based
Clements (2009, 2010) and Engelberg, Manski, and model (DSGE) and the statistical representation (VAR or
Williams (2009) considered the consistency between the Factor Augmented VAR), as illustrated by Del Negro and
point predictions and histogram forecasts. Reporting prac- Schorfheide (2004) and Consolo et al. (2009).
tices such as ‘rounding’ have also been considered (Binder, Moreover, several studies discuss how prediction per-
2017; Clements, 2011; Manski & Molinari, 2010). formance could depend on the parameters’ estimation. Ko-
Clements (2019) reviews macroeconomic survey ex- lasa and Rubaszek (2015b) suggest that updating DSGE
pectations. model parameters only once a year is enough to have
accurate and efficient predictions about the main macro
variables.
102 See, for example, Rosenblatt (1952), Shephard (1994), Kim,
Shephard, and Chib (1998), Diebold et al. (1998) and Berkowitz (2001). 103 This subsection was written by Alessia Paccagnini.
772
Table 2
Alternative competitors to DSGE models.
Competitor Reference
Hybrid Models US: Del Negro and Schorfheide (2004), Consolo et al. (2009)
Random Walk US: Gürkaynak, Kısacıkoğlu, and Rossi (2013), Euro Area: Warne et al. (2010), Smets, Warne,
and Wouters (2014)
Bayesian VAR US: Smets and Wouters (2007), Gürkaynak et al. (2013), Wolters (2015), Bekiros and
Paccagnini (2014), Bekiros and Paccagnini (2015a), Bekiros and Paccagnini (2015b), Euro Area:
(Warne et al., 2010)
Time-Varying VAR and Markov-Switching US: Bekiros, Cardani, Paccagnini, and Villa (2016), Euro Area: Bekiros and Paccagnini (2016)
Institutional Forecasts US: Edge and Gürkaynak (2010), Kolasa et al. (2012), Del Negro and Schorfheide (2013),
Wolters (2015)
3.3.3. Forecasting unemployment104 (2016) applies a flow approach to unemployment fore-

Unemployment has significant implications at both the casting and finds improvements, as does Smith (2011).
micro and macro levels, influencing individual living stan- One approach that does yield accurate forecasts is to
dards, health and well-being, as well as imposing di- use a measure of profitability as the explanatory variable,
rect costs on the economy. Given its importance, policy- assuming that unemployment will fall when hiring is
makers put unemployment at the heart of their economic profitable. Hendry (2001) proxies profitability (π ) by the
plans, and as such require accurate forecasts to feed into gap between the real interest rate (reflecting costs) and
economic policy decisions. Unemployment is described as the real growth rate (reflecting the demand side), such
a lagging indicator of the economy, with characteristics that the unemployment rate rises when the real interest
including business cycles and persistence. Despite this, rate exceeds the real growth rate, and vice versa:
forecasting the unemployment rate is difficult, because
the data are highly non-stationary with abrupt distribu- πt = (RL − ∆p − ∆y)t
tional shifts, but persistence within regimes. In this sec- where RL is the long-term interest rate, ∆p is a measure
tion we focus on methods used to forecast the aggregate of inflation and ∆y is a measure of output growth. This is
unemployment rate. then embedded within a dynamic equilibrium correction
Unemployment is the outcome of supply and demand model, using impulse indicator saturation (IIS: Hendry,
for labour, aggregated across all prospective workers, Johansen, & Santos, 2008b; Johansen & Nielsen, 2009)
with labour demand derived from demand for goods and and step indicator saturation (SIS: Castle et al., 2015c)
services. This implies a highly complex data generat- to capture outliers, breaks and regime shifts, as well as
ing process. Empirical forecasting models tend to sim- allowing for any non-linearities using Taylor expansions
plify this relationship, with two approaches dominating for the regressors. The resulting forecasts perform well
the literature. The first is based on the Phillips (1958) over the business cycle relative to alternative statistical
curve capturing a non-linear relationship between nom- models (also see Hendry, 2015, and Castle, Hendry, &
inal wage inflation and the unemployment rate, or the Martinez, 2020c).
relation between unemployment and output described
Forecasts from models of unemployment could be im-
as Okun’s 1962 Law. The second uses the time-series
proved with either better economic theories of aggregate
properties of the data to produce statistical forecasts,
unemployment,105 or more general empirical models that
such as univariate linear models (for example, ARIMA or
tackle stochastic trends, breaks, dynamics, non-linearities
unobserved component models; see Sections 2.3.4 and
and interdependence,106 or better still, both. The COVID-
2.3.6), multivariate linear models (for example, VARMA or
19 pandemic and subsequent lockdown policies highlight
CVAR; see Section 2.3.9), various threshold autoregressive
just how important forecasts of unemployment are (Cas-
models (see Section 2.3.13), Markov Switching models
tle, Doornik, & Hendry, 2021).
(see Section 2.3.12) and Artificial Neural Networks (see
Section 2.7.8).
3.3.4. Forecasting productivity107
The empirical literature is inconclusive as to the ‘best’
The growth of labour productivity, measured by the
forecasting models for unemployment, which varies by
percent change in output per hours worked, has varied
country, time period and forecast horizon. There is some
dramatically over the last 260 years. In the UK it ranged
evidence that non-linear statistical models tend to out-
from −5.8% at the onset of the 1920 Depression to just
perform within business cycle contractions or expansions,
but perform worse across business cycles (see, for ex- over 7% in 1971; see panel A in Fig. 6. Productivity growth
ample, Koop & Potter, 1999; Montgomery, Zarnowitz, is very volatile and has undergone large historical shifts
Tsay, & Tiao, 1998; Rothman, 1998), whereas Proietti with productivity growth averaging around 1% between
(2003) finds that linear models characterised by higher
persistence perform significantly better. Evidence of non- 105 There are many relevant theories based on microfoundations,
linearities is found by Johnes (1999), Milas and Rothman including search and matching, loss of skills, efficiency wages, and
(2008) and Peel and Speight (2000), and Gil-Alana (2001) insider-outsider models, see Layard, Nickell, and Jackman (1991) for
a summary.
finds evidence of long-memory. Barnichon and Garda 106 See Hendry and Doornik (2014) for an approach to jointly tackling
all of these issues.
104 This subsection was written by Jennifer L. Castle. 107 This subsection was written by Andrew B. Martinez.
773
Fig. 6. Productivity Growth (Output per total hours worked).

Source: Bank of England and Penn World Table Version 10.0.
1800–1950 followed by an increase in the average annual They show that are able to broadly replicate the OBR’s
growth to 3% between 1950–1975. Since the mid-1970’s forecasts using a quasi-transformed autoregressive model
productivity growth has gradually declined in many de- with one lag, a constant, and a trend. The estimated long-
veloped economies; see panel B of Fig. 6. In the decade run trend is just over 2% per year through 2007 Q4 which
since 2009, 2% annual productivity growth was an upper is consistent with the OBR’s assumptions about the long-
bound for most G7 countries. run growth rate of productivity (OBR, 2019). However, it
The most common approach for forecasting produc- is possible to dramatically improve upon OBR’s forecasts
tivity is to estimate the trend growth in productivity in real-time by allowing for the long-term trend forecast
using aggregate data. For example, Gordon (2003) con- to adjust based on more recent historical patterns. By
siders three separate approaches for calculating trend taking a local average of the last four years of growth
labor productivity in the United States based on (i) av- rates, Martinez et al. (2021) generate productivity fore-
erage historical growth rates outside of the business cy- casts whose RMSE is on average more than 75% smaller
cle, (ii) filtering the data using the HP filter (Hodrick than OBR’s forecasts extending five-years-ahead and is
& Prescott, 1997), and (iii) filtering the data using the 84% smaller at the longest forecast horizon.
Kalman filter (see Kalman, 1960). The Office for Budget
Responsibility (OBR) in the UK and the Congressional Bud- 3.3.5. Fiscal forecasting for government budget surveillance109
get Office (CBO) in the US follow similar approaches for Recent economic recessions have led to a renewed
generating its forecasts of productivity based on average interest in fiscal forecasting, mainly for deficit and debt
historical growth rates as well as judgments about factors surveillance. This was certainly true in the case of the
that may cause productivity to deviate from its histor- 2008 recession, and looks to become even more impor-
ical trend in the short-term.108 Alternative approaches
tant in the current economic crisis brought on by the
include forecasting aggregate productivity using disaggre-
COVID-19 pandemic. This is particularly important in Eu-
gated firm-level data (see Bartelsman, Kurz, & Wolf, 2011;
rope, where countries are subject to strong fiscal mon-
Bartelsman & Wolf, 2014, and Section 2.10.1) and using
itoring mechanisms. Two main themes can be detected
time-series models (see Žmuk, Dumičić, & Palić, 2018, and
in the fiscal forecasting literature (Leal, Pérez, Tujula, &
Section 2.3.4).
Vidal, 2008). First, investigate the properties of forecasts
In the last few decades there have been several at-
in terms of bias, efficiency and accuracy. Second, check
tempts to test for time-varying trends in productivity and
the adequacy of forecasting procedures.
to allow for them. However, the focus of these approaches
The first topic has its own interest for long, mainly
has been primarily on the United States (Hansen, 2001;
restricted to international institutions (Artis & Marcellino,
Roberts, 2001), which saw a sharp rise in productivity
2001). Part of the literature, however, argue that fiscal
growth in the 1990’s that was not mirrored in other
forecasts are politically biased, mainly because there is
countries (Basu, Fernald, Oulton, & Srinivasan, 2003). Test
for shifts in productivity growth rates in other advanced usually no clear distinction between political targets and
economies did not find evidence of a changes in pro- rigorous forecasts (Frankel & Schreger, 2013; Strauch,
ductivity growth until well after the financial crisis in Hallerberg, & Hagen, 2004). In this sense, the availabil-
2007 (Benati, 2007; Glocker & Wegmüller, 2018; Turner ity of forecasts from independent sources is of great
& Boulhol, 2011). value (Jonung & Larch, 2006). But it is not as easy as saying
A more recent approach by Martinez et al. (2021) al- that independent forecasters would improve forecasts
lows for a time-varying long-run trend in UK productivity. due to the absence of political bias, because forecast-
ing accuracy is compromised by complexities of data,
108 See https://obr.uk/forecasts-in-depth/the-economy-forecast/
potential-output-and-the-output-gap. (Accessed: 2020-09-05) 109 This subsection was written by Diego J. Pedregal.
774
country-specific factors, outliers, changes in the definition asset pricing and risk management. Indeed, the value of
of fiscal variables, etc. Very often some of these issues interest rate–sensitive securities depends on the value of
are known by the staff of organisations in charge of the riskless rate. Besides, the short interest rate is a fun-
making the official statistics and forecasts long before damental ingredient in the formulation and transmission
the general public, and some information never leaves of the monetary policy (see, for example, Section 2.3.15).
such institutions. So this insider information is actually a However, many popular models of the short rate (for
valuable asset to improve forecasting accuracy (Leal et al., instance, continuous time, diffusion models) fail to deliver
2008). accurate out-of-sample forecasts. Their poor predictive
As for the second issue, namely the accuracy of fore- performance may depend on the fact that the stochastic
casting methods, the literature can be divided into two behaviour of short interest rates may be time-varying (for
parts, one based on macroeconomic models with specific instance, it may depend on the business cycle and on the
fiscal modules that allows to analyse the effects of fis- stance of monetary policy).
cal policy on macro variables and vice versa (see Favero Notably, the presence of nonlinearities in the condi-
and Marcellino (2005) and references therein), and the tional mean and variance of the short-term yield influ-
other based on pure forecasting methods and compar- ences the behaviour of the entire term structure of spot
isons among them. This last stream of research basically rates implicit in riskless bond prices. For instance, the
resembles closely what is seen in other forecasting areas: level of the short-term rate directly affects the slope of
(i) there is no single method outperforming the rest gen- the yield curve. More generally, nonlinear rate dynamics
erally, (ii) judgmental forecasting is especially important imply a nonlinear equilibrium relationship between short
due to data problems (see Section 2.11), and (iii) combi- and long-term yields. Accordingly, recent research has
nation of methods tends to outperform individual ones, reported that dynamic econometric models with regime
see Leal et al. (2008) and Section 2.6. shifts in parameters, such as Markov switching (MS; see
Part of the recent literature focused on the genera- Section 2.3.12) and threshold models (see Section 2.3.13),
tion of very short-term public finance monitoring systems are useful at forecasting rates.
using models that combine annual information with intra- The usefulness of MS VAR models with term structure
annual fiscal data (Pedregal & Pérez, 2010) by time aggre- data had been established since Hamilton (1988) and Gar-
gation techniques (see Section 2.10.2), often set up in a cia and Perron (1996): single-state, VARMA models are
SS framework (see Section 2.3.6). The idea is to produce overwhelmingly rejected in favour of multi-state models.
global annual end-of-year forecasts of budgetary variables Subsequently, a literature has emerged that has docu-
based on the most frequently available fiscal indicators, mented that MS models are required to successfully fore-
so that changes throughout the year in the indicators can cast the yield curve. Lanne and Saikkonen (2003) showed
be used as early warnings to infer the changes in the an- that a mixture of autoregressions with two
nual forecasts and deviations from fiscal targets (Pedregal, regimes improves the predictions of US T-bill rates. Ang
Pérez, & Sánchez, 2014). and Bekaert (2002) found support for MS dynamics in the
The level of disaggregation of the indicator variables short-term rates for the US, the UK, and Germany. Cai
are established according to the information available (1994) developed a MS ARCH model to examine volatil-
and the particular objectives. The simplest options are ity persistence, reflecting a concern that it may be in-
the accrual National Accounts annual or quarterly fiscal flated by regimes. Gray (1996) generalised this attempt
balances running on their cash monthly counterparts. A to MS GARCH and reported improvements in pseudo out-
somewhat more complex version is the previous one with of-sample predictions. Further advances in the methods
all the variables broken down into revenues and expendi- and applications of MS GARCH are in Haas, Mittnik, and
tures. Other disaggregation schemes have been applied, Paolella (2004) and Smith (2002). A number of papers
namely by region, by administrative level (regional, mu- have also investigated the presence of regimes in the typ-
nicipal, social security, etc.), or by items within revenue ical factors (level, slope, and convexity) that characterise
and/or expenditure (VAT, income taxes, etc. Asimakopou- the no-arbitrage dynamics of the term structure, showing
los, Paredes, & Warmedinger, 2020; Paredes, Pedregal, & the predictive benefits of incorporating MS (see, for ex-
Pérez, 2014). ample, Guidolin & Pedio, 2019; Hevia, Gonzalez-Rozada,
Unfortunately, what is missing is a comprehensive and Sola, & Spagnolo, 2015).
transparent forecasting system, independent of Member Alternatively, a few studies have tried to capture the
States, capable of producing consistent forecasts over time time-varying, nonlinear dynamics of interest rates us-
and across countries. This is certainly a challenge that no ing threshold models. As discussed by Pai and Pedersen
one has yet dared to take up. (1999), threshold models have an advantage compared to
MS ones: the regimes are not determined by an unob-
3.3.6. Interest rate prediction110 served latent variable, thus fostering interpretability. In
The (spot) rate on a (riskless) bond represents the ex- most of the applications to interest rates, the regimes are
ante return (yield) to maturity which equates its market determined by the lagged level of the short rate itself,
price to a theoretical valuation. Modelling and predicting in a self-exciting fashion. For instance, Pfann, Schotman,
default-free, short-term interest rates are crucial tasks in and Tschernig (1996) explored nonlinear dynamics of the
US short-term interest rate using a (self-exciting) thresh-
110 This subsection was written by Massimo Guidolin & Manuela old autoregressive model augmented by conditional het-
Pedio. eroskedasticity (namely, a TAR-GARCH model) and found
775
strong evidence of the presence of two regimes. More re- the world.112 At the same time, a substantial empirical
cently, also Gospodinov (2005) used a TAR-GARCH to pre- literature has developed that deals with predicting future
dict the short-term rate and showed that this model can house price movements (for a comprehensive survey see
capture some well-documented features of the data, such Ghysels, Plazzi, Valkanov, & Torous, 2013). Although this
as high persistence and conditional heteroskedasticity. literature concentrates almost entirely on the US (see, for
Another advantage of nonlinear models is that they example, Bork & Møller, 2015; Rapach & Strauss, 2009),
can reproduce the empirical puzzles that plague the ex- there are many other countries, such as the UK, where
pectations hypothesis of interest rates (EH), according to house price forecastability is of prime importance. Sim-
which it is a weighted average of short-term rates to drive ilarly to the US, in the UK, housing activities account
longer-term rates (see, for example, Bansal, Tauchen, & for a large fraction of GDP and of households’ expendi-
Zhou, 2004; Dai, Singleton, & Yang, 2007). For instance, tures; real estate property comprises a significant com-
while Bekaert, Hodrick, and Marshall (2001) show single- ponent of private wealth and mortgage debt constitutes a
state VARs cannot generate distributions consistent with main liability of households (Office for National Statistics,
the EH, Guidolin and Timmermann (2009) find that the 2019).
optimal combinations of lagged short and forward rates The appropriate forecasting model has to reflect the
depend on regimes so that the EH holds only in some dynamics of the specific real estate market and take into
states. account its particular characteristics. In the UK, for in-
As widely documented (see, for instance, Guidolin stance, there is a substantial empirical literature that doc-
& Thornton, 2018), the predictable component in mean uments the existence of strong spatial linkages between
rates is hardly significant. As a result, the random walk regional markets, whereby the house price shocks em-
remains a hard benchmark to outperform as far as the anating from southern regions of the country, and in
prediction of the mean is concerned. However, density particular Greater London, have a tendency to spread out
forecasts reflect all moments and the models that capture and affect neighbouring regions with a time lag (see, for
the dynamics of higher-order moments tend to perform example, Antonakakis, Chatziantoniou, Floros, & Gabauer,
best. MS models appear at the forefront of a class of 2018; Cook & Thomas, 2003; Holly, Pesaran, & Yamagata,
non-linear models that produce accurate density predic- 2010, inter alia); see also Section 2.3.10 on forecasting
tions (see, for example, Hong, Li, & Zhao, 2004; Maheu & functional data.
Yang, 2016). Alternatively, Pfann et al. (1996) and more Recent evidence also suggests that the relationship be-
recently Dellaportas, Denison, and Holmes (2007) esti- tween real estate valuations and conditioning macro and
mated TAR models to also forecast conditional higher financial variables displayed a complex of time-varying
order moments and all report reasonable accuracy. patterns over the previous decades (Aizenman & Jinjarak,
Finally, a literature has strived to fit rates not only un- 2013). Hence, predictive methods that do not allow for
der the physical measure, i.e., in time series, but to predict time-variation in both predictors and their marginal ef-
rates when MS enters the pricing kernel, the fundamental fects may not be able to capture the complex house price
pricing operator. A few papers have assumed that regimes dynamics in the UK (see Yusupova, Pavlidis, & Pavlidis,
represent a new risk factor (see, for instance, Dai & Single- 2019, for a comparison of forecasting accuracy of a battery
ton, 2003). This literature reports that MS models lead to a of static and dynamic econometric methods).
range of shapes for nominal and real term structures (see, An important recent trend is to attempt to incorpo-
for instance, Veronesi & Yared, 1999). Often the model rate information from novel data sources (such as news-
specifications that are not rejected by formal tests include paper articles, social media, etc.) in forecasting models
regimes (Ang, Bekaert, & Wei, 2008; Bansal & Zhou, 2002). as a measure of expectations and perceptions of eco-
To conclude, it is worthwhile noting that, while thresh- nomic agents (see also Section 2.9.3). It has been shown
old models are more interpretable, MS remain a more that changes in uncertainty about house prices impact
popular alternative for the prediction of interest rates. on housing investment and real estate construction de-
This is mainly due to the fact that statistical inference cisions (Banks, Blundell, Oldfield, & Smith, 2015; Cun-
for threshold regime switching models poses some chal- ningham, 2006; Oh & Yoon, 2020), and thus incorporat-
lenges, because the likelihood function is discontinuous ing a measure of uncertainty in the forecasting model
with respect to the threshold parameters. can improve the forecastability of real estate prices. For
instance in the UK, the House Price Uncertainty (HPU)
3.3.7. House price forecasting111 index (Yusupova, Pavlidis, Paya, & Peel, 2020), constructed
The boom and bust in housing markets in the early using the methodology outlined in Baker, Bloom, and
and mid 2000s and its decisive role in the Great Reces- Davis (2016),113 was found to be important in predicting
sion has generated a vast interest in the dynamics of
house prices and emphasised the importance of accu- 112 For instance, the International Monetary Fund recently established
rately forecasting property price movements during tur-
the Global Housing Watch, the Globalisation and Monetary Policy
bulent times. International organisations, central banks Institute of the Federal Reserve Bank of Dallas initiated a project on
and research institutes have become increasingly engaged monitoring international property price dynamics, and the UK Housing
in monitoring the property price developments across Observatory initiated a similar project for the UK national and regional
housing markets.
113 For a comparison of alternative text-based measures of economic
111 This subsection was written by Alisa Yusupova. uncertainty see Kalamara, Turrell, Redl, Kapetanios, and Kapadia (2020).
776
property price inflation ahead of the house price col- a simple method assuming gradual adjustment of the ex-
lapse of the third quarter of 2008 and during the bust change rate towards the level implied by the Purchasing
phase (Yusupova et al., 2019). Along with capturing the Power Parity (PPP) performs well over shorter as well as
two recent recessions (in the early 1990s and middle longer horizon. This result is consistent with the results of
2000s) this index also reflects the uncertainly related to Ca’ Zorzi et al. (2017) and Eichenbaum, Johannsen, and
the EU Referendum, Brexit negotiations and COVID-19 Rebelo (2017), who showed that exchange rates are pre-
pandemic. dictable within a general equilibrium DSGE framework
(see Section 2.3.15), which encompasses an adjustment of
3.3.8. Exchange rate forecasting114 the exchange rate to a PPP equilibrium. Finally, Ca’ Zorzi,
Exchange rates have long fascinated and puzzled re- Cap, Mijakovic, and Rubaszek (2020) discuss how extend-
searchers in international finance. The reason is that fol- ing the PPP framework for other fundamentals within
lowing the seminal paper of Meese and Rogoff (1983), the the BEER framework is not helping in exchange rate
common wisdom is that macroeconomic models cannot forecasting. Overall, at the current juncture it might be
outperform the random walk in exchange rate forecast- claimed that ‘‘exchange rate disconnect puzzle’’ is still
ing (see Rossi, 2013, for a survey). This view is difficult puzzling, with some evidence that methods based on PPP
to reconcile with the strong belief that exchange rates and controlling the estimation forecast error can deliver
are driven by fundamentals, such as relative productivity, more accurate forecast than the random walk benchmark.
external imbalances, terms of trade, fiscal policy or in- A way forward to account for macroeconomic variables
terest rate disparity (Couharde, Delatte, Grekou, Mignon, in exchange rate forecasting could be to use variable
& Morvillier, 2018; Lee, Milesi-Ferretti, & Ricci, 2013; selection methods that allow to control for the estimation
MacDonald, 1998). These two contradicting assertions by error (see Section 2.5.3).
the academic literature is referred to as ‘‘exchange rate
disconnect puzzle’’. 3.3.9. Financial time series forecasting with range-based
The literature provides several explanations for this volatility models115
The range-based (RB) volatility models is a general
puzzle. First, it can be related to the forecast estimation
term for the models constructed with high and low prices,
error (see Section 2.5.2). The studies in which models
and most often with their difference i.e., the price range.
are estimated with a large panels of data (Engel, Mark,
A short review and classification of such models is con-
& West, 2008; Ince, 2014; Mark & Sul, 2001), long time
tained in Section 2.3.14. From practical point of view, it
series (Lothian & Taylor, 1996) or calibrated (Ca’ Zorzi
is important that low and high prices are almost always
& Rubaszek, 2020) deliver positive results on exchange
available with daily closing prices for financial series.
rate forecastability. Second, there is ample evidence that
The price range (or its logarithm) is a significantly more
the adjustment of exchange rates to equilibrium is non-
efficient estimator of volatility than the estimator based
linear (Curran & Velic, 2019; Taylor & Peel, 2000), which
on closing prices (Alizadeh et al., 2002). Similarly the
might diminish the out-of-sample performance of macroe-
co-range (the covariance based on price ranges) is a sig-
conomic models (Kilian & Taylor, 2003; Lopez-Suarez &
nificantly more efficient estimator of the covariance of re-
Rodriguez-Lopez, 2011). Third, few economists argue that
turns than the estimator based on closing prices (Brunetti
the role of macroeconomic fundamentals may be varying
& Lildholdt, 2002). For these reasons models based on the
over time and this should be accounted for in a forecasting
price range and the co-range better describe variances
setting (Beckmann & Schussler, 2016; Byrne, Korobilis, &
and covariances of financial returns than the ones based
Ribeiro, 2016).
on closing prices.
The dominant part of the exchange rate forecasting Forecasts of volatility from simple models like moving
literature investigates which macroeconomic model per- average, EWMA, AR, ARMA based on the RB variance
forms best out-of-sample. The initial studies explored the estimators are more accurate than the forecasts from
role of monetary fundamentals to find that these models the same models based on squared returns of closing
deliver inaccurate short-term and not so bad long-term prices (Rajvanshi, 2015; Vipul & Jacob, 2007). Forecasts
predictions in comparison to the random walk (Mark, of volatility from the AR model based on the Parkinson
1995; Meese & Rogoff, 1983). In a comprehensive study estimator are more precise even than the forecasts from
from mid-2000s, Cheung, Chinn, and Pascual (2005) the standard GARCH models (see Section 2.3.11) based on
showed that neither monetary, uncovered interest parity closing prices (Li & Hong, 2011).
(UIP) nor behavioural equilibrium exchange rate (BEER) In plenty of studies it was shown that forecasts of
model are able to outperform the no-change forecast. volatility of financial returns from the univariate RB mod-
A step forward was made by Molodtsova and Papell els are more accurate than the forecasts from standard
(2009), who proposed a model combining the UIP and GARCH models based on closing prices (see, for exam-
Taylor rule equations and showed that it delivers comple, Mapa, 2003 for the GARCH-PARK-R model;
petitive exchange rate forecasts. This result, however, Chou, 2005 for the CARR model; Fiszeder, 2005 for the
has not been confirmed by more recent studies (Cheung, GARCH-TR model; Brandt and Jones, 2006 for the RE-
Chinn, Pascual, & Zhang, 2019; Engel, Lee, Liu, Liu, & Wu, GARCH model; Chen et al., 2008 for the TARR model; Lin
2019). In turn, Ca’ Zorzi and Rubaszek (2020) argue that et al., 2012 for the STARR model; Fiszeder and Perczak, 2016
114 This subsection was written by Michał Rubaszek. 115 This subsection was written by Piotr Fiszeder.
777
for the GARCH model estimated with low, high and clos- by a real-valued latent variable, which corresponds to
ing prices during crisis periods; Molnár, 2016 for the the Fisher transformation of Kendall’s τ . Li and Kang
RGARCH model). (2018) use a covariate-dependent copula framework to
The use of daily low and high prices in the multi- forecast the time varying dependence that improves both
variate volatility models leads to more accurate forecasts the probabilistic forecasting performance and the fore-
of covariance or covariance matrix of financial returns casting interpretability. Liquidity risk is another focus in
than the forecasts from the models based on closing finance. Weiß and Supper (2013) forecast three types of
prices (see, for example, Chou et al., 2009 for the RB DCC liquidity-adjusted intraday Value-at-Risk (L-IVaR) with a
model; Harris and Yilmaz, 2010 for the hybrid EWMA vine copula structure. The liquidity-adjusted intraday VaR
model; Fiszeder, 2018 for the BEKK-HL model; Fiszeder is based on simulated portfolio values, and the results are
and Fałdziński, 2019 for the co-range DCC model; Fiszeder compared with the realised portfolio profits and losses.
et al., 2019 for the DCC-RGARCH model). In macroeconomic forecasting, most existing reduced-
The RB models were used in many financial applica- form models for multivariate time series produce sym-
tions. They lead for example to more precise forecasts metric forecast densities. Gaussian copulas with skew
of value-at-risk measures in comparison to the applica- Student’s-t margins depict asymmetries in the predic-
tion of only closing prices (see, for example, Chen, Ger- tive distributions of GDP growth and inflation (Smith
lach, Hwang, and McAleer, 2012 for the threshold CAViaR & Vahey, 2016). Real-time macroeconomic variables are
model; Asai and Brugal, 2013 for the HVAR model; Fiszeder forecasted with heteroscedastic inversion copulas (Smith
et al., 2019 for the DCC-RGARCH model; Meng and Tay- & Maneesoonthorn, 2018) that allow for asymmetry in the
lor, 2020 for scoring functions). The application of the density forecasts, and both serial and cross-sectional de-
multivariate RB models provides also the increase in the pendence could be captured by the copula function (Loaiza-
efficiency of hedging strategies (see, for example, Chou Maya & Smith, 2020).
et al., 2009 for the RB DCC model; Harris and Yilmaz, 2010 Copulas are also widely used to detect and forecast
for the hybrid EWMA model; Su and Wu, 2014 for the RB- default correlation, which is a random variable called
MS-DCC model). Moreover, the RB volatility models have time-until-default to denote the survival time of each de-
more significant economic value than the return-based faultable entity or financial instrument (Li, 2000). Then
ones in the portfolio construction (Chou and Liu, 2010 for copulas are used in modelling the dependent defaults (Li,
the RB DCC model; Wu and Liang, 2011 for the RB-copula 2000), forecasting credit risk (Bielecki & Rutkowski, 2013),
model). Some studies show that based on the forecasts and credit derivatives market forecasting (Schönbucher,
from the volatility models with low and high prices it is 2003). A much large volume of literature is available
possible to construct profitable investment strategies (He, for this specific area. See the aforementioned references
Kwok, and Wan, 2010 for the VECM model; Kumar, 2015 therein. For particular applications in default risk and
for the CARRS model). credit default swap (CDS) forecasting, see Li and He (2019)
and Oh and Patton (2018) respectively.
3.3.10. Copula forecasting with multivariate dependent fi- In energy economics, Aloui, Hammoudeh, and Nguyen
nancial times series116 (2013) employ the time-varying copula approach, where
In this section, we focus on the practical advances on the marginal models are from ARMA(p, q)–GARCH(1,1)
jointly forecasting multivariate financial time series with to investigate the conditional dependence between the
copulas. In the copula framework (see Section 2.4.3), be- Brent crude oil price and stock markets in the Central and
cause marginal models and copula models are separable, Eastern European transition economies. Bessa, Miranda,
point forecasts are straightforward with marginal models, Botterud, Zhou, and Wang (2012) propose a time-adaptive
but dependence information is ignored. A joint probabilis- quantile-copula where the copula density is estimated
tic forecast with copulas involves both estimations of the with a kernel density forecast method. The method is
copula distribution and marginal models. applied to wind power probabilistic forecasting (see also
In financial time series, an emerging interest is to Section 3.4.6) and shows its advantages for both system
model and forecast the asymmetric dependence. A typical operators and wind power producers. Vine copula models
asymmetric dependence phenomenon is that two stock are also used to forecast wind power farms’ uncertainty
returns exhibit greater correlation during market down- in power system operation scheduling. Wang, Wang, Liu,
turns than market upturns. Patton (2006) employs the Wang, and Hou (2017) shows vine copulas have advan-
asymmetric dependence between exchange rates with a tages of providing reliable and sharp forecast intervals,
time-varying copula construction with AR and GARCH especially in the case with limited observations available.
margins. A similar study for measuring financial conta-
gion with copulas allows the parameters of the copula to 3.3.11. Financial forecasting with neural networks117
change with the states of the variance to identify shifts Neural Networks (NNs; see Section 2.7.8) are capable
in the dependence structure in times of crisis (Rodriguez, of successfully modelling non-stationary and non-linear
2007). series. This property has made them one of the most
In stock forecasting, Almeida and Czado (2012) employ popular (if not the most) non-linear specification used by
a stochastic copula autoregressive model to model DJI practitioners and academics in Finance. For example, 89%
and Nasdaq, and the dependence at the time is modelled of European banks use NNs to their operations (European
116 This subsection was written by Feng Li. 117 This subsection was written by Georgios Sermpinis.
778
Banking Federation, 2019) while 25.4% of the NNs appli- 3.3.12. Forecasting returns to investment style119
cations in total is in Finance (Wong, Bodnovich, & Selvi, Investment style or factor portfolios are constructed
1995). from constituent securities on the basis of a variety of a-
The first applications of NNs in Finance and currently priori observable characteristics, thought to affect future
the most widespread, is in financial trading. In the mid- returns. For example a ‘Momentum’ portfolio might be
80s when computational power became cheaper and more constructed with positive (‘long’) exposures to stocks with
accessible, hedge fund managers started to experiment positive trailing 12-month returns, and negative (‘short’)
with NNs in trading. Their initial success led to even more exposure to stocks with negative trailing 12-month re-
practitioners to apply NNs and nowadays 67% of hedge turns (for full background and context, see, for example
fund managers use NNs to generate trading ideas (Bar- Bernstein, 1995; Haugen, 2010).120 Explanations as to
clayHedge, 2018). A broad measure of the success of NNs why such characteristics seem to predict returns fall in to
in financial trading is provided by the Eurekahedge AI two main camps: firstly that the returns represent a risk
Hedge Fund Index118 where it is noteworthy the 13.02% premium, earned by the investor in return for taking on
annualised return of the selected AI hedge funds over the some kind of (undiversifiable) risk, and secondly that such
last 10 years. returns are the result of behavioural biases on the part of
In academia, financial trading with NNs is the focus investors. In practice, both explanations are likely to drive
style returns to a greater or lesser extent. Several such
of numerous papers. Notable applications of NNs in trad-
strategies have generated reasonably consistent positive
ing financial series were provided by Dunis, Laws, and
risk-adjusted returns over many decades, but as with
Sermpinis (2010), Kaastra and Boyd (1996), Panda and
many financial return series, return volatility is large rel-
Narasimhan (2007), Tenti (1996), and Zhang and Ming
ative to the mean, and there can be periods of months or
(2008). The aim of these studies is to forecast the sign
even years when returns deviate significantly from their
or the return of financial trading series and based on
long-run averages. The idea of timing exposure to styles
these forecasts to generate profitable trading strategies.
is therefore at least superficially attractive, although the
These studies are closely related to the ones presented
feasibility of doing so is a matter of some debate (Arnott,
in Section 3.3.13 but the focus is now in profitability. Beck, Kalesnik, & West, 2016; Asness, 2016; Bender, Sun,
The second major field of applications of NNs in Fi- Thomas, & Zdorovtsov, 2018). Overconfidence in timing
nance is in derivatives pricing and financial risk man- ability has a direct cost in terms of trading frictions and
agement. The growth of the financial industry and the opportunity cost in terms of potential expected returns
provided financial services have made NNs and other and diversification forgone.
machine learning algorithms a necessity for tasks such A number of authors write on the general topic of style
as fraud detection, information extraction and credit risk timing (recent examples include Dichtl, Drobetz, Lohre,
assessment (Buchanan, 2019). In derivatives pricing, NNs Rother, & Vosskamp, 2019; Hodges, Hogan, Peterson, &
try to fill the limitations of the Black–Scholes model Ang, 2017), and several forecasting methodologies have
and are being used in options pricing and hedging. In been suggested, falling in to three main camps:
academia notable applications of NNs in risk management
are provided by Liu (2005) and Locarek-Junge and Prinzler 1. Serial Correlation: Perhaps the most promising ap-
(1998) and in derivatives by Bennell and Sutcliffe (2004) proach is exploiting serial correlation in style re-
and Psaradellis and Sermpinis (2016). turns. Babu, Levine, Ooi, Pedersen, and Stame-
As discussed before, financial series due to their non- los (2020) and Tarun and Bryan (2019) outline
linear nature and their wide applications in practice seems two such approaches and Ehsani and Linnainmaa
the perfect forecasting data set for researchers that want (2020) explore the relationship between momen-
tum in factor portfolios and momentum in un-
to test their NN topologies. As a result, there are thou-
derlying stock returns. As with valuation spreads
sands of forecasting papers in the field of NNs in financial
mentioned below, there is a risk that using momen-
forecasting. However, caution is needed in interpreta-
tum signals to time exposure to momentum factor
tion of their results. NNs are sensitive to the choice of
portfolios risks unwittingly compounding exposure.
their hyperparameters. For a simple MLP, a practitioner
A related strand of research relates (own) factor
needs to set (among others) the number and type of
volatility to future returns, in particular for mo-
inputs, the number of hidden nodes, the momentum, the
mentum factors (Barroso, 2015; Daniel & Moskowitz,
learning rate, the number of epochs and the batch size.
2016).
This complexity in NN modelling leads inadvertently to
2. Valuation Spreads: Using value signals (aggregated
the data snooping bias (see also Section 2.12.6). In other
from individual stock value exposures) to time ex-
words, a researcher that experiments long enough with posure to various fundamental-based strategies is
the parameters of a NN topology can have excellent in- a popular and intuitively appealing approach (As-
sample and out-of-sample results for a series. However, ness, 2016); however evidence of value added from
this does not mean that the results of his NN can be doing so is mixed, and the technique seems to
generalised. This issue has led the related literature to compound risk exposure to value factors.
be stained by studies cannot be extended in different
samples. 119 This subsection was written by Ross Hollyman.
120 The website of Kenneth French is an excellent source of data on
118 https://www.eurekahedge.com/Indices/IndexView/Eurekahedge/ investment style factor data and research. http://mba.tuck.dartmouth.
683/Eurekahedge-AI-Hedge-fund-Index (Accessed: 2020-09-01) edu/pages/faculty/ken.french/data_library.html
779
3. Economic & Financial Conditions: Polk, Haghbin, show that forecast combination can significantly improve
and de Longis (2020) explore how economic and out-of-sample market return forecasts. They first con-
financial conditions affect style returns (an idea struct return forecasts via individual univariate predictive
that dates back at least to Bernstein (1995) and regressions based on numerous popular predictors from
references therein). the literature (Goyal & Welch, 2008). They then compute
a simple combination forecast by taking the average of
Style returns exhibit distinctly non-normal distribu- the individual forecasts. Rapach et al. (2010) demonstrate
tions. On a univariate basis, most styles display returns that forecast combination exerts a strong shrinkage effect,
which are highly negatively skewed and demonstrate thereby helping to guard against overfitting.
significant kurtosis. The long-run low correlation between An emerging literature uses machine-learning tech-
investment styles is often put forward as a benefit of niques to construct forecasts of stock returns based on
style-based strategies, but more careful analysis reveals large sets of predictors. In an investigation of lead–lag
that non-normality extends to the co-movements of in- relationships among developed equity markets, Rapach,
vestment style returns; factors exhibit significant tail de- Strauss, and Zhou (2013) appear to be the first to em-
pendence. Christoffersen and Langlois (2013) explores ploy machine-learning tools to predict market returns.
this issue, also giving details of the skew and kurtosis They use the elastic net (ENet: Zou & Hastie, 2005), a
of weekly style returns. These features of the data mean generalisation of the popular least absolute shrinkage and
that focusing solely on forecasting the mean may not be selection operator (LASSO: Tibshirani, 1996). The LASSO
sufficient, and building distributional forecasts becomes and ENet employ penalised regression to guard against
important for proper risk management. Jondeau (2007) overfitting in high-dimensional settings by shrinking the
writes extensively on modelling non-gaussian distribu- parameter estimates toward zero. Chinco, Clark-Joseph,
tions. and Ye (2019) use the LASSO to forecast high-frequency
(one-minute-ahead) individual stock returns and report
3.3.13. Forecasting stock returns121 improvements in out-of-sample fit, while Rapach et al.
Theory and intuition suggest a plethora of potentially (2019) use the LASSO to improve monthly forecasts of
relevant predictors of stock returns. Financial statement industry returns.
data (e.g., Chan & Genovese, 2001; Yan & Zheng, 2017) Incorporating insights from Diebold and Shin (2019), Han,
provide a wealth of information, and variables relating to He, Rapach, and Zhou (2021) use the LASSO to form com-
liquidity, price trends, and sentiment, among numerous bination forecasts of cross-sectional stock returns based
other concepts, have been used extensively by academics on a large number of firm characteristics from the cross-
and practitioners alike to predict stock returns. The era sectional literature (e.g., Harvey, Liu, & Zhu, 2016; Hou,
of big data further increases the data available for fore- Xue, & Zhang, 2020; McLean & Pontiff, 2016), extend-
casting returns. When forecasting with large numbers ing the conventional OLS approach of Green, Hand, and
of predictors, conventional ordinary least squares (OLS) Zhang (2017), Haugen and Baker (1996), and Lewellen
estimation is highly susceptible to overfitting, which is ex- (2015). Dong, Li, Rapach, and Zhou (2021) and Rapach
acerbated by the substantial noise in stock return data (re- and Zhou (2020) use the ENet to compute combination
flecting the intrinsically large unpredictable component in forecasts of the market return based on popular predictors
returns); see Section 2.7.11. from the time-series literature and numerous anomalies
Over the last decade or so, researchers have explored from the cross-sectional literature, respectively. Forecast-
methods for forecasting returns with large numbers of ing individual stock returns on the basis of firm char-
predictors. Principal component regression extracts the acteristics in a panel framework, Freyberger, Neuhierl,
first few principal components (or factors) from the set of and Weber (2020) and Gu, Kelly, and Xiu (2020) employ
predictors; the factors then serve as predictors in a low- machine-learning techniques – such as the nonparametric
dimensional predictive regression, which is estimated via additive LASSO (Huang, Horowitz, & Wei, 2010), random
OLS (see Section 2.7.1). Intuitively, the factors combine forests (Breiman, 2001), and artificial neural networks –
the information in the individual predictors to reduce that allow for nonlinear predictive relationships.
the dimension of the regression, which helps to guard
against overfitting. Ludvigson and Ng (2007) find that a 3.3.14. Forecasting crashes in stock markets122
few factors extracted from hundreds of macroeconomic Time series data on financial asset returns have spe-
and financial variables improve out-of-sample forecasts of cial features. Returns themselves are hard to forecast,
the US market return. Kelly and Pruitt (2013) and Huang, while it seems that volatility of returns can be predicted.
Jiang, Tu and Zhou (2015) use partial least squares (Wold, Empirical distributions of asset returns show occasional
1966) to construct target-relevant factors from a cross clusters of large positive and large negative returns. Large
section of valuation ratios and a variety of sentiment mea- negative returns, that is, crashes seem to occur more
sures, respectively, to improve market return forecasts. frequently than large positive returns. Forecasting upcom-
Since Bates and Granger (1969), it has been known that ing increases or decreases in volatility can be achieved
combinations of individual forecasts often perform better by using variants of the Autoregressive Conditional Het-
than the individual forecasts themselves (Timmermann, eroskedasticity (ARCH) model (Bollerslev, 1986; Engle,
2006, and Section 2.6.1). Rapach, Strauss, and Zhou (2010) 1982, and Section 2.3.11) or realized volatility models
121 This subsection was written by David E. Rapach. 122 This subsection was written by Philip Hans Franses.
780
(Taylor, 1986a). These models take (functions of) past energy management and planning by supporting a large
volatility and past returns as volatility predictors, al- variety of optimisation procedures.
though also other explanatory variables can be incorpo- The main challenge in electricity consumption fore-
rated in the regression. casting is that building energy systems are complex in
An important challenge that remains is to predict nature, with their behaviour depending on various factors
crashes. Sornette (2003) summarises potential causes for related to the type (e.g., residential, office, entertainment,
crashes and these are computer trading, increased trading business, and industrial) and the end-uses (e.g., heating,
in derivatives, illiquidity, trade and budget deficits, and cooling, hot water, and lighting) of the building, its con-
especially, herding behaviour of investors. Yet, forecast- struction, its occupancy, the occupants’ behaviour and
ing the exact timing of crashes may seem impossible, schedule, the efficiency of the installed equipment, and
but on the other hand, it may be possible to forecast the weather conditions (Zhao & Magoulès, 2012). Special
the probability that a crash may occur within a fore- events, holidays, and calendar effects can also affect the
seeable future. Given the herding behaviour, any model behaviour of the systems and further complicate the con-
to use for prediction should include some self-exciting sumption patterns, especially when forecasting at hourly
behaviour. For that purpose, Aït-Sahalia, Cacho-Diaz, and or daily level (see Section 2.3.5). As a result, producing
Laeven (2015) propose mutually exciting jump processes, accurate forecasts typically requires developing tailored,
where jumps can excite new jumps, also across assets or building-specific methods.
markets (see also Chavez-Demoulin, Davison, & McNeil, To deal with this task, the literature focuses on three
2005). Another successful approach is the Autoregressive main classes of forecasting methods, namely engineering,
Conditional Duration (ACD) model (Engle & Russell, 1997, statistical, and ML (Mat Daut, Hassan, Abdullah, Rahman,
1998), which refers to a time series model for durations Abdullah, & Hussin, 2017). Engineering methods, typically
between (negative) events. utilised through software tools such as DOE-2, Energy-
An alternative view on returns’ volatility and the po- Plus, BLAST, and ESP-r, build on physical models that
tential occurrence of crashes draws upon the earthquake forecast consumption through detailed equations which
literature (Ogata, 1978, 1988). The idea is that tensions account for the particularities of the building (Al-Homoud,
in and across tectonic plates build up, until an erup- 2001; Foucquier, Robert, Suard, Stéphan, & Jay, 2013;
tion, and after that, tension starts to build up again un- Zhao & Magoulès, 2012). Statistical methods usually in-
til the next eruption. By modelling the tension-building- volve linear regression (see Section 2.3.2), ARIMA/ARIMAX
up process using so-called Hawkes processes (Hawkes, (see Section 2.3.4), and exponential smoothing (see Sec-
1971, 2018; Hawkes & Oakes, 1974; Ozaki, 1979), one tion 2.3.1) models that forecast consumption using past
can exploit the similarities between earthquakes and fi- consumption data or additional explanatory variables,
nancial crashes (see also Section 2.8.4). Gresnigt, Kole, such as weather or occupancy and calendar related in-
and Franses (2015) take Hawkes processes to daily S&P formation (Deb, Zhang, Yang, Lee, & Shah, 2017). Finally,
500 data and show that it is possible to create reliable ML methods (see Section 2.7.10) typically involve neural
probability predictions of a crash occurrence within the networks (see Section 2.7.8), support vector machines,
next five days. Gresnigt, Kole, and Franses (2017a, 2017b) and grey models that account for multiple non-linear
further develop a specification strategy for any type of as- dependencies between the electricity consumed and the
set returns, and document that there are spillovers across factors influencing its value (Ahmad, Hassan, Abdullah,
assets and markets. Rahman, Hussin, Abdullah, & Saidur, 2014). Till present,
Given investor behaviour, past crashes can ignite fu- the literature has been inconclusive about which class
ture crashes. Hawkes processes are particularly useful to of methods is the most appropriate, with the conclu-
describe this feature and can usefully be implemented sions drawn being subject to the examined building type,
to predict the probability of nearby crashes. By the way, data set used, forecasting horizon considered, and data
these processes can also be useful to predict social con- frequency at which the forecasts are produced (Wei, Li,
flicts, as also there one may discern earthquake-like pat- Peng, Zeng, & Lu, 2019). To mitigate this problem, com-
terns. van den Hengel and Franses (2020) document their binations of methods (see Section 2.6) and hybrids (see
forecasting power for social conflicts in Africa. Section 2.7.13) have been proposed, reporting encourag-
ing results (Mohandes, Zhang, & Mahdiyar, 2019; Zhao &
3.4. Energy Magoulès, 2012).
Other practical issues refer to data pre-processing.
3.4.1. Building energy consumption forecasting and optimi- Electricity consumption data is typically collected at high
sation123 frequencies through smart meters and therefore display
In Europe, buildings account for 40% of total energy noise and missing or extreme values due to monitor-
consumed and 36% of total CO2 emissions (Patti, Acqua- ing issues (see Section 2.7.11). As a result, verifying the
viva, Jahn, Pramudianto, Tomasi, Rabourdin, Virgone, & quality of the input data through diagnostics and data
Macii, 2016). Given that energy consumption of buildings cleansing techniques (see Section 2.2.3 and Section 2.2.4),
is expected to increase in the coming years, forecasting as well as optimising the selected time frames, are im-
electricity consumption becomes critical for improving portant for improving forecasting performance (Bourdeau,
qiang Zhai, Nefzaoui, Guo, & Chatellier, 2019). Similarly,
123 This subsection was written by Christoph Bergmeir & Evangelos it is critical to engineer (see Section 2.2.5) and select
Spiliotis. (see Section 2.5.3) appropriate regressor variables which
781
are of high quality and possible to accurately predict More generally, many challenges in the space of build-
to assist electricity consumption forecasting. Finally, it ing energy optimisation are classical examples of so-
must be carefully decided whether the bottom-up, the called ‘‘predict then optimise’’ problems (Demirovic et al.,
top-down or a combination method (see Section 2.10.1) 2019; Elmachtoub & Grigas, 2017). Here, different pos-
will be used for producing reconciled forecasts at both sible scenario predictions are obtained from different
building and end-use level (Kuster, Rezgui, & Mourshed, assumptions in the form of input parameters. These input
2017), being also possibly mixed with temporal aggre- parameters are then optimised to achieve a desired pre-
gation approaches (Spiliotis, Petropoulos et al., 2020, but dicted outcome. As both prediction and optimisation are
also Section 2.10.3). difficult problems, they are usually treated separately (El-
Provided that accurate forecasts are available, effective machtoub & Grigas, 2017), though there are now recent
energy optimisation can take place at a building level or works where they are considered together (Demirovic
across blocks of buildings (see Section 3.4.10) to reduce et al., 2019; El Balghiti, Elmachtoub, Grigas, & Tewari,
energy cost, improve network stability, and support ef- 2019), and this will certainly be an interesting avenue for
forts towards a carbon-free future, by exploiting smart future research.
grid, internet of things (IoT), and big data technologies
along with recommendation systems (Marinakis et al., 3.4.2. Electricity price forecasting124
2020). Forecasting electricity prices has various challenges
An example for a typical application in this area is the that are highlighted in the detailed review papers by
optimisation of heating, ventilation, and air conditioning Weron (2014). Even though there are economically well
(HVAC) systems. The goal is to minimise the energy use motivated fundamental electricity price models, forecast-
of the HVAC system under the constraints of maintaining ing models based on evaluating historic price data are
certain comfort levels in the building (Marinakis, Doukas, the dominating the academic literature. In recent years
Spiliotis, & Papastamatiou, 2017). Though this is predom- the focus on probabilistic forecasting grew rapidly, as
inantly an optimisation exercise, forecasting comes in at they are highly relevant for many applications in energy
different points of the system as input into the optimisa- trading and risk management, storage optimisation and
tion, and many problems in this space involve forecasting predictive maintenance, (Nowotarski & Weron, 2018; Ziel
as a sub-problem, including energy consumption fore- & Steinert, 2018). Electricity price data is highly complex
casting, room occupancy forecasting, inside temperature and is influenced by regulation. However, there is electric-
forecasting, (hyper-local) forecasts of outside tempera- ity trading based on auctions and on continuous trading.
ture, and air pressure forecasting for ventilation, among Many markets like the US and European markets organ-
others. For instance, Krüger and Givoni (2004) use a linear ise day-ahead auctions for electricity prices, see Fig. 7.
regression approach to predict inside temperatures in 3 Thus, we have to predict multivariate time series type
houses in Brazil, and Ruano, Crispim, Conceiçao, and Lúcio data (Ziel & Weron, 2018). In contrast, intraday markets
(2006) propose the use of a neural network to predict usually apply continuous trading to manage short term
temperatures in a school building. Madaus, McDermott, variations due to changes in forecasts of renewable energy
Hacker, and Pullen (2020) predict hyper-local extreme and demand, and outages (Kiesel & Paraschiv, 2017).
heat events, combining global climate models and ma- The key challenge in electricity price forecasting is
chine learning models. Jing, Cai, Chen, Zhai, Cui, and Yin to address all potential characteristics of the considered
(2018) predict air pressure to tackle the air balancing market, most notably (some of them visible in Fig. 7):
problem in ventilation systems, using a support vector
1. (time-varying) autoregressive effects and
machine.
(in)stationarity,
Predicting energy demand on a building/household
2. calendar effects (daily, weekly and annual season-
level from smart meter data is an important research topic
ality, holiday effects, clock-change),
not only for energy savings. In the building space, Ahmad,
3. (time-varying) volatility and higher moment ef-
Mourshed, and Rezgui (2017), Touzani, Granderson, and
fects,
Fernandes (2018), and Wang, Wang, Zeng, Srinivasan, and
4. price spikes (positive and negative), and
Ahrentzen (2018) predict building energy consumption of
5. price clustering.
residential and commercial buildings using decision tree-
based algorithms (random forests and gradient boosted Some of those impacts can be explained by external
trees) and neural networks to improve energy efficiency. inputs, that partially have to be predicted in advance:
A recent trend in forecasting are global forecasting
1. load/demand/consumption (see Section 3.4.3),
models, built across sets of time series (Januschowski
2. power generation, especially from the renewable
et al., 2020). (Recurrent) neural networks (Bandara et al.,
energy sources (RES) of wind and solar (see Sec-
2020a; Hewamalage et al., 2021) are particularly suitable
tions 3.4.6 and 3.4.8),
for this type of processing due to their capabilities to deal
3. relevant fuel prices (especially oil, coal, natural gas;
with external inputs and cold-start problems. Such capa-
see also Section 3.4.4),
bilities are necessary if there are different regimes in the
4. prices of emission allowances (CO2 e costs),
simulations under which to predict, an example of such a
system for HVAC optimisation is presented by Godahewa,
Deng, Prouzeau, and Bergmeir (2020). 124 This subsection was written by Luigi Grossi & Florian Ziel.
782
Fig. 7. Hourly German day-ahead electricity price data resulting from a two-sided auction (top left) with corresponding 24 sale/supply and
purchase/demand curves for 24 May 2020 and highlighted curves for 17:00 (top right), power generation and consumption time series (bottom
left), and bid structure of 24 May 2020 17:00 (bottom right).
5. related power market prices (future, balancing and In recent years, statistical and machine learning meth-
neighboring markets), ods gained a lot of attraction in day-ahead electricity price
6. availabilities of power plants and interconnectors, forecasting. Even though the majority of effects is linear
7. import/export flow related data, and there are specific non-linear dependencies that can be
8. weather effects (e.g. temperature due to cooling explored by using non-linear models, especially neural
and heating and combined heat and power (CHP) networks (Dudek, 2016; Lago, De Ridder, & De Schutter,
effects; see also Section 3.5.2). 2018; Marcjasz, Uniejewski, & Weron, 2019; Ugurlu, Ok-
suz, & Tas, 2018). Of course this comes along with higher
Note that other weather effects might be relevant as well, computational costs compared to linear models. Fezzi and
but should be covered from the fundamental point of view Mosetti (2020) illustrate that even simple linear mod-
by the listed external inputs. Obvious examples are wind els can give highly accurate forecasts, if correctly cali-
speed for the wind power prediction, cloud cover for the brated. However, there seems to be consensus that fore-
solar power production and illumination effects in the cast combination is appropriate, particularly for mod-
electricity consumption. els that have different structures or different calibra-
Many of those external effects may be explained by tion window length (Gaillard, Goude, & Nedellec, 2016;
standard economic theory from fundamental electricity Hubicka, Marcjasz, & Weron, 2018; Mirakyan, Meyer-
price models (Cludius, Hermann, Matthes, & Graichen, Renschhausen, & Koch, 2017).
2014; Kulakov & Ziel, 2021). Even the simple supply stack Another increasing stream of electricity price forecast-
model (merit order model), see Fig. 8, explains many ing models do not focus on the electricity price itself, but
features and should be kept in mind when designing an the bid/sale/sell/supply and ask/sell/purchase/demand
appropriate electricity price forecasting model. curves of the underlying auctions (see Fig. 7, but also
783
Fig. 8. Illustrative example of a supply stack model with inelastic demand for different power plant types, roughly covering the situation in Germany
2020.
Kulakov, 2020; Mestre, Portela, San Roque, & Alonso, and Pavlov (2008) who adopt Markov switching mod-
2020; Shah & Lisi, 2020; Ziel & Steinert, 2016). This so- els for spikes prediction. Christensen, Hurn, and Lindsay
phisticated forecasting problem allows more insights for (2009, 2012) suggest treating and forecasting price spikes
trading applications and the capturing of price clusters. through Poisson autoregressive and discrete-time pro-
In forecasting intraday markets the literature just cesses, respectively. Herrera and González (2014) use a
started to grow quickly. As the aforementioned mar- Hawkes model combined with extreme events theory.
ket characteristics get less distinct if information from
Interregional links among different electricity markets are
day-ahead markets is taken into account appropriately.
used by Clements, Herrera, and Hurn (2015) and Man-
However, intraday prices are usually more volatile and
ner, Türk, and Eichler (2016) to forecast electricity price
exhibit stronger price spikes. Thus, probabilistic fore-
spikes. A new procedure for the simulation of electricity
casting is even more relevant (Janke & Steinke, 2019;
Narajewski & Ziel, 2020b). Recent studies showed that spikes has been recently proposed by Muniain and Ziel
European markets are close to weak-form efficiency. Thus (2020) utilising bivariate jump components in a mean
naive point forecasting benchmarks perform remarkably reverting jump diffusion model in the residuals.
well (Marcjasz, Uniejewski, & Weron, 2020; Narajewski & The second stream of literature includes papers de-
Ziel, 2020a; Oksuz & Ugurlu, 2019). veloping outlier detection methods or robust estimators
As pointed out above, predicting price spikes is par- to improve the forecasting performance of the models.
ticularly important in practice, due to the high impact Martínez-Álvarez, Troncoso, Riquelme, and Aguilar-Ruiz
in decision making problems which occur usually in ex- (2011) tackle the issue of outlier detection and prediction
treme situations, see Fig. 8. Very high electricity prices are defining ‘‘motifs’’, that is patches of units preceding obser-
usually observed in connection to high demand and low vations marked as anomalous in a training set. Janczura,
renewable energy generation, sometimes together with Trück, Weron, and Wolff (2013) focus on the detection
sudden power plant failures. In contrast, negative price
and treatment of outliers in electricity prices. A very sim-
spikes occur in oversupply situation, when there is low
ilar approach, based on seasonal autoregressive models
demand but high penetration from wind and solar power.
and outlier filtering, is followed by Afanasyev and Fe-
The presence of spikes is explored in two main streams in
dorova (2019). Grossi and Nan (2019) introduced a pro-
literature: spike forecasting and prediction of prices under
normal regime through robust estimators. cedure for the robust statistical prediction of electric-
Within the first set of papers, spikes are often mod- ity prices. The econometric framework is represented by
elled as one regime of non-linear models for time series. the robust estimation of non-linear SETAR processes. A
This approach is followed by Mount, Ning, and Cai (2006) similar approach has been followed by Wang, Yang, Du,
focusing on regime-switching models with parameters and Niu (2020) using an outlier-robust machine learning
driven by time-varying variables and by Becker, Hurn, algorithm.
784
3.4.3. Load forecasting125 Li, Tan, and Zhou (2018) combined empirical mode de-
Load forecasting forms the basis where power system composition (EMD), ARIMA, and wavelet neural networks
operation and planning builds upon. Based on the time (WNN) optimised by the fruit fly algorithm on Australian
horizon of the forecasts, load forecasting can be classified Market data and New York City data. Their approach was
into very short-term (VSTLF), that refers to horizon from to separate the linear and nonlinear components from
several minutes ahead up to 1 h, short-term (STLF), that original electricity load; ARIMA is used for linear part
spans from 1 h to 168 h ahead, medium-term (MTLF), while the WNN for the non-linear one.
that spans from 168 h to 1 year ahead and finally, and Sideratos, Ikonomopoulos, and Hatziargyriou (2020)
long-term (LTLF) that concerns predictions from 1 year proposed that a radial basis network that performs the
to several years ahead. In VSTLF and STLF applications, initial forecasting could serve as input to a convolutional
the focus is on the sub-hourly or hourly load. In MTLF neural network that performs the final forecasting. The
and LTLF, the variables of interest can be either monthly proposed model led to lower error compared to the per-
electricity peak load and total demand for energy. sistence model, NN, and SVM. Semero, Zhang, and Zheng
Inputs differ in the various horizons. In VSTLF and (2020) focused on the energy management of a micro-
STLF, apart from meteorological data, day type identifi- grid located in China using EMD to decompose the load,
cation codes are used. In LTLF, macroeconomic data are adaptive neuro-fuzzy inference system (ANFIS) for fore-
used since total demand of energy is influenced by the casting and particle swarm intelligence (PSO) to opti-
long-term modifications of the social and economic envi- mize ANFIS parameters. The results show that the pro-
ronments. Among the horizons, special attention is placed posed approach yielded superior performance over four
at STLF. This is reflected by the research momentum that other methods. Faraji, Ketabi, Hashemi-Dezaki, Shafie-
have been placed in the load forecasting related litera- Khah, and Catalão (2020) proposed a hybrid system for
ture by other researchers (Hong & Fan, 2016). Processes the scheduling of a prosumer microgrid in Iran. Various
like unit commitment and optimal power flow rely on machine learning algorithms provided load and weather
STLF (Bo & Li, 2012; Saksornchai, Lee, Methaprayoon, forecasts. Through an optimisation routine, the best in-
Liao, & Ross, 2005). Additionally, since competitive energy dividual forecast is selected. The hybrid system displayed
markets continually evolve, STLF becomes vital for new better accuracy from the sole application of the individual
market entities such as retailers, aggregators, and pro- forecasters.
sumers for applications such as strategic bidding, portfolio
optimisation, and tariff design (Ahmad, Javaid, Mateen, 3.4.4. Crude oil price forecasting126
Awais, & Khan, 2019; Danti & Magnani, 2017). Crude oil, one of the leading energy resources, has
The models that can be found in the load forecasting contributed to over one-third of the world’s energy con-
related literature can in general categorised into three sumption (Alvarez-Ramirez, Soriano, Cisneros, & Suarez,
types: time-series, machine learning, and hybrid. Time- 2003). The fluctuations of the crude oil price have a sig-
series models historically precede the others. Typical ex- nificant impact on industries, governments as well as
amples of this family are ARMA, ARIMA, and others (see individuals, with substantial up-and-downs of the crude
also Section 2.3.4). In the machine learning models, the oil price bringing dramatic uncertainty for the economic
structure is usually determined via the training process. and political development (Cunado & De Gracia, 2005;
NNs are commonly used. Once a NN is sufficiently trained, Kaboudan, 2001). Thus, it is critical to develop reliable
it can provide forecasts for all types of forecasting hori- methods to accurately forecast crude oil price movement,
zons (Hippert, Pedreira, & Souza, 2001). The third category so as to guard against the crude oil market extreme risks
of models refers to the integration of two or more individ- and improve macroeconomic policy responses. However,
ual forecasting approaches (see also see Section 2.7.13). the crude oil price movement suffers from complex fea-
For instance, a NN can be combined with time series tures such as nonlinearity, irregularities, dynamics and
methods, with unsupervised machine learning algorithms, high volatility (Alquist, Kilian, & Vigfusson, 2013; Herrera,
data transformation, and with meta-heuristics algorithms Hu, & Pastor, 2018; Kang, Kang, & Yoon, 2009, and also
(Bozkurt, Biricik, & Tayşi, 2017; El-Hendawi & Wang, Section 2.3.11), making the crude oil price forecasting still
2020; López, Zhong, & Zheng, 2017; Lu, Azimi, & Iseley, one of the most challenging forecasting problems.
2019). Some prior studies have suggested that the crude oil
Hybrid systems has been tested on validation data price movement is inherently unpredictable, and it would
(through forecasting competitions), power system aggre- be pointless and futile to attempt to forecast future prices,
gated load, and application oriented tasks. Ma (2021) see Miao, Ramchander, Wang, and Yang (2017) for a de-
proposed an ensemble method based on a combination tailed summary. These agnostics consider the naive no-
of various single forecasters on GEFCom2012 forecasting change forecast as the best available forecast value of
competition data that outperformed benchmark forecast- future prices. In recent years, however, numerous stud-
ers such as Theta method, NN, ARIMA, and others (see ies result in forecasts that are more accurate than naive
Section 2.12.7 for further discussions on forecasting com- no-change forecasts, making the forecasting activities of
petitions). For aggregated load cases, researchers focus crude oil prices promising (Alquist et al., 2013; Baumeis-
on different countries and energy markets. Zhang, Wei, ter, Guérin, & Kilian, 2015).
125 This subsection was written by Ioannis Panapakidis. 126 This subsection was written by Xiaoqian Wang.
785
Extensive research on crude oil price forecasting has According to Rao and Kishore (2010), renewables’ typical
focused predominantly on the econometric models, such characteristics such as low load factor, need for energy
as VAR, ARCH-type, ARIMA, and Markov models (see, for storage, small size, high upfront costs create a compet-
example, Agnolucci, 2009; e Silva, Legey, & e Silva, 2010; itive disadvantage, while Meade and Islam (2015b) sug-
Mirmirani & Li, 2004; Mohammadi & Su, 2010, and Sec- gested that renewable technologies are different from
tion 2.3). In the forecasting literature, unit root tests (see other industrial technological innovations because, in the
Section 2.3.4) are commonly applied to examine the sta- absence of focused support, they are not convenient from
tionarity of crude oil prices prior to econometric mod- a financial point of view. In this sense, policy measures
elling (Rahman & Serletis, 2012; Serletis & Rangel-Ruiz, and incentive mechanisms, such as feed-in tariffs, have
2004; Silvapulle & Moosa, 1999). It is well-documented been used to stimulate the market. As highlighted in Lee
that crude oil prices are driven by a large set of exter- and Huh (2017b), forecasting RETs requires to capture
nal components, which are themselves hard to predict, different socio-economic aspects, such as policy choices
including supply and demand forces, stock market ac- by governments, carbon emissions, macroeconomic fac-
tivities, oil-related events (e.g., war, weather conditions), tors, economic and financial development of a country,
political factors, etc. In this context, researchers have fre- competitive strength of traditional energy technologies.
quently considered structural models (see Section 2.3.9), The complex and uncertain environment concerning
which relate the oil price movements to a set of economic RETs deployment has been faced in literature in several
factors. With so many econometric models, is there an op- ways, in order to account for various determinants of the
timal one? Recently, de Albuquerquemello, de Medeiros, transition process. A first stream of research employed a
da Nóbrega Besarria, and Maia (2018) proposed a SE- bottom-up approach, where forecasts at a lower level are
TAR model, allowing for predictive regimes changing af- aggregated to higher levels within the forecasting hier-
ter a detected threshold, and achieved performance im- archy. For instance Park, Yun, Yun, Lee, and Choi (2016)
provements over six widely used econometric models. realised a bottom-up analysis to study the optimum re-
Despite their high computational efficiency, the econo- newable energy portfolio, while Lee and Huh (2017a)
metric models are generally limited in the ability to non- performed a three-step forecasting analysis, to reflect
linear time series modelling. the specificities of renewable sources, by using different
forecasting methods for each of the sources considered.
On the other hand, artificial intelligence and machine
A similar bottom-up perspective was adopted in Zhang,
learning techniques, such as belief networks, support vec-
Bauer, Yin, and Xie (2020), by conducting a multi-region
tor machines (SVMs), recurrent neural networks (RNNs),
study, to understand how multi-level learning may affect
and extreme gradient boosting (XGBoost), provided pow-
RETs dynamics, with the regionalised model of invest-
erful solutions to recognise the nonlinear and irregular
ment and technological development, a general equilib-
patterns of the crude oil price movement with high au-
rium model linking a macro-economic growth with a
tomation (see, for example, Abramson & Finizza, 1991;
bottom-up engineering-based energy system model.
Gumus & Kiran, 2017; Mingming & Jinliang, 2012; Xie,
The relative newness of RETs has posed the challenge
Yu, Xu, & Wang, 2006). However, challenges also ex-
of forecasting with a limited amount of data: in this per-
ist in these techniques, such as computational cost and
spective, several contributions applied the ‘Grey System’
overfitting. In addition, a large number of studies have
theory, a popular methodology for dealing with systems
increasingly focused on the hybrid forecasting models
with partially unknown parameters (Kayacan, Ulutas, &
(see also Section 2.7.13) based on econometrics models
Kaynak, 2010). Grey prediction models for RETs forecast-
and machine learning techniques (Baumeister & Kilian, ing were proposed in Liu and Wu (2021), Lu (2019),
2015; Chiroma, Abdulkareem, & Herawan, 2015; He, Yu, Moonchai and Chutsagulprom (2020), Tsai, Xue, Zhang,
& Lai, 2012; Jammazi & Aloui, 2012), achieving improved Chen, Liu, Zhou, and Dong (2017) and Wu, Ma, Zeng,
performance. Notably, the vast majority of the literature Wang, and Cai (2019).
has focused primarily on the deterministic prediction, Other studies developed forecasting procedures based
with much less attention paid to the probabilistic predic- on growth curves and innovation diffusion models (see
tion and uncertainty analysis. However, the high volatility Sections 2.3.18–2.3.20): from the seminal work by
of crude oil prices makes probabilistic prediction more Marchetti and Nakicenovic (1979), contributions on the
crucial to reduce the risk in decision-making (Abramson diffusion of RETs were proposed by Bunea, Della Posta,
& Finizza, 1995; Sun, Sun, Wang, & Wei, 2018). Guidolin, and Manfredi (2020), Dalla Valle and Furlan
(2011), Guidolin and Mortarino (2010), Lee and Huh (2017b)
3.4.5. Forecasting renewable energy technologies127 and Meade and Islam (2015b). Forecasting the diffusion
The widespread adoption of renewable energy tech- of renewable energy technologies was also considered
nologies, RETs, plays a driving role in the transition to within a competitive environment in Furlan and Mor-
low-carbon energy systems, a key challenge to face cli- tarino (2018), Guidolin and Alpcan (2019), Guidolin and
mate change and energy security problems. Forecasting Guseo (2016) and Huh and Lee (2014).
the diffusion of RETs is critical for planning a suitable
energy agenda and setting achievable targets in terms of 3.4.6. Wind power forecasting128
electricity generation, although the available time series Wind energy is a leading source of renewable energy,
are often very short and pose difficulties in modelling. meeting 4.8% of global electricity demand in 2018, more
127 This subsection was written by Mariangela Guidolin. 128 This subsection was written by Jethro Browell.
786
than twice that of solar energy (IEA, 2020). Kinetic en- and Zareipour (2020) and Zhang, Wang, and Wang (2014),
ergy in the wind is converted into electrical energy by and research is ongoing in a range of directions including:
wind turbines according to a characteristic ‘power curve’. improving accuracy and reducing uncertainty in short-
Power production is proportion to the cube of the wind term forecasting, extending forecast horizons to weeks
speed at low-to-moderate speeds, and above this is con- and months ahead, and improving very short-term fore-
stant at the turbine’s rated power. At very high or low cast with remote sensing and data sharing (Sweeney,
wind speeds no power is generated. Furthermore, the Bessa, Browell, & Pinson, 2019, and Section 3.4.10).
power curve is influenced by additional factors including
air density, icing, and degradation of the turbine’s blades. 3.4.7. Wave forecasting129
Forecasts of wind energy production are required from Ocean waves are primarily generated by persistent
minutes to days-ahead to inform the operation of wind winds in one direction. The energy thus propagated by
farms, participation in energy markets and power sys- the wind is referred to as wave energy flux and follows
tems operations. However, the limited predictability of a linear function of wave height squared and wave pe-
the weather (see also Section 3.5.2) and the complex- riod. Wave height is typically measured as significant
ity of the power curve make this challenging. For this wave height, the average height of the highest third of
reason, probabilistic forecasts are increasing used in prac- the waves. The mean wave period, typically measured
tice (Bessa et al., 2017). Their value for energy trading is in seconds, is the average time between the arrival of
clear (Pinson, Chevallier, & Kariniotakis, 2007), but quan- consecutive crests, whereas the peak wave period is the
tifying value for power system operation is extremely wave period at which the highest energy occurs at a
complex. Wind power forecasting may be considered a specific point.
mature technology as many competing commercial of- The benefit of wave energy is that it requires signif-
ferings exist, but research and development efforts to icantly less reserve compared to those from wind (see
produce novel and enhanced products is ongoing (see also Section 3.4.6) and solar (see Section 3.4.8) renewable en-
Section 3.4.5). ergy sources (Hong et al., 2016). For example, the forecast
Short-term forecasts (hours to days ahead) of wind error at one hour ahead for the simulated wave farms is
power production are generally produced by combining typically in the range of 5%–7%, while the forecast errors
numerical weather predictions (NWP) with a model of for solar and wind are 17 and 22% respectively (Reikard,
the wind turbine, farm or even regional power curve, Pinson, & Bidlot, 2011). Solar power is dominated by
depending on the objective. The power curve may be diurnal and annual cycles but also exhibits nonlinear vari-
modelled using physical information, e.g. provided by the ability due to factors such as cloud cover, temperature and
turbine manufacturer, in which case it is also necessary precipitation. Wind power is dominated by large ramp
to post-process NWP wind speeds to match the same events such as irregular transitions between states of high
height-above-ground as the turbine’s rotor. More accurate and low power. Wave energy exhibits annual cycles and
forecasts can be produced by learning the NWP-to-energy is generally smoother although there are still some large
relationship from historic data when it is available. State- transitions, particularly during the winter months. In the
of-the-art methods for producing wind power forecasts first few hours of forecasting wave energy, time series
leverage large quantities of NWP data to produce a single models are known to be more accurate than numerical
forecast (Andrade, Filipe, Reis, & Bessa, 2017) and detailed wave prediction. Beyond these forecast horizons, numer-
information about the target wind farm (Gilbert, Browell, ical wave prediction models such as SWAN (Simulating
& McMillan, 2020a). A number of practical aspects may WAves Nearshore, Booij, Ris, & Holthuijsen, 1999) and
also need to be considered by users, such as mainte- ®
WAVEWATCH III (Tolman, 2008) are widely used. As
nance outages and requirements to reduce output for
there is as yet no consensus on the most efficient model
other reasons, such as noise control or electricity network
for harnessing wave energy, potential wave energy is
issues.
primarily measured with energy flux, but the wave en-
Very short-term forecast (minutes to a few hours
ergy harnessed typically follows non-linear functions of
ahead) are also of value, and on these time scales recent
wave height and wave period in the observations of the
observations are the most significant input to forecasting
six different types of wave energy converters (Reikard,
models and more relevant than NWP. Classical time series
Robertson, Buckham, Bidlot, & Hiles, 2015).
methods perform well (see Section 2.3), and those which
To model the dependencies of wind speed, wave height,
are able to capture spatial dependency between multiple
wave period and their lags, Reikard et al. (2011) uses
wind farms are state-of-the-art, notably vector autore-
linear regressions, which were then converted to fore-
gressive models and variants (Cavalcante, Bessa, Reis, &
casts of energy flux. Pinson, Reikard, and Bidlot (2012)
Browell, 2016; Messner & Pinson, 2018). Care must be
use Reikard et al.’s (2011) regression model and log-
taken when implementing these models as wind power
normal distribution assumptions to produce probabilis-
time series are bounded by zero and the wind farm’s
tic forecasts. López-Ruiz, Bergillos, and Ortega-Sánchez
rated power meaning that errors may not be assumed
(2016) model the temporal dependencies of significant
to be normally distributed. The use of transformations is
wave heights, peak wave periods and mean wave di-
recommended (see also Section 2.2.1), though the choice
rection using a vector autoregressive model, and used
of transformation depends on the nature of individual
them to produce medium to long term wave energy
time series (Pinson, 2012).
Wind power forecasting is reviewed in detail in Giebel
and Kariniotakis (2017), Hong, Pinson, Wang, Weron, Yang, 129 This subsection was written by Jooyoung Jeon.
787
forecasts. Jeon and Taylor (2016) model the temporal Ogliari, Dolara, Manzolini, and Leva (2017) compared
dependencies of significant wave heights and peak wave the PV output power day-ahead forecasts performed by
periods using a bivariate VARMA-GARCH (see also Sec- deterministic (based on three and five parameters electric
tion 2.3.11) to convert the two probabilistic forecasts equivalent circuit) and stochastic hybrid (based on artifi-
into a probabilistic forecast of wave energy flux, find- cial neural network models) methods aiming to find the
ing this approach worked better than either univariate best performance conditions. In general, there is no sig-
modelling of wave energy flux or bivariate modelling of nificant difference between the two deterministic models,
wave energy flux and wind speed. Taylor and Jeon (2018) with the three-parameter approach being slightly more
produce probabilistic forecasts for wave heights using a accurate. Fig. 11 shows the daily value of normalised
bivariate VARMA-GARCH model of wave heights and wind mean absolute error (NMAE%) for 216 days evaluated by
speeds, and using forecasts so as to optimise decision using PHANN and three parameters electric circuit. The
making for scheduling offshore wind farm maintenance PHANN hybrid method achieves the best forecasting re-
vessels dispatched under stochastic uncertainty. On the sults, and only a few days of training can provide accurate
same subject, Gilbert, Browell, and McMillan (2020b) use forecasts.
statistical post-processing of numerical wave predictions Dolara et al. (2018) analysed the effect of different
to produce probabilistic forecasts of wave heights, wave approaches in the composition of a training data-set for
periods and wave direction and a logistic regression to de- the day-ahead forecasting of PV power production based
termine the regime of the variables. They further applied on NN. In particular, the influence of different data-set
the Gaussian copula to model temporal dependency but compositions on the forecast outcome has been investi-
this did not improve their probabilistic forecasts of wave gated by increasing the size of the training set size and by
varying the lengths of the training and validation sets, in
heights and periods.
order to assess the most effective training method of this
machine learning approach. As a general comment on the
3.4.8. Solar power forecasting130 reported results, it can be stated that a method that em-
Over the past few years, a number of forecasting tech- ploys the same chronologically consecutive samples for
niques for photovoltaic (PV) power systems has been training is best suited when the availability of historical
developed and presented in the literature. In general, the data is limited (for example, in newly deployed PV plant),
quantitative comparison among different forecast tech- while training based on randomly mixed samples method,
niques is challenging, as the factors influencing the per- appears to be most effective in the case of a greater data
formance are numerous: the historical data, the weather availability. Generally speaking, ensembles composed of
forecast, the temporal horizon and resolution, and the independent trials are most effective.
installation conditions. A recent review by Sobri, Koohi-
Kamali, and Rahim (2018) presents a comparative anal- 3.4.9. Long-term simulation for large electrical power sys-
ysis of previous works, also including statistical errors. tems131
However, since the conditions and metrics used in each In large electrical power systems with renewable en-
work were different, the comparison is not very meaning- ergy dependence, the power generators need to be sched-
ful. Dolara, Grimaccia, Leva, Mussetta, and Ogliari (2018) uled to supply the system demand (de Queiroz, 2016). In
present relevant evaluation metrics for PV forecasting general, for modelling long-term renewables future be-
accuracy, while Leva, Mussetta, & Ogliari, 2019 com- haviour, such as hydro, wind and solar photovoltaics (PV),
pare their effectiveness and immediate comprehension. stochastic scenarios should be included in the scheduling,
In term of forecast horizon for PV power systems, in- usually in a dispatch optimisation problem under uncer-
traday (Nespoli et al., 2019) and the 24 h of the next tainty – like described, for small systems, in Section 3.4.1
day (Mellit et al., 2020) are considered the most impor- and, for wave forecasting, in Section 3.4.7. Due to the
tant. complexity and uncertainly associated, this problem is, in
Nespoli et al. (2019) compared two of the most widely general, modelled with time series scenarios and multi-
used and effective methods for the forecasting of the PV stage stochastic approaches. de Queiroz (2016) presented
production: a method based on Multi-Layer Perceptron a review for hydrothermal systems, with a focus on the
(MLP) and a hybrid method using artificial neural network optimisation algorithms. Sections 3.4.6 and 3.4.8 explore
combined with clear sky solar radiation (see also Sec- the up-to-date methods for wind and PV solar power
tions 2.7.8 and 2.7.13). In the second case, the simulations forecasting.
are based on a feed-forward neural network (FFNN) but, Here, we emphasise the importance of forecasting with
among the inputs, the irradiation in clear sky conditions simulation in the long-term renewable energy planning,
is provided. This method is called Physical Hybrid Artifi- especially in hydroelectric systems. In this context, due
cial Neural Network (PHANN) and is graphically depicted to the data spatial and temporal dependence structure,
in Fig. 9 (Dolara, Grimaccia, Leva, Mussetta, & Ogliari, time series models are useful for future scenarios gen-
2015). PHANN method demonstrates better performance eration. Although the proposal could be forecasting for
than classical NN methods. Fig. 10 shows a compari- short-term planning and scheduling (as described in Sec-
son between the measured and forecasted hourly output tion 3.4.6, 3.4.7, and Section 3.4.8), simulation strategies
power of the PV plant for both sunny and cloudy days.
The PHANN method shows good forecasting performance, 130 This subsection was written by Sonia Leva.
especially for sunny days. 131 This subsection was written by Fernando Luiz Cyrino Oliveira.
788
Fig. 9. Physical Hybrid Artificial Neural Network (PHANN) for PV power forecasting.
Fig. 10. Measured versus forecasted output power by MLP and PHANN methods.
Fig. 11. Daily NMAE% of the PHANN method trained with 10 days (left) and with 215 days (right) compared with the three-parameters model.
are explored for considering and estimating uncertainty ones, are, in general, stationary. One of the main features
in medium and/or long-term horizons. of hydroelectric generation systems is the strong depen-
According to Hipel and McLeod (1994), stochastic pro- dence on hydrological regimes. To deal with this task,
cesses of natural phenomena, such as the renewables the literature focuses on two main classes for forecasting/
789
simulation streamflow data: physical and data-driven mod- In order to tackle these limitations, recent research
els (Zhang, Peng, Zhang, & Wang, 2015). Water resources in energy time series forecasting is exploring two al-
management for hydropower generation and energy plan- ternative (and potentially complementary) pathways: (i)
ning is one of the main challenges for decision-makers. privacy-preserving analytics, and (ii) data markets.
At large, the hydrological data are transformed into the The role of privacy-preserving techniques applied col-
so-called affluent natural energy, that is used for scenar- laborative forecasting is to combine time series data from
ios simulation and serve as input for the optimisation multiple data owners in order to improve forecasting
algorithms (Oliveira, Souza, & Marcato, 2015). The current accuracy and keep data private at the same time. For so-
state-of-the-art models for this proposal are the periodic lar energy forecasting, Berdugo, Chaussin, Dubus, Hebrail,
ones. Hipel and McLeod (1994) presented a wide range
and Leboucher (2011) described a method based on local
of possibilities, but the univariate periodic autoregressive
and global analog-search that uses solar power time series
(PAR, a periodic extension version of the ones presented
from neighbouring sites, where only the timestamps and
in Section 2.3.4) is still the benchmark, with several en-
normalised weights (based on similarity) are exchanged
hanced versions. The approach fits a model to each period
and not the time series data. Zhang and Wang (2018) pro-
of the historical data and the residuals are simulated to
generate new future versions of the time series, consid- posed, for wind energy forecasting with spatia-temporal
ered stationary. Among many others, important variations data, a combination of ridge linear quantile regression and
and alternative proposals to PAR with bootstrap proce- Alternating Direction Method of Multipliers (ADMM) that
dures (see bootstrap details in Section 2.7.5), Bayesian enables each data owner to autonomously solve its fore-
dynamic linear models, spatial information and copulas casting problem, while collaborating with the others to
versions (for copulas references, see Section 2.4.3) are improve forecasting accuracy. However, as demonstrated
detailed in Souza, Marcato, Dias, and Oliveira (2012), by Gonçalves, Bessa, and Pinson (2021a), the mathemat-
Marangon Lima, Popova, and Damien (2014), Lohmann, ical properties of these algorithms should be carefully
Hering, and Rebennack (2016), and de Almeida Pereira analysed in order to avoid privacy breaches (i.e., when a
and Veiga (2019), respectively. third party recovers the original data without consent).
It is worth considering the need for renewables portfo- An alternative approach is to design a market (or auc-
lio simulation. This led Pinheiro Neto, Domingues, Coim- tion) mechanism for time series or forecasting data where
bra, de Almeida, Alves, and Calixto (2017) to propose a the data owners are willing to sell their private (or con-
model to integrate hydro, wind and solar power scenarios fidential) data in exchange for an economic compensa-
for Brazilian data. For the Eastern United States, Shahri- tion (Agarwal, Dahleh, & Sarkar, 2019). The basic concept
ari and Blumsack (2018) add to the literature on the consists in pricing data as a function of privacy loss, but it
wind, solar and blended portfolios over several spatial and can be also pricing data as a function of tangible benefits
temporal scales. For China, Liu et al. (2020) proposed a such as electricity market profit maximization. Gonçalves,
multi-variable model, with a unified framework, to sim-
Pinson, and Bessa (2021b) adapted for renewable energy
ulate wind and PV scenarios to compensate hydropower
forecasting the model described in Agarwal et al. (2019),
generation. However, in light of the aforementioned, one
by considering the temporal nature of the data and re-
of the key challenges and trends for renewable electrical
lating data price with the extra revenue obtained in the
power systems portfolio simulation are still related to the
inclusion of exogenous variables, such as climate, mete- electricity market due to forecasting accuracy improve-
orological, calendar and economic ones, as mentioned in ment. The results showed a benefit in terms of higher
Section 3.4.2. revenue resulting from the combination of electricity and
data markets. With the advent of peer-to-peer energy
3.4.10. Collaborative forecasting in the energy sector132 markets at the domestic consumer level (Parag & Sova-
As mentioned in Section 3.4.6, the combination of geo- cool, 2016), smart meter data exchange between peers is
graphically distributed time series data, in a collaborative also expected to increase and enable collaborative fore-
forecasting (or data sharing) framework, can deliver sig- casting schemes. For this scenario, Yassine, Shirehjini, and
nificant improvements in the forecasting accuracy of each Shirmohammadi (2015) proposed a game theory mecha-
individual renewable energy power plant. The same is nism where a energy consumer maximizes its reward by
valid for hierarchical load forecasting (Hong et al., 2019) sharing consumption data and a data aggregator can this
and energy price forecasting (see Section 3.4.2). A review data with a data analyst (which seeks data with the lowest
of multivariate time series forecasting methods can be possible price).
found in Section 2.3.9 2.3.11 and Section 2.4.3. However, Finally, promoting data sharing via privacy-preserving
this data might have different owners, which are un- or data monetisation can also solve data scarcity prob-
willing to share their data due to the following reasons: lems in some use cases of the energy sector, such as
(i) personal or business sensitive information, (ii) lack forecasting the condition of electrical grid assets (Fan,
of understanding about which data can and cannot be Nowaczyk, & Röognvaldsson, 2020). Moreover, combina-
shared, and (iii) lack of information about economic (and tion of heterogeneous data sources (e.g., numerical, tex-
technical) benefits from data sharing. tual, categorical) is a challenging and promising avenue of
future research in collaborative forecasting (Obst, Ghattas,
132 This subsection was written by Ricardo Bessa. Claudel, Cugliari, Goude, & Oppenheim, 2019).
790
Fig. 12. (a) UK emissions, (b) energy sources in megatonnes (Mt) and megatonnes of oil equivalent (Mtoe), (c) economic variables, and (d) multi-step
forecasts of CO2 emissions in Mt.
3.5. Environmental applications (Croll, 1875 and Milankovitch, 1969), allowing 100,000
years of multi-step forecasts at endogenous emissions.
3.5.1. Forecasting two aspects of climate change133 Humanity has affected climate since 10 thousand years
First into the Industrial Revolution, the UK is one ago (kya: Ruddiman, 2005), so we commence forecasts
of the first out: in 2013 its per capita CO2 emissions there. Forecasts over −10 to 100 with time series from
dropped below their 1860 level, despite per capita real 400kya in panels (d) to (f) of Fig. 13 show paths within
incomes being around 7-fold higher (Hendry, 2020). The the ranges of past data ±2.2SE (Pretis & Kaufmann, 2018).
model for forecasting UK CO2 emissions was selected Atmospheric CO2 already exceeds 400 ppm (parts per
from annual data 1860–2011 on CO2 emissions, coal and million), dramatically outside the Ice-Age range (Sundquist
oil usage, capital and GDP, their lags and non-linearities & Keeling, 2009). Consequently, we conditionally forecast
(see Section 3.5.2 for higher frequency weather forecasts). the next 100,000 years, simulating the potential climate
Figs. 12(a) to 12(c) show the non-stationary time series for anthropogenic CO2 (Castle & Hendry, 2020b) not-
with strong upward then downward trends, punctuated ing the ‘greenhouse’ temperature is proportional to the
by large outliers from world wars, miners strikes plus logarithm of CO2 (Arrhenius, 1896). The orbital drivers
shifts from legislation and technological change: (Castle will continue to influence all three variables but that
& Hendry, 2020a). Saturation estimation at 0.1% using
relation is switched off in the scenario for ‘exogenised’
Autometrics (Doornik, 2018) retaining all other regres-
CO2 . The 110 dynamic forecasts conditional on 400 ppm
sors, detected 4 step shifts coinciding with major policy
and 560 ppm with ±2SE bands are shown in Fig. 14,
interventions like the 2008 Climate Change Act, plus nu-
panels (a) and (b) for Ice and Temperature respectively.
merous outliers, revealing a cointegrated relation. The
The resulting global temperature rises inferred from these
multi-step forecasts over 2012—2017 from a VAR in panel
Antarctic temperatures would be dangerous, at more than
(d) of Fig. 12 show the advantage of using step-indicator
5 ◦ C, with Antarctic temperatures positive for thousands
saturation (SIS: Castle et al., 2015b).
We formulated a 3-equation simultaneous model of of years (Pretis & Kaufmann, 2020; Vaks, Mason, Breiten-
atmospheric CO2 and Antarctic Temperature and Ice vol- bach, et al., 2019).
ume over 800,000 years of Ice Ages in 1000-year fre-
quency (Kaufmann & Juselius, 2013; Paillard, 2001). Driven 3.5.2. Weather forecasting134
by non-linear functions of eccentricity, obliquity, and The weather has a huge impact on our lives, affecting
precession (see panels (a), (b), and (c) of Fig. 13 respec- health, transport, agriculture (see also Section 3.8.10), en-
tively), the model was selected with saturation estima- ergy use (see also Section 3.4), and leisure. Since Bjerknes
tion. Earth’s orbital path is calculable into the future (1904) introduced hydrodynamics and thermodynamics
133 This section was written by David F. Hendry. 134 This subsection was written by Thordis Thorarinsdottir.
791
Fig. 13. Ice-Age data, model fits, and forecasts with endogenous CO2 .
Fig. 14. Ice-Age simulations with exogenous CO2 .
into meteorology, weather prediction has been based on Leuenberger et al., 2020). However, for fixed computa-
merging physical principles and observational informational resources, there is a trade-off between grid resolu-
tion. Modern weather forecasting is based on numerical tion and ensemble size, with a larger ensemble generally
weather prediction (NWP) models that rely on accurate providing a better estimate of the prediction uncertainty.
estimates of the current state of the climate system, in- Recent advances furthermore include machine learning
cluding ocean, atmosphere and land surface. Uncertainty approaches (see Section 2.7.10) to directly model the
in these estimates is propagated through the NWP model small-scale processes, in particular cloud processes (see,
by running the model for an ensemble of perturbed ini-
for example, Gentine, Pritchard, Rasp, Reinaudi, & Yacalis,
tial states, creating a weather forecast ensemble (Buizza,
2018; Rasp, Pritchard, & Gentine, 2018).
2018; Toth & Buizza, 2019).
Despite rapid progress in NWP modelling, the raw en-
One principal concern in NWP modelling is that small-
scale phenomena such as clouds and convective precipi- semble forecasts exhibit systematic errors in both magni-
tation are on too small a scale to be represented directly tude and spread (Buizza, 2018). Statistical post-processing
in the models and must, instead, be represented by ap- is thus routinely used to correct systematic errors in cal-
proximations known as parameterisations. Current NWP ibration and accuracy before a weather forecast is is-
model development aims at improving both the grid res- sued; see Vannitsem, Wilks, and Messner (2018) for a
olution and the observational information that enters the recent review but also Sections 2.12.4 and 2.12.5. A funda-
models (Bannister, Chipilski, & Martinez-Alvarado, 2020; mental challenge here is to preserve physical consistency
792
across space, time and variables (see, for example, Hein- the systems offered reasonable performance, with corre-
rich, Hellton, Lenkoski, & Thorarinsdottir, 2020; Möller, lation coefficients ranging from 0.6 (lazy learning method)
Lenkoski, & Thorarinsdottir, 2013; Schefzik, Thorarins- to 0.75 (neural network). The work also demonstrated
dottir, & Gneiting, 2013). This is particularly important that the performance of the ensemble of the three sys-
when the weather forecast is used as input for further tems was better than the best model for each monitoring
prediction modelling, e.g., in hydrometeorology (Hemri, station (see also Section 2.6 for further discussions on
2018; Hemri, Lisniak, & Klein, 2015). forecast combinations).
At time scales beyond two weeks, the weather noise Starting from the results of this work, a second config-
that arises from the growth of the initial uncertainty, uration was implemented, using as input also the wind
becomes large (Royer, 1993). Sources of long-range pre- speed measured in the meteorological monitoring sta-
dictability are usually associated with the existence of tion closest to the measurement point of PM. The re-
slowly evolving components of the earth system, includ- searchers observed an improvement in all performance
ing the El Niño Southern Oscillation (ENSO), monsoon indices, with the median of the correlation for the best
rains, the Madden Julian Oscillation (MJO), the Indian model (neural networks) increasing from 0.75 to 0.82 and
Ocean dipole, and the North Atlantic Oscillation (NAO), the RMSE dropping from 15µg /m3 to 7µg /m3 .
spanning a wide range of time scales from months to One of the main drawbacks of data-driven models for
decades (Hoskins, 2013; Vitart, Robertson, & Anderson, air quality is that they provide information only in the
2012). It is expected that, if a forecasting system is capable point where the measurements are available. To over-
of reproducing these slowly evolving components, they come this limitation, recent literature has presented mixed
may also be able to forecast them (Van Schaeybroeck & deterministic and data-driven approaches (see, for exam-
Vannitsem, 2018). The next step is then to find relation- ple, Carnevale, Angelis, Finzi, Turrini, & Volta, 2020) which
ships between modes of low-frequency variability and use the data assimilation procedure and offer promising
the information needed by forecast users such as predic- forecasting performance.
tions of surface temperature and precipitation (Roulin & From a practical point of view, critical issues regarding
Vannitsem, 2019; Smith, Scaife, Eade, Athanasiadis, Bel- forecasting air quality include:
lucci, Bethke, Bilbao, Borchert, Caron, Counillon, Danaba-
soglu, Delowrth, Doblas-Reyes, Dunstone, Estella-Perez, • Information collection and data access: even if re-
Flavoni, Hermanson, Keenlyside, Kharin, Kimoto, Merry- gional authorities have to publicly provide data and
field, Mignot, Mochizuki, Modali, Moneri, Müller, Nicolí, information related to air quality and meteorology,
Ortega, Pankatz, Pholman, Robson, Ruggieri, Sospedra- the measured data are not usually available in real-
Alfonso, Swingedouw, Wang, Wild, Yeager, Yang, & Zhang, time and the interfaces are sometimes not auto-
2020). mated;
• Data quantity: the amount of information required
3.5.3. Air quality forecasting135 by air quality forecasting systems is usually large, in
To preserve human health, European Commission stated particular towards the definition of the training and
in the Directive (2008/50/EC) that member states have validation sets;
to promptly inform the population when the particu- • Non-linear relationships: the phenomenon of accu-
late matter (PM) daily mean value exceeds (or is ex- mulation of pollutants in atmosphere is usually af-
pected to exceed) the threshold of 50µg /m3 . Therefore, fected by strong nonlinearities, which significantly
systems have been designed in order to produce fore- impact the selection of the models and their perfor-
casts for up to three days in advance using as input mance;
the measured value of concentration and meteorological • Unknown factors: it is a matter of fact that the
conditions. These systems can be classified in (i) data- dynamic of pollutants in atmosphere is affected by
driven models (Carnevale, Finzi, Pisoni, & Volta, 2016; a large number of non-measurable variables (such
Corani, 2005; Stadlober, Hormann, & Pfeiler, 2018, and as meteorological variables or the interaction with
Section 2.7), and (ii) deterministic chemical and transport other non-measurable pollutants), largely affecting
models (Honoré, Menut, Bessagnet, Meleux, & Rou, 2007; the capability of the models to reproduce the state
Manders, Schaap, & Hoogerbrugge, 2009). In this section, a of the atmosphere.
brief overview of the application of these systems to the
high polluted area of Lombardy region, in Italy, will be 3.5.4. Forecasting and decision making for floods and water
presented. resources management136
Carnevale, Finzi, Pederzoli, Turrini, and Volta (2018) In Water Resources and Flood Risk Management, de-
compared the results of three different forecasting sys- cision makers are frequently confronted with the need
tems based on neural networks, lazy learning models, and of taking the most appropriate decisions not knowing
regression trees respectively. A single model has been what will occur in the future. To support their decision-
identified for each monitoring station. In the initial con- making under uncertainty, decision theory (Berger, 1985;
figuration, only the last three PM measurements available Bernardo, 1994; DeGroot, 2004) invokes Bayesian informed
were used to produce the forecast. In this configuration, decision approaches, which find the most appropriate
135 This subsection was written by Claudio Carnevale. 136 This subsection was written by Ezio Todini.
793
decision by maximising (or minimising) the expected 3.6. Social good and demographic forecasting
value of a ‘‘utility function’’, thus requiring its definition,
together with the estimation of a ‘‘predictive probability’’ 3.6.1. Healthcare137
density (Berger, 1985) due to the fact that utility func- There are many decisions that depend on the quality
tions are rarely linear or continuous. Consequently, their of forecasts in the health care system, from capacity plan-
expected value does not coincide with the value assumed ning to layout decisions to the daily schedules. In general,
on the predicted ‘‘deterministic’’ expected value. Accord- the role of forecasting in health care is to inform both clin-
ical and non-clinical decisions. While the former concerns
ingly, overcoming the classical 18th century ‘‘mechanis-
decisions related to patients and their treatments (Makri-
tic’’ view by resorting into probabilistic forecasting ap-
dakis, Kirkham, Wakefield, Papadaki, Kirkham, & Long,
proaches becomes essential (see also Section 2.6.2).
2019), the latter involves policy/management, and supply
The failure of decision-making based on deterministic
chain decisions that support the delivery of high-quality
forecasts in the case of Flood Risk Management is easily care for patients.
shown through a simple example. At a river section, the A number of studies refer to the use of forecasting
future water level provided by a forecast is uncertain and methods to inform clinical decision making. These meth-
can be described by a Normal distribution with mean ods are used to screen high risk patients for preventative
10 meters and standard deviation of 5 m. Given a dike health care (Chen, Wang, & Hung, 2015; Santos, Abreu,
elevation of 10.5 m, damages may be expected as zero Garca-Laencina, Simão, & Carvalho, 2015; Uematsu, Ku-
if water level falls below the dike elevation and linearly nisawa, Sasaki, Ikai, & Imanaka, 2014; van der Mark,
growing when level exceeds it with a factor of 106 dol- van Wonderen, Mohrs, van Aalderen, ter Riet, & Bindels,
lars. If one assumes the expect value of forecast as the 2014), to predict mental health issues (Shen, Jia, Nie,
deterministic prediction to compute the damage the latter Feng, Zhang, Hu, Chua, & Zhu, 2017; Tran, Phung, Luo,
will result equal to zero, while if one correctly integrates Harvey, Berk, & Venkatesh, 2013), to assist diagnosis and
the damage function times the predictive density the disease progression (Ghassemi et al., 2015; Ma, Chitta,
estimated expected damage will results into 6.59 millions Zhou, You, Sun, & Gao, 2017; Pierce, Hess, Kline, Shah,
Breslin, Branda, Pencille, Asplin, Nestler, Sadosty, Stiell,
of dollars and educated decisions on alerting or not the
Ting, & Montori, 2010; Qiao, Wu, Ge, & Fan, 2019), to
population or evacuating or not a flood-prone area can be
determine prognosis (Dietzel et al., 2010; Ng, Stein, Ning,
appropriately taken (see also Section 3.6).
& Black-Schaffer, 2007), and to recommend treatments for
Water resources management, and in particular reser-
patients (Kedia & Williams, 2003; Scerri, De Goumoens,
voirs management, aim at deriving appropriate operat- Fritsch, Van Melle, Stiefel, & So, 2006; Shang, Ma, Xiao, &
ing rules via long term expected benefits maximisation. Sun, 2019). Common forecasting methods to inform clin-
Nonetheless, during flood events decision makers must ical decisions include time series (see Sections 2.3.1, 2.3.4
decide how much to preventively release from multi- and 2.3.5), regression models (see Section 2.3.2), classi-
purpose reservoirs in order to reduce dam failure and fication trees (see Section 2.7.12), neural networks (see
downstream flooding risks the optimal choice descending Section 2.7.8), Markov models (see Section 2.3.12) and
from trading-off between loosing future water resource vs Bayesian networks. These models utilise structured and
the reduction of short term expected losses. unstructured data including clinician notes
This is obtained by setting up an objective function (Austin & Kusumoto, 2016; Labarere, Bertrand, & Fine,
based on the linear combination of long and short term 2014) which makes the data pre-processing a crucial part
‘‘expected losses’’, once again based on the available prob- of the forecasting process in clinical health care.
abilistic forecast. This Bayesian adaptive reservoir man- One of the aspects of the non-clinical forecasting that
agement approach incorporating into the decision mech- has received the most attention in both research and ap-
plication is the policy and management. Demand forecast-
anism the forecasting information described by the short-
ing is regularly used in Emergency Departments (Arora,
term predictive probability density, was implemented on
Taylor, & Mak, 2020; Choudhury & Urena, 2020; Khaldi,
the lake Como since 1997 (Todini, 1999, 2017) as an
El Afia, & Chiheb, 2019; Rostami-Tabar & Ziel, 2020),
extension of an earlier original idea (Todini, 1991). This
ambulance services (Al-Azzani, Davari, & England, 2020;
resulted into: Setzler, Saydam, & Park, 2009; Vile, Gillard, Harper, &
• a reduction of over 30% of the city of Como fre- Knight, 2012; Zhou & Matteson, 2016) and hospitals with
quency; several different specialities (McCoy, Pellegrini, & Perlis,
• an average reduction of 12% of the water deficit; 2018; Ordu, Demir, & Tofallis, 2019; Zhou, Zhao, Wu,
Cheng, & Huang, 2018) to inform operational, tactical
• an increase of 3% in the electricity production.
and strategic planning. The common methods used for
Lake Como example clearly shows that instead of bas- this purpose include classical ARIMA and exponential
ing decisions on the deterministic prediction, the use of smoothing methods, regression, singular spectrum anal-
a Bayesian decision scheme, in which model forecasts ysis, Prophet, Double-Seasonal Holt-Winter, TBATS and
describe the predictive probability density, increases the Neural Networks. In public health, forecasting can guide
reliability of the management scheme by essentially re- policy and planning. Although it has a wider definition,
ducing the probability of wrong decisions (Todini, 2017,
2018). 137 This section was written by Bahman Rostami-Tabar.
794
the most attention is given to Epidemic forecasting (see cases. Shaman and Karspeck (2012) used Kalman filter
also Section 3.6.2). based SIR epidemiological models to forecast the peak
Forecasting is also used in both national and global time of influenza 6–7 weeks ahead.
health care supply chains, not only to ensure the avail- For COVID-19, Petropoulos and Makridakis (2020) ap-
ability of medical products for the population but also plied a multiplicative exponential smoothing model (see
to avoid excessive inventory. Additionally, the lack of also Section 2.3.1) for predicting global number of con-
accurate demand forecast in a health supply chain may firmed cases, with very successful results both for point
cost lives (Baicker, Chandra, & Skinner, 2012) and has forecasts and prediction intervals. This article got serious
exacerbated risks for suppliers (Levine, Pickett, Sekhri, traction with 100,000 views and 300 citations in the first
& Yadav, 2008). Classical exponential smoothing, ARIMA, twelve months since its publication, thus evidencing the
regression and Neural Network models have been applied importance of such empirical investigations. There has
to estimate the drug utilisation and expenditures (Dolgin, been a series of studies focusing on predicting deaths
2010; Linnér, Eriksson, Persson, & Wettermark, 2020), in the USA and European countries for the first wave of
blood demand (Fortsch & Khapalova, 2016), hospital sup- the COVID-19 pandemic (IHME COVID-19 health service
plies (Gebicki, Mooney, Chen, & Mazur, 2014; Riahi, Hosseini- utilization forecasting team & Murray, 2020a, 2020b). Fur-
Motlagh, & Teimourpour, 2013) and demand for global thermore, Petropoulos, Makridakis, and Stylianou (2020)
medical items (Amarasinghe, Wichmann, Margolis, & Ma- expanded their investigation to capture the continuation
honey, 2010; Hecht & Gandhi, 2008; van der Laan, van of both cases and deaths as well as their uncertainty,
Dalen, Rohrmoser, & Simpson, 2016). It is important to achieving high levels of forecasting accuracy for ten-days-
note that, while the demand in a health care supply chain ahead forecasts over a period of four months. Along the
has often grouped and hierarchical structures (Mircetica, same lines, Doornik et al. (2020b) have been publish-
Rostami-Tabar, Nikolicica, & Maslarica, 2020, see also ing real-time accurate forecasts of confirmed cases and
Section 2.10.1), this has not been well investigated and deaths from mid-March 2020 onwards. Their approach is
needs more attention. based on extraction of trends from the data using machine
learning.
3.6.2. Epidemics and pandemics138 Pinson and Makridakis (2020) organised a debate be-
Pandemics and epidemics both refer to disease out- tween Taleb and Ioannidis on forecasting pandemics. Ioan-
breaks. An epidemic is a disease outbreak that spreads nidis, Cripps, and Tanner (2020) claim that forecasting for
across a particular region. A pandemic is defined as spread COVID-19 has by and large failed. However they give rec-
of a disease worldwide. Forecasting the evolution of a ommendations of how this can be averted. They suggest
pandemic or an epidemic, the growth of cases and fatal- that the focus should be on predictive distributions and
models should be continuously evaluated. Moreover, they
ities for various horizons and levels of granularity, is a
emphasise the importance of multiple dimensions of the
complex task with raw and limited data – as each disease
problem (and its impact). Taleb et al. (2020) discuss the
outbreak type has unique features with several factors
dangers of using naive, empirical approaches for fat-tailed
affecting the severity and the contagiousness. Be that as
variables and tail risk management. They also reiterate
it may, forecasting becomes an paramount task for the
the inefficiency of point forecasts for such phenomena.
countries to prepare and plan their response (Nikolopou-
Finally, Nikolopoulos, Punia, Schäfers, Tsinopoulos, and
los, 2020), both in healthcare and the supply chains (Be-
Vasilakis (2020) focused on forecast-driven planning, pre-
liën & Forcé, 2012, see also Section 3.6.1 and Section 3.2.2).
dicting the growth of COVID-19 cases and the respective
Successful forecasting methods for the task include
disruptions across the supply chain at country level with
time-series methods (see Section 2.3), epidemiological
data from the USA, India, UK, Germany, and Singapore.
and agent-based models (see Section 2.7.3), metapopula-
Their findings confirmed the excess demand for groceries
tion models, approaches in metrology (Nsoesie, Mararthe,
and electronics, and reduced demand for automotive –
& Brownstein, 2013), machine and deep learning meth-
but the model also proved that the earlier a lock-down
ods (Yang, Zeng, Wang, Wong, Liang, Zanin, Liu, Cao,
is imposed, the higher the excess demand will be for gro-
Gao, Mai, Liang, Liu, Li, Li, Ye, Guan, Yang, Li, Luo, Xie,
ceries. Therefore, governments would need to secure high
Liu, Wang, Zhang, Wang, Zhong, & He, 2020). Anders- volumes of key products before imposing lock-downs;
son, Kühlmann-Berenzon, Linde, Schiöler, Rubinova, and and, when this is not possible, seriously consider more
Frisén (2008) used regression models for the prediction radical interventions such as rationing.
of the peak time and volume of cases for a pandemic Dengue is one of the most common epidemic diseases
with evidence from seven outbreaks in Sweden. Yaffee, in tropical and sub-tropical regions of the world. Esti-
Nikolopoulos, Reilly, Crone, Wagoner, Douglas, Amman, mates of World Health Organisation reveals that about
Ksiazek, and Mills (2011) forecasted the evolution of half of the world’s population is now at risk for Dengue
the Hantavirus epidemic in USA and compared causal infection (Romero, Olivero, Real, & Guerrero, 2019). Aedes
and machine-learning methods with time-series methods aegypti and Aedes albopictus are the principal vectors of
and found that univariate methods quite successful. Soe- dengue transmission and they are highly domesticated
biyanto, Adimi, and Kiang (2010) used ARIMA models for mosquitoes. Rainfall, temperature and relative humidity
successfully short-term forecasting of influenza weekly are thought of as important factors attributing towards
the growth and dispersion of mosquito vectors and po-
138 This subsection was written by Konstantinos Nikolopoulos & tential of dengue outbreaks (Banu, Hu, Hurst, & Tong,
Thiyanga S. Talagala. 2011).
795
In reviewing the existing literature, two data types The analysis of mortality data is fundamental to pub-
have been used to forecast dengue incidence: (i) spatio- lic health authorities and policymakers to make deci-
temporal data: incidence of laboratory-confirmed dengue sions or evaluate the effectiveness of prevention and
cases among the clinically suspected patients (Naish, Dale, response strategies. When facing a new pandemic, mor-
Mackenzie, McBride, Mengersen, & Tong, 2014), (ii) web- tality surveillance is essential for monitoring the over-
based data: Google trends, tweets associated with Dengue all impact on public health in terms of disease sever-
cases (de Almeida Marques-Toledo et al., 2017). ity and mortality (Setel, AbouZahr, Atuheire, Bratschi,
SARIMA models (see also Section 2.3.4) have been Cercone, Chinganya, Clapham, Clark, Congdon, de Sav-
quite popular in forecasting laboratory-confirmed dengue igny, Karpati, Nichols, Jakob, Mwanza, Muhwava, Nah-
cases (Gharbi et al., 2011; Martinez & Silva, 2011; Promprou, mias, Ortiza, & Tshangelab, 2020; Vestergaard, Nielsen,
Jaroensutasinee, & Jaroensutasinee, 2006). Chakraborty, Richter, Schmid, Bustos, Braeye, Denissov, Veideman, Lu-
Chattopadhyay, and Ghosh (2019) used a hybrid model
omala, Möttönen, Fouillet, Caserio-Schönemann, an der
combining ARIMA and neural network autoregressive
Heiden, Uphoff, Lytras, Gkolfinopoulou, Paldy, Domegan,
(NNAR) to forecast dengue cases. In light of biological
O’Donnell, de’ Donato, Noccioli, Hoffmann, Velez, England,
relationships between climate and transmission of Aedes
van Asten, White, Tønnessen, da Silva, Rodrigues, Larrauri,
mosquitoes, several studies have used additional covari-
Delgado-Sanz, Farah, Galanis, Junker, Perisa, Sinnathamby,
ates such as, rainfall, temperature, wind speed, and hu-
midity to forecasts dengue incidence (Banu et al., 2011; Andrews, O’Doherty, Marquess, Kennedy, Olsen, Pebody,
Naish et al., 2014; Talagala, 2015). Poisson regression ECDC Public Health Emergency Team for COVID-19, Krause,
model has been widely used to forecast dengue incidence & Mlbak, 2020). A useful metric is excess mortality and
using climatic factors and lagged time between dengue is the difference between the observed number of deaths
incidence and weather variables (Hii, Zhu, Ng, Ng, & Rock- and the expected number of deaths under ‘‘normal’’ con-
löv, 2012; Koh, Spindler, Sandgren, & Jiang, 2018). Several ditions (Aron & Muellbauer, 2020; Checchi & Roberts,
researchers looked at the use of Quasi-Poisson and neg- 2005). Thus, it can only be estimated with accurate and
ative binomial regression models to accommodate over high-quality data from previous years. Excess mortality
dispersion in the counts (Lowe et al., 2011; Wang, Jiang, has been used to measure the impact of heat events (Li-
Fan, Wang, & Liu, 2014). Cazelles, Chavez, McMichael, maye, Vargo, Harkey, Holloway, & Patz, 2018; Matte,
and Hales (2005) used wavelet analysis to explore the Lane, & Ito, 2016), pandemic influenza (Nielsen, Mazick,
dynamic of dengue incidence and wavelet coherence anal- Andrews, Detsis, Fenech, Flores, Foulliet, Gergonne, Green,
yses was used to identify time and frequency specific Junker, Nunes, O’Donnell, Oza, Paldy, Pebody, Reynolds,
association with climatic variables. de Almeida Marques- Sideroglou, E, Simon-Sofia, Uphoff, Van Asten, Virtanen,
Toledo et al. (2017) took a different perspective and look Wuillaume, & Molbak, 2013; Nunes, Viboud, Machado,
at weekly tweets to forecast Dengue cases. Rangarajan, Ringholz, Rebelo-de Andrade, Nogueira, & Miller, 2011),
Mody, and Marathe (2019) used Google trend data to and nowadays COVID-19 (Nogueira, de Araújo Nobre,
forecast Dengue cases. Authors hypothesised that web Nicola, Furtado, & Carneiro, 2020; Ritchie, Ortiz-Ospina,
query search related to dengue disease correlated with Beltekian, Mathieu, Hasell, Macdonald, Giattino, & Roser,
the current level of dengue cases and thus may be helpful 2020; Shang & Xu, 2021; Sinnathamby, Whitaker, Cough-
in forecasting dengue cases.
lan, Bernal, Ramsay, & Andrews, 2020, and Section 3.6.2),
A direction for future research in this field is to explore
among others. Excess mortality data have been making
the use of spatio-temporal hierarchical forecasting (see
available by the media publications The Economist, The
Section 2.10).
New York Times and The Financial Times. Moreover, a mon-
3.6.3. Forecasting mortality139 itoring system of the weekly excess mortality in Europe
Actuarial, Demographic, and Health studies are some has been performed by the EuroMOMO project (Vester-
examples where mortality data are commonly used. A gaard et al., 2020).
valuable source of mortality information is the Human An essential use of mortality data for those individuals
Mortality Database (HMD), a database that provides mor- at age over 60 is in the pension and insurance industries,
tality and population data for 41 mainly developed coun- whose profitability and solvency crucially rely on accu-
tries. Additionally, at least five country-specific databases rate mortality forecasts to adequately hedge longevity
are devoted to subnational data series: Australian, Cana- risks (see, e.g., Shang & Haberman, 2020a, 2020b). Longevity
dian, and French Human Mortality Databases, United risk is a potential systematic risk attached to the increas-
States and Japan Mortality Databases. In some situations, ing life expectancy of annuitants, and it is an important
the lack of reliable mortality data can be a problem, espe- factor to be considered when determining a sustainable
cially in developing countries, due to delays in registering government pension age (see, e.g., Hyndman, Zeng, &
or miscounting deaths (Checchi & Roberts, 2005). Analysis Shang, 2021, for Australia). The price of a fixed-term or
of National Causes of Death for Action (ANACONDA) is a lifelong annuity is a random variable, as it depends on the
valuable tool that assesses the accuracy and complete- value of zero-coupon bond price and mortality forecasts.
ness of data for mortality and cause of death by check- The zero-coupon bond price is a function of interest rate
ing for potential errors and inconsistencies (Mikkelsen, (see Section 3.3.6) and is comparably more stable than the
Moesgaard, Hegnauer, & Lopez, 2020). retirees’ mortality forecasts.
Several methodologies were developed for mortality
139 This subsection was written by Clara Cordeiro & Han Lin Shang. modelling and forecasting (Booth & Tickle, 2008; Janssen,
796
2018). These methods can be grouped into three cate- behaviour is influenced by numerous factors acting at
gories: expectation, explanation, and extrapolation (Booth different levels, from individual characteristics to soci-
& Tickle, 2008). etal change (Balbo, Billari, & Mills, 2013). An important
The expectation approach is based on the subjective methodological challenge for many low- and middle-
opinion of experts (see also Section 2.11.4), who set a income countries is fertility estimation, due to deficien-
long-run mortality target. Methods based on expectation cies in vital statistics caused by inadequate birth regis-
make use of experts’ opinions concerning future mortal- tration systems (AbouZahr, de Savigny, Mikkelsen, Setel,
ity or life expectancy with a specified path and speed Lozano, & Lopez, 2015; Moultrie, Dorrington, Hill, Hill,
of progression towards the assumed value (Continuous Timæ us, & Zaba, 2013; Phillips, Adair, & Lopez, 2018).
Mortality Investigation, 2020). The advantage of this ap- Such countries are often also in the process of transition-
proach is that demographic, epidemiological, medical, and ing from high to low fertility, which induces greater fore-
other relevant information may be incorporated into the cast uncertainty compared to low-fertility countries (Pro-
forecasts. The disadvantages are that such information is gramme, 2019).
subjective and biased towards experts’ opinions, and it A range of statistical models have been proposed to
only produces scenario-based (see Section 2.11.5) deter- forecast fertility – see Booth (2006), Bohk-Ewald, Li, and
ministic forecasts (Ahlburg & Vaupel, 1990; Wong-Fupuy Myrskylä (2018), and Shang and Booth (2020) for reviews.
& Haberman, 2004). The Lee–Carter model (Lee & Carter, 1992) originally
The explanation approach captures the correlation be- developed for mortality forecasting (see Section 3.6.3)
tween mortality and the underlying cause of death. Meth-
has been applied to fertility (Lee, 1993), with exten-
ods based on the explanation approach incorporate med-
sions in functional data (Hyndman & Ullah, 2007) and
ical, social, environmental, and behavioural factors into
Bayesian (Wiśniowski, Smith, Bijak, Raymer, & Forster,
mortality modelling. Example include smoking and disease-
2015) contexts. Other notable extrapolative methods in-
related mortality models. The benefit of this approach is
clude the cohort-ARIMA model of De Beer (1985, 1990)
that mortality change can be understood from changes
– see Section 2.3.4 – and the linear extrapolation method
in related explanatory variables; thus, it is attractive in
of Myrskylä, Goldstein, and Cheng (2013). Many paramet-
terms of interpretability (Gutterman & Vanderhoof, 1998).
ric models have been specified to describe the shapes of
The extrapolative approach is considered more ob-
jective, easy to use and more likely to obtain better fertility curves (Brass, 1974; Evans, 1986; Hoem, Madsen,
forecast accuracy than the other two approaches (Janssen, Nielsen, Ohlsen, Hansen, & Rennermalm, 1981; Schmert-
2018). The extrapolation approach identifies age patterns mann, 2003), with forecasts obtained through time series
and trends in time which can be then forecasted via extrapolations of the parameters (Congdon, 1990; De Iaco
univariate and multivariate time series models (see Sec- & Maggio, 2016; Knudsen, McNown, & Rogers, 1993).
tion 2.3). In the extrapolation approach, many parametric Bayesian methods have been used to borrow strength
and nonparametric methods have been proposed (see, across countries (for example, Alkema et al., 2011; Schmert-
e.g., Alho & Spencer, 2005; Hyndman & Ullah, 2007; mann, Zagheni, Goldstein, & Myrskylä, 2014), with El-
Shang, Booth, & Hyndman, 2011). Among the parametric lison, Dodd, and Forster (2020) developing a hierarchi-
methods, the method of Heligman and Pollard (1980) cal model in the spirit of the latter. The top-down ap-
is well-known. Among the nonparametric methods, the proach (see Section 2.10.1) of the former, which is used by
Lee–Carter model (Lee & Carter, 1992), Cairns-Blake-Dowd the United Nations, projects the aggregate Total Fertility
model (Cairns et al., 2009; Dowd, Cairns, Blake, Cough- Rate (TFR) measure probabilistically (also see Tuljapurkar
lan, Epstein, & Khalaf-Allah, 2010), and functional data & Boe, 1999) before decomposing it by age. Hyppöla,
model (Hyndman & Ullah, 2007, and Section 2.3.10), as Tunkelo, and Törnqvist (1949) provide one of the earli-
well as their extensions and generalisations are dominant. est known examples of probabilistic fertility forecasting
The time-series extrapolation approach has the advan- (Alho & Spencer, 2005).
tage of obtaining a forecast probability distribution rather Little work has been done to compare forecast per-
than a deterministic point forecast and, also, enable the formance across this broad spectrum of approaches. The
determination of forecast intervals (Booth & Tickle, 2008). study of Bohk-Ewald et al. (2018) is the most comprehen-
Janssen (2018) presents a review of the advances in sive to date. Most striking is their finding that few meth-
mortality forecasting and possible future research chal- ods can better the naive freezing of age-specific rates, and
lenges. those that can differ greatly in method complexity (see
also Section 2.5.2). A recent survey of fertility forecast-
3.6.4. Forecasting fertility140 ing practice in European statistical offices (Gleditsch &
Aside from being a driver of population forecasts (see Syse, 2020) found that forecasts tend to be deterministic
Section 2.3.7), fertility forecasts are vital for planning and make use of expert panels (see Section 2.11.4). Ex-
maternity services and anticipating demand for school pert elicitation techniques are gaining in sophistication,
places. The key challenge relates to the existence of, highlighted by the protocol of Statistics Canada (Dion,
and interaction between, the quantum (how many?) and Galbraith, & Sirag, 2020) which requests a full probability
tempo (when?) components (Booth, 2006). This intrinsic distribution of the TFR.
dependence on human decisions means that childbearing A promising avenue is the development of forecasting
methods that incorporate birth order (parity) information,
140 This subsection was written by Joanne Ellison. supported by evidence from individual-level analyses (for
797
example, Fiori, Graham, & Feng, 2014). Another underex- more predictable than other (e.g., asylum). Another recog-
plored area is the integration of survey data into fertil- nised problem with models using covariates is that those
ity forecasting models, which tend to use vital statistics can be endogenous to migration (e.g., population) and also
alone when they are of sufficient quality (see Rendall, need predicting, which necessitates applying structured
Handcock, & Jonsson, 2009; Zhang & Bryant, 2019, for models to prevent uncertainty from exploding.
Bayesian fertility estimation with imperfect census data). The methodological gaps and current work in migra-
Alternative data sources also have great potential. For tion forecasting concentrate in a few key areas, notably
example, Wilde, Chen, and Lohmann (2020) use Google including causal (mechanistic) forecasting based on the
data to predict the effect of COVID-19 on US fertility process of migrant decision making (Willekens, 2018); as
in the absence of vital statistics. Lastly, investigation of well as early warnings and ‘nowcasting’ of rapidly chang-
the possible long-term impacts of delayed motherhood ing trends, for example in asylum migration (Napierała,
in high-income countries, alongside developments in as- Hilton, Forster, Carammia, & Bijak, 2021). In the context
sisted reproduction technology such as egg freezing, is of early warnings, forays into data-driven methods for
required (see, for example, Sobotka & Beaujouan, 2018). changepoint detection, possibly coupled with the digital
trace and other high-frequency ‘Big data’, bear particular
3.6.5. Forecasting migration141 promise. At the same time, coherent uncertainty descrip-
Migration forecasts are needed both as crucial input tion across a range of time horizons, especially in the long
into population projections (see Section 2.3.7), as well as range (Azose & Raftery, 2015), remains a challenge, which
standalone predictions, made for a range of users, chiefly needs addressing for the sake of proper calibration of er-
in the areas of policy and planning. At the same time, mi- rors in the population forecasts, to which these migration
gration processes are highly uncertain and complex, with components contribute.
many underlying and interacting drivers, which evade
precise conceptualisation, definitions, measurement, and 3.6.6. Forecasting risk for violence and wars142
theoretical description (Bijak & Czaika, 2020). Given the Can we predict the occurrence of WW3 in the next
high level of the predictive uncertainty, and the non- 20 years? Is there any trend in the severity of wars?
stationary character of many migration processes (Bijak & The study of armed conflicts and atrocities, both in
Wiśniowski, 2010), the current state of the art of forward- terms of frequency over time and the number of casu-
looking migration studies reflects therefore a shift from alties, has received quite some attention in the scientific
prediction to the use of forecasts as contingency planning literature and the media (e.g., Cederman, 2003; Fried-
tools (idem). man, 2015; Hayes, 2002; Norton-Taylor, 2015; Richard-
Reviews of migration forecasting methods are avail- son, 1948, 1960), falling within the broader discussion
able in Bijak (2010) and Sohst, Tjaden, de Valk, and Melde about violence (Berlinski, 2009; Goldstein, 2011; Spagat,
(2020). The applications in official statistics, with a few Mack, Cooper, & Kreutz, 2009), with the final goal of
exceptions, are typically based on various forms scenario- understanding whether humanity is becoming less bel-
based forecasting with judgment (see Section 2.11.5), ligerent (Pinker, 2011), or not (Braumoeller, 2019).
based on pre-set assumptions (for an example, see Abel, Regarding wars and atrocities, the public debate has
2018). Such methods are particularly used for longer time focused its attention on the so-called Long Peace The-
horizons, of a decade or more, so typically in the con- ory (Gaddis, 1989), according to which, after WW2, hu-
text of applications in population projections, although manity has experienced the most peaceful period in his-
even for such long time horizons calibrated probabilistic tory, with a decline in the number and in the severity
methods have been used as well (Azose et al., 2016). of bloody events. Scholars like Mueller (2009a, 2009b)
The mainstream developments in migration forecast- and Pinker (2011, 2018) claim that sociological arguments
ing methodology, however, include statistical and econo- and all statistics suggest we live in better times, while
metric methods discussed in Section 2.3, such as time others like Gray (2015a, 2015b) and Mann (2018) main-
series models, both uni- and multivariate (for exam- tain that those statistics are often partial and misused,
ple, Bijak, 2010; Bijak, Disney, Findlay, Forster, Smith, & the derived theories weak, and that war and violence are
Wiśniowski, 2019; Gorbey, James, & Poot, 1999), econo- not declining but only being transformed. For Mann, the
metric models (for example, Brücker & Siliverstovs, 2006; Long Peace proves to be ad-hoc, as it only deals with
Cappelen, Skjerpen, & Tønnessen, 2015), Bayesian hier- Western Europe and North America, neglecting the rest
archical models (Azose & Raftery, 2015), and dedicated of the world, and the fact that countries like the US have
methods, for example for forecasting data structured by been involved in many conflicts out of their territories
age (Raymer & Wiśniowski, 2018). In some cases, the after WW2.
methods additionally involve selection and combining Recent statistical analyses confirm Gray’s and Mann’s
forecasts through Bayesian model selection and averag- views: empirical data do not support the idea of a decline
ing (Bijak, 2010, see also Section 2.5 and Section 2.6). Such in human belligerence (no clear trend appears), and in
models can be expected to produce reasonable forecasts its severity. Armed conflicts show long inter-arrival times,
(and errors) for up to a decade ahead (Bijak & Wiśniowski, therefore a relative peace of a few decades means nothing
2010), although this depends on the migration flows be- statistically (Cirillo & Taleb, 2016b). Moreover, the distri-
ing forecast, with some processes (e.g., family migration) bution of war casualties is extremely fat-tailed (Clauset,
141 This subsection was written by Jakub Bijak. 142 This subsection was written by Pasquale Cirillo.
798
2018; Clauset & Gleditsch, 2018), often with a tail ex- relevant information, analytical models, judgment, visual-
ponent ξ = α1 > 1 (Cirillo & Taleb, 2016b), indicating isations, and feedback. To that end, FSSs must (i) elabo-
a possibly infinite mean, i.e., a tremendously erratic and rate accurate, efficient, and automatic statistical forecast-
unforeseeable phenomenon (see Section 2.3.22). An only ing methods, (ii) enable users to effectively incorporate
apparently infinite-mean phenomenon though (Cirillo & their judgment, (iii) allow the users to track and interact
Taleb, 2019), because no single war can kill more than the with the whole forecasting procedure, and (iv) be easily
entire world population, therefore a finite upper bound customised based on the context of the company.
exists, and all moments are necessarily finite, even if Indeed, nowadays, most off-the-self solutions, such
difficult to estimate. Extreme value theory (Embrechts as SAP, SAS, JDEdwards, and ForecastPro, offer a vari-
et al., 2013) can thus be used to correctly model tail risk ety of both standard and advanced statistical forecasting
and make prudential forecasts (with many caveats like in methods (see Section 2.3), as well as data pre-processing
Scharpf, Schneider, Nöh, & Clauset, 2014), while avoiding (see Section 2.2) and performance evaluation algorithms
naive extrapolations (Taleb et al., 2020). (see Section 2.12). On the other hand, many of them
As history teaches (Nye, 1990), humanity has already still struggle to incorporate state-of-the-art methods that
experienced periods of relative regional peace, like the can further improve forecasting accuracy, such as auto-
famous Paces Romana and Sinica. The present Pax Ameri- matic model selection algorithms and temporal aggrega-
cana is not enough to claim that we are structurally living tion (see also Section 2.10.2), thus limiting the options of
in a more peaceful era. The Long Peace risks to be an- the users (Petropoulos, 2015). Similarly, although many
other apophenia, another example of Texan sharpshooter FSSs support judgmental forecasts (see Section 2.11.1) and
fallacy (Carroll, 2003). judgmental adjustments of statistical forecasts (see Sec-
Similar mistakes have been made in the past. Buckle tion 2.11.2), this is not done as suggested by the literature,
(1858) wrote: ‘‘that [war] is, in the progress of society, i.e., in a guided way under a well-organised framework.
steadily declining, must be evident, even to the most As a result, the capabilities of the users are restrained
hasty reader of European history. If we compare one and methods that could be used to mitigate biases, over-
country with another, we shall find that for a very long shooting, anchoring, and unreasonable or insignificant
period wars have been becoming less frequent; and now changes that do not rationalise the time wasted, are
so clearly is the movement marked, that, until the late largely ignored (Fildes & Goodwin, 2013; Fildes et al.,
commencement of hostilities, we had remained at peace 2006).
for nearly forty years: a circumstance unparalleled [...] in Other practical issues of FSSs are related with their
the affairs of the world’’. Sadly, Buckle was victim of the engine and interfaces which are typically designed so that
illusion coming from the Pax Britannica (Johnston, 2008): they are generic and capable to serve different compa-
the century following his prose turned out to be the most nies and organisations of diverse needs (Kusters et al.,
murderous in human history. 2006). From a developing and economic perspective, this
is a reasonable choice. However, the lack of flexibility
and customisability can lead to interfaces with need-
3.7. Systems and humans less options, models, tools, and features that may con-
fuse inexperienced users and undermine their perfor-
3.7.1. Support systems143 mance (Fildes et al., 2006). Thus, simple, yet exhaus-
Forecasting in businesses is a complicated procedure, tive interfaces should be designed in the future to bet-
especially when predicting numerous, diverse series (see ter serve the needs of each company and fit its partic-
Section 2.7.4), dealing with unstructured data of multi- ular requirements (Spiliotis, Raptis, & Assimakopoulos,
ple sources (see Section 2.7.1), and incorporating human 2015). Ideally, the interfaces should be adapted to the
judgment (Lim & O’Connor, 1996a, but also Section 2.11). strengths and weaknesses of the user, providing useful
In this respect, since the early 80’s, various Forecasting feedback when possible (Goodwin, Fildes, Lawrence, &
Support Systems (FSSs) have been developed to facilitate Nikolopoulos, 2007). Finally, web-based FSSs could re-
forecasting and support decision making (Kusters, Mc- place windows-based ones that are locally installed and
Cullough, & Bell, 2006). Rycroft (1993) provides an early therefore of limited accessibility, availability, and compat-
comparative review of such systems, while many studies ibility (Asimakopoulos & Dix, 2013). Cloud computing and
strongly support their utilisation over other forecasting web-services could be exploited in that direction.
alternatives (Sanders & Manrodt, 2003; Tashman & Leach,
1991). 3.7.2. Cloud resource capacity forecasting144
In a typical use-case scenario, the FSSs will retrieve One of the central promises in cloud computing is
the data required for producing the forecasts, will provide that of elasticity. Customers of cloud computing services
some visualisations and summary statistics to the user, can add compute resources in real-time to meet and
allow for data pre-processing, and then produce forecasts satisfy increasing demand and, when demand for a cloud-
that may be adjusted according to the preferences of the hosted application goes down, it is possible for cloud
user. However, according to Ord and Fildes (2013), effec- computing customers to down-scale. The benefit of the
tive FSS should be able to produce forecasts by combining latter is particularly economically interesting during the
143 This subsection was written by Vassilios Assimakopoulos. 144 This subsection was written by Tim Januschowski.
799
current pandemic. Popular recent cloud computing offer- 3.7.3. Judgmental forecasting in practice145
ings take this elasticity concept one step further. They Surveys of forecasting practice (De Baets, 2019) have
abstract away the computational resources completely shown that the use of pure judgmental forecasting by
from developers, so that developers can build serverless practitioners has become less common. About 40 years
applications. In order for this to work, the cloud provider ago, Sparkes and McHugh (1984) found that company ac-
handles the addition and removal of compute resources tion was more likely to be influenced by judgmental fore-
‘‘behind the scenes’’. casts than by any other type of forecast. In contrast, Fildes
and Petropoulos (2015) found that only 15.6% of forecasts
To keep the promise of elasticity, a cloud provider
in the surveyed companies were made by judgment alone.
must address a number of forecasting problems at vary-
The majority of forecasts (55.6%) were made using a com-
ing scales along the operational, tactical and strategic
bination of statistical and judgmental methods. In this
problem dimensions (Januschowski & Kolassa, 2019). As
section, we discuss forecasting using unaided judgment
an example for a strategic forecasting problems: where (pure judgmental forecasting; see also Section 2.11.1),
should data centres be placed? In what region of a country judgmental adjustments (judgment in combination with
and in what geographic region? As an example for tactical statistical models; see also Section 2.11.2), and the role of
forecasting problems, these must take into account energy judgment in forecasting support systems.
prices (see Section 3.4.2) and also, classic supply chain On the first theme, the survey results discussed above
problems (Larson, Simchi-Levi, Kaminsky, & Simchi-Levi, beg the question of whether pure judgmental forecasting
2001). After all, physical servers and data centres are what is still relevant and reliable. Answers here depend on the
enables the cloud and these must be ordered and have type of information on which the judgmental forecasts
a lead-time. The careful incorporation of life cycles of are based (Harvey, 2007, see also Section 2.11.1). For
compute types is important (e.g., both the popularity of instance, people have difficulty making cross-series fore-
certain machine types and the duration of a hard disk). casts, as they have difficulty learning the correlation be-
Analogous to the retail sector, cloud resource providers tween variables and using it to make their forecasts (Har-
have tactical cold-start forecasting problems. For example, vey, Bolger, & McClelland, 1994; Lim & O’Connor, 1996b,
1996c). Additionally, they appear to take account of the
while GPU or TPU instances are still relatively recent
noise as well as the pattern when learning the relation
but already well estabilished, the demand for quantum
between variables; hence, when later using one of the
computing is still to be decided. In the class of opera-
variables to forecast the other, they add noise to their
tional forecasting problems, cloud provider can choose to
forecasts (Gray, Barnes, & Wilkinson, 1965). Judgmen-
address short-term resource forecasting problems for ap- tal extrapolation from a single time series is subject to
plications such as adding resources to applications predic- various effects. First, people are influenced by optimism.
tively and make this available to customers (Barr, 2018). For example, they over-forecast time series labelled as
The forecasting of the customer’s spend for cloud com- ‘profits’ but under-forecast the same series labelled as
puting is another example. For serverless infrastructure, a ‘losses’ (Harvey & Reimers, 2013). Second, they add noise
number of servers is often maintained in a ready state (Gias to their forecasts so that a sequence of forecasts looks
& Casale, 2020) and the forecasting of the size of this similar to (‘represents’) the data series (Harvey, 1995).
‘warmpool’ is another example. We note that cloud com- Third, they damp trends in the data (Eggleton, 1982;
puting customers have forecasting problems that mirror Harvey & Reimers, 2013; Lawrence & Makridakis, 1989).
the forecasting challenges of the cloud providers. Interest- Fourth, forecasts from un-trended independent series do
ingly, forecasting itself has become a software service that not lie on the series mean but between the last data point
cloud computing companies offer (Januschowski, Arpin, and the mean; this is what we would expect if people
Salinas, Flunkert, Gasthaus & Lorenzo, 2018; Liberty et al., perceived a positive autocorrelation in the series (Reimers
& Harvey, 2011). These last two effects can be explained
2020; Poccia, 2019)
in terms of the under-adjustment that characterises use
Many challenges in this application area are not unique
of the anchor-and-adjust heuristic: forecasters anchor on
to cloud computing. Cold start problems exist elsewhere
the last data point and adjust towards the trend line or
for example. What potentially stands out in cloud com-
mean – but do so insufficiently. However, in practice,
puting forecasting problems may be the scale (e.g., there this under-adjustment may be appropriate because real
are a lot of physical servers available), the demands on the linear trends do become damped and real series are more
response time and granularity of a forecast and the degree likely to contain a modest autocorrelation than be inde-
of automation. Consider the operational forecasting prob- pendent (Harvey, 2011). We should therefore be reluctant
lem of predictive scaling. Unlike in retail demand forecast- to characterise these last two effects as biases.
ing, no human operator will be able to control this and Given these inherent flaws in people’s decision mak-
response times to forecasts are in seconds. It will be inter- ing, practitioners might be hesitant to base their predic-
esting to see whether approaches based on reinforcement tions on judgment. However, the reality is that companies
learning (Dempster, Payne, Romahi, & Thompson, 2001; persist in incorporating judgment into their forecasting.
Gamble & Gao, 2018) can partially replace the need to
have forecasting models (Januschowski, Gasthaus, Wang, 145 This subsection was written by Shari De Baets, M. Sinan Gönül, &
Rangapuram & Callot, 2018). Nigel Harvey.
800
Assertions that they are wrong to do so represent an over- selection, (iii) when baseline predictions are supplemented
simplified view of the reality in which businesses operate. by additional supportive materials such as scenarios and
Statistical models are generally not able to account for alternative forecasts, (iv) when the forecasting source is
external events, events with low frequency, or a patchy believed to be trustworthy and reliable, and (v) when
and insufficient data history (Armstrong & Collopy, 1998; organisational policy or culture prohibits judgmental ad-
Goodwin, 2002; Hughes, 2001). Hence, a balance may justments. In these circumstances, the baseline forecasts
be found in the combination of statistical models and are more easily accepted by practitioners and their ad-
judgment (see Section 2.11.2). justments tend to be less frequent.
In this respect, judgmental adjustments to statistical Ideally, a Forecast Support System (FSS; see
model outputs are the most frequent form of judgmental Section 3.7.1) should be designed to ensure that it encour-
forecasting in practice (Arvan et al., 2019; Eksoz et al., ages adjustment or non-adjustment whichever is appro-
2019; Lawrence et al., 2006; Petropoulos et al., 2016). priate (Fildes et al., 2006). But how can this be achieved?
Judgmental adjustments give practitioners a quick and The perceived quality and accessibility of a FSS can be
influenced by its design. More on this can be found in
convenient way to incorporate their insights, their experi-
the literature on the Technology Acceptance Model (Davis,
ence and the additional information that they possess into
Bagozzi, & Warshaw, 1989) and decision making (for
a set of statistical baseline forecasts. Interestingly, Fildes
instance, by means of framing, visual presentation or
et al. (2009) examined the judgmental adjustment appli-
nudging; e.g., Gigerenzer, 1996; Kahneman & Tversky,
cations in four large supply-chain companies and found
1996; Payne, 1982; Thaler & Sunstein, 2009). A number
evidence that the adjustments in a ‘negative’ direction of studies have investigated the design aspects of FSS,
improved the accuracy more than the adjustments in with varying success. One of the more straightforward
a ‘positive’ direction. This effect may be attributable to approaches is to change the look and feel of the FSS as
wishful thinking or optimism that may underlie positive well as its presentation style. Harvey and Bolger (1996)
adjustments. Adjustments that were ‘larger’ in magnitude found that trends were more easily discernible when
were also more beneficial in terms of the final forecast ac- the data was displayed in graphical rather than tabular
curacy than ‘smaller’ adjustments (Fildes et al., 2009). This format. Additionally, simple variations in presentation
may simply be because smaller adjustments are merely such as line graphs versus point graphs can alter accuracy
a sign of tweaking the numbers, but large adjustments (Theocharis, Smith, & Harvey, 2018). The functionalities
are carried out when there is a highly valid reason to of the FSS can also be modified (see Section 2.11.2).
make them. These findings have been confirmed in other Goodwin (2000b) investigated three ways of improving
studies (see, for example, Franses & Legerstee, 2009b; judgmental adjustment via changes in the FSS: a ‘no ad-
Syntetos, Nikolopoulos, Boylan, Fildes, & Goodwin, 2009). justment’ default, requesting forecasters specify the size
What are the main reasons behind judgmental adjust- of an adjustment rather than give a revised forecast, and
ments? Önkal and Gönül (2005) conducted a series of in- requiring a mandatory explanation for the adjustment.
terviews and a survey on forecasting practitioners (Gönül, Only the default option and the explanation feature were
Önkal, & Goodwin, 2009) to explore these. The main rea- successful in increasing the acceptance of the statistical
sons given were (i) to incorporate the practitioners’ in- forecast and so improving forecast accuracy.
tuition and experience about the predictions generated Goodwin et al. (2011) reported an experiment that
externally, (ii) to accommodate sporadic events and ex- investigated the effects of (i) ‘guidance’ in the form of pro-
ceptional occasions, (iii) to integrate confidential/insider viding information about when to make adjustments and
information that may have not been captured in the fore- (ii) ‘restriction’ of what the forecaster could do (e.g., pro-
casts, (iv) to hold responsibility and to gain control of the hibiting small adjustments). They found that neither re-
forecasting process, (v) to incorporate the expectations strictiveness nor guidance was successful in improving
and viewpoints of the practitioners, and (vi) to compen- accuracy, and both were met with resistance by the fore-
casters. While these studies focused on voluntary integra-
sate for various judgmental biases that are believed to
tion, Goodwin (2000a, 2002) examined the effectiveness
exist in the predictions. These studies also revealed that
of various methods of mechanical integration and con-
forecasting practitioners are very fond of judgmental ad-
cluded that automatic correction for judgmental biases by
justments and perceive them as a prominent way of ‘com-
the FSS was more effective than combining judgmental
pleting’ and ‘owning’ the predictions that are generated
and statistical inputs automatically with equal or varying
by others. weights. Another approach to mechanical integration was
While the first three reasons represent the integration investigated by Baecke et al. (2017). They compared ordi-
of an un-modelled component into the forecast, poten- nary judgmental adjustment with what they termed ‘‘in-
tially improving accuracy, the other reasons tend to harm tegrative judgment’’. This takes the judgmental informa-
accuracy rather than improve it. In such cases, the fore- tion into account as a predictive variable in the forecasting
cast would be better off if left unadjusted. Önkal and model and generates a new forecast. This approach im-
Gönül (2005) and Gönül et al. (2009) report that the proved accuracy. It also had the advantage that forecasters
occasions when forecasters refrain from adjustments are still had their input into the forecasting process and so
(i) when the practitioners are adequately informed and the resistance found by Goodwin et al. (2011) should not
knowledgeable about the forecasting method(s) that are occur. Finally, it is worth emphasising that an effective
used to generate the baseline forecasts, (ii) when there are FSS should not only improve forecast accuracy but should
accompanying explanations and convincing communica- also be easy to use, understandable, and acceptable (Fildes
tions that provide the rationale behind forecast method et al., 2006, see also Section 2.11.6 and Section 3.7.1).
801
3.7.4. Trust in forecasts146 objectivity and expertise. In parallel, forecast users face
Regardless of how much effort is poured into train- challenges in openly communicating their expectations
ing forecasters and developing elaborate forecast support from forecasts (Gönül et al., 2009), as well as their needs
systems, decision-makers will either modify or discard for explanations and other informational addendum to
the predictions if they do not trust them (see also Sec- gauge the uncertainties surrounding the forecasts. Organ-
tion 2.11.2, Section 2.11.6, Section 3.7.1, and Section 3.7.3). isational challenges include investing in forecast man-
Hence, trust is essential for forecasts to be actually used agement and designing resilient systems for collaborative
in making decisions (Alvarado-Valencia & Barrero, 2014; forecasting.
Önkal et al., 2019).
Given that trust appears to be the most important 3.7.5. Communicating forecast uncertainty147
attribute that promotes a forecast, what does it mean to Communicating forecast uncertainty is a critical issue
practitioners? Past work suggests that trusting a forecast in forecasting practice. Effective communication allows
is often equated with trusting the forecaster, their exper- forecasters to influence end-users to respond appropri-
tise and skills so that predictions could be used without ately to forecasted uncertainties. Some frameworks for
adjustment to make decisions (Önkal et al., 2019). It is effective communication have been proposed by decom-
argued that trust entails relying on credible forecasters posing the communication process into its elements: the
that make the best use of available information while communicator, object of uncertainty, expression format,
using correctly applied methods and realistic assump- audience, and its effect (National Research Council, 2006;
tions (Gönül et al., 2009) with no hidden agendas (Gönül, van der Bles, van der Linden, Freeman, Mitchell, Galvao,
Önkal, & Goodwin, 2012). Research suggests that trust Zaval, & Spiegelhalter, 2019).
is not only about trusting forecaster’s competence; users Forecasters have long studied part of this problem fo-
also need to be convinced that no manipulations are made cusing mostly in the manner by which we express forecast
for personal gains and/or to mislead decisions (Twyman, uncertainties. Gneiting and Katzfuss (2014) provides a
Harvey, & Harries, 2008). review of recent probabilistic forecasting methods (see
Surveys with practitioners show that key determinants also Section 2.12.4 and Section 2.12.5). Forecasting prac-
of trust revolve around (i) forecast support features and tice however revealed that numeracy skills and cognitive
tools (e.g., graphical illustrations, rationale for forecasts), load can often inhibit end-users from correctly interpret-
(ii) forecaster competence/credibility, (iii) forecast coming these uncertainties (Joslyn & Nichols, 2009; Raftery,
binations (from multiple forecasters/methods), and (iv) 2016). Attempts to improve understanding through the
forecast user’s knowledge of forecasting methods (Önkal use of less technical vocabulary also creates new chal-
et al., 2019). lenges. Research in psychology show that wording and
What can be done to enhance trust? If trust translates verbal representation play important roles in dissem-
into accepting guidance for the future while acknowledg- inating uncertainty (Joslyn, Nadav-Greenberg, Taing, &
ing and tolerating potential forecast errors, then both the Nichols, 2009). Generally forecasters are found to be con-
providers and users of forecasts need to work as part- sistent in their use of terminology, but forecast end-users
ners towards shared goals and expectations. Important often have inconsistent interpretation of these terms even
pathways to accomplish this include (i) honest communi- those commonly used (Budescu & Wallsten, 1985; Clark,
cation of forecaster’s track record and relevant accuracy 1990; Ülkümen, Fox, & Malle, 2016). Pretesting verbal ex-
targets (Önkal et al., 2019), (ii) knowledge sharing (Özer pressions and avoiding commonly misinterpreted terms
et al., 2011; Renzl, 2008) and transparency of forecasting are some easy ways to significantly reduce biases and
methods, assumptions and data (Önkal et al., 2019), (iii) improve comprehension.
communicating forecasts in the correct tone and jargon- Visualisations can also be powerful in communicating
free language to appeal to the user audience (Taylor & uncertainty. Johnson and Slovic (1995) and Spiegelhalter,
Thomas, 1982), (iv) users to be supported with forecasting Pearson, and Short (2011) propose several suggestions for
training (Merrick, Hardin, & Walker, 2006), (v) providing effective communication (e.g., multiple-format use, avoid-
explanations/rationale behind forecasts (Gönül, Önkal, & ing framing bias, and acknowledging limitations), but also
Lawrence, 2006; Önkal, Gönül, & Lawrence, 2008), (vi) recognise the limited amount of existing empirical evi-
presenting alternative forecasts under different scenarios dence. Some domain-specific studies do exist. For exam-
(see Section 2.11.5), and (vii) giving combined forecasts as ple, Riveiro, Helldin, Falkman, and Lebram (2014) showed
benchmarks (Önkal et al., 2019). uncertainty visualisation helped forecast comprehension
Trust must be earned and deserved (Maister et al., in a homeland security context.
2012) and is based on building a relationship that benefits With respect to the forecaster and her audience, is-
both the providers and users of forecasts. Take-aways sues such as competence, trust, respect, and optimism
for those who make forecasts and those who use them have been recently examined as a means to improve
converge around clarity of communication as well as per- uncertainty communication. Fiske and Dupree (2014) dis-
ceptions of competence and integrity. Key challenges for cusses how forecast recipients often infer apparent intent
forecasters are to successfully engage with users through- and competence from the uncertainty provided and use
out the forecasting process (rather than relying on a fore- these to judge trust and respect (see also Sections 2.11.6
cast statement at the end) and to convince them of their and 3.7.4 for discussion on trust and forecasting). This
146 This subsection was written by Dilek Önkal. 147 This subsection was written by Victor Richmond R. Jose.
802
suggests that the amount of uncertainty information pro- Traditionally, typical micro-economic demand drivers
vided should be audience dependent (Han, Klein, Lehman, (own price, competitors’ prices, and income) and some
Massett, Lee, & Freedman, 2009; Politi, Han, & Col, 2007). more tourism-specific demand drivers (source-market
Raftery (2016) acknowledges this by using strategies de- population, marketing expenditures, consumer tastes, habit
pending on the audience type (e.g., low-stakes user, risk persistence, and dummy variables capturing one-off events
avoider, etc.). Fischhoff and Davis (2014) suggests a simi- or qualitative characteristics) have been employed as pre-
lar approach by examining how people are likely to use dictors in tourism demand forecasting (Song et al., 2008).
the information (e.g., finding a signal, generating new One caveat of some of these economic demand drivers is
options, etc.) their publication lag and their low frequency, for instance,
When dealing with the public, experts assert that when real GDP (per capita) is employed as a proxy for
communicating uncertainty helps users understand fore- travellers’ income.
casts better and avoid a false sense of certainty (Morss, The use of leading indicators, such as industrial pro-
Demuth, & Lazo, 2008). Research however shows that duction as a leading indicator for real GDP (see also
hesitation to include forecast uncertainty exists among Section 3.3.2), has been proposed for short-term tourism
experts because it provides an opening for criticism and demand forecasting and nowcasting (Chatziantoniou, De-
the possibility of misinterpration by the public (Fischhoff, giannakis, Eeckels, & Filis, 2016). During the past couple
2012). This is more challenging when the public has of years, web-based leading indicators have also been
prior beliefs on a topic or trust has not been established. employed in tourism demand forecasting and have, in
Uncertainty can be used by individuals to reinforce a general, shown improvement in terms of forecast accu-
motivated-reasoning bias that allows them to ‘‘see what racy. However, this has not happened in each and every
they want to see’’ (Dieckmann, Gregory, Peters, & Hart- case, thereby confirming the traded wisdom that there is
man, 2017). Recent work however suggests that increas- no single best tourism demand forecasting approach (Li,
ing transparency for uncertainty does not necessarily Song, & Witt, 2005). Examples of those web-based lead-
affect trust in some settings. van der Bles, van der Linden, ing indicators include Google Trends indices (Bangwayo-
Freeman, and Spiegelhalter (2020) recently showed in Skeete & Skeete, 2015), Google Analytics indicators (Gunter
a series of experiments that people recognise greater & Önder, 2016), as well as Facebook ‘likes’ (Gunter, Önder,
uncertainty with more information but expressed only a & Gindl, 2019).
small decrease in trust in the report and trustworthiness The reason why these expressions of interaction of
of the source. users with the Internet have proven worthwhile as pre-
dictors in a large number of cases is that it is sensible to
3.8. Other applications assume potential travellers gather information about their
destination of interest prior to the actual trip, with the
3.8.1. Tourism demand forecasting148 Internet being characterised by comparably low search
As seen throughout 2020, (leisure) tourism demand costs, ergo allowing potential travellers to forage infor-
is very sensitive to external shocks such as natural and mation (Pirolli & Card, 1999) with only little effort (Zipf,
human-made disasters, making tourism products and ser- 2016). A forecaster should include this information in
vices extremely perishable (Frechtling, 2001). As the ma- their own set of relevant information at the forecast origin
jority of business decisions in the tourism industry require Lütkepohl (2005), if taking it into account results in an
reliable demand forecasts (Song, Witt, & Li, 2008), improv- improved forecast accuracy, with web-based leading in-
ing their accuracy has continuously been on the agenda of dicators thus effectively Granger-causing (Granger, 1969)
tourism researchers and practitioners alike. This contin- actual tourism demand (see Section 2.5.1).
uous interest has resulted in two tourism demand fore- Naturally, tourism demand forecasting is closely re-
casting competitions to date (Athanasopoulos et al., 2011; lated to aviation forecasting (see Section 3.8.2), as well as
Song & Li, 2021), the current one with a particular focus traffic flow forecasting (see Section 3.8.3). A sub-discipline
on tourism demand forecasting during the COVID-19 pan- of tourism demand forecasting can be found with hotel
demic (for forecasting competitions, see Section 2.12.7). room demand forecasting. The aforementioned perisha-
Depending on data availability, as well as on geographical bility of tourism products and services is particularly
aggregation level, tourism demand is typically measured evident for hotels as a hotel room not sold is lost revenue
in terms of arrivals, bed-nights, visitors, export receipts, that cannot be regenerated. Accurate hotel room demand
import expenditures, etc. forecasts are crucial for successful hotel revenue manage-
Since there are no specific tourism demand forecast ment (Pereira, 2016) and are relevant for planning pur-
models, standard univariate and multivariate statistical poses such as adequate staffing during MICE (i.e., Meet-
models, including common aggregation and combination ings, Incentives, Conventions, and Exhibitions/Events)
techniques, etc., have been used in quantitative tourism times, scheduling of renovation periods during low sea-
demand forecasting (see, for example, Jiao & Chen, 2019; sons, or balancing out overbookings and ‘‘no shows’’ given
Song, Qiu, & Park, 2019, for recent reviews). Machine constrained hotel room supply (Ivanov & Zhechev, 2012).
learning and other artificial intelligence methods, as well Particularly since the onset of the COVID-19 pandemic
as hybrids of statistical and machine learning models, in 2020, which has been characterised by global travel
have recently been employed more frequently. restrictions and tourism businesses being locked down to
varying extents, scenario forecasting and other forms of
148 This subsection was written by Ulrich Gunter. hybrid and judgmental forecasting played an important
803
role (Zhang, Song, Wen, & Liu, 2021, see Section 2.11.5), as exponential smoothing (see Section 2.3.1) and ARIMA
thereby highlighting an important limitation of quantita- (see Section 2.3.4) have been discussed in Botimer (1997),
tive tourism demand forecasting as currently practised. Sa (1987), and Wickham (1995). There are also studies
Based on the rapid development of information technol- focusing on disaggregate airline demand forecasting. For
ogy and artificial intelligence, Li and Jiao (2020), however, example, Martinez and Sanchez (1970) apply empirical
envisage a ‘‘super-smart tourism forecasting system’’ (Li & probability distributions to predict bookings and can-
Jiao, 2020, p. 264) for the upcoming 75 years of tourism cellations of individual passengers travelling with Iberia
demand forecasting. According to these authors, this sys- Airlines. Carson, Cenesizoglu, and Parker (2011) show
tem will be able to automatically produce forecasts at the that aggregating the forecasts of individual airports using
micro level (i.e., for the individual traveller and tourism airport-specific data could provide better forecasts at a
business) in real time while drawing on a multitude of national level. More recently, machine learning meth-
data sources and integrating multiple (self-developing) ods have also been introduced to generate forecasts for
forecast models. airlines. This can be seen in Weatherford, Gentry, and
Wilamowski (2003) where they apply neural networks
3.8.2. Forecasting for aviation149 to forecast the time series of the number of reservations.
Airports and airlines have long invested in forecasting Moreover, Hopman, Koole, and van der Mei (2021) show
arrivals and departures of aircrafts. These forecasts are that an extreme gradient boosting model which forecasts
important in measuring airspace and airport congestions, itinerary-based bookings using ticket price, social media
designing flight schedules, and planning for the assign- posts and airline reviews outperforms traditional time
ment of stands and gates (Barnhart & Cohn, 2004). Various series forecasts.
techniques have been applied to forecast aircrafts’ arrivals Forecasting passenger arrivals and delays in the air-
and departures. For instance, Rebollo and Balakrishnan ports have received also some attention in the litera-
(2014) apply random forests to predict air traffic delays ture, particularly in the past decade. Wei and Hansen
of the National Airspace System using both temporal and (2006) build an aggregate demand model for air passenger
network delay states as covariates. Manna, Biswas, Kundu, traffic in a hub-and-spoke network. The model is a log-
Rakshit, Gupta, and Barman (2017) develop a statistical linear regression that uses airline service variables such
model based on a gradient boosting decision tree to pre- as aircraft size and flight distance as predictors. Barnhart,
dict arrival and departure delays, using the data taken Fearing, and Vaze (2014) develop a multinomial logit
from the United States Department of Transportation (Bu- regression model, designed to predict delays of US do-
reau of Transportation Statistics, 2020). Rodríguez-Sanz,
mestic passengers. Their study also uses data from the US
Comendador, Valdés, Pérez-Castán, Montes, and Serrano
Department of Transportation (Bureau of Transportation
(2019) develop a Bayesian Network model to predict
Statistics, 2020). Guo, Grushka-Cockayne, and De Reyck
flight arrivals and delays using the radar data, aircraft his-
(2020) recently develop a predictive system that gener-
torical performance and local environmental data. There
ates distributional forecasts of connection times for trans-
are also a few studies that have focused on generating
fer passengers at an airport, as well as passenger flows
probabilistic forecasts of arrivals and departures, mov-
at the immigration and security areas. Their approach is
ing beyond point estimates. For example, Tu, Ball, and
based on the application of regression trees combined
Jank (2008) develop a predictive system for estimating
with copula-based simulations. This predictive system has
flight departure delay distributions using flight data from
been implemented at Heathrow airport since 2017.
Denver International Airport. The system employs the
With an increasing amount of available data that is as-
smoothing spline method to model seasonal trends and
sociated with activities in the aviation industry, predictive
daily propagation patterns. It also uses mixture distribu-
analyses and forecasting methods face new challenges as
tions to estimate the residual errors for predicting the
well as opportunities, especially in regard to updating
entire distribution.
In the airline industry, accurate forecasts on demand forecasts in real time. The predictive system developed
and booking cancellations are crucial to revenue manage- by Guo et al. (2020) is able to generate accurate forecasts
ment, a concept that was mainly inspired by the airline using real-time flight and passenger information on a
and hotel industries (Lee, 1990; McGill & Van Ryzin, 1999, rolling basis. The parameters of their model, however, do
see also Section 3.8.1 for a discussion on hotel occu- not update over time. Therefore, a key challenge in this
pancy forecasting). The proposals of forecasting models area is for future studies to identify an efficient way to
for flight demand can be traced back to Beckmann and dynamically update model parameters in real time.
Bobkoski (1958), where these authors demonstrate that
Poisson and Gamma models can be applied to fit airline 3.8.3. Traffic flow forecasting150
data. Then, the use of similar flights’ short-term booking Traffic flow forecasting is an important task for traffic
information in forecasting potential future bookings has management bodies to reduce traffic congestion, perform
been discussed by airline practitioners such as Adams and planning and allocation tasks, as well as for travelling
Michael (1987) at Quantas as well as Smith, Leimkuh- individuals to plan their trips. Traffic flow is complex
ler, and Darrow (1992) at American Airlines. Regressions spatial and time-series data exhibiting multiple season-
models (see Section 2.3.2) and time series models such alities and affected by spatial exogenous influences such
149 This subsection was written by Xiaojia Guo. 150 This subsection was written by Alexander Dokumentov.
804
as social and economic activities and events, various gov- volume or call arrival rates is an important part of call
ernment regulations, planned road works, weather, traffic centre management.
accidents, etc. (Polson & Sokolov, 2017). There are several properties to call arrival data. De-
Methods to solve traffic flow forecasting problems pending on the level of aggregation and the frequency
vaguely fall into three categories. The first uses paramet- with which data is collected, e.g., hourly, call arrival
ric statistical methods such as ARIMA, seasonal ARIMA, data may exhibit intraday (within-day), intraweek, and
space–time ARIMA, Kalman filters, etc. (see, for example, intrayear multiple seasonal patterns (Avramidis, Deslau-
Kamarianakis & Prastacos, 2005; Vlahogianni, Golias, & riers, & L’Ecuyer, 2004; Brown et al., 2005a, and Sec-
Karlaftis, 2004; Vlahogianni, Karlaftis, & Golias, 2014; tion 2.3.5). In addition, arrival data may also exhibit inter-
Whittaker, Garside, & Lindveld, 1997). The second set day and intraday dependencies, with different time peri-
of approaches uses purely of neural networks (Mena- ods within the same day, or across days within the same
Oreja & Gozalvez, 2020). The third group of methods week, showing strong levels of autocorrelation (Brown
uses various machine learning, statistical non-parametric et al., 2005a; Shen & Huang, 2005; Tanir & Booth, 1999).
techniques or mixture of them (see, for example, Hong, Call arrivals may also be heteroscedastic with variance
2011; Zhang, Qi, Henrickson, Tang, & Wang, 2017; Zhang, at least proportional to arrival counts (Taylor, 2008), and
Zou, Tang, Ash, & Wang, 2016, but also Section 2.7.8 and overdispersed under a Poisson assumption having vari-
Section 2.7.10 for an overview of NN and ML methods). ance per time period typically much larger than its ex-
Although neural networks are probably the most pected value (Avramidis et al., 2004; Jongbloed & Koole,
promising technique for traffic flow forecasting (see, for 2001; Steckley, Henderson, & Mehrotra, 2005). These prop-
example, Do, Vu, Vo, Liu, & Phung, 2019; Polson & Sokolov, erties have implications for various approaches to mod-
elling and forecasting call arrivals.
2017), statistical techniques, such as Seasonal-Trend de-
The first family of methods are time series meth-
composition based on Regression (STR, see Section 2.2.2),
ods requiring no distributional assumptions. Early studies
can outperform when little data is available or they can be
employed auto regressive moving average (ARMA; see
used for imputation, de-noising, and other pre-processing
Section 2.3.4) models (Andrews & Cunningham, 1995;
before feeding data into neural networks which often
Antipov & Meade, 2002; Tandberg, Easom, & Qualls, 1995;
become less powerful when working with missing or very
Xu, 1999), exponential smoothing (Bianchi, Jarrett, & Hanu-
noisy data.
mara, 1993, 1998, see Section 2.3.1), fast Fourier trans-
Traffic flow forecasting is illustrated below using ve-
forms (Lewis, Herbert, & Bell, 2003), and regression (Tych,
hicle flow rate data from road camera A1.GT.24538 on
Pedregal, Young, & Davies, 2002, see Section 2.3.2). The
A1 highway in Luxembourg (des Mobilités, 2020) from
first methods capable of capturing multiple seasonality
2019-11-19 06:44:00 UTC to 2019-12-23 06:44:00 UTC.
were evaluated by Taylor (2008) and included double
Most of the data points are separated by 5 min intervals. seasonal exponential smoothing (Taylor, 2003b) and mul-
Discarding points which do not follow this schedule leads tiplicative double seasonal ARMA (SARMA). Since then
to a data set where all data points are separated by 5 min several advanced time series methods have been devel-
intervals, although values at some points are missing. oped and evaluated (De Livera et al., 2011; Taylor, 2010;
The data is split into training and test sets by setting Taylor & Snyder, 2012), including artificial neural net-
aside last 7 days of data. As Hou, Edara, and Sun (2014) works (Li, Huang, & Gong, 2011; Millán-Ruiz & Hidalgo,
and Polson and Sokolov (2017) suggest, spatial factors 2013; Pacheco, Millán-Ruiz, & Vélez, 2009) and models
are less important for long term traffic flow forecasting, for density forecasting (Taylor, 2012).
and therefore they are not taken into account and only Another family of models relies on the assumption
temporal data is used. Application of STR (Dokumentov, of a time-inhomogeneous Poisson process adopting fixed
2017) as a forecasting technique to the log transformed (Brown et al., 2005a; Jongbloed & Koole, 2001; Shen &
data leads to a forecast with Mean Squared Error 102.4, Huang, 2008a; Taylor, 2012) and mixed modelling (Aldor-
Mean Absolute Error 62.8, and Mean Absolute Percent- Noiman, Feigin, & Mandelbaum, 2009; Avramidis et al.,
age Error (MAPE) 14.3% over the test set, outperforming 2004; Ibrahim & L’Ecuyer, 2013) approaches to account
Double-Seasonal Holt-Winters by 44% in terms of MAPE. for the overdispersed nature of the data, and in some
The decomposition and the forecast obtained by STR are cases, interday and intraday dependence.
shown on Fig. 15 and the magnified forecast and the The works by Soyer and Tarimcilar (2008) and Wein-
forecasting errors are on Fig. 16. berg, Brown, and Stroud (2007) model call volumes from
a Bayesian point of view. Other Bayesian inspired ap-
3.8.4. Call arrival forecasting151 proaches have been adopted mainly for estimating var-
Forecasting of inbound call arrivals for call centres ious model parameters, but also allowing for intraday
supports a number of key decisions primarily around updates of forecasts (Aktekin & Soyer, 2011; Landon, Rug-
staffing (Aksin, Armony, & Mehrotra, 2007). This typ- geri, Soyer, & Tarimcilar, 2010).
A further class of approach addresses the dimension-
ically involves matching staffing level requirements to
ality challenge related to high frequency call data using
service demand as summarised in Fig. 17. To achieve
Singular Value Decomposition (SVD). Shen and Huang
service level objectives, an understanding of the call load
is required in terms of the call arrivals (Gans, Koole, &
Mandelbaum, 2003). As such, forecasting of future call 151 This subsection was written by Devon K. Barrow.
805
Fig. 15. STR decomposition of the log transformed training data and the forecasts for the traffic flow data.
Fig. 16. Left: forecast (red) and the test data (black); Right: the prediction error over time for the traffic flow data. (For interpretation of the
references to colour in this figure legend, the reader is referred to the web version of this article.)
Fig. 17. The staffing decision process in call centres.
(2005) and Shen and Huang (2008a) use the same tech- mechanism. Several further studies have extended the
nique to achieve dimensionality reduction of arrival data, basic SVD approach to realise further modelling innova-
and to create a forecasting model that provides both in- tions, for example, to forecast call arrival rate profiles and
terday forecasts of call volume, and an intraday updating generate smooth arrival rate curves (Shen, 2009; Shen &
806
Huang, 2008b; Shen, Huang, & Lee, 2007). A more com- & Wlezien, 2012; Williams & Reade, 2016; Wolfers &
prehensive coverage of different forecasting approaches Zitzewitz, 2004), even studying their links with opinion
for call arrival rate and volume can be found in a recent polls (Brown, Reade, & Vaughan Williams, 2019). Practice
review paper by Ibrahim, Ye, L’Ecuyer, and Shen (2016). has also developed econometric models (Fair, 1978) that
exploit structural information available months before
3.8.5. Elections forecasting152 the election (e.g., the evolution of the economy or the
With the exception of weather forecasts, there are few incumbent popularity). Lewis-Beck has had great success
forecasts which have as much public exposure as election in publishing dozens of papers using this approach (see,
forecasts. They are frequently published by mass media, e.g., Lewis-Beck, 2005).
with their number and disclosure reaching a frenzy as Special mention also goes to Election-Day forecast-
the Election Day approaches. This explains the signif- ing strategies, which have been systematically commis-
icant amount of methods, approaches and procedures sioned since the 1950s (Mitofsky, 1991). Exit (and en-
proposed and the paramount role these forecasts play in trance) polls (Klofstad & Bishin, 2012; Pavía, 2010), quick-
shaping people’s confidence in (soft/social) methods of counts (Pavía-Miralles & Larraz-Iribas, 2008), and statis-
forecasting. tical models (Bernardo, 1984; Moshman, 1964; Pavía-
The problem escalates because, regardless whether the Miralles, 2005) have been used to anticipate outcomes
goal of the election forecast is an attempt to ascertain on Election Day. Some of these strategies (mainly ran-
the winner in two-choice elections (e.g., a referendum or dom quick-counts) can be also employed as auditing tools
a Presidential election) or to reach estimates within the to disclose manipulation and fraud in weak democra-
margins of error in Parliamentary systems, the knowl- cies (Scheuren & Alvey, 2008).
edge of the forecasts influences electors’ choices (Pavía,
Gil-Carceller, Rubio-Mataix, Coll, Alvarez-Jareño, Aybar, & 3.8.6. Sports forecasting153
Carrasco-Arroyo, 2019). Election forecasts not only affect
Forecasting is inherent to sport. Strategies employed
voters but also political parties, campaign organizations
by participants in sporting contests rely on forecasts, and
and (international) investors, who are also watchful of
the decision by promoters to promote, and consumers
their evolution.
to attend such events are conditioned on forecasts: pre-
Scientific approaches to election forecasting include
dictions of how interesting the event will be. First in
polls, information (stock) markets and statistical models.
this section, we look at forecast competitions in sport,
They can also be sorted by when they are performed;
and following this we consider the role forecasts play in
and new methods, such as social media surveillance (see
sporting outcomes.
also Section 2.9.3), are also emerging (Ceron, Curini, &
Forecast competitions are common; see Section 2.12.7.
Iacus, 2016; Huberty, 2015). Probabilistic (representative)
Sport provides a range of forecast competitions, perhaps
polls are the most commonly used instrument to gauge
most notably the competition between bookmakers and
public opinions. The progressive higher impact of non-
their customers – betting. A bet is a contingent contract,
sampling errors (coverage issues, non-response bias, mea-
a contract whose payout is conditional on specified fu-
surement error: Biemer, 2010) is, however, severely test-
ture events occurring. Bets occur fundamentally because
ing this approach. Despite this, as Kennedy, Wojcik, and
two agents disagree about the likelihood of that event
Lazer (2017) show in a recent study covering 86 coun-
tries and more than 500 elections, polls are still pow- occurring, and hence it is a forecast.
erful and robust predictors of election outcomes after Bookmakers have been extensively analysed as fore-
adjustments (see also Jennings, Lewis-Beck, & Wlezien, casters; Forrest, Goddard, and Simmons (2005) evalu-
2020). The increasing need of post-sampling adjustments ated biases in the forecasts implied by bookmaker odds
of probabilistic samples has led to a resurgence of interest over a period where the betting industry became more
in non-probabilistic polls (Elliott & Valliant, 2017; Pavía & competitive, and found that relative to expert forecasts,
Larraz, 2012; Wang, Rothschild, Goel, & Gelman, 2015), bookmaker forecasts improved.
abandoned in favour of probabilistic sampling in 1936, With the internet age, prediction markets have
when Gallup forecasted Roosevelt’s triumph over Landon emerged, financial exchanges where willing participants
using a small representative sample despite Literacy Di- can buy and sell contingent contracts. In theory, such de-
gest failing to do so with a sample of near 2.5 million centralised market structures ought to provide the most
responses (Squire, 1988). efficient prices and hence efficient forecasts (Nordhaus,
A person knows far more than just her/his voting 1987). A range of papers have tested this in the sport-
intention (Rothschild, 2009) and when s/he makes a bet, ing context (Angelini & De Angelis, 2019; Croxson &
the rationality of her/his prediction is reinforced because Reade, 2014; Gil & Levitt, 2007), with conclusions tending
s/he wants to win. Expectation polls try to exloit the towards a lack of efficiency.
first issue (Graefe, 2014), while prediction markets, as Judgmental forecasts by experts are commonplace too
efficient aggregators of information, exploit both these (see also Section 2.11); traditionally in newspapers, but
issues to yield election forecasts (see also Sections 2.6.4 more recently on television and online. Reade, Singleton,
and 2.11.4). Several studies have proven the performance and Brown (2020) evaluate forecasts of scorelines from
of these approaches (Berg, Nelson, & Rietz, 2008; Erikson two such experts against bookmaker prices, a statistical
152 This subsection was written by Jose M. Pavía. 153 This subsection was written by J. James Reade.
807
model, and the forecasts from users of an online fore- a need megaprojects especially those that deliver social
casting competition. Singleton, Reade, and Brown (2019) and economic goods and create economic growth (Flyvb-
find that when forecasters in the same competition revise jerg, Bruzelius, & Rothengatter, 2003). Typical features of
their forecasts, their forecast performance worsens. This megaprojects include some or all the following: (i) deliv-
forecasting competition is also analysed by Butler, Butler, ering a substantial piece of physical infrastructure with a
and Eakins (2020) and Reade et al. (2020). life expectancy that spans across decades, (ii) main con-
Sport is a spectacle, and its commercial success is contractor or group of contractors are privately owned and fi-
ditioned on this fact. Hundreds of millions of people glob- nanced, and (iii) the contractor could retain an ownership
ally watch events like the Olympics and the FIFA World stake in the project and the client is often a government
Cup – but such interest is conditioned on anticipation, a or public sector organisation (Sanderson, 2012).
forecast that something interesting will happen. A super- However, megaprojects are heavily laced with extreme
star is going to be performing, the match will be a close human and technical complexities making their deliv-
encounter, or it will matter a lot for a bigger outcome (the ery and implementation difficult and often unsuccess-
championship, say). These are the central tenets of sport ful (Merrow, McDonnell, & Arguden, 1988; The R.F.E.
economics back to Neale (1964) and Rottenberg (1956), Working Group Report, 2015). This is largely due to the
most fundamentally the ‘uncertainty of outcome hypoth- challenge of managing megaprojects including extreme
complexity, increased risk, tight budget and deadlines,
esis’. A multitude of sport attendance prediction studies
lofty ideals (Fiori & Kovaka, 2005). Due to the possibility
investigate this (see, for example, Coates & Humphreys,
and consequences of megaproject failure (Mišić & Radu-
2010; Forrest & Simmons, 2006; Hart, Hutton, & Sharot,
jković, 2015), forecasting the outcomes of megaprojects
1975; Sacheti, Gregory-Smith, & Paton, 2014; van Ours,
is becoming of growing importance. In particular, it is
2021), and Van Reeth (2019) considers this for forecasting
crucial to identify and assess the risks and uncertain-
TV audiences for the Tour de France.
ties as well as other factors that contribute to disap-
Cities and countries bid to host large events like the
pointing outcomes of megaprojects in order to mitigate
World Cup based on forecasts regarding the impact of them (Flyvbjerg et al., 2003; Miller & Lessard, 2007).
hosting such events. Forecasts that are often inflated Literature review in forecasting in megaprojects is
for political reasons (Baade & Matheson, 2016). Equally, scarce. However, there are a few themes that have
franchise-based sports like many North American sports emerged in the extant literature as characteristics of
attract forecasts regarding the impact of a team locat- megaprojects that should be skilfully managed to provide
ing in a city, usually resulting in public subsidies for a guideline for the successful planning and construction
the construction of venues for teams to play at Coates of megaprojects (Fiori & Kovaka, 2005; Flyvbjerg, 2007;
and Humphreys (1999). Governments invest in sporting Sanderson, 2012). Turner and Zolin (2012) even claim
development, primarily to achieve better performances that we cannot even properly define what success is.
at global events, most notably the Olympics (Bernard & They argue that we need to reliable scales in order to
Busse, 2004). predict multiple perspectives by multiple stakeholders
Many sporting events themselves rely on forecasts to over multiple time frames — so definitely a very diffi-
function; high jumpers predict what height they will be cult long term problem. This could be done via a set of
able to jump over, and free diving contestants must state leading performance indicators that will enable managers
the depth they will dive to. Less formally, teams will set of Megaprojects to forecast during project execution how
themselves goals: to win matches, to win competitions, various stakeholders will perceive success months or even
to avoid the ‘wooden spoon’. Here, forecast outcomes are years into the operation. At the very early stages of a
influenced by the teams, and competitors, taking part in project’s lifecycle, a number of decisions must been taken
competitions and, as such, are perhaps less commonly and are of a great importance for the performance and
thought of as genuine forecasts. Important works pre- successful deliverables/outcomes. Flyvbjerg (2007) stress
dicting outcomes range from Dixon and Coles (1997) in the importance of the front-end considerations particu-
soccer, to Kovalchik and Reid (2019) for tennis, while the larly for Megaprojects Failure to account for unforeseen
increasing abundance of data means that machine learn- events frequently lead to cost overruns.
ing and deep learning methods are beginning to domi- Litsiou et al. (2019) suggest that forecasting the suc-
nate the landscape. See, for example, Hubáček, Šourek, cess of megaprojects is particularly a challenging and
and Železnỳ (2019) and Maymin (2019) for basketball, critical task due to the characteristics of such projects.
and Mulholland and Jensen (2019) for NFL. Megaproject stakeholders typically implement impact as-
sessments and/or cost benefit Analysis tools (Litsiou et al.,
3.8.7. Forecasting for megaprojects154 2019). As Makridakis, Hogarth, and Gaba (2010) sug-
Megaprojects are significant activities characterised by gested, judgmental forecasting is suitable where quan-
titative data is limited, and the level of uncertainty is
a multi-organisation structure, which produces highly vis-
very high; elements that we find in megaprojects. By
ible infrastructure or asset with very crucial social im-
comparing the performance of three judgmental methods,
pacts (Aaltonen, 2011). Megaprojects are complex, require
unaided judgment, semi-structured analogies (sSA), and
huge capital investment, several stakeholders are identi-
interaction groups (IG), used by a group of 69 semi-
fied and, usually a vast number of communities and the
experts, Litsiou et al. (2019) found that, the use of sSA out-
public are the receivers of the project’s benefits. There is
performs unaided judgment in forecasting performance
(see also Section 2.11.4). The difference is amplified fur-
154 This subsection was written by Konstantia Litsiou. ther when pooling of analogies through IG is introduced.
808
3.8.8. Competing products155 cassettes and compact discs for pre-recorded music in
Competition among products or technologies affects the US market. Obtained results of LVch outperform re-
prediction due to local systematic deviations and saturat- stricted and unrestricted UCRCD analyses. In this context
ing effects related to policies, and evolving interactions. the residual market is not perfectly accessible to both
The corresponding sales time series must be jointly mod- competitors and this fact, combined with WOM com-
elled including the time varying reciprocal influence. Fol- ponents, allows for better interpretation and forecasting
lowing the guidelines in subsection Section 2.3.20, some especially in medium and long-term horizons.
examples are reported below. A further application of the LVch model, Lotka–Volterra
Based on IMS-Health quarterly number of cimetidine with asymmetric churn (LVac), is proposed in Guidolin
and ranitidine packages sold in Italy, the CRCD model and Guseo (2020). It is based on a statistical reduction:
(Guseo & Mortarino, 2012) was tested to evaluate a di- The late entrant behaves as a standard Bass (1969) model
achronic competition that produced substitution. Cimeti- that modifies the dynamics and the evolution of the
dine is a histamine antagonist that inhibits the production first entrant in a partially overlapped market. The case
of stomach acid and was introduced by Smith, Kline & study is offered by a special form of competition where
French in 1976. Ranitidine is an alternative active prin- the iPhone produced an inverse cannibalisation of the
ciple introduced by Glaxo in 1981 and was found to have iPad. The former suffered a local negative interaction
far-improved tolerability and a longer-lasting action. The with some benefits: A long-lasting life cycle and a larger
main effect in delayed competition is that the first com- market size induced by the iPad.
pound spread fast but was suddenly outperformed by the A limitation in models for diachronic competition re-
new one principle that modified its stand-alone regime. lates to high number of rivals, implying complex para-
Guseo and Mortarino (2012) give some statistical and metric representations with respect to the observed in-
forecasting comparisons with the restricted Krishnan- formation. A second limitation, but also an opportunity,
Bass-Kummar Diachronic model (KBKD) by Krishnan et al. is the conditional nature of forecasting if the processes
(2000). Previous results were improved with the UCRCD partially depend upon exogenous control functions (new
model in Guseo and Mortarino (2014) by considering a policy regulations, new radical innovations, regular and
decomposition of word-of-mouth (WOM) effects in two promotional prices, etc.). These tools may be used to
parts: within-brand and cross-brand contributions. The simulate the effect of strategic interventions, but a lack of
new active compound exploited a large cross-brand WOM knowledge of such future policies may affect prediction.
and a positive within-brand effect. After the start of com-
petition, cimetidine experienced a negative WOM effect 3.8.9. Forecasting under data integrity attacks157
from its own adopters and benefited from the increase of Data integrity attacks, where unauthorized parties ac-
the category’s market potential driven by the antagonist. cess protected or confidential data and inject false infor-
Forecasting is more realistic with the UCRCD approach mation using various attack templates such as ramping,
and it avoids mistakes in long-term prediction. scaling, random attacks, pulse and smooth-curve, has be-
Restricted and unrestricted UCRCD models were ap- come a major concern in data integrity control in forecast-
plied in Germany by Guidolin and Guseo (2016) to the ing (Giani, Bitar, Garcia, McQueen, Khargonekar, & Poolla,
competition between nuclear power technologies and re- 2013; Singer & Friedman, 2014; Sridhar & Govindarasu,
newable energy technologies (wind and solar; see also 2014; Yue, 2017).
Sections 3.4.5, 3.4.6 and 3.4.8) in electricity production. Several previous studies have given attention in
Due to the ‘Energiewende’ policy started around 2000, the anomaly detection pre-processing step in forecasting work-
substitution effect, induced by competition, is confirmed flow with varying degree of emphasis. However, accord-
by the electricity production data provided by BP.156 An ing to Yue (2017), the detection of data integrity attacks
advance is proposed in Furlan, Mortarino, and Zahangir is very challenging as such attacks are done by highly
(2020) with three competitors (nuclear power, wind, and skilled adversaries in a coordinated manner without no-
solar technologies) and exogenous control functions ob- table variations in the historical data patterns (Liang, He,
taining direct inferences that provide a deeper analy- & Chen, 2019). These attacks can cause over-forecasts
sis and forecasting improvements in energy transition that demand unnecessary expenses for the upgrade and
context. maintenance, and can eventually lead to poor planning
Previous mentioned intersections between Lotka– and business decisions (Luo et al., 2018a; Luo, Hong, &
Volterra approach and diffusion of innovations competi- Fang, 2018b; Wu, Yu, Cui, & Lu, 2020).
tion models suggested a more modulated access to the Short-term load forecasting (see Section 3.4.3) is one
residual carrying capacity. The Lotka–Volterra with churn major field that are vulnerable to malicious data integrity
model (LVch) by Guidolin and Guseo (2015) represents attacks as many power industry functions such as eco-
‘churn effects’ preserving within and cross-brand effects nomic dispatch, unit commitment and automatic gen-
in a synchronic context. eration control heavily depend on accurate load fore-
An application of LVch model is discussed with ref- casts (Liang et al., 2019). The cyberattack on U.S. power
erence to the competition/substitution between compact grid in 2018 is one such major incident related to the
topic. According to the study conducted by Luo et al.
155 This subsection was written by Renato Guseo. (2018a), the widely used load forecasting models fail to
156 https://www.bp.com/en/global/corporate/energy-economics/
statistical-review-of-world-energy.html (Accessed: 2020-09-01). 157 This subsection was written by Priyanga Dilini Talagala.
809
produce reliable load forecast in the presence of such applied forecasting. The literature on the latter connection
large malicious data integrity attacks. A submission to can consider many different aspects in modelling as we
the Global Energy Forecasting Competition 2014 (GEF- illustrate below. We can identify two literature strands,
Com2014) incorporated an anomaly detection a much larger one on the various connections of energy
pre-processing step with a fixed anomalous threshold to with commodities and the agricultural sector (and in this
their load forecasting framework (Xie & Hong, 2016). The strand we include forecasting agricultural series) and a
method was later improved by Luo, Hong, Yue (2018) by smaller one that explores the issue of common factors.
replacing the fixed threshold with a data driven anoma- An early reference of the impact of energy on the agri-
lous threshold. Sridhar and Govindarasu (2014) also pro- cultural sector is Tewari (1990) and then after a decade
posed a general framework to detect scaling and ramp we find Cohin and Chantret (2010) on the long-run impact
attacks in power systems. Akouemo and Povinelli (2016) of energy prices on global agricultural markets.
investigated the impact towards the gas load forecasting Byrne, Fazio, and Fiess (2013) is an early reference for co-
using hybrid approach based on Bayesian maximum like- movement of commodity prices followed by
lihood classifier and a forecasting model. In contrast to Daskalaki, Kostakis, and Skiadopoulos (2014) on com-
the previous model based attempts, Yue, Hong, and Wang mon factors of commodity future returns and then a
(2019) proposed a descriptive analytic-based approach very recent paper from Alquist, Bhattarai, and Coibion
to detect cyberattacks including long anomalous sub- (2020) who link global economic activity with commod-
sequences (see Section 2.2.3), that are difficult to detect ity price co-movement. The impact of energy shocks on
by the conventional anomaly detection methods. US agricultural productivity was investigated by Wang
The problem of data integrity attacks is not limited and McPhail (2014) while Koirala, Mishra, D’Antoni, and
to load forecasting. Forecasting fields such as elections Mehlhorn (2015) explore the non-linear correlations of
forecasting (see Section 3.8.5), retail forecasting (see Sec- energy and agricultural prices with Albulescu, Tiwari,
tion 3.2.4), airline flight demand forecasting (see Sec- and Ji (2020) exploring the latter issue further, the last
tion 3.8.2) and stock price forecasting (see Section 3.3.13) two papers using copulas. Xiong, Li, Bao, Hu, and Zhang
are also vulnerable to data integrity attacks (Luo et al., (2015) is an early reference of forecasting agricultural
2018a; Seaman, 2018). For instant, Wu et al. (2020) ex- commodity prices while Kyriazi et al. (2019), Li, Li, Liu,
plored the vulnerability of traffic modelling and forecast- Zhu and Wei (2020), and Wang, Wang, Li, and Zhou (2019)
ing in the presence of data integrity attacks with the consider three novel and completely different approaches
aim of providing useful guidance for constrained network on forecasting agricultural prices and agricultural futures
resource planning and scheduling.
returns. López Cabrera and Schulz (2016) explore volatil-
However, despite of the increasing attention toward
ity linkages between energy and agricultural commodity
the topic, advancements in cyberattacks on critical in-
prices and then Tian, Yang, and Chen (2017) start a mini-
frastructure raise further data challenges. Fooling existing
stream on volatility forecasting on agricultural series fol-
anomaly detection algorithms via novel cyberattack tem-
lowed among others by the work of Luo, Klein, Ji, and
plates is one such major concern. In response to the
Hou (2019) and of Degiannakis, Filis, Klein, and Walther
above concern, Liang et al. (2019) proposed a data poi-
(2020). de Nicola, De Pace, and Hernandez (2016) exam-
soning algorithm that can fool existing load forecasting
ine the co-movement of energy and agricultural returns
approaches with anomaly detection component while de-
while Kagraoka (2016) and Lübbers and Posch (2016) ex-
manding further investigation into advanced anomaly de-
amine common factors in commodity prices. Pal and Mitra
tection methods. Further, adversaries can also manipulate
(2019) and Wei Su, Wang, Tao, and Oana-Ramona (2019)
other related input data without damaging the target
both investigate the linkages of crude oil and agricultural
data series. Therefore, further research similar to (Sobhani
prices. Finally, Tiwari, Nasreen, Shahbaz, and Hammoudeh
et al., 2020) are required to handle such data challenges.
(2020) examine the time-frequency causality between
3.8.10. The forecastability of agricultural time series158 various commodities, including agricultural and metals.
The forecasting of agricultural time series falls under There is clearly room for a number of applications in
the broader group of forecasting commodities, of which the context of this recent research, such along the lines
agricultural and related products are a critical subset. of further identifying and then using common factors in
While there has been considerable work in the economet- constructing forecasting models, exploring the impact of
rics and forecasting literature on common factor models the COVID-19 crisis in agricultural production or that of
in general there is surprisingly little work so far on the climate changes on agricultural prices.
application of such models for commodities and agri-
cultural time series – and this is so given that there 3.8.11. Forecasting in the food and beverage industry159
is considerable literature in the linkage between energy Reducing the ecological impact and waste, and increas-
and commodities, including agricultural products, their ing the efficiency of the food and beverage industry are
prices and futures prices, their returns and volatilities. currently major worldwide issues. To this direction, effi-
Furthermore, a significant number of papers is fairly re- cient and sustainable management of perishable food and
cent which indicates that there are many open avenues the control of the beverage quality is of paramount impor-
of future research on these topics, and in particular for tance. A particular focus on this topic is placed on supply
158 This subsection was written by Dimitrios Thomakos. 159 This subsection was written by Daniele Apiletti.
810
chain forecasting (see Section 3.2.2), with advanced mon- a low-quality coffee due to excessive percolation time. If
itoring technologies able to track the events impacting the amount of coffee ground is low, the ground is coarse,
and affecting the food and beverage processes (La Scalia, and the pressure is high, then we can forecast a low-
Micale, Miglietta, & Toma, 2019). Such technologies are quality coffee due to excessive flow rate. Furthermore, the
typically deployed inside manufacturing plants, yielding coarseness of coffee ground generates an excessive flow
to Industry 4.0 solutions (Ojo, Shah, Coutroubis, Jiménez, rate forecast, despite the optimal values of dosage and
& Ocana, 2018) that are enabled by state-of-the-art fore- pressure, with very high confidence.
casting applications in smart factories. The transition from
plain agriculture techniques to smart solutions for food 3.8.12. Dealing with logistic forecasts in practice160
processing is a trend that fosters emerging forecasting
The forecaster faces three major difficulties when us-
data-driven solutions in many parts of the world, with
ing the logistic equation (S curve); see also Section 2.3.19.
special attention to the sustainability aspects (Zailani, Je-
A first dilemma is whether he or she should fit an S
yaraman, Vengadasan, & Premkumar, 2012).
curve to the cumulative number or to the number per
Various forecasting approaches have been successfully
unit of time. Here the forecaster must exercise wise judg-
applied in the context of the food and beverage indus-
ment. What is the ‘‘species’’ and what is the niche that is
try, from Monte Carlo simulations based on a shelf-life
being filled? To the frustration of business people there
model (La Scalia et al., 2019), to association rule min-
is no universal answer. When forecasting the sales of
ing (see Section 2.9.2) applied to sensor-based equip-
a new product it is often clear that one should fit the
ment monitoring measurements (Apiletti & Pastor, 2020),
cumulative sales because the product’s market niche is
multi-objective mathematical models for perishable sup-
ply chain configurations, forecasting costs, delivery time, expected to eventually fill up. But if we are dealing with
and emissions (Wang, Nhieu, Chung, & Pham, 2021), and something that is going to stay with us for a long time
intelligent agent technologies for network optimisation in (for example, the Internet or a smoking habit), then one
the food and beverage logistics management (Mangina & should not fit cumulative numbers. At times this distinc-
Vlachos, 2005). tion may not be so obvious. For example, when COVID-19
We now focus on the case of forecasting the quality first appeared many people (often amateurs) began fit-
of beverages, and particularly coffee. Espresso coffee is ting S curves to the cumulative number of infections
among the most popular beverages, and its quality is (for other attempts on forecasting COVID-19, see Sec-
one of the most discussed and investigated issues. Be- tion 3.6.2). Some of them were rewarded because in-
sides human-expert panels, electronic noses, and chem- deed the diffusion of the virus in some countries behaved
ical techniques, forecasting the quality of espresso by accordingly (Debecker & Modis, 1994). But many were
means of data-driven approaches, such as association rule frustrated and tried to ‘‘fix’’ the logistic equation by intro-
mining, is an emerging research topic (Apiletti & Pastor, ducing more parameters, or simply gave up on trying to
2020; Apiletti, Pastor, Callà, & Baralis, 2020; Kittichotsat- use logistics with COVID 19. And yet, many cases (e.g., the
sawat, Jangkrajarng, & Tippayawong, 2021). US) can be illuminated by logistic fits but on the daily
The forecasting model of the espresso quality is built number of infections, not on the cumulative number. As
from a real-world dataset of espresso brewing by profes- of August 1, 2020, leaving out the three eastern states
sional coffee-making machines. Coffee ground size, cof- that had gotten things under control, the rest of the US
fee ground amount, and water pressure have been se- displayed two classic S curve steps followed by plateaus
lected among the most influential external variables. The (see Fig. 18). The two plateaus reflect the number of
ground-truth quality evaluation has been performed for infections that American society was willing to tolerate
each shot of coffee based on three well-known quality at the time, as the price to pay for not applying measures
variables selected by domain experts and measured by to restrict the virus diffusion.
specific sensors: the extraction time, the average flow The second difficulty in using the logistic equation has
rate, and the espresso volume. An exhaustive set of more to do with its ability to predict from relatively early mea-
than a thousand coffees has been produced to train a surements the final ceiling. The crucial question is how
model able to forecast the effect of non-optimal values on early can the final ceiling be determined and with what
the espresso quality. accuracy. Some people claim that before the midpoint no
For each variable considered, different categorical val- determination of a final level is trustworthy (Marinakis &
ues are considered: ground size can be coarse, optimal, Walsh, 2021). Forecasters usually abstain from assigning
or fine; ground amount can be high, optimal, or low; quantitative uncertainties on the parameters of their S
brewing water pressure can be high, optimal, or low. The curve forecasts mostly because there is no theory behind
experimental setting of categorical variables enables the it. However, there is a unique study by Debecker and
application of association rule mining (see Section 2.9.2), Modis (2021) that quantifies the uncertainties on the pa-
a powerful data-driven exhaustive and explainable ap- rameters determined by logistic fits. The study was based
proach (Han et al., 2011; Tan et al., 2005), successfully ex- on 35,000 S curve fits on simulated data, smeared by
ploited in different application contexts (Acquaviva et al., random noise and covering a variety of conditions. The
2015; Di Corso et al., 2018). fits were carried out via a χ 2 minimisation technique. The
Several interesting findings emerged. If the water pres-
sure is low, the amount of coffee ground is too high, and
the grinding is fine, then we can forecast with confidence 160 This subsection was written by Theodore Modis.
811
Fig. 18. Two logistic-growth steps during the early diffusion of COVID-19 in America (March to July, 2020).
study produced lookup tables and graphs for determining processes during the past decades to convince many busi-
the uncertainties expected on the three parameters of nesses to adopt systematic forecasting procedures, leaving
the logistic equation as a function of the range of the S a wide swath of commerce under the guidance of ad hoc
curve populated by data, the error per data point, and the judgment and intuition. At the other extreme, we see
confidence level required. companies with implementations that combine state-of-
The third difficulty using the logistic equation comes the-art methodology with sophisticated accommodations
from the fact that no matter what fitting program one of computing time and costs, as well as consideration
uses, the fitted S curve will flatten toward a ceiling as of the requirements and capabilities of a diverse group
early and as low as it is allowed by the constraints of the of stakeholders (Yelland, Baz, & Serafini, 2019). So, it is
procedure. As a consequence fitting programs may yield not hyperbole to state that business forecasting prac-
logistic fits that are often biased toward a low ceiling. tices are all over the place. What surely is hyperbole,
Bigger errors on the data points accentuate this bias by however, are the ubiquitous claims of software providers
permitting larger margins for the determination of the S about their products accurately forecasting sales, reduc-
ing costs, integrating functions, and elevating the bottom
curve parameters. To compensate for this bias the user
line (Makridakis, Bonneli et al., 2020; Sorensen, 2020).
must explore several fits with different weights on the
For this section, we grilled a dozen practitioners and
data points during the calculation of the χ 2 . He or she
thought leaders (‘‘the Group’’) about developments play-
should then favour the answer that gives the highest
ing out in the next decade of forecasting practice, and
ceiling for the S curve (most often obtained by weighting
have categorised their responses:
more heavily the recent historical data points). Of course,
this must be done with good justification; here again the • Nature of forecasting challenges;
forecaster must exercise wise judgment. • Changes in the forecasting toolbox;
• Evolution in forecasting processes such as integra-
3.9. The future of forecasting practice161 tion of planning functions;
• Expectations of forecasters; and
Plus ça change, plus c’est la même chose.
• Scepticism about real change.
Forecasting Challenges: Focusing on operations, the
Jean-Baptiste Karr (1849) Group sees demand forecasting becoming ever more dif-
ficult due to product/channel proliferation, shorter lead
It would be a more straightforward task to make pre- times, shorter product histories, and spikes in major dis-
dictions about the future of forecasting practice if we had ruptions.
a better grasp of the present state of forecasting prac-
• Operational forecasts will have shorter forecast hori-
tice. For that matter, we lack even a common definition
zons to increase strategic agility required by busi-
of forecasting practice. In a recent article, Makridakis,
ness to compete, sustain, and survive.
Bonneli et al. (2020) lamented the failure of truly no-
• New models will need to incorporate supply-chain
table advances in forecasting methodologies, systems, and
disruption. Demand chains will need to be restarted,
shortening historical data sets and making tradi-
161 This subsection was written by Len Tashman. tional models less viable due to limited history.
812
• Lead times will decrease as companies see the prob- • The interest around disease models increases our
lems in having distant suppliers. Longer lead times awareness of the strengths and weaknesses of math-
make accurate forecasting more difficult. ematical models. Forecasters may need to become
more measured in their claims, or do more to resist
Forecasting Tool Box: Unsurprisingly, this category re-
their models being exploited.
ceived most of the Group’s attention. All predict greater
• We will see a transformation from demand planner
reliance on AI/ML for automating supply-and-demand
to demand analyst, requiring additional skill sets
planning tasks and for reconciling discrepancies in hi-
including advanced decision making, data and risk
erarchical forecasting. Longer-horizon causal forecasting
analysis, communication, and negotiation.
models will be facilitated by big data, social media, and
• Professional forecasters will be rare except in com-
algorithmic improvements by quantum computing. Post-
panies where this expertise is valued. Fewer stu-
COVID, we will see a greater focus on risk
dents are now educated or interested in statistical
management/mitigation. The Cloud will end the era of
modelling, and time is not generally available for
desktop solutions.
training.
• Quantum computers will improve algorithms used • Forecasters will learn the same lesson as optimisa-
in areas like financial forecasting (e.g., Monte Carlo tion folks in the 1990s and 2000s: the importance
simulations), and will change our thinking about of understanding the application area—community
forecasting and uncertainty. intelligence.
• Although social media is a tool for ‘‘what’s trend-
Scepticism: Many were sceptical about the current en-
ing now’’, new models will be developed to use
thusiasm for AI/ML methods; disappointed about the slow
social-media data to predict longer-term behaviour.
adoption of promising new methods into software sys-
Step aside Brown (exponential smoothing) and Bass
tems and, in turn, by companies that use these systems;
(diffusion).
and pessimistic about the respect given to and influence
• Greater automation of routine tasks (data loading,
of forecasters in the company’s decision making.
scrubbing, forecast generation and tuning, etc.)
through AI/ML-powered workflow, configurable lim- • While AI/ML are important additions to the fore-
its, and active alerts. More black box under the hood, caster’s toolbox, they will not automatically solve
but more clarity on the dashboard. forecasting issues. Problems include data hunger,
• Greater focus on risk management/mitigation through capacity brittleness, dubious input data, fickle trust
what-if scenarios, simulations, and probabilistic fore- by users (Kolassa, 2020c), and model bias.
casting. • Practices in the next decade will look very similar to
the present. Not that much has changed in the last
Forecasting Processes and Functional Integration: Sys-
decade, and academic developments are slow to be
tems will become more integrated, promoting greater
translated into practice.
collaboration across functional areas and coordination be-
• Politics, gaming, and the low priority given to fore-
tween forecast teams and those who rely upon them.
casting are the prime drivers of practice, thus limit-
Achieving supply-chain resilience will become as impor-
ing interest in adopting new methodologies.
tant as production efficiency, and new technology such
• None of the topical items (AI/ML, big data, demand
as Alert and Root Cause Analysis systems will mitigate
sensing, new forecasting applications) will have much
disruptions.
of an impact on forecasting practice. Forecasting de-
• S&OP will expand from its home in operations to partments hop from one trend to the other without
more fully integrate with other functions such as making much progress towards better forecasting
finance and performance management, especially in accuracy.
larger multinationals. • Software companies will struggle, despite good of-
• The pandemic has forced firms to consider upping ferings. Most companies do not want to invest in
supply-chain resilience. Firms are building in capac- excellent forecasting engines; whatever came with
ity, inventory, redundancy into operations—somewhat their ERP system is ‘‘good enough’’.
antithetical to the efficiency plays that forecasting • Forecasting will continue to suffer from neglect by
brings to the table. higher levels of management, particularly when fore-
• Forecasting will be more closely tied to Alert and casts are inconveniently contrary to the messages
Root Cause Analysis systems, which identify break- management hopes to convey.
downs in processes/systems contributing to adverse
Note finally that the COVID-19 pandemic has elevated
events, and prevent their recurrence.
practitioner concerns about disruptions to normal pat-
Expectations of Forecasters: Agreement was universal terns as well as the fear of an increasingly volatile en-
that the forecaster’s job description will broaden and be- vironment in which forecasts must be made. There are
come more demanding, but that technology will allow indications that companies will place more stress on judg-
some redirection of effort from producing forecasts to mental scenarios, likely in conjunction with statistical/ML
communicating forecasting insights. methods.
813
4. Forecasting: benefits, practices, value, and limita- whose usefulness is misunderstood. Executives want to
tions162 know the exact sales of their firm for next month to
set up their production schedule. Instead, they are given
prediction intervals (PIs) around such forecast and told
Mr. Buffett said his advice for the cash left to his
that most of the time, sales will be within this inter-
wife was that 10 per cent should go to short-term
val, assuming the fluctuations follow some distributional
government bonds and 90 per cent into a very
low-cost S&P 500 index fund.
assumptions. They argue that forecasting must decrease,
not amplify, uncertainty and that the PIs are too wide
and ‘uninformative’ to be used for making practical busi-
ness decisions. The trouble is that these PIs are based
The purpose of this unique article is to provide an en- on past fluctuations and present the best estimation of
cyclopedic knowledge about the various aspects of fore- future uncertainty, even if they seem too wide. Worse,
empirical research has shown that they are too narrow,
casting. In this article, there are more than 140 sections
underestimating uncertainty often considerably.
and subsections, with more than 2,100 references, written
Assessing Uncertainty and Dealing with its Implied Risks:
by 80 of some of the best-known forecasting researchers
Uncertainty entails risks, requiring action to minimise
and practitioners in the world, making it into a selective,
their negative consequences. There are two kinds of un-
encyclopedic piece covering, into a single source, a great certainty that can be illustrated by a commuting example.
deal of the available knowledge about the theory and The first relates to fluctuations in the commuting time
practice of forecasting. We hope that this article will serve under normal driving conditions when there are no se-
as an easy-to-use reference source. We aim to convert it rious accidents, road works or major snowstorms. Such
into an online resource that will be regularly updated as fluctuations are small and can be captured by a normal
new information becomes available. curve that allows to balance the risk of arriving ear-
But some people argue if there is any value in at- lier or later than the desired time. In the opposite case,
tempting to predict the future and if forecasting is any uncertainty is fat-tailed and hard to estimate, as delays
different than fortune telling, given the large numbers can be substantial depending upon the seriousness of the
of mistaken forecasts made in the past, including our accident or that of the snowstorm while the risk of being
inability to accurately predict the progression of COVID- early to work is smaller than being late. Moreover, such
19 and its economic and human consequences? What risk is substantially different when going to the airport
is, therefore, the usefulness of a paper like the present to catch a flight, requiring starting much earlier than the
one when crystal balling is not possible, and uncertainty average time it takes to go to the airport to minimise the
reigns? It is the aim of this concluding article to set the risk of missing the flight.
record straight, explaining the benefits and practical value More Accurate Ways of Forecasting and Assessing Uncer-
of forecasting while reporting its limitations too. tainty: Extensive empirical research, including forecasting
The Myriad of Forecasts: All planning and the great ma- competitions, has shown that systematic approaches im-
jority of decisions we make require forecasting. Deciding prove the accuracy of forecasting and the correct assess-
ment of uncertainty resulting in substantial benefits when
what time to get up in the morning, not to be late for work
compared to ad-hoc judgmental alternatives (Makridakis,
implies a correct prediction of the commuting time to go
Bonneli et al., 2020). The biggest advantage of such ap-
to the office. Determining what to study is another de-
proaches is their ability to identify and estimate, in a
cision requiring elaborate predictions about the demand
mathematically optimal manner, past patterns and rela-
for future jobs decades away. In the business world, firms
tionships that are subsequently extrapolated to predict
must decide/forecast how many products to manufacture, their continuation, avoiding the over optimism and wish-
the price they should be sold, how much money to spend ful thinking associated with judgmental approaches. At
on advertising and promotion, how much and in what the same time, it must be clear that the accuracy of
type of new technologies to invests and a plethora of the forecasts and the correctness of uncertainty will de-
other future-oriented decisions requiring both predictions pend on the established patterns/relationship not chang-
and assessing their inevitable uncertainty. Whether we ing much during the forecasting period.
like it or not, we have no choice but making these fore- Using Benchmarks to Evaluate the Value of Forecasting:
casts to benefit as much as possible from their value, The accuracy of the forecasts and the correct assessment
knowing perfectly well that all predictions are uncertain of uncertainty must be judged not on their own but in
while some may turn out to be wrong. comparison to some simple, readily available benchmarks.
The Pervasiveness of Uncertainty: Apart from some areas In stock market forecasts, for instance, the accuracy of
of hard sciences, all other forecasts are uncertain and predictions is compared to that of today’s price used as
must be accompanied with a measure of its magnitude, the forecast for future periods. Empirical comparisons
expressed as a prediction interval, or as a probability have shown that such a benchmark beats the great ma-
distribution around the most likely forecast. Although the jority of professional forecasters, hence Buffet’s advice in
value and usage of forecasts is clear, that of uncertainty is the epigram for his wife to invest in a low-cost index
not. Worse, it becomes an unwelcome source of anxiety fund that selects stocks randomly. In weather forecast-
ing, meteorologists are judged by the improvement of
their forecasts over the naive prediction that tomorrow’s
162 This subsection was written by Spyros Makridakis. weather will be the same as today.
814
Concluding remark: Accepting the advantages and lim- David F. Hendry gratefully acknowledges funding from
itations of systematic forecasting methods and most im- the Robertson Foundation, USA and Nuffield College, UK.
portantly avoiding any exaggerated expectations of what Yanfei Kang acknowledges the support of the National
it can achieve is critical. Such methods do not possess any Natural Science Foundation of China (number 11701022)
prophetic powers, they simply extrapolate established and the National Key Research and Development Program,
patterns and relationships to predict the future and assess China (number 2019YFB1404600).
its uncertainty. Their biggest advantage is their objectivity Stephan Kolassa would like to thank Tilmann Gneiting
and ability for optimal extrapolation. Their biggest disad- for some very helpful tips.
vantages are: (i) the patterns and the relationships must Gael M. Martin has been supported by Australian Re-
remain fairly constant during the forecasting phase for the search Council (ARC) Discovery Grants DP170100729 and
forecasts to be accurate, and (ii) uncertainty must not be DP200101414.
fat-tailed so it can be measured quantitatively. Alessia Paccagnini acknowledges the research support
by COST Action ‘‘Fintech and Artificial Intelligence in Fi-
Declaration of competing interest nance - Towards a transparent financial industry’’ (FinAI)
CA19130.
The authors declare that they have no known com- Jose M. Pavía acknowledges the support of the Spanish
peting financial interests or personal relationships that Ministry of Science, Innovation and Universities and the
could have appeared to influence the work reported in Spanish Agency of Research, co-funded with FEDER funds,
this paper. grant ECO2017-87245-R, and of Consellería d’Innovació,
Universitats, Ciència i Societat Digital, Generalitat Valen-
Acknowledgments
ciana – grant number AICO/2019/053.
Diego J. Pedregal and Juan Ramon Trapero Arenas
Fotios Petropoulos would like to thank all the co-
acknowledge the support of the European Regional De-
authors of this article for their very enthusiastic response
velopment Fund and Junta de Comunidades de Castilla-La
and participation in this initiave. He would also like to
Mancha (JCCM/FEDER, UE) under the project
thank Pierre Pinson for inviting this paper to be submitted
SBPLY/19/180501/000151 and by the Vicerrectorado de
to the International Journal of Forecasting. The constructive
Investigación y Política Científica from UCLM, Spain through
comments and suggestions from this advisory board were
the research group fund program PREDILAB;
vital in improving the paper. He also thanks Artur Taras-
DOCM 26/02/2020 [2020-GRIN-28770].
sow for offering a list of Gretl’s software functionalities.
David E. Rapach thanks Ilias Filippou and Guofu Zhou
Jakub Bijak’s work received funding from the Euro-
pean Union’s Horizon 2020 research and innovation pro- for valuable comments.
gramme, grant 870299 QuantMig: Quantifying Migration J. James Reade and Han Lin Shang acknowledge Shix-
Scenarios for Better Policy. uan Wang for his constructive comments.
Clara Cordeiro is partially financed by national funds Michał Rubaszek is thankful for the financial support
through FCT – Fundação para a Ciência e a Tecnologia, provided by the National Science Centre, Poland, grant
Portugal under the project UIDB/00006/2020. No. 2019/33/B/HS4/01923 entitled ‘‘Predictive content of
Fernando Luiz Cyrino Oliveira acknowledges the sup- equilibrium exchange rate models’’.
port of the Coordination for the Improvement of Higher
Level Personnel (CAPES), Brazil – grant number 001, the Disclaimer
Brazilian National Council for Scientific and Technological
Development (CNPq) – grant number 307403/2019-0, and The views expressed in this paper are those of the
the Carlos Chagas Filho Research Support Foundation of authors and do not necessarily reflect the views of their
the State of Rio de Janeiro (FAPERJ) – grant numbers affiliated institutions and organisations.
202.673/2018 and 211.086/2019.
Shari De Baets was funded by the FWO Research Foun-
Appendix A. List of acronyms
dation Flanders.
Joanne Ellison acknowledges the support of the ESRC
FertilityTrends project (grant number ES/S009477/1) and ABC Approximate Bayesian
the ESRC Centre for Population Change (grant number Computation
ES/R009139/1). ACC Ant Colony Clustering
Piotr Fiszeder was supported by the National Science ACD Autoregressive Conditional
Centre, Poland project number 2016/21/B/HS4/00662 en- Duration
titled ‘‘Multivariate volatility models - the application of ADIDA Aggregate–Disaggregate
low and high prices’’. Intermittent Demand
David T. Frazier has been supported by Australian Re- Approach
search Council (ARC) Discovery Grants DP170100729 and ADL Autoregressive Distributed Lag
DP200101414, and ARC Early Career Researcher Award ADMM Alternating Direction Method
DE200101070. of Multipliers
Mariangela Guidolin acknowledges the support of the AI Artificial Intelligence
University of Padua, Italy, through the grant BIRD188753/18.
815
AIC Akaike’s Information Criterion CRPS Continuous Ranked Probability

AICc Akaike’s Information Criterion Score
corrected (for small sample CSR Complete Subset Regression
sizes) CV Cross-Validation
ANACONDA Analysis of National Causes of DA Deterministic Annealing
Death for Action DC Distribution Centre
ANFIS Adaptive Neuro-Fuzzy DCC Dynamic Conditional
Inference System Correlation
ANN Artificial Neural Network DCC-RGARCH Range GARCH DCC
AO Additive Outlier DFM Dynamic Factor Model
AR AutoRegressive (model) DGP Data Generating Process
ARX AutoRegressive with DJI Dow Jones Industrial
eXogenous variables (model) DM Diebold–Mariano (test)
ARCH AutoRegressive Conditional DSGE Dynamic Stochastic General
Heteroskedasticity Equilibrium
ARMA AutoRegressive-Moving DSHW Double Seasonal Holt-Winters
Average (model) DSTCC-CARR Double Smooth Transition
ARIMA AutoRegressive Integrated Conditional Correlation CARR
Moving Average (model) DT Delay Time
ARIMAX AutoRegressive Integrated EEG ElectroEncephaloGram
Moving Average with EGARCH Exponential GARCH
eXogenous variables (model) EH Expectations Hypothesis
B&M Brick and Mortar EMD Empirical Mode
BATS Box–Cox transform, ARMA Decomposition
errors, Trend, and Seasonal ENet Elastic Net
components (model) ENSO El Niño Southern Oscillation
BEER Behavioural Equilibrium ERCOT Electric Reliability Council of
Exchange Rate Texas
BEKK Baba-Engle-Kraft-Kroner ES Expected Shortfall
GARCH ESP-r Environmental Systems
BEKK-HL BEKK High Low Performance – research
BIC Bayesian Information Criterion ESTAR Exponential STAR
BLAST Building Loads Analysis and ETS ExponenTial Smoothing (or
System Thermodynamics Error, Trend, Seasonality)
BM Bass Model EVT Extreme Value Theory
BMC Bootstrap Model Combination EWMA Exponentially Weighted
BPNN Back-Propagation Neural Moving Average
Network FAR Functional AutoRegressive
CARGPR Conditional AutoRegressive (model)
Geometric Process Range FASSTER Forecasting with Additive
(model) Switching of Seasonality,
CARR Conditional AutoRegressive Trend and Exogenous
Range (model) Regressors
CARRS Conditional AutoRegressive FCM Fuzzy C-Means
Rogers and Satchell (model) FIGARCH Fractionally Integrated GARCH
CBC Choice Based Conjoint FIS Fuzzy Inference System
(analysis) FFNN Feed-Forward Neural Network
CBO Congressional Budget Office FFORMA Feature-based FORecast Model
CDS Credit Default Swap Averaging
CNN Convolutional Neural Network FMCG Fast Moving Consumer Goods
COVID-19 Coronavirus disease 2019 FPCA Functional Principal
CVAR Cointegrated Vector Component Analysis
AutoRegressive (model) FRB/EDO Federal Reserve Board’s
CAViaR Conditional AutoRegressive Estimated, Dynamic,
Value At Risk Optimisation-based (model)
CL Cross Learning FSS Forecasting Support System
CPFR Collaborative Planning, FVA Forecast Value Added
Forecasting, and GARCH General AutoRegressive
Replenishment Conditional Heteroscedasticity
CRCD Competition and Regime GARCH-PARK-R GARCH PARKinson Range
Change Diachronic (model) GARCH-TR GARCH True Range
816
GB Givon-Bass (model) MASE Mean Absolute Scaled Error

GBM Generalised Bass Model MCMC Markov Chain Monte Carlo
GDP Gross Domestic Product MFI Marginal Forecast Interval
GGM Guseo-Guidolin Model (GGM) MICE Meetings, Incentives,
GJR-GARCH Glosten-Jagannathan-Runkle Conventions, and
GARCH Exhibitions/Events
GM Generalised M-estimator MIDAS MIxed DAta Sampling
GMM Generalised Methods of MJO Madden Julian Oscillation
Moments ML Machine Learning
GPU Graphics Processing Unit MLP MultiLayer forward Perceptron
GRNN Generalised Regression Neural MLR Multiple Linear Regression
Network MS Markov Switching
HAR Heterogeneous AutoRegressive MS VAR Markov Switching VAR
(model) MSARIMA Multiple/Multiplicative
HDFS Hadoop Distributed File Seasonal ARIMA
System MSC Multiple Seasonal Cycles
HMD Human Mortality Database MSE Mean Squared Error
HP Hodrick–Prescott MSRB Markov-Switching
HPU House Price Uncertainty Range-Based
HVAC Heating, Ventilation, and Air MTA Multiple Temporal
Conditioning (system) Aggregation
HAR Heterogeneous AutoRegressive MTLF Medium-Term Load
(model) Forecasting
HQ Hannan-Quinn NAO North Atlantic Oscillation
IEA International Energy Agency NLS Nonlinear Least Squares
IG Interaction Groups NLTK Natural Language Toolkit
iid independent and identically NMAE Normalised Mean Absolute
distributed Error
IIS Impulse Indicator Saturation NN Neural Network
IO Innovation Outlier NNAR Neural Network
IT Information Technology AutoRegressive
KBKD Krishnan-Bass-Kummar NOB Non-Overlapping Blocks
Diachronic (model) NPF New Product Forecasting
KISS Keep It Simple, Stupid NWP Numerical Weather Prediction
(principle) OB Overlapping Blocks
kNN k Nearest Neighbours OBR Office for Budget
KPSS Kwiatkowski–Phillips– Responsibility
Schmidt–Shin ODE Ordinary Differential
L-IVaR Liquidity-adjusted Intraday Equations
Value-at-Risk OLS Ordinary Least Squares
LASSO Least Absolute Shrinkage and OWA Overall Weighted Average
Selection Operator PAR Periodic AutoRegressive
LH Low and High (model)
LLN Law of Large Numbers PCA Principal Components Analysis
LN-CASS Logit-Normal Continuous pdf probability density function
Analogue of the PdM Predictive Maintenance
Spike-and-Slab PFEM Point Forecast Error Measure
LS (or LPS) Logarithmic (Predictive) Score PHANN Physical Hybrid Artificial
(log-score) Neural Network
LSTAR Logistic STAR pHDR predictive Highest Density
LSTM Long Short-Term Memory Region
Networks PI Prediction Interval
LTLF Long-Term Load Forecasting PIT Probability Integral Transform
LV Lotka–Volterra (model) PL Product Level
LVac Lotka–Volterra with PLS Partial Least Squares
asymmetric churn (model) PM Particulate Matter
LVch Lotka–Volterra with churn POT Peak Over Threshold
(model) PPP Purchasing Power Parity
MAE Mean Absolute Error PSO Particle Swarm Intelligence
MAPE Mean Absolute Percentage PV PhotoVoltaic
Error
817
Qα quantile score or pinball loss STR Seasonal-Trend decomposition

for a level α ∈ (0, 1) based on Regression
RAF Royal Air Force (UK) SV Stochastic Volatility
RB Range-Based SVA Stochastic Value Added
RB-copula Range-Based copula SVD Singular Value Decomposition
RB-DCC Range-Based DCC SVM Support Vector Machine
RB-MS-DCC Range-Based SWAN Simulating WAves Nearshore
Markov-Switching DCC S&OP Sales and Operations Planning
RBF Radial Basis Function S&P Standard & Poor’s
REGARCH Range-Based Exponential TAR Threshold AutoRegressive
GARCH (model)
RET Renewable Energy Technology TARMA Threshold AutoRegressive
RGARCH Range GARCH Moving Average (model)
RMSE Root Mean Squared Error TARMASE Threshold AutoRegressive
RNN Recurrent Neural Network Moving Average (model)
RR-HGADCC Return and Range TARR Range-Based Threshold
Heterogeneous General conditional AutoRegressive
Asymmetric DCC (model)
RTV Real Time Vintage TBATS Exponential Smoothing state
SA Structured Analogies space model with Box–Cox
SARIMA Seasonal AutoRegressive transformation, ARMA errors,
Integrated Moving Average Trend and Seasonal
(model) components
SARIMAX Seasonal AutoRegressive TFR Total Fertility Rate
Integrated Moving Average TGARCH Threshold GARCH
with eXogenous variables TMA Threshold Moving Average
SARMA Seasonal AutoRegressive (model)
Moving Average (model) TPU Tensor Processing Unit
SARMAX Seasonal AutoRegressive TRAMO Time series Regression with
Moving Average with ARIMA noise, Missing values
eXogenous variables and Outliers
SBA Syntetos-Boylan TSB Teunter-Syntetos-Babai
Approximation (method)
SBC Syntetos-Boylan-Croston UCRCD Unbalanced Competition and
(classification) Regime Change Diachronic
SEATS Seasonal Extraction in ARIMA (model)
Time Series UIP Uncovered Interest Party
SES Simple (or Single) Exponential VaR Value at Risk
Smoothing VAR Vector AutoRegressive (model)
SETAR Self-Exciting Threshold VARX VAR with eXogenous variables
AutoRegressive (model) (model)
SFI Simultaneous Forecast Interval VARMA Vector AutoRegressive Moving
SKU Stock Keeping Unit Average (model)
SGD Stochastic Gradient Descent VAT Value Added Tax
SIS Step Indicator Saturation VARIMAX Vector AutoRegressive
SL Serial number Level Integrated Moving Average
SMA Simple Moving Average with eXogenous variables
sMAPE symmetric Mean Absolute (model)
Percentage Error VECM (or VEC) Vector Error Correction Model
SOM Self-Organising Map VEqCM Vector Equilibrium-Correction
SS State Space Model
sSA semi-Structured Analogies VSTLF Very Short-Term Load
SSARIMA Several Seasonalities (or State Forecasting
Space) ARIMA WLS Weighted Least Squares
STAR Smooth Transition WNN Wavelet Neural Network
AutoRegressive (model) WOM Word-Of-Mouth
STARR Smooth Transition conditional WW2 World War 1
AutoRegressive Range (model) WW2 World War 2
STL Seasonal Trend decomposition WW3 World War 3
using Loess XGBoost eXtreme Gradient Boosting
STLF Short-Term Load Forecasting
818
F. Petropoulos, D. Apiletti, V. Assimakopoulos et al.
Table B.1
A list of indicative free or open-source packages, libraries, and toolboxes linking to the theory sections of this article. The authors assume no liability for the software listed below; interested users are
strongly advised to read the respective documentations and licences terms.
Related section Software Package/ Function(s) Comments
Library/
Toolbox
Section 2.2.1. Box–Cox R forecast BoxCox; InvBoxCox; Functions to transform the input variable using a Box–Cox transformation, reverse the
transformations BoxCox.lambda; transformation and find optimal parameters. https://cran.r-project.org/package=forecast
Section 2.2.2. Box–Cox R stats decompose; stl Classical decomposition method (additive and multiplicative), and STL decomposition method.
transformations Time series https://stat.ethz.ch/R-manual/R-devel/library/stats/html/00Index.html
decomposition
R forecast seasadj; seasonal; mstl; msts; Tools for extracting components, and multiple seasonal decomposition methods.
tbats.components https://cran.r-project.org/package=forecast
R tsutils decomp; seasplot Classical decomposition method, and functions for seasonal plots.
https://cran.r-project.org/package=tsutils
R stR AutoSTR; STR; heuristicSTR; Seasonal-Trend decomposition based on Regression. https://cran.r-project.org/package=stR
plot.STR; seasadj.STR
R seasonal seas Functions for X-11, SEATS, and X-13-ARIMA-SEATS decomposition methods.
https://cran.r-project.org/package=seasonal
Gretl buys_ballot Plot seasonal time series components. http://gretl.sourceforge.net/
Gretl season_plot set_season_plot Plot seasonal time-series components. http://gretl.sourceforge.net/
Gretl tsfcst decompcl Classical time series decomposition. http://gretl.sourceforge.net/
Gretl StrucTiSM STSM_components Decomposition using structural timeseries model. http://gretl.sourceforge.net/
819
Section 2.2.3. Anomaly R anomalize time_decompose; anomalize; A ‘‘tidy’’ workflow for detecting anomalies in data.
detection and time series time_recompose https://cran.r-project.org/package=anomalize
forecasting
R oddstream find_odd_streams; Early detection of anomalous series within a large collection of streaming time series data.
extract_tsfeatures; https://cran.r-project.org/package=oddstream
set_outlier_threshold
R tsoutliers tso; locate.outliers.oloop; Detection of outliers in time series such as Innovational outliers, additive outliers, level shifts,
remove.outliers temporary changes and seasonal level shifts. https://cran.r-project.org/package=tsoutliers

R stray find_HDoutliers; Anomaly detection in high dimensional and temporal data.
find_threshold; https://cran.r-project.org/package=stray
display_HDoutliers
R forecast tsoutliers; tsclean Provides some simple heuristic methods for identifying and correcting outliers.
https://cran.r-project.org/package=seasonal
R OutliersO3 OutliersO3; O3plotM; O3plotT; Draws overview of outliers (O3) Plots. https://cran.r-project.org/package=OutliersO3
O3prep
R CRAN Task Anomaly Detection with R Contains a list of R packages that can be used for anomaly detection.
View https://github.com/pridiltal/ctv-AnomalyDetection
Gretl tramolin Outlier detection/correction and missing data interpolation. http://gretl.sourceforge.net/
Section 2.2.4. Robust handling R gets isat Function for running impulse and step indicator saturation.
of outliers in time series https://cran.r-project.org/package=gets
forecasting
(continued on next page)

Table B.1 (continued).
Library/
Toolbox
Section 2.3.1. Exponential R forecast ets; forecast.ets; ses; Functions for simple exponential smoothing and automatic exponential smoothing modelling.
smoothing models https://cran.r-project.org/package=forecast
R smooth es Function for automatic exponential smoothing modelling.
https://cran.r-project.org/package=smooth
Gretl tsfcst expsmpars Simple exponential smoothing minimising the sum of squared errors.
http://gretl.sourceforge.net/
Section 2.3.2. Time-series R stats lm Fitting linear regression models.
regression models https://stat.ethz.ch/R-manual/R-devel/library/stats/html/00Index.html
R leaps regsubsets Functions for selecting linear regression models. https://cran.r-project.org/package=leaps
R relaimpo Relative importance of regressors in linear models. https://cran.r-project.org/package=relaimpo
R MASS stepAIC Choose a model by AIC in a stepwise algorithm. https://cran.r-project.org/package=MASS
Gretl ols, lad, midasreg Regression models with OLS, LAD, and MIDAS with functionality for forecasting.
Section 2.3.3. Theta method R forecast thetaf Returns forecasts and prediction intervals for a theta method forecast.
and models https://cran.r-project.org/package=forecast
R forecTheta stheta; stm; otm; dstm; dotm Functions for forecasting univariate time series using Theta Models.
https://cran.r-project.org/package=forecTheta
R tsutils theta Estimate Theta method. https://cran.r-project.org/package=tsutils
Gretl tsfcst stheta Theta-method for univariate forecasting. http://gretl.sourceforge.net/
Section 2.3.4. Autoregressive R forecast auto.arima; Arima; arfima; Functions for fitting and forecasting with ARIMA models.
820
integrated moving average arima.errors; arimaorder https://cran.r-project.org/package=forecast

(ARIMA) models
R smooth auto.msarima; auto.ssarima; State-space and multiple seasonalities implementations of ARIMA models.
msarima; ssarima https://cran.r-project.org/package=smooth
Gretl arima Functions for fitting and forecasting with SARIMAX models. http://gretl.sourceforge.net/
Gretl auto_arima Find best fitting SARIMAX model with functions for forecasting. http://gretl.sourceforge.net/
Gretl armax Automatically determine the best ARMAX model. http://gretl.sourceforge.net/
R aTSA adf.test Augmented Dickey–Fuller test. https://cran.r-project.org/package=aTSA

R tseries kpss.test Kwiatkowski-Phillips–Schmidt–Shin (KPSS) test. https://cran.r-project.org/package=tseries
R forecast ndiffs; nsdiffs Estimates the number of (seasonal) differences required to make a given time series
stationary. https://cran.r-project.org/package=forecast
Gretl adf; adf-gls; kpss; levinlin Various unit-root tests for time-series and panel data. http://gretl.sourceforge.net/
Gretl DHF_test Package for Dickey-Hasza-Fuller seasonal Unit Root Test. http://gretl.sourceforge.net/
Gretl DP Package for testing for a double unit root. http://gretl.sourceforge.net/
Gretl GHegy Package Seasonal unit roots tests. http://gretl.sourceforge.net/
Gretl Kapetanios Package for Kapetanios’ unit root test with possible structural breaks.
Gretl PPtest Package for running Phillips–Perron unit root test. http://gretl.sourceforge.net/
Gretl VSG_test Package for test proposed by Ventosa-Santaulària and Gómez-Zaldívar.
Section 2.3.5. Forecasting for R smooth msarima; ssarima Functions for forecasting data with multiple seasonal cycles.
multiple seasonal cycles https://cran.r-project.org/package=smooth
R fable model; forecast; fasster; ETS; Forecasting models for tidy time series. https://cran.r-project.org/package=fable
ARIMA; TSLM
Library/
Toolbox
R, Python prophet Facebook’s automatic forecasting procedure. https://cran.r-project.org/package=prophet
R tidymodels Collection of packages for modelling and machine learning using tidyverse principles.
https://cran.r-project.org/package=tidymodels
R forecast tbats; dshw Functions for forecasting data with multiple seasonal cycles.
https://cran.r-project.org/package=forecast
R fable.prophet prophet; forecast A tidy R interface to the prophet forecasting procedure using fable.
https://github.com/mitchelloharawild/fable.prophet
Section 2.3.6. State-space Matlab SSpace General modelling of linear, non-linear and non-Gaussian State Space systems.
models https://github.com/djpedregal/SSpace
Matlab SSM General modelling of linear, non-linear and non-Gaussian State Space systems.
https://www.mathworks.com/help/econ/ssm-class.html
Matlab SSMMATLAB A Set of MATLAB Programs for the Statistical Analysis of State Space Models.
https://github.com/vgomezenriquez/ssmmatlab
Matlab E4 A MATLAB toolbox for time series analysis in State Space form. https://www.ucm.es/e-4
R UComp Automatic identification of Unobserved Components models in State Space form.
https://cran.r-project.org/package=UComp
R statespacer State Space modelling, mainly ARIMA and Basic Structural Models.
https://cran.r-project.org/package=statespacer
821
R smooth Forecasting using single error State Space models. https://cran.r-project.org/package=smooth

R bssm Bayesian Inference of Non-Gaussian State Space Models.
https://cran.r-project.org/package=bssm
R mssm Multivariate State Space models. https://cran.r-project.org/package=mssm
R KFAS Kalman Filter and Smoother for Exponential Family State Space Models.
https://cran.r-project.org/package=KFAS
R TSSS Time Series Analysis with State Space Model, based on the methods in Kitagawa (1993).
https://cran.r-project.org/package=TSSS
R dlm Bayesian and Likelihood Analysis of Dynamic Linear Models (Gaussian State Space models).

https://cran.r-project.org/package=dlm
Python statsmodels statespace Time Series Analysis by State Space Methods. https://www.statsmodels.org/stable/index.html
Gretl kfilter; ksmooth; kdsmooth; State Space Modeling functionality with function for forecasting. http://gretl.sourceforge.net/
ksimul
Gretl StrucTiSM STSM_fcast Harvey-style Structural Time Series Models with function for forecasting.
Section 2.3.7. Models for R dembase General-purpose tools for demographic analysis. https://github.com/StatisticsNZ/dembase
population processes
R demest Bayesian statistical methods for demography. https://github.com/StatisticsNZ/demest
R demlife Creating and working with life tables. https://github.com/StatisticsNZ/demlife
R BayesPop Generating population projections for all countries of the world using several probabilistic
components, such as total fertility rate and life expectancy.
https://cran.r-project.org/package=bayesPop
Library/
Toolbox
R bayesTFR Making probabilistic projections of total fertility rate for all countries of the world, using a
Bayesian hierarchical model. https://cran.r-project.org/package=bayesTFR
R bayesLife Making probabilistic projections of life expectancy for all countries of the world, using a
Bayesian hierarchical model. https://cran.r-project.org/package=bayesLife
Spread- DAPPS Demographic Analysis and Population Projection System: Standalone spreadsheet-based
sheet software for demographic estimation and projections, prepared by the US Census Bureau.
https://www.census.gov/data/software/dapps.Overview.html
Section 2.3.9. Forecasting with R gets getsm, getsv, isat, isatvar Package that implements general to specific model selection, indicator saturation, with
many variables functionality for forecasting. https://cran.r-project.org/package=gets
R vars Functions and routines for VAR Modelling. https://cran.r-project.org/package=vars
Gretl var; system Fitting system-models with functionality for forecasting. http://gretl.sourceforge.net/
Section 2.3.10. Functional time R ftsa ftsm; farforecast; T_stationarity Functional time series analysis. https://cran.r-project.org/package=ftsa
series models
Section 2.3.11. ARCH/GARCH R tseries garch Fit GARCH models to time series. https://cran.r-project.org/package=tseries
models
Python PyFlux Time series analysis and prediction tools that focus on autoregressive methods (ARIMA, ARCH,
GARCH, etc.). https://pyflux.readthedocs.io/en/latest/index.html
Gretl arch, garch Fit (G)ARCH models to time series. http://gretl.sourceforge.net/
Gretl gig gig_estimate; gig_var_fcast Estimate various types of GARCH models. http://gretl.sourceforge.net/
Section 2.3.12. Markov R MSwM Fitting Markov switching models. https://cran.r-project.org/package=MSwM
822
switching models
R NHMSAR Non-homogeneous Markov switching autoregressive models.
https://cran.r-project.org/package=NHMSAR
Section 2.3.13. Threshold R TAR Bayesian modelling of autoregressive threshold time series models.
models https://cran.r-project.org/package=TAR
R TSA tar; star Functions for threshold models (and general time series analysis).
https://cran.r-project.org/package=TSA
Gretl Threshold_Panel THRESH_SETUP Hansen’s panel threshold model. http://gretl.sourceforge.net/

Gretl SETAR Estimation of a SETAR model. http://gretl.sourceforge.net/
Section 2.3.15. Forecasting with R BMR forecast Bayesian Macroeconometrics in R (BMR) is a package for estimating and forecasting Bayesian
DSGE models VAR and DSGE. https://github.com/kthohr/BMR
Mat- Dynare Software platform for solving, estimating, and making forecasts with DSGE.
lab/GNU https://www.dynare.org/
Octave
Section 2.3.18. Innovation R DIMORA Estimation of Bass Model, Generalised Bass Model, GGM, UCRCD.
diffusion models https://cran.r-project.org/package=DIMORA
R diffusion diffusion Various diffusion models to forecast new product growth. Currently the package contains Bass,
Gompertz and Gamma/Shifted Gompertz curves. https://cran.r-project.org/package=diffusion
Section 2.3.19. The natural law R LS2Wstat scurve An S curve function between two constant values. https://cran.r-project.org/package=LS2Wstat
of growth in competition
Section 2.3.21. Estimation and R hdrcde cde Conditional kernel density estimation to produce marginal distributions (uncertainty
representation of uncertainty forecasts). https://cran.r-project.org/package=hdrcde
R gamlss gamlss Semi-parametric models for uncertainty forecasting. https://cran.r-project.org/package=gamlss
Library/
Toolbox
R gamboostLSS mboostLSS Semi-parametric component-wise gradient boosting models for uncertainty forecasting.
https://cran.r-project.org/package=gamboostLSS
Python scikit-learn GradientBoostingRegressor; Machine learning models (gradient boosting trees and random forests) for quantile
RandomForestQuantileRegres- forecasting. https://scikit-learn.org/stable/
sor
R quantreg rq; lprq; nlqr Estimation and inference methods for models of conditional quantiles.
https://cran.r-project.org/package=quantreg
R EnvStats FcnsByCatPredInts; pointwise Functions for computing prediction intervals and simultaneous prediction intervals.
https://cran.r-project.org/package=EnvStats
R rmgarch dccfit-methods; Multivariate GARCH Models (e.g., forecasting covariance matrix).
dccforecast-methods https://cran.r-project.org/package=rmgarch
Section 2.3.22. Forecasting R FatTailsR Functions for Kiener distributions and fat tails. https://cran.r-project.org/package=FatTailsR
under fat tails
Section 2.4.3. Bayesian R VineCopula Statistical analysis of vine copula models. https://cran.r-project.org/package=VineCopula
forecasting with copulas
R cdcopula Covariate-dependent copula models. https://github.com/feng-li/cdcopula
R FactorCopula Factor Copula Models for Mixed Continuous and Discrete Data.
https://cran.r-project.org/package=FactorCopula
823
Section 2.5.1. Leading R lmtest grangertest Test for Granger causality. https://cran.r-project.org/package=lmtest
indicators and Granger
causality
Gretl var; omit Standard Granger-causality test. http://gretl.sourceforge.net/
Gretl BreitungCande- Breitung-Candelon test of frequency-wise Granger (non-)causality. http://gretl.sourceforge.net/
lonTest
Section 2.5.3. Variable Selection R glmnet Generalised linear models with Lasso or elastic net regularisation.

https://cran.r-project.org/package=glmnet
Gretl omit Sequential removing of variables to a model. http://gretl.sourceforge.net/
Gretl addlist Sequential addition of variables to a model. http://gretl.sourceforge.net/
Gretl regls Add-on for regularised least squares such as Ridge, Lasso and Elastic-Net.
Gretl fsboost fsreg Forward-stagewise boosted regression estimates with functionality for forecasting.
Section 2.5.4. Model Selection R gets Functions for automatic general to specific model selection.
https://cran.r-project.org/package=gets
Section 2.5.5. Cross-validation R forecast CVar k-fold Cross-Validation applied to an autoregressive model.
for time-series data https://cran.r-project.org/package=forecast
Gretl fcast Forecasting command with functionality for recursive-window forecasts.

Library/
Toolbox
Section 2.6.1. Forecast R forecastHybrid Functions for ensemble time series forecasts. https://cran.r-project.org/package=forecastHybrid
combination: a brief review of
statistical approaches
Section 2.7.2. Forecasting on Database InfluxDB Scalable datastore for metrics, events, and real-time analytics.
distributed systems https://www.influxdata.com/time-series-database/
Database OpenTSDB A scalable, distributed Time Series Database. http://opentsdb.net/
Database RRDtool A program for easily maintaining a database of time-series data. https://oss.oetiker.ch/rrdtool/
Database Timely A time series database application that provides secure access to time series data.
https://code.nsa.gov/timely/
Python, R darima Implementations of distributed ARIMA models on Spark platform.
& Spark https://github.com/xqnwang/darima
Section 2.7.3. Agent-based R SpaDES Spatially explicit discrete event simulation models. https://cran.r-project.org/package=SpaDES
models
Section 2.7.4. Feature-based R tsfeatures tsfeatures Methods for extracting various features from time series data.
time series forecasting https://cran.r-project.org/package=tsfeatures
Python tsfresh Calculates a large number of time series characteristics, the so called features. Further the
package contains methods to evaluate the explaining power and importance of such
characteristics for regression or classification tasks. https://tsfresh.readthedocs.io/en/latest/
Matlab hctsa Code framework that enables the extraction of thousands of time-series features from a time
series (or a time-series dataset). It also provides a range of tools for visualising and analysing
the resulting time-series feature matrix. https://github.com/benfulcher/hctsa
824
Python pyopy Python binding for hctsa. https://github.com/strawlab/pyopy

R fforma Tools for forecasting using a model combination approach. It can be used for model averaging
or model selection. It works by training a ‘classifier’ that learns to select/combine different
forecast models. https://github.com/pmontman/fforma
R gratis Efficient algorithms for generating time series with diverse and controllable characteristics,
which can be used as the training data in feature-based time series forecasting.
https://cran.r-project.org/package=gratis
R seer Implementations of a novel framework for forecast model selection using time series features;

FFORMS (Feature-based FORecast Model Selection). https://cran.r-project.org/package=seer
Section 2.7.5. Forecasting with R bootstrap Various bootstrapping functions. https://cran.r-project.org/package=bootstrap
bootstrap
Gretl uniFCextensions uniFCboot Estimate an interval forecast without assuming Gaussian innovations.
Section 2.7.6. Bagging for time R forecast baggedETS Returns forecasts and other information for bagged ETS models.
series forecasting https://cran.r-project.org/package=forecast
R tshacks baggedClusterETS; treatedETS Returns forecasts for bagged Cluster and Treated ETS models.
https://github.com/tiagomendesdantas/tshacks
Section 2.7.8. Neural Networks Python, GluonTS Framework for building deep learning based models including a number of pre-built models
MXNet & such as feed-forward neural networks. https://github.com/awslabs/gluon-ts
PyTorch
R nnfor mlp; elm Time Series Forecasting with Neural Networks. https://cran.r-project.org/package=nnfor
Python neural prophet Reimplementation of prophet in PyTorch, and extensions to it.
https://github.com/ourownstory/neural_prophet
Python RNNs for forecasting in Tensorflow. https://github.com/HansikaPH/time-series-forecasting
Library/
Toolbox
R ANN2 neuralnetwork Artificial Neural Networks. https://cran.r-project.org/package=ANN2
R nnet nnet Feed-Forward Neural Networks and Multinomial Log-Linear Models.
https://cran.r-project.org/package=nnet
R forecast nnetar Auto-regressive Neural Network for time series forecasting.
Section 2.7.9. Deep Python, GluonTS Framework for building deep learning based models including a number of pre-built models
Probabilistic models MXNet & such as DeepAR, DeepState models and NBEATS. https://github.com/awslabs/gluon-ts
PyTorch
Python, PyTorchTS Clone of GluonTS in PyTorch. https://github.com/zalandoresearch/pytorch-ts
PyTorch
Section 2.7.10. Machine R RSNNS mlp; rbf; dlvq; elman; jordan; Neural Networks using the Stuttgart Neural Network Simulator (SNNS).
learning som https://cran.r-project.org/package=RSNNS
R rpart rpart; prune Recursive partitioning and regression trees. https://cran.r-project.org/package=rpart
R caret Classification and regression training. https://cran.r-project.org/package=caret
R e1071 svm Misc ML functions of the Department of Statistics, Probability Theory Group.
https://cran.r-project.org/package=e1071
R kernlab gausspr Gaussian processes for regression and classification. https://cran.r-project.org/package=kernlab
R brnn brnn Bayesian Regularisation for Feed-Forward Neural Networks.
825
https://cran.r-project.org/package=brnn
R grnn grnn General regression neural network. https://cran.r-project.org/package=grnn
R randomForest randomForest Breiman and Cutler’s Random Forests for Classification and Regression.
https://cran.r-project.org/package=randomForest
R gbm gbm Generalised Boosted regression models. https://cran.r-project.org/package=gbm
R neuralnet neuralnet Training of simple Neural Networks. https://cran.r-project.org/package=neuralnet
Python Tensorflow A framework, developed by Google, offering tools for designing, building, and deploying ML
models. https://tensorflow.org/api_docs/python/tf
Python Keras API A deep learning API built on top of Tensorflow. It provides high level blocks for building and

training NN models. https://keras.io/
R Tensorflow R Interface to Tensorflow. https://tensorflow.rstudio.com/
R deepnet Deep learning toolkit. https://cran.r-project.org/package=deepnet
R h2o R Interface for the ’H2O’ Scalable Machine Learning Platform.
https://cran.r-project.org/package=h2o
R Apache MXNet A flexible library for deep learning. https://mxnet.apache.org/
Python scikit-learn Ordinary Least Squares, Ridge regression, Lasso, Bayesian Regression, Generalized Linear
Regression, Stochastic Gradient Descent and Polynomial regression, Support Vector Machines,
Nearest Neighbors, Gaussian Processes, Decision Trees, Ensemble methods (Forests of
randomised trees, AdaBoost and Gradient Tree Boosting), Multi-layer Perceptrons.
https://scikit-learn.org/stable/
Python CNTK A framework, developed by Microsoft, that provides tools for building ML and DL models.
https://docs.microsoft.com/en-us/cognitive-toolkit/
Python PyTorch A framework, developed by Facebook, for building ML and DL models. https://pytorch.org/

Library/
Toolbox
Section 2.7.12. Clustering-based R tsfknn tsfknn Time Series Forecasting Using Nearest Neighbours. https://cran.r-project.org/package=tsfknn
forecasting
Section 2.7.13. Hybrid methods Python ESRNN-GPU A GPU-enabled version of the hybrid model used by the winner of M4 competition.
https://github.com/damitkwr/ESRNN-GPU
Section 2.8.1. Parametric R tsintermittent crost; tsb Parametric forecasting methods for intermitternt demand.
methods for intermittent https://cran.r-project.org/package=tsintermittent
demand forecasting
R forecast croston Forecasts for intermittent demand using Croston’s method.
Section 2.8.2. Non-parametric R tsintermittent imapa MAPA for intermittent demand data. https://cran.r-project.org/package=tsintermittent
intermittent demand methods
Section 2.8.3. Classification R tsintermittent idclass Time series categorisation for intermittent demand.
methods https://cran.r-project.org/package=tsintermittent
R tsutils abc; xyz; abcxyz Classification functions and routines. https://cran.r-project.org/package=tsutils
Section 2.9.3. Forecasting with Python NLTK The Natural Language Toolkit in Python. https://www.nltk.org/
text information
Python SpaCy An open source library for advanced Natural Language Processing in Python. https://spacy.io/
Section 2.10.1. Cross-sectional R hts Functions and routines for hierarchical and grouped time series forecasting.
hierarchical forecasting https://cran.r-project.org/package=hts
Section 2.10.2. Temporal R MAPA mapa; mapasimple Functions and wrappers for using the Multiple Aggregation Prediction Algorithm (MAPA) for
826
aggregation time series forecasting. https://cran.r-project.org/package=MAPA

R thief thief Temporal Hierarchical Forecasting. https://cran.r-project.org/package=thief
R tsintermittent imapa MAPA for intermittent demand data with automatic model selection based on the PK
classification. https://cran.r-project.org/package=tsintermittent
Section 2.10.4. Ecological R ei ei Returns local and global forecasts of inner cells in 2 × 2 tables.
inference forecasting https://cran.r-project.org/package=ei
R eiPack ei.MD.bayes; ei.reg; ei.reg.bayes Returns local and global forecasts of inner cells in R × C tables under a Multinomial Dirichlet
model or using ecological regression. https://cran.r-project.org/package=eiPack

R lphom lphom; tslphom; nslphom Returns forecasts of inner cells of a R × C table using linear programming optimisation.
https://CRAN.R-project.org/package=lphom
R eiCompare ei_est_gen; ei_good; ei_rxc Returns forecasts of inner cells of a R × C tables using iterative versions of 2 × 2 methods
and the Multinomial Dirichlet model. https://cran.r-project.org/package=eiCompare
Section 2.12.2. Point, interval, R forecast accuracy Accuracy measures for a forecast model. https://cran.r-project.org/package=forecast
and pHDR forecast error
measures
Section 2.12.4. Evaluating R scoringRules Scoring rules for parametric and simulated distribution forecasts.
probabilistic forecasts https://cran.r-project.org/package=scoringRules
R verification crps Continuous ranked probability score. https://cran.r-project.org/package=verification
Section 2.12.6. Statistical tests R forecast dm.test Diebold–Mariano test for predictive accuracy. https://cran.r-project.org/package=forecast
of forecast performance
R tsutils nemenyi Nonparametric multiple comparisons (Nemenyi test). https://cran.r-project.org/package=tsutils
Gretl FEP doMZtest; doHPtest; doEKTtest; Various statistical tests on forecast unbiasedness, efficiency, asymmetric loss and directional
doPTtest; doDLtest; doDMtest; changes. http://gretl.sourceforge.net/
doGWtest; doCWtest
Gretl DiebMar Diebold–Mariano test. http://gretl.sourceforge.net/
Table C.2
A list of indicative publicly available data sets.
Related section Description Link
Section 2.7.9. Deep Data for wikipedia page views, Dominicks https://gluon-ts.mxnet.io/api/gluonts/gluonts.
Probabilistic Models retail, electricity consumption, traffic lane dataset.repository.datasets.html
occupation.
Section 2.9.3. Forecasting Movie reviews data provided by the Stanford https://nlp.stanford.edu/sentiment/code.html
with text Information NLP group.
Section 2.10.4. Ecological Party registration in South-East North Carolina https://www2.ncleg.net/RnR/Redistricting/
inference forecasting (eiPack R package). BaseData2001
ei.Datasets: Real Datasets for Assessing https://cran.csiro.au/web/packages/ei.Datasets/
Ecological Inference Algorithms. index.html
Section 2.12.7. Data for the M, M2, M3 and M4 forecasting https://forecasters.org/resources/time-series-
Forecasting competitions competitions. data/
Time Series Competition Data (R package) https://github.com/robjhyndman/tscompdata
Mcomp: Data for the M and M3 forecasting https://cran.r-project.org/package=Mcomp
competitions (R package).
M4comp2018: Data for the M4 forecasting https://github.com/carlanetto/M4comp2018
competition (R package).
Data for the M4 forecasting competition (csv https://github.com/Mcompetitions/M4-
files). methods/tree/master/Dataset
Tcomp: Data from the 2010 Tourism https://cran.r-project.org/package=Tcomp
forecasting competition (R package)
Data for the M5 forecasting competition (csv https://github.com/Mcompetitions/M5-
files). methods/tree/master/Dataset
Section 3.2.3. Forecasting Grupo Bimbo Inventory Demand. https://www.kaggle.com/c/grupo-bimbo-
for inventories inventory-demand
Section 3.2.4. Forecasting Rossmann Store Sales. https://www.kaggle.com/c/rossmann-store-
in retail sales
Corporación Favorita Grocery Sales Forecasting. https://www.kaggle.com/c/favorita-grocery-
sales-forecasting
Walmart Recruiting – Store Sales Forecasting. https://www.kaggle.com/c/walmart-recruiting-
store-sales-forecasting
Walmart Recruiting II: Sales in Stormy https://www.kaggle.com/c/walmart-recruiting-
Weather. sales-in-stormy-weather
Store Item Demand Forecasting Challenge. https://www.kaggle.com/c/demand-forecasting-
kernels-only
Online Product Sales. https://www.kaggle.com/c/online-sales
Section 3.2.8. Predictive Robot Execution Failures. https://archive.ics.uci.edu/ml/datasets/Robot+
maintenance Execution+Failures
Gearbox Fault Detection. https://c3.nasa.gov/dashlink/resources/997/
Air Pressure System Failure at Scania Trucks. https://archive.ics.uci.edu/ml/datasets/
IDA2016Challenge
Generic, Scalable and Decentralised Fault https:
Detection for Robot Swarms //zenodo.org/record/831471#.WwQIPUgvxPY
Wind turbine data (e.g., failures). https://opendata.edp.com/pages/homepage/
Section 3.3. Economics Two Sigma Financial Modelling Challenge. https://www.kaggle.com/c/two-sigma-
and finance financial-modeling/overview
Financial, economic, and alternative data sets, https://www.quandl.com/
serving investment professionals.
Section 3.3.2. Forecasting Repository website with Dynare codes and https://github.com/johannespfeifer/dsge_mod
GDP and Inflation data sets to estimate different DSGE models
and use them for forecasting.
Data set for Macroeconomic variables for US https://fred.stlouisfed.org/
economy.
Data set for Macroeconomic variables for https://data.oecd.org/
OECD economy.
Data set for Macroeconomic variables for EU https://ec.europa.eu/eurostat/data/database
economy.
Section 3.3.7. House Zillow Prize: Zillow’s Home Value Prediction https://www.kaggle.com/c/zillow-prize-1
price forecasting (Zestimate).
Sberbank Russian Housing Market. https://www.kaggle.com/c/sberbank-russian-
housing-market
Appendix B. Software Appendix C. Data sets
See Table B.1. See Table C.2.

827
Table C.2 (continued).

Western Australia Rental Prices. https://www.kaggle.com/c/deloitte-western-australia-
rental-prices
Section 3.3.12. Algorithmic Trading Challenge. https://www.kaggle.com/c/AlgorithmicTradingChallenge
Forecasting returns to
investment style
Section 3.3.13. The Winton Stock Market Challenge. https://www.kaggle.com/c/the-winton-stock-market-
Forecasting stock returns challenge
The Big Data Combine Engineered by BattleFin. https://www.kaggle.com/c/battlefin-s-big-data-combine-
forecasting-challenge/data
Section 3.4. Energy VSB Power Line Fault Detection. https://www.kaggle.com/c/vsb-power-line-fault-
detection
ASHRAE – Great Energy Predictor III https://www.kaggle.com/c/ashrae-energy-prediction
Section 3.4.3. Hybrid Global Energy Forecasting Competition 2012 – https://www.kaggle.com/c/global-energy-forecasting-
machine learning system Load Forecasting. competition-2012-load-forecasting
for short-term load
forecasting
Section 3.4.6. Wind Global Energy Forecasting Competition 2012 – https://www.kaggle.com/c/GEF2012-wind-forecasting
power forecasting Wind Forecasting.
Section 3.4.8. Solar Power measurements from a PV power plant https://doi.org/10.25747/edf8-m258
power forecasting and grid of numerical weather predictions.
AMS 2013–2014 Solar Energy Prediction https://www.kaggle.com/c/ams-2014-solar-energy-
Contest. prediction-contest
SolarTechLab data set. https://ieee-dataport.org/open-access/photovoltaic-
power-and-weather-parameters
Section 3.4.9. Long-term Brazilian National Electric Systems Operator http://www.ons.org.br/Paginas/resultados-da-
simulation for large (hydro, solar, wind, nuclear and thermal operacao/historico-da-operacao/geracao_energia.aspx
electrical power systems generation data).
Section 3.4.10. Solar power time series from 44 small-scale https://doi.org/10.25747/gywm-9457
Collaborative forecasting PV in Évora, Portugal.
in the energy sector
Australian Electricity Market Operator (AEMO) https://doi.org/10.15129/9e1d9b96-baa7-4f05-93bd-
5 Minute Wind Power Data. 99c5ae50b141
Electrical energy consumption data from
domestic consumers.
Electric vehicles charging data (arrivals, https://eatechnology.com/consultancy-insights/my-
departures, current, voltage, etc.). electric-avenue/
Wind power plant data and numerical weather https://challengedata.ens.fr/challenges/34
predictions from CNR (France).
Section 3.5.2. Weather How Much Did It Rain? https://www.kaggle.com/c/how-much-did-it-rain-ii
forecasting
Section 3.5.3. Air quality EMC Data Science Global Hackathon (Air https://www.kaggle.com/c/dsg-hackathon/overview
forecasting Quality Prediction).
Section 3.6. Social good LANL Earthquake Prediction. https://www.kaggle.com/c/LANL-Earthquake-Prediction
and demographic
forecasting
Section 3.6.1. Healthcare Flu Forecasting. https://www.kaggle.com/c/genentech-flu-forecasting
West Nile Virus Prediction. https://www.kaggle.com/c/predict-west-nile-virus
Section 3.6.2. Epidemics COVID-19 Data Repository by the Center for https://github.com/CSSEGISandData/COVID-19
and pandemics Systems Science and Engineering (CSSE) at
Johns Hopkins University.
Section 3.6.3. Forecasting Human Mortality Database. https://www.mortality.org
mortality
EuroMOMO. https://www.euromomo.eu/
The Economist. https://github.com/TheEconomist/covid-19-excess-
deaths-tracker
The New York Times. https://github.com/Financial-Times/coronavirus-excess-
mortality-data
The Financial Times. https://github.com/nytimes/covid-19-
data/tree/master/excess-deaths
ANACONDA- Quality assessment of mortality https://crvsgateway.info/anaconda
data.
Australian Human Mortality Database. https://demography.cass.anu.edu.au/research/australian-
human-mortality-database
828

Canadian Human Mortality Database. http://www.bdlc.umontreal.ca/CHMD/
French Human Mortality Database. https://frdata.org/fr/french-human-mortality-database
Section 3.6.4. Forecasting Human Fertility Database: fertility data for https://www.humanfertility.org/
fertility developed countries with complete birth
registration based on official vital statistics.
World Fertility Data: UN’s collection of fertility https://www.un.org/development/desa/pd/data/world-
data based on additional data sources such as fertility-data
surveys.
Section 3.6.5. Forecasting Integrated Modelling of European Migration https://www.imem.cpc.ac.uk/
migration (IMEM) Database, with estimates of migration
flows between 31 European countries by
origin, destination, age and sex, for 2002–2008.
QuantMig data inventory: meta-inventory on https://quantmig.eu/data_inventory/
different sources of data on migration and its
drivers, with European focus.
Bilateral international migration flow estimates https://doi.org/10.1038/s41597-019-0089-3
for 200 countries (Abel & Cohen, 2019).
UN World Population Prospects: UN global https://population.un.org/wpp/
population estimates and projections, including
probabilistic
Section 3.8. Other Forecast Eurovision Voting. https://www.kaggle.com/c/Eurovision2010
applications
Reducing Commercial Aviation Fatalities. https://www.kaggle.com/c/reducing-commercial-
aviation-fatalities
Porto Seguro’s Safe Driver Prediction. https://www.kaggle.com/c/porto-seguro-safe-driver-
prediction
Recruit Restaurant Visitor Forecasting. https://www.kaggle.com/c/recruit-restaurant-visitor-
forecasting
Restaurant Revenue Prediction. https://www.kaggle.com/c/restaurant-revenue-prediction
Coupon Purchase Prediction. https://www.kaggle.com/c/coupon-purchase-prediction
Bike Sharing Demand https://www.kaggle.com/c/bike-sharing-demand
Google Analytics Customer Revenue Prediction. https://www.kaggle.com/c/ga-customer-revenue-
prediction
Santander Value Prediction Challenge. https://www.kaggle.com/c/santander-value-prediction-
challenge
Santander Customer Transaction Predictio.n https://www.kaggle.com/c/santander-customer-
transaction-prediction
Acquire Valued Shoppers Challenge https://www.kaggle.com/c/acquire-valued-shoppers-
challenge
Risky Business https://www.kaggle.com/c/risky-business
Web Traffic Time Series Forecasting. https://www.kaggle.com/c/web-traffic-time-series-
forecasting
A repository of data sets, including time series https://github.com/awesomedata/awesome-public-
ones, that can be used for benchmarking datasets
forecasting methods in various applications of
interest.
WSDM – KKBox’s Churn Prediction Challenge. https://www.kaggle.com/c/kkbox-churn-prediction-
challenge/overview
Homesite Quote Conversion https://www.kaggle.com/c/homesite-quote-conversion
Liberty Mutual Group: Property Inspection https://www.kaggle.com/c/liberty-mutual-group-
Prediction. property-inspection-prediction
Liberty Mutual Group – Fire Peril Loss Cost. https://www.kaggle.com/c/liberty-mutual-fire-peril
A set of more than 490,000 time series (micro, http://fsudataset.com/
macro, demographic, finance, other) to
download.
A repository of data sets, including time series https://github.com/awesomedata/awesome-public-
ones, that can be used for benchmarking datasets
forecasting methods in various applications of
interest.
829

Section 3.8.1. Tourism TourMIS: Database with annual and monthly https://www.tourmis.info/index_e.html
demand forecasting tourism time series (e.g., arrivals and
bednights) covering European countries,
regions, and cities (free registration required).
Tourism Forecasting. https://www.kaggle.com/c/tourism1;
https://www.kaggle.com/c/tourism2
Section 3.8.2. Forecasting Airline and Airport performance data provided https://www.transtats.bts.gov/
for aviation by the U.S. Department of Transportation
Section 3.8.3. Traffic flow New York City Taxi Fare Prediction. https://www.kaggle.com/c/demand-forecasting-kernels-
forecasting only
RTA Freeway Travel Time Prediction https://www.kaggle.com/c/RTA
ECML/PKDD 15: Taxi Trip Time Prediction. https://www.kaggle.com/c/pkdd-15-taxi-trip-time-
prediction-ii
BigQuery-Geotab Intersection Congestion. https://www.kaggle.com/c/bigquery-geotab-intersection-
congestion/overview
Traffic volume counts collected by DOT for https://data.cityofnewyork.us/Transportation/Traffic-
New York Metropolitan Transportation Council. Volume-Counts-2014--2019-/ertz-hr4r
Section 3.8.5. Elections New Zealand General Elections – Official https://www.electionresults.govt.nz
forecasting results and statistics.
Spanish Elections – Official results and https://dataverse.harvard.edu/dataverse/SEA
statistics.
Section 3.8.6. Sports NFL Big Data Bowl. https://www.kaggle.com/c/nfl-big-data-bowl-2020
forecasting
Section 3.8.9. Forecasting Microsoft Malware Prediction. https://www.kaggle.com/c/microsoft-malware-prediction
under data integrity
attacks
Acquaviva, A., Apiletti, D., Attanasio, A., Baralis, E., Castagnetti, F.

B., Cerquitelli, T., et al. (2015). Enhancing energy awareness
through the analysis of thermal energy consumption. In EDBT/ICDT
References workshops (pp. 64–71).
Adams, W., & Michael, V. (1987). Short-term forecasting of passenger
Aaltonen, K. (2011). Project stakeholder analysis as an environmental demand and some application in quantas. In AGIFORS symposium
interpretation process. International Journal of Project Management, proc, vol. 27.
Afanasyev, D. O., & Fedorova, E. A. (2019). On the impact of outlier
29(2), 165–183.
filtering on the electricity price forecasting accuracy. Applied Energy,
Aastveit, K. A., Mitchell, J., Ravazzolo, F., & van Dijk, H. K. (2019).
236, 196–210.
The evolution of forecast density combinations in economics. Oxford Agarwal, A., Dahleh, M., & Sarkar, T. (2019). A marketplace for data:
University Press. An algorithmic solution. In Proceedings of the 2019 ACM conference
Abadi, M., Chu, A., Goodfellow, I., McMahan, H. B., Mironov, I., Tal- on economics and computation (pp. 701–726).
war, K., et al. (2016). Deep learning with differential privacy. Aggarwal, C., & Zhai, C. (2012). Mining text data. Springer Science &
In Proc. of the 2016 ACM SIGSAC conference on computer and Business Media.
communications security (ACM CCS) (pp. 308–318). Agnolucci, P. (2009). Volatility in crude oil futures: A comparison of the
Abel, G. J. (2018). Non-zero trajectories for long-run net migration predictive ability of GARCH and implied volatility models. Energy
assumptions in global population projection models. Demographic Economics, 31(2), 316–321.
Research, 38(54), 1635–1662. Ahlburg, D. A., & Vaupel, J. W. (1990). Alternative projections of the
Abel, G. J., & Cohen, J. E. (2019). Bilateral international migration flow U.S. population. Demography, 27(4), 639–652.
Ahmad, A., Hassan, M., Abdullah, M., Rahman, H., Hussin, F., Abdul-
estimates for 200 countries. Scientific Data, 6(1), 82.
lah, H., et al. (2014). A review on applications of ANN and SVM for
Abouarghoub, W., Nomikos, N. K., & Petropoulos, F. (2018). On rec-
building electrical energy consumption forecasting. Renewable and
onciling macro and micro energy transport forecasts for strategic
Sustainable Energy Reviews, 33, 102–109.
decision making in the tanker industry. Transportation Research Part Ahmad, A., Javaid, N., Mateen, A., Awais, M., & Khan, Z. A. (2019).
E: Logistics and Transportation Review, 113, 225–238. Short-term load forecasting in smart grids: An intelligent modular
AbouZahr, C., de Savigny, D., Mikkelsen, L., Setel, P. W., Lozano, R., approach. Energies, 12(1), 164.
& Lopez, A. D. (2015). Towards universal civil registration and Ahmad, M. W., Mourshed, M., & Rezgui, Y. (2017). Trees vs neurons:
vital statistics systems: the time is now. The Lancet, 386(10001), Comparison between random forest and ANN for high-resolution
1407–1418. prediction of building energy consumption. Energy and Buildings,
Abraham, B., & Box, G. E. P. (1979). BayesIan analysis of some outlier 147, 77–89.
problems in time series. Biometrika, 66(2), 229–236. Aït-Sahalia, Y., Cacho-Diaz, J., & Laeven, R. J. A. (2015). Modeling
Abraham, B., & Chuang, A. (1989). Outlier detection and time series financial contagion using mutually exciting jump processes. Journal
modeling. Technometrics, 31(2), 241–248. of Financial Economics, 117(3), 585–606.
Aizenman, J., & Jinjarak, Y. (2013). Real estate valuation, current
Abramson, B., & Finizza, A. (1991). Using belief networks to forecast
account and credit growth patterns, before and after the 2008-9
oil prices. International Journal of Forecasting, 7(3), 299–315.
crisis. Tech. Rep. 19190, National Bureau of Economic Research.
Abramson, B., & Finizza, A. (1995). Probabilistic forecasts from proba- Akaike, H. (1973). Information theory and an extension of the max-
bilistic models: a case study in the oil market. International Journal imum likelihood principle. In B. N. Petrov, & F. Csáki (Eds.),
of Forecasting, 11(1), 63–72. Proceedings of the second international symposium on information
Abramson, G., & Zanette, D. (1998). Statistics of extinction and survival theory (pp. 267–281). Budapest: Csáki.
in Lotka–Volterra systems. Physical Review E, 57, 4572–4577. Akouemo, H. N., & Povinelli, R. J. (2016). Probabilistic anomaly detection
Achen, C. H., & Phillips Shively, W. (1995). Cross-level inference. in natural gas time series data. International Journal of Forecasting,
University of Chicago Press. 32(3), 948–956.
830
Akram, F., Binning, A., & Maih, J. (2015). Joint prediction bands for Anderson, J. L. (1996). A method for producing and evaluating prob-
macroeconomic risk management. Tech. Rep. No. 5/2016, Centre for abilistic forecasts from ensemble model integrations. Journal of
Applied Macro- and Petroleum economics (CAMP) Working Paper Climate, 9, 1518–1530.
Series. Anderson, B. D. O., & Moore, J. B. (1979). Optimal filtering. Englewood
Aksin, Z., Armony, M., & Mehrotra, V. (2007). The modern call center: A Cliffs, NJ: Prentice-Hall.
multi-disciplinary perspective on operations management research. Anderson, V. O., & Nochmals, U. (1914). The elimination of spurious
Production and Operations Management, 16(6), 665–688. correlation due to position in time or space. Biometrika, 10(2/3),
Aktekin, T., & Soyer, R. (2011). Call center arrival modeling: A Bayesian 269–279.
state-space approach. Naval Research Logistics, 58(1), 28–42. Andersson, E., Kühlmann-Berenzon, S., Linde, A., Schiöler, L., Rubi-
Al-Azzani, M. A., Davari, S., & England, T. J. (2020). An empirical nova, S., & Frisén, M. (2008). Predictions by early indicators of
investigation of forecasting methods for ambulance calls-a case the time and height of the peaks of yearly influenza outbreaks in
study. Health Systems, 1–18. Sweden. Scandinavian Journal of Public Health, 36(5), 475–482.
Al-Homoud, M. S. (2001). Computer-aided building energy analysis Andrade, J., Filipe, J., Reis, M., & Bessa, R. (2017). Probabilistic price fore-
techniques. Building and Environment, 36(4), 421–433. casting for day-ahead and intraday markets: Beyond the statistical
Albon, C. (2018). Python machine learning cookbook. O’Reilly UK Ltd.. model. Sustainability, 9(1990), 1–29.
Albulescu, C. T., Tiwari, A. K., & Ji, Q. (2020). Copula-based local Andrawis, R. R., Atiya, A. F., & El-Shishiny, H. (2011). Combina-
dependence among energy, agriculture and metal commodities tion of long term and short term forecasts, with application to
markets. Energy, 202, Article 117762. tourism demand forecasting. International Journal of Forecasting,
Aldor-Noiman, S., Feigin, P. D., & Mandelbaum, A. (2009). Workload 27(3), 870–886.
forecasting for a call center: Methodology and a case study. The Andrés, M. A., Peña, D., & Romo, J. (2002). Forecasting time series with
Annals of Applied Statistics, 3(4), 1403–1447. sieve bootstrap. Journal of Statistical Planning and Inference, 100(1),
Alexandrov, A., Benidis, K., Bohlke-Schneider, M., Flunkert, V., 1–11.
Gasthaus, J., Januschowski, T., et al. (2019). GluonTS: Probabilistic Andrews, B. H., & Cunningham, S. M. (1995). LL Bean improves
time series models in python. Journal of Machine Learning Research. call-center forecasting. Interfaces, 25(6), 1–13.
Alho, J. M., Hougaard Jensen, S. E., & Lassila, J. (Eds.), (2008). Uncertain Andrews, R. L., Currim, I. S., Leeflang, P., & Lim, J. (2008). Estimating the
demographics and fiscal sustainability. Cambridge University Press. SCAN* PRO model of store sales: HB, FM or just OLS? International
Alho, J. M., & Spencer, B. D. (1985). Uncertain population forecasting. Journal of Research in Marketing, 25(1), 22–33.
Journal of the American Statistical Association, 80(390), 306–314. Andrieu, C., Doucet, A., & Holenstein, R. (2011). Particle Markov
Alho, J. M., & Spencer, B. D. (2005). Statistical demography and chain Monte Carlo. Journal of the Royal Statistical Society. Series B.
forecasting. New York: Springer. Statistical Methodology, 72(2), 269–342.
Andrieu, C., & Roberts, G. (2009). The pseudo-marginal approach for
Ali, M. M., & Boylan, J. E. (2011). Feasibility principles for downstream
efficient Monte Carlo computations. The Annals of Statistics, 37(2),
demand inference in supply chains. Journal of the Operational
697–725.
Research Society, 62(3), 474–482.
Aneiros-Pérez, G., & Vieu, P. (2008). Nonparametric time series
Ali, M. M., Boylan, J. E., & Syntetos, A. A. (2012). Forecast errors
prediction: A semi-functional partial linear modeling. Journal of
and inventory performance under forecast information sharing.
Multivariate Analysis, 99(5), 834–857.
International Journal of Forecasting, 28(4), 830–841.
Ang, A., & Bekaert, G. (2002). Short rate nonlinearities and regime
Alizadeh, S., Brandt, M. W., & Diebold, F. X. (2002). Range-based
switches. Journal of Economic Dynamics & Control, 26(7), 1243–1274.
estimation of stochastic volatility models. The Journal of Finance,
Ang, A., Bekaert, G., & Wei, M. (2008). The term structure of real rates
57(3), 1047–1091.
and expected inflation. The Journal of Finance, 63(2), 797–849.
Alkema, L., Raftery, A. E., Gerland, P., Clark, S. J., Pelletier, F., Buettner, T.,
Angelini, G., & De Angelis, L. (2019). Efficiency of online football betting
et al. (2011). Probabilistic projections of the total fertility rate for
markets. International Journal of Forecasting, 35(2), 712–721.
all countries. Demography, 48(3), 815–839.
Angus, J. E. (1992). Asymptotic theory for bootstrapping the extremes.
Almeida, C., & Czado, C. (2012). Efficient Bayesian inference for stochas-
Communications in Statistics. Theory and Methods, 22(1), 15–30.
tic time-varying copula models. Computational Statistics & Data Anselin, L., & Tam Cho, W. K. (2002). Spatial effects and ecological
Analysis, 56(6), 1511–1527. inference. Political Analysis, 10(3), 276–297.
Aloui, R., Hammoudeh, S., & Nguyen, D. K. (2013). A time-varying Antipov, A., & Meade, N. (2002). Forecasting call frequency at a financial
copula approach to oil and stock market dependence: The case of services call centre. Journal of the Operational Research Society, 53(9),
transition economies. Energy Economics, 39, 208–221. 953–960.
Alquist, R., Bhattarai, S., & Coibion, O. (2020). Commodity-price Antonakakis, N., Chatziantoniou, I., Floros, C., & Gabauer, D. (2018). The
comovement and global economic activity. Journal of Monetary dynamic connectedness of U.K. regional property returns. Urban
Economics, 112, 41–56. Studies, 55(14), 3110–3134.
Alquist, R., Kilian, L., & Vigfusson, R. J. (2013). Forecasting the price Apiletti, D., Baralis, E., Cerquitelli, T., Garza, P., Michiardi, P., & Pul-
of oil. vol. 2, In Handbook of economic forecasting (pp. 427–507). virenti, F. (2015). Pampa-HD: A parallel MapReduce-based frequent
Elsevier. pattern miner for high-dimensional data. In 2015 IEEE international
Alvarado-Valencia, J. A., & Barrero, L. H. (2014). Reliance, trust and conference on data mining workshop (ICDMW) (pp. 839–846). IEEE.
heuristics in judgmental forecasting. Computers in Human Behavior, Apiletti, D., Baralis, E., Cerquitelli, T., Garza, P., Pulvirenti, F., &
36, 102–113. Michiardi, P. (2017). A parallel mapreduce algorithm to effi-
Alvarado-Valencia, J., Barrero, L. H., Önkal, D., & Dennerlein, J. T. (2017). ciently support itemset mining on high dimensional data. Big Data
Expertise, credibility of system forecasts and integration methods in Research, 10, 53–69.
judgmental demand forecasting. International Journal of Forecasting, Apiletti, D., & Pastor, E. (2020). Correlating espresso quality with
33(1), 298–313. coffee-machine parameters by means of association rule mining.
Alvarez-Ramirez, J., Soriano, A., Cisneros, M., & Suarez, R. (2003). Electronics, 9(1), 100.
Symmetry/anti-symmetry phase transitions in crude oil markets. Apiletti, D., Pastor, E., Callà, R., & Baralis, E. (2020). Evaluating espresso
Physica A: Statistical Mechanics and Its Applications, 322, 583–596. coffee quality by means of time-series feature engineering. In
Amarasinghe, A., Wichmann, O., Margolis, H. S., & Mahoney, R. T. EDBT/ICDT workshops.
(2010). Forecasting dengue vaccine demand in disease endemic and Archak, N., Ghose, A., & Ipeirotis, P. (2011). Deriving the pricing power
non-endemic countries. Human Vaccines, 6(9), 745–753. of product features by mining consumer reviews. Management
Amendola, A., Niglio, M., & Vitale, C. (2006). The moments of SETARMA Science, 57(8), 1485–1509.
models. Statistics & Probability Letters, 76(6), 625–633. Arinze, B. (1994). Selecting appropriate forecasting models using rule
Amisano, G., & Giacomini, R. (2007). Comparing density forecasts via induction. Omega, 22(6), 647–658.
weighted likelihood ratio tests. Journal of Business & Economic Arlot, S., & Celisse, A. (2010). A survey of cross-validation procedures
Statistics, 25(2), 177–190. for model selection. Statistics Surveys, 4, 40–79.
An, S., & Schorfheide, F. (2007). Bayesian analysis of DSGE models. Armstrong, J. S. (2001a). Principles of forecasting: A handbook for
Econometric Reviews, 26(2–4), 113–172. researchers and practitioners. Springer Science & Business Media.
831
Armstrong, J. S. (2001b). Combining forecasts. In International series in Atiya, A. F. (2020). Why does forecast combination work so
operations research & management science, Principles of forecasting well? International Journal of Forecasting, 36(1), 197–200, URL https:
(pp. 417–439). Springer, Boston, MA. //www.sciencedirect.com/science/article/pii/S0169207019300779,
Armstrong, J. S. (2007). Significance tests harm progress in forecasting. M4 Competition.
International Journal of Forecasting, 23(2), 321–327. Atiya, A. F., El-shoura, S. M., Shaheen, S. I., & El-sherif, M. S. (1999).
Armstrong, C. (2017). Omnichannel retailing and demand planning. The A comparison between neural-network forecasting techniques–case
Journal of Business Forecasting, 35(4), 10–15. study: river flow forecasting. IEEE Transactions on Neural Networks,
Armstrong, J. S., & Collopy, F. (1998). Integration of statistical methods 10(2), 402–409.
and judgment for time series forecasting: Principles from empirical Atkinson, A. C., Riani, M., & Corbellini, A. (2021). The Box–Cox
research. In G. Wright, & P. Goodwin (Eds.), Forecasting with transformation: Review and extensions. Statistical Science, 36(2),
judgment (pp. 269–293). New York: John Wiley & Sons Ltd. 239–255.
Armstrong, J. S., & Green, K. C. (2018). Forecasting methods and Aue, A., Norinho, D. D., & Hörmann, S. (2015). On the prediction of
principles: Evidence-based checklists. Journal of Global Scholars of stationary functional time series. Journal of the American Statistical
Marketing Science, 28(2), 103–159. Association, 110(509), 378–392.
Armstrong, J. S., Green, K. C., & Graefe, A. (2015). Golden rule of Austin, C., & Kusumoto, F. (2016). The application of big data in
forecasting: Be conservative. Journal of Business Research, 68(8), medicine: current implications and future directions. Journal of
1717–1731. Interventional Cardiac Electrophysiology, 47(1), 51–59.
Arnott, R. D., Beck, N., Kalesnik, V., & West, J. (2016). How can ‘smart Avramidis, A. N., Deslauriers, A., & L’Ecuyer, P. (2004). Modeling daily
beta’ go horribly wrong? SSRN:3040949. arrivals to a telephone call center. Management Science, 50(7),
Aron, J., & Muellbauer, J. (2020). Measuring excess mortality: 896–908.
the case of England during the Covid-19 pandemic. https: Axelrod, R. (1997). Advancing the art of simulation in the social
//www.oxfordmartin.ox.ac.uk/publications/measuring-excess- sciences. In Simulating social phenomena (pp. 21–40). Springer.
mortality-the-case-of-england-during-the-covid-19-pandemic/, Ayton, P., Önkal, D., & McReynolds, L. (2011). Effects of ignorance and
Accessed on 2020-08-20. information on judgments and decisions. Judgment and Decision
Arora, S., Taylor, J. W., & Mak, H.-Y. (2020). Probabilistic forecasting Making, 6(5), 381–391.
of patient waiting times in an emergency department. arXiv:2006. Azose, J. J., & Raftery, A. E. (2015). Bayesian probabilistic projection of
00335. international migration. Demography, 52(5), 1627–1650.
Azose, J. J., Ševčíková, H., & Raftery, A. E. (2016). Probabilistic pop-
Arrhenius, S. A. (1896). On the influence of carbonic acid in the
ulation projections with migration uncertainty. Proceedings of the
air upon the temperature of the ground. London, Edinburgh, and
National Academy of Sciences of the United States of America, 113(23),
Dublin Philosophical Magazine and Journal of Science (Fifth Series),
6460–6465.
41, 237–275.
Baade, R. A., & Matheson, V. A. (2016). Going for the gold: The
Artis, M., & Marcellino, M. (2001). Fiscal forecasting: The track record
economics of the olympics. Journal of Economic Perspectives, 30(2),
of the IMF, OECD and EC. The Econometrics Journal, 4(1), S20–S36.
201–218.
Arvan, M., Fahimnia, B., Reisi, M., & Siemsen, E. (2019). Integrating
Baardman, L., Levin, I., Perakis, G., & Singhvi, D. (2018). Leveraging
human judgement into quantitative forecasting methods: A review.
comparables for new product sales forecasting. Production and
Omega, 86, 237–252.
Operations Management, 27(12), 2340–2343.
Asai, M. (2013). Heterogeneous asymmetric dynamic conditional cor-
Babai, M. Z., Ali, M. M., & Nikolopoulos, K. (2012). Impact of temporal
relation model with stock return and range. Journal of Forecasting,
aggregation on stock control performance of intermittent demand
32(5), 469–480.
estimators: Empirical analysis. Omega, 40(6), 713–721.
Asai, M., & Brugal, I. (2013). Forecasting volatility via stock return,
Babai, M. Z., Dallery, Y., Boubaker, S., & Kalai, R. (2019). A new method
range, trading volume and spillover effects: The case of Brazil. North
to forecast intermittent demand in the presence of inventory
American Journal of Economics and Finance, 25, 202–213.
obsolescence. International Journal of Production Economics, 209,
Asimakopoulos, S., & Dix, A. (2013). Forecasting support systems
30–41.
technologies-in-practice: A model of adoption and use for product Babai, M. Z., Syntetos, A., & Teunter, R. (2014). Intermittent de-
forecasting. International Journal of Forecasting, 29(2), 322–336. mand forecasting: An empirical study on accuracy and the risk
Asimakopoulos, S., Paredes, J., & Warmedinger, T. (2020). Real-time of obsolescence. International Journal of Production Economics, 157,
fiscal forecasting using mixed-frequency data. Scandinavian Journal 212–219.
of Economics, 122, 369–390. Babai, M. Z., Tsadiras, A., & Papadopoulos, C. (2020). On the empirical
Askanazi, R., Diebold, F. X., Schorfheide, F., & Shin, M. (2018). On the performance of some new neural network methods for forecasting
comparison of interval forecasts. Journal of Time Series Analysis, intermittent demand. IMA Journal of Management Mathematics,
39(6), 953–965. 31(3), 281–305.
Asness, C. S. (2016). Invited editorial comment: the siren song of Babu, A., Levine, A., Ooi, Y. H., Pedersen, L. H., & Stamelos, E.
factor timing aka ‘‘smart beta timing’’ aka ‘‘style timing’’. Journal (2020). Trends everywhere. Journal of Investment Management,
of Portfolio Management, 42(5), 1–6. 18(1), 52–68.
Assimakopoulos, V., & Nikolopoulos, K. (2000). The theta model: a Bacchetti, A., & Saccani, N. (2012). Spare parts classification and de-
decomposition approach to forecasting. International Journal of mand forecasting for stock control: Investigating the gap between
Forecasting, 16(4), 521–530. research and practice. Omega, 40(6), 722–737.
Assmus, G. (1984). New product forecasting. Journal of Forecasting, 3(2), Baccianella, S., Esuli, A., & Sebastiani, F. (2010). Sentiwordnet 3.0:
121–138. an enhanced lexical resource for sentiment analysis and opinion
Athanasopoulos, G., Ahmed, R. A., & Hyndman, R. J. (2009). Hierarchical mining. In Proceedings of the seventh international conference on
forecasts for Australian domestic tourism. International Journal of language resources and evaluation (LREC’10), vol. 10 (pp. 2200–2204).
Forecasting, 25(1), 146–166. Bacha, H., & Meyer, W. (1992). A neural network architecture for
Athanasopoulos, G., Hyndman, R. J., Kourentzes, N., & Petropoulos, F. load forecasting. In IJCNN international joint conference on neural
(2017). Forecasting with temporal hierarchies. European Journal of networks, vol. 2 (pp. 442–447).
Operational Research, 262(1), 60–74. Baecke, P., De Baets, S., & Vanderheyden, K. (2017). Investigating
Athanasopoulos, G., Hyndman, R. J., Song, H., & Wu, D. C. (2011). The the added value of integrating human judgement into statistical
tourism forecasting competition. International Journal of Forecasting, demand forecasting systems. International Journal of Production
27(3), 822–844. Economics, 191, 85–96.
Athanasopoulos, G., Song, H., & Sun, J. A. (2018). Bagging in tourism Baicker, K., Chandra, A., & Skinner, J. S. (2012). Saving money or
demand modeling and forecasting. Journal of Travel Research, 57(1), just saving lives? Improving the productivity of US health care
52–68. spending. Annual Review of Economics, 4(1), 33–56.
Athey, S. (2018). The impact of machine learning on economics. In Baillie, R. T., & Bollerslev, T. (1992). Prediction in dynamic models
A. Agrawal, J. Gans, & A. Goldfarb (Eds.), The economics of artificial with time-dependent conditional variances. Journal of Econometrics,
intelligence: An Agenda (pp. 507–547). University of Chicago Press. 1–2(52), 91–113.
832
Baillie, R. T., Bollerslev, T., & Mikkelsen, H. O. (1996). Fractionally in- Barnichon, R., & Garda, P. (2016). Forecasting unemployment across
tegrated generalized autoregressive conditional heteroskedasticity. countries: The ins and outs. European Economic Review, 84,
Journal of Econometrics, 74(1), 3–30. 165–183.
Baker, J. (2021). Maximizing forecast value added through machine Barr, J. (2018). New – predictive scaling for EC2, powered by machine
learning and nudges. Foresight: The International Journal of Applied learning. In AWS news blog. https://aws.amazon.com/blogs/aws/
Forecasting, 60. new-predictive-scaling-for-ec2-powered-by-machine-learning,
Baker, S. R., Bloom, N., & Davis, S. J. (2016). Measuring economic policy Accessed on 2020-09-01.
uncertainty. Quarterly Journal of Economics, 131(4), 1593–1636. Barron, A. R. (1994). Approximation and estimation bounds for artificial
Balbo, N., Billari, F. C., & Mills, M. (2013). Fertility in advanced societies: neural networks. Machine Learning, 14(1), 115–133.
A review of research. European Journal of Population, 29(1), 1–38. Barroso, P. (2015). Momentum has its moments. Journal of Financial
Balke, N. S. (1993). Detecting level shifts in time series. Journal of Economics, 116(1), 111–121.
Business & Economic Statistics, 11(1), 81–92. Barrow, D. K., & Crone, S. F. (2016a). A comparison of AdaBoost algo-
Balke, N. S., & Fomby, T. B. (1997). Threshold cointegration. International rithms for time series forecast combination. International Journal of
Economic Review, 38(3), 627–645. Forecasting, 32(4), 1103–1119.
Bańbura, M., Giannone, D., & Reichlin, L. (2010). Large Bayesian vector Barrow, D. K., & Crone, S. F. (2016b). Cross-validation aggregation for
auto regressions. Journal of Applied Econometrics, 25(1), 71–92. combining autoregressive neural network forecasts. International
Bańbura, M., Giannone, D., & Reichlin, L. (2011). Nowcasting (chapter Journal of Forecasting, 32(4), 1120–1137.
7). In M. P. Clements, & D. F. Hendry (Eds.), The oxford handbook of Barrow, D. K., & Kourentzes, N. (2016). Distributions of forecast-
economic forecasting. Oxford University Press. ing errors of forecast combinations: implications for inventory
Bandara, K., Bergmeir, C., & Hewamalage, H. (2020a). LSTM-msnet: management. International Journal of Production Economics, 177,
leveraging forecasts on sets of related time series with multiple 24–33.
seasonal patterns. IEEE Transactions on Neural Networks and Learning Barrow, D., Kourentzes, N., Sandberg, R., & Niklewski, J. (2020).
Systems. Automatic robust estimation for exponential smoothing: Perspec-
Bandara, K., Bergmeir, C., & Smyl, S. (2020b). Forecasting across time tives from statistics and machine learning. Expert Systems with
series databases using recurrent neural networks on groups of sim- Applications, 160, Article 113637.
ilar series: a clustering approach. Expert Systems with Applications, Bartelsman, E. J., Kurz, C. J., & Wolf, Z. (2011). Using census microdata
140, Article 112896. to forecast US aggregate productivity. In Working paper.
Bartelsman, E. J., & Wolf, Z. (2014). Forecasting aggregate productivity
Bandyopadhyay, S. (2009). A dynamic model of cross-category com-
using information from firm-level data. The Review of Economics
petition: theory, tests and applications. Journal of Retailing, 85(4),
and Statistics, 96(4), 745–755.
468–479.
Bartezzaghi, E., Verganti, R., & Zotteri, G. (1999). A simulation frame-
Bangwayo-Skeete, P. F., & Skeete, R. W. (2015). Can google data im-
work for forecasting uncertain lumpy demand. International Journal
prove the forecasting performance of tourist arrivals? Mixed-data
of Production Economics, 59(1), 499–510.
sampling approach. Tourism Management, 46, 454–464.
Bass, F. (1969). A new product growth model for consumer durables.
Banks, J., Blundell, R., Oldfield, Z., & Smith, J. P. (2015). House price
Management Science, 15, 215–227.
volatility and the housing ladder. Tech. Rep. 21255, National Bureau
Bass, F. M., Gordon, K., Ferguson, T. L., & Githens, M. L. (2001). DIRECTV:
of Economic Research.
Forecasting diffusion of a new technology prior to product launch.
Bannister, R. N., Chipilski, H. G., & Martinez-Alvarado, O. (2020).
INFORMS Journal on Applied Analytics, 31(3S), S82–S93.
Techniques and challenges in the assimilation of atmospheric water
Bass, F., Krishnan, T., & Jain, D. (1994). Why the bass model fits without
observations for numerical weather prediction towards convective
decision variables. Marketing Science, 13, 203–223.
scales. Quarterly Journal of the Royal Meteorological Society, 146(726),
Bassetti, F., Casarin, R., & Ravazzolo, F. (2018). Bayesian nonparametric
1–48.
calibration and combination of predictive distributions. Journal of
Bansal, R., Tauchen, G., & Zhou, H. (2004). Regime shifts, risk premiums
the American Statistical Association, 113(522), 675–685.
in the term structure, and the business cycle. Journal of Business &
Basturk, N., Borowska, A., Grassi, S., Hoogerheide, L., & van Dijk, H.
Economic Statistics, 22(4), 396–409.
(2019). Forecast density combinations of dynamic models and data
Bansal, R., & Zhou, H. (2002). Term structure of interest rates with driven portfolio strategies. Journal of Econometrics, 210(1), 170–186.
regime shifts. The Journal of Finance, 57(5), 1997–2043. Basu, S., Fernald, J. G., Oulton, N., & Srinivasan, S. (2003). The case of
Banu, S., Hu, W., Hurst, C., & Tong, S. (2011). Dengue transmission the missing productivity growth, or does information technology
in the Asia-Pacific region: impact of climate change and socio- explain why productivity accelerated in the United States but not
environmental factors. Tropical Medicine & International Health, in the United Kingdom? NBER Macroeconomics Annual, 18, 9–63.
16(5), 598–607. Bates, J. M., & Granger, C. W. J. (1969). The combination of forecasts.
Bao, Y., Lee, T.-H., & Saltoglu, B. (2007). Comparing density forecast Journal of the Operational Research Society, 20(4), 451–468.
models. Journal of Forecasting, 26(3), 203–225. Baumeister, C., Guérin, P., & Kilian, L. (2015). Do high-frequency
Baptista, M., Sankararaman, S., de Medeiros, I. P., Nascimento, C., financial data help forecast oil prices? The midas touch at work.
Prendinger, H., & Henriques, E. M. P. (2018). Forecasting fault events International Journal of Forecasting, 31(2), 238–252.
for predictive maintenance using data-driven techniques and ARMA Baumeister, C., & Kilian, L. (2015). Forecasting the real price of oil
modeling. Computers & Industrial Engineering, 115, 41–53. in a changing world: a forecast combination approach. Journal of
Barbetta, S., Coccia, G., Moramarco, T., Brocca, L., & Todini, E. (2017). Business & Economic Statistics, 33(3), 338–351.
The multi temporal/multi-model approach to predictive uncertainty Beare, B. K., Seo, J., & Seo, W. (2017). Cointegrated linear processes in
assessment in real-time flood forecasting. Journal of Hydrology, 551, Hilbert space. Journal of Time Series Analysis, 38(6), 1010–1027.
555–576. Becker, R., Hurn, S., & Pavlov, V. (2008). Modelling spikes in electricity
BarclayHedge (2018). Survey: Majority of hedge fund pros use prices. The Economic Record, 83(263), 371–382.
AI/Machine learning in investment strategies. In BarclayHedge. Beckmann, M., & Bobkoski, F. (1958). Airline demand: An analysis
https://www.barclayhedge.com/insider/barclayhedge-survey- of some frequency distributions. Naval Research Logistics Quarterly,
majority-of-hedge-fund-pros-use-ai-machine-learning-in- 5(1), 43–51.
investment-strategies, Accessed on 2020-09-01. Beckmann, J., & Schussler, R. (2016). Forecasting exchange rates under
Barker, J. (2020). Machine learning in M4: What makes a good parameter and model uncertainty. Journal of International Money
unstructured model?. International Journal of Forecasting, 36(1), and Finance, 60, 267–288.
150–155. Behera, M. K., Majumder, I., & Nayak, N. (2018). Solar photovoltaic
Barnhart, C., & Cohn, A. (2004). Airline schedule planning: Accom- power forecasting using optimized modified extreme learning ma-
plishments and opportunities. Manufacturing & Service Operations chine technique. Engineering Science and Technology An International
Management, 6(1), 3–22. Journal, 21(3).
Barnhart, C., Fearing, D., & Vaze, V. (2014). Modeling passenger travel Bekaert, G., Hodrick, R. J., & Marshall, D. A. (2001). Peso problem
and delays in the national air transportation system. Operations explanations for term structure anomalies. Journal of Monetary
Research, 62(3), 580–601. Economics, 48(2), 241–270.
833
Bekiros, S., Cardani, R., Paccagnini, A., & Villa, S. (2016). Dealing with Berkowitz, J. (2001). Testing density forecasts, with applications to
financial instability under a DSGE modeling approach with banking risk management. Journal of Business & Economic Statistics, 19(4),
intermediation: A predictability analysis versus TVP-vars. Journal of 465–474.
Financial Stability, 26(C), 216–227. Berlinski, D. (2009). The devil’s delusion: atheism and its scientific
Bekiros, S. D., & Paccagnini, A. (2014). Bayesian forecasting with small pretensions. Basic Books.
and medium scale factor-augmented vector autoregressive DSGE Bernanke, B. S., Boivin, J., & Eliasz, P. (2005). Measuring the ef-
models. Computational Statistics & Data Analysis, 71(C), 298–323. fects of monetary policy: a factor-augmented vector autoregressive
Bekiros, S., & Paccagnini, A. (2015a). Estimating point and density (FAVAR) approach. Quarterly Journal of Economics, 120(1), 387–422.
forecasts for the US economy with a factor-augmented vec- Bernard, A., & Busse, M. (2004). Who wins the olympic games: Eco-
tor autoregressive DSGE model. Studies in Nonlinear Dynamics & nomic resources and medal totals. The Review of Economics and
Econometrics, 19(2), 107–136. Statistics, 86(1), 413–417.
Bekiros, S. D., & Paccagnini, A. (2015b). Macroprudential policy and Bernardini Papalia, R., & Fernandez Vazquez, E. (2020). Entropy-based
forecasting using hybrid DSGE models with financial frictions and solutions for ecological inference problems: A composite estimator.
state space Markov-switching tvp-vars. Macroeconomic Dynamics, Entropy, 22(7), 781.
19(7), 1565–1592. Bernardo, J. M. (1984). Monitoring the 1982 spanish socialist victory:
Bekiros, S. D., & Paccagnini, A. (2016). Policy oriented macroeconomic A Bayesian analysis. Journal of the American Statistical Association,
forecasting with hybrid DGSE and time varying parameter VAR 79(387), 510–515.
models. Journal of Forecasting, 35(7), 613–632. Bernardo, J. M. (1994). Bayesian theory. Wiley.
Bélanger, A., & Sabourin, P. (2017). Microsimulation and population Bernstein, R. (1995). Style investing. New York: John Wiley & Sons.
dynamics: an introduction to modgen 12. Springer, Cham. Berry, L. R., & West, M. (2020). BayesIan forecasting of many count-
Beliën, J., & Forcé, H. (2012). Supply chain management of blood prod- valued time series. Journal of Business & Economic Statistics, 38(4),
ucts: A literature review. European Journal of Operational Research, 872–887.
217(1), 1–16. Bertsimas, D., & Pachamanova, D. (2008). Robust multiperiod portfolio
Bell, F., & Smyl, S. (2018). Forecasting at uber: An introduction. management in the presence of transaction costs. Computers &
URL https://eng.uber.com/forecasting-introduction/, Accessed on Operations Research, 35(1), 3–17.
2020-09-02. Bessa, R. J., Miranda, V., Botterud, A., Zhou, Z., & Wang, J. (2012). Time-
Ben Taieb, S. (2014). Machine learning strategies for multi-step-ahead adaptive quantile-copula for wind power probabilistic forecasting.
time series forecasting. (Ph.D. thesis), Free University of Brussels Renewable Energy, 40(1), 29–39.
(ULB). Bessa, R., Möhrlen, C., Fundel, V., Siefert, M., Browell, J., Haglund
Ben Taieb, S., & Atiya, A. F. (2015). A bias and variance analysis for El Gaidi, S., et al. (2017). Towards improved understanding of the
multistep-ahead time series forecasting. IEEE Transactions on Neural applicability of uncertainty forecasts in the electric power industry.
Networks and Learning Systems, PP(99), 1. Energies, 10(9).
Ben Taieb, S., Bontempi, G., Atiya, A. F., & Sorjamaa, A. (2012). A Besse, P., Cardot, H., & Stephenson, D. (2000). Autoregressive forecast-
review and comparison of strategies for multi-step ahead time ing of some functional climatic variations. Scandinavian Journal of
series forecasting based on the NN5 forecasting competition. Expert Statistics, 27(4), 673–687.
Systems with Applications, 39(8), 7067–7083. Beyaztas, U., & Shang, H. L. (2019). Forecasting functional time se-
Ben Taieb, S., & Hyndman, R. (2014). Boosting multi-step autoregressive ries using weighted likelihood methodology. Journal of Statistical
forecasts. In Proceedings of the 31st international conference on Computation and Simulation, 89(16), 3046–3060.
machine learning (pp. 109–117). Bhansali, R. J., & Kokoszka, P. S. (2002). Computation of the forecast
Ben Taieb, S., Sorjamaa, A., & Bontempi, G. (2010). Multiple- coefficients for multistep prediction of long-range dependent time
output modeling for multi-step-ahead time series forecasting. series. International Journal of Forecasting, 18(2), 181–206.
Neurocomputing, 73(10–12), 1950–1957. Bianchi, L., Jarrett, J. E., & Hanumara, R. C. (1993). Forecasting incoming
Ben Taieb, S., Taylor, J. W., & Hyndman, R. J. (2020). Hierarchical calls to telemarketing centers. The Journal of Business Forecasting,
probabilistic forecasting of electricity demand with smart meter 12(2), 3.
data. Journal of the American Statistical Association. Bianchi, L., Jarrett, J., & Hanumara, R. C. (1998). Improving forecasting
Benati, L. (2007). Drift and breaks in labor productivity. Journal of for telemarketing centers by ARIMA modeling with intervention.
Economic Dynamics and Control, 31(8), 2847–2877. International Journal of Forecasting, 14(4), 497–504.
Bender, J., Sun, X., Thomas, R., & Zdorovtsov, V. (2018). The promises Bianco, A. M., García Ben, M., Martínez, E. J., & Yohai, V. J. (2001).
and pitfalls of factor timing. Journal of Portfolio Management, 44(4), Outlier detection in regression models with ARIMA errors using
79–92. robust estimates. Journal of Forecasting, 20(8), 565–579.
Bendre, M., & Manthalkar, R. (2019). Time series decomposition and Bickel, J. E. (2007). Some comparisons among quadratic, spherical, and
predictive analytics using MapReduce framework. Expert Systems logarithmic scoring rules. Decision Analysis, 4(2), 49–65.
with Applications, 116, 108–120. Bickel, P. J., & Doksum, K. A. (1981). An analysis of transformations
Benidis, K., Rangapuram, S. S., Flunkert, V., Wang, B., Maddix, D., revisited. Journal of the American Statistical Association, 76(374),
Turkmen, C., et al. (2020). Neural forecasting: Introduction and 296–311.
literature overview. arXiv:2004.10240. Bickel, P. J., & Freedman, D. A. (1981). Some asymptotic theory for the
Bennell, J., & Sutcliffe, C. (2004). Black-scholes versus artificial neu- bootstrap. The Annals of Statistics, 1196–1217.
ral networks in pricing FTSE 100 options. Intelligent Systems in Bielecki, T. R., & Rutkowski, M. (2013). Credit risk: modeling, valuation
Accounting, Finance & Management, 12(4), 243–260. and hedging. Springer Science & Business Media.
Berdugo, V., Chaussin, C., Dubus, L., Hebrail, G., & Leboucher, V. (2011). Biemer, P. P. (2010). Total survey error: Design, implementation, and
Analog method for collaborative very-short-term forecasting of evaluation. Public Opinion Quarterly, 74(5), 817–848.
powergeneration from photovoltaic systems. In Next generation data Bijak, J. (2010). Forecasting international migration in Europe: A Bayesian
mining summit (pp. 1–5). Athens, Greece. view. Springer, Dordrecht.
Berg, J. E., Nelson, F. D., & Rietz, T. A. (2008). Prediction market accuracy Bijak, J., & Czaika, M. (2020). Black swans and grey rhinos: Migration
in the long run. International Journal of Forecasting, 24(2), 285–300. policy under uncertainty. Migration Policy Practice, X(4), 14–20.
Berger, J. O. (1985). Statistical decision theory and bayesian analysis. Bijak, J., Disney, G., Findlay, A. M., Forster, J. J., Smith, P. W. F., &
Springer. Wiśniowski, A. (2019). Assessing time series models for forecasting
Bergmeir, C., & Benítez, J. M. (2012). On the use of cross-validation for international migration: Lessons from the United Kingdom. Journal
time series predictor evaluation. Information Sciences, 191, 192–213. of Forecasting, 38(6), 470–487.
Bergmeir, C., Hyndman, R. J., & Benítez, J. M. (2016). Bagging expo- Bijak, J., & Wiśniowski, A. (2010). Bayesian forecasting of immigration
nential smoothing methods using STL decomposition and Box–Cox to selected European countries by using expert knowledge. Journal
transformation. International Journal of Forecasting, 32(2), 303–312. of the Royal Statistical Society. Series A, 173(4), 775–796.
Bergmeir, C., Hyndman, R. J., & Koo, B. (2018). A note on the va- Billio, M., Casarin, R., Ravazzolo, F., & van Dijk, H. K. (2013).
lidity of cross-validation for evaluating autoregressive time series Time-varying combinations of predictive densities using nonlinear
prediction. Computational Statistics & Data Analysis, 120, 70–83. filtering. Journal of Econometrics, 177(2), 213–232.
834
Binder, C. C. (2017). Measuring uncertainty based on rounding: Bordignon, S., Bunn, D. W., Lisi, F., & Nan, F. (2013). Combining day-
New method and application to inflation expectations. Journal of ahead forecasts for british electricity prices. Energy Economics, 35,
Monetary Economics, 90(C), 1–12. 88–103.
Bisaglia, L., & Canale, A. (2016). Bayesian nonparametric forecasting for Bordley, R. F. (1982). The combination of forecasts: a Bayesian
INAR models. Computational Statistics & Data Analysis, 100, 70–78. approach. Journal of the Operational Research Society, 33(2),
Bisaglia, L., & Gerolimetto, M. (2019). Model-based INAR bootstrap for 171–174.
forecasting INAR(p) models. Computational Statistics, 34, 1815–1848. Bork, L., & Møller, S. V. (2015). Forecasting house prices in the 50 states
Bishop, C. M. (2006). Pattern recognition and machine learning. New using dynamic model averaging and dynamic model selection.
York, N.Y.: Springer. International Journal of Forecasting, 31(1), 63–78.
Bjerknes, V. (1904). Das problem der wettervorhersage, betrachtet Bosq, D. (2000). Linear processes in function spaces. New York: Lecture
vom standpunkte der mechanik und der physik. Meteorologische Notes in Statistics.
Zeitschrift, 21, 1–7. Bosq, D., & Blanke, D. (2007). Inference and prediction in large
Blanchard, O. J., & Kahn, C. M. (1980). The solution of linear dif- dimensions. West Sussex, England: John Wiley & Sons.
ference models under rational expectations. Econometrica, 48(5), Botimer, T. (1997). Select ideas on forecasting with sales relative to
1305–1311. bucketing and ‘seasonality’. Company Report, Continental Airlines,
Blei, D. M., Kucukelbir, A., & McAuliffe, J. D. (2017). Variational infer- Inc.
ence: A review for statisticians. Journal of the American Statistical Bourdeau, M., qiang Zhai, X., Nefzaoui, E., Guo, X., & Chatellier, P.
Association, 112(518), 859–877. (2019). Modeling and forecasting building energy consumption: A
Bo, R., & Li, F. (2012). Probabilistic LMP forecasting under AC optimal review of data-driven techniques. Sustainable Cities and Society, 48,
power flow framework: Theory and applications. Electric Power Article 101533.
Systems Research, 88, 16–24. Box, G. E. P., & Cox, D. R. (1964). An analysis of transformations. Journal
Boccara, N. (2004). Modeling complex systems. New York: Springer- of the Royal Statistical Society. Series B. Statistical Methodology, 26(2),
Verlag. 211–243.
Bohk-Ewald, C., Li, P., & Myrskylä, M. (2018). Forecast accuracy hardly Box, George, E. P., Jenkins, & Gwilym (1976). Time series analysis
improves with method complexity when completing cohort fertil- forecasting and control. San Francisco, CA: Holden-Day.
ity. Proceedings of the National Academy of Sciences of the United Box, G. E. P., Jenkins, G. M., & Reinsel, G. C. (2008). Time series analysis:
States of America, 115(37), 9187–9192. forecasting and control (4th ed.). New Jersey: Wiley.
Boje, D. M., & Murnighan, J. K. (1982). Group confidence pressures in Boylan, J. E., & Babai, M. Z. (2016). On the performance of overlapping
iterative decisions. Management Science, 28(10), 1187–1196. and non-overlapping temporal demand aggregation approaches.
Bojer, C. S., & Meldgaard, J. P. (2020). Kaggle’s forecasting competi-
International Journal of Production Economics, 181, 136–144.
tions: An overlooked learning opportunity. International Journal of
Boylan, J. E., & Syntetos, A. A. (2003). Intermittent demand forecasting:
Forecasting.
size-interval methods based on averaging and smoothing. In C.
Bolger, F., & Harvey, N. (1993). Context-sensitive heuristics in statistical
C. Frangos (Ed.), Proceedings of the international conference on
reasoning. The Quarterly Journal of Experimental Psychology Section
quantitative methods in industry and commerce (pp. 87–96). Athens:
A, 46(4), 779–811.
Technological Educational Institute.
Bollerslev, T. (1986). Generalized autoregressive conditional het-
Boylan, J., & Syntetos, A. (2006). Accuracy and accuracy implication
eroskedasticity. Journal of Econometrics, 31(3), 307–327.
metrics for intermittent demand. Foresight: The International Journal
Bollerslev, T. (1987). A conditionally heteroskedastic time series model
of Applied Forecasting, 4, 39–42.
for speculative prices and rates of return. The Review of Economics
Boylan, J. E., & Syntetos, A. A. (2021). Intermittent demand forecasting -
and Statistics, 69(3), 542–547.
context, methods and applications. Wiley.
Bollerslev, T. (1990). Modelling the coherence in short-run nominal
Boylan, J. E., Syntetos, A. A., & Karakostas, G. C. (2008). Classification
exchange rates: A multivariate generalized arch model. The Review
for forecasting and stock control: a case study. Journal of the
of Economics and Statistics, 72(3), 498–505.
Operational Research Society, 59(4), 473–481.
Bonaccio, S., & Dalal, R. S. (2006). Advice taking and decision-making:
Bozkurt, O. O., Biricik, G., & Tayşi, Z. C. (2017). Artificial neural network
An integrative literature review, and implications for the orga-
and SARIMA based models for power load forecasting in Turkish
nizational sciences. Organizational Behavior and Human Decision
electricity market. PLoS One, 12(4), Article e0175915.
Processes, 101(2), 127–151.
Bonaldo, D. (1991). Competizione tra prodotti farmaceutici: strumenti di Brandt, M. W., & Jones, C. S. (2006). Volatility forecasting with range-
previsione. (Master’s thesis), University of Padua. based EGARCH models. Journal of Business & Economic Statistics,
Boneva, L., Fawcett, N., Masolo, R. M., & Waldron, M. (2019). Forecasting 24(4), 470–486.
the UK economy: Alternative forecasting methodologies and the Brass, W. (1974). Perspectives in population prediction: Illustrated by
role of off-model information. International Journal of Forecasting, the statistics of England and Wales. Journal of the Royal Statistical
35(1), 100–120. Society. Series A, 137(4), 532–583.
Bonham, C., & Cohen, R. (2001). To aggregate, pool, or neither: Testing Braumoeller, B. F. (2019). Only the dead: the persistence of war in the
the rational expectations hypothesis using survey data. Journal of modern age. Oxford University Press.
Business & Economic Statistics, 190, 278–291. Brehmer, J., & Gneiting, T. (2020). Scoring interval forecasts:
Bontempi, G., & Ben Taieb, S. (2011). Conditionally dependent strategies Equal-tailed, shortest, and modal interval. arXiv:2007.05709.
for multiple-step-ahead prediction in local learning. International Breiman, L. (1996). Bagging predictors. Machine Learning, 24(2),
Journal of Forecasting, 27(3), 689–699. 123–140.
Bontempi, G., Birattari, M., & Bersini, H. (1999). Local learning for iter- Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5–32.
ated time series prediction. In International conference on machine Brennan, J. (2020). Can novices trust themselves to choose trustwor-
learning. In (pp. 32–38). thy experts? Reasons for (reserved) optimism. Social Epistemology,
Booij, N., Ris, R. C., & Holthuijsen, L. H. (1999). A third-generation 34(3), 227–240.
wave model for coastal regions 1. Model description and validation. Brier, G. W. (1950). Verification of forecasts expressed in terms of
Journal of Geophysical Research: Oceans, 104(C4), 7649–7666. probability. Monthly Weather Review, 78(1), 1–3.
Boone, T., & Ganeshan, R. (2008). The value of information shar- Brighton, H., & Gigerenzer, G. (2015). The bias bias. Journal of Business
ing in the retail supply chain: Two case studies. Foresight: The Research, 68(8), 1772–1784.
International Journal of Applied Forecasting, 9, 12–17. Broer, T., & Kohlhas, A. (2018). Forecaster (mis-)behavior. CEPR
Booth, H. (2006). Demographic forecasting: 1980 to 2005 in review. Discussion Papers 12898, C.E.P.R. Discussion Papers.
International Journal of Forecasting, 22(3), 547–581. Brooks, S., Gelman, A., Jones, G., & Meng, X. (2011). Handbook of Markov
Booth, H., & Tickle, L. (2008). Mortality modelling and forecasting: A Chain Monte Carlo. Taylor & Francis.
review of methods. Annals of Actuarial Science, 3(1–2), 3–43. Brown, L., Gans, N., Mandelbaum, A., Sakov, A., Shen, H., Zeltyn, S., et al.
Bordalo, P., Gennaioli, N., Ma, Y., & Shleifer, A. (2018). Over-reaction in (2005a). Statistical analysis of a telephone call center: A queueing-
macroeconomic expectations. NBER Working Papers 24932, National science perspective. Journal of the American Statistical Association,
Bureau of Economic Research, Inc. 100(469), 36–50.
835
Brown, A., Reade, J. J., & Vaughan Williams, L. (2019). When are Cai, J. (1994). A Markov model of switching-regime ARCH. Journal of
prediction market prices most informative? International Journal of Business & Economic Statistics, 12(3), 309–316.
Forecasting, 35(1), 420–428. Cairns, A. J. G., Blake, D., Dowd, K., Coughlan, G. D., Epstein, D., Ong, A.,
Brown, G., Wyatt, J., Harris, R., & Yao, X. (2005b). Diversity creation et al. (2009). A quantitative comparison of stochastic mortality
methods: a survey and categorisation. Information Fusion, 6(1), models using data from England and Wales and the United States.
5–20. North American Actuarial Journal, 13(1), 1–35.
Brücker, H., & Siliverstovs, B. (2006). On the estimation and forecasting Calvo, E., & Escolar, M. (2003). The local voter: A geographically
of international migration: how relevant is heterogeneity across weighted approach to ecological inference. American Journal of
countries? Empirical Economics, 31(3), 735–754. Political Science, 47(1), 189–204.
Brunetti, C., & Lildholdt, P. M. (2002). Return-based and range-based Campbell, J. Y., & Thompson, S. B. (2008). Predicting excess
(co)variance estimation - with an application to foreign exchange stock returns out of sample: Can anything beat the historical
markets. SSRN:296875. average? Review of Financial Studies, 21(4), 1509–1531.
Bryant, J., & Zhang, J. L. (2018). Bayesian demographic estimation and Canale, A., & Ruggiero, M. (2016). Bayesian nonparametric forecasting
forecasting. CRC Press. of monotonic functional time series. Electronic Journal of Statistics,
Bu, R., & McCabe, B. P. (2008). Model selection, estimation and forecast- 10(2), 3265–3286.
ing in INAR(p) models: A likelihood-based Markov chain approach. Cappelen, Å., Skjerpen, T., & Tønnessen, M. (2015). Forecasting im-
International Journal of Forecasting, 24(1), 151–162. migration in official population projections using an econometric
Buchanan, B. G. (2019). Artificial intelligence in finance. London: The model. International Migration Review, 49(4), 945–980.
Alan Turing Institute. Cardani, R., Paccagnini, A., & Villa, S. (2015). Forecasting in a DSGE
Buckle, H. T. (1858). vol. 1, History of civilization in England. John W. model with banking intermediation: Evidence from the US. Working
Parker and Son. Paper 292, University of Milano-Bicocca, Department of Economics.
Budescu, D. V., & Wallsten, T. S. (1985). Consistency in interpretation Cardani, R., Paccagnini, A., & Villa, S. (2019). Forecasting with instabili-
of probabilistic phrases. Organizational Behavior and Human Decision ties: An application to DSGE models with financial frictions. Journal
Processes, 36(3), 391–405. of Macroeconomics, 61(C), Article 103133.
Bühlmann, P. (1997). Sieve bootstrap for time series. Bernoulli, 3(2), Carlstein, E. (1990). Resampling techniques for stationary time-
123–148. series: some recent developments. Tech. rep., North Carolina State
Buizza, R. (2018). Ensemble forecasting and the need for calibration. In University, Department of Statistics.
Statistical postprocessing of ensemble forecasts (pp. 15–48). Elsevier. Carmo, J. L., & Rodrigues, A. J. (2004). Adaptive forecasting of irregular
Bunea, A., Della Posta, P., Guidolin, M., & Manfredi, P. (2020). What do demand processes. Engineering Applications of Artificial Intelligence,
adoption patterns of solar panels observed so far tell about gov- 17(2), 137–143.
ernments’ incentive? Insights from diffusion models. Technological Carnevale, C., Angelis, E. D., Finzi, G., Turrini, E., & Volta, M. (2020). Ap-
Forecasting and Social Change, 160, Article 120240. plication of data fusion techniques to improve air quality forecast:
Bunn, D. W. (1975). A Bayesian approach to the linear combination of A case study in the northern Italy. Atmosphere, 11(3).
forecasts. Journal of the Operational Research Society, 26(2), 325–329. Carnevale, C., Finzi, G., Pederzoli, A., Turrini, E., & Volta, M. (2018). An
Bunn, D. W., & Salo, A. A. (1993). Forecasting with scenarios. European integrated data-driven/data assimilation approach for the forecast
Journal of Operational Research, 68(3), 291–303. of PM10 levels in northern Italy. In C. Mensink, & G. Kallos
Burch, T. K. (2018). Model-based demography: essays on integrating data, (Eds.), Air pollution modeling and its application XXV (pp. 225–229).
technique and theory. Springer, Cham. Springer International Publishing.
Bureau of Transportation Statistics (2020). Reporting carrier on-time Carnevale, C., Finzi, G., Pisoni, E., & Volta, M. (2016). Lazy learning based
performance (1987 - present). Accessed on 2020-09-09. surrogate models for air quality planning. Environmental Modelling
Burgman, M. A. (2016). Trusting judgements: how to get the best out of and Software, 83, 47–57.
experts. Cambridge University Press. Carriero, A., Clements, M. P., & Galvão, A. B. (2015). Forecasting with
Burman, P., Chow, E., & Nolan, D. (1994). A cross-validatory method Bayesian multivariate vintage-based VARs. International Journal of
for dependent data. Biometrika, 81(2), 351–358. Forecasting, 31(3), 757–768.
Burridge, P., & Robert Taylor, A. (2006). Additive outlier detection Carroll, R. (2003). The skeptic’s dictionary: A collection of strange beliefs,
via extreme-value theory. Journal of Time Series Analysis, 27(5), amusing deceptions, and dangerous delusions. Wiley.
685–701. Carson, R., Cenesizoglu, T., & Parker, R. (2011). Forecasting (aggregate)
Burton, J. W., Stein, M.-k., & Jensen, T. B. (2020). A systematic review demand for US commercial air travel. International Journal of
of algorithm aversion in augmented decision making. Journal of Forecasting, 27(3), 923–941.
Behavioral Decision Making, 33(2), 220–239. Caruana, R. (1997). Multitask learning. Machine Learning, 28(1), 41–75.
Busetti, F., & Marcucci, J. (2013). Comparing forecast accuracy: a Monte Carvalho, C. M., Polson, N. G., & Scott, J. G. (2010). The horseshoe
Carlo investigation. International Journal of Forecasting, 29(1), 13–27. estimator for sparse signals. Biometrika, 97(2), 465–480.
Butler, D., Butler, R., & Eakins, J. (2020). Expert performance and Carvalho, T. P., Soares, F. A. A. M. N., Vita, R., Francisco, R. d. P.,
crowd wisdom: Evidence from english premier league predictions. Basto, J. P., & Alcalá, S. G. S. (2019). A systematic literature review
European Journal of Operational Research, 288, 170–182. of machine learning methods applied to predictive maintenance.
Buys-Ballot, C. H. D. (1847). Les changements périodiques de temperature. Computers & Industrial Engineering, 137, Article 106024.
Utrecht: Kemink Et Fils. Casals, J., Garcia-Hiernaux, A., Jerez, M., Sotoca, S., & Trindade, A. (2016).
Byrne, J. P., Fazio, G., & Fiess, N. (2013). Primary commodity prices: State-space methods for time series analysis: theory, applications and
Co-movements, common factors and fundamentals. Journal of software. Chapman-Hall / CRC Press.
Development Economics, 101, 16–26. Casarin, R., Leisen, F., Molina, G., & ter Horst, E. (2015). A Bayesian beta
Byrne, J. P., Korobilis, D., & Ribeiro, P. J. (2016). Exchange rate pre- Markov random field calibration of the term structure of implied
dictability in a changing world. Journal of International Money and risk neutral densities. Bayesian Analysis, 10(4), 791–819.
Finance, 62, 1–24. Castle, J. L., Clements, M. P., & Hendry, D. F. (2015a). Robust approaches
Ca’ Zorzi, M., Cap, A., Mijakovic, A., & Rubaszek, M. (2020). The to forecasting. International Journal of Forecasting, 31(1), 99–112.
predictive power of equilibrium exchange rate models. Working Castle, J. L., Doornik, J. A., & Hendry, D. F. (2018a). Selecting a model
Paper Series 2358, European Central Bank. for forecasting. In Working paper. Economics Department, Oxford
Ca’ Zorzi, M., Kolasa, M., & Rubaszek, M. (2017). Exchange rate forecast- University.
ing with DSGE models. Journal of International Economics, 107(C), Castle, J. L., Doornik, J. A., & Hendry, D. F. (2020a). Modelling
127–146. non-stationary ‘big data’. International Journal of Forecasting.
Ca’ Zorzi, M., Muck, J., & Rubaszek, M. (2016). Real exchange rate Castle, J. L., Doornik, J. A., & Hendry, D. F. (2020b). Robust discovery
forecasting and PPP: This time the random walk loses. Open of regression models. Working paper 2020-W04, Nuffield College,
Economies Review, 27(3), 585–609. Oxford University.
Ca’ Zorzi, M., & Rubaszek, M. (2020). Exchange rate forecasting on Castle, J., Doornik, J., & Hendry, D. (2021). The value of robust statistical
a napkin. Journal of International Money and Finance, 104, Article forecasts in the COVID-19 pandemic, Vol. 256 (pp. 19–43). National
102168. Institute Economic Review.
836
Castle, J. L., Doornik, J. A., Hendry, D. F., & Pretis, F. (2015b). Detecting Chase, C. (2021). Assisted demand planning using machine learning.
location shifts during model selection by step-indicator saturation. In M. Gilliland, L. Tashman, & U. Sglavo (Eds.), Business forecasting:
Econometrics, 3(2), 240–264. the emerging role of artificial intelligence and machine learning (pp.
Castle, J., Doornik, J., Hendry, D., & Pretis, F. (2015c). Detecting lo- 110–114). Wiley.
cation shifts during model selection by step-indicator saturation. Chatfield, C. (1986). Simple is best? International Journal of Forecasting,
Econometrics, 3(2), 240–264. 2(4), 401–402.
Castle, J. L., Fawcett, N. W., & Hendry, D. F. (2010). Forecasting with Chatziantoniou, I., Degiannakis, S., Eeckels, B., & Filis, G. (2016).
equilibrium-correction models during structural breaks. Journal of Forecasting tourist arrivals using origin country macroeconomics.
Econometrics, 158(1), 25–36. Applied Economics, 48(27), 2571–2585.
Castle, J. L., & Hendry, D. F. (2010). Nowcasting from disaggregates in Chavez-Demoulin, V., Davison, A. C., & McNeil, A. J. (2005). Estimating
the face of location shifts. Journal of Forecasting, 29, 200–214. value-at-risk: a point process approach. Quantitative Finance, 5(2),
Castle, J. L., & Hendry, D. F. (2020a). Climate econometrics: An 227–234.
overview. Foundations and Trends in Econometrics , 10, 145–322. Checchi, F., & Roberts, L. (2005). Interpreting and using mortality data
Castle, J. L., & Hendry, D. F. (2020b). Identifying the causal role of CO2 in humanitarian emergencies. Humanitarian Practice Network, 52.
during the ice ages. Discussion paper 898, Economics Department, Chen, R. (1995). Threshold variable selection in open-loop thresh-
Oxford University. old autoregressive models. Journal of Time Series Analysis, 16(5),
Castle, J. L., Hendry, D. F., & Kitov, O. I. (2018). Forecasting and now- 461–481.
casting macroeconomic variables: A methodological overview. In Chen, C. W. S., Chiang, T. C., & So, M. K. P. (2003). Asymmetrical reaction
EuroStat (Ed.), Handbook on rapid estimates (pp. 53–107). Brussels: to US stock-return news: evidence from major stock markets based
UN/EuroStat. on a double-threshold model. Journal of Economics and Business,
Castle, J. L., Hendry, D. F., & Martinez, A. B. (2020c). The paradox of 55(5), 487–502.
stagnant real wages yet rising ‘living standards’ in the UK. Tech. Chen, C. W., Gerlach, R., Hwang, B. B., & McAleer, M. (2012). Forecasting
rep., VoxEU. value-at-risk using nonlinear regression quantiles and the intra-day
Caswell, H. (2019a). Sensitivity analysis: matrix methods in demography range. International Journal of Forecasting, 28(3), 557–574.
and ecology. Springer, Cham. Chen, C. W., Gerlach, R., & Lin, E. M. (2008). Volatility forecasting
Caswell, H. (2019b). The formal demography of kinship: A matrix using threshold heteroskedastic models of the intra-day range.
formulation. Demographic Research, 41(24), 679–712. Computational Statistics & Data Analysis, 52(6), 2990–3010.
Caswell, H. (2020). The formal demography of kinship II: Multi- Chen, T., & Guestrin, C. (2016). Xgboost: A scalable tree boosting sys-
state models, parity, and sibship. Demographic Research, 42(38), tem. In Proceedings of the 22nd ACM SIGKDD international conference
1097–1146. on knowledge discovery and data mining (pp. 785–794). ACM.
Catalán, B., & Trívez, F. J. (2007). Forecasting volatility in GARCH models Chen, Y., Kang, Y., Chen, Y., & Wang, Z. (2020). Probabilistic forecasting
with additive outliers. Quantitative Finance, 7(6), 591–596. with temporal convolutional neural network. Neurocomputing, 399,
Cavalcante, L., Bessa, R. J., Reis, M., & Browell, J. (2016). LASSO 491–501.
vector autoregression structures for very short-term wind power Chen, J., Li, K., Rong, H., Bilal, K., Li, K., & Philip, S. Y. (2019a). A
forecasting. Wind Energy, 20, 657–675. periodicity-based parallel time series prediction algorithm in cloud
Cazelles, B., Chavez, M., McMichael, A. J., & Hales, S. (2005). Nonsta- computing environments. Information Sciences, 496, 506–537.
tionary influence of el nino on the synchronous dengue epidemics Chen, C., & Liu, L.-M. (1993a). Forecasting time series with outliers.
in thailand. PLoS Medicine, 2(4), Article e106. Journal of Forecasting, 12(1), 13–35.
Cederman, L.-E. (2003). Modeling the size of wars: From billiard balls Chen, C., & Liu, L.-M. (1993b). Joint estimation of model parameters
to sandpiles. The American Political Science Review, 97(1), 135–150. and outlier effects in time series. Journal of the American Statistical
Ceron, A., Curini, L., & Iacus, S. M. (2016). Politics and big data: Association, 88(421), 284–297.
nowcasting and forecasting elections with social media. Routledge. Chen, Y., Marron, J. S., & Zhang, J. (2019b). Modeling seasonality
Chae, Y. T., Horesh, R., Hwang, Y., & Lee, Y. M. (2016). Artificial and serial dependence of electricity price curves with warping
neural network model for forecasting sub-hourly electricity usage functional autoregressive dynamics. The Annals of Applied Statistics,
in commercial buildings. Energy and Buildings, 111, 184–194. 13(3), 1590–1616.
Chakraborty, T., Chattopadhyay, S., & Ghosh, I. (2019). Forecasting Chen, C. W. S., & So, M. K. P. (2006). On a threshold heteroscedastic
dengue epidemics using a hybrid methodology. Physica A: Statistical model. International Journal of Forecasting, 22(1), 73–89.
Mechanics and its Applications, 527, Article 121266. Chen, M.-F., Wang, R.-H., & Hung, S.-L. (2015). Predicting health-
Chan, N. H., & Genovese, C. R. (2001). A comparison of linear and promoting self-care behaviors in people with pre-diabetes by
nonlinear statistical techniques in performance attribution. IEEE applying bandura social learning theory. Applied Nursing Research,
Transactions on Neural Networks, 12(4), 922–928. 28(4), 299–304.
Chan, C. K., Kingsman, B. G., & Wong, H. (1999). The value of combin- Chen, R., Yang, L., & Hafner, C. (2004). Nonparametric multistep-ahead
ing forecasts in inventory management–a case study in banking. prediction in time series analysis. Journal of the Royal Statistical
European Journal of Operational Research, 117(2), 199–210. Society. Series B. Statistical Methodology, 66(3), 669–686.
Chan, J. S., Lam, C. P., Yu, P. L., Choy, S. T., & Chen, C. W. (2012). Cheng, G., & Yang, Y. (2015). Forecast combination with outlier
A Bayesian conditional autoregressive geometric process model protection. International Journal of Forecasting, 31(2), 223–237.
for range data. Computational Statistics & Data Analysis, 56(11), Cheng, C., Yu, L., & Chen, L. J. (2012). Structural nonlinear damage
3006–3019. detection based on ARMA-GARCH model. Applied Mechanics and
Chan, F., & Pauwels, L. L. (2018). Some theoretical results on forecast Materials, 204–208, 2891–2896.
combinations. International Journal of Forecasting, 34(1), 64–74. Cheung, Y.-W., Chinn, M. D., & Pascual, A. G. (2005). Empirical exchange
Chan, K. S., & Tong, H. (1986). On estimating thresholds in rate models of the nineties: Are any fit to survive? Journal of
autoregressive models. Journal of Time Series Analysis, 7(3), 179–190. International Money and Finance, 24(7), 1150–1175.
Chan, J. C., & Yu, X. (2020). Fast and accurate variational inference Cheung, Y.-W., Chinn, M. D., Pascual, A. G., & Zhang, Y. (2019). Exchange
for large Bayesian VARs with stochastic volatility. In CAMA working rate prediction redux: New models, new data, new currencies.
paper. Journal of International Money and Finance, 95, 332–362.
Chandola, V., Banerjee, A., & Kumar, V. (2007). Outlier detection: A Chevillon, G. (2007). Direct multi-step estimation and forecasting.
survey. ACM Computing Surveys, 14, 15. Journal of Economic Surveys, 21(4), 746–785.
Chandola, V., Banerjee, A., & Kumar, V. (2009). Anomaly detection: A Chew, V. (1968). Simultaneous prediction intervals. Technometrics,
survey. ACM Computing Surveys, 41(3), 1–58. 10(2), 323–330.
Chang, Y., Kim, C. S., & Park, J. (2016). Nonstationarity in time series Chiang, M. H., & Wang, L. M. (2011). Volatility contagion: A range-based
of state densities. Journal of Econometrics, 192(1), 152–167. volatility approach. Journal of Econometrics, 165(2), 175–189.
Chaouch, M. (2014). Clustering-based improvement of nonparamet- Chicco, G., Cocina, V., Di Leo, P., Spertino, F., & Massi Pavan, A. (2015).
ric functional time series forecasting: Application to intra-day Error assessment of solar irradiance forecasts and AC power from
household-level load curves. IEEE Transactions on Smart Grid, 5(1), energy conversion model in grid-connected photovoltaic systems.
411–419. Energies, 9(1), 8.
837
Chinco, A., Clark-Joseph, A. D., & Ye, M. (2019). Sparse signals in the Clark, T., & West, K. (2006). Using out-of-sample mean squared predic-
cross-section of returns. The Journal of Finance, 74(1), 449–492. tion errors to test the martingale difference hypothesis. Journal of
Chiroma, H., Abdulkareem, S., & Herawan, T. (2015). Evolutionary Econometrics, 135(1–2), 155–186.
neural network model for west texas intermediate crude oil price Clauset, A. (2018). Trends and fluctuations in the severity of interstate
prediction. Applied Energy, 142, 266–273. wars. Science Advances, 4(2), eaao3580.
Choi, E., Özer, O., & Zheng, Y. (2020). Network trust and trust behaviors Clauset, A., & Gleditsch, K. S. (2018). Trends in conflicts: What
among executives in supply chain interactions. Management Science. do we know and what can we know? In A. Gheciu, & W.
Chong, Y. Y., & Hendry, D. F. (1986). Econometric evaluation of C. Wohlforth (Eds.), The oxford handbook of international security.
linear macro-economic models. Review of Economic Studies, 53(4), Oxford University Press.
671–690. Cleave, N., Brown, P. J., & Payne, C. D. (1995). Evaluation of methods
Chou, R. Y.-T. (2005). Forecasting financial volatilities with extreme for ecological inference. Journal of the Royal Statistical Society, Series
values: The conditional autoregressive range (CARR) model. Journal A,, 158(1), 55–72.
of Money, Credit, and Banking, 37(3), 561–582. Clemen, R. T. (1989). Combining forecasts: A review and annotated
Chou, R. Y., & Cai, Y. (2009). Range-based multivariate volatility model bibliography. International Journal of Forecasting, 5(4), 559–583.
with double smooth transition in conditional correlation. Global Clemen, R. T. (2008). Comment on cooke’s classical method. Reliability
Finance Journal, 20(2), 137–152. Engineering & System Safety, 93(5), 760–765.
Chou, R. Y., Chou, H., & Liu, N. (2015). Range volatility: A review Clemen, R. T., & Winkler, R. L. (1986). Combining economic forecasts.
of models and empirical studies. In C. F. Lee, & J. C. Lee (Eds.), Journal of Business & Economic Statistics, 4(1), 39–46.
Handbook of financial econometrics and statistics (pp. 2029–2050). Clements, M. P. (2009). Internal consistency of survey respondents’
Springer New York. forecasts: Evidence based on the survey of professional forecasters.
Chou, R. Y., & Liu, N. (2010). The economic value of volatility timing In J. L. Castle, & N. Shephard (Eds.), The methodology and practice
using a range-based volatility model. Journal of Economic Dynamics of econometrics. a festschrift in honour of David F. Hendry. Chapter 8
and Control, 34(11), 2288–2301. (pp. 206–226). Oxford: Oxford University Press.
Chou, R. Y., Wu, C. C., & Liu, N. (2009). Forecasting time-varying covari- Clements, M. P. (2010). Explanations of the inconsistencies in survey
ance with a range-based dynamic conditional correlation model. respondents forecasts. European Economic Review, 54(4), 536–549.
Review of Quantitative Finance and Accounting, 33(4), 327–345. Clements, M. P. (2011). An empirical investigation of the effects of
Choudhury, A., & Urena, E. (2020). Forecasting hourly emergency rounding on the SPF probabilities of decline and output growth
department arrival using time series analysis. British Journal of histograms. Journal of Money, Credit and Banking, 43(1), 207–220.
Healthcare Management, 26(1), 34–43. Clements, M. P. (2014a). Forecast uncertainty - ex ante and ex post:
US inflation and output growth. Journal of Business & Economic
Christ, M., Braun, N., Neuffer, J., & Kempa-Liehr, A. W. (2018). Time
Statistics, 32(2), 206–216.
series feature extraction on basis of scalable hypothesis tests
Clements, M. P. (2014b). US inflation expectations and heterogeneous
(tsfresh – a python package). Neurocomputing, 307, 72–77.
loss functions, 1968–2010. Journal of Forecasting, 33(1), 1–14.
Christensen, T., Hurn, S., & Lindsay, K. (2009). It never rains but it
Clements, M. P. (2017). Assessing macro uncertainty in real-time when
pours: Modeling the persistence of spikes in electricity prices.
data are subject to revision. Journal of Business & Economic Statistics,
Energy Journal, 30(1), 25–48.
35(3), 420–433.
Christensen, T. M., Hurn, A. S., & Lindsay, K. A. (2012). Forecasting
Clements, M. P. (2018). Are macroeconomic density forecasts
spikes in electricity prices. International Journal of Forecasting, 28(2),
informative? International Journal of Forecasting, 34, 181–198.
400–411.
Clements, M. P. (2019). Macroeconomic survey expectations. Palgrave
Christiano, L. J., Eichenbaum, M. S., & Trabandt, M. (2018). On DSGE
Texts in Econometrics. Palgrave Macmillan.
models. Journal of Economic Perspectives, 32(3), 113–140.
Clements, M. P., & Galvão, A. B. (2012). Improving real-time estimates
Christoffersen, P., & Langlois, H. (2013). The joint dynamics of equity
of output gaps and inflation trends with multiple-vintage VAR
market factors. Journal of Financial and Quantitative Analysis, 48(5),
models. Journal of Business & Economic Statistics, 30(4), 554–562.
1371–1404.
Clements, M. P., & Galvão, A. B. (2013a). Forecasting with vector
Chung, H., Kiley, M. T., & Laforte, J.-P. (2010). Documentation of the autoregressive models of data vintages: US output growth and
estimated, dynamic, optimization-based (EDO) model of the U.S. inflation. International Journal of Forecasting, 29(4), 698–714.
economy: 2010 version. Finance and Economics Discussion Series Clements, M. P., & Galvão, A. B. (2013b). Real-time forecasting of
2010–29, Board of Governors of the Federal Reserve System (U.S.). inflation and output growth with autoregressive models in the
Chung, C., Niu, S.-C., & Sriskandarajah, C. (2012). A sales forecast presence of data revisions. Journal of Applied Econometrics, 28(3),
model for short-life-cycle products: New releases at blockbuster. 458–477.
Production and Operations Management, 21(5), 851–873. Clements, M. P., & Galvão, A. B. (2017). Data revisions and real-time
Cirillo, P., & Taleb, N. N. (2016a). Expected shortfall estimation for probabilistic forecasting of macroeconomic variables. Discussion
apparently infinite-mean models of operational risk. Quantitative Paper ICM-2017-01, ICMA, Henley Business School, Reading.
Finance, 16(10), 1485–1494. Clements, M. P., & Galvão, A. B. (2019). Data revisions and real-
Cirillo, P., & Taleb, N. N. (2016b). On the statistical properties and time forecasting. The Oxford Research Encyclopedia of Economics and
tail risk of violent conflicts. Physica A: Statistical Mechanics and its Finance.
Applications, 452, 29–45. Clements, M. P., & Harvey, D. I. (2011). Combining probability forecasts.
Cirillo, P., & Taleb, N. N. (2019). The decline of violent conflicts: What International Journal of Forecasting, 27(2), 208–223.
do the data really say? In A. Toje, & B. N. V. Steen (Eds.), The causes Clements, M. P., & Hendry, D. F. (1998). Forecasting economic time series.
of peace: what we know (pp. 57–86). The Causes of Peace: What We Cambridge University Press.
Know. Clements, M. P., & Hendry, D. F. (1999). Zeuthen lecture book series,
Claeskens, G., Magnus, J. R., Vasnev, A. L., & Wang, W. (2016). The Forecasting non-stationary economic time series. Cambridge, MA: MIT
forecast combination puzzle: A simple theoretical explanation. Press.
International Journal of Forecasting, 32(3), 754–762. Clements, M. P., & Hendry, D. F. (2005). Evaluating a model by
Clark, D. A. (1990). Verbal uncertainty expressions: A critical review of forecast performance. Oxford Bulletin of Economics and Statistics, 67,
two decades of research. Current Psychology, 9(3), 203–235. 931–956.
Clark, T. E., & McCracken, M. W. (2001). Tests of equal forecast accu- Clements, A. E., Herrera, R., & Hurn, A. S. (2015). Modelling interregional
racy and encompassing for nested models. Journal of Econometrics, links in electricity price spikes. Energy Economics, 51, 383–393.
105(1), 85–110. Cleveland, R. B., Cleveland, W. S., McRae, J. E., & Terpenning, I. (1990).
Clark, T. E., & McCracken, M. W. (2009). Tests of equal predictive ability STL: A seasonal-trend decomposition procedure based on loess.
with real-time data. Journal of Business & Economic Statistics, 27(4), Journal of Official Statistics, 6(1), 3–73.
441–454. Clottey, T., Benton Jr., W. C., & Srivastava, R. (2012). Forecasting product
Clark, T., & McCracken, M. (2013). Advances in forecast evaluation. vol. returns for remanufacturing operations. Decision Sciences, 43(4),
2, In Handbook of economic forecasting (pp. 1107–1201). Elsevier. 589–614.
838
Cludius, J., Hermann, H., Matthes, F. C., & Graichen, V. (2014). The Couharde, C., Delatte, A.-L., Grekou, C., Mignon, V., & Morvillier, F.
merit order effect of wind and photovoltaic electricity generation (2018). EQCHANGE: A world database on actual and equilibrium
in Germany 2008–2016: Estimation and distributional implications. effective exchange rates. International Economics, 156, 206–230.
Energy Economics, 44, 302–313. Courgeau, D. (2012). Probability and social science: methodologial rela-
Coates, D., & Humphreys, B. R. (1999). The growth effects of sport fran- tionships between the two approaches?. Tech. Rep. 43102, Germany:
chises, stadia, and arenas. Journal of Policy Analysis and Management, University Library of Munich.
18(4), 601–624. Creal, D., Koopman, S. J., & Lucas, A. (2013). Generalized autoregressive
Coates, D., & Humphreys, B. (2010). Week-to-week attendance and score models with applications. Journal of Applied Econometrics, 28,
competitive balance in the national football league. International 777–795.
Journal of Sport Finance, 5(4), 239. Creal, D. D., & Tsay, R. S. (2015). High dimensional dynamic stochastic
Coccia, G. (2011). Analysis and developments of uncertainty processors copula models. Journal of Econometrics, 189(2), 335–345.
for real time flood forecasting. (Ph.D. thesis), Alma Mater Studiorum Croll, J. (1875). Climate and time in their geological relations, a theory of
University of Bologna. secular changes of the earth’s climate. New York: D. Appleton.
Coccia, G., & Todini, E. (2011). Recent developments in predictive Crone, S. F., Hibon, M., & Nikolopoulos, K. (2011). Advances in
uncertainty assessment based on the model conditional processor forecasting with neural networks? Empirical evidence from the
approach. Hydrology and Earth System Sciences, 15(10), 3253–3274. NN3 competition on time series prediction. International Journal of
Cohin, A., & Chantret, F. (2010). The long-run impact of energy prices on Forecasting, 27(3), 635–660.
world agricultural markets: The role of macro-economic linkages. Cross, J. L. (2020). Macroeconomic forecasting with large Bayesian
Energy Policy, 38(1), 333–339. VARs: Global-local priors and the illusion of sparsity. International
Coibion, O., & Gorodnichenko, Y. (2012). What can survey forecasts tell Journal of Forecasting, 36(3), 899–916.
us about information rigidities? Journal of Political Economy, 120(1), Cross, R., & Sproull, L. (2004). More than an answer: Information
116–159. relationships for actionable knowledge. Organization Science, 15(4),
Coibion, O., & Gorodnichenko, Y. (2015). Information rigidity and the 446–462.
expectations formation process: A simple framework and new facts. Croston, J. D. (1972). Forecasting and stock control for intermittent
American Economic Review, 105(8), 2644–2678. demands. Operational Research Quarterly, 23(3), 289–303.
Collopy, F., & Armstrong, J. S. (1992). Rule-based forecasting: develop- Croushore, D. (2006). Forecasting with real-time macroeconomic data.
ment and validation of an expert systems approach to combining In G. Elliott, C. Granger, & A. Timmermann (Eds.), Handbook of
time series extrapolations. Management Science, 38(10), 1394–1414. economic forecasting, volume 1. Handbook of economics 24 (pp.
Commandeur, J. J. F., Koopman, S. J., & Ooms, M. (2011). Statistical 961–982). Elsevier, Horth-Holland.
software for state space methods. Journal of Statistical Software, Croushore, D. (2011a). Forecasting with real-time data vintages (chap-
41(1), 1–18. ter 9). In M. P. Clements, & D. F. Hendry (Eds.), The oxford handbook
of economic forecasting (pp. 247–267). Oxford University Press.
Congdon, P. (1990). Graduation of fertility schedules: an analysis of
Croushore, D. (2011b). Frontiers of real-time data analysis. Journal of
fertility patterns in London in the 1980s and an application to
Economic Literature, 49, 72–100.
fertility forecasts. Regional Studies, 24(4), 311–326.
Croushore, D., & Stark, T. (2001). A real-time data set for
Consolo, A., Favero, C., & Paccagnini, A. (2009). On the statistical
macroeconomists. Journal of Econometrics, 105(1), 111–130.
identification of DSGE models. Journal of Econometrics, 150(1),
Croxson, K., & Reade, J. J. (2014). Information and efficiency: goal arrival
99–115.
in soccer. The Economic Journal, 124(575), 62–91.
Continuous Mortality Investigation (2020). The CMI mortality pro-
Cunado, J., & De Gracia, F. P. (2005). Oil prices, economic activity and
jections model, cmi_2019. Working paper, London: Institute of
inflation: Evidence for some Asian countries. The Quarterly Review
Actuaries and Faculty of Actuaries.
of Economics and Finance, 45(1), 65–83.
Cook, S., & Thomas, C. (2003). An alternative approach to examining the
Cunningham, C. R. (2006). House price uncertainty, timing of develop-
ripple effect in U.K. house prices. Applied Economics Letters, 10(13),
ment, and vacant land prices: Evidence for real options in seattle.
849–851.
Journal of Urban Economics, 59(1), 1–31.
Cooke, R. M. (1991). Experts in uncertainty: opinion and subjective
Cunningham, A., Eklund, J., Jeffery, C., Kapetanios, G., & Labhard, V.
probability in science. Oxford University Press.
(2009). A state space approach to extracting the signal from
Copeland, M. T. (1915). Statistical indices of business conditions. uncertain data. Journal of Business & Economic Statistics, 30,
Quarterly Journal of Economics, 29(3), 522–562. 173–180.
Corani, G. (2005). Air quality prediction in milan: Feed-forward neural Curran, M., & Velic, A. (2019). Real exchange rate persistence and
networks, pruned neural networks and lazy learning. Ecological country characteristics: A global analysis. Journal of International
Modelling, 185, 513–529. Money and Finance, 97, 35–56.
Cordeiro, C., & Neves, M. (2006). The bootstrap methodology in time Cybenko, G. (1989). Approximation by superpositions of a sigmoidal
series forecasting. In J. Black, & A. White (Eds.), Proceedings of function. Mathematics of Control, Signals, and Systems, 2(4), 303–314.
CompStat2006 (pp. 1067–1073). Springer Verlag. Czado, C., Gneiting, T., & Held, L. (2009). Predictive model assessment
Cordeiro, C., & Neves, M. (2009). Forecasting time series with for count data. Biometrics, 65(4), 1254–1261.
BOOT.EXPOS procedure. REVSTAT-Statistical Journal, 7(2), 135–149. Dagum, E. B. (1988). The X11ARIMA/88 seasonal adjustment method:
Cordeiro, C., & Neves, M. M. (2010). Boot.EXPOS In NNGC competition. foundations and user’s manual. Statistics Canada, Time Series
In The 2010 international joint conference on neural networks (IJCNN) Research and Analysis Division.
(pp. 1–7). IEEE. Dai, Q., & Singleton, K. (2003). Term structure dynamics in theory and
Cordeiro, C., & Neves, M. M. (2013). Predicting and treating missing reality. Review of Financial Studies, 16(3), 631–678.
data with boot.expos. In Advances in regression, survival analysis, Dai, Q., Singleton, K. J., & Yang, W. (2007). Regime shifts in a dynamic
extreme values, markov processes and other statistical applications term structure model of U.S. Treasury bond yields. Review of
(pp. 131–138). Springer. Financial Studies, 20(5), 1669–1706.
Cordeiro, C., & Neves, M. M. (2014). Forecast intervals with boot.expos. Dalkey, N. C. (1969). The delphi method: An experimental study of
In New advances in statistical modeling and applications (pp. group opinion. Research Memoranda, RM-5888-PR.
249–256). Springer. Dalla Valle, A., & Furlan, C. (2011). Forecasting accuracy of wind power
Corominas, A., Lusa, A., & Dolors Calvet, M. (2015). Computing voter technology diffusion models across countries. International Journal
transitions: The elections for the catalan parliament, from 2010 of Forecasting, 27(2), 592–601.
to 2012. Journal of Industrial Engineering and Management, 8(1), Daniel, K., & Moskowitz, T. J. (2016). Momentum crashes. Journal of
122–136. Financial Economics, 122(2), 221–247.
Corradi, V., Swanson, N. R., & Olivetti, C. (2001). Predictive ability with Dantas, T. M., & Cyrino Oliveira, F. L. (2018). Improving time series
cointegrated variables. Journal of Econometrics, 104(2), 315–358. forecasting: An approach combining bootstrap aggregation, clusters
Corsi, F. (2009). A simple approximate long-memory model of realized and exponential smoothing. International Journal of Forecasting,
volatility. Journal of Financial Econometrics, 7(2), 174–196. 34(4), 748–761.
839
Dantas, T. M., Cyrino Oliveira, F. L., & Varela Repolho, H. M. (2017). de Oliveira, E. M., & Cyrino Oliveira, F. L. (2018). Forecasting mid-
Air transportation demand forecast through bagging holt winters long term electric energy consumption through bagging ARIMA and
methods. Journal of Air Transport Management, 59, 116–123. exponential smoothing methods. Energy, 144, 776–788.
Danti, P., & Magnani, S. (2017). Effects of the load forecasts mismatch de Queiroz, A. R. (2016). Stochastic hydro-thermal scheduling optimiza-
on the optimized schedule of a real small-size smart prosumer. tion: An overview. Renewable and Sustainable Energy Reviews, 62,
Energy Procedia, 126, 406–413. 382–395.
Dantzig, G. B., & Infanger, G. (1993). Multi-stage stochastic linear Dean, J., & Ghemawat, S. (2008). MapReduce: Simplified data processing
programs for portfolio optimization. Annals of Operations Research, on large clusters. Communications of the ACM, 51(1), 107–113.
45, 59–76. Deb, C., Zhang, F., Yang, J., Lee, S. E., & Shah, K. W. (2017). A review on
Das, S., & Chen, M. (2007). Yahoo! for Amazon: Sentiment extraction time series forecasting techniques for building energy consumption.
from small talk on the web. Management Science, 53(9), 1375–1388. Renewable and Sustainable Energy Reviews, 74, 902–924.
Daskalaki, C., Kostakis, A., & Skiadopoulos, G. (2014). Are there common Debecker, A., & Modis, T. (1994). Determination of the uncertainties
factors in individual commodity futures returns? Journal of Banking in S-curve logistic fits. Technological Forecasting and Social Change,
& Finance, 40(C), 346–363. 46(2), 153–173.
Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1989). User acceptance Debecker, A., & Modis, T. (2021). Poorly known aspects of flattening
of computer technology: A comparison of two theoretical models. the curve of COVID 19. Technological Forecasting and Social Change,
Management Science, 35(8), 982–1003. 163(120432).
Dawid, A. P. (1982). The well-calibrated Bayesian. Journal of the Dees, S., Mauro, F. d., Pesaran, M. H., & Smith, L. V. (2007). Exploring
American Statistical Association, 77(379), 605–610. the international linkages of the euro area: a global VAR analysis.
Dawid, A. P. (1984). Statistical theory: The prequential approach (with Journal of Applied Economics, 22(1), 1–38.
discussion and rejoinder). Journal of the Royal Statistical Society, Degiannakis, S. A., Filis, G., Klein, T., & Walther, T. (2020). Forecasting
Series A, 147, 278–292. realized volatility of agricultural commodities. International Journal
Dawid, A. P. (1985). Calibration-based empirical probability. The Annals of Forecasting.
of Statistics, 13(4), 1251–1274. DeGroot, M. H. (2004). Optimal statistical decisions. Hoboken, N.J:
Dawid, A. P., DeGroot, M. H., Mortera, J., Cooke, R., French, S., Genest, C., Wiley-Interscience.
et al. (1995). Coherent combination of experts’ opinions. Test, 4(2), Dekker, M., van Donselaar, K., & Ouwehand, P. (2004). How to use
263–313. aggregation and combined forecasting to improve seasonal de-
de Albuquerquemello, V. P., de Medeiros, R. K., da Nóbrega Besarria, C., mand forecasts. International Journal of Production Economics, 90(2),
& Maia, S. F. (2018). Forecasting crude oil price: Does exist an 151–167.
optimal econometric model? Energy, 155, 578–591.
Del Negro, M., & Schorfheide, F. (2004). Priors from general equilibrium
de Almeida Marques-Toledo, C., Degener, C. M., Vinhal, L., Coelho, G.,
models for VARS. International Economic Review, 45(2), 643–673.
Meira, W., Codeço, C. T., et al. (2017). Dengue prediction by the
Del Negro, M., & Schorfheide, F. (2006). How good is what you’ve
web: Tweets are a useful tool for estimating and forecasting dengue
got? DGSE-VAR as a toolkit for evaluating DSGE models. Economic
at country and city level. PLoS Neglected Tropical Diseases, 11(7),
Review-Federal Reserve Bank of Atlanta, 91(2), 21.
Article e0005729.
Del Negro, M., & Schorfheide, F. (2013). DSGE model-based forecasting.
de Almeida Pereira, G. A., & Veiga, A. (2019). Periodic copula autore-
In G. Elliott, & A. Timmermann (Eds.), vol. 2, Handbook of economic
gressive model designed to multivariate streamflow time series
forecasting (pp. 57–140). Amsterdam, Horth-Holland.
modelling. Water Resources Management, 33(10), 3417–3431.
Dellaportas, P., Denison, D. G. T., & Holmes, C. (2007). Flexible threshold
De Baets, S. (2019). Surveying forecasting: a review and directions
models for modelling interest rate volatility. Econometric Reviews,
for future research. International Journal of Information and Decision
26(2–4), 419–437.
Sciences.
Delle Monache, L., Hacker, J. P., Zhou, Y., Deng, X., & Stull, R. B. (2006).
De Baets, S., & Harvey, N. (2020). Using judgment to select and adjust
Probabilistic aspects of meteorological and ozone regional ensemble
forecasts from statistical models. European Journal of Operational
forecasts. Journal of Geophysical Research: Atmospheres, 111(D24).
Research, 284(3), 882–895.
Demirovic, E., Stuckey, P. J., Bailey, J., Chan, J., Leckie, C., Ramamoha-
De Beer, J. (1985). A time series model for cohort data. Journal of the
narao, K., et al. (2019). Predict+optimise with ranking objectives:
American Statistical Association, 80(391), 525–530.
De Beer, J. (1990). Projecting age-specific fertility rates by using Exhaustively learning linear functions. In IJCAI (pp. 1078–1085).
time-series methods. European Journal of Population, 5(4), 315–346. Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood
De Beer, J. (2008). Forecasting international migration: Time series from incomplete data via the EM algorithm. Journal of the Royal
projections vs argument-based forecasts. In International migration Statistical Society. Series B. Statistical Methodology, 39, 1–38.
in Europe (pp. 283–306). Chichester, UK: John Wiley & Sons, Ltd. Dempster, M., Payne, T., Romahi, Y., & Thompson, G. (2001). Computa-
de Brito, M. P., & van der Laan, E. A. (2009). Inventory control with tional learning techniques for intraday FX trading using popular
product returns: The impact of imperfect information. European technical indicators. IEEE Transactions on Neural Networks, 12,
Journal of Operational Research, 194(1), 85–101. 744–754.
De Gooijer, J. (1998). On threshold moving-average models. Journal of Di Corso, E., Cerquitelli, T., & Apiletti, D. (2018). Metatech: Meteoro-
Time Series Analysis, 19(1), 1–18. logical data analysis for thermal energy characterization by means
De Gooijer, J. G., & Hyndman, R. J. (2006). 25 Years of time series of self-learning transparent models. Energies, 11(6), 1336.
forecasting. International Journal of Forecasting, 22, 443–473. Diab, D. L., Pui, S.-Y., Yankelevich, M., & Highhouse, S. (2011). Lay
De Iaco, S., & Maggio, S. (2016). A dynamic model for age-specific perceptions of selection decision aids in US and non-US samples.
fertility rates in Italy. Spatial Statistics, 17, 105–120. International Journal of Selection and Assessment, 19(2), 209–216.
de Kok, S. (2017). The quest for a better forecast error metric: Dichtl, H., Drobetz, W., Lohre, H., Rother, C., & Vosskamp, P. (2019).
Measuring more than the average error. Foresight: The International Optimal timing and tilting of equity factors. Financial Analysts
Journal of Applied Forecasting, 46, 36–45. Journal, 75(4), 84–102.
De Livera, A. M., Hyndman, R. J., & Snyder, R. D. (2011). Forecasting Dickersin, K. (1990). The existence of publication bias and risk factors
time series with complex seasonal patterns using exponential for its occurrence. Jama, 263(10), 1385–1389.
smoothing. Journal of the American Statistical Association, 106(496), Dickey, D. A., & Fuller, W. A. (1979). Distribution of the estimators for
1513–1527. autoregressive time series with a unit root. Journal of the American
De Mare, J. (1980). Optimal prediction of catastrophes with applications Statistical Association, 74(366), 427–431.
to Gaussian processes. The Annals of Probability, 8(4), 841–850. Dickey, D. A., & Pantula, S. G. (1987). Determining the order of differ-
De Menezes, L. M., Bunn, D. W., & Taylor, J. W. (2000). Review of encing in autoregressive processes. Journal of Business & Economic
guidelines for the use of combined forecasts. European Journal of Statistics, 5(4), 455–461.
Operational Research, 120(1), 190–204. Diebold, F. X. (2015). Comparing predictive accuracy, twenty years
de Nicola, F., De Pace, P., & Hernandez, M. A. (2016). Co-movement of later: A personal perspective on the use and abuse of Diebold–
major energy, agricultural, and food commodity price returns: A Mariano tests. Journal of Business & Economic Statistics, 33(1),
time-series assessment. Energy Economics, 57(C), 28–41. 1.
840
Diebold, F. X., Gunther, T. A., & Tay, A. S. (1998). Evaluating den- Doornik, J. A., & Hendry, D. F. (2015). Statistical model selection with
sity forecasts with applications to financial risk management. ‘‘big data’’. Cogent Economics & Finance, 3(1).
International Economic Review, 39(4), 863–883. Doucet, A. N. d. F., & Gordon, N. J. (2001). Sequential Monte Carlo
Diebold, F. X., & Mariano, R. S. (1995). Comparing predictive accuracy. methods in practice. New York: Springer Verlag.
Journal of Business & Economic Statistics, 13(3), 253–263. Dowd, K., Cairns, A. J. G., Blake, D., Coughlan, G. D., Epstein, D., &
Diebold, F. X., & Pauly, P. (1987). Structural change and the combination Khalaf-Allah, M. (2010). Evaluating the goodness of fit of stochas-
of forecasts. Journal of Forecasting, 6(1), 21–40. tic mortality model. Insurance: Mathematics & Economics, 47(3),
Diebold, F. X., & Pauly, P. (1990). The use of prior information 255–265.
in forecast combination. International Journal of Forecasting, 6(4), Draper, D., & Krnjajić, M. (2013). Calibration results for Bayesian model
503–508. specification. Tech. rep., Department of Applied Mathematics and
Diebold, F. X., & Shin, M. (2019). Machine learning for regularized Statistics, University of California.
survey forecast combination: Partially-egalitarian lasso and its Dudek, G. (2013). Forecasting time series with multiple seasonal
derivatives. International Journal of Forecasting, 35(4), 1679–1691. cycles using neural networks with local learning. In International
Dieckmann, N. F., Gregory, R., Peters, E., & Hartman, R. (2017). Seeing conference on artificial intelligence and soft computing (pp. 52–63).
what you want to see: How imprecise uncertainty ranges enhance Springer.
motivated reasoning. Risk Analysis, 37(3), 471–486. Dudek, G. (2015). Generalized regression neural network for forecasting
Dietrich, J. K., & Joines, D. H. (1983). Rational expectations, informa- time series with multiple seasonal cycles. In Intelligent systems’2014
tional efficiency, and tests using survey data: A comment. The (pp. 839–846). Springer.
Review of Economics and Statistics, 65(3), 525–529. Dudek, G. (2016). Multilayer perceptron for GEFCom2014 probabilistic
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015). Algorithm aversion: electricity price forecasting. International Journal of Forecasting,
people erroneously avoid algorithms after seeing them err. Journal 32(3), 1057–1060.
of Experimental Psychology: General, 144(1), 114–126. Duncan, O. D., & Davis, B. (1953). An alternative to ecological
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2018). Overcoming correlation. American Sociological Review, 18, 665–666.
algorithm aversion: People will use imperfect algorithms if they can Dungey, M., Martin, V. L., Tang, C., & Tremayne, A. (2020). A threshold
even slightly modify them. Management Science, 64(3), 1155–1170. mixed count time series model: estimation and application. Studies
Dietzel, M., Baltzer, P. A., Vag, T., Gröschel, T., Gajda, M., Camara, O., et in Nonlinear Dynamics & Econometrics, 24(2).
al. (2010). Application of breast MRI for prediction of lymph node Dunis, C. L., Laws, J., & Sermpinis, G. (2010). Modelling and trading the
metastases–systematic approach using 17 individual descriptors EUR/USd exchange rate at the ECB fixing. The European Journal of
and a dedicated decision tree. Acta Radiologica, 51(8), 885–894. Finance, 16(6), 541–560.
Ding, R., Wang, Q., Dang, Y., Fu, Q., Zhang, H., & Zhang, D. (2015).
Dunn, D. M., Williams, W. H., & Dechaine, T. L. (1976). Aggregate
Yading: Fast clustering of large-scale time series data. Proceedings
versus subaggregate models in local area forecasting. Journal of the
of the VLDB Endowment, 8(5), 473–484.
Dion, P., Galbraith, N., & Sirag, E. (2020). Using expert elicitation to
Durante, F., & Sempi, C. (2015). Principles of copula theory. CRC Press.
build long-term projection assumptions. In S. Mazzuco, & N. Keil-
Durbin, J., & Koopman, S. J. (2012). Time series analysis by state space
man (Eds.), Developments in demographic forecasting (pp. 43–62).
methods. Oxford: Oxford University Press.
Cham: Springer International Publishing.
e Silva, E. G. d. S., Legey, L. F., & e Silva, E. A. d. S. (2010). Forecasting
Dissanayake, G. S., Peiris, M. S., & Proietti, T. (2018). Fractionally
oil price trends using wavelets and hidden Markov models. Energy
differenced gegenbauer processes with long memory: A review.
Economics, 32(6), 1507–1519.
Statistical Science, 33, 413–426.
Easingwood, C. J., Mahajan, V., & Muller, E. (1983). A nonuniform
Divakar, S., Ratchford, B. T., & Shankar, V. (2005). CHAN4CAST: A
influence innovation diffusion model of new product acceptance.
multichannel, multiregion sales forecasting model and decision
Marketing Science, 2(3), 273–295.
support system for consumer packaged goods. Marketing Science,
Eastwood, J., Snook, B., & Luther, K. (2012). What people want from
24(3), 334–350.
their professionals: Attitudes toward decision-making strategies.
Dixon, M. J., & Coles, S. C. (1997). Modelling association football scores
Journal of Behavioral Decision Making, 25(5), 458–468.
and inefficiencies in the football betting market. Applied Statistics,
Eaves, A. H. C., & Kingsman, B. G. (2004). Forecasting for the ordering
47(3), 265—280.
and stock-holding of spare parts. Journal of the Operational Research
Do, L., Vu, H., Vo, B., Liu, Z., & Phung, D. (2019). An effective spatial-
Society, 55(4), 431–437.
temporal attention based neural network for traffic flow prediction.
Transportation Research Part C (Emerging Technologies), 108, 12–28. Eberhardt, M. (2012). Estimating panel time-series models with
Doan, T., Litterman, R., & Sims, C. (1984). Forecasting and conditional heterogeneous slopes. The Stata Journal, 12(1), 61–71.
projection using realistic prior distributions. Econometric Reviews, Economou, T., Stephenson, D. B., Rougier, J. C., Neal, R. A., & Mylne, K. R.
3(1), 1–100. (2016). On the use of Bayesian decision theory for issuing natural
Dokumentov, A. (2017). Smoothing, decomposition and forecasting of hazard warnings. Proceedings of the Royal Society: Mathematical,
multidimensional and functional time series using regularisation. Physical, and Engineering Sciences, 472(2194), Article 20160295.
Monash University. Edge, R. M., & Gürkaynak, R. (2010). How useful are estimated DSGE
Dokumentov, A., & Hyndman, R. J. (2018). stR: STR Decomposition. R model forecasts for central bankers? Brookings Papers on Economic
package version 0.4. Activity, 41(2 (Fall)), 209–259.
Dolara, A., Grimaccia, F., Leva, S., Mussetta, M., & Ogliari, E. (2015). A Edwards, D. G., & Hsu, J. C. (1983). Multiple comparisons with the best
physical hybrid artificial neural network for short term forecasting treatment. Journal of the American Statistical Association, 78(384),
of PV plant power output. Energies, 8(2), 1–16. 965–971.
Dolara, A., Grimaccia, F., Leva, S., Mussetta, M., & Ogliari, E. (2018). Efron, B. (1979). Bootstrap methods: Another look at the jackknife. The
Comparison of training approaches for photovoltaic forecasts by Annals of Statistics, 7(1), 1–26.
means of machine learning. Applied Sciences, 8(2), 228. Efron, B., & Tibshirani, R. (1986). Bootstrap methods for standard errors,
Dolgin, E. (2010). Better forecasting urged to avoid drug waste. Nature confidence intervals, and other measures of statistical accuracy.
Publishing Group. Statistical Science, 54–75.
Dong, X., Li, Y., Rapach, D. E., & Zhou, G. (2021). Anomalies and the Eggleton, I. R. C. (1982). Intuitive time-series extrapolation. Journal of
expected market return. The Journal of Finance, in press. Accounting Research, 20(1), 68–102.
Doornik, J. A. (2018). Autometrics. In J. L. Castle, & N. Shephard (Eds.), Ehsani, S., & Linnainmaa, J. T. (2020). Factor momentum and the
The methodology and practice of econometrics (pp. 88–121). Oxford: momentum factor. SSRN:3014521.
Oxford University Press. Eichenbaum, M., Johannsen, B. K., & Rebelo, S. (2017). Monetary policy
Doornik, J. A., Castle, J. L., & Hendry, D. F. (2020a). Card forecasts for and the predictability of nominal exchange rates. NBER Working
M4. International Journal of Forecasting, 36, 129–134. Papers 23158, National Bureau of Economic Research, Inc.
Doornik, J. A., Castle, J. L., & Hendry, D. F. (2020b). Short-term Eksoz, C., Mansouri, S. A., Bourlakis, M., & Önkal, D. (2019). Judgmental
forecasting of the coronavirus pandemic. International Journal of adjustments through supply integration for strategic partnerships
Forecasting. in food chains. Omega, 87, 20–33.
841
El Balghiti, O., Elmachtoub, A. N., Grigas, P., & Tewari, A. Fan, S., Mao, C., & Chen, L. (2006). Electricity peak load forecasting with
(2019). Generalization bounds in the predict-then-optimize frame- self-organizing map and support vector regression. IEEJ Transactions
work. In Advances in neural information processing systems (pp. on Electrical and Electronic Engineering, 1(3), xxxi.
14412–14421). Fan, Y., Nowaczyk, S., & Röognvaldsson, T. (2020). Transfer learning for
El-Hendawi, M., & Wang, Z. (2020). An ensemble method of full wavelet remaining useful life prediction based on consensus self-organizing
packet transform and neural network for short term electrical load models. Reliability Engineering & System Safety, 203, Article 107098.
forecasting. Electric Power Systems Research, 182, Article 106265. Fan, Y., & Tang, C. Y. (2013). Tuning parameter selection in high
Elangasinghe, M. A., Singhal, N., Dirks, K. N., Salmond, J. A., & Samaras- dimensional penalized likelihood. Journal of the Royal Statistical
inghe, S. (2014). Complex time series analysis of PM10 and PM2.5 Society. Series B. Statistical Methodology, 75(3), 531–552.
for a coastal site using artificial neural network modelling and Fan, J., & Yao, Q. (2005). Springer series in statistics, Nonlinear time series:
k-means clustering. Atmospheric Enviroment, 94, 106–116. nonparametric and parametric methods (p. 576). New York: Springer.
Elliott, G. (2015). Complete subset regressions with large-dimensional Faraji, J., Ketabi, A., Hashemi-Dezaki, H., Shafie-Khah, M., & Catalão, J.
sets of predictors. Journal of Economic Dynamics & Control, 54, P. S. (2020). Optimal day-ahead self-scheduling and operation of
86–111. prosumer microgrids using hybrid machine learning-based weather
Elliott, G., Timmermann, A., & Komunjer, I. (2005). Estimation and and load forecasting. IEEE Access, 8, 157284–157305.
testing of forecast rationality under flexible loss. Review of Economic Farmer, J. D., & Foley, D. (2009). The economy needs agent-based
Studies, 72(4), 1107–1125. modelling. Nature, 460(7256), 685–686.
Elliott, M. R., & Valliant, R. (2017). Inference for nonprobability samples. Faust, J., & Wright, J. H. (2013). Forecasting inflation. In G. Elliott, &
Statistical Science, 32(2), 249–264. A. Timmermann (Eds.), Handbook of economic forecasting: vol. 2, (pp.
Ellison, J., Dodd, E., & Forster, J. J. (2020). Forecasting of cohort 2–56). Elsevier.
fertility under a hierarchical Bayesian approach. Journal of the Royal Favero, C. A., & Marcellino, M. (2005). Modelling and forecasting
Statistical Society. Series A, 183(3), 829–856. fiscal variables for the euro area. Oxford Bulletin of Economics and
Elmachtoub, A. N., & Grigas, P. (2017). Smart ‘‘predict, then optimize’’. Statistics, 67, 755–783.
arXiv:1710.08005. Fernandes, M., de Sá Mota, B., & Rocha, G. (2005). A multivariate
Elsbach, K. D., & Elofson, G. (2000). How the packaging of decision conditional autoregressive range model. Economics Letters, 86(3),
explanations affects perceptions of trustworthiness. Academy of 435–440.
Management Journal, 43(1), 80–89. Fernández-Villaverde, J., & Guerrón-Quintana, P. A. (2020). Estimating
Embrechts, P., Klüppelberg, C., & Mikosch, T. (2013). Modelling extremal DSGE models: Recent advances and future challenges. Working
events: for insurance and finance. Springer Science & Business Media. Paper 27715, National Bureau of Economic Research.
Engel, C., Lee, D., Liu, C., Liu, C., & Wu, S. P. Y. (2019). The uncovered Fezzi, C., & Mosetti, L. (2020). Size matters: Estimation sample length
interest parity puzzle, exchange rate forecasting, and taylor rules. and electricity price forecasting accuracy. The Energy Journal, 41(4).
Journal of International Money and Finance, 95, 317–331. Fifić, M., & Gigerenzer, G. (2014). Are two interviewers better than
Engel, C., Mark, N. C., & West, K. D. (2008). Exchange rate models are one? Journal of Business Research, 67(8), 1771–1779.
not as bad as you think. In D. Acemoglu, K. Rogoff, & M. Woodford Figlewski, S., & Wachtel, P. (1981). The formation of inflationary
(Eds.), NBER chapters: vol. 22, NBER macroeconomics annual 2007 expectations. The Review of Economics and Statistics, 63(1), 1–10.
(pp. 381–441). National Bureau of Economic Research, Inc. Figlewski, S., & Wachtel, P. (1983). Rational expectations, informational
Engelberg, J., Manski, C. F., & Williams, J. (2009). Comparing the point efficiency, and tests using survey data: A reply. The Review of
predictions and subjective probability distributions of professional Economics and Statistics, 65(3), 529–531.
forecasters. Journal of Business & Economic Statistics, 27(1), 30–41. Fildes, R. (2017). Research into forecasting practice. Foresight: The
Engle, R. F. (1982). Autoregressive conditional heteroscedasticity with International Journal of Applied Forecasting, 44, 39–46.
estimates of the variance of United Kingdom inflation. Econometrica, Fildes, R., & Goodwin, P. (2007). Against your better judgment? How
50(4), 987. organizations can improve their use of management judgment in
Engle, R. (2002). Dynamic conditional correlation. Journal of Business & forecasting. Interfaces, 37(6), 570–576.
Economic Statistics, 20(3), 339–350. Fildes, R., & Goodwin, P. (2013). Forecasting support systems: What we
Engle, R. (2004). Risk and volatility: Econometric models and financial know, what we need to know. International Journal of Forecasting,
practice. American Economic Review, 94(3), 405–420. 29(2), 290–294.
Engle, R. F., Ghysels, E., & Sohn, B. (2013). Stock market volatility Fildes, R., Goodwin, P., & Lawrence, M. (2006). The design features
and macroeconomic fundamentals. The Review of Economics and of forecasting support systems and their effectiveness. Decision
Statistics, 95(3), 776–797. Support Systems, 42(1), 351–361.
Engle, R. F., & Kroner, K. F. (1995). Multivariate simultaneous Fildes, R., Goodwin, P., Lawrence, M., & Nikolopoulos, K. (2009).
generalized ARCH. Economic Theory, 11(1), 122–150. Effective forecasting and judgmental adjustments: an empiri-
Engle, R. F., & Russell, J. R. (1997). Forecasting the frequency of cal evaluation and strategies for improvement in supply-chain
changes in quoted foreign exchange prices with the autoregres- planning. International Journal of Forecasting, 25(1), 3–23.
sive conditional duration model. Journal of Empirical Finance, 4(2), Fildes, R., Goodwin, P., & Önkal, D. (2019a). Use and misuse of informa-
187–212. tion in supply chain forecasting of promotion effects. International
Engle, R. F., & Russell, J. R. (1998). Autoregressive conditional duration: Journal of Forecasting, 35(1), 144–156.
A new model for irregularly spaced transaction data. Econometrica, Fildes, R., Ma, S., & Kolassa, S. (2019b). Retail forecasting: research and
66(5), 1127–1162. practice. International Journal of Forecasting.
Erikson, R. S., & Wlezien, C. (2012). Markets vs. polls as elec- Fildes, R., & Petropoulos, F. (2015). Improving forecast quality in
tion predictors: An historical assessment. Electoral Studies, 31(3), practice. Foresight: The International Journal of Applied Forecasting,
532–539. 36(Winter), 5–12.
European Banking Federation (2019). EBF position paper on AI in the Filippou, I., Rapach, D. E., Taylor, M. P., & Zhou, G. (2020). Exchange rate
banking industry. EBF_037419. prediction with machine learning and a smart carry trade portfolio.
Evans, M. D. R. (1986). American fertility patterns: A comparison SSRN:3455713.
of white and nonwhite cohorts Born 1903-56. Population and Findley, D. F. (2005). Some recent developments and directions in
Development Review, 12(2), 267–293. seasonal adjustment. Journal of Official Statistics, 21(2), 343.
Fahimnia, B., Sanders, N., & Siemsen, E. (2020). Human judgment in Findley, D. F., Monsell, B. C., Bell, W. R., Otto, M. C., & Chen, B.-C.
supply chain forecasting. Omega, 94, Article 102249. (1998). New capabilities and methods of the X-12-ARIMA seasonal-
Fair, R. C. (1978). The effect of economic events on votes for president. adjustment program. Journal of Business & Economic Statistics, 16(2),
The Review of Economics and Statistics, 60(2), 159–173. 127–152.
Fan, S., Chen, L., & Lee, W.-J. (2008). Machine learning based switch- Fiori, F., Graham, E., & Feng, Z. (2014). Geographical variations in
ing model for electricity load forecasting. Energy Conversion & fertility and transition to second and third birth in britain. Advances
Management, 49(6), 1331–1344. in Life Course Research, 21, 149–167.
842
Fiori, C., & Kovaka, M. (2005). Defining megaprojects: Learning from Foucquier, A., Robert, S., Suard, F., Stéphan, L., & Jay, A. (2013). State of
construction at the edge of experience. Construction Research the art in building modelling and energy performances prediction:
Congress 2005, 1–10. A review. Renewable and Sustainable Energy Reviews, 23, 272–288.
Fiorucci, J. A., Pellegrini, T. R., Louzada, F., Petropoulos, F., & Koehler, A. Fox, A. J. (1972). Outliers in time series. Journal of the Royal Statistical
B. (2016). Models for optimising the theta method and their rela- Society. Series B. Statistical Methodology, 34(3), 350–363.
tionship to state space models. International Journal of Forecasting, Frankel, J., & Schreger, J. (2013). Over-optimistic official forecasts
32(4), 1151–1161. and fiscal rules in the eurozone. Review of World Economics, 149,
Fioruci, J. A., Pellegrini, T. R., Louzada, F., & Petropoulos, F. (2015). The 247–272.
optimised theta method. arXiv:1503.03529. Franses, P. H. (1991). Seasonality, non-stationarity and the forecasting
Firebaugh, G. (1978). A rule for inferring individual-level relationships of monthly time series. International Journal of Forecasting, 7(2),
from aggregate data. American Sociological Review, 43(4), 557–572. 199–208.
Fischhoff, B. (2007). An early history of hindsight research. Social Franses, P. H., van Dijk, D., & Opschoor, A. (2014). Time series models
Cognition, 25(1), 10–13. for business and economic forecasting. Cambridge University Press.
Fischhoff, B. (2012). Communicating uncertainty fulfilling the duty to Franses, P. H., & Ghijsels, H. (1999). Additive outliers, GARCH and
inform. Issues in Science and Technology, 28(4), 63–70. forecasting volatility. International Journal of Forecasting, 15(1), 1–9.
Fischhoff, B., & Davis, A. L. (2014). Communicating scientific Franses, P. H., & Legerstee, R. (2009a). Do experts’ adjustments on
uncertainty. Proceedings of the National Academy of Sciences, model-based SKU-level forecasts improve forecast quality? Journal
111(Supplement 4), 13664–13671. of Forecasting, 36.
Fisher, J. C., & Pry, R. H. (1971). A simple substitution model of Franses, P. H., & Legerstee, R. (2009b). Properties of expert adjust-
technological change. Technological Forecasting and Social Change, ments on model-based SKU-level forecasts. International Journal of
3, 75–88. Forecasting, 25(1), 35–47.
Fiske, S. T., & Dupree, C. (2014). Gaining trust as well as respect Franses, P. H., & Legerstee, R. (2009c). A unifying view on multi-step
in communicating to motivated audiences about science topics. forecasting using an autoregression. Journal of Economic Surveys,
Proceedings of the National Academy of Sciences, 111(Supplement 4), 24(3), 389–401.
13593–13597. Frazier, D. T., Loaiza-Maya, R., Martin, G. M., & Koo, B. (2021).
Fissler, T., Frongillo, R., Hlavinová, J., & Rudloff, B. (2020). Forecast Loss-based variational Bayes prediction. arXiv:2104.14054.
evaluation of quantiles, prediction intervals, and other set-valued Frazier, D. T., Maneesoonthorn, W., Martin, G. M., & McCabe, B. P.
functionals. arXiv:1910.07912. (2019). Approximate Bayesian forecasting. International Journal of
Fiszeder, P. (2005). Forecasting the volatility of the polish stock index Forecasting, 35(2), 521–539.
– WIG20. In W. Milo, & P. Wdowiński (Eds.), Forecasting finan- Frechtling, D. C. (2001). Forecasting tourism demand: methods and
cial markets. theory and applications (pp. 29–42). Wydawnictwo strategies. Routledge.
Uniwersytetu Łódzkiego. Freedman, D. A. (1981). Bootstrapping regression models. The Annals of
Statistics, 9(6), 1218–1228.
Fiszeder, P. (2018). Low and high prices can improve covariance fore-
Freedman, D. A., Klein, S. P., Ostland, M., & Roberts, M. (1998). Review
casts: The evidence based on currency rates. Journal of Forecasting,
of ‘a solution to the ecological inference problem’. Journal of the
37(6), 641–649.
Fiszeder, P., & Fałdziński, M. (2019). Improving forecasts with the co-
Freeland, K., & McCabe, B. P. (2004). Forecasting discrete valued
range dynamic conditional correlation model. Journal of Economic
low count time series. International Journal of Forecasting, 20(3),
Dynamics and Control, 108, Article 103736.
427–434.
Fiszeder, P., Fałdziński, M., & Molnár, P. (2019). Range-based DCC
Freyberger, J., Neuhierl, A., & Weber, M. (2020). Dissecting char-
models for covariance and value-at-risk forecasting. Journal of
acteristics nonparametrically. Review of Financial Studies, 33,
Empirical Finance, 54, 58–76.
2326–2377.
Fiszeder, P., & Perczak, G. (2013). A new look at variance estimation
Friedman, J. A. (2015). Using power laws to estimate conflict size. The
based on low, high and closing prices taking into account the drift.
Journal of Conflict Resolution, 59(7), 1216–1241.
Statistica Neerlandica, 67(4), 456–481.
Fry, C., & Brundage, M. (2020). The M4 forecasting competition –
Fiszeder, P., & Perczak, G. (2016). Low and high prices can improve
a practitioner’s view. International Journal of Forecasting, 36(1),
volatility forecasts during periods of turmoil. International Journal
156–160.
of Forecasting, 32(2), 398–410.
Fuhrer, J. C. (2018). Intrinsic expectations persistence: Evidence from
Fixler, D. J., & Grimm, B. T. (2005). Reliability of the NIPA estimates of
professional and household survey expectations: Working Papers 18–9,
U.S. economic activity. Survey of Current Business, 85, 9–19.
Federal Reserve Bank of Boston.
Fixler, D. J., & Grimm, B. T. (2008). The reliability of the GDP and GDI Fulcher, B. D., & Jones, N. S. (2014). Highly comparative feature-based
estimates. Survey of Current Business, 88, 16–32. time-series classification. IEEE Transactions on Knowledge and Data
Fliedner, G. (2003). CPFR: an emerging supply chain tool. Industrial Engineering, 26(12), 3026–3037.
Management & Data Systems, 103(1), 14–21. Fulcher, B. D., Little, M. A., & Jones, N. S. (2013). Highly comparative
Flyvbjerg, B. (2007). Policy and planning for large-infrastructure time-series analysis: the empirical structure of time series and
projects: Problems, causes, cures. Environment and Planning: B, their methods. Journal of the Royal Society Interface, 10(83), Article
Planning & Design, 34(4), 578–597. 20130048.
Flyvbjerg, B., Bruzelius, N., & Rothengatter, W. (2003). Megaprojects and Funahashi, K.-I. (1989). On the approximate realization of continuous
risk: an anatomy of ambition. Cambridge University Press. mappings by neural networks. Neural Networks, 2(3), 183–192.
Forcina, A., & Pellegrino, D. (2019). Estimation of voter transitions and Furlan, C., & Mortarino, C. (2018). Forecasting the impact of renewable
the ecological fallacy. Quality & Quantity, 53(4), 1859–1874. energies in competition with non-renewable sources. Renewable
Forni, M., Hallin, M., Lippi, M., & Reichlin, L. (2003). Do financial and Sustainable Energy Reviews, 81, 1879–1886.
variables help forecasting inflation and real activity in the euro Furlan, C., Mortarino, C., & Zahangir, M. S. (2020). Interaction among
area? Journal of Monetary Economics, 50(6), 1243–1255. three substitute products: An extended innovation diffusion model.
Forrest, D., Goddard, J., & Simmons, R. (2005). Odds-setters as forecast- Statistical Methods & Applications, in press.
ers: The case of english football. International Journal of Forecasting, Gaddis, J. L. (1989). The long peace: Inquiries into the history of the cold
21, 551–564. war. The Long Peace: Inquiries Into the History of the Cold War.
Forrest, D., & Simmons, R. (2006). New issues in attendance demand: Gaillard, P., Goude, Y., & Nedellec, R. (2016). Additive models and
The case of the english football league. Journal of Sports Economics, robust aggregation for GEFCom2014 probabilistic electric load and
7(3), 247–263. electricity price forecasting. International Journal of Forecasting,
Fortsch, S. M., & Khapalova, E. A. (2016). Reducing uncertainty in 32(3), 1038–1050.
demand for blood. Operations Research for Health Care, 9, 16–28. Galbreth, M. R., Kurtuluş, M., & Shor, M. (2015). How collaborative fore-
Fortuin, L. (1984). Initial supply and re-order level of new service parts. casting can reduce forecast accuracy. Operations Research Letters,
European Journal of Operational Research, 15(3), 310–319. 43(4), 349–353.
843
Gali, J. (2008). Monetary policy, inflation, and the business cycle: An Geweke, J., & Amisano, G. (2011). Optimal prediction pools. Journal of
introduction to the new keynesian framework. Princeton and Oxford: Econometrics, 164(1), 130–141.
Princeton University Press. Geweke, J., Koop, G., & van Dijk, H. (2011). The oxford handbook of
Galicia, A., Talavera-Llames, R., Troncoso, A., Koprinska, I., & Martínez- bayesian econometrics. OUP.
Álvarez, F. (2019). Multi-step forecasting for big data time Geweke, J., & Whiteman, C. (2006). BayesIan forecasting. The Handbook
series based on ensemble learning. Knowledge-Based Systems, 163, of Economic Forecasting, 1, 3–98.
830–841. Gharbi, M., Quenel, P., Gustave, J., Cassadou, S., La Ruche, G., Girdary, L.,
Galicia, A., Torres, J. F., Martínez-Álvarez, F., & Troncoso, A. (2018). et al. (2011). Time series analysis of dengue incidence in Guade-
A novel Spark-based multi-step forecasting algorithm for big data loupe, French West Indies: forecasting models using climate
time series. Information Sciences, 467, 800–818. variables as predictors. BMC Infectious Diseases, 11(1), 1–13.
Galvão, A. B. (2017). Data revisions and DSGE models. Journal of Ghassemi, M., Pimentel, M. A., Naumann, T., Brennan, T., Clifton, D.
Econometrics, 196(1), 215–232. A., Szolovits, P., et al. (2015). A multivariate timeseries modeling
Galvão, A. B., Giraitis, L., Kapetanios, G., & Petrova, K. (2016). A time approach to severity of illness assessment and forecasting in ICU
varying DSGE model with financial frictions. Journal of Empirical with sparse, heterogeneous clinical data. 2015, In Proceedings of the
Finance, 38, 690–716. AAAI conference on artificial intelligence. aaai conference on artificial
Gamble, C., & Gao, J. (2018). Safety-first AI for autonomous data centre intelligence (p. 446). NIH Public Access.
cooling and industrial control. In DeepMind. Accessed on 2020-09- Ghysels, E., Lee, H. S., & Noh, J. (1994). Testing for unit roots in
01, https://deepmind.com/blog/article/safety-first-ai-autonomous- seasonal time series: Some theoretical extensions and a Monte
data-centre-cooling-and-industrial-control. Carlo investigation. Journal of Econometrics, 62(2), 415–442.
Gans, N., Koole, G., & Mandelbaum, A. (2003). Telephone call centers: Ghysels, E., Plazzi, A., Valkanov, R., & Torous, W. (2013). Forecasting
Tutorial, review, and research prospects. Manufacturing & Service real estate prices. In G. Elliott, & A. Timmermann (Eds.), vol. 2,
Operations Management, 5(2), 79–141. Handbook of economic forecasting (pp. 509–580). Elsevier.
García, F. P., Pedregal, D. J., & Roberts, C. (2010). Time series methods Giacomini, R., & Rossi, B. (2016). Model comparisons in unstable
applied to failure prediction and detection. Reliability Engineering & environments. International Economic Review, 57(2), 369–392.
System Safety, 95(6), 698–703. Giacomini, R., & White, H. (2006). Tests of conditional predictive ability.
Garcia, R., & Perron, P. (1996). An analysis of the real interest rate Econometrica, 74(6), 1545–1578.
under regime shifts. The Review of Economics and Statistics, 78(1), Giani, A., Bitar, E., Garcia, M., McQueen, M., Khargonekar, P., & Poolla, K.
111–125. (2013). Smart grid data integrity attacks. IEEE Transactions on Smart
Gardner, E. S. (1985). Exponential smoothing: The state Grid, 4(3), 1244–1253.
of the art. In ICT Monograph: Journal of Forecasting, In ICT Giannone, D. L., & Primiceri, G. M. (2017). Macroeconomic prediction
Monograph: 4(1).1–28, with big data: The illusion of sparsity. The Fedral Reserve Bank of
Gardner, E. S. (2006). Exponential smoothing: The state of the art -
New York.
part II. International Journal of Forecasting, 22(4), 637–666.
Gias, A. U., & Casale, G. (2020). COCOA: COld start aware capacity
Gardner, E., & Koehler, A. B. (2005). Comments on a patented boot-
planning for function-as-a-service platforms. ArXiv:2007.01222.
strapping method for forecasting intermittent demand. International
Giebel, G., & Kariniotakis, G. (2017). Wind power forecasting—a review
Journal of Forecasting, 21(3), 617–618.
of the state of the art. In Renewable energy forecasting (pp. 59–109).
Garman, M. B., & Klass, M. J. (1980). On the estimation of security price
Elsevier.
volatilities from historical data. Journal of Business, 53(1), 67–78.
Gigerenzer, G. (1996). On narrow norms and vague heuristics: A reply
Garratt, A., Lee, K., Mise, E., & Shields, K. (2008). Real time representa-
to Kahneman and Tversky. Psychological Review, 103(3), 592–596.
tions of the output gap. The Review of Economics and Statistics, 90,
Gigerenzer, G. (2007). Gut feelings: The intelligence of the unconscious.
792–804.
Viking.
Gartner, W. B., & Thomas, R. J. (1993). Factors affecting new product
Gil, R. G. R., & Levitt, S. D. (2007). Testing the efficiency of markets
forecasting accuracy in new firms. Journal of Product Innovation
in the 2002 World Cup. The Journal of Prediction Markets, 1(3),
Management, 10(1), 35–52.
255–270.
Gasthaus, J., Benidis, K., Wang, Y., Rangapuram, S. S., Salinas, D.,
Gil-Alana, L. (2001). A fractionally integrated exponential model for UK
Flunkert, V., et al. (2019).
Gebicki, M., Mooney, E., Chen, S.-J. G., & Mazur, L. M. (2014). Evaluation unemployment. Journal of Forecasting, 20(5), 329–340.
of hospital medication inventory policies. Health Care Management Gilbert, K. (2005). An ARIMA supply chain model. Management Science,
Science, 17(3), 215–229. 51(2), 305–310.
Gelman, A., Park, D. K., Ansolabehere, S., Price, P. N., & Minnite, L. Gilbert, C., Browell, J., & McMillan, D. (2020a). Leveraging turbine-
C. (2001). Models, assumptions and model checking in ecological level data for improved probabilistic wind power forecasting. IEEE
regressions. Journal of the Royal Statistical Society, Series A, 164(1), Transactions on Sustainable Energy, 11(3), 1152–1160.
101–118. Gilbert, C., Browell, J., & McMillan, D. (2020b). Probabilistic access
Gelper, S., Fried, R., & Croux, C. (2009). Robust forecasting with forecasting for improved offshore operations. International Journal
exponential and holt-winters smoothing. Journal of Forecasting, 11. of Forecasting.
Gentine, P., Pritchard, M., Rasp, S., Reinaudi, G., & Yacalis, G. (2018). Gilliland, M. (2002). Is forecasting a waste of time? Supply Chain
Could machine learning break the convection parameterization Management Review, 6(4), 16–23.
deadlock? Geophysical Research Letters, 45(11), 5742–5751. Gilliland, M. (2010). The business forecasting deal: Exposing myths,
George, E. I., & McCulloch, R. E. (1993). Variable selection via Gibbs eliminating bad practices, providing practical solutions. John Wiley
sampling. Journal of the American Statistical Association, 88(423), & Sons.
881–890. Giraitis, L., Kapetanios, G., & Price, S. (2013). Adaptive forecasting in
Gerlach, R., Chen, C. W. S., Lin, D. S. Y., & Huang, M.-H. (2006). the presence of recent and ongoing structural change. Journal of
Asymmetric responses of international stock markets to trading Econometrics, 177(2), 153–170.
volume. Physica A: Statistical Mechanics and its Applications, 360(2), Givon, M., Mahajan, W., & Müller, E. (1995). Software piracy: Estimation
422–444. of the lost sales and the impact on software diffusion. Journal of
Gerland, P., Raftery, A. E., Ševčíková, H., Li, N., Gu, D., Spoorenberg, T., Marketing, 59, 29–37.
et al. (2014). World population stabilization unlikely this century. Glahn, H. R., & Lowry, D. A. (1972). The use of model output statis-
Science, 346(6206), 234–237. tics (MOS) in objective weather forecasting. Journal of Applied
Geweke, J. (1977). The dynamic factor analysis of economic time series. Meteorology, 11(8), 1203–1211.
In Latent Variables in Socio-Economic Models. North-Holland. Gleditsch, R. F., & Syse, A. (2020). Ways to project fertility in Europe. Per-
Geweke, J. (2001). BayesIan econometrics and forecasting. Journal of ceptions of current practices and outcomes: Tech. Rep. 929, Statistics
Econometrics, 100(1), 11–15. Norway, Research Department.
Geweke, J., & Amisano, G. (2010). Comparing and evaluating Bayesian Glocker, C., & Wegmüller, P. (2018). International evidence of time-
predictive distributions of asset returns. International Journal of variation in trend labor productivity growth. Economics Letters, 167,
Forecasting, 26(2), 216–230. 115–119.
844
Glosten, L. R., Jagannathan, R., & Runkle, D. E. (1993). On the relation Gönül, M. S., Önkal, D., & Goodwin, P. (2012). Why should I trust your
between the expected value and the volatility of the nominal excess forecasts? Foresight: The International Journal of Applied Forecasting,
return on stocks. The Journal of Finance, 48(5), 1779–1801. 27, 5–9.
Glynn, A., & Wakefield, J. (2010). Ecological inference in the social Gönül, M. S., Önkal, D., & Lawrence, M. (2006). The effects of structural
sciences. Statistical Methodology, 7(3), 307–322. characteristics of explanations on use of a DSS. Decision Support
Gneiting, T. (2011a). Making and evaluating point forecasts. Journal of Systems, 42(3), 1481–1493.
the American Statistical Association, 106(494), 746–762. Goodman, L. A. (1953). Ecological regressions and behavior of
Gneiting, T. (2011b). Quantiles as optimal point forecasts. International individuals. American Sociological Review, 18, 663–664.
Journal of Forecasting, 27(2), 197–207. Goodman, L. A. (1959). Some alternatives to ecological correlation. The
Gneiting, T., Balabdaoui, F., & Raftery, A. E. (2007). Probabilistic fore- American Journal of Sociology, 64(6), 610–625.
casts, calibration and sharpness. Journal of the Royal Statistical Goodwin, P. (2000a). Correct or combine? Mechanically integrating
Society. Series B. Statistical Methodology, 69, 243–268. judgmental forecasts with statistical methods. International Journal
Gneiting, T., & Katzfuss, M. (2014). Probabilistic forecasting. Annual of Forecasting, 16(2), 261–275.
Goodwin, P. (2000b). Improving the voluntary integration of statistical
Review of Statistics and Its Application, 1, 125–151.
forecasts and judgment. International Journal of Forecasting, 16(1),
Gneiting, T., & Raftery, A. E. (2007). Strictly proper scoring rules, predic-
85–99.
tion, and estimation. Journal of the American Statistical Association,
Goodwin, P. (2002). Integrating management judgment and statistical
102(477), 359–378.
methods to improve short-term forecasts. Omega, 30(2), 127–135.
Gneiting, T., Raftery, A. E., Westveld, A. H., & Goldman, T. (2005).
Goodwin, P. (2014). Getting real about uncertainty. Foresight: The
Calibrated probabilistic forecasting using ensemble model output
International Journal of Applied Forecasting, 33, 4–7.
statistics and minimum CRPS estimation. Monthly Weather Review,
Goodwin, P., Dyussekeneva, K., & Meeran, S. (2013). The use of analo-
133(5), 1098–1118.
gies in forecasting the annual sales of new electronics products.
Gneiting, T., & Ranjan, R. (2013). Combining predictive distributions. IMA Journal of Management Mathematics, 24(4), 407–422.
Electronic Journal of Statistics, 7, 1747–1782. Goodwin, P., & Fildes, R. (1999). Judgmental forecasts of time series
Gneiting, T., Stanberry, L. I., Grimit, E. P., Held, L., & Johnson, N. A. affected by special events: does providing a statistical forecast
(2008). Assessing probabilistic forecasts of multivariate quantities, improve accuracy? Journal of Behavioural Decision Making, 12(1),
with applications to ensemble predictions of surface winds (with 37–53.
discussion and rejoinder). Test, 17, 211–264. Goodwin, P., Fildes, R., Lawrence, M., & Nikolopoulos, K. (2007). The
Godahewa, R., Deng, C., Prouzeau, A., & Bergmeir, C. (2020). Simula- process of using a forecasting support system. International Journal
tion and optimisation of air conditioning systems using machine of Forecasting, 23(3), 391–404.
learning. ArXiv:2006.15296. Goodwin, P., Fildes, R., Lawrence, M., & Stephens, G. (2011). Re-
Godbole, N., Srinivasaiah, M., & Skiena, S. (2007). Large-scale sentiment strictiveness and guidance in support systems. Omega, 39(3),
analysis for news and blogs. ICWSM, 7(21), 219–222. 242–253.
Godet, M. (1982). From forecasting to ‘la prospective’ a new way of Goodwin, P., Gönül, M. S., & Önkal, D. (2013). Antecedents and effects
looking at futures. Journal of Forecasting, 1(3), 293–301. of trust in forecasting advice. International Journal of Forecasting,
Goh, T. N., & Varaprasad, N. (1986). A statistical methodology for the 29(2), 354–366.
analysis of the life-cycle of reusable containers. IIE Transactions, Goodwin, P., Gönül, M. S., & Önkal, D. (2019a). When providing opti-
18(1), 42–47. mistic and pessimistic scenarios can be detrimental to judgmental
Goia, A., May, C., & Fusai, G. (2010). Functional clustering and lin- demand forecasts and production decisions. European Journal of
ear regression for peak load forecasting. International Journal of Operational Research, 273(3), 992–1004.
Forecasting, 26(4), 700–711. Goodwin, P., Gönül, M. S., Önkal, D., Kocabıyıkoğlu, A., & Göğüş, I.
Goldberg, Y. (2017). Neural network methods for natural language (2019b). Contrast effects in judgmental forecasting when assessing
processing. Synthesis Lectures on Human Language Technologies, the implications of worst- and best-case scenarios. Journal of
10(1), 1–309. Behavioral Decision Making, 32(5), 536–549.
Goldstein, J. S. (2011). Winning the war on war: The decline of armed Goodwin, P., Petropoulos, F., & Hyndman, R. J. (2017). A note on upper
conflict worldwide. Penguin. bounds for forecast-value-added relative to naïve forecasts. Journal
Goldstein, D. G., & Gigerenzer, G. (2002). Models of ecological ra- of the Operational Research Society, 68(9), 1082–1084.
tionality: the recognition heuristic. Psychological Review, 109(1), Goodwin, P., & Wright, G. (2010). The limits of forecasting methods in
75–90. anticipating rare events. Technological Forecasting and Social Change,
77(3), 355–368.
Golestaneh, F., Pinson, P., & Gooi, H. B. (2019). Polyhedral predictive
Google code (2013). The Word2Vec project. Accessed on 2020-09-05,
regions for power system applications. IEEE Transactions on Power
https://code.google.com/archive/p/word2vec.
Systems, 34(1), 693–704.
Gorbey, S., James, D., & Poot, J. (1999). Population forecasting with
Goltsos, T., & Syntetos, A. (2020). Forecasting for remanufacturing.
endogenous migration: An application to trans-tasman migration.
Foresight: The International Journal of Applied Forecasting, 56, 10–17.
International Regional Science Review, 22(1), 69–101.
Goltsos, T. E., Syntetos, A. A., & van der Laan, E. (2019). Forecasting for
Gordon, R. J. (2003). Exploding productivity growth: context, causes,
remanufacturing: The effects of serialization. Journal of Operations
and implications. Brookings Papers on Economic Activity, 2003(2),
Management, 65(5), 447–467.
207–298.
Gomez Munoz, C. Q., De la Hermosa Gonzalez-Carrato, R. R., Trap- Gospodinov, N. (2005). Testing for threshold nonlinearity in short-term
ero Arenas, J. R., & Garcia Marquez, F. P. (2014). A novel approach interest rates. Journal of Financial Econometrics, 3(3), 344–371.
to fault detection and diagnosis on wind turbines. GlobalNEST Gould, P. G., Koehler, A. B., Ord, J. K., Snyder, R. D., Hyndman, R. J.,
International Journal, 16(6), 1029–1037. & Vahid-Araghi, F. (2008). Forecasting time series with multiple
Gonçalves, R. (2015). Minimizing symmetric mean absolute percentage seasonal patterns. European Journal of Operational Research, 191(1),
error (SMAPE). Cross Validated URL: https://stats.stackexchange. 207–222.
com/q/145490 (version: 2016-04-15). Goyal, A., & Welch, I. (2008). A comprehensive look at the empirical
Gonçalves, C., Bessa, R. J., & Pinson, P. (2021a). A critical overview performance of equity premium prediction. Review of Financial
of privacy-preserving approaches for collaborative forecasting. Studies, 21(4), 1455–1508.
International Journal of Forecasting, 37(1), 322–342. Graefe, A. (2014). Accuracy of vote expectation surveys in forecasting
Gonçalves, C., Pinson, P., & Bessa, R. J. (2021b). Towards data markets elections. Public Opinion Quarterly, 78(1), 204–232.
in renewable energy forecasting. IEEE Transactions on Sustainable Graefe, A., & Armstrong, J. S. (2011). Comparing face-to-face meetings,
Energy, 12(1), 533–542. nominal groups, Delphi and prediction markets on an estimation
Gönül, M. S., Önkal, D., & Goodwin, P. (2009). Expectations, use task. International Journal of Forecasting, 27(1), 183–195.
and judgmental adjustment of external financial and economic Graefe, A., Armstrong, J. S., Jones Jr, R. J., & Cuzán, A. G. (2014). Com-
forecasts: an empirical investigation. Journal of Forecasting, 28(1), bining forecasts: An application to elections. International Journal of
19–37. Forecasting, 30(1), 43–54.
845
Granger, C. W. J. (1969). Investigating causal relations by econometric Guidolin, M., & Guseo, R. (2015). Technological change in the U.S. music
models and cross-spectral methods. Econometrica, 37(3), 424–438. industry: Within-product, cross–product and churn effects between
Granger, C. W. J., & Newbold, P. (1976). Forecasting transformed competing blockbusters. Technological Forecasting and Social Change,
series. Journal of the Royal Statistical Society. Series B. Statistical 99, 35–46.
Methodology, 38(2), 189–203. Guidolin, M., & Guseo, R. (2016). The german energy transition:
Granger, C. W. J., & Pesaran, M. H. (2000). Economic and statistical Modeling competition and substitution between nuclear power and
measures of forecast accuracy. Journal of Forecasting, 19, 537–560. renewable energy technologies. Renewable and Sustainable Energy
Granger, C. W., & Ramanathan, R. (1984). Improved methods of Reviews, 60, 1498–1504.
combining forecasts. Journal of Forecasting, 3(2), 197–204. Guidolin, M., & Guseo, R. (2020). Has the iPhone cannibalized the
Granger, C. W. J., & Swanson, N. (1996). Future developments in the iPad? an asymmetric competition model. Applied Stochastic Models
study of cointegrated variables. Oxford Bulletin of Economics and in Business and Industry, 36, 465–476.
Statistics, 58(3), 537–553. Guidolin, M., & Mortarino, C. (2010). Cross-country diffusion of pho-
Graves, S. C. (1999). A single-item inventory model for a nonstationary tovoltaic systems: modelling choices and forecasts for national
demand process. Manufacturing & Service Operations Management, adoption patterns. Technological Forecasting and Social Change, 77(2),
1(1), 50–61. 279–296.
Gray, S. F. (1996). Modeling the conditional distribution of interest rates Guidolin, M., & Pedio, M. (2018). Essentials of time series for financial
as a regime-switching process. Journal of Financial Economics, 42(1), applications. Academic Press.
27–62. Guidolin, M., & Pedio, M. (2019). Forecasting and trading monetary
Gray, J. (2015a). Heresies: Against Progress and Other Illusions. Granta policy effects on the riskless yield curve with regime switching
Books. Nelson–Siegel models. Journal of Economic Dynamics & Control, 107,
Gray, J. (2015b). Steven pinker is wrong about violence and war. In Article 103723.
the Guardian. Accessed on 2018-05-02, http://www.theguardian. Guidolin, M., & Thornton, D. L. (2018). Predictions of short-term rates
com/books/2015/mar/13/john-gray-steven-pinker-wrong-violence- and the expectations hypothesis. International Journal of Forecasting,
war-declining. 34(4), 636–664.
Gray, C. W., Barnes, C. B., & Wilkinson, E. F. (1965). The process of Guidolin, M., & Timmermann, A. (2006). Term structure of risk un-
prediction as a function of the correlation between two scaled der alternative econometric specifications. Journal of Econometrics,
variables. Psychonomic Science, 3(1), 231. 131(1), 285–308.
Green, K. C., & Armstrong, J. S. (2007). Structured analogies for Guidolin, M., & Timmermann, A. (2009). Forecasts of US short-term
forecasting. International Journal of Forecasting, 23(3), 365–376. interest rates: A flexible forecast combination approach. Journal of
Econometrics, 150(2), 297–311.
Green, K. C., & Armstrong, J. S. (2015). Simple versus complex
Gumus, M., & Kiran, M. S. (2017). Crude oil price forecasting using
forecasting: The evidence. Journal of Business Research, 68(8),
XGBoost. In 2017 International conference on computer science and
1678–1685.
engineering (UBMK) (pp. 1100–1103). IEEE.
Green, J., Hand, J. R. M., & Zhang, X. F. (2017). The characteristics that
Gunter, U., & Önder, I. (2016). Forecasting city arrivals with Google
provide independent information about average U.S. monthly stock
Analytics. Annals of Tourism Research, 61, 199–212.
returns. Review of Financial Studies, 30(12), 4389–4436.
Gunter, U., Önder, I., & Gindl, S. (2019). Exploring the predictive ability
Greenberg, E. (2008). Introduction to bayesian econometrics. CUP.
of LIKES of posts on the facebook pages of four major city DMOs
Greiner, D. J. (2007). Ecological inference in voting rights act disputes:
in Austria. Tourism Economics, 25(3), 375–401.
Where are we now, and where do we want to be? Jurimetrics, 47(2),
Guo, X., Grushka-Cockayne, Y., & De Reyck, B. (2020). Forecasting
115–167.
airport transfer passenger flow using real-time data and machine
Greiner, D. J., & Quinn, K. M. (2010). Exit polling and racial bloc voting:
learning. Manufacturing & Service Operations Management.
combining individual-level and rxc ecological data. The Annals of
Gupta, S. (1994). Managerial judgment and forecast combination: An
Applied Statistics, 4(4), 1774–1796.
experimental study. Marketing Letters, 5(1), 5–17.
Gresnigt, F., Kole, E., & Franses, P. H. (2015). Interpreting financial
Gupta, M., Gao, J., Aggarwal, C. C., & Han, J. (2013). Outlier detection
market crashes as earthquakes: A new early warning system for
for temporal data: A survey. IEEE Transactions on Knowledge and
medium term crashes. Journal of Banking & Finance, 56, 123–139.
Data Engineering, 26(9), 2250–2267.
Gresnigt, F., Kole, E., & Franses, P. H. (2017a). Exploiting spillovers to Gürkaynak, R. S., Kısacıkoğlu, B., & Rossi, B. (2013). Do DSGE models
forecast crashes. Journal of Forecasting, 36(8), 936–955. forecast more accurately out-of-sample than VAR models? Advances
Gresnigt, F., Kole, E., & Franses, P. H. (2017b). Specification testing in in Econometrics,VAR Models in Macroeconomics – New Developments
hawkes models. Journal of Financial Econometrics, 15(1), 139–171. and Applications: Essays in Honor of Christopher A. Sims, 32, 27–79.
Gromenko, O., Kokoszka, P., & Reimherr, M. (2017). Detection of change Guseo, R. (2010). Partial and ecological correlation: a common three-
in the spatiotemporal mean function. Journal of the Royal Statistical term covariance decomposition. Statistical Methods & Applications,
Society. Series B. Statistical Methodology, 79(1), 29–50. 19(1), 31–46.
Gross, C. W., & Sohl, J. E. (1990). Disaggregation methods to expedite Guseo, R., & Guidolin, M. (2009). Modelling a dynamic market po-
product line forecasting. Journal of Forecasting, 9(3), 233–254. tential: A class of automata networks for diffusion of innovations.
Grossi, L., & Nan, F. (2019). Robust forecasting of electricity prices: Sim- Technological Forecasting and Social Change, 76, 806–820.
ulations, models and the impact of renewable sources. Technological Guseo, R., & Guidolin, M. (2011). Market potential dynamics in innova-
Forecasting and Social Change, 141, 305–318. tion diffusion: Modelling the synergy between two driving forces.
Grushka-Cockayne, Y., & Jose, V. R. R. (2020). Combining prediction Technological Forecasting and Social Change, 78, 13–24.
intervals in the M4 competition. International Journal of Forecasting, Guseo, R., & Mortarino, C. (2010). Correction to the paper ‘‘optimal
36(1), 178–185. product launch times in a Duopoly: Balancing life-cycle revenues
Grushka-Cockayne, Y., Jose, V. R. R., & Lichtendahl, K. C. (2017). En- with product cost’’. Operations Research, 58, 1522–1523.
sembles of overfit and overconfident forecasts. Management Science, Guseo, R., & Mortarino, C. (2012). Sequential market entries and com-
63(4), 1110–1130. petition modelling in multi-innovation diffusions. European Journal
Grushka-Cockayne, Y., Lichtendahl, K. C., Jose, V. R. R., & Winkler, R. L. of Operational Research, 216, 658–667.
(2017). Quantile evaluation, sensitivity to bracketing, and sharing Guseo, R., & Mortarino, C. (2014). Within-brand and cross-brand word-
business payoffs. Operations Research, 65(3), 712–728. of-mouth for sequential multi-innovation diffusions. IMA Journal of
Gu, S., Kelly, B., & Xiu, D. (2020). Empirical asset pricing via machine Management Mathematics, 25, 287–311.
learning. Review of Financial Studies, 33(5), 2223–2273. Guseo, R., & Mortarino, C. (2015). Modeling competition between two
Guerrero, V. M. (1993). Time-series analysis supported by power pharmaceutical drugs using innovation diffusion models. The Annals
transformations. Journal of Forecasting, 12(1), 37–48. of Applied Statistics, 9, 2073–2089.
Guidolin, M., & Alpcan, T. (2019). Transition to sustainable energy gen- Gutierrez, R. S., Solis, A. O., & Mukhopadhyay, S. (2008). Lumpy
eration in Australia: Interplay between coal, gas and renewables. demand forecasting using neural networks. International Journal of
Renewable Energy, 139, 359–367. Production Economics, 111(2), 409–420.
846
Gutterman, S., & Vanderhoof, I. T. (1998). Forecasting changes in Harvey, N. (2007). Use of heuristics: Insights from forecasting research.
mortality: A search for a law of causes and effects. North American Thinking & Reasoning, 13(1), 5–24.
Actuarial Journal, 2(4), 135–138. Harvey, N. (2011). Anchoring and adjustment: A Bayesian heuristic? In
Haas, M., Mittnik, S., & Paolella, M. S. (2004). A new approach to W. Brun, G. Keren, G. Kirkebøen, & H. Montgomery (Eds.), Perspec-
Markov-switching GARCH models. Journal of Financial Econometrics, tives on Thinking, Judging, and Decision Making (pp. 98–108). Oslo:
2(4), 493–530. Universitetsforlaget.
Hahn, M., Frühwirth-Schnatter, S., & Sass, J. (2010). Markov chain Harvey, A. C. (2013). Econometric Society Monographs, Dynamic models
Monte Carlo methods for parameter estimation in multidimensional for volatility and heavy tails: With applications to financial and
continuous time Markov switching models. Journal of Financial economic time series. Cambridge University Press.
Econometrics, 8(1), 88–121. Harvey, N. (2019). Commentary: Algorithmic aversion and judgmental
Hajnal, J. (1955). The prospects for population forecasts. Journal of the wisdom. Foresight: The International Journal of Applied Forecasting,
American Statistical Association, 50(270), 309–322. 54, 13–14.
Hall, P. (1990). Using the bootstrap to estimate mean squared error and Harvey, N., & Bolger, F. (1996). Graphs versus tables: Effects of data pre-
select smoothing parameter in nonparametric problems. Journal of sentation format on judgemental forecasting. International Journal
Multivariate Analysis, 32(2), 177–203. of Forecasting, 12(1), 119–137.
Hall, S. G., & Mitchell, J. (2007). Combining density forecasts. Harvey, N., Bolger, F., & McClelland, A. (1994). On the nature of
International Journal of Forecasting, 23(1), 1–13. expectations. British Journal of Psychology, 85(2), 203–229.
Hall, S. G., & Mitchell, J. (2009). Recent developments in density Harvey, D. I., Leybourne, S. J., & Newbold, P. (1998). Tests for fore-
forecasting. In T. C. Mills, & K. Patterson (Eds.), Palgrave handbook cast encompassing. Journal of Business & Economic Statistics, 16(2),
of econometrics, Volume 2: Applied econometrics (pp. 199–239). 254–259.
Palgrave MacMillan. Harvey, C. R., Liu, Y., & Zhu, H. (2016). ... And the cross-section of
Hamill, T. M., & Colucci, S. J. (1997). Verification of eta-RSM short-range expected returns. Review of Financial Studies, 29(1), 5–68.
ensemble forecasts. Monthly Weather Review, 125(6), 1312–1327. Harvey, N., & Reimers, S. (2013). Trend damping: Under-adjustment,
Hamilton, J. D. (1988). Rational-expectations econometric analysis of experimental artifact, or adaptation to features of the natural
changes in regime: An investigation of the term structure of environment? Journal of Experimental Psychology. Learning, Memory,
interest rates. Journal of Economic Dynamics & Control, 12(2), and Cognition, 39(2), 589–607.
385–423. Hasbrouck, J. (1995). One security, many markets: Determining the
Hamilton, J. D. (1990). Analysis of time series subject to changes in contributions to price discovery. The Journal of Finance, 50(4),
regime. Journal of Econometrics, 45(1), 39–70. 1175–1199.
Hamilton, J. D. (2016). Macroeconomic regimes and regime shifts. In J. Hasni, M., Aguir, M. S., Babai, M. Z., & Jemai, Z. (2019a). On the
B. Taylor, & H. Uhlig (Eds.), vol. 2, Handbook of macroeconomics (pp. performance of adjusted bootstrapping methods for intermittent
163–201). Elsevier. demand forecasting. International Journal of Production Economics,
Han, Y., He, A., Rapach, D. E., & Zhou, G. (2021). Expected stock returns
216, 145–153.
and firm characteristics: E-LASSO, assessment, and implications.
Hasni, M., Aguir, M. S., Babai, M. Z., & Jemai, Z. (2019b). Spare
SSRN:3185335.
parts demand forecasting: a review on bootstrapping methods.
Han, P. K., Klein, W. M., Lehman, T. C., Massett, H., Lee, S. C., & Freed-
International Journal of Productions Research, 57(15–16), 4791–4804.
man, A. N. (2009). Laypersons’ responses to the communication
Hassan, S., Arroyo, J., Galán Ordax, J. M., Antunes, L., & Pavón Mes-
of uncertainty regarding cancer risk estimates. Medical Decision
tras, J. (2013). Asking the oracle: Introducing forecasting principles
Making, 29(3), 391–403.
into agent-based modelling. Journal of Artificial Societies and Social
Han, J., Pei, J., & Kamber, M. (2011). Data mining: Concepts and
Simulation, 16(3).
techniques. Elsevier.
Hassani, H., & Silva, E. S. (2015). Forecasting with big data: A review.
Han, W., Wang, X., Petropoulos, F., & Wang, J. (2019). Brain imaging
Annals of Data Science, 2, 5–19.
and forecasting: Insights from judgmental model selection. Omega,
Hastie, T. J., & Tibshirani, R. J. (1990). vol. 43, Generalized additive models.
87, 1–9.
CRC Press.
Hand, D. J. (2009). Mining the past to determine the future - problems
Hastie, T., Tibshirani, R., & Friedman, J. (2009). The elements of statistical
and possibilities. International Journal of Forecasting, 25(3), 441–451.
learning. Springer-Verlag GmbH.
Hanley, J. A., Joseph, L., Platt, R. W., Chung, M. K., & Belisle, P. (2001).
Haugen, R. A. (2010). The new finance, overreaction, complexity, and their
Visualizing the median as the minimum-deviation location. The
consequences (4th ed.). Pearson Education.
American Statistician, 55(2), 150–152.
Hannan, E. J., & Quinn, B. G. (1979). The determination of the order Haugen, R. A., & Baker, N. L. (1996). Commonality in the determinants
of an autoregression. Journal of the Royal Statistical Society, B, 41, of expected stock returns. Journal of Financial Economics, 41(3),
190–195. 401–439.
Hansen, B. E. (2001). The new econometrics of structural change: Hawkes, A. G. (1969). An approach to the analysis of electoral swing.
Dating breaks in US labour productivity. Journal of Economic Journal of the Royal Statistical Society, Series A, 132(1), 68–79.
Perspectives, 15(4), 117–128. Hawkes, A. G. (1971). Point spectra of some mutually exciting point
Hansen, P. R. (2005). A test for superior predictive ability. Journal of processes. Journal of the Royal Statistical Society. Series B. Statistical
Business & Economic Statistics, 23(4), 365–380. Methodology, 33(3), 438–443.
Harford, T. (2014). Big data: A big mistake? Significance, 11, 14–19. Hawkes, A. G. (2018). Hawkes processes and their applications to
Harrell, F. E. (2015). Regression modeling strategies: with applications to finance: a review. Quantitative Finance, 18(2), 193–198.
linear models, logistic and ordinal regression, and survival analysis Hawkes, A. G., & Oakes, D. (1974). A cluster process representation of a
(2nd Ed). New York, USA: Springer. self-exciting process. Journal of Applied Probability, 11(3), 493–503.
Harris, D., Martin, G. M., Perera, I., & Poskitt, D. S. (2019). Construction Hayes, B. (2002). Computing science: Statistics of deadly quarrels.
and visualization of confidence sets for frequentist distributional American Scientist, 90, 10–14.
forecasts. Journal of Computational and Graphical Statistics, 28(1), He, A. W., Kwok, J. T., & Wan, A. T. (2010). An empirical model of daily
92–104. highs and lows of west texas intermediate crude oil prices. Energy
Harris, R. D., & Yilmaz, F. (2010). Estimation of the conditional variance- Economics, 32(6), 1499–1506.
covariance matrix of returns using the intraday range. International He, K., Yu, L., & Lai, K. K. (2012). Crude oil price analysis and
Journal of Forecasting, 26(1), 180–194. forecasting using wavelet decomposed ensemble model. Energy,
Hart, R., Hutton, J., & Sharot, T. (1975). A statistical analysis of 46(1), 564–574.
association football attendances. Applied Statistics, 24(1), 17–27. Hecht, R., & Gandhi, G. (2008). Demand forecasting for preventive AIDS
Harvey, A. C. (1990). Forecasting, structural time series models and the vaccines. Pharmacoeconomics, 26(8), 679–697.
Kalman filter. Cambridge University Press. Hedonometer (2020). Hedonometer word list. Accessed on 2020-09-05,
Harvey, N. (1995). Why are judgments less consistent in less https://hedonometer.org/words/labMT-en-v2/.
predictable task situations? Organizational Behavior and Human Heinrich, C. (2014). The mode functional is not elicitable. Biometrika,
Decision Processes, 63(3), 247–263. 101(1), 245–251.
847
Heinrich, C. (2020). On the number of bins in a rank histogram. Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., & Salakhut-
Quarterly Journal of the Royal Meteorological Society. dinov, R. R. (2012). Improving neural networks by preventing
Heinrich, C., Hellton, K. H., Lenkoski, A., & Thorarinsdottir, T. L. (2020). co-adaptation of feature detectors. ArXiv:1207.0580.
Multivariate postprocessing methods for high-dimensional seasonal Hinton, H. L., Jr. (1999). Defence inventory, continuing challenger in
weather forecasts. Journal of the American Statistical Association. managing inventories and avoiding adverse operational effects: Tech.
Heligman, L., & Pollard, J. H. (1980). The age pattern of mortality. rep., US General Accounting Office.
Journal of the Institute of Actuaries, 107, 49–80. Hipel, K. W., & McLeod, A. I. (1994). Time series modelling of water
Hemri, S. (2018). Applications of postprocessing for hydrological resources and environmental systems. Elsevier.
forecasts. In Statistical postprocessing of ensemble forecasts (pp. Hippert, H. S., Pedreira, C. E., & Souza, R. C. (2001). Neural networks
219–240). Elsevier. for short-term load forecasting: a review and evaluation. IEEE
Hemri, S., Lisniak, D., & Klein, B. (2015). Multivariate postprocess- Transactions on Power Systems, 16(1), 44–55.
ing techniques for probabilistic hydrological forecasting. Water Hobijn, B., Franses, P. H., & Ooms, M. (2004). Generalizations of the
Resources Research, 51(9), 7436–7451. KPSS-test for stationarity. Statistica Neerlandica, 58(4), 483–502.
Hendriks, F., Kienhues, D., & Bromme, R. (2015). Measuring laypeo- Hodges, P., Hogan, K., Peterson, J. R., & Ang, A. (2017). Factor timing
ple’s trust in experts in a digital age: The muenster epis- with cross-sectional and time-series predictors. Journal of Portfolio
temic trustworthiness inventory (METI). PLoS One, 10(10), Article Management, 44(1), 30–43.
Hodrick, R. J., & Prescott, E. C. (1997). Postwar US business cycles: An
e0139309.
empirical investigation. Journal of Money, Credit, and Banking, 1–16.
Hendry, D. F. (2001). Modelling UK inflation, 1875-1991. Journal of
Hoem, J. M., Madsen, D., Nielsen, J. L., Ohlsen, E. M., Hansen, H. O.,
Applied Econometrics, 16, 255–275.
& Rennermalm, B. (1981). Experiments in modelling recent danish
Hendry, D. F. (2006). Robustifying forecasts from equilibrium-correction
fertility curves. Demography, 18(2), 231–244.
systems. Journal of Econometrics, 135(1–2), 399–426.
Hoeting, J. A., Madigan, D., Raftery, A. E., & Volinsky, C. T. (1999).
Hendry, D. F. (2010). Equilibrium-correction models. In Macroeconomet- BayesIan model averaging: A tutorial (with discussion). Statistical
rics and time series analysis (pp. 76–89). Springer. Science, 214, 382–417.
Hendry, D. F. (Ed.), (2015). Introductory macro-econometrics: A new Hofmann, E., & Rutschmann, E. (2018). Big data analytics and demand
approach. London: Timberlake Consultants Press. forecasting in supply chains: a conceptual analysis. The International
Hendry, D. F. (2020). First-in, First-out: Modelling the UK’s CO2 Emissions, Journal of Logistics Management, 29(2), 739–766.
1860–2016: Working paper 2020-W02, Oxford University: Nuffield Hogarth, R. M., & Makridakis, S. (1981). Forecasting and planning: An
College. evaluation. Management Science, 27(2), 115–138.
Hendry, D., & Clements, M. (2001). Forecasting non-stationary economic Holly, S., Pesaran, M. H., & Yamagata, T. (2010). Spatial and temporal
time series. Cambridge, Mass.: MIT Press. diffusion of house prices in the U.K.: Discussion Papers 4694, Institute
Hendry, D. F., & Doornik, J. A. (2014). Empirical model discovery and of Labor Economics (IZA).
theory evaluation. Cambridge MA: MIT Press. Hollyman, R., Petropoulos, F., & Tipping, M. E. (2021). Understanding
Hendry, D. F., Johansen, S., & Santos, C. (2008a). Automatic selection of forecast reconciliation. European Journal of Operational Research.
indicators in a fully saturated regression. Computational Statistics, Holt, C. C. (2004). Forecasting seasonals and trends by exponentially
33, 317–335, Erratum, 337–339. weighted moving averages. International Journal of Forecasting,
Hendry, D. F., Johansen, S., & Santos, C. (2008b). Automatic selection of 20(1), 5–10.
indicators in a fully saturated regression.. Computational Statistics Homburg, A., Weiß, C. H., Alwan, L. C., Frahm, G., & Göb, R. (2019).
& Data Analysis, 33, 317–335. Evaluating approximate point forecasting of count processes.
Hendry, D. F., & Mizon, G. E. (2012). Open-model forecast-error tax- Econometrics, 7(3), 1–28.
onomies. In X. Chen, & N. R. Swanson (Eds.), Recent advances and Homburg, A., Weiß, C. H., Alwan, L. C., Frahm, G., & Göb, R. (2020). A
future directions in causality, prediction, and specification analysis (pp. performance analysis of prediction intervals for count time series.
219–240). Springer. Journal of Forecasting.
Herbst, E., & Schorfheide, F. (2016). Bayesian estimation of DSGE models Hong, W.-C. (2011). Traffic flow forecasting by seasonal SVR with
(1st ed.). Princeton University Press. chaotic simulated annealing algorithm. Neurocomputing, 74(12–13),
Herrera, R., & González, N. (2014). The modeling and forecasting of 2096–2107.
extreme events in electricity spot markets. International Journal of Hong, T., & Fan, S. (2016). Probabilistic electric load forecasting: A
Forecasting, 30(3), 477–490. tutorial review. International Journal of Forecasting, 32(3), 914–938.
Herrera, A. M., Hu, L., & Pastor, D. (2018). Forecasting crude oil price Hong, Y., Li, H., & Zhao, F. (2004). Out-of-sample performance of
volatility. International Journal of Forecasting, 34(4), 622–635. discrete-time spot interest rate models. Journal of Business &
Economic Statistics, 22(4), 457–473.
Herron, M. C., & Shotts, K. W. (2004). Logical inconsistency in EI-based
Hong, T., & Pinson, P. (2019). Energy forecasting in the big data world.
second-stage regressions. American Journal of Political Science, 48(1),
172–183.
Hong, T., Pinson, P., & Fan, S. (2014). Global energy forecast-
Hertzum, M. (2002). The importance of trust in software engineers’
ing competition 2012. International Journal of Forecasting, 30(2),
assessment and choice of information sources. Information and
357–363.
Organization, 12(1), 1–18.
Hong, T., Pinson, P., Fan, S., Zareipour, H., Troccoli, A., & Hyndman, R. J.
Hertzum, M. (2014). Expertise seeking: A review. Information Processing (2016). Probabilistic energy forecasting: Global energy forecasting
& Management, 50(5), 775–795. competition 2014 and beyond. International Journal of Forecasting,
Hevia, C., Gonzalez-Rozada, M., Sola, M., & Spagnolo, F. (2015). Esti- 32(3), 896–913.
mating and forecasting the yield curve using a Markov switching Hong, T., Pinson, P., Wang, Y., Weron, R., Yang, D., & Zareipour, H.
dynamic nelson and Siegel model. Journal of Applied Economics, (2020). Energy forecasting: A review and outlook. IEEE Open Access
30(6), 987–1009. Journal of Power and Energy, 7, 376–388.
Hewamalage, H., Bergmeir, C., & Bandara, K. (2021). Recurrent neural Hong, T., Xie, J., & Black, J. (2019). Global energy forecasting competi-
networks for time series forecasting: Current status and future tion 2017: Hierarchical probabilistic load forecasting. International
directions. International Journal of Forecasting, 37(1), 388–427. Journal of Forecasting, 35(4), 1389–1399.
Hii, Y. L., Zhu, H., Ng, N., Ng, L. C., & Rocklöv, J. (2012). Forecast of Honnibal, M. (2015). spaCy: INdustrial-strength natural language pro-
dengue incidence using temperature and rainfall. PLoS Neglected cessing (NLP) with python and cython. Accessed on 2020-09-10,
Tropical Diseases, 6(11), Article e1908. https://spacy.io.
Hill, C. A., Zhang, G. P., & Miller, K. E. (2018). Collaborative planning, Honoré, C., Menut, L., Bessagnet, B., Meleux, F., & Rou, L. (2007).
forecasting, and replenishment & firm performance: An empirical An integrated air quality forecast system for a metropolitan area.
evaluation. International Journal of Production Economics, 196, 12–23. Development in Environmental Science, 6, 292–300.
Hillebrand, E., & Medeiros, M. C. (2010). The benefits of bagging for Hooker, R. H. (1901). The suspension of the Berlin produce exchange
forecast models of realized volatility. Econometric Reviews, 29(5–6), and its effect upon corn prices. Journal of the Royal Statistical Society,
571–593. 64(4), 574–613.
848
Hopman, D., Koole, G., & van der Mei, R. (2021). A machine learning ap- Huh, S.-Y., & Lee, C.-Y. (2014). Diffusion of renewable energy
proach to itinerary-level booking prediction in competitive airline technologies in South Korea on incorporating their competitive
markets. ArXiv:2103.08405. interrelationships. Energy Policy, 69, 248–257.
Hora, S. C. (2004). Probability judgments for continuous quantities: Hui, F. K. C., Warton, D. I., & Foster, S. D. (2015). Tuning parameter
Linear combinations and calibration. Management Science, 50(5), selection for the adaptive lasso using ERIC. Journal of the American
597–604. Statistical Society, 110(509), 262–269.
Hörmann, S., Horváth, L., & Reeder, R. (2013). A functional version of Hylleberg, S., Engle, R. F., Granger, C. W. J., & Yoo, B. S. (1990).
the ARCH model. Economic Theory, 29(2), 267–288. Seasonal integration and cointegration. Journal of Econometrics,
Hornik, K. (1991). Approximation capabilities of multilayer feedforward 44(1), 215–238.
networks. Neural Networks, 4(2), 251–257. Hyndman, R. J. (1996). Computing and graphing Highest Density
Hornik, K., Stinchcombe, M., & White, H. (1989). Multilayer feedfor- Regions. The American Statistician, 50(2), 120–126.
ward networks are universal approximators. Neural Networks, 2(5), Hyndman, R. J. (2020). Quality measure for predictive Highest Density
359–366. Regions. Cross Validated, Accessed on 2020-08-20, URL https://
Horrace, W. C., & Schmidt, P. (2000). Multiple comparisons with the stats.stackexchange.com/q/483882.
best, with economic applications. Journal of Applied Econometrics, Hyndman, R. J., Ahmed, R. A., Athanasopoulos, G., & Shang, H. L.
15(1), 1–26. (2011). Optimal combination forecasts for hierarchical time series.
Horst, E. T., Rodriguez, A., Gzyl, H., & Molina, G. (2012). Stochas- Computational Statistics & Data Analysis, 55(9), 2579–2589.
tic volatility models including open, close, high and low prices. Hyndman, R. J., & Athanasopoulos, G. (2018). Forecasting: Principles and
Quantitative Finance, 12(2), 199–212. practice (2nd ed.). Melbourne, Australia: OTexts.
Horváth, L., & Kokoszka, P. (2012). Inference for Functional Data with Hyndman, R. J., & Athanasopoulos, G. (2021). Forecasting: Principles
Applications. New York: Springer. and practice (3rd ed.). Melbourne, Australia: OTexts, URL https:
Horváth, L., Kokoszka, P., & Rice, G. (2014). Testing stationarity of //otexts.com/fpp3/.
functional time series. Journal of Econometrics, 179(1), 66–82. Hyndman, R., Athanasopoulos, G., Bergmeir, C., Caceres, G., Chhay, L.,
Horváth, L., Liu, Z., Rice, G., & Wang, S. (2020). A functional time O’Hara-Wild, M., et al. (2020). Forecast: Forecasting functions for
series analysis of forward curves derived from commodity futures. time series and linear models. R package version 8.12.
International Journal of Forecasting, 36(2), 646–665. Hyndman, R. J., Bashtannyk, D. M., & Grunwald, G. K. (1996). Estimating
Hoskins, B. (2013). The potential for skill across the range of the and visualizing conditional densities. Journal of Computational and
seamless weather-climate prediction problem: a stimulus for Graphical Statistics, 5(4), 315–336.
our science. Quarterly Journal of the Royal Meteorological Society, Hyndman, R. J., & Billah, B. (2003). Unmasking the theta method.
139(672), 573–584. International Journal of Forecasting, 19(2), 287–290.
Hossin, M., & Sulaiman, M. (2015). A review on evaluation metrics for Hyndman, R. J., Koehler, A. B., Ord, J. K., & Snyder, R. D. (2008).
data classification evaluations. International Journal of Data Mining Forecasting with exponential smoothing: The state space approach.
& Knowledge Management Process, 5(2), 1–11. Berlin: Springer Verlag.
Hou, Y., Edara, P., & Sun, C. (2014). Traffic flow forecasting for urban Hyndman, R. J., Koehler, A. B., Snyder, R., & Grose, S. (2002). A
work zones. IEEE Transactions on Intelligent Transportation Systems, state space framework for automatic forecasting using exponen-
16(4), 1761–1770. tial smoothing methods. International Journal of Forecasting, 18(3),
Hou, K., Xue, C., & Zhang, L. (2020). Replicating anomalies. Review of 439–454.
Financial Studies, 33(5), 2019–2133. Hyndman, R. J., & Shang, H. L. (2009). Forecasting functional time series
Hsu, J. C. (1981). Simultaneous confidence intervals for all distances (with discussions). Journal of the Korean Statistical Society, 38(3),
from the ‘‘best". The Annals of Statistics, 1026–1034. 199–221.
Hu, K., Acimovic, J., Erize, F., Thomas, D. J., & Van Mieghem, J. A. (2019). Hyndman, R. J., & Ullah, M. S. (2007). Robust forecasting of mortal-
Forecasting new product life cycle curves: Practical approach and ity and fertility rates: A functional data approach. Computational
empirical analysis. Manufacturing & Service Operations Management, Statistics & Data Analysis, 51(10), 4942–4956.
21(1), 66–85. Hyndman, R. J., Zeng, Y., & Shang, H. L. (2021). Forecasting the old-age
Huang, C., Chen, S., Yang, S., & Kuo, C. (2015). One-day-ahead hourly dependency ratio to determine a sustainable pension age. Australian
forecasting for photovoltaic power generation using an intelligent & New Zealand Journal of Statistics, in press.
method with weather-based forecasting models. IET Generation, Hyppöla, J., Tunkelo, A., & Törnqvist, L. (Eds.), (1949). Suomen
Transmission and Distribution, 9(14), 1874–1882. väestöä, sen uusiutumista ja tulevaa kehitystä koskevia laskelmia.
Huang, T., Fildes, R., & Soopramanien, D. (2019). Forecasting retailer Tilastollisia tiedonantoja: vol. 38, Helsinki: Statistics Finland.
product sales in the presence of structural change. European Journal Ibrahim, R., & L’Ecuyer, P. (2013). Forecasting call center arrivals:
of Operational Research, 279(2), 459–470. Fixed-effects, mixed-effects, and bivariate models. Manufacturing &
Huang, J., Horowitz, J. L., & Wei, F. (2010). Variable selection in Service Operations Management, 15(1), 72–85.
nonparametric additive models. The Annals of Statistics, 38(4), Ibrahim, R., Ye, H., L’Ecuyer, P., & Shen, H. (2016). Modeling and
2282–2313. forecasting call center arrivals: A literature survey and a case study.
Huang, D., Jiang, F., Tu, J., & Zhou, G. (2015). Investor sentiment aligned: International Journal of Forecasting, 32(3), 865–874.
A powerful predictor of stock returns. Review of Financial Studies, IEA, P. (2020). Electricity information: Overview. URL www.iea.org/
28(3), 791–837. reports/electricity-information-overview.
Huard, D., Évin, G., & Favre, A.-C. (2006). BayesIan copula selection. IHME COVID-19 health service utilization forecasting team, & Mur-
Computational Statistics & Data Analysis, 51(2), 809–822. ray, C. J. L. (2020a). Forecasting COVID-19 impact on hospital
Hubáček, O., Šourek, G., & Železnỳ, F. (2019). Exploiting sports-betting bed-days, ICU-days, ventilator-days and deaths by US state in
market using machine learning. International Journal of Forecasting, the next 4 months. http://dx.doi.org/10.1101/2020.03.27.20043752,
35(2), 783–796. Medrxiv;2020.03.27.20043752v1.
Huber, J., & Stuckenschmidt, H. (2020). Daily retail demand forecasting IHME COVID-19 health service utilization forecasting team, & Mur-
using machine learning with emphasis on calendric special days. ray, C. J. L. (2020b). Forecasting the impact of the first wave of the
International Journal of Forecasting. COVID-19 pandemic on hospital demand and deaths for the USA
Huberty, M. (2015). Can we vote with our tweet? On the perennial and European economic area countries. http://dx.doi.org/10.1101/
difficulty of election forecasting with social media. International 2020.04.21.20074732, Medrxiv;2020.04.21.20074732v1.
Journal of Forecasting, 31(3), 992–1007. Ince, O. (2014). Forecasting exchange rates out-of-sample with panel
Hubicka, K., Marcjasz, G., & Weron, R. (2018). A note on averaging day- methods and real-time data. Journal of International Money and
ahead electricity price forecasts across calibration windows. IEEE Finance, 43(C), 1–18.
Transactions on Sustainable Energy, 10(1), 321–323. Inoue, A., Jin, L., & Rossi, B. (2017). Rolling window selection for
Hughes, M. C. (2001). Forecasting practice: organisational issues. Journal out-of-sample forecasting with time-varying parameters. Journal of
of the Operational Research Society, 52(2), 143–149. Econometrics, 196(1), 55–67.
849
Inoue, A., & Kilian, L. (2008). How useful is bagging in forecasting Joe, H. (2014). Dependence Modeling with Copulas. CRC Press.
economic time series? A case study of US consumer price inflation. Johansen, S., & Nielsen, B. (2009). An analysis of the indicator satu-
Journal of the American Statistical Association, 103(482), 511–522. ration estimator as a robust regression estimator. In J. Castle, &
ifo Institute (2020). ifo business climate index for Germany. Ifo In- N. Shephard (Eds.), The methodology and practice of econometrics: A
stitute, Accessed on 2020-09-07, https://www.ifo.de/en/survey/ifo- festschrift in honour of David F. Hendry (pp. 1–35). Oxford and New
business-climate-index. York: Oxford University Press.
Ioannidis, J. P. A., Cripps, S., & Tanner, M. A. (2020). Forecasting for Johnes, G. (1999). Forecasting unemployment. Applied Economics Letters,
COVID-19 has failed. International Journal of Forecasting. 6(9), 605–607.
Irwin, G. A., & Meeter, D. A. (1969). Building voter transition models Johnson, B. B., & Slovic, P. (1995). Presenting uncertainty in health risk
from aggregate data. Midwest Journal of Political Science, 13(4), assessment: Initial studies of its effects on risk perception and trust.
545–566. Risk Analysis, 15(4), 485–494.
Islam, T., & Meade, N. (2000). Modelling diffusion and replacement. Johnston, D. M. (2008). The historical foundations of world order: The
European Journal of Operational Research, 125(3), 551–570. tower and the arena. Martinus Nijhoff Publishers.
Ivanov, S., & Zhechev, V. (2012). Hotel revenue management – a critical Johnston, R., & Pattie, C. (2000). Ecological inference and entropy-
literature review. Tourism: An International Interdisciplinary Journal, maximizing: An alternative estimation procedure for split-ticket
60(2), 175–197. voting. Political Analysis, 8(4), 333–345.
Jacobs, J. P. A. M., & van Norden, S. (2011). Modeling data revi- Johnstone, D. J., Jose, V. R. R., & Winkler, R. L. (2011). Tailored scoring
sions: Measurement error and dynamics of ‘true’ values. Journal rules for probabilities. Decision Analysis, 8(4), 256–268.
of Econometrics, 161, 101–109. Joiner, T. A., Leveson, L., & Langfield-Smith, K. (2002). Technical lan-
James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013). An introduction guage, advice understandability, and perceptions of expertise and
to statistical learning with applications in R. New York, USA: Springer. trustworthiness: The case of the financial planner. Australian Journal
Jammazi, R., & Aloui, C. (2012). Crude oil price forecasting: Experi- of Management, 27(1), 25–43.
mental evidence from wavelet decomposition and neural network Jondeau, E. (2007). Financial modelling under non-Gaussian distributions
modeling. Energy Economics, 34(3), 828–841. (1st ed.). London: Springer.
Janczura, J., Trück, S., Weron, R., & Wolff, R. C. (2013). Identifying spikes Jongbloed, G., & Koole, G. (2001). Managing uncertainty in call centres
and seasonal components in electricity spot price data: A guide to using Poisson mixtures. Applied Stochastic Models in Business and
robust modeling. Energy Economics, 38, 96–110. Industry, 17(4), 307–318.
Janke, T., & Steinke, F. (2019). Forecasting the price distribution of
Jonung, L., & Larch, M. (2006). Improving fiscal policy in the EU: the
continuous intraday electricity trading. Energies, 12(22), 4262.
case for independent forecasts. Economic Policy, 21(47), 491–534.
Janssen, F. (2018). Advances in mortality forecasting: introduction.
Jordá, O., Knüppelc, M., & Marcellino, M. (2013). Empirical simulta-
Genus, 74(21).
neous prediction regions for path-forecasts. International Journal of
Januschowski, T., Arpin, D., Salinas, D., Flunkert, V., Gasthaus, J.,
Forecasting, 29(3), 456–468.
Stella, L., et al. (2018). Now available in Amazon SageMaker:
Jore, A. S., Mitchell, J., & Vahey, S. P. (2010). Combining forecast
DeepAR algorithm for more accurate time series forecasting.
densities from VARs with uncertain instabilities. Journal of Applied
In AWS machine learning blog. Accessed on 2020-09-01,
Econometrics, 25(4), 621–634.
https://aws.amazon.com/blogs/machine-learning/now-available-
Jose, V. R. R., Grushka-Cockayne, Y., & Lichtendahl, K. C. (2014).
in-amazon-sagemaker-deepar-algorithm-for-more-accurate-time-
Trimmed opinion pools and the crowd’s calibration problem.
series-forecasting/.
Management Science, 60(2), 463–475.
Januschowski, T., Gasthaus, J., Wang, Y., Rangapuram, S. S., & Callot, L.
Jose, V. R. R., Nau, R. F., & Winkler, R. L. (2008). Scoring rules,
(2018). Deep learning for forecasting: Current trends and chal-
generalized entropy, and utility maximization. Operations Research,
lenges. Foresight: The International Journal of Applied Forecasting, 51,
56(5), 1146–1157.
42–47.
Jose, V. R. R., & Winkler, R. L. (2008). Simple robust averages of fore-
Januschowski, T., Gasthaus, J., Wang, Y., Salinas, D., Flunkert, V., Bohlke-
casts: Some empirical results. International Journal of Forecasting,
Schneider, M., et al. (2020). Criteria for classifying forecasting
24(1), 163–169.
methods. International Journal of Forecasting, 36(1), 167–177.
Januschowski, T., & Kolassa, S. (2019). A classification of business Jose, V. R. R., & Winkler, R. L. (2009). Evaluating quantile assessments.
forecasting problems. Foresight: The International Journal of Applied Operations Research, 57(5), 1287–1297.
Forecasting, 52, 36–43. Joslyn, S. L., Nadav-Greenberg, L., Taing, M. U., & Nichols, R. M. (2009).
Jardine, A. K. S., Lin, D., & Banjevic, D. (2006). A review on ma- The effects of wording on the understanding and use of uncertainty
chinery diagnostics and prognostics implementing condition-based information in a threshold forecasting decision. Applied Cognitive
maintenance. Mechanical Systems and Signal Processing, 20(7), Psychology, 23(1), 55–72.
1483–1510. Joslyn, S. L., & Nichols, R. M. (2009). Probability or frequency? Express-
Jennings, W., Lewis-Beck, M., & Wlezien, C. (2020). Election forecasting: ing forecast uncertainty in public weather forecasts. Meteorological
Too far out? International Journal of Forecasting, 36(3), 949–962. Applications, 16(3), 309–314.
Jeon, J., Panagiotelis, A., & Petropoulos, F. (2019). Probabilistic forecast Julier, S. J., & Uhlmann, J. K. (1997). New extension of the Kalman filter
reconciliation with applications to wind power and electric load. to nonlinear systems. In I. Kadar (Ed.), 3068, Signal processing, sensor
European Journal of Operational Research. fusion, and target recognition VI (pp. 182–193). SPIE, International
Jeon, J., & Taylor, J. (2016). Short-term density forecasting of wave Society for Optics and Photonics.
energy using ARMA-GARCH models and kernel density estimation. Jung, R. C., & Tremayne, A. (2006). Coherent forecasting in integer time
International Journal of Forecasting, 32(3), 991–1004. series models. International Journal of Forecasting, 22(2), 223–238.
Jiang, J. J., Muhanna, W. A., & Pick, R. A. (1996). The impact of Kaastra, I., & Boyd, M. (1996). Designing a neural network for fore-
model performance history information on users’ confidence in de- casting financial and economic time series. Neurocomputing, 10(3),
cision models: An experimental examination. Computers in Human 215–236.
Behavior, 12(2), 193–207. Kaboudan, M. (2001). Compumetric forecasting of crude oil prices. In
Jiao, E. X., & Chen, J. L. (2019). Tourism forecasting: A review Proceedings of the 2001 congress on evolutionary computation (IEEE
of methodological developments over the last decade. Tourism Cat. No. 01TH8546) (pp. 283–287). IEEE.
Economics, 25(3), 469–492. Kagraoka, Y. (2016). Common dynamic factors in driving commod-
Jing, G., Cai, W., Chen, H., Zhai, D., Cui, C., & Yin, X. (2018). An air ity prices: Implications of a generalized dynamic factor model.
balancing method using support vector machine for a ventilation Economic Modelling, 52, 609–617.
system. Building and Environment, 143, 487–495. Kahn, K. B. (2002). An exploratory investigation of new product fore-
Joe, H. (1997). Multivariate models and dependence concepts. Chapman casting practices. Journal of Product Innovation Management, 19(2),
& Hall, London. 133–143.
Joe, H. (2005). Asymptotic efficiency of the two-stage estimation Kahneman, D. (2011). Thinking, fast and slow. London: Penguin books.
method for copula-based models. Journal of Multivariate Analysis, Kahneman, D., & Tversky, A. (1973). On the psychology of prediction.
94(2), 401–419. Psychological Review, 80(4), 237–251.
850
Kahneman, D., & Tversky, A. (1996). On the reality of cognitive illusions. Keyfitz, N. (1981). The limits of population forecasting. Population and
Psychological Review, 103(3), 582–91; discusion 592–6. Development Review, 7(4), 579–593.
Kalamara, E., Turrell, A., Redl, C., Kapetanios, G., & Kapadia, S. (2020). Khaldi, R., El Afia, A., & Chiheb, R. (2019). Forecasting of weekly patient
Making text count: economic forecasting using newspaper text: Tech. visits to emergency department: real case study. Procedia Computer
Rep. 865, Bank of England. Science, 148, 532–541.
Kalman, R. E. (1960). A new approach to linear filtering and prediction Kiesel, R., & Paraschiv, F. (2017). Econometric analysis of 15-minute
problems. Journal of Fluids Engineering, 82(1), 35–45. intraday electricity prices. Energy Economics, 64, 77–90.
Kamarianakis, Y., & Prastacos, P. (2005). Space–time modeling of traffic Kilian, L., & Inoue, A. (2004). Bagging time series models: Tech. Rep. 110,
flow. Computers & Geosciences, 31(2), 119–133. Econometric Society.
Kamisan, N. A. B., Lee, M. H., Suhartono, S., Hussin, A. G., & Zubairi, Y. Kilian, L., & Taylor, M. P. (2003). Why is it so difficult to beat the
Z. (2018). Load forecasting using combination model of multiple random walk forecast of exchange rates? Journal of International
linear regression with neural network for Malaysian city. Sains Economics, 60(1), 85–107.
Malaysiana, 47(2), 419–426. Kim, C.-J., Charles, K. C.-J. N., & Nelson, C. R. (1999). State-space models
Kang, Y. (2012). Real-time change detection in time series based
with regime switching: Classical and Gibbs-sampling approaches with
on growing feature quantization. In The 2012 international joint
applications. MIT Press.
conference on neural networks (IJCNN)(pp. 1–6).
Kim, T. Y., Dekker, R., & Heij, C. (2017). Spare part demand forecasting
Kang, Y., Belušić, D., & Smith-Miles, K. (2014). Detecting and classifying
for consumer goods using installed base information. Computers &
events in noisy time series. Journal of the Atmospheric Sciences,
Industrial Engineering, 103, 201–215.
71(3), 1090–1104.
Kim, S., Shephard, N., & Chib, S. (1998). Stochastic volatility: likelihood
Kang, Y., Belušić, D., & Smith-Miles, K. (2015). Classes of structures
inference and comparison with ARCH models. Review of Economic
in the stable atmospheric boundary layer. Quarterly Journal of the
Studies, 81, 361–393.
Royal Meteorological Society, 141(691), 2057–2069.
Kang, Y., Hyndman, R. J., & Li, F. (2020). GRATIS: GeneRAting time series Kim, H. H., & Swanson, N. R. (2014). Forecasting financial and macroe-
with diverse and controllable characteristics. Statistical Analysis and conomic variables using data reduction methods: New empirical
Data Mining, 13(4), 354–376. evidence. Journal of Econometrics, 178, 352–367.
Kang, Y., Hyndman, R. J., & Smith-Miles, K. (2017). Visualising fore- King, G. (1997). A solution to the ecological inference problem: re-
casting algorithm performance using time series instance spaces. constructing individual behavior from aggregate data. Princeton
International Journal of Forecasting, 33(2), 345–358. University Press.
Kang, S. H., Kang, S.-M., & Yoon, S.-M. (2009). Forecasting volatility of King, G., Rosen, O., & Tanner, M. A. (1999). Binomial-beta hierarchical
crude oil markets. Energy Economics, 31(1), 119–125. models for ecological inference. Sociological Methods & Research,
Kang, Y., Spiliotis, E., Petropoulos, F., Athiniotis, N., Li, F., & As- 28(1), 61–90.
simakopoulos, V. (2020). Déjà vu: A Data-centric forecasting King, G., Tanner, M. A., & Rosen, O. (2004). Ecological inference: New
approach through time series cross-similarity. Journal of Business methodological strategies. Cambridge University Press.
Research. Kingma, D. P., & Ba, J. (2015). Adam: A method for stochastic optimiza-
Kapetanios, G., Mitchell, J., Price, S., & Fawcett, N. (2015). Gener- tion. San Diego, Third Annual International Conference on Learning
alised density forecast combinations. Journal of Econometrics, 188(1), Representations.
150–165. Kishor, N. K., & Koenig, E. F. (2012). VAR estimation and forecasting
Kargin, V., & Onatski, A. (2008). Curve forecasting by functional when data are subject to revision. Journal of Business & Economic
autoregression. Journal of Multivariate Analysis, 99(10), 2508–2526. Statistics, 30(2), 181–190.
Karniouchina, E. V. (2011). Are virtual markets efficient predictors of Kittichotsatsawat, Y., Jangkrajarng, V., & Tippayawong, K. Y. (2021).
new product success? The case of the Hollywood stock exchange. Enhancing coffee supply chain towards sustainable growth with
The Journal of Product Innovation Management, 28(4), 470–484. big data and modern agricultural technologies. Sustainability, 13(8),
Kascha, C., & Ravazzolo, F. (2010). Combining inflation density forecasts. 4593.
Journal of Forecasting, 29(1–2), 231–250. Klepsch, J., & Klüppelberg, C. (2017). An innovations algorithm for
Katz, R. W., & Lazo, J. K. (Eds.), (2011). The Oxford Handbook of Economic the prediction of functional linear processes. Journal of Multivariate
Forecasting, Economic value of weather and climate forecasts. Oxford Analysis, 155, 252–271.
University Press. Klepsch, J., Klüppelberg, C., & Wei, T. (2017). Prediction of functional
Kaufmann, R., & Juselius, K. (2013). Testing hypotheses about glacial ARMA processes with an application to traffic data. Econometrics
cycles against the observational record. Paleoceanography, 28, and Statistics, 1, 128–149.
175–184. Klima, A., Schlesinger, T., Thurner, P. W., & Küchenhoff, H. (2019).
Kayacan, E., Ulutas, B., & Kaynak, O. (2010). Grey system theory-based
Combining aggregate data and exit polls for the estimation of voter
models in time series prediction. Expert Systems with Applications,
transitions. Sociological Methods & Research, 48(2), 296–325.
37(2), 1784–1789.
Klima, A., Thurner, P. W., Molnar, C., Schlesinger, T., & Küchen-
Keane, M. P., & Runkle, D. E. (1990). Testing the rationality of price fore-
hoff, H. (2016). Estimation of voter transitions based on ecological
casts: new evidence from panel data. American Economic Review,
inference. AStA. Advances in Statistical Analysis, 2, 133–159.
80(4), 714–735.
Kline, D. M. (2004). Methods for multi-step time series forecasting with
Kedia, S., & Williams, C. (2003). Predictors of substance abuse treatment
neural networks. In G. P. Zhang (Ed.), Neural networks in business
outcomes in Tennessee. Journal of Drug Education, 33(1), 25–47.
forecasting (pp. 226–250). Information Science Publishing.
Kehagias, A., & Petridis, V. (1997). Time-series segmentation using
predictive modular neural networks. Neural Computation, 9(8), Klofstad, C. A., & Bishin, B. G. (2012). Exit and entrance polling:
1691–1709. A comparison of election survey methods. Field Methods, 24(4),
Keiding, N., & Hoem, J. M. (1976). Stochastic stable population theory 429–437.
with continuous time. I. Scandinavian Actuarial Journal, 1976(3), Knudsen, C., McNown, R., & Rogers, A. (1993). Forecasting fertility:
150–175. An application of time series methods to parameterized model
Kelle, P., & Silver, E. A. (1989). Forecasting the returns of reusable schedules. Social Science Research, 22(1), 1–23.
containers. Journal of Operations Management, 8(1), 17–35. Koenig, E. F., Dolmas, S., & Piger, J. (2003). The use and abuse of real-
Kelly, B., & Pruitt, S. (2013). Market expectations in the cross-section time data in economic forecasting. The Review of Economics and
of present values. The Journal of Finance, 68(5), 1721–1756. Statistics, 85(3), 618–628.
Kennedy, W. J., Wayne Patterson, J., & Fredendall, L. D. (2002). Koenker, R. (2005). Econometric Society Monographs, Quantile regression.
An overview of recent literature on spare parts inventories. Cambridge University Press.
International Journal of Production Economics, 76(2), 201–215. Koh, Y.-M., Spindler, R., Sandgren, M., & Jiang, J. (2018). A model com-
Kennedy, R., Wojcik, S., & Lazer, D. (2017). Improving election parison algorithm for increased forecast accuracy of dengue fever
prediction internationally. Science, 355(6324), 515–520. incidence in Singapore and the auxiliary role of total precipitation
Keyfitz, N. (1972). On future population. Journal of the American information. International Journal of Environmental Health Research,
Statistical Association, 67(338), 347–363. 28(5), 535–552.
851
Koirala, K. H., Mishra, A. K., D’Antoni, J. M., & Mehlhorn, J. E. (2015). En- Krishnan, T., Bass, F., & Kummar, V. (2000). Impact of a late entrant
ergy prices and agricultural commodity prices: Testing correlation on the diffusion of a new product/service. Journal of Marketing
using copulas method. Energy, 81, 430–436. Research, 37, 269–278.
Kokoszka, P., & Reimherr, M. (2013). Determining the order of the Krüger, E., & Givoni, B. (2004). Predicting thermal performance in
functional autoregressive model. Journal of Time Series Analysis, occupied dwellings. Energy and Buildings, 36(3), 301–307.
34(1), 116–129. Krzysztofowicz, R. (1999). BayesIan theory of probabilistic forecasting
Kokoszka, P., Rice, G., & Shang, H. L. (2017). Inference for the via deterministic hydrologic model. Water Resources Research, 35(9),
autocovariance of a functional time series under conditional 2739–2750.
heteroscedasticity. Journal of Multivariate Analysis, 162, 32–50. Krzysztofowicz, R. (2014). Probabilistic flood forecast: Exact and
Kolasa, M., & Rubaszek, M. (2015a). Forecasting using DSGE models approximate predictive distributions. Journal of Hydrology, 517,
with financial frictions. International Journal of Forecasting, 31(1), 643–651.
1–19. Kück, M., Crone, S. F., & Freitag, M. (2016). Meta-learning with neural
Kolasa, M., & Rubaszek, M. (2015b). How frequently should we reesti- networks and landmarking for forecasting model selection an em-
mate DSGE models? International Journal of Central Banking, 11(4), pirical evaluation of different feature sets applied to industry data.
279–305. In 2016 international joint conference on neural networks (IJCNN) (pp.
Kolasa, M., Rubaszek, M., & Skrzypczynśki, P. (2012). Putting the New 1499–1506). IEEE.
Keynesian DSGE model to the real-time forecasting test. Journal of Kuhn, M., & Johnson, K. (2019). Feature engineering and selection. Taylor
Money, Credit and Banking, 44(7), 1301–1324. & Francis Ltd.
Kolassa, S. (2011). Combining exponential smoothing forecasts using Kulakov, S. (2020). X-model: further development and possible
akaike weights. International Journal of Forecasting, 27(2), 238–251. modifications. Forecasting, 2(1), 20–35.
Kolassa, S. (2016). Evaluating predictive count data distributions in Kulakov, S., & Ziel, F. (2021). The impact of renewable energy
retail sales forecasting. International Journal of Forecasting, 32(3), forecasts on intraday electricity prices. Economics of Energy and
788–803. Environmental Policy, 10, 79–104.
Kolassa, S. (2020a). Quality measure for predictive Highest Density Kulkarni, G., Kannan, P. K., & Moe, W. (2012). Using online search
Regions. Cross Validated, Accessed on 2020-08-20, URL https:// data to forecast new product sales. Decision Support Systems, 52(3),
stats.stackexchange.com/q/483878. 604–611.
Kolassa, S. (2020b). Why the ‘‘best’’ point forecast depends on the Kumar, D. (2015). Sudden changes in extreme value volatility estima-
error or accuracy measure. International Journal of Forecasting, 36(1), tor: Modeling and forecasting with economic significance analysis.
208–211. Economic Modelling, 49, 354–371.
Kolassa, S. (2020c). Will deep and machine learning solve our fore- Künsch, H. R. (1989). The jackknife and the bootstrap for general
casting problems? Foresight: The International Journal of Applied stationary observations. The Annals of Statistics, 17(3), 1217–1241.
Forecasting, 57, 13–18. Kupiszewski, M., & Kupiszewska, D. (2011). MULTIPOLES: A Revised
Kolassa, S., & Siemsen, E. (2016). Demand forecasting for managers. multiregional model for improved capture of international migra-
Business Expert Press. tion. In J. Stillwell, & M. Clarke (Eds.), Population dynamics and
Kon Kam King, G., Canale, A., & Ruggiero, M. (2019). BayesIan func- projection methods (pp. 41–60). Dordrecht: Springer Netherlands.
tional forecasting with locally-autoregressive dependent processes. Kuster, C., Rezgui, Y., & Mourshed, M. (2017). Electrical load forecasting
Bayesian Analysis, 14(4), 1121–1141. models: A critical systematic review. Sustainable Cities and Society,
Koning, A. J., Franses, P. H., Hibon, M., & Stekler, H. O. (2005). The M3 35, 257–270.
competition: Statistical tests of the results. International Journal of Kusters, U., McCullough, B., & Bell, M. (2006). Forecasting software:
Forecasting, 21(3), 397–409. Past, present and future. International Journal of Forecasting, 22(3),
Koop, G. M. (2003). Bayesian econometrics. John Wiley & Sons Inc.. 599–615.
Koop, G., & Korobilis, D. (2018). Variational Bayes inference in Kwiatkowski, D., Phillips, P. C. B., Schmidt, P., & Shin, Y. (1992). Testing
high-dimensional time-varying parameter models. Journal of the null hypothesis of stationarity against the alternative of a unit
Econometrics. root: How sure are we that economic time series have a unit
Koop, G., & Potter, S. M. (1999). Dynamic asymmetries in U.S. root? Journal of Econometrics, 54(1), 159–178.
unemployment. Journal of Business & Economic Statistics, 17(3), Kyriazi, F., Thomakos, D. D., & Guerard, J. B. (2019). Adaptive learning
298–312. forecasting, with applications in forecasting agricultural prices.
Kostenko, A. V., & Hyndman, R. J. (2006). A note on the categorization of International Journal of Forecasting, 35(4), 1356–1369.
demand patterns. Journal of the Operational Research Society, 57(10), La Scalia, G., Micale, R., Miglietta, P. P., & Toma, P. (2019). Re-
1256–1257. ducing waste and ecological impacts through a sustainable and
Kotchoni, R., Leroux, M., & Stevanovic, D. (2019). Macroeconomic efficient management of perishable food based on the Monte Carlo
forecast accuracy in a data-rich environment. Journal of Applied simulation. Ecological Indicators, 97, 363–371.
Econometrics, 34(7), 1050–1072. Labarere, J., Bertrand, R., & Fine, M. J. (2014). How to derive and validate
Kourentzes, N., & Athanasopoulos, G. (2019). Cross-temporal coherent clinical prediction models for use in intensive care medicine.
forecasts for Australian tourism. Annals of Tourism Research, 75, Intensive Care Medicine, 40(4), 513–527.
393–409. Ladiray, D., & Quenneville, B. (2001). Lecture notes in statistics 158,
Kourentzes, N., & Athanasopoulos, G. (2020). Elucidate structure Seasonal adjustment with the X-11 method. New York, USA: Springer.
in intermittent demand series. European Journal of Operational Lago, J., De Ridder, F., & De Schutter, B. (2018). Forecasting spot elec-
Research. tricity prices: Deep learning approaches and empirical comparison
Kourentzes, N., Barrow, D., & Petropoulos, F. (2019). Another look at of traditional algorithms. Applied Energy, 221, 386–405.
forecast selection and combination: Evidence from forecast pooling. Lahiri, S. K., & Lahiri, N. (2003). Resampling methods for dependent data
International Journal of Production Economics, 209, 226–235. (Springer series in statistics). Springer.
Kourentzes, N., & Petropoulos, F. (2016). Forecasting with multivari- Lai, G., Chang, W.-C., Yang, Y., & Liu, H. (2018. Modeling long-and short-
ate temporal aggregation: The case of promotional modelling. term temporal patterns with deep neural networks. In The 41st
International Journal of Production Economics, 181, Part A, 145–153. International ACM SIGIR Conference on Research & Development in
Kourentzes, N., Petropoulos, F., & Trapero, J. R. (2014). Improving Information Retrieval (pp. 95–104).
forecasting by estimating time series structural components across Landon, J., Ruggeri, F., Soyer, R., & Tarimcilar, M. M. (2010). Modeling
multiple frequencies. International Journal of Forecasting, 30(2), latent sources in call center arrival data. European Journal of
291–302. Operational Research, 204(3), 597–603.
Kourentzes, N., Rostami-Tabar, B., & Barrow, D. K. (2017). Demand Lanne, M., & Saikkonen, P. (2003). Modeling the U.S. short-term inter-
forecasting by temporal aggregation: Using optimal or multiple est rate by mixture autoregressive processes. Journal of Financial
aggregation levels? Journal of Business Research, 78, 1–9. Econometrics, 1(1), 96–125.
Kovalchik, S., & Reid, M. (2019). A calibration method with dynamic Larrick, R. P., & Soll, J. B. (2006). Intuitions about combining opinions:
updates for within-match forecasting of wins in tennis. International Misappreciation of the averaging principle. Management Science,
Journal of Forecasting, 35(2), 756–766. 52(1), 111–127.
852
Larson, P. D., Simchi-Levi, D., Kaminsky, P., & Simchi-Levi, E. (2001). Leslie, P. H. (1948). Some further notes on the use of matrices in
Designing and managing the supply chain: Concepts, strategies, and population mathematics. Biometrika, 35(3/4), 213–245.
case studies. Journal of Business Logistics, 22(1), 259–261. Leuenberger, D., Haefele, A., Omanovic, N., Fengler, M., Martucci, G.,
Law, R., Li, G., Fong, D. K. C., & Han, X. (2019). Tourism demand Calpini, B., et al. (2020). Improving high-impact numerical weather
forecasting: A deep learning approach. Annals of Tourism Research, prediction with lidar and drone observations. Bulletin of the
75, 410–423. American Meteorological Society, 101(7), E1036–E1051.
Lawrence, M. (2000). What does it take to achieve adoption in sales Leva, S., Mussetta, M., & Ogliari, E. (2019). PV module fault diagnosis
forecasting? International Journal of Forecasting, 16(2), 147–148. based on microconverters and day-ahead forecast. IEEE Transactions
Lawrence, M., Goodwin, P., & Fildes, R. (2002). Influence of user on Industrial Electronics, 66(5), 3928–3937.
participation on DSS use and decision accuracy. Omega, 30(5), Levine, R., Pickett, J., Sekhri, N., & Yadav, P. (2008). Demand forecasting
381–392. for essential medical technologies. American Journal of Law &
Lawrence, M., Goodwin, P., O’Connor, M., & Önkal, D. (2006). Judg- Medicine, 34(2–3), 225–255.
mental forecasting: A review of progress over the last 25 years. Lewellen, J. (2015). The cross-section of expected stock returns. Critical
International Journal of Forecasting, 22(3), 493–518. Finance Review, 4(1), 1–44.
Lewis, B., Herbert, R., & Bell, R. (2003). The application of fourier
Lawrence, M., & Makridakis, S. (1989). Factors affecting judgmental
analysis to forecasting the inbound call time series of a call
forecasts and confidence intervals. Organizational Behavior and
centre. In Proceedings of the international congress on modeling
Human Decision Processes, 43(2), 172–187.
and simulation (MODSIM03); Townsville, Australia (pp. 1281–1286).
Lawrence, M., & O’Connor, M. (1992). Exploring judgemental
Citeseer.
forecasting. International Journal of Forecasting, 8(1), 15–26.
Lewis-Beck, M. S. (2005). Election forecasting: Principles and practice.
Layard, R., Nickell, S. J., & Jackman, R. (1991). Unemployment, macroeco-
British Journal of Politics and International Relations, 7(2), 145–164.
nomic performance and the labour market. Oxford: Oxford University
L’heureux, A., Grolinger, K., Elyamany, H. F., & Capretz, M. A. (2017).
Press.
Machine learning with big data: Challenges and approaches. IEEE
Le, Q., & Mikolov, T. (2014). Distributed representations of sentences
Access, 5, 7776–7797.
and documents. In International conference on machine learning (pp. Li, D. X. (2000). On default correlation: A copula function approach.
1188–1196). The Journal of Fixed Income, 9(4), 43–54.
Leadbetter, M. R. (1991). On a basis for ‘Peaks over Threshold’ Li, J. S.-H., & Chan, W.-S. (2011). Time-simultaneous prediction bands:
modeling. Statistics & Probability Letters, 12(4), 357–362. A new look at the uncertainty involved in forecasting mortality.
Leal, T., Pérez, J. J., Tujula, M., & Vidal, J. P. (2008). Fiscal forecast- Insurance: Mathematics & Economics, 49(1), 81–88.
ing: Lessons from the literature and challenges. Fiscal Studies, 29, Li, W., Han, Z., & Li, F. (2008). Clustering analysis of power load fore-
347–386. casting based on improved ant colony algorithm. In 2008 7th world
Ledolter, J. (1989). The effect of additive outliers on the forecasts from congress on intelligent control and automation (pp. 7492–7495).
ARIMA models. International Journal of Forecasting, 5(2), 231–240. Li, F., & He, Z. (2019). Credit risk clustering in a business group: which
Ledolter, J. (1991). Outliers in time series analysis: Some comments on matters more, systematic or idiosyncratic risk? Cogent Economics &
their impact and their detection. Image. Finance, Article 1632528.
Lee, A. (1990). Airline reservations forecasting: Probabilistic and statistical Li, H., & Hong, Y. (2011). Financial volatility forecasting with range-
models of the booking process: Tech. rep., Cambridge, Mass.: Flight based autoregressive volatility model. Finance Research Letters, 8(2),
Transportation Laboratory, Dept. of Aeronautics and Astronautics, 69–76.
Massachusetts Institute of Technology. Li, M., Huang, L., & Gong, L. (2011). Research on the challenges and
Lee, R. D. (1993). Modeling and forecasting the time series of US solutions of design large-scale call center intelligent scheduling
fertility: Age distribution, range, and ultimate level. International system. Procedia Engineering, 15, 2359–2363.
Journal of Forecasting, 9(2), 187–202. Li, G., & Jiao, E. (2020). Tourism forecasting research: a perspective
Lee, K. L., & Billings, S. a. (2003). A new direct approach of computing article. Tourism Review.
multi-step ahead predictions for non-linear models. International Li, F., & Kang, Y. (2018). Improving forecasting performance us-
Journal of Control, 76(8), 810–822. ing covariate-dependent copula models. International Journal of
Lee, R. D., & Carter, L. R. (1992). Modeling and forecasting US mortality. Forecasting, 34(3), 456–476.
Journal of the American Statistical Association, 87(419), 659–671. Li, X., Kang, Y., & Li, F. (2020). Forecasting with time series imaging.
Lee, W. Y., Goodwin, P., Fildes, R., Nikolopoulos, K., & Lawrence, M. Expert Systems with Applications, 160, Article 113680.
(2007). Providing support for the use of analogies in de- Li, J., Li, G., Liu, M., Zhu, X., & Wei, L. (2020). A novel text-based
mand forecasting tasks. International Journal of Forecasting, 23(3), framework for forecasting agricultural futures using massive online
377–390. news headlines. International Journal of Forecasting.
Li, J., Liao, Z., & Quaedvlieg, R. (2020). Conditional superior predictive
Lee, C.-Y., & Huh, S.-Y. (2017a). Forecasting new and renewable energy
ability. SSRN:3536461.
supply through a bottom-up approach: The case of South Korea.
Li, L., Noorian, F., Moss, D. J., & Leong, P. H. (2014). Rolling window time
Renewable and Sustainable Energy Reviews, 69, 207–217.
series prediction using MapReduce. In Proceedings of the 2014 IEEE
Lee, C.-Y., & Huh, S.-Y. (2017b). Forecasting the diffusion of renewable
15th international conference on information reuse and integration
electricity considering the impact of policy and oil prices: The case
(IEEE IRI 2014) (pp. 757–764). IEEE.
of South Korea. Applied Energy, 197, 29–39.
Li, D., Robinson, P. M., & Shang, H. L. (2020d). Nonstationary fractionally
Lee, J., Milesi-Ferretti, G. M., & Ricci, L. A. (2013). Real exchange rates
integrated functional time series: Working paper, University of York.
and fundamentals: A cross-country perspective. Journal of Money, Li, D., Robinson, P. M., & Shang, H. L. (2020e). Long-range dependent
Credit and Banking, 45(5), 845–865. curve time series. Journal of the American Statistical Association,
Lee, H. L., Padmanabhan, V., & Whang, S. (2004). Information distortion 115(530), 957–971.
in a supply chain: The Bullwhip effect. Management Science, 50, Li, D., Robinson, P. M., & Shang, H. L. (2021). Local whittle estimation
1875–1886. of long range dependence for functional time series. Journal of Time
Leigh, C., Alsibai, O., Hyndman, R. J., Kandanaarachchi, S., King, O. C., Series Analysis, In Press.
McGree, J. M., et al. (2019). A framework for automated anomaly Li, G., Song, H., & Witt, S. F. (2005). Recent developments in econo-
detection in high frequency water-quality data from in situ sensors. metric modeling and forecasting. Journal of Travel Research, 44(1),
Science of the Total Environment, 664, 885–898. 82–99.
Lemke, C., & Gabrys, B. (2010). Meta-learning for time series forecasting Liang, Y., He, D., & Chen, D. (2019). Poisoning attack on load forecasting.
and forecast combination. Neurocomputing, 73(10–12), 2006–2016. In 2019 IEEE innovative smart grid technologies-Asia (ISGT Asia) (pp.
Lerch, S., Baran, S., Möller, A., Groß, J., Schefzik, R., Hemri, S., et 1230–1235). IEEE.
al. (2020). Simulation-based comparison of multivariate ensemble Liberty, E., Karnin, Z., Xiang, B., Rouesnel, L., Coskun, B., Nallapati, R.,
post-processing methods. Nonlinear Processes in Geophysics, 27(2), et al. (2020). Elastic machine learning algorithms in amazon
349–371. SageMaker. In SIGMOD ’20, Proceedings of the 2020 International
Leslie, P. H. (1945). On the use of matrices in certain population Conference on Management of Data (pp. 731–737). New York, NY,
mathematics. Biometrika, 33(3), 183–212. USA: ACM.
853
Lichtendahl, K. C., Grushka-Cockayne, Y., & Winkler, R. L. (2013). Is it López, C., Zhong, W., & Zheng, M. (2017). Short-term electric load
better to average probabilities or quantiles? Management Science, forecasting based on wavelet neural network, particle swarm op-
59(7), 1594–1611. timization and ensemble empirical mode decomposition. Energy
Lichtendahl Jr, K. C., & Winkler, R. L. (2020). Why do some com- Procedia, 105, 3677–3682.
binations perform better than others? International Journal of López Cabrera, B., & Schulz, F. (2016). Volatility linkages between
Forecasting, 36(1), 142–149. energy and agricultural commodity prices. Energy Economics, 54(C),
Lildholdt, P. M. (2002). Centre for Analytical Finance, Estimation of GARCH 190–203.
models based on open, close, high, and low prices. Aarhus School of López-Ruiz, A., Bergillos, R. J., & Ortega-Sánchez, M. (2016). The
Business, Centre for Analytical Finance, Aarhus School of Business. importance of wave climate forecasting on the decision-making
Lim, J. S., & O’Connor, M. (1996a). Judgmental forecasting with inter- process for nearshore wave energy exploitation. Applied Energy, 182,
active forecasting support systems. Decision Support Systems, 16(4), 191–203.
339–357. Lopez-Suarez, C. F., & Rodriguez-Lopez, J. A. (2011). Nonlinear exchange
Lim, J. S., & O’Connor, M. (1996b). Judgmental forecasting with inter- rate predictability. Journal of International Money and Finance, 30(5),
active forecasting support systems. Decision Support Systems, 16(4), 877–895.
339–357. Lothian, J. R., & Taylor, M. P. (1996). Real exchange rate behavior: The
Lim, J. S., & O’Connor, M. (1996c). Judgmental forecasting with time recent float from the perspective of the past two centuries. Journal
series and causal information. International Journal of Forecasting, of Political Economy, 104(3), 488–509.
12(1), 139–153.
Lotka, A. J. (1907). Relation between birth rates and death rates. Science,
Limaye, V. S., Vargo, J., Harkey, M., Holloway, T., & Patz, J. A. (2018).
26(653), 21–22.
Climate change and heat-related excess mortality in the eastern
Lotka, A. (1920). Undamped oscillations derived from the law of mass
USA. EcoHealth, 15(3), 485–496.
action. Journal of the American Chemical Society, 42, 1595–1599.
Lin, E. M., Chen, C. W., & Gerlach, R. (2012). Forecasting volatility with
Lotka, A. J. (1925). Elements of physical biology. Williams & Wilkins.
asymmetric smooth transition dynamic range models. International
Journal of Forecasting, 28(2), 384–399. Lovins, J. (1968). Development of a stemming algorithm. Mechanical
Lin, J. L., & Granger, C. (1994). Forecasting from non-linear models in Translation and Computational Linguistics, 11(1–2), 22–31.
practice. Journal of Forecasting, 13, 1–9. Lowe, R., Bailey, T. C., Stephenson, D. B., Graham, R. J., Coelho, C. A.,
Lin, C.-F. J., & Teräsvirta, T. (1994). Testing the constancy of regres- Carvalho, M. S., et al. (2011). Spatio-temporal modelling of climate-
sion parameters against continuous structural change. Journal of sensitive disease risk: Towards an early warning system for dengue
Econometrics, 62(2), 211–228. in Brazil. Computers & Geosciences, 37(3), 371–381.
Ling, S. (1999). On the probabilistic properties of a double thresh- Lu, S.-L. (2019). Integrating heuristic time series with modified grey
old ARMA conditional heteroskedastic model. Journal of Applied forecasting for renewable energy in Taiwan. Renewable Energy, 133,
Probability, 36(3), 688–705. 1436–1444.
Ling, S., Tong, H., & Li, D. (2007). Ergodicity and invertibility of Lu, Y. (2021). The predictive distributions of thinning-based count
threshold moving-average models. Bernoulli, 13(1), 161–168. processes. Scandinavian Journal of Statistics, 48(1), 42–67.
Linnér, L., Eriksson, I., Persson, M., & Wettermark, B. (2020). Forecast- Lu, H., Azimi, M., & Iseley, T. (2019). Short-term load forecasting
ing drug utilization and expenditure: ten years of experience in of urban gas using a hybrid model based on improved fruit fly
stockholm. BMC Health Services Research, 20, 1–11. optimization algorithm and support vector machine. Energy Reports,
Litsiou, K., Polychronakis, Y., Karami, A., & Nikolopoulos, K. (2019). 5, 666–677.
Relative performance of judgmental methods for forecasting the Lübbers, J., & Posch, P. N. (2016). Commodities’ common factor: An
success of megaprojects. International Journal of Forecasting. empirical assessment of the markets’ drivers. Journal of Commodity
Liu, Y. (2005). Value-at-Risk Model Combination Using Artificial Neural Markets, 4(1), 28–40.
Networks. Ermory University Working Papers. Lucas, R. E. (1976). Econometric policy evaluation: A critique.
Liu, L., & Wu, L. (2021). Forecasting the renewable energy consumption Carnegie-Rochester Conference Series on Public Policy, 1, 19–46.
of the European countries by an adjacent non-homogeneous grey Lucas, A., Schwaab, B., & Zhang, X. (2014). Conditional euro area
model. Applied Mathematical Modelling, 89, 1932–1948. sovereign default risk. Journal of Business & Economic Statistics,
Liu, W., Zhu, F., Zhao, T., Wang, H., Lei, X., Zhong, P.-A., et al. (2020). 32(2), 271–284.
Optimal stochastic scheduling of hydropower-based compensation Ludvigson, S. C., & Ng, S. (2007). The empirical risk-return relation:
for combined wind and photovoltaic power outputs. Applied Energy, A factor analysis approach. Journal of Financial Economics, 83(1),
276, Article 115501. 171–222.
Ljung, G. M., & Box, G. E. (1978). On a measure of lack of fit in time Luo, J., Hong, T., & Fang, S.-C. (2018a). Benchmarking robustness of
series models. Biometrika, 65(2), 297–303. load forecasting models under data integrity attacks. International
Loaiza-Maya, R., Martin, G. M., & Frazier, D. T. (2020). Focused Bayesian Journal of Forecasting, 34(1), 89–104.
prediction. Journal of Applied Econometrics.
Luo, J., Hong, T., & Fang, S.-C. (2018b). Robust regression models for
Loaiza-Maya, R., & Smith, M. S. (2020). Real-time macroeconomic fore-
load forecasting. IEEE Transactions on Smart Grid, 10(5), 5397–5404.
casting with a heteroscedastic inversion copula. Journal of Business
Luo, J., Hong, T., & Yue, M. (2018). Real-time anomaly detection for
& Economic Statistics, 38(2), 470–486.
very short-term load forecasting. Journal of Modern Power Systems
Loaiza-Maya, R., Smith, M. S., Nott, D. J., & Danaher, P. J. (2020). Fast
and Clean Energy, 6(2), 235–243.
and accurate variational inference for models with many latent
Luo, J., Klein, T., Ji, Q., & Hou, C. (2019). Forecasting realized volatility
variables. ArXiv:2005.07430.
Locarek-Junge, H., & Prinzler, R. (1998). Estimating value-at-risk using of agricultural commodity futures with infinite hidden Markov HAR
neural networks. In Informationssysteme in der finanzwirtschaft (pp. models. International Journal of Forecasting.
385–397). Springer Berlin Heidelberg. Lütkepohl, H. (2005). New introduction to multiple time series analysis.
Logg, J. M., Minson, J. A., & Moore, D. A. (2019). Algorithm apprecia- Springer, Berlin, Heidelberg.
tion: People prefer algorithmic to human judgment. Organizational Lütkepohl, H. (2011). Forecasting nonlinear aggregates and aggregates
Behavior and Human Decision Processes, 151, 90–103. with time-varying weights. JahrbÜCher FÜR NationalÖKonomie Und
Lohmann, T., Hering, A. S., & Rebennack, S. (2016). Spatio-temporal Statistik, 231(1), 107–133.
hydro forecasting of multireservoir inflows for hydro-thermal Lutz, W., Butz, W. P., & Samir, K. C. (2017). World population and human
scheduling. European Journal of Operational Research, 255(1), capital in the twenty-first century: An overview. Oxford University
243–258. Press.
Loper, E., & Bird, S. (2002). NLTK: The natural language toolkit. ArXiv: Lux, T. (2008). The Markov-switching multifractal model of asset
Cs/0205028. returns. Journal of Business & Economic Statistics, 26(2), 194–210.
López, M., Valero, S., Senabre, C., Aparicio, J., & Gabaldon, A. (2012). Lynn, G. S., Schnaars, S. P., & Skov, R. B. (1999). A survey of new
Application of SOM neural networks to short-term load forecasting: product forecasting practices in industrial high technology and
The spanish electricity market case study. Electric Power Systems low technology businesses. Industrial Marketing Management, 28(6),
Research, 91, 18–27. 565–571.
854
Ma, S. (2021). A hybrid deep meta-ensemble networks with application Makridakis, S., Spiliotis, E., & Assimakopoulos, V. (2021). The M5 ac-
in electric utility industry load forecasting. Information Sciences, 544, curacy competition: Results, findings and conclusions. International
183–196. Journal of Forecasting.
Ma, F., Chitta, R., Zhou, J., You, Q., Sun, T., & Gao, J. (2017). Makridakis, S., Spiliotis, E., Assimakopoulos, V., Chen, Z., Winkler, R. L.,
Ma, S., & Fildes, R. (2017). A retail store SKU promotions optimization et al. (2021). The M5 uncertainty competition: Results, findings and
model for category multi-period profit maximization. European conclusions. International Journal of Forecasting.
Journal of Operational Research, 260(2), 680–692. Makridakis, S., & Winkler, R. L. (1989). Sampling distributions of post-
Ma, S., Fildes, R., & Huang, T. (2016). Demand forecasting with high sample forecasting errors. Journal of the Royal Statistical Society.
dimensional data: The case of SKU retail sales forecasting with Series C. Applied Statistics, 38(2), 331–342.
intra-and inter-category promotional information. European Journal Mamdani, E. H., & Assilian, S. (1975). An experiment in linguistic
of Operational Research, 249(1), 245–257. synthesis with a fuzzy logic controller. International Journal of
Macaulay, F. R. (1931). The smoothing of time series. In NBER Books. Man-Machine Studies, 7, 1–15.
National Bureau of Economic Research, Inc. Mandal, P., Madhira, S. T. S., Haque, A. U., Meng, J., & Pineda, R.
MacDonald, R. (1998). What determines real exchange rates? The L. (2012). Forecasting power output of solar photovoltaic sys-
long and the short of it. Journal of International Financial Markets, tem using wavelet transform and artificial intelligence techniques.
Institutions and Money, 8(2), 117–153. Procedia Computer Science, 12, 332–337.
MacDonald, R., & Marsh, I. W. (1994). Combining exchange rate Mandelbrot, B. (1963). The variation of certain speculative prices.
forecasts: What is the optimal consensus measure? Journal of Journal of Business, 36(4), 394.
Forecasting, 13(3), 313–332. Mandelbrot, B. B. (1983). The fractal geometry of nature. Henry Holt and
Madaus, L., McDermott, P., Hacker, J., & Pullen, J. (2020). Hyper- Company.
local, efficient extreme heat projection and analysis using machine Manders, A., Schaap, M., & Hoogerbrugge, R. (2009). Testing the
learning to augment a hybrid dynamical-statistical downscaling capability of the chemistry transport model LOTOS-EUROS to fore-
technique. Urban Climate, 32, Article 100606. cast PM10 levels in the Netherlands. Atmospheric Enviroment, 46,
Maddix, D. C., Wang, Y., & Smola, A. (2018). Deep factors with Gaussian 4050–4059.
processes for forecasting. ArXiv:1812.00098. Mangina, E., & Vlachos, I. P. (2005). The changing role of information
Madhavan, P., & Wiegmann, D. A. (2007). Similarities and differ- technology in food and beverage logistics management: beverage
ences between human–human and human–automation trust: an network optimisation using intelligent agent technology. Journal of
integrative review. Theoretical Issues in Ergonomics Science, 8(4), Food Engineering, 70(3), 403–420.
277–301. Mankiw, N. G., & Reis, R. (2002). Sticky information versus sticky prices:
Magdon-Ismail, M., & Atiya, A. F. (2003). A maximum likelihood A proposal to replace the New Keynesian Phillips curve. Quarterly
approach to volatility estimation for a Brownian motion using high, Journal of Economics, 117, 1295–1328.
low and close price data. Quantitative Finance, 3(5), 376–384.
Mankiw, N. G., Reis, R., & Wolfers, J. (2003). Disagreement about
Mahajan, V., Muller, E., & Bass, F. (1990). New product diffusion models
inflation expectations. Tech. rep., Cambridge MA: National Bureau
in marketing: a review and directions of future research. Journal of
of Economic Research.
Marketing, 54, 1–26.
Mann, M. (2018). Have wars and violence declined? Theory and Society,
Maheu, J. M., & Yang, Q. (2016). An infinite hidden Markov model for
47(1), 37–60.
short-term interest rates. Journal of Empirical Finance, 38, 202–220.
Manna, S., Biswas, S., Kundu, R., Rakshit, S., Gupta, P., & Barman, S.
Maister, D. H., Galford, R., & Green, C. (2012). The trusted advisor. Simon
(2017). A statistical approach to predict flight delay using gra-
and Schuster.
dient boosted decision tree. In 2017 International conference on
Makridakis, S., Andersen, A., Carbone, R., Fildes, R., Hibon, M.,
computational intelligence in data science (ICCIDS) (pp. 1–5). IEEE.
Lewandowski, R., et al. (1982). The accuracy of extrapolation (time
Manner, H., Türk, D., & Eichler, M. (2016). Modeling and forecasting
series) methods: Results of a forecasting competition. Journal of
multivariate electricity price spikes. Energy Economics, 60, 255–265.
Forecasting, 1(2), 111–153.
Mannes, A. E., Larrick, R. P., & Soll, J. B. (2012). The social psychology
Makridakis, S., Bonneli, E., Clarke, S., Fildes, R., Gilliland, M., Hover, J., et
of the wisdom of crowds. Social Judgment and Decision Making, 297,
al. (2020). The benefits of systematic forecasting for organizations:
227–242.
The UFO project. Foresight: The International Journal of Applied
Forecasting, 59, 45–56. Mannes, A. E., Soll, J. B., & Larrick, R. P. (2014). The wisdom of
select crowds. Journal of Personality and Social Psychology, 107(2),
Makridakis, S., Chatfield, C., Hibon, M., Lawrence, M., Mills, T., Ord, K.,
276–299.
et al. (1993). The M2-competition: A real-time judgmentally based
forecasting study. International Journal of Forecasting, 9(1), 5–22. Manning, C., Schütze, H., & Raghavan, P. (2008). Introduction to
Makridakis, S., Fry, C., Petropoulos, F., & Spiliotis, E. (2021). The information retrieval. Cambridge University Press.
future of forecasting competitions: Design attributes and principles. Manski, C. F., & Molinari, F. (2010). Rounding probabilistic expectations
INFORMS Journal on Data Science. in surveys. Journal of Business & Economic Statistics, 28(2), 219–231.
Makridakis, S., & Hibon, M. (1979). Accuracy of forecasting: An em- Mapa, D. (2003). A range-based GARCH model for forecasting volatility.
pirical investigation. Journal of the Royal Statistical Society: Series A The Philippine Review of Economics, 60(2), 73–90.
(General), 142(2), 97–125. Marangon Lima, L. M., Popova, E., & Damien, P. (2014). Modeling and
Makridakis, S., & Hibon, M. (2000). The M3-competition: results, con- forecasting of Brazilian reservoir inflows via dynamic linear models.
clusions and implications. International Journal of Forecasting, 16(4), International Journal of Forecasting, 30(3), 464–476.
451–476. Marcellino, M., Stock, J. H., & Watson, M. W. (2006). A compari-
Makridakis, S. G., Hogarth, R. M., & Gaba, A. (2010). Dance with chance: son of direct and iterated multistep AR methods for forecasting
Making luck work for you. Oneworld Publications. macroeconomic time series. Journal of Econometrics, 135(1–2),
Makridakis, S., Hyndman, R. J., & Petropoulos, F. (2020). Forecasting 499–526.
in social settings: The state of the art. International Journal of Marchetti, C. (1983). The automobile in a system context: The past 80
Forecasting, 36(1), 15–28. years and the next 20 years. Technological Forecasting and Social
Makridakis, S., Kirkham, R., Wakefield, A., Papadaki, M., Kirkham, J., Change, 23(1), 3–23.
& Long, L. (2019). Forecasting, uncertainty and risk; perspectives Marchetti, C., & Nakicenovic, N. (1979). The dynamics of energy systems
on clinical decision-making in preventive and curative medicine. and the logistic substitution model RR-79-13, 1–71.
International Journal of Forecasting, 35(2), 659–666. Marcjasz, G., Uniejewski, B., & Weron, R. (2019). On the importance of
Makridakis, S., Spiliotis, E., & Assimakopoulos, V. (2018). Statistical and the long-term seasonal component in day-ahead electricity price
machine learning forecasting methods: Concerns and ways forward. forecasting with NARX neural networks. International Journal of
PLoS One, 13(3), 1–26. Forecasting, 35(4), 1520–1532.
Makridakis, S., Spiliotis, E., & Assimakopoulos, V. (2020). The M4 Marcjasz, G., Uniejewski, B., & Weron, R. (2020). Beating the Naïve—
competition: 100,000 time series and 61 forecasting methods. Combining LASSO with Naïve Intraday Electricity Price Forecasts.
International Journal of Forecasting, 36(1), 54–74. Energies, 13(7), 1667.
855
Marczak, M., & Proietti, T. (2016). Outlier detection in structural McLean, R. D., & Pontiff, J. (2016). Does academic research destroy
time series models: The indicator saturation approach. International return predictability? The Journal of Finance, 71(1), 5–32.
Journal of Forecasting, 32(1), 180–202. McNames, J. (1998). A nearest trajectory strategy for time series
Marinakis, V., Doukas, H., Spiliotis, E., & Papastamatiou, I. (2017). Deci- prediction. In Proceedings of the international workshop on advanced
sion support for intelligent energy management in buildings using black-box techniques for nonlinear modeling (pp. 112–128). Citeseer.
the thermal comfort model. International Journal of Computational McNees, S. K. (1990). The role of judgment in macroeconomic
Intelligence Systems, 10, 882–893. forecasting accuracy. International Journal of Forecasting, 6(3),
Marinakis, V., Doukas, H., Tsapelas, J., Mouzakitis, S., Sicilia, A., 287–299.
Madrazo, L., et al. (2020). From big data to smart energy ser- McNeil, A. J., Frey, R., & Embrechts, P. (2015). Quantitative risk man-
vices: An application for intelligent energy management. Future agement: Concepts, techniques and tools - revised edition. Princeton
Generation Computer Systems, 110, 572–586. University Press.
Marinakis, Y., & Walsh, S. (2021). Parameter instability and structural Meade, N. (1984). The use of growth curves in forecasting market
change in s-curve-based technology diffusion forecasting. Working development - a review and appraisal. Journal of Forecasting, 3(4),
Paper. 429–451.
Mark, N. C. (1995). Exchange rates and fundamentals: Evidence Meade, N. (2000). Evidence for the selection of forecasting methods.
on long-horizon predictability. American Economic Review, 85(1), Journal of Forecasting, 19(6), 515–535.
201–218. Meade, N., & Islam, T. (2001). Forecasting the diffusion of innovations:
Mark, N. C., & Sul, D. (2001). Nominal exchange rates and monetary Implications for time-series extrapolation. In J. S. Armstrong (Ed.),
fundamentals: Evidence from a small post-Bretton Woods panel. Principles of forecasting: A handbook for researchers and practitioners
Journal of International Economics, 53(1), 29–52. (pp. 577–595). Boston, MA: Springer US.
Markowitz, H. (1952). Portfolio selection. The Journal of Finance, 7(1), Meade, N., & Islam, T. (2006). Modelling and forecasting the diffusion of
77–91. innovation – a 25-year review. International Journal of Forecasting,
Marron, J. S., & Wand, M. P. (1992). Exact mean integrated squared 22, 519–545.
error. The Annals of Statistics, 20(2), 712–736. Meade, N., & Islam, T. (2015a). Forecasting in telecommunications and
Martin, G. M., Frazier, D. T., & Robert, C. P. (2020). Computing Bayes: ICT - a review. International Journal of Forecasting, 31(4), 1105–1126.
Bayesian computation from 1763 to the 21st century. ArXiv:2004. Meade, N., & Islam, T. (2015b). Modelling European usage of renew-
06425. able energy technologies for electricity generation. Technological
Martinez, A. B., Castle, J. L., & Hendry, D. F. (2021). Smooth robust Forecasting and Social Change, 90, 497–509.
multi-horizon forecasts. Advances in Econometrics, Forthcoming. Meehl, P. (2013). Clinical versus statistical prediction: A theoretical
Martinez, R., & Sanchez, M. (1970). Automatic booking level control.
analysis and a review of the evidence. Echo Point Books & Media.
10, In AGIFORS symposium proc.
Meeran, S., Dyussekeneva, K., & Goodwin, P. (2013). Sales forecasting
Martinez, E. Z., & Silva, E. A. S. d. (2011). Predicting the number of
using a combination of diffusion model and forecast market: an
cases of dengue infection in ribeirão preto, são paulo state, Brazil,
adaptation of prediction/preference markets. In Proceedings of the
using a SARIMA model. Cadernos de Saude Publica, 27, 1809–1818.
7th IFAC conference on manufacturing modelling, management, and
Martínez-Álvarez, F., Troncoso, A., Riquelme, J. C., & Aguilar-Ruiz, J. S.
control (pp. 87–92).
(2011). Discovery of motifs to forecast outlier occurrence in time
Meeran, S., Jahanbin, S., Goodwin, P., & Quariguasi Frota Neto, J.
series. Pattern Recognition Letters, 32(12), 1652–1665.
(2017). When do changes in consumer preferences make forecasts
Martinez Alvarez, F., Troncoso, A., Riquelme, J. C., & Aguilar Ruiz, J. S.
from choice-based conjoint models unreliable? European Journal of
(2011). Energy time series forecasting based on pattern sequence
Operational Research, 258(2), 512–524.
similarity. IEEE Transactions on Knowledge and Data Engineering,
Meese, R. A., & Rogoff, K. (1983). Empirical exchange rate models of
23(8), 1230–1243.
the seventies: Do they fit out of sample? Journal of International
Masarotto, G. (1990). Bootstrap prediction intervals for autoregressions.
Economics, 14(1–2), 3–24.
Meinshausen, N. (2006). Quantile regression forests. Journal of Machine
Mat Daut, M. A., Hassan, M. Y., Abdullah, H., Rahman, H. A., Abdullah, M.
Learning Research, 7, 983–999.
P., & Hussin, F. (2017). Building electrical energy consumption
Meira, E., Cyrino Oliveira, F. L., & Jeon, J. (2020). Treating and pruning:
forecasting analysis using conventional and artificial intelligence
new approaches to forecasting model selection and combination
methods: A review. Renewable and Sustainable Energy Reviews, 70,
using prediction intervals. International Journal of Forecasting.
1108–1118.
Matte, T. D., Lane, K., & Ito, K. (2016). Excess mortality attributable to Melacini, M., Perotti, S., Rasini, M., & Tappia, E. (2018). E-fulfilment
extreme heat in new york city, 1997–2013. Health Security, 14(2), and distribution in omni-channel retailing: a systematic literature
64–70. review. International Journal of Physical Distribution and Logistics
Maymin, P. (2019). Wage against the machine: A generalized deep- Management, 48(4), 391–414.
learning market test of dataset value. International Journal of Mellit, A., Massi Pavan, A., Ogliari, E., Leva, S., & Lughi, V. (2020).
Forecasting, 35(2), 776–782. Advanced methods for photovoltaic output power forecasting: A
McAlinn, K., Aastveit, K. A., Nakajima, J., & West, M. (2020). Multivariate review. Applied Sciences, 10(2), 487.
Bayesian predictive synthesis in macroeconomic forecasting. Journal Mello, J. (2009). The impact of sales forecast game playing on supply
of the American Statistical Association, 115(531), 1092–1110. chains. Foresight: The International Journal of Applied Forecasting, 13,
McAlinn, K., & West, M. (2019). Dynamic Bayesian predictive synthesis 13–22.
in time series forecasting. Journal of Econometrics, 210(1), 155–169. Mello, J. (2010). Corporate culture and s&op: Why culture counts.
McCabe, B. P., & Martin, G. (2005). BayesIan predictions of low count Foresight: The International Journal of Applied Forecasting, 16, 46–49.
time series. International Journal of Forecasting, 21(2), 315–330. Mena-Oreja, J., & Gozalvez, J. (2020). A comprehensive evaluation of
McCabe, B. P. M., Martin, G. M., & Harris, D. (2011). Efficient proba- deep learning-based techniques for traffic prediction. IEEE Access,
bilistic forecasts for counts. Journal of the Royal Statistical Society. 8, 91188–91212.
Series B. Statistical Methodology, 73(2), 253–272. Meng, X., Bradley, J., Yavuz, B., Sparks, E., Venkataraman, S., Liu, D.,
McCarthy, C., & Ryan, T. M. (1977). Estimates of voter transition et al. (2016). MLlib: MAchine learning in apache spark. Journal of
probabilities from the british general elections of 1974. Journal of Machine Learning Research, 17(1), 1235–1241.
the Royal Statistical Society, Series A, 140(1), 78–85. Meng, X., & Taylor, J. W. (2020). Estimating value-at-risk and expected
McCoy, T. H., Pellegrini, A. M., & Perlis, R. H. (2018). Assessment shortfall using the intraday low and range data. European Journal
of time-series machine learning methods for forecasting hospital of Operational Research, 280(1), 191–202.
discharge volume. JAMA Network Open, 1(7), Article e184087. Meng, X., Taylor, J. W., Ben Taieb, S., & Li, S. (2020). Scoring functions
McFadden, D. (1977). Modelling the Choice of residential location: Tech. for multivariate distributions and level sets. ArXiv:2002.09578.
Rep. 477, Cowles Foundation for Research in Economics, Yale Merkle, E. C., & Steyvers, M. (2013). Choosing a strictly proper scoring
University. rule. Decision Analysis, 10(4), 292–304.
McGill, J., & Van Ryzin, G. (1999). Revenue management: Research Merrick, J. R. W., Hardin, J. R., & Walker, R. (2006). Partnerships in
overview and prospects. Transportation Science, 33(2), 233–256. training. INFORMS Journal on Applied Analytics, 36(4), 359–370.
856
Merrow, E. W., McDonnell, L. M., & Arguden, R. Y. (1988). Understanding Modis, T. (2007). The normal, the natural, and the harmonic.
the outcomes of mega-projects. RAND Corporation. Technological Forecasting and Social Change, 74(3), 391–399.
Messner, J. W., & Pinson, P. (2018). Online adaptive lasso estimation Modis, T. (2013). Natural laws in the service of the decision maker: How
in vector autoregressive models for high dimensional wind power to use science-based methodologies to see more clearly further into
forecasting. International Journal of Forecasting. the future. Growth Dynamics.
Mestre, G., Portela, J., San Roque, A. M. n., & Alonso, E. (2020). Forecast- Modis, T., & Debecker, A. (1992). Chaoslike states can be expected
ing hourly supply curves in the Italian day-ahead electricity market before and after logistic growth. Technological Forecasting and Social
with a double-seasonal SARMAHX model. International Journal of Change, 41(2), 111–120.
Electrical Power & Energy Systems, 121, Article 106083. Moghaddam, A. H., Moghaddam, M. H., & Esfandyari, M. (2016). Stock
Miao, H., Ramchander, S., Wang, T., & Yang, D. (2017). Influential factors market index prediction using artificial neural network. Journal of
in crude oil price forecasting. Energy Economics, 68, 77–88. Economics, Finance and Administrative Science, 21(41), 89–93.
Miao, D. W. C., Wu, C. C., & Su, Y. K. (2013). Regime-switching in Mohammadi, H., & Su, L. (2010). International evidence on crude
volatility and correlation structure using range-based models with oil price dynamics: Applications of ARIMA-GARCH models. Energy
Markov-switching. Economic Modelling, 31(1), 87–93. Economics, 32(5), 1001–1008.
Mikkelsen, L., Moesgaard, K., Hegnauer, M., & Lopez, A. D. (2020). Mohandes, S. R., Zhang, X., & Mahdiyar, A. (2019). A comprehensive
ANACONDA: A new tool to improve mortality and cause of death review on the application of artificial neural networks in building
data. BMC Medicine, 18(1), 1–13. energy analysis. Neurocomputing, 340, 55–75.
Milankovitch, M. (1969). Canon of insolation and the ice-age problem. Molenaers, A., Baets, H., Pintelon, L., & Waeyenbergh, G. (2012). Critical-
Washington, D.C: National Science Foundation, English translation ity classification of spare parts: A case study. International Journal
by the Israel Program for Scientific Translations of Kanon der of Production Economics, 140(2), 570–578.
Erdbestrahlung und seine Anwendung auf das Eiszeitenproblem, Möller, A., Lenkoski, A., & Thorarinsdottir, T. L. (2013). Multivariate
Textbook Publishing Company, Belgrade, 1941. probabilistic forecasting using ensemble Bayesian model averaging
Milas, C., & Rothman, P. (2008). Out-of-sample forecasting of unem- and copulas. Quarterly Journal of the Royal Meteorological Society,
ployment rates with pooled STVECM forecasts. International Journal 139(673), 982–991.
of Forecasting, 24(1), 101–121. Molnár, P. (2016). High-low range in GARCH models of stock return
Millán-Ruiz, D., & Hidalgo, J. I. (2013). Forecasting call centre arrivals. volatility. Applied Economics, 48(51), 4977–4991.
Journal of Forecasting, 32(7), 628–638. Molodtsova, T., & Papell, D. H. (2009). Out-of-sample exchange rate
Miller, R., & Lessard, D. (2007). Evolving strategy: Risk management and predictability with Taylor rule fundamentals. Journal of International
the shaping of large engineering projects: Tech. Rep. 37157, MIT Sloan Economics, 77(2), 167–180.
School of Management, Massachusetts Institute of Technology. Monsell, B., Aston, J., & Koopman, S. (2003). Toward x-13? In Proceed-
Min, A., & Czado, C. (2011). BayesIan model selection for D-vine
ings of the american statistical association, section on business and
pair-copula constructions. The Canadian Journal of Statistics, 39(2),
economic statistics (pp. 1–8). U.S. Census Bureau.
239–258.
Montero Jimenez, J. J., Schwartz, S., Vingerhoeds, R., Grabot, B., &
Min, C.-k., & Zellner, A. (1993). BayesIan and non-Bayesian methods
Salaün, M. (2020). Towards multi-model approaches to predictive
for combining models and forecasts with applications to forecasting
maintenance: A systematic literature survey on diagnostics and
international growth rates. Journal of Econometrics, 56(1–2), 89–118.
prognostics. Journal of Manufacturing Systems, 56, 539–557.
Mincer, J., & Zarnowitz, V. (1969). The evaluation of economic forecasts.
Montero-Manso, P., Athanasopoulos, G., Hyndman, R. J., & Tala-
In J. Mincer (Ed.), Economic forecasts and expectations: analysis of
gala, T. S. (2020). FFORMA: Feature-based forecast model averaging.
forecasting behavior and performance (pp. 3–46). National Bureau of
Economic Research, Inc.
Montero-Manso, P., & Hyndman, R. J. (2020). Principles and algorithms
Mingming, T., & Jinliang, Z. (2012). A multiple adaptive wavelet recur-
for forecasting groups of time series: Locality and globality. arXiv:
rent neural network model to analyze crude oil prices. Journal of
2008.00444.
Economics and Business, 64(4), 275–286.
Montgomery, A. L., Zarnowitz, V., Tsay, R. S., & Tiao, G. C. (1998).
Mirakyan, A., Meyer-Renschhausen, M., & Koch, A. (2017). Compos-
Forecasting the U.S. unemployment rate. Journal of the American
ite forecasting approach, application for next-day electricity price
Statistical Association, 93, 478–493.
forecasting. Energy Economics, 66, 228–237.
Moon, M. A., Mentzer, J. T., & Smith, C. D. (2003). Conducting a sales
Mircetica, D., Rostami-Tabar, B., Nikolicica, S., & Maslarica, M. (2020).
forecasting audit. International Journal of Forecasting, 19(1), 5–25.
Forecasting hierarchical time series in supply chains: an empirical
investigation. Cardiff University. Moon, S., Simpson, A., & Hicks, C. (2013). The development of a
Mirko, K., & Kantelhardt, J. W. (2013). Hadoop. TS: Large-scale time- classification model for predicting the performance of forecasting
series processing. International Journal of Computer Applications, methods for naval spare parts demand. International Journal of
74(17). Production Economics, 143(2), 449–454.
Mirmirani, S., & Li, H. C. (2004). A comparison of VAR and neural net- Moonchai, S., & Chutsagulprom, N. (2020). Short-term forecasting of
works with genetic algorithm in forecasting price of oil. Advances renewable energy consumption: Augmentation of a modified grey
in Econometrics, 19, 203–223. model with a Kalman filter. Applied Soft Computing, 87, Article
Mišić, S., & Radujković, M. (2015). Critical drivers of megaprojects 105994.
success and failure. Procedia Engineering, 122, 71–80. Mori, H., & Yuihara, A. (2001). Deterministic annealing clustering for
Mitchell, T. J., & Beauchamp, J. J. (1988). BayesIan variable selection ANN-based short-term load forecasting. IEEE Transactions on Power
in linear regression. Journal of the American Statistical Association, Systems, 16(3), 545–551.
83(404), 1023–1032. Morlidge, S. (2014a). Do forecasting methods reduce avoidable error?
Mitofsky, W. (1991). A short history of exit polls. In P. J. Lavrakas, Evidence from forecasting competitions. Foresight: The International
& J. K. Holley (Eds.), Polling and presidential election coverage (pp. Journal of Applied Forecasting, 32, 34–39.
83–99). Newbury Park, CA: Sage. Morlidge, S. (2014b). Forecast quality in the supply chain. Foresight:
des Mobilités, L. F. (2020). Motorway traffic in Luxembourg. Accessed The International Journal of Applied Forecasting, 33, 26–31.
on 2020-09-01, URL https://www.kaggle.com/fabmob/motorway- Morlidge, S. (2014c). Using relative error metrics to improve forecast
traffic-in-luxembourg?select=datexDataA1.csv. quality in the supply chain. Foresight: The International Journal of
Modis, T. (1992). Predictions: Society’s telltale signature reveals the past Applied Forecasting, 34, 39–46.
and forecasts the future. Simon & Schuster. Morris, S., & Pratt, D. (2003). Analysis of the Lotka–Volterra competi-
Modis, T. (1994). Fractal aspects of natural growth. Technological tion equations as a technological substitution model. Technological
Forecasting and Social Change, 47(1), 63–73. Forecasting and Social Change, 77, 103–133.
Modis, T. (1997). Genetic re-engineering of corporations. Technological Morss, R. E., Demuth, J. L., & Lazo, J. K. (2008). Communicating uncer-
Forecasting and Social Change, 56(2), 107–118. tainty in weather forecasts: A survey of the US public. Weather and
Modis, T. (1998). Conquering uncertainty: Understanding corporate cycles Forecasting, 23(5), 974–991.
and positioning your company to survive the changing environment. Morwitz, V. (1997). Why consumers don’t always accurately predict
McGraw-Hill. their own future behavior. Marketing Letters, 8(1), 57–70.
857
Moshman, J. (1964). The role of computers in election night broadcast- Ng, Y. S., Stein, J., Ning, M., & Black-Schaffer, R. M. (2007). Comparison
ing. In F. L. Alt, & M. Rubinoff (Eds.), vol. 5, Advances in computers of clinical characteristics and functional outcomes of ischemic
(pp. 1–21). Elsevier. stroke in different vascular territories. Stroke, 38(8), 2309–2314.
Moultrie, T., Dorrington, R., Hill, A., Hill, K., Timæ us, I., & Zaba, B. Nicol-Harper, A., Dooley, C., Packman, D., Mueller, M., Bijak, J., Hodg-
(2013). Tools for demographic estimation. Paris: International son, D., et al. (2018). Inferring transient dynamics of human
Union for the Scientific Study of Population. populations from matrix non-normality. Population Ecology, 60(1),
Mount, T. D., Ning, Y., & Cai, X. (2006). Predicting price spikes in elec- 185–196.
tricity markets using a regime-switching model with time-varying Nielsen, J., Mazick, A., Andrews, N., Detsis, M., Fenech, T., Flores, V., et
parameters. Energy Economics, 28(1), 62–80. al. (2013). Pooling European all-cause mortality: methodology and
Mueller, J. (2009a). Retreat from Doomsday: The obsolescence of major findings for the seasons 2008/2009 to 2010/2011. Epidemiology &
war. Zip Publishing. Infection, 141(9), 1996–2010.
Mueller, J. (2009b). War has almost ceased to exist: An assessment. Nielsen, M., Seo, W., & Seong, D. (2019). Inference on the dimension of
Political Science Quarterly, 124(2), 297–321. the nonstationary subspace in functional time series 1420. Queen’s
Mukhopadhyay, S., & Sathish, V. (2019). Predictive likelihood for co- Economics Department.
herent forecasting of count time series. Journal of Forecasting, 38(3), Nikolopoulos, K. (2020). We need to talk about intermittent demand
222–235. forecasting. European Journal of Operational Research.
Mulholland, J., & Jensen, S. T. (2019). Optimizing the allocation of Nikolopoulos, K., Assimakopoulos, V., Bougioukos, N., Litsa, A., &
funds of an NFL team under the salary cap. International Journal Petropoulos, F. (2012). The theta model: An essential forecasting
of Forecasting, 35(2), 767–775. tool for supply chain planning. In Advances in automation and
Muniain, P., & Ziel, F. (2020). Probabilistic forecasting in day- robotics, Vol. 2 (pp. 431–437). Springer Berlin Heidelberg.
ahead electricity markets: Simulating peak and off-peak prices. Nikolopoulos, K. I., Babai, M. Z., & Bozos, K. (2016). Forecasting
International Journal of Forecasting, 36(4), 1193–1210. supply chain sporadic demand with nearest neighbor approaches.
Murphy, A. H. (1993). What is a good forecast? An essay on the nature International Journal of Production Economics, 177, 139–148.
of goodness in weather forecasting. Weather and Forecasting, 8(2), Nikolopoulos, K., Goodwin, P., Patelis, A., & Assimakopoulos, V. (2007).
281–293. Forecasting with cue information: A comparison of multiple re-
Muth, J. F. (1961). Rational expectations and the theory of price gression with alternative forecasting approaches. European Journal
movements. Econometrica, 29, 315–335. of Operational Research, 180(1), 354–368.
Nikolopoulos, K., Litsa, A., Petropoulos, F., Bougioukos, V., & Kham-
Myrskylä, M., Goldstein, J. R., & Cheng, Y.-H. A. (2013). New cohort
mash, M. (2015). Relative performance of methods for forecasting
fertility forecasts for the developed world: Rises, falls, and reversals.
special events. Journal of Business Research, 68(8), 1785–1791.
Population and Development Review, 39(1), 31–56.
Nikolopoulos, K., & Petropoulos, F. (2018). Forecasting for big data:
Nagi, J., Yap, K. S., Nagi, F., Tiong, S. K., & Ahmed, S. K. (2011). A
Does suboptimality matter? Computers & Operations Research, 98,
computational intelligence scheme for the prediction of the daily
322–329.
peak load. Applied Soft Computing, 11(8), 4773–4788.
Nikolopoulos, K., Punia, S., Schäfers, A., Tsinopoulos, C., & Vasilakis, C.
Naish, S., Dale, P., Mackenzie, J. S., McBride, J., Mengersen, K., & Tong, S.
(2020). Forecasting and planning during a pandemic: COVID-19
(2014). Climate change and dengue: a critical and systematic re-
growth rates, supply chain disruptions, and governmental decisions.
view of quantitative modelling approaches. BMC Infectious Diseases,
European Journal of Operational Research.
14(1), 1–14.
Nikolopoulos, K., Syntetos, A. A., Boylan, J. E., Petropoulos, F., & As-
Nanopoulos, A., Alcock, R., & Manolopoulos, Y. (2001). Feature-based
simakopoulos, V. (2011). An aggregate - disaggregate intermittent
classification of time-series data. In Information processing and
demand approach (ADIDA) to forecasting: An empirical proposition
technology (pp. 49–61). USA: Nova Science Publishers, Inc..
and analysis. Journal of the Operational Research Society, 62(3),
Napierała, J., Hilton, J., Forster, J. J., Carammia, M., & Bijak, J. (2021).
544–554.
Towards an early warning system for monitoring asylum-related
Nikolopoulos, K. I., & Thomakos, D. D. (2019). Forecasting with the theta
migration flows in Europe. International Migration Review, in press.
method: theory and applications. John Wiley & Sons.
Narajewski, M., & Ziel, F. (2020a). Econometric modelling and forecast-
Nogueira, P. J., de Araújo Nobre, M., Nicola, P. J., Furtado, C., &
ing of intraday electricity prices. Journal of Commodity Markets, 19,
Carneiro, A. V. (2020). Excess mortality estimation during the
Article 100107.
COVID-19 pandemic: preliminary data from Portugal. Acta Médica
Narajewski, M., & Ziel, F. (2020b). Ensemble forecasting for intra- Portuguesa, 33(13).
day electricity prices: Simulating trajectories. Applied Energy, 279, Nordhaus, W. D. (1987). Forecasting efficiency: Concepts and
Article 115801. applications. The Review of Economics and Statistics, 69(4), 667–674.
National Research Council (2000). Beyond Six Billion: Forecasting the Norton-Taylor, R. (2015). Global armed conflicts becoming more deadly,
World’s Population. National Academies Press. major study finds. In the Guardian. Accessed on 2020-09-12, http:
National Research Council (2006). Completing the Forecast: Charac- //www.theguardian.com/world/2015/may/20/.
terizing and Communicating Uncertainty for Better Decisions using Nowotarski, J., & Weron, R. (2018). Recent advances in electricity price
Weather and Climate Forecasts. National Academies Press. forecasting: A review of probabilistic forecasting. Renewable and
Neal, P., & Kypraios, T. (2015). Exact Bayesian inference via data Sustainable Energy Reviews, 81, 1548–1568.
augmentation. Statistics and Computing, 25, 333–347. Nsoesie, E., Mararthe, M., & Brownstein, J. (2013). Forecasting peaks of
Neale, W. C. (1964). The peculiar economics of professional sports. seasonal influenza epidemics. PLoS Currents, 5.
Quarterly Journal of Economics, 78(1), 1–14. Nunes, B., Viboud, C., Machado, A., Ringholz, C., Rebelo-de Andrade, H.,
Nelsen, R. (2006). An introduction to copulas. Springer Verlag. Nogueira, P., et al. (2011). Excess mortality associated with in-
Nelson, D. B. (1991). Conditional heteroskedasticity in asset returns: A fluenza epidemics in Portugal, 1980 to 2004. PLoS One, 6(6), Article
new approach. Econometrica, 59(2), 347. e20661.
Nelson, C. R., & Plosser, C. R. (1982). Trends and random walks Nye, J. S. (1990). The changing nature of world power. Political Science
in macroeconomic time series: Some evidence and implications. Quarterly, 105(2), 177–192.
Journal of Monetary Economics, 10(2), 139–162. Nymoen, R., & Sparrman, V. (2015). Equilibrium unemployment dynam-
Nespoli, A., Ogliari, E., Leva, S., Massi Pavan, A., Mellit, A., Lughi, V., ics in a panel of OECD countries. Oxford Bulletin of Economics and
et al. (2019). Day-ahead photovoltaic forecasting: A comparison of Statistics, 77(2), 164–190.
the most effective techniques. Energies, 12(9), 1621. Nystrup, P., Lindström, E., Pinson, P., & Madsen, H. (2020). Temporal
Neves, M. M., & Cordeiro, C. (2020). Modellling (and forecasting) hierarchies with autocorrelation for load forecasting. European
extremes in time series: a naive approach. In Atas do XXIII congresso Journal of Operational Research, 280(3), 876–888.
da SPE (pp. 189–202). Sociedade Portuguesa de Estatística. OBR (2019). Long-term economic determinants. London: Office for
Newbold, P., & Granger, C. W. (1974). Experience with forecasting Budget Responsibility.
univariate time series and the combination of forecasts. Journal of Obst, D., Ghattas, B., Claudel, S., Cugliari, J., Goude, Y., & Oppenheim, G.
the Royal Statistical Society: Series A (General), 137(2), 131–146. (2019). Textual data for time series forecasting. arXiv:1910.12618.
858
O’Connor, M., Remus, W., & Griggs, K. (1993). Judgemental forecast- Ord, K., & Fildes, R. (2013). Principles of business forecasting (1st ed.).
ing in times of change. International Journal of Forecasting, 9(2), South-Western Cengage Learning, Mason, OH and Andover, UK.
163–172. Ord, J. K., Fildes, R., & Kourentzes, N. (2017). Principles of business
Office for National Statistics (2019). U.K. national accounts, the blue forecasting (2nd ed). Wessex Press Publishing Co..
book: 2019. Office for National Statistics. Ordu, M., Demir, E., & Tofallis, C. (2019). A comprehensive modelling
Ogata, Y. (1978). The asymptotic behaviour of maximum likelihood framework to forecast the demand for all hospital services. The
estimators for stationary point processes. Annals of the Institute of International Journal of Health Planning and Management, 34(2),
Statistical Mathematics, 30(2), 243–261. e1257–e1271.
Ogata, Y. (1988). Statistical models for earthquake occurrences and Oreshkin, B. N., Carpov, D., Chapados, N., & Bengio, Y. (2020a). Meta-
residual analysis for point processes. Journal of the American learning framework with applications to zero-shot time-series
Statistical Association, 83(401), 9–27. forecasting. arXiv:2002.02887.
Ogliari, E., Dolara, A., Manzolini, G., & Leva, S. (2017). Physical and Oreshkin, B. N., Carpov, D., Chapados, N., & Bengio, Y. (2020b). N-
hybrid methods comparison for the day ahead PV output power BEATS: Neural basis expansion analysis for interpretable time series
forecast. Renewable Energy, 113, 11–21. forecasting. arXiv:1905.10437.
Ogliari, E., Niccolai, A., Leva, S., & Zich, R. E. (2018). Computational Ozaki, T. (1979). Maximum likelihood estimation of Hawkes’ self-
intelligence techniques applied to the day ahead PV output power exciting point processes. Annals of the Institute of Statistical
forecast: PHANN, SNO and mixed. Energies, 11(6), 1487. Mathematics, 31(1), 145–155.
Oh, D. H., & Patton, A. J. (2016). High-dimensional copula-based Ozer, M. (2011). Understanding the impacts of product knowledge
distributions with mixed frequency data. Journal of Econometrics, and product type on the accuracy of intentions-based new prod-
193(2), 349–366. uct predictions. European Journal of Operational Research, 211(2),
Oh, D. H., & Patton, A. J. (2018). Time-varying systemic risk: Evidence 359–369.
from a dynamic copula model of cds spreads. Journal of Business & Özer, O., Zheng, Y., & Chen, K.-Y. (2011). Trust in forecast information
Economic Statistics, 36(2), 181–195. sharing. Management Science, 57(6), 1111–1137.
Oh, H., & Yoon, C. (2020). Time to build and the real-options channel Paccagnini, A. (2017). Dealing with misspecification in DSGE models:
of residential investment. Journal of Financial Economics, 135(1), A survey 82914. University Library of Munich, Germany.
255–269. Pacheco, J., Millán-Ruiz, D., & Vélez, J. L. (2009). Neural networks for
O’Hagan, A., Buck, C. E., Daneshkhah, A., Richard Eiser, J., Garthwaite, P. forecasting in a multi-skill call centre. In International conference on
H., Jenkinson, D. J., et al. (2006). Uncertain judgements: Eliciting engineering applications of neural networks (pp. 291–300). Springer.
Pai, J., & Pedersen, H. (1999). Threshold models of the term structure
experts’ probabilities. Wiley.
of interest rate. In Joint day proceedings volume of the XXXth
O’Hagan, A., & Forster, J. (2004). Kendall’s advanced theory of statistics:
international ASTIN colloquium/9th international AFIR colloquium (pp.
Bayesian inference, vol. 2B (second ed). Arnold.
O’Hagan, A., & West, M. (2010). The oxford handbook of applied bayesian 387–400).
Paillard, D. (2001). Glacial cycles: towards a new paradigm. Reviews of
analysis. OUP.
Geophysics, 39, 325–346.
O’Hara-Wild, M., & Hyndman, R. (2020). Fasster: Fast additive switching
Pal, D., & Mitra, S. K. (2019). Correlation dynamics of crude oil with
of seasonality, trend and exogenous regressors. R package version
agricultural commodities: A comparison between energy and food
0.1.0.9100.
crops. Economic Modelling, 82, 453–466.
Ojo, O. O., Shah, S., Coutroubis, A., Jiménez, M. T., & Ocana, Y. M. (2018).
Palm, F. C., & Zellner, A. (1992). To combine or not to combine? Issues
Potential impact of industry 4.0 in sustainable food supply chain
of combining forecasts. Journal of Forecasting, 11(8), 687–701.
environment. In 2018 IEEE international conference on technology
Panagiotelis, A., Athanasopoulos, G., Hyndman, R. J., Jiang, B., & Vahid, F.
management, operations and decisions (pp. 172–177). IEEE.
(2019). Macroeconomic forecasting for Australia using a large
Oksuz, I., & Ugurlu, U. (2019). Neural network based model comparison
number of predictors. International Journal of Forecasting, 35(2),
for intraday electricity price forecasting. Energies, 12(23), 4557.
616–633.
Okun, A. M. (1962). Potential GNP: Its measurement and significance.
Panagiotelis, A., Czado, C., & Joe, H. (2012). Pair copula constructions
American Statistical Association, Proceedings of the Business and
for multivariate discrete data. Journal of the American Statistical
Economics Statistics Section, 98–104.
Association, 107(499), 1063–1072.
Oliva, R., & Watson, N. (2009). Managing functional biases in organiza- Panagiotelis, A., Czado, C., Joe, H., & Stöber, J. (2017). Model selection
tional forecasts: A case study of consensus forecasting in supply for discrete regular vine copulas. Computational Statistics & Data
chain planning. International Journal of Operations & Production Analysis, 106, 138–152.
Management, 18(2), 138–151. Panagiotelis, A., Gamakumara, P., Athanasopoulos, G., & Hyndman, R.
Oliveira, J. M., & Ramos, P. c. (2019). Assessing the performance of J. (2021). Forecast reconciliation: A geometric view with new
hierarchical forecasting methods on the retail sector. Entropy, 21(4). insights on bias correction. International Journal of Forecasting, 37(1),
Oliveira, F. L. C., Souza, R. C., & Marcato, A. L. M. (2015). A time series 343–359.
model for building scenarios trees applied to stochastic optimisa- Panahifar, F., Byrne, P. J., & Heavey, C. (2015). A hybrid approach to
tion. International Journal of Electrical Power & Energy Systems, 67, the study of CPFR implementation enablers. Production Planning and
315–323. Control, 26(13), 1090–1109.
Omar, H., Klibi, W., Babai, M. Z., & Ducq, Y. (2021). Basket data-driven Panda, C., & Narasimhan, V. (2007). Forecasting exchange rate better
approach for omnichannel demand forecasting. working paper. with artificial neural network. Journal of Policy Modeling, 29(2),
Önkal, D., & Gönül, M. S. (2005). Judgmental adjustment: A challenge 227–236.
for providers and users of forecasts. Foresight: The International Pankratz, A., & Dudley, U. (1987). Forecasts of power-transformed
Journal of Applied Forecasting, 1, 13–17. series. Journal of Forecasting, 6(4), 239–248.
Önkal, D., Gönül, M. S., & De Baets, S. (2019). Trusting forecasts. Futures Parag, Y., & Sovacool, B. K. (2016). Electricity market design for the
& Foresight Science, 1, Article e19. prosumer era. Nature Energy, 1(4), 1–6.
Önkal, D., Gönül, M. S., Goodwin, P., Thomson, M., & Öz, E. (2017). Eval- Paredes, J., Pedregal, D. J., & Pérez, J. J. (2014). Fiscal policy analysis in
uating expert advice in forecasting: Users’ reactions to presumed the euro area: Expanding the toolkit. Journal of Policy Modeling, 36,
vs. experienced credibility. International Journal of Forecasting, 33(1), 800–823.
280–297. Park, B.-J. (2002). An outlier robust GARCH model and forecasting
Önkal, D., Gönül, M. S., & Lawrence, M. (2008). Judgmental adjustments volatility of exchange rate returns. In Handbook of E: Journal of
of previously adjusted forecasts. Decision Sciences, 39(2), 213–238. Forecasting, In Handbook of E: 21(5).381–393,
Önkal, D., Goodwin, P., Thomson, M., Gönül, M. S., & Pollock, A. (2009). Park, J., & Sandberg, I. W. (1991). Universal approximation using
The relative influence of advice from human experts and statistical radial-basis-function networks. Neural Computation, 3(2), 246–257.
methods on forecast adjustments. Journal of Behavioral Decision Park, S. Y., Yun, B.-Y., Yun, C. Y., Lee, D. H., & Choi, D. G. (2016).
Making, 22(4), 390–409. An analysis of the optimum renewable energy portfolio using the
Önkal, D., Sayım, K. Z., & Gönül, M. S. (2013). Scenarios as channels of bottom–up model: Focusing on the electricity generation sector
forecast advice. Technological Forecasting and Social Change, 80(4), in South Korea. Renewable and Sustainable Energy Reviews, 53,
772–788. 319–329.
859
Parkinson, M. (1980). The extreme value method for estimating the Peng, R. (2015). The reproducibility crisis in science: A statistical
variance of the rate of return. Journal of Business, 53(1), 61–65. counterattack. Significance, 12(3), 30–32.
Pastore, E., Alfieri, A., Zotteri, G., & Boylan, J. E. (2020). The impact Pennings, C. L., & van Dalen, J. (2017). Integrated hierarchical
of demand parameter uncertainty on the bullwhip effect. European forecasting. European Journal of Operational Research, 263(2),
Journal of Operational Research, 283(1), 94–107. 412–418.
Patel, J. (1989). Prediction intervals - a review. Communications in Pennings, C. L. P., van Dalen, J., & Rook, L. (2019). Coordinating
Statistics. Theory and Methods, 18(7), 2393–2465. judgmental forecasting: Coping with intentional biases. Omega, 87,
Patterson, K. D. (1995). An integrated model of the data measurement 46–56.
and data generation processes with an application to consumers’ Pereira, L. N. (2016). An introduction to helpful forecasting methods
expenditure. The Economic Journal, 105, 54–76. for hotel revenue management. International Journal of Hospitality
Patti, E., Acquaviva, A., Jahn, M., Pramudianto, F., Tomasi, R., Rabour- Management, 58, 13–23.
din, D., et al. (2016). Event-driven user-centric middleware for Perera, H. N., Hurley, J., Fahimnia, B., & Reisi, M. (2019). The human
energy-efficient buildings and public spaces. IEEE Systems Journal, factor in supply chain forecasting: A systematic review. European
10(3), 1137–1146. Journal of Operational Research, 274(2), 574–600.
Patton, A. (2006). Modelling asymmetric exchange rate dependence. Peres, R., Muller, E., & Mahajan, V. (2010). Innovation diffusion and new
International Economic Review, 47(2), 527–556. product growth models: A critical review and research directions.
Patton, A. J. (2006a). Estimation of multivariate models for time series International Journal of Research in Marketing, 27, 91–106.
of possibly different lengths. Journal of Applied Econometrics, 21(2), Pesaran, M. H., Pick, A., & Pranovich, M. (2013). Optimal forecasts in
147–173. the presence of structural breaks. Journal of Econometrics, 177(2),
Patton, A. (2013). Copula methods for forecasting multivariate time 134–152.
series. In Handbook of economic forecasting, vol. 2 (pp. 899–960). Pesaran, M. H. M. H., Pick, A., & Timmermann, A. (2011). Variable
Elsevier. selection, estimation and inference for multi-period forecasting
Patton, A. J., & Timmermann, A. (2007). Testing forecast optimality problems. Journal of Econometrics, 164(250), 173–187.
under unknown loss. Journal of the American Statistical Association, Pesaran, M. H., Shin, Y., & Smith, R. P. (1999). Pooled mean group es-
102, 1172–1184. timation of dynamic heterogeneous panels. Journal of the American
Pavía, J. M. (2010). Improving predictive accuracy of exit polls. Statistical Association, 94(446), 621–634.
International Journal of Forecasting, 26(1), 68–81. Peters, J., Janzing, D., & Schölkopf, B. (2017). Elements of causal inference.
Pavia, J. M., Cabrer, B., & Sala, R. (2009). Updating input–output matri- MIT Press.
ces: assessing alternatives through simulation. Journal of Statistical
Petropoulos, F. (2015). Forecasting support systems: Ways forward.
Computation and Simulation, 79(12), 1467–1482.
Foresight: The International Journal of Applied Forecasting, 39, 5–11.
Pavía, J. M., Gil-Carceller, I., Rubio-Mataix, A., Coll, V., Alvarez-Jareño, J.
Petropoulos, F., Fildes, R., & Goodwin, P. (2016). Do ‘big losses’
A., Aybar, C., et al. (2019). The formation of aggregate expectations:
in judgmental adjustments to statistical forecasts affect ex-
wisdom of the crowds or media influence? Contemporary Social
perts’ behaviour? European Journal of Operational Research, 249(3),
Science, 14(1), 132–143.
842–852.
Pavía, J. M., & Larraz, B. (2012). Nonresponse bias and superpopulation
Petropoulos, F., Goodwin, P., & Fildes, R. (2017). Using a rolling
models in electoral polls. Reis, 137(1), 237–264.
training approach to improve judgmental extrapolations elicited
Pavía, J. M., & Romero, R. (2021). Improving estimates accuracy of voter
from forecasters with technical knowledge. International Journal of
transitions. Two new algorithms for ecological inference based on
Forecasting, 33(1), 314–324.
linear programming. Advance.
Petropoulos, F., Hyndman, R. J., & Bergmeir, C. (2018a). Exploring
Pavía-Miralles, J. M. (2005). Forecasts from nonrandom samples. Journal
the sources of uncertainty: Why does bagging for time series
of the American Statistical Association, 100(472), 1113–1122.
forecasting work? European Journal of Operational Research, 268(2),
Pavía-Miralles, J. M., & Larraz-Iribas, B. (2008). Quick counts from
545–554.
non-selected polling stations. Journal of Applied Statistics, 35(4),
Petropoulos, F., & Kourentzes, N. (2014). Improving forecasting via
383–405.
multiple temporal aggregation. Foresight: The International Journal
Payne, J. W. (1982). Contingent decision behavior. Psychological Bulletin,
of Applied Forecasting, 34, 12–17.
92(2), 382–402.
Peña, I., Martinez-Anido, C. B., & Hodge, B.-M. (2018). An extended Petropoulos, F., & Kourentzes, N. (2015). Forecast combinations for
IEEE 118-bus test system with high renewable penetration. IEEE intermittent demand. Journal of the Operational Research Society,
Transactions on Power Systems, 33(1), 281–289. 66(6), 914–924.
Pearl, J. (2009). Causality: Models, reasoning, and inference (2nd ed.). Petropoulos, F., Kourentzes, N., Nikolopoulos, K., & Siemsen, E. (2018b).
Cambridge University Press. Judgmental selection of forecasting models. Journal of Operations
Pearl, R., & Reed, L. J. (1920). On the rate of growth of the population of Management, 60, 34–46.
the United States since 1790 and its mathematical representation. Petropoulos, F., & Makridakis, S. (2020). Forecasting the novel
Proceedings of the National Academy of Sciences of the United States coronavirus COVID-19. PLoS One, 15(3), Article e0231236.
of America, 6(6), 275–288. Petropoulos, F., Makridakis, S., Assimakopoulos, V., & Nikolopoulos, K.
Pedregal, D. J., & Carmen Carnero, M. (2006). State space models for (2014). ‘Horses for courses’ in demand forecasting. European Journal
condition monitoring: a case study. Reliability Engineering & System of Operational Research, 237(1), 152–163.
Safety, 91(2), 171–180. Petropoulos, F., Makridakis, S., & Stylianou, N. (2020). COVID-19:
Pedregal, D. J., García, F. P., & Roberts, C. (2009). An algorithmic Forecasting confirmed cases and deaths with a simple time-series
approach for maintenance management based on advanced state model. International Journal of Forecasting.
space systems and harmonic regressions. Annals of Operations Pettenuzzo, D., & Ravazzolo, F. (2016). Optimal portfolio choice under
Research, 166(1), 109–124. decision-based model combinations. Journal of Applied Econometrics,
Pedregal, D. J., & Pérez, J. J. (2010). Should quarterly gov- 31(7), 1312–1332.
ernment finance statistics be used for fiscal surveillance in Pfann, G. A., Schotman, P. C., & Tschernig, R. (1996). Nonlinear interest
europe? International Journal of Forecasting, 26, 794–807. rate dynamics and implications for the term structure. Journal of
Pedregal, D. J., Pérez, J. J., & Sánchez, A. J. (2014). A toolkit to strengthen Econometrics, 74(1), 149–176.
government budget surveillance. Review of Public Economics, 211, Phillips, A. W. H. (1958). The relation between unemployment and
117–146. the rate of change of money wage rates in the United Kingdom,
Peel, D. A., & Speight, A. (2000). Threshold nonlinearities in unemploy- 1861–1957. Economica, 25, 283–299.
ment rates: Further evidence for the UK and G3 economies. Applied Phillips, P. C. B. (1987). Time series regression with a unit root.
Economics, 32(6), 705–715. Econometrica, 55(2), 277–301.
Pegels, C. C. (1969). Exponential forecasting: Some new variations. Phillips, D. E., Adair, T., & Lopez, A. D. (2018). How useful are registered
Management Sience, 15(5), 311–315. birth statistics for health and social policy? A global systematic
Pelletier, D. (2006). Regime switching for dynamic correlations. Journal assessment of the availability and quality of birth registration data.
of Econometrics, 131(1), 445–473. Population Health Metrics, 16(1), 1–13.
860
Pierce, M. A., Hess, E. P., Kline, J. A., Shah, N. D., Breslin, M., Branda, M. Prak, D., Teunter, R., & Syntetos, A. (2017). On the calculation of safety
E., et al. (2010). The chest pain choice trial: a pilot randomized stocks when demand is forecasted. European Journal of Operational
trial of a decision aid for patients with chest pain in the emergency Research, 256(2), 454–461.
department. Trials, 11(1), 1–8. Preston, S., Heuveline, P., & Guillot, M. (2000). Demography: Measuring
Piironen, J., & Vehtari, A. (2017). Comparison of Bayesian predic- and modeling population processes. Wiley.
tive methods for model selection. Statistics and Computing, 27(3), Prestwich, S. D., Tarim, S. A., Rossi, R., & Hnich, B. (2014). Fore-
711–735. casting intermittent demand by hyperbolic-exponential smoothing.
Pinheiro Neto, D., Domingues, E. G., Coimbra, A. P., de Almeida, A. International Journal of Forecasting, 30(4), 928–933.
T., Alves, A. J., & Calixto, W. P. (2017). Portfolio optimization of Pretis, F. (2020). Econometric modelling of climate systems: The
renewable energy assets: Hydro, wind, and photovoltaic energy in equivalence of energy balance models and cointegrated vector
the regulated market in Brazil. Energy Economics, 64, 238–250. autoregressions. Journal of Econometrics, 214(1), 256–273.
Pinker, S. (2011). The better angels of our nature: the decline of violence Pretis, F., & Kaufmann, R. K. (2018). Out-of-sample paleo-climate
in history and its causes. Penguin UK. simulations: Testing hypotheses about the mid-brunhes event, the
Pinker, S. (2018). Enlightenment now: the case for reason, science, stage 11 paradox, and orbital variations. Canada: University of
humanism, and progress. Penguin. Victoria.
Pinson, P. (2012). Very-short-term probabilistic forecasting of wind Pretis, F., & Kaufmann, R. K. (2020). Managing carbon emissions to
power with generalized logit-normal distributions. Journal of the avoid the next ice age. Canada: University of Victoria.
Royal Statistical Society. Series C. Applied Statistics, 555–576. Pretis, F., Reade, J., & Sucarrat, G. (2017). gets: GEneral-to-specific
Pinson, P., Chevallier, C., & Kariniotakis, G. (2007). Trading wind (GETS) modelling and indicator saturation methods. R package
generation from short-term probabilistic forecasts of wind power. version 0.12.
IEEE Transaction on Power Systems, 22(3), 1148–1156. Pretis, F., Reade, J., & Sucarrat, G. (2018). Automated general-to-specific
Pinson, P., Madsen, H., Nielsen, H. A., Papaefthymiou, G., & Klöckl, B. (GETS) regression modeling and indicator saturation for outliers
(2009). From probabilistic forecasts to statistical scenarios of and structural breaks. Journal of Statistical Software, 86(3).
short-term wind power production. Wind Energy, 12(1), 51–62. Pretis, F., Schneider, L., & Smerdon, J. (2016). Detecting volcanic erup-
Pinson, P., & Makridakis, S. (2020). Pandemics and forecasting: The way tions in temperature reconstructions by designed break-indicator
forward through the Taleb-Ioannidis debate. International Journal of saturation. Journal of Economic Surveys, 30(3), 403–429.
Forecasting. Pritularga, K. F., Svetunkov, I., & Kourentzes, N. (2021). Stochastic
Pinson, P., Reikard, G., & Bidlot, J.-R. (2012). Probabilistic forecasting of coherency in forecast reconciliation. International Journal of Produc-
the wave energy flux. Applied Energy, 93, 364–370. tion Economics, 240, Article 108221, URL https://www.sciencedirect.
Pinson, P., & Tastu, J. (2013). Discrimination ability of the energy score. com/science/article/pii/S0925527321001973.
Technical University of Denmark (DTU). Programme, U. N. D. (2019). Population facts no. 2019/6, december
Pirolli, P., & Card, S. (1999). Information foraging. Psychological Review, 2019: How certain are the united nations global population pro-
106(4), 643–675. jections?. (2019/6), Department of Economic and Social Affairs,
Pitt, M., Chan, D., & Kohn, R. (2006). Efficient Bayesian inference for Population Division.
Gaussian copula regression models. Biometrika, 93(3), 537–554. Proietti, T. (2003). Forecasting the US unemployment rate. Computa-
Plescia, C., & De Sio, L. (2018). An evaluation of the performance and tional Statistics & Data Analysis, 42, 451–476.
suitability of r × c methods for ecological inference with known Promprou, S., Jaroensutasinee, M., & Jaroensutasinee, K. (2006). Fore-
true values. Quality & Quantity, 52(2), 669–683. casting dengue haemorrhagic fever cases in Southern Thailand
Plott, C., & Chen, K.-Y. (2002). Information aggregation mechanisms: using ARIMA models. Dengue Bulletin, 30, 99–106.
Concept, design and implementation for a sales forecasting prob- Prudêncio, R. B., & Ludermir, T. B. (2004). Meta-learning approaches to
lem, 1131. California Institute of Technology, Division of the selecting time series models. Neurocomputing, 61, 121–137.
Humanities and Social Sciences. Psaradellis, I., & Sermpinis, G. (2016). Modelling and trading the u.s.
Poccia, D. (2019). Amazon forecast – now generally available. implied volatility indices. Evidence from the VIX, VXN and VXD
In AWS news blog. https://aws.amazon.com/blogs/aws/amazon- indices. International Journal of Forecasting, 32(4), 1268–1283.
forecast-now-generally-available/ (Accessed on 01 September Puig, X., & Ginebra, J. (2015). Ecological inference and spatial variation
2020). of individual behavior: National divide and elections in catalonia.
Politi, M. C., Han, P. K., & Col, N. F. (2007). Communicating the Geographical Analysis, 47(3), 262–283.
uncertainty of harms and benefits of medical interventions. Medical Qiao, Z., Wu, X., Ge, S., & Fan, W. (2019). MNN: multimodal attentional
Decision Making, 27(5), 681–695. neural networks for diagnosis prediction. Extraction, 1, A1.
Politis, D. N., & Romano, J. P. (1992). A circular block-resampling Qu, X., Kang, X., Zhang, C., Jiang, S., & Ma, X. (2016). Short-term
procedure for stationary data. Exploring the Limits of Bootstrap, prediction of wind power based on deep long short-term memory.
2635270. In 2016 IEEE PES asia-pacific power and energy engineering conference
Politis, D. N., & Romano, J. P. (1994). The stationary bootstrap. Journal (pp. 1148–1152).
of the American Statistical Association, 89(428), 1303–1313. Quaedvlieg, R. (2019). Multi-horizon forecast comparison. Journal of
Polk, C., Haghbin, M., & de Longis, A. (2020). Time-series variation Business & Economic Statistics, 1–14.
in factor premia: The influence of the business cycle. Journal of Quiroz, M., Nott, D. J., & Kohn, R. (2018). Gaussian variational ap-
Investment Management, 18(1). proximation for high-dimensional state space models. arXiv:1801.
Polson, N. G., & Sokolov, V. O. (2017). Deep learning for short-term 07873.
traffic flow prediction. Transportation Research Part C (Emerging R Core Team (2020). R: A language and environment for statistical
Technologies), 79, 1–17. computing. Vienna, Austria: R Foundation for Statistical Computing.
Porras, E., & Dekker, R. (2008). An inventory control system for spare Rabanser, S., Januschowski, T., Flunkert, V., Salinas, D., & Gasthaus, J.
parts at a refinery: An empirical comparison of different re-order (2020). The effectiveness of discretization in forecasting: An
point methods. European Journal of Operational Research, 184(1), empirical study on neural time series models. arXiv:2005.10111.
101–132. Racine, J. (2000). Consistent cross-validatory model-selection for de-
Powell, W. B. (2019). A unified framework for stochastic optimization. pendent data: hv-block cross-validation. Journal of Econometrics,
European Journal of Operational Research, 275(3), 795–821. 99(1), 39–61.
Poynting, J. H. (1884). A comparison of the fluctuations in the price of Raftery, A. E. (1993). BayesIan model selection in structural equation
wheat and in the cotton and silk imports into great britain. Journal models. In K. Bollen, & J. Long (Eds.), Testing structural equation
of the Statistical Society of London, 47(1), 34–74. models (pp. 163–180). Newbury Park, CA: Sage.
Pradeepkumar, D., & Ravi, V. (2017). Forecasting financial time se- Raftery, A. E. (2016). Use and communication of probabilistic forecasts.
ries volatility using particle swarm optimization trained quantile Statistical Analysis and Data Mining: The ASA Data Science Journal,
regression neural network. Applied Soft Computing, 58, 35–52. 9(6), 397–410.
Prahl, A., & Van Swol, L. (2017). Understanding algorithm aversion: Raftery, A. E., Madigan, D., & Hoeting, J. A. (1997). BayesIan model
When is advice from automation discounted? Journal of Forecasting, averaging for linear regression models. Journal of the American
36(6), 691–702. Statistical Association, 92(437), 179–191.
861
Rahman, S., & Serletis, A. (2012). Oil price uncertainty and the Canadian Rees, P. H., & Wilson, A. G. (1973). Accounts and models for spatial de-
economy: Evidence from a VARMA, GARCH-in-mean, asymmetric mographic analysis I: Aggregate population. Environment & Planning
BEKK model. Energy Economics, 34(2), 603–610. A, 5(1), 61–90.
Rajvanshi, V. (2015). Performance of range and return based volatility Reggiani, P., & Boyko, O. (2019). A Bayesian processor of uncertainty for
estimators: evidence from Indian crude oil futures market. Global precipitation forecasting using multiple predictors and censoring.
Economy and Finance Journal, 8(1), 46–66. Monthly Weather Review, 147(12), 4367–4387.
Ramos, M.-H., Mathevet, T., Thielen, J., & Pappenberger, F. (2010). Com- Reid, D. (1972). A comparison of forecasting techniques on economic
municating uncertainty in hydro-meteorological forecasts: mission time series. Forecasting in Action. Operational Research Society and
impossible? Meteorological Applications, 17(2), 223–235. the Society for Long Range Planning.
Ramos, P., & Oliveira, J. M. (2016). A procedure for identification of Reikard, G., Pinson, P., & Bidlot, J.-R. (2011). Forecasting ocean wave
appropriate state space and ARIMA models based on time-series energy: The ECMWF wave model and time series methods. Ocean
cross-validation. Algorithms, 9(4), 76. Engineering, 38(10), 1089–1099.
Ramos, P., Santos, N., & Rebelo, R. (2015). Performance of state space Reikard, G., Robertson, B., Buckham, B., Bidlot, J.-R., & Hiles, C. (2015).
and ARIMA models for consumer retail sales forecasting. Robotics Simulating and forecasting ocean wave energy in western Canada.
and Computer-Integrated Manufacturing, 34, 151–163. Ocean Engineering, 103, 223–236.
Ranawana, R., & Palade, V. (2006). Optimized precision-a new measure Reimers, S., & Harvey, N. (2011). Sensitivity to autocorrelation in judg-
for classifier performance evaluation. In 2006 IEEE international mental time series forecasting. International Journal of Forecasting,
conference on evolutionary computation (pp. 2254–2261). IEEE. 27(4), 1196–1214.
Rangapuram, S. S., de Bezenac, E., Benidis, K., Stella, L., & Rendall, M. S., Handcock, M. S., & Jonsson, S. H. (2009). BayesIan
Januschowski, T. (2020). Normalizing Kalman filters for multivariate estimation of hispanic fertility hazards from survey and population
time series analysis. In Advances in neural information processing data. Demography, 46(1), 65–83.
systems (pp. 7785–7794). Renzl, B. (2008). Trust in management and knowledge sharing: The
Rangapuram, S. S., Seeger, M. W., Gasthaus, J., Stella, L., Wang, Y., & mediating effects of fear and knowledge documentation. Omega,
Januschowski, T. (2018). Deep state space models for time series 36(2), 206–220.
forecasting. In Advances in neural information processing systems (pp. Riahi, N., Hosseini-Motlagh, S.-M., & Teimourpour, B. (2013). A three-
7785–7794). phase hybrid times series modeling framework for improved
Rangarajan, P., Mody, S. K., & Marathe, M. (2019). Forecasting dengue hospital inventory demand forecast. International Journal of Hospital
and influenza incidences using a sparse representation of google Research, 2(3), 133–142.
trends, electronic health records, and time series data. PLoS Rice, G., Wirjanto, T., & Zhao, Y. (2020). Tests for conditional het-
Computational Biology, 15(11), Article e1007518. eroscedasticity of functional data. Journal of Time Series Analysis,
Ranjan, R., & Gneiting, T. (2010). Combining probability forecasts. Jour-
41(6), 733–758.
nal of the Royal Statistical Society. Series B. Statistical Methodology,
Richardson, L. F. (1948). Variation of the frequency of fatal quarrels
72(1), 71–91.
with magnitude. Journal of the American Statistical Association,
Rao, J. K., Anderson, L. A., Sukumar, B., Beauchesne, D. A., Stein, T., &
43(244), 523–546.
Frankel, R. M. (2010). Engaging communication experts in a delphi
Richardson, L. F. (1960). Statistics of deadly quarrels. Boxwood Press.
process to identify patient behaviors that could enhance commu-
Riedel, K. (2021). The value of the high, low and close in the estimation
nication in medical encounters. BMC Health Services Research, 10,
of Brownian motion. Statistical Inference for Stochastic Processes, 24,
97.
179–210.
Rao, K., & Kishore, V. (2010). A review of technology diffusion models
Rios, I., Wets, R. J.-B., & Woodruff, D. L. (2015). Multi-period fore-
with special reference to renewable energy technologies. Renewable
casting and scenario generation with limited data. Computational
and Sustainable Energy Reviews, 14(3), 1070–1078.
Management Science, 12, 267–295.
Rao, Y., & McCabe, B. (2016). Real-time surveillance for abnormal
Ritchie, H., Ortiz-Ospina, E., Beltekian, D., Mathieu, E., Hasell, J., Mac-
events: the case of influenza outbreaks. Statistics in Medicine,
donald, B., et al. (2020). Coronavirus pandemic (COVID-19). https:
35(13), 2206–2220.
//ourworldindata.org/coronavirus (Accessed on 20 August 2020).
Rapach, D. E., & Strauss, J. K. (2009). Differences in housing price
Riveiro, M., Helldin, T., Falkman, G., & Lebram, M. (2014). Effects of
forecastability across U.S. states. International Journal of Forecasting,
visualizing uncertainty on decision-making in a target identification
25(2), 351–372.
Rapach, D. E., Strauss, J. K., Tu, J., & Zhou, G. (2019). Industry return scenario. Computers & Graphics, 41, 84–98.
predictability: A machine learning approach. The Journal of Financial Roberts, J. M. (2001). Estimates of the productivity trend using time-
Data Science, 1(3), 9–28. varying parameter techniques. The BE Journal of Macroeconomics,
Rapach, D. E., Strauss, J. K., & Zhou, G. (2010). Out-of-sample equity 1(1).
premium prediction: Combination forecasts and links to the real Robinson, W. S. (1950). Ecological correlations and the behavior of
economy. Review of Financial Studies, 23(2), 821–862. individuals. American Sociological Review, 15(3), 351–357.
Rapach, D. E., Strauss, J. K., & Zhou, G. (2013). International stock return Rodriguez, J. C. (2007). Measuring financial contagion: A copula
predictability: What is the role of the United States? The Journal of approach. Journal of Empirical Finance, 14(3), 401–423.
Finance, 68(4), 1633–1662. Rodríguez-Sanz, A., Comendador, F., Valdés, R., Pérez-Castán, J.,
Rapach, D. E., & Zhou, G. (2020). Time-series and cross-sectional stock Montes, R. B., & Serrano, S. (2019). Assessment of airport arrival
return forecasting: New machine learning methods. In E. Jurczenko congestion and delay: Prediction and reliability. Transportation
(Ed.), Machine learning for asset management: New developments and Research Part C (Emerging Technologies), 98, 255–283.
financial applications (pp. 1–34). Hoboken, NJ: Wiley. Rogers, A. (1975). Introduction to multiregional mathematical demogra-
Rasp, S., Pritchard, M. S., & Gentine, P. (2018). Deep learning to phy. New York: Wiley.
represent subgrid processes in climate models. Proceedings of the Rogers, L. C. G., & Satchell, S. E. (1991). Estimating variance from high,
National Academy of Sciences, 115(39), 9684–9689. low and closing prices. Annals of Applied Probability, 1(4), 504–512.
Ravishanker, N., yen Wu, L. S., & Glaz, J. (1991). Multiple prediction Romero, D., Olivero, J., Real, R., & Guerrero, J. C. (2019). Applying fuzzy
intervals for time series: comparison of simultaneous and marginal logic to assess the biogeographical risk of dengue in south america.
intervals. Journal of Forecasting, 10(5), 445–463. Parasites & Vectors, 12(1), 1–13.
Raymer, J., & Wiśniowski, A. (2018). Applying and testing a forecasting Romero, R., Pavía, J. M., Martín, J., & Romero, G. (2020). Assessing
model for age and sex patterns of immigration and emigration. uncertainty of voter transitions estimated from aggregated data.
Population Studies, 72(3), 339–355. Application to the 2017 french presidential election. Journal of
Reade, J., Singleton, C., & Brown, A. (2020). Evaluating strange forecasts: Applied Statistics, 47(13–15), 2711—2736.
The curious case of football match scorelines. Scottish Journal of Rosen, O., Jiang, W., King, G., & Tanner, M. A. (2001). BayesIan
Political Economy. and frequentist inference for ecological inference: The rxc case.
Rebollo, J., & Balakrishnan, H. (2014). Characterization and predic- Statistica Neerlandica, 55(2), 134–156.
tion of air traffic delays. Transportation Research Part C (Emerging Rosenblatt, M. (1952). Remarks on a multivariate transformation. The
Technologies), 44, 231–241. Annals of Mathematical Statistics, 23, 470–472.
862
Rossi, B. (2005). Testing long-horizon predictive ability with high Salinas, D., Flunkert, V., Gasthaus, J., & Januschowski, T. (2019). DeepAR:
persistence, and the meese–rogoff puzzle. International Economic Probabilistic forecasting with autoregressive recurrent networks.
Review, 46(1), 61–92. International Journal of Forecasting.
Rossi, B. (2013). Exchange rate predictability. Journal of Economic Salway, R., & Wakefield, J. (2004). A common framework for ecolog-
Literature, 51(4), 1063–1119. ical inference in epidemiology, political science and sociology. In
Rossi, B., & Sekhposyan, T. (2016). Forecast rationality tests in the Ecological inference: new methodological strategies (pp. 303–332).
presence of instabilities, with applications to federal reserve and Cambridge University Press.
survey forecasts. Journal of Applied Econometrics, 31(3), 507–532. Sanders, N. R., & Manrodt, K. B. (2003). Forecasting software in practice:
Rostami-Tabar, B., Babai, M. Z., Syntetos, A., & Ducq, Y. (2013). Demand Use, satisfaction, and performance. Interfaces, 33(5), 90–93.
forecasting by temporal aggregation. Naval Research Logistics, 60(6), Sanderson, J. (2012). Risk, uncertainty and governance in megapro-
479–498. jects: A critical discussion of alternative explanations. International
Rostami-Tabar, B., & Ziel, F. (2020). Anticipating special events Journal of Project Management, 30(4), 432–443.
in emergency department forecasting. International Journal of Santos, M. S., Abreu, P. H., Garca-Laencina, P. J., Simão, A., & Carvalho, A.
Forecasting. (2015). A new cluster-based oversampling method for improving
Rothman, P. (1998). Forecasting asymmetric unemployment rates. The survival prediction of hepatocellular carcinoma patients. Journal of
Review of Economics and Statistics, 80(1), 164–168. Biomedical Informatics, 58, 49–59.
Rothschild, D. (2009). Forecasting elections: Comparing prediction
Sardinha-Lourenço, A., Andrade-Campos, A., Antunes, A., & Oliveira, M.
markets, polls, and their biases. Public Opinion Quarterly, 73(5),
S. (2018). Increased performance in the short-term water de-
895–916.
mand forecasting through the use of a parallel adaptive weighting
Rottenberg, S. (1956). The baseball players’ labor market. Journal of strategy. Journal of Hydrology, 558, 392–404.
Political Economy, 64(3), 242–258.
Savin, S., & Terwiesch, C. (2005). Optimal product launch times
Roulin, E., & Vannitsem, S. (2019). Post-processing of seasonal predic-
in a duopoly: balancing life-cycle revenues with product cost.
tions – case studies using EUROSIP hindcast data base. Nonlinear
Operations Research, 53, 26–47.
Processes in Geophysics.
Scerri, M., De Goumoens, P., Fritsch, C., Van Melle, G., Stiefel, F., &
Rousseau, D. M., Sitkin, S. B., Burt, R. S., & Camerer, C. F. (1998). Not
So, A. (2006). The INTERMED questionnaire for predicting return
so different after all: A cross-discipline view of trust. Academy of
to work after a multidisciplinary rehabilitation program for chronic
Management Review, 23(3), 393–404.
low back pain. Joint Bone Spine, 73(6), 736–741.
Rowe, G., & Wright, G. (1999). The delphi technique as a forecasting
tool: issues and analysis. International Journal of Forecasting, 15(4), Schäfer, A. M., & Zimmermann, H. G. (2006). Recurrent neural networks
353–375. are universal approximators. In Artificial Neural Networks – ICANN
Rowe, G., & Wright, G. (2001). Expert opinions in forecasting: The 2006 (pp. 632–640). Springer Berlin Heidelberg.
role of the delphi technique. In J. S. Armstrong (Ed.), Principles Scharpf, A., Schneider, G., Nöh, A., & Clauset, A. (2014). Forecasting the
of forecasting: A handbook for researchers and practitioners (pp. risk of extreme massacres in Syria. European Review of International
125–144). Boston, MA: Springer US. Studies, 1(2), 50–68.
Royer, J. F. (1993). Review of recent advances in dynamical extended Schefzik, R., Thorarinsdottir, T. L., & Gneiting, T. (2013). Uncertainty
range forecasting for the extratropics. In J. Shukla (Ed.), Prediction quantification in complex simulation models using ensemble copula
of interannual climate variations (pp. 49–69). Berlin, Heidelberg: coupling. Statistical Science, 28(4), 616–640.
Springer. Scheuerer, M., & Hamill, T. M. (2015). Variogram-based proper scoring
Ruano, A. E., Crispim, E. M., Conceiçao, E. Z., & Lúcio, M. M. J. (2006). rules for probabilistic forecasts of multivariate quantities. Monthly
Prediction of building’s temperature using neural networks models. Weather Review, 143(4), 1321–1334.
Energy and Buildings, 38(6), 682–694. Scheuren, F. J., & Alvey, W. (2008). Elections and exit polling. Hoboken
Rubaszek, M. (2020). Forecasting crude oil prices with DSGE models. (New Yersey): John Wiley & Sons.
International Journal of Forecasting. Schmertmann, C. P. (2003). A system of model fertility schedules with
Ruddiman, W. (2005). Plows, plagues and petroleum: how humans took graphically intuitive parameters. Demographic Research, 9, 81–110.
control of climate. Princeton: Princeton University Press. Schmertmann, C., Zagheni, E., Goldstein, J. R., & Myrskylä, M. (2014).
Rycroft, R. S. (1993). Microcomputer software of interest to fore- BayesIan forecasting of cohort fertility. Journal of the American
casters in comparative review: An update. International Journal of Statistical Association, 109(506), 500–513.
Forecasting, 9(4), 531–575. Schnaars, S. P., & Topol, M. T. (1987). The use of multiple scenarios
Sa, J. (1987). Reservations forecasting in airline yield management. in sales forecasting: An empirical test. International Journal of
(Ph.D. thesis), Massachusetts Institute of Technology. Forecasting, 3(3), 405–419.
Sacheti, A., Gregory-Smith, I., & Paton, D. (2014). Uncertainty of Schoemaker, P. J. H. (1991). When and how to use scenario planning:
outcome or strengths of teams: An economic analysis of atten- A heuristic approach with illustration. Journal of Forecasting, 10(6),
dance demand for international cricket. Applied Economics, 46(17), 549–564.
2034–2046.
Schoen, R. (1987). Modeling multigroup populations. Springer Science &
Sah, S., Moore, D. A., & MacCoun, R. J. (2013). Cheap talk and credibility: Business Media.
The consequences of confidence and accuracy on advisor credibility
Schönbucher, P. J. (2003). Credit derivatives pricing models: models,
and persuasiveness. Organizational Behavior and Human Decision
pricing and implementation. John Wiley & Sons.
Processes, 121(2), 246–255.
Schubert, S., & Rickard, R. (2011). Using forecast value added anal-
Sakamoto, Y., Ishiguro, M., & Kitagawa, G. (1986). Akaike information
ysis for data-driven forecasting improvement. IBF Best Practices
criterion statistics. Dordrecht, the Netherlands: D. Reidel, 81.
Conference.
Sakata, S., & White, H. (1998). High breakdown point conditional
dispersion estimation with application to S&P 500 daily returns Schwanenberg, D., Fan, F. M., Naumann, S., Kuwajima, J. I., Montero, R.
volatility. Econometrica, 66(3), 529–568. A., & Assis dos Reis, A. (2015). Short-term reservoir optimization
Sakia, R. M. (1992). The box-cox transformation technique: A review. for flood mitigation under meteorological and hydrological forecast
Journal of the Royal Statistical Society: Series D (the Statistician), uncertainty. Water Resources Management, 29(5), 1635–1651.
41(2), 169–178. Schwarz, G. (1978). Estimating the dimension of a model. The Annals
Saksornchai, T., Lee, W.-J., Methaprayoon, K., Liao, J. R., & Ross, R. of Statistics, 6(2), 461–464.
J. (2005). Improve the unit commitment scheduling by us- Scott Armstrong, J. (2006). Should the forecasting process eliminate
ing the neural-network-based short-term load forecasting. IEEE face-to-face meetings? Foresight: The International Journal of Applied
Transactions on Industry Applications, 41(1), 169–179. Forecasting, 5, 3–8.
Salinas, D., Bohlke-Schneider, M., Callot, L., Medico, R., & Gasthaus, J. Seaman, B. (2018). Considerations of a retail forecasting practitioner.
(2019). High-dimensional multivariate forecasting with low-rank International Journal of Forecasting, 34(4), 822–829.
Gaussian copula processes. In Advances in neural information Seifert, D. (2003). Collaborative planning, forecasting, and replenishment:
processing systems (pp. 6827–6837). How to create a supply chain advantage. New York: AMACOM.
863
Semenoglou, A.-A., Spiliotis, E., Makridakis, S., & Assimakopoulos, V. Shen, G., Jia, J., Nie, L., Feng, F., Zhang, C., Hu, T., et al. (2017).
(2021). Investigating the accuracy of cross-learning time series Depression detection via harvesting social media: A multimodal
forecasting methods. International Journal of Forecasting, 37(3), dictionary learning solution.. In IJCAI (pp. 3838–3844).
1072–1084, URL https://www.sciencedirect.com/science/article/pii/ Sheng, C., Zhao, J., Leung, H., & Wang, W. (2013). Extended Kalman
S0169207020301850. filter based echo state network for time series prediction using
Semero, Y. K., Zhang, J., & Zheng, D. (2020). EMD–PSO–ANFIS-Based MapReduce framework. In 2013 IEEE 9th International Conference
hybrid approach for short-term load forecasting in microgrids. IET on Mobile Ad-Hoc and Sensor Networks (pp. 175–180). IEEE.
Generation, Transmission and Distribution, 14(3), 470–475. Shephard, N. (1994). Partial non-Gaussian state space. Biometrika, 81,
Seong, Y., & Bisantz, A. M. (2008). The impact of cognitive feedback on 115–131.
judgment performance and trust with decision aids. International Shi, Q., Yin, J., Cai, J., Cichocki, A., Yokota, T., Chen, L., et al. (2020). Block
Journal of Industrial Ergonomics, 38(7), 608–625. Hankel tensor ARIMA for multiple short time series forecasting. In
Serletis, A., & Rangel-Ruiz, R. (2004). Testing for common features in AAAI (pp. 5758–5766).
North American energy markets. Energy Economics, 26(3), 401–414. Shishkin, J., Young, A. H., & Musgrave, J. C. (1967). The x-11 variant of
Setel, P., AbouZahr, C., Atuheire, E. B., Bratschi, M., Cercone, E., Chin- the census II method seasonal adjustment program. (15), Bureau of
ganya, O., et al. (2020). Mortality surveillance during the COVID-19 the Census, US Department of Commerce.
pandemic. Bulletin of the World Health Organization, 98(6), 374. Shumway, R. H., & Stoffer, D. S. (2017). Time Series Analysis and Its
Applications: With R Examples. Springer.
Setzler, H., Saydam, C., & Park, S. (2009). EMS call volume predic-
Shvachko, K., Kuang, H., Radia, S., & Chansler, R. (2010). The hadoop
tions: A comparative study. Computers & Operations Research, 36(6),
distributed file system. In 2010 IEEE 26th Symposium on Mass
1843–1851.
Storage Systems and Technologies (MSST) (pp. 1–10). IEEE.
Shackleton, M. B., Taylor, S. J., & Yu, P. (2010). A multi-horizon
Si, X.-S., Wang, W., Hu, C.-H., & Zhou, D.-H. (2011). Remaining useful
comparison of density forecasts for the S&P 500 using index returns
life estimation – a review on the statistical data driven approaches.
and option prices. Journal of Banking & Finance, 34(11), 2678–2693.
European Journal of Operational Research, 213(1), 1–14.
Shah, I., & Lisi, F. (2020). Forecasting of electricity price through
Sideratos, G., Ikonomopoulos, A., & Hatziargyriou, N. D. (2020). A novel
a functional prediction of sale and purchase curves. Journal of
fuzzy-based ensemble model for load forecasting using hybrid
Forecasting, 39(2), 242–259.
deep neural networks. Electric Power Systems Research, 178, Article
Shahriari, M., & Blumsack, S. (2018). The capacity value of optimal wind 106025.
and solar portfolios. Energy, 148, 992–1005. Silvapulle, P., & Moosa, I. A. (1999). The relationship between spot
Shale, E. A., Boylan, J. E., & Johnston, F. R. (2006). Forecasting for in- and futures prices: evidence from the crude oil market. Journal
termittent demand: the estimation of an unbiased average. Journal of Futures Markets: Futures, Options, and Other Derivative Products,
of the Operational Research Society, 57(5), 588–592. 19(2), 175–193.
Shaman, J., & Karspeck, A. (2012). Forecasting seasonal outbreaks of Simon, H., & Sebastian, K.-H. (1987). Diffusion and advertising: The
influenza. Proceedings of the National Academy of Sciences, 109(50), german telephone campaign. Management Science, 33(4), 451–466.
20425–20430. Simpson, E. H. (1951). The interpretation of interaction in contingency
Shang, H. L., & Booth, H. (2020). Synergy in fertility forecasting: tables. Journal of the Royal Statistical Society. Series B. Statistical
Improving forecast accuracy through model averaging. Genus, 76. Methodology, 13(2), 238–241.
Shang, H. L., Booth, H., & Hyndman, R. J. (2011). Point and interval Sims, C. (1980). Macroeconomics and reality. Econometrica, 48(1), 1–48.
forecasts of mortality rates and life expectancy: A comparison Sims, C. (2002). Solving linear rational expectations models. Computa-
of ten principal component methods. Demographic Research, 25, tional Economics, 20(1–2), 1–20.
173–214. Sims, C. A. (2003). Implications of rational inattention. Journal of
Shang, H. L., & Haberman, S. (2020a). Forecasting age distribution of Monetary Economics, 50, 665–690.
death counts: An application to annuity pricing. Annals of Actuarial Singer, P. W., & Friedman, A. (2014). Cybersecurity: What Everyone Needs
Science, 14(1), 150–169. To Know. OUP USA.
Shang, H. L., & Haberman, S. (2020b). Retiree mortality forecasting: A Singleton, C., Reade, J., & Brown, A. (2019). Going with your gut: The
partial age-range or a full age-range model? Risks, 8(3), 69. (in)accuracy of forecast revisions in a football score prediction
Shang, H. L., & Hyndman, R. J. (2017). Grouped functional time series game. Journal of Behavioral and Experimental Economics, Article
forecasting: An application to age-specific mortality rates. Journal 101502.
of Computational and Graphical Statistics, 26(2), 330–343. Sinnathamby, M. A., Whitaker, H., Coughlan, L., Bernal, J. L., Ramsay, M.,
Shang, J., Ma, T., Xiao, C., & Sun, J. (2019). Pre-training of graph & Andrews, N. (2020). All-cause excess mortality observed by age
augmented transformers for medication recommendation. arXiv: group and regions in the first wave of the COVID-19 pandemic in
1906.00346. England. Eurosurveillance, 25(28), Article 2001239.
Sisson, S. A., Fan, Y., & Beaumont, M. (2019). Handbook of Approximate
Shang, G., McKie, E. C., Ferguson, M. E., & Galbreth, M. R. (2020). Using
Bayesian Computation. Chapman & Hall/CRC.
transactions data to improve consumer returns forecasting. Journal
Smets, F., Warne, A., & Wouters, R. (2014). Professional forecasters and
of Operations Management, 66(3), 326–348.
real-time forecasting with a DSGE model. International Journal of
Shang, H. L., & Xu, R. (2021). Change point detection for COVID-19
Forecasting, 30(4), 981–995.
excess deaths in Belgium. Journal of Population Research, in press.
Smets, F., & Wouters, R. (2007). Shocks and frictions in US business
Sharpe, W. F. (1964). Capital asset prices: A theory of market equi-
cycles: A Bayesian dsge approach. American Economic Review, 97(3),
librium under conditions of risk. The Journal of Finance, 19(3),
586–606.
425.
Smith, D. R. (2002). Markov-Switching and stochastic volatility diffusion
Shen, H. (2009). On modeling and forecasting time series of smooth models of short-term interest rates. Journal of Business & Economic
curves. Technometrics, 51(3), 227–238. Statistics, 20(2), 183–197.
Shen, H., & Huang, J. Z. (2005). Analysis of call centre arrival data using Smith, M. (2010). Modeling longitudinal data using a pair-copula de-
singular value decomposition. Applied Stochastic Models in Business composition of serial dependence. Journal of the American Statistical
and Industry, 21(3), 251–263. Association, 105(492), 1467–1479.
Shen, H., & Huang, J. Z. (2008a). Forecasting time series of inhomoge- Smith, J. C. (2011). The ins and outs of UK unemployment. The Economic
neous Poisson processes with application to call center workforce Journal, 121, 402–444.
management. The Annals of Applied Statistics, 2(2), 601–623. Smith, M., & Khaled, M. (2012). Estimation of copula models with
Shen, H., & Huang, J. Z. (2008b). Interday forecasting and intraday discrete margins via Bayesian data augmentation. Journal of the
updating of call center arrivals. Manufacturing & Service Operations American Statistical Association, 107(497), 290–303.
Management, 10(3), 391–410. Smith, B., Leimkuhler, J., & Darrow, R. (1992). Yield management at
Shen, H., Huang, J. Z., & Lee, C. (2007). Forecasting and dynamic American airlines. Interfaces, 22(1), 8–31.
updating of uncertain arrival rates to a call center. In 2007 IEEE Smith, M. S., & Maneesoonthorn, W. (2018). Inversion copulas from
International Conference on Service Operations and Logistics, and nonlinear state space models with an application to inflation
Informatics (pp. 1–6). IEEE. forecasting. International Journal of Forecasting, 34(3), 389–407.
864
Smith, D. M., Scaife, A. A., Eade, R., Athanasiadis, P., Bellucci, A., Soyer, R., & Tarimcilar, M. M. (2008). Modeling and analysis of call
Bethke, I., et al. (2020). North atlantic climate far more predictable center arrival data: A Bayesian approach. Management Science,
than models imply. Nature, 583(7818), 796–800. 54(2), 266–278.
Smith, M. S., & Vahey, S. P. (2016). Asymmetric forecast densities for US Spagat, M., Mack, A., Cooper, T., & Kreutz, J. (2009). Estimating war
macroeconomic variables from a Gaussian copula model of cross- deaths: An arena of contestation. The Journal of Conflict Resolution,
sectional and serial dependence. Journal of Business & Economic 53(6), 934–950.
Statistics, 34(3), 416–434. Sparkes, J. R., & McHugh, A. K. (1984). Awareness and use of forecasting
Smith, J., & Wallis, K. F. (2009a). A simple explanation of the forecast techniques in british industry. Journal of Forecasting, 3(1), 37–42.
combination puzzle. Oxford Bulletin of Economics and Statistics, Spencer, J. (1904). On the graduation of the rates of sickness and
71(3), 331–355. mortality presented by the experience of the manchester unity of
Smith, J., & Wallis, K. F. (2009b). A simple explanation of the forecast oddfellows during the period 1893-97. Journal of the Institute of
combination puzzle. Oxford Bulletin of Economics and Etatistics, Actuaries, 38(4), 334–343.
71(3), 331–355. Spiegelhalter, D., Pearson, M., & Short, I. (2011). Visualizing uncertainty
Smyl, S. (2020). A hybrid method of exponential smoothing and about the future. Science, 333(6048), 1393–1400.
recurrent neural networks for time series forecasting. International Spiliotis, E., Assimakopoulos, V., & Makridakis, S. (2020). Generalizing
Journal of Forecasting, 36(1), 75–85. the theta method for automatic forecasting. European Journal of
Sniezek, J. A., & Henry, R. A. (1989). Accuracy and confidence in group Operational Research, 284(2), 550–558.
judgment. Organizational Behavior and Human Decision Processes, Spiliotis, E., Assimakopoulos, V., & Nikolopoulos, K. (2019). Forecasting
43(1), 1–28. with a hybrid method utilizing data smoothing, a variation of
Snyder, R. D., Ord, J. K., & Beaumont, A. (2012). Forecasting the the theta method and shrinkage of seasonal factors. International
intermittent demand for slow-moving inventories: A modelling Journal of Production Economics, 209, 92–102.
approach. International Journal of Forecasting, 28(2), 485–496. Spiliotis, E., Kouloumos, A., Assimakopoulos, V., & Makridakis, S.
Sobhani, M., Hong, T., & Martin, C. (2020). Temperature anomaly detec- (2020). Are forecasting competitions data representative of the
tion for electric load forecasting. International Journal of Forecasting, reality? International Journal of Forecasting, 36(1), 37–53.
36(2), 324–333. Spiliotis, E., Petropoulos, F., & Assimakopoulos, V. (2019). Improving the
Sobotka, T., & Beaujouan, E. (2018). Late motherhood in low-fertility forecasting performance of temporal hierarchies. PLoS One, 14(10),
countries: Reproductive intentions, trends and consequences. In Article e0223422.
D. Stoop (Ed.), Preventing Age Related Fertility Loss (pp. 11–29). Spiliotis, E., Petropoulos, F., Kourentzes, N., & Assimakopoulos, V.
Cham: Springer International Publishing. (2020). Cross-temporal aggregation: Improving the forecast accu-
Sobri, S., Koohi-Kamali, S., & Rahim, N. A. (2018). Solar photovoltaic racy of hierarchical electricity consumption. Applied Energy, 261,
generation forecasting methods: A review. Energy Conversion & Article 114339.
Management, 156, 459–497. Spiliotis, E., Raptis, A., & Assimakopoulos, V. (2015). Off-the-shelf vs.
Soebiyanto, R. P., Adimi, F., & Kiang, R. K. (2010). Modeling and Customized forecasting support systems. Foresight: The International
predicting seasonal influenza transmission in warm regions using Journal of Applied Forecasting, Issue 43, 42–48.
climatological parameters. PLoS One, 5(3), Article e9450. Spithourakis, G., Petropoulos, F., Babai, M. Z., Nikolopoulos, K., &
Sohst, R. R., Tjaden, J., de Valk, H., & Melde, S. (2020). The future of Assimakopoulos, V. (2011). Improving the performance of popu-
migration to europe: A systematic review of the literature on mi- lar supply chain forecasting techniques. Supply Chain Forum, An
gration scenarios and forecasts. Geneva: International Organization International Journal, 12(4), 16–25.
for Migration. Spithourakis, G., Petropoulos, F., Nikolopoulos, K., & Assimakopoulos, V.
Sommer, B., Pinson, P., Messner, J. W., & Obst, D. (2020). Online (2014). A systemic view of ADIDA framework. IMA Journal of
distributed learning in wind power forecasting. International Journal Management Mathematics, 25, 125–137.
of Forecasting. Squire, P. (1988). Why the 1936 literary digest poll failed. Public Opinion
Son, N., Yang, S., & Na, J. (2019). Hybrid forecasting model for Quarterly, 52(1), 125–133.
short-term wind power prediction using modified long short-term Sridhar, S., & Govindarasu, M. (2014). Model-based attack detection
memory. Energies, 12(20), 3901. and mitigation for automatic generation control. IEEE Transactions
Song, H., & Li, G. (2021). Editorial: Tourism forecasting competition in on Smart Grid, 5(2), 580–591.
the time of COVID-19. Annals of Tourism Research, Article 103198. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., & Salakhutdi-
Song, H., Qiu, R. T. R., & Park, J. (2019). A review of research on tourism nov, R. (2014). Dropout: A simple way to prevent neural networks
demand forecasting: Launching the annals of tourism research from overfitting. Journal of Machine Learning Research, 15(56),
curated collection on tourism demand forecasting. Annals of Tourism 1929–1958.
Research, 75, 338–362. Stadlober, E., Hormann, S., & Pfeiler, B. (2018). Quality and performance
Song, H., Witt, S. F., & Li, G. (2008). The Advanced Econometrics of of a PM10 daily forecasting model. Atmospheric Enviroment, 42,
Tourism Demand. Routledge. 1098–1109.
Sopadjieva, E., Dholakia, U. M., & Benjamin, B. (2017). A study of 46,000 Stanford NLP Group (2013). Code for deeply moving: Deep learn-
shoppers shows that omnichannel retailing works. Harvard Business ing for sentiment analysis. https://nlp.stanford.edu/sentiment/code.
Review, Reprint H03D7A. html (Accessed on 05 September 2020).
Sorensen, D. (2020). Strategic IBP: Driving profitable growth in complex Staszewska-Bystrova, A. (2011). Bootstrap prediction bands for forecast
global organizations. Foresight: The International Journal of Applied paths from vector autoregressive models. Journal of Forecasting,
Forecasting, 56, 36–45. 30(8), 721–735.
Sorjamaa, A., Hao, J., Reyhani, N., Ji, Y., & Lendasse, A. (2007). Method- Steckley, S. G., Henderson, S. G., & Mehrotra, V. (2005). Performance
ology for long-term prediction of time series. Neurocomputing, measures for service systems with a random arrival rate. In
70(16–18), 2861–2869. Proceedings of the Winter Simulation Conference, 2005.. IEEE.
Sorjamaa, A., & Lendasse, A. (2006). Time series prediction using dirrec Steurer, J. (2011). The delphi method: an efficient procedure to
strategy. In M. Verleysen (Ed.), ESANN, European Symposium on generate knowledge. Skeletal Radiology, 40(8), 959–961.
Artificial Neural Networks, European Symposium on Artificial Neural Stillwell, J., & Clarke, M. (Eds.), (2011). Population dynamics and
Networks (pp. 143–148). Citeseer, European Symposium on Artificial projection methods. Springer, Dordrecht.
Neural Networks. Stock, J. H., & Watson, M. W. (1998). A comparison of linear and
Sornette, D. (2003). Critical market crashes. Physics Reports, 378(1), nonlinear univariate models for forecasting macroeconomic time
1–98. series. (6607), National Bureau of Economic Research, Inc.
Soule, D., Grushka-Cockayne, Y., & Merrick, J. R. W. (2020). A heuristic Stock, J. H., & Watson, M. W. (2002). Forecasting using principal
for combining correlated experts. SSRN:3680229. components from a large number of predictors. Journal of the
Souza, R. C., Marcato, A. L. M., Dias, B. H., & Oliveira, F. L. C. (2012). Opti- American Statistical Association, 97(460), 1167–1179.
mal operation of hydrothermal systems with hydrological scenario Stock, J. H., & Watson, M. W. (2012). Generalized shrinkage methods for
generation through bootstrap and periodic autoregressive models. forecasting using many predictors. Journal of Business & Economic
European Journal of Operational Research, 222(3), 606–615. Statistics, 30, 481–493.
865
Stone, M. (1961). The opinion pool. The Annals of Mathematical Statistics, Taillardat, M., Mestre, O., Zamo, M., & Naveau, P. (2016). Cali-
32(4), 1339–1342. brated ensemble forecasts using quantile regression forests and
Stone, M. (1974). Cross-validatory choice and assessment of statistical ensemble model output statistics. Monthly Weather Review, 144(6),
predictions. Journal of the Royal Statistical Society. Series B. Statistical 2375–2393.
Methodology, 36(2), 111–133. Talagala, T. (2015). Distributed lag nonlinear modelling approach to
Strähl, C., & Ziegel, J. (2017). Cross-calibration of probabilistic forecasts. identify relationship between climatic factors and dengue incidence
Electronic Journal of Statistics, 11(1), 608–639. in colombo district, Sri Lanka. Epidemiology, Biostatistics and Public
Strauch, R., Hallerberg, M., & Hagen, J. (2004). Budgetary forecasts in Health, 12(4).
europe – the track record of stability and convergence programmes. Talagala, T. S., Hyndman, R. J., & Athanasopoulos, G. (2018). Meta-
ECB Working Paper 307. learning how to forecast time series. (6/18), Monash University,
Strijbosch, L. W. G., & Moors, J. J. A. (2005). The impact of unknown de- Department of Econometrics and Business Statistics.
mand parameters on (r,s)-inventory control performance. European Talagala, P. D., Hyndman, R. J., Leigh, C., Mengersen, K., & Smith-
Journal of Operational Research, 162(3), 805–815. Miles, K. (2019). A feature-based procedure for detecting technical
Su, Y.-K., & Wu, C.-C. (2014). A new range-based regime-switching outliers in water-quality data from in situ sensors. Water Resources
dynamic conditional correlation model for minimum-variance Research, 55(11), 8547–8568.
hedging. Journal of Mathematical Finance, 04(03), 207–219. Talagala, P. D., Hyndman, R. J., & Smith-Miles, K. (2020). Anomaly
detection in high dimensional data. Journal of Computational and
Sugeno, M. (1985). Industrial Applications of Fuzzy Control. Elsevier
Graphical Statistics, in press, 1–32.
Science Inc.
Talagala, P. D., Hyndman, R. J., Smith-Miles, K., Kandanaarachchi, S., &
Sun, S., Sun, Y., Wang, S., & Wei, Y. (2018). Interval decomposition en-
Muñoz, M. A. (2020). Anomaly detection in streaming nonstationary
semble approach for crude oil price forecasting. Energy Economics,
temporal data. Journal of Computational and Graphical Statistics,
76, 274–287.
29(1), 13–27.
Sun, J., Sun, Y., Zhang, X., & McCabe, B. (2021). Model averaging of
Talavera-Llames, R. L., Pérez-Chacón, R., Martínez-Ballesteros, M., Tron-
integer-valued autoregressive model with covariates. https://Ssrn.
coso, A., & Martínez-Álvarez, F. (2016). A nearest neighbours-based
Com.
algorithm for big time series data forecasting. In International
Sundquist, E. T., & Keeling, R. F. (2009). The mauna loa carbon diox- Conference on Hybrid Artificial Intelligence Systems (pp. 174–185).
ide record: Lessons for long-term earth observations. Geophysical Springer.
Monograph Series, 183, 27–35. Taleb, N. N. (2008). The Black Swan: The Impact of the Highly Improbable
Surowiecki, J. (2005). The Wisdom of Crowds: Why the Many are Smarter (New Edition). Penguin.
than the Few (New). Abacus. Taleb, N. N. (2020). Statistical Consequences of Fat Tails: Real World
Svensson, A., Holst, J., Lindquist, R., & Lindgren, G. (1996). Optimal pre- Preasymptotics, Epistemology, and Applications. STEM Academic
diction of catastrophes in autoregressive moving-average processes. Press.
Journal of Time Series Analysis, 17(5), 511–531. Taleb, N. N., Bar-Yam, Y., & Cirillo, P. (2020). On single point forecasts
Svetunkov, I., & Boylan, J. E. (2020). State-space ARIMA for supply-chain for fat tailed variables. International Journal of Forecasting.
forecasting. International Journal of Productions Research, 58(3), Tam Cho, W. K. (1998). Iff the assumption fits...: A comment on the
818–827. king ecological inference solution. Political Analysis, 7, 143–163.
Swanson, N. R., & Xiong, W. (2018). Big data analytics in economics: Tan, B. K., Panagiotelis, A., & Athanasopoulos, G. (2019). BayesIan
What have we learned so far, and where should we go from inference for the one-factor copula model. Journal of Computational
here? Canadian Journal of Economics, 51(3), 695–746. and Graphical Statistics, 28(1), 155–173.
Sweeney, C., Bessa, R. J., Browell, J., & Pinson, P. (2019). The future of Tan, P.-N., Steinbach, M., & Kumar, V. (2005). Introduction To Data
forecasting for renewable energy. Wiley Interdisciplinary Reviews: Mining, (First Edition). Boston, MA, USA: Addison-Wesley Longman
Energy and Environment. Publishing Co., Inc..
Syntetos, A. A., Babai, Z., Boylan, J. E., Kolassa, S., & Nikolopoulos, K. Tandberg, D., Easom, L. J., & Qualls, C. (1995). Time series forecasts of
(2016). Supply chain forecasting: Theory, practice, their gap and poison center call volume. Journal of Toxicology: Clinical Toxicology,
the future. European Journal of Operational Research, 252(1), 1–26. 33(1), 11–18.
Syntetos, A. A., Babai, M. Z., & Luo, S. (2015). Forecasting of compound Tanir, O., & Booth, R. J. (1999). Call center simulation in bell
erlang demand. Journal of the Operational Research Society, 66(12), Canada. 2, In WSC’99. 1999 Winter Simulation Conference
2061–2074. Proceedings.’simulation-a Bridge To the Future’(Cat. No. 99CH37038)
Syntetos, A. A., & Boylan, J. E. (2001). On the bias of intermittent (pp. 1640–1647). IEEE.
demand estimates. International Journal of Production Economics, Tarun, G., & Bryan, K. (2019). Factor momentum everywhere. Journal
71(1), 457–466. of Portfolio Management, 45(3), 13–36.
Tashman, L. J. (2000). Out-of-sample tests of forecasting accuracy:
Syntetos, A. A., & Boylan, J. E. (2005). The accuracy of intermit-
an analysis and review. International Journal of Forecasting, 16(4),
tent demand estimates. International Journal of Forecasting, 21(2),
437–450.
303–314.
Tashman, L. J., & Leach, M. L. (1991). Automatic forecasting software:
Syntetos, A. A., & Boylan, J. E. (2006). On the stock control perfor-
A survey and evaluation. International Journal of Forecasting, 7(2),
mance of intermittent demand estimators. International Journal of
209–230.
Production Economics, 103(1), 36–47.
Tay, A. S., & Wallis, K. F. (2000). Density forecasting: a survey. Journal
Syntetos, A. A., Boylan, J. E., & Croston, J. D. (2005). On the categoriza-
of Forecasting, 19(4), 235–254.
tion of demand patterns. Journal of the Operational Research Society,
Taylor, S. (1986a). Modelling Financial Time Series. Wiley.
56(5), 495–503. Taylor, J. M. (1986b). The retransformed mean after a fitted power
Syntetos, A. A., Kholidasari, I., & Naim, M. M. (2016). The effects of transformation. Journal of the American Statistical Association,
integrating management judgement into OUT levels: In or out of 81(393), 114–118.
context? European Journal of Operational Research, 249(3), 853–863. Taylor, J. W. (2003a). Exponential smoothing with a damped
Syntetos, A. A., Nikolopoulos, K., Boylan, J. E., Fildes, R., & Goodwin, P. multiplicative trend. International Journal of Forecasting, 19(4),
(2009). The effects of integrating management judgement into 715–725.
intermittent demand forecasts. International Journal of Production Taylor, J. W. (2003b). Short-term electricity demand forecasting using
Economics, 118(1), 72–81. double seasonal exponential smoothing. Journal of the Operational
Syntetos, A. A., Zied Babai, M., & Gardner, E. S. (2015). Forecasting Research Society, 54(8), 799–805.
intermittent inventory demands: simple parametric methods vs. Taylor, J. W. (2007). Forecasting daily supermarket sales using exponen-
bootstrapping. Journal of Business Research, 68(8), 1746–1752. tially weighted quantile regression. European Journal of Operational
Syring, N., & Martin, R. (2020). Gibbs posterior concentration rates Research, 178(1), 154–167.
under sub-exponential type losses. arXiv:2012.04505. Taylor, J. W. (2008). A comparison of univariate time series methods for
Szozda, N. (2010). Analogous forecasting of products with a short life forecasting intraday arrivals at a call center. Management Science,
cycle. Decision Making in Manufacturing and Services, 4(1–2), 71–85. 54(2), 253–265.
866
Taylor, J. W. (2010). Exponentially weighted methods for forecasting Thorarinsdottir, T. L., Scheuerer, M., & Heinz, C. (2016). Assessing
intraday time series with multiple seasonal cycles. International the calibration of high-dimensional ensemble forecasts using rank
Journal of Forecasting, 26(4), 627–646. histograms. Journal of Computational and Graphical Statistics, 25(1),
Taylor, J. W. (2012). Density forecasting of intraday call center arrivals 105–122.
using models based on exponential smoothing. Management Science, Thorarinsdottir, T. L., & Schuhen, N. (2018). Verification: assessment
58(3), 534–549. of calibration and accuracy. In Statistical Postprocessing of Ensemble
Taylor, J. W., & Bunn, D. W. (1999). A quantile regression approach Forecasts (pp. 155–186). Elsevier.
to generating prediction intervals. Management Science, 45(2), Tian, J., & Anderson, H. M. (2014). Forecast combinations under struc-
131–295. tural break uncertainty. International Journal of Forecasting, 30(1),
Taylor, J., & Jeon, J. (2018). Probabilistic forecasting of wave height for 161–175.
offshore wind turbine maintenance. European Journal of Operational Tian, F., Yang, K., & Chen, L. (2017). Realized volatility forecasting
Research, 267(3). of agricultural commodity futures using the HAR model with
Taylor, S. J., & Letham, B. (2018). Forecasting at scale. The American time-varying sparsity. International Journal of Forecasting, 33(1),
Statistician, 72(1), 37–45. 132–152.
Taylor, J. W., McSharry, P. E., & Buizza, R. (2009). Wind power density Tibshirani, R. (1996). Regression shrinkage and selection via the
forecasting using ensemble predictions and time series models. IEEE LASSO. Journal of the Royal Statistical Society. Series B. Statistical
Transactions on Energy Conversion, 24(3), 775–782. Methodology, 58, 267–288.
Taylor, M. P., & Peel, D. A. (2000). Nonlinear adjustment, long-run equi-
Timmermann, A. (2000). Moments of Markov switching models. Journal
librium and exchange rate fundamentals. Journal of International
of Econometrics, 96(1), 75–111.
Money and Finance, 19(1), 33–53.
Timmermann, A. (2006). Forecast combinations. In G. Elliott, C. W.
Taylor, J. W., & Snyder, R. D. (2012). Forecasting intraday time se-
J. Granger, & A. Timmermann (Eds.), vol. 1, Handbook of Economic
ries with multiple seasonal cycles using parsimonious seasonal
Forecasting (pp. 135–196). Amsterdam: Elsevier.
exponential smoothing. Omega, 40(6), 748–757.
Timmermann, A., & Zhu, Y. (2019). Comparing forecasting performance
Taylor, P. F., & Thomas, M. E. (1982). Short term forecasting: Horses for
with panel data. SSRN:3380755.
courses. Journal of the Operational Research Society, 33(8), 685–694.
Tenti, P. (1996). Forecasting foreign exchange rates using recurrent Tiwari, A. K., Nasreen, S., Shahbaz, M., & Hammoudeh, S. (2020).
neural networks. Applied Artificial Intelligence, 10(6), 567–582. Time-frequency causality and connectedness between international
Teräsvirta, T. (1994). Specification, estimation, and evaluation of prices of energy, food, industry, agriculture and metals. Energy
smooth transition autoregressive models. Journal of the American Economics, 85, Article 104529.
Statistical Association, 89(425), 208–218. Todini, E. (1991). Coupling real-time forecasting in the aswan dam
Teräsvirta, T., Tjostheim, D., & Granger, C. W. J. (2010). Modelling reservoir management. In Workshop on Monitoring, Forecasting and
Nonlinear Economic Time Series. OUP Oxford. Simulation of River Basins for Agricultural Production. Rome: Land
Teunter, R. H., & Duncan, L. (2009). Forecasting intermittent demand: a and Water Development Division.
comparative study. Journal of the Operational Research Society, 60(3), Todini, E. (1999). Using phase-space modelling for inferring forecasting
321–329. uncertainty in non-linear stochastic decision schemes. Journal of
Teunter, R. H., Syntetos, A. A., & Zied Babai, M. (2011). Intermittent Hydroinformatics, 1(2), 75–82.
demand: Linking forecasting to inventory obsolescence. European Todini, E. (2008). A model conditional processor to assess predictive
Journal of Operational Research, 214(3), 606–615. uncertainty in flood forecasting. International Journal of River Basin
Tewari, D. D. (1990). Energy-price impacts modelling in the agriculture Management, 6(2), 123–137.
sector. Energy Economics, 12(2), 147–158. Todini, E. (2016). Predictive uncertainty assessment and decision
Thaler, R. H., & Sunstein, C. R. (2009). Nudge: Improving Decisions About making. In V. P. Singh (Ed.), Handbook of Applied Hydrology (pp.
Health, Wealth, and Happiness. Penguin Books. 26.1–26.16). New York: McGraw Hill.
The Conference Board (2020). Global business cycle indicators. In Todini, E. (2017). Flood forecasting and decision making in the new
the Conference Board. https://conference-board.org/data/bcicountry. millennium. Where are we? Water Resources Management, 31(10),
cfm?cid=1 (Accessed on 2020-09-07). 3111–3129.
The R. F. E. Working Group Report (2015). Risk in the front end of Todini, E. (2018). Paradigmatic changes required in water resources
megaprojects. European Cooperation in Science and Technology. management to benefit from probabilistic forecasts. Water Security,
Theocharis, Z., & Harvey, N. (2019). When does more mean worse? 3, 9–17.
Accuracy of judgmental forecasting is nonlinearly related to length Toktay, L. B. (2003). Forecasting product returns. In V. D. R. Guide, Jr., &
of data series. Omega, 87, 10–19. L. N. van Wassenhove (Eds.), Business Aspects of Closed-Loop Supply
Theocharis, Z., Smith, L. A., & Harvey, N. (2018). The influence of Chains (pp. 203–209). Pittsburgh: Carnegie Mellon University Press.
graphical format on judgmental forecasting accuracy: Lines versus Toktay, L. B., Wein, L. M., & Zenios, S. A. (2000). Inventory manage-
points. Futures & Foresight Science, 13, Article e7. ment of remanufacturable products. Management Science, 46(11),
Theodosiou, M. (2011). Disaggregation & aggregation of time series 1412–1426.
components: A hybrid forecasting approach using generalized re-
Tolman, H. L. (2008). A mosaic approach to wind wave modeling. Ocean
gression neural networks and the theta method. Neurocomputing,
Modelling, 25(1–2), 35–47.
74(6), 896–905.
Tong, H. (1978). On a threshold model. In C. Chen (Ed.), NATO ASI
Thomakos, D., & Nikolopoulos, K. (2012). Fathoming the theta method
Series E: Applied Sc., Pattern Recognition and Signal Processing (pp.
for a unit root process. IMA Journal of Management Mathematics,
575–586). Netherlands: Sijthoff & Noordhoff.
25(1), 105–124.
Thomakos, D. D., & Nikolopoulos, K. (2015). Forecasting multivariate Tong, H. (1990). Non-Linear Time Series: A Dynamical System Approach.
time series with the theta method: Multivariate theta method. Clarendon Press.
Journal of Forecasting, 34(3), 220–229. Toth, Z., & Buizza, R. (2019). Weather forecasting: What sets the
Thomé, A. M. T., Hollmann, R. L., & Scavarda do Carmo, L. F. R. forecast skill horizon? In Sub-Seasonal To Seasonal Prediction (pp.
R. (2014). Research synthesis in collaborative planning forecast 17–45). Elsevier.
and replenishment. Industrial Management & Data Systems, 114(6), Touzani, S., Granderson, J., & Fernandes, S. (2018). Gradient boosting
949–965. machine for modeling the energy consumption of commercial
Thomé, A. M. T., Scavarda, L. F., Fernandez, N. S., & Scavarda, A. J. (2012). buildings. Energy and Buildings, 158, 1533–1543.
Sales and operations planning: A research synthesis. International Tracy, M., Cerdá, M., & Keyes, K. M. (2018). Agent-based modeling in
Journal of Production Economics, 138(1), 1–13. public health: current applications and future directions. Annual
Thomson, W., Jabbari, S., Taylor, A., Arlt, W., & Smith, D. (2019). Review of Public Health, 39, 77–94.
Simultaneous parameter estimation and variable selection via Tran, M.-N., Nott, D. J., & Kohn, R. (2017). Variational Bayes with in-
the logit-normal continuous analogue of the spike-and-slab prior. tractable likelihood. Journal of Computational and Graphical Statistics,
Journal of the Royal Society Interface, 16(150). 26(4), 873–882.
867
Tran, T., Phung, D., Luo, W., Harvey, R., Berk, M., & Venkatesh, S. (2013). Vaks, A., Mason, A. J., Breitenbach, S. F. M., et al. (2019). Palaeoclimate
An integrated framework for suicide risk prediction. In Proceedings evidence of vulnerable permafrost during times of low sea ice.
of the 19th ACM SIGKDD International Conference on Knowledge Nature, 577, 221–225.
Discovery and Data Mining (pp. 1410–1418). Van de Ven, A., & Delbeco, A. L. (1971). Nominal versus interact-
Trapero, J. R., Kourentzes, N., & Fildes, R. (2015). On the identification ing group processes for committee decision-making effectiveness.
of sales forecasting models in the presence of promotions. Journal Academy of Management Journal. Academy of Management, 14(2),
of the Operational Research Society, 66(2), 299–307. 203–212.
Trapero, J. R., Pedregal, D. J., Fildes, R., & Kourentzes, N. (2013). Van den Broeke, M., De Baets, S., Vereecke, A., Baecke, P., & Vander-
Analysis of judgmental adjustments in the presence of promotions. heyden, K. (2019). Judgmental forecast adjustments over different
International Journal of Forecasting, 29(2), 234–243. time horizons. Omega, 87, 34–45.
Triguero, I., Peralta, D., Bacardit, J., García, S., & Herrera, F. (2015). van den Hengel, G., & Franses, P. H. (2020). Forecasting social con-
MRPR: A MapReduce solution for prototype reduction in big data flicts in africa using an epidemic type aftershock sequence model.
classification. Neurocomputing, 150, 331–345. Forecasting, 2(3), 284–308.
Trivedi, P. K., & Zimmer, D. M. (2007). Copula Modeling: An Introduction Van der Auweraer, S., & Boute, R. (2019). Forecasting spare part demand
for Practitioners. Now Publishers Inc. using service maintenance information. International Journal of
Tsai, S.-B., Xue, Y., Zhang, J., Chen, Q., Liu, Y., Zhou, J., et al. (2017). Mod- Production Economics, 213, 138–149.
els for forecasting growth trends in renewable energy. Renewable van der Bles, A. M., van der Linden, S., Freeman, A. L., Mitchell, J.,
and Sustainable Energy Reviews, 77, 1169–1178. Galvao, A. B., Zaval, L., et al. (2019). Communicating uncertainty
Tsay, R. S. (1986). Time series model specification in the presence about facts, numbers and science. Royal Society Open Science, 6(5),
of outliers. Journal of the American Statistical Association, 81(393), Article 181870.
132–141. van der Bles, A. M., van der Linden, S., Freeman, A. L., & Spiegelhalter, D.
Tse, Y. K., & Tsui, A. K. C. (2002). A multivariate generalized J. (2020). The effects of communicating uncertainty on public
autoregressive conditional heteroscedasticity model with time- trust in facts and numbers. Proceedings of the National Academy of
varying correlations. Journal of Business & Economic Statistics, 20(3), Sciences, 117(14), 7672–7683.
351–362. van der Laan, E., van Dalen, J., Rohrmoser, M., & Simpson, R. (2016).
Tsyplakov, A. (2013). Evaluation of probabilistic forecasts: proper Demand forecasting and order planning for humanitarian logistics:
scoring rules and moments. SSRN:2236605. An empirical assessment. Journal of Operations Management, 45,
Tu, Y., Ball, M., & Jank, W. (2008). Estimating flight departure delay 114–122.
distributions – a statistical approach with long-term trend and van der Mark, L. B., van Wonderen, K. E., Mohrs, J., van Aalderen, W. M.,
short-term pattern. Journal of the American Statistical Association, ter Riet, G., & Bindels, P. J. (2014). Predicting asthma in preschool
103(481), 112–125. children at high risk presenting in primary care: development of
Tuljapurkar, S., & Boe, C. (1999). Validation, probability-weighted priors, a clinical asthma prediction score. Primary Care Respiratory Journal,
and information in stochastic forecasts. International Journal of 23(1), 52–59.
Forecasting, 15(3), 259–271. Van Dijk, D., Franses, P. H., & Lucas, A. (1999). Testing for smooth tran-
Turkman, M. A. A., & Turkman, K. F. (1990). Optimal alarm systems sition nonlinearity in the presence of outliers. Journal of Business &
for autoregressive processes: A Bayesian approach. Computational Economic Statistics, 17(2), 217–235.
Statistics & Data Analysis, 10(3), 307–314. Van Heerde, H. J., Leeflang, P. S., & Wittink, D. R. (2002). How
Turkmen, A. C., Wang, Y., & Januschowski, T. (2019). Intermittent promotions work: Scan* PRO-based evolutionary model building.
demand forecasting with deep renewal processes. arXiv:1911. Schmalenbach Business Review, 54(3), 198–220.
10416. van Ours, J. C. (2021). Common international trends in football stadium
Turner, D. S. (1990). The role of judgement in macroeconomic attendance. PLoS One, 16(3), Article e0247761.
forecasting. Journal of Forecasting, 9(4), 315–345. Van Reeth, D. (2019). Forecasting tour de France TV audiences: A
Turner, L., & Boulhol, H. (2011). Recent trends and structural breaks multi-country analysis. International Journal of Forecasting, 35(2),
in the US and EU15 labour productivity growth. Applied Economics, 810–821.
43(30), 4769–4784. Van Schaeybroeck, B., & Vannitsem, S. (2018). Postprocessing of long-
Turner, R., & Zolin, R. (2012). Forecasting success on large projects: range forecasts. In S. Vannitsem, D. S. Wilks, & J. W. Messner
Developing reliable scales to predict multiple perspectives by mul- (Eds.), Statistical Postprocessing of Ensemble Forecasts (pp. 267–290).
tiple stakeholders over multiple time frames. Project Management Amsterdam, Netherlands: Elsevier.
Journal, 43(5), 87–99. van Wingerden, E., Basten, R. J. I., Dekker, R., & Rustenburg, W. D.
Turnovsky, S. J., & Wachter, M. L. (1972). A test of the ‘‘expectations (2014). More grip on inventory control through improved forecast-
hypothesis’’ using directly observed wage and price expectations. ing: A comparative study at three companies. International Journal
The Review of Economics and Statistics, 54(1), 47–54. of Production Economics, 157, 220–237.
Twyman, M., Harvey, N., & Harries, C. (2008). Trust in motives, trust in Vannitsem, S., Wilks, D. S., & Messner, J. (2018). Statistical Postprocessing
competence: Separate factors determining the effectiveness of risk of Ensemble Forecasts. Elsevier.
communication. Judgment and Decision Making, 3(1), 111–120. Varian, H. R. (2014). Big data: New tricks for econometrics. Journal of
Tych, W., Pedregal, D. J., Young, P. C., & Davies, J. (2002). An unobserved Economic Perspectives, 28(2), 3–28.
component model for multi-rate forecasting of telephone call Vaughan Williams, L., & Reade, J. (2016). Prediction markets, social
demand: the design of a forecasting support system. International media and information efficiency. Kyklos, 69(3), 518–556.
Journal of Forecasting, 18(4), 673–695. Venkatramanan, S., Lewis, B., Chen, J., Higdon, D., Vullikanti, A., &
Tziafetas, G. (1986). Estimation of the voter transition matrix. Marathe, M. (2018). Using data-driven agent-based models for
Optimization, 17(2), 275–279. forecasting emerging infectious diseases. Epidemics, 22, 43–49.
Uematsu, H., Kunisawa, S., Sasaki, N., Ikai, H., & Imanaka, Y. (2014). Venter, J. H., De Jongh, P. J., & Griebenow, G. (2005). NIG-Garch models
Development of a risk-adjusted in-hospital mortality prediction based on open, close, high and low prices. South African Statistical
model for community-acquired pneumonia: a retrospective analysis Journal, 39(2), 79–101.
using a Japanese administrative database. BMC Pulmonary Medicine, Verhulst, P. (1838). Notice sur la loi que la population suit dans
14(1), 203. son accroissement. Correspondance Mathématique Et Physique, 10,
Ugurlu, U., Oksuz, I., & Tas, O. (2018). Electricity price forecasting using 113–121.
recurrent neural networks. Energies, 11(5), 1255. Verhulst, P. F. (1845). Recherches mathématiques sur la loi
Ülkümen, G., Fox, C. R., & Malle, B. F. (2016). Two dimensions of d’accroissement de la population. Nouveaux MÉMoires de L’AcadÉMie
subjective uncertainty: Clues from natural language. Journal of Royale Des Sciences Et Belles-Lettres de Bruxelles, 18, 14–54.
Experimental Psychology: General, 145(10), 1280–1297. Vermue, M., Seger, C. R., & Sanfey, A. G. (2018). Group-based bi-
Unwin, A. (2019). Multivariate outliers and the O3 plot. Journal of ases influence learning about individual trustworthiness. Journal of
Computational and Graphical Statistics, 28(3), 635–643. Experimental Social Psychology, 77, 36–49.
868
Veronesi, P., & Yared, F. (1999). Short and long horizon term and Wang, X., Smith-Miles, K., & Hyndman, R. J. (2006). Characteristic-
inflation risk premia in the US term structure: Evidence from an based clustering for time series data. Data Mining and Knowledge
integrated model for nominal and real bond prices under regime Discovery, 13(3), 335–364.
shifts. CRSP Working Paper, 508. Wang, X., Smith-Miles, K., & Hyndman, R. J. (2009). Rule induction for
Vestergaard, L. S., Nielsen, J., Richter, L., Schmid, D., Bustos, N., forecasting method selection: meta-learning the characteristics of
Braeye, T., et al. (2020). Excess all-cause mortality during the univariate time series. Neurocomputing, 72(10–12), 2581–2594.
COVID-19 pandemic in europe–preliminary pooled estimates from Wang, Y., Smola, A., Maddix, D., Gasthaus, J., Foster, D., &
the euromomo network, march to april 2020. Eurosurveillance, Januschowski, T. (2019). Deep factors for forecasting. In Interna-
25(26), Article 2001214. tional Conference on Machine Learning (pp. 6607–6617).
Vile, J. L., Gillard, J. W., Harper, P. R., & Knight, V. A. (2012). Predicting Wang, W., & Syntetos, A. A. (2011). Spare parts demand: Linking
ambulance demand using singular spectrum analysis. Journal of the forecasting to equipment maintenance. Transportation Research Part
Operational Research Society, 63(11), 1556–1565. E: Logistics and Transportation Review, 47(6), 1194–1209.
Villegas, M. A., & Pedregal, D. J. (2018). Supply chain decision sup- Wang, J., Wang, Z., Li, X., & Zhou, H. (2019). Artificial bee colony-based
port systems based on a novel hierarchical forecasting approach. combination approach to forecasting agricultural commodity prices.
Decision Support Systems, 114, 29–36. International Journal of Forecasting.
Vipul, & Jacob, J. (2007). Forecasting performance of extreme-value Wang, Z., Wang, W., Liu, C., Wang, Z., & Hou, Y. (2017). Probabilistic
volatility estimators. Journal of Futures Markets, 27(11), 1085–1105. forecast for multiple wind farms based on regular vine copulas.
Vitart, F., Robertson, A. W., & Anderson, D. L. T. (2012). Subseasonal to IEEE Transactions on Power Systems, 33(1), 578–589.
seasonal prediction project: Bridging the gap between weather and Wang, Z., Wang, Y., Zeng, R., Srinivasan, R. S., & Ahrentzen, S. (2018).
climate. Bulletin of the World Meteorological Organization, 61(2), 23. Random forest based hourly building energy prediction. Energy and
Vlahogianni, E. I., Golias, J. C., & Karlaftis, M. G. (2004). Short-term Buildings, 171, 11–25.
traffic forecasting: Overview of objectives and methods. Transport Wang, J., Yang, W., Du, P., & Niu, T. (2018). A novel hybrid forecasting
Reviews, 24(5), 533–557. system of wind speed based on a newly developed multi-objective
Vlahogianni, E. I., Karlaftis, M. G., & Golias, J. C. (2014). Short- sine cosine algorithm. Energy Conversion and Management, 163,
term traffic forecasting: Where we are and where we’re going. 134–150.
Transportation Research Part C (Emerging Technologies), 43, 3–19. Wang, J., Yang, W., Du, P., & Niu, T. (2020). Outlier-robust hybrid elec-
Volterra, V. (1926a). Fluctuations in the abundance of a species tricity price forecasting model for electricity market management.
considered mathematically. Nature, 118, 558–560. Journal of Cleaner Production, 249, Article 119318.
Wang, H., & Yeung, D.-Y. (2016). A survey on Bayesian deep learning.
Volterra, V. (1926b). Variazioni e fluttuazioni del numero d’individui in
arXiv:1604.01662.
specie animali conviventi. Memoria Della Reale Accademia Nazionale
Warne, A., Coenen, G., & Christoffel, K. (2010). Forecasting with DSGE
Dei Lincei, 2, 31–113.
models. (1185), European Central Bank.
Wakefield, J. (2004). Ecological inference for 2x2 tables (with dis-
Wasserstein, R. L., & Lazar, N. A. (2016). The ASA statement on p-
cussion). Journal of the Royal Statistical Society, Series A, 167(3),
values: Context, process, and purpose. The American Statistician,
385–445.
70(2), 129–133.
Wallentin, G., Kaziyeva, D., & Reibersdorfer-Adelsberger, E. (2020).
Weatherford, L., Gentry, T., & Wilamowski, B. (2003). Neural network
COVID-19 intervention scenarios for a long-term disease manage-
forecasting for airlines: A comparative analysis. Journal of Revenue
ment. International Journal of Health Policy and Management.
and Pricing Management, 1(4), 319–331.
Wallström, P., & Segerstedt, A. (2010). Evaluation of forecasting
Weaver, W. T. (1971). The delphi forecasting method. The Phi Delta
error measurements and techniques for intermittent demand.
Kappan, 52(5), 267–271.
International Journal of Production Economics, 128(2), 625–636.
Webby, R., O’Connor, M., & Edmundson, B. (2005). Forecasting support
Walton, D., Reed, C., & Macagno, F. (2008). Argumentation Schemes.
systems for the incorporation of event information: An empirical
Cambridge University Press.
investigation. International Journal of Forecasting, 21(3), 411–423.
Wang, C., Jiang, B., Fan, J., Wang, F., & Liu, Q. (2014). A study of the
Wei, W., & Hansen, M. (2006). An aggregate demand model for air
dengue epidemic and meteorological factors in guangzhou, China,
passenger traffic in the hub-and-spoke network. Transportation
by using a zero-inflated Poisson regression model. Asia Pacific
Research Part A: Policy and Practice, 40(10), 841–851.
Journal of Public Health, 26(1), 48–57.
Wei, W., & Held, L. (2014). Calibration tests for count data. Test, 23,
Wang, X., Kang, Y., Hyndman, R. J., & Li, F. (2020). Distributed ARIMA 787–805.
models for ultra-long time series. arXiv:2007.09577. Wei, N., Li, C., Peng, X., Zeng, F., & Lu, X. (2019). Conventional models
Wang, X., Kang, Y., Petropoulos, F., & Li, F. (2021). The uncertainty and artificial intelligence-based models for energy consumption
estimation of feature-based forecast combinations. Journal of the forecasting: A review. Journal of Petroleum Science and Engineering,
Operational Research Society. 181, Article 106187.
Wang, H., Li, B., & Leng, C. (2009). Shrinkage tuning parameter selec- Wei Su, C., Wang, X.-Q., Tao, R., & Oana-Ramona, L. (2019). Do oil prices
tion with a diverging number of parameters. Journal of the Royal drive agricultural commodity prices? Further evidence in a global
Statistical Society. Series B. Statistical Methodology, 71(3), 671–683. bio-energy context. Energy, 172, 691–701.
Wang, P., Liu, B., & Hong, T. (2016). Electric load forecasting with Weinberg, J., Brown, L. D., & Stroud, J. R. (2007). BayesIan forecasting
recency effect: A big data approach. International Journal of of an inhomogeneous Poisson process with applications to call
Forecasting, 32(3), 585–597. center data. Journal of the American Statistical Association, 102(480),
Wang, S. L., & McPhail, L. (2014). Impacts of energy shocks on US agri- 1185–1198.
cultural productivity growth and commodity prices—A structural Weiß, C. H., Homburg, A., Alwan, L. C., Frahm, G., & Göb, R. (2021). Effi-
VAR analysis. Energy Economics, 46(C), 435–444. cient accounting for estimation uncertainty in coherent forecasting
Wang, C.-N., Nhieu, N.-L., Chung, Y.-C., & Pham, H.-T. (2021). of count processes. Journal of Applied Statistics, 1–22.
Multi-objective optimization models for sustainable perishable Weiß, G. N., & Supper, H. (2013). Forecasting liquidity-adjusted intraday
intermodal multi-product networks with delivery time window. value-at-risk with vine copulas. Journal of Banking & Finance, 37(9),
Mathematics, 9(4), 379. 3334–3350.
Wang, W., Pedrycz, W., & Liu, X. (2015). Time series long-term fore- Wen, R., Torkkola, K., Narayanaswamy, B., & Madeka, D. (2017). A
casting model based on information granules and fuzzy clustering. multi-horizon quantile recurrent forecaster. arXiv:1711.11053.
Engineering Applications of Artificial Intelligence, 41, 17–24. Weron, R. (2014). Electricity price forecasting: A review of the state-
Wang, X., & Petropoulos, F. (2016). To select or to combine? The of-the-art with a look into the future. International Journal of
inventory performance of model and expert forecasts. International Forecasting, 30(4), 1030–1081.
Journal of Productions Research, 54(17), 5271–5282. West, K. D. (1996). Asymptotic inference about predictive ability.
Wang, W., Rothschild, D., Goel, S., & Gelman, A. (2015). Forecasting Econometrica, 1067–1084.
elections with non-representative polls. International Journal of White, H. (2000). A reality check for data snooping. Econometrica, 68(5),
Forecasting, 31(3), 980–991. 1097–1126.
869
Whitt, W., & Zhang, X. (2019). Forecasting arrivals and occupancy levels Wright, G., & Goodwin, P. (1999). Future-focussed thinking: combining
in an emergency department. Operations Research for Health Care, scenario planning with decision analysis. Journal of Multi-Criteria
21, 1–18. Decision Analysis, 8(6), 311–321.
Whittaker, J., Garside, S., & Lindveld, K. (1997). Tracking and predicting Wright, G., & Goodwin, P. (2009). Decision making and planning
a network traffic process. International Journal of Forecasting, 13(1), under low levels of predictability: Enhancing the scenario method.
51–61. International Journal of Forecasting, 25(4), 813–825.
Wicke, L., Dhami, M. K., Önkal, D., & Belton, I. K. (2019). Using scenarios Wright, M. J., & Stern, P. (2015). Forecasting new product trial with
to forecast outcomes of a refugee crisis. International Journal of analogous series. Journal of Business Research, 68(8), 1732–1738.
Forecasting. Wu, S., & Chen, R. (2007). Threshold variable determination and thresh-
Wickham, R. (1995). Evaluation of forecasting techniques for short-term old variable driven switching autoregressive models. Statistica
demand of air transportation. (Ph.D. thesis), Massachusetts Institute Sinica, 17(1), 241–S38.
of Technology. Wu, C. C., & Liang, S. S. (2011). The economic value of range-based
Wickramasuriya, S. L., Athanasopoulos, G., & Hyndman, R. J. (2019). covariance between stock and bond returns with dynamic copulas.
Optimal forecast reconciliation for hierarchical and grouped time Journal of Empirical Finance, 18(4), 711–727.
series through trace minimization. Journal of the American Statistical Wu, W., Ma, X., Zeng, B., Wang, Y., & Cai, W. (2019). Forecasting
Association, 114(526), 804–819. short-term renewable energy consumption of China using a novel
fractional nonlinear grey Bernoulli model. Renewable Energy, 140,
Wilde, J., Chen, W., & Lohmann, S. (2020). COVID-19 and the future of
70–87.
US fertility: what can we learn from google?. (WP-2020-034), Max
Wu, Y., Yu, W., Cui, Y., & Lu, C. (2020). Data integrity attacks against
Planck Institute for Demographic Research, Rostock, Germany.
traffic modeling and forecasting in M2m communications. In ICC
Wilkd, D. S. (2005). Science & Technology, Statistical Methods in the
2020-2020 IEEE International Conference on Communications (ICC)
Atmospheric Sciences (2nd). Elsevier.
(pp. 1–6). IEEE.
Wilks, D. S. (2004). The minimum spanning tree histogram as verifica-
Xiao, Y., & Han, J. (2016). Forecasting new product diffusion with
tion tool for multidimensional ensemble forecasts. Montly Weather
agent-based models. Technological Forecasting and Social Change,
Review, 132, 1329–1340.
105, 167–178.
Wilks, D. S. (2019). Indices of rank histogram flatness and their Xie, Y. (2000). Demography: Past, present, and future. Journal of the
sampling properties. Monthly Weather Review, 147(2), 763–769. American Statistical Association, 95(450), 670–673.
Willekens, F. (2018). Towards causal forecasting of international Xie, T., & Ding, J. (2020). Forecasting with multiple seasonality. arXiv:
migration. Vienna Yearbook of Population Research, 16, 199–218. 2008.12340.
Willemain, T. R., Smart, C. N., & Schwarz, H. F. (2004). A new approach Xie, J., & Hong, T. (2016). Gefcom2014 probabilistic electric load
to forecasting intermittent demand for service parts inventories. forecasting: An integrated solution with forecast combination
International Journal of Forecasting, 20(3), 375–387. and residual simulation. International Journal of Forecasting, 32(3),
Williams, L. V., & Reade, J. J. (2016). Forecasting elections. Journal of 1012–1016.
Forecasting, 35(4). Xie, W., Yu, L., Xu, S., & Wang, S. (2006). A new method for crude oil
Wilms, I., Rombouts, J., & Croux, C. (2021). Multivariate volatility fore- price forecasting based on support vector machines. In International
casts for stock market indices. International Journal of Forecasting, Conference on Computational Science (pp. 444–451). Springer.
37(2), 484–499. Xiong, T., Li, C., Bao, Y., Hu, Z., & Zhang, L. (2015). A combination
Wind, Y. (Ed.), (1981). New product forecasting: models and method for interval forecasting of agricultural commodity futures
applications. Lexington, MA: Lexington Books. prices. Knowledge-Based Systems, 77, 92–102.
Winkler, R. L. (1972). A decision-theoretic approach to interval es- Xu, W. (1999). Long range planning for call centers at fedex. The Journal
timation. Journal of the American Statistical Association, 67(337), of Business Forecasting, 18(4), 7.
187–191. Xu, Y., Liu, H., & Long, Z. (2020). A distributed computing framework
Winkler, R. L., Grushka-Cockayne, Y., Lichtendahl, K. C., & Jose, V. R. for wind speed big data forecasting on apache spark. Sustainable
R. (2019). Probability forecasts and their combination: A research Energy Technologies and Assessments, 37, Article 100582.
perspective. Decision Analysis, 16(4), 239–260. Xu, R., & Wunsch, D. (2005). Survey of clustering algorithms. IEEE
Winkler, R. L., Muñoz, J., Cervera, J. L., Bernardo, J. M., Blattenberger, G., Transactions on Neural Networks, 16(3), 645–678.
Kadane, J. B., et al. (1996). Scoring rules and the evaluation of Yaffee, R. A., Nikolopoulos, K., Reilly, D. P., Crone, S. F., Wagoner, K.
probabilities. Test, 5(1), 1–60. D., Douglas, R. J., et al. (2011). An experiment in epidemiological
Winters, P. R. (1960). Forecasting sales by exponentially weighted forecasting: A comparison of forecast accuracies among differ-
moving averages. Management Science, 6(3), 324–342. ent methods of forecasting deer mouse population densities in
montana. In Federal Forecaster’s Brown Bag Lunch.
Wiśniowski, A., Smith, P. W. F., Bijak, J., Raymer, J., & Forster, J. J. (2015).
Yagli, G. M., Yang, D., & Srinivasan, D. (2019). Reconciling solar
BayesIan population forecasting: Extending the lee-carter method.
forecasts: Sequential reconciliation. Solar Energy, 179, 391–397.
Demography, 52(3), 1035–1059.
Yan, X. S., & Zheng, L. (2017). Fundamental analysis and the cross-
Wold, H. (1966). Estimation of principal components and related
section of stock returns: A data-mining approach. Review of
models by iterative least squares. In P. Krishnajah (Ed.), Multivariate
Financial Studies, 30(4), 1382–1423.
Analysis (pp. 391–420). New York: Academic Press.
Yang, Z., Zeng, Z., Wang, K., Wong, S.-S., Liang, W., Zanin, M., et al.
Wolfers, J., & Zitzewitz, E. (2004). Prediction markets. Journal of
(2020). Modified SEIR and AI prediction of the epidemics trend
Economic Perspectives, 18(2), 107–126.
of COVID-19 in China under public health interventions. Journal
Wolpert, D. H., & Macready, W. G. (1997). No free lunch theorems for of Thoracic Disease, 12(3), 165–174.
optimization. IEEE Transactions on Evolutionary Computation, 1(1), Yang, D., & Zhang, Q. (2000). Drift-independent volatility estimation
67–82. based on high, low, open, and close prices. Journal of Business, 73(3),
Wolters, M. H. (2015). Evaluating point and density forecasts of DSGE 477–491.
models. Journal of Applied Econometrics, 30(1), 74–96. Yassine, A., Shirehjini, A. N., & Shirmohammadi, S. (2015). Smart meters
Wong, B. K., Bodnovich, T. A., & Selvi, Y. (1995). A bibliography of big data: Game theoretic model for fair data sharing in deregulated
neural network business applications research: 1988-september smart grids. IEEE Access, 3, 2743–2754.
1994. Expert Systems, 12(3), 253–261. Yelland, P., Baz, Z. E., & Serafini, D. (2019). Forecasting at scale: The
Wong-Fupuy, C., & Haberman, S. (2004). Projecting mortality trends: architecture of a modern retail forecasting system. Foresight: The
recent developments in the United Kingdom and the United States. International Journal of Applied Forecasting, 55, 10–18.
North American Actuarial Journal, 8(2), 56–83. Yue, M. (2017). An integrated anomaly detection method for load
Woodford, M. (2002). Imperfect common knowledge and the effects of forecasting data under cyberattacks. In 2017 IEEE Power & Energy
monetary policy. In P. Aghion, R. Frydman, J. Stiglitz, & M. Wood- Society General Meeting (pp. 1–5). IEEE.
ford (Eds.), Knowledge, Information, and Expectations in Modern Yue, M., Hong, T., & Wang, J. (2019). Descriptive analytics-
Macroeconomics: in Honor of Edmund Phelps (pp. 25–58). Princeton based anomaly detection for cybersecure load forecasting. IEEE
University Press. Transactions on Smart Grid, 10(6), 5964–5974.
870
Yusupova, A., Pavlidis, N. G., & Pavlidis, E. G. (2019). Adaptive dynamic Zhang, H., Song, H., Wen, L., & Liu, C. (2021). Forecasting tourism
model averaging with an application to house price forecasting. recovery amid COVID-19. Annals of Tourism Research, 87, Article
arXiv:1912.04661. 103149.
Yusupova, A., Pavlidis, E., Paya, I., & Peel, D. (2020). UK housing Zhang, Y., & Wang, J. (2018). A distributed approach for wind power
price uncertainty index (HPU). UK Housing Observatory, Dept. of probabilistic forecasting considering spatio-temporal correlation
Economics, Lancaster University Management School. without directaccess to off-site information. IEEE Transactions on
Zadeh, L. (1965). Fuzzy sets. Information and Control, 8(3), 338–353. Power Systems, 33(5), 5714–5726.
Zagdański, A. (2001). Prediction intervals for stationary time series Zhang, Y., Wang, J., & Wang, X. (2014). Review on probabilistic
using the sieve bootstrap method. Demonstratio Mathematica, 34(2), forecasting of wind power generation. Renewable and Sustainable
257–270. Energy Reviews, 32, 255–270.
Zaharia, M., Xin, R. S., Wendell, P., Das, T., Armbrust, M., Dave, A., et Zhang, J., Wei, Y.-M., Li, D., Tan, Z., & Zhou, J. (2018). Short term
al. (2016). Apache Spark: A unified engine for big data processing. electricity load forecasting using a hybrid model. Energy, 158,
Communications of the ACM, 59(11), 56–65. 774–781.
Zaidi, A., Harding, A., & Williamson, P. (Eds.), (2009). New frontiers in Zhang, L., Zhou, W.-D., Chang, P.-C., Yang, J.-W., & Li, F.-Z. (2013). Iter-
microsimulation modelling. Farnham: Ashgate. ated time series prediction with multiple support vector regression
Zailani, S., Jeyaraman, K., Vengadasan, G., & Premkumar, R. (2012). Sus- models. Neurocomputing, 99, 411–422.
tainable supply chain management (SSCM) in Malaysia: A survey. Zhang, W., Zou, Y., Tang, J., Ash, J., & Wang, Y. (2016). Short-term
International Journal of Production Economics, 140(1), 330–340. prediction of vehicle waiting queue at ferry terminal based on
Zaki, M. J. (2000). Scalable algorithms for association mining. IEEE machine learning method. Journal of Marine Science and Technology,
Transactions on Knowledge and Data Engineering, 12(3), 372–390. 21(4), 729–741.
Zakoian, J.-M. (1994). Threshold heteroskedastic models. Journal of Zhao, H.-X., & Magoulès, F. (2012). A review on the prediction of
Economic Dynamics and Control, 18(5), 931–955. building energy consumption. Renewable and Sustainable Energy
Zang, H., Cheng, L., Ding, T., Cheung, K. W., Liang, Z., Wei, Z., et al. Reviews, 16(6), 3586–3592.
(2018). Hybrid method for short-term photovoltaic power forecast- Zheng, J., Xu, C., Zhang, Z., & Li, X. (2017). Electric load forecasting in
ing based on deep convolutional neural network. IET Generation, smart grids using long-short-term-memory based recurrent neural
Transmission and Distribution, 12(20), 4557–4567. network. In 2017 51st Annual Conference on Information Sciences and
Zarnowitz, V. (1985). Rational expectations and macroeconomic Systems (CISS) (pp. 1–6). IEEE.
forecasts. Journal of Business & Economic Statistics, 3(4), 293–311. Zhou, Z., & Matteson, D. S. (2016). Predicting melbourne ambulance
Zelterman, D. (1993). A semiparametric bootstrap technique for sim- demand using kernel warping. The Annals of Applied Statistics, 10(4),
ulating extreme order statistics. Journal of the American Statistical 1977–1996.
Association, 88(422), 477–485. Zhou, C., & Viswanathan, S. (2011). Comparison of a new bootstrapping
Zhang, S., Bauer, N., Yin, G., & Xie, X. (2020). Technology learning and method with parametric approaches for safety stock determina-
diffusion at the global and local scales: A modeling exercise in the tion in service parts inventory systems. International Journal of
REMIND model. Technological Forecasting and Social Change, 151, Production Economics, 133(1), 481–485.
Article 119765. Zhou, L., Zhao, P., Wu, D., Cheng, C., & Huang, H. (2018). Time series
Zhang, J. L., & Bryant, J. (2019). Combining multiple imperfect data model for forecasting the number of new admission inpatients. BMC
sources for small area estimation: a Bayesian model of provincial Medical Informatics and Decision Making, 18(1), 39.
fertility rates in cambodia. Statistical Theory and Related Fields, 3(2), Zhu, S., Dekker, R., van Jaarsveld, W., Renjie, R. W., & Koning, A.
178–185. J. (2017). An improved method for forecasting spare parts de-
Zhang, J.-L., Chen, J., & Lee, C.-Y. (2008). Joint optimization on pric- mand using extreme value theory. European Journal of Operational
ing, promotion and inventory control with stochastic demand. Research, 261(1), 169–181.
International Journal of Production Economics, 116(2), 190–198. Ziegel, J. F., & Gneiting, T. (2014). Copula calibration. Electronic Journal
Zhang, G., Eddy Patuwo, B., & Y. Hu, M. (1998). Forecasting with of Statistics, 8(2), 2619–2638.
artificial neural networks:: The state of the art. International Journal Ziel, F., & Berk, K. (2019). Multivariate forecasting evaluation: On
of Forecasting, 14(1), 35–62. sensitive and strictly proper scoring rules. arXiv:1910.07325.
Zhang, X., & Hutchinson, J. (1994). Simple architectures on fast ma- Ziel, F., & Steinert, R. (2016). Electricity price forecasting using sale and
chines: practical issues in nonlinear time series prediction. In purchase curves: The X-Model. Energy Economics, 59, 435–454.
A. S. Weigend, & N. A. Gershenfeld (Eds.), Time Series Prediction Ziel, F., & Steinert, R. (2018). Probabilistic mid-and long-term electricity
Forecasting the Future and Understanding the Past (pp. 219–241). price forecasting. Renewable and Sustainable Energy Reviews, 94,
Addison-Wesley, Santa Fe Institute. 251–266.
Zhang, & Ming (2008). Artificial higher order neural networks for Ziel, F., & Weron, R. (2018). Day-ahead electricity price forecasting with
economics and business. IGI Global. high-dimensional structures: Univariate vs. multivariate modeling
Zhang, Y., & Nadarajah, S. (2018). A review of backtesting for value frameworks. Energy Economics, 70, 396–420.
at risk. Communications in Statistics. Theory and Methods, 47(15), Zipf, G. K. (2016). Human Behavior and the Principle of Least Effort: An
3616–3639. Introduction To Human Ecology. Ravenio Books.
Zhang, X., Peng, Y., Zhang, C., & Wang, B. (2015). Are hybrid models Žmuk, B., Dumičić, K., & Palić, I. (2018). Forecasting labour productivity
integrated with data preprocessing techniques suitable for monthly in the European union member states: Is labour productivity chang-
streamflow forecasting? Some experiment evidences. Journal of ing as expected? Interdisciplinary Description of Complex Systems,
Hydrology, 530, 137–152. 16(3-B), 504–523.
Zhang, G. P., & Qi, M. (2005). Neural network forecasting for seasonal Zou, H., & Hastie, T. (2005). Regularization and variable selection via the
and trend time series. European Journal of Operational Research, elastic net. Journal of the Royal Statistical Society. Series B. Statistical
160(2), 501–514. Methodology, 67, 301–320.
Zhang, W., Qi, Y., Henrickson, K., Tang, J., & Wang, Y. (2017). Vehicle Zwijnenburg, J. (2015). Revisions of quarterly GDP in selected OECD
traffic delay prediction in ferry terminal based on Bayesian multiple countries. OECD Statistics Briefing, July 2015 - No. 22, 1–12.
models combination method. Transportmetrica A: Transport Science,
13(5), 467–490.
871

Forecasting - Theory and Practice

Uploaded by

Copyright:

Available Formats

Forecasting - Theory and Practice

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Forecasting - Theory and Practice

Uploaded by

Copyright:

Available Formats

International Journal of Forecasting 38 (2022) 705–871

Contents lists available at ScienceDirect

International Journal of Forecasting