A new approach for detecting shifts in forecast accuracy

Ching Wai (Jeremy) Chiu*, Simon Hayes, George Kapetanios, Konstantinos Theodoridis

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

1 Citation (Scopus)

Abstract

Forecasts play a critical role at inflation-targeting central banks, such as the Bank of England. Breaks in the forecast performance of a model can potentially incur important policy costs. However, commonly-used statistical procedures implicitly place a lot of weight on type I errors (or false positives), which results in a relatively low power of the tests to identify forecast breakdowns in small samples. We develop a procedure which aims to capture the policy cost of missing a break. We use data-based rules to find the test size that optimally trades off the costs associated with false positives with those that can result from a break going undetected for too long. In so doing, we also explicitly study forecast errors as a multivariate system. The covariance between forecast errors for different series, although often overlooked in the forecasting literature, not only enables us to consider testing in a multivariate setting, but also increases the test power. As a result, we can tailor our choice of the critical values for each series not only to the in-sample properties of each series, but also to the way in which the series of forecast errors covary.

Original languageEnglish
Pages (from-to)1596-1612
Number of pages17
JournalINTERNATIONAL JOURNAL OF FORECASTING
Volume35
Issue number4
Early online date24 May 2019
DOIs
Publication statusPublished - 1 Oct 2019

Keywords

  • Central banking
  • Forecast breaks
  • Hypothesis testing with small sample sizes
  • Optimal test sizes
  • Statistical decision making

Fingerprint

Dive into the research topics of 'A new approach for detecting shifts in forecast accuracy'. Together they form a unique fingerprint.

Cite this