To be submitted ELECTRONICALLY via Moodle
before 15.00 pm on Wednesday the 10th of January 2018
This assignment carries 70% of your total mark in this unit
The Task
The objective of this coursework is to propose and build a framework for batch forecasting of fast
moving time series. Once a set of suitable forecasting models is identified (Part A), you are asked to
propose a model selection strategy that will automatically choose the most appropriate forecasting
model for each time series individually (Part B). The proposed strategy should then be applied to all
time series. Finally, performance evaluation (Part C) and residuals diagnostics (Part D) should be
carried out.
The Data
Using the library Mcomp of the R statistical software, consider the quarterly time series of the M3-
Competition with IDs within [701, 1400], so that the last digit of the series ID matches the last digit of
your Student ID. For example, if your Student ID is 1456789, then you should select all the series with
IDs finishing at 9, that means: 709, 719, 729, 739, ..., 1399. Following this procedure, you should end
up with a set of 70 quarterly time series. You should be able to access a single time series (e.g. the
time series 709) using the command M3[[709]]. Note that each time series is split in an in-sample
(M3[[709]]$x) and an out-of-sample (M3[[709]]$xx) set of observations. Other useful variables
include the size of the in-sample (M3[[709]]$n), the size of the out-of-sample (equal to the
required forecast horizon, M3[[709]]$h), and the category of the data (micro, macro, industry,
demographic, finance or other, M3[[709]]$type). For simplicity, the length of the out-of-sample
set is always 8 quarters. You are expected to use only the in-sample set in order to generate statistical
forecasts for the out-of-sample set (forecasting horizon equal to 8 periods/ quarters). Then,
forecasting performance should be evaluated by comparing the produced forecasts with the withheld
out-of-sample set of observations.
The data is also provided in an Excel file (MN50640CourseworkData.xls). The file is arranged as follows:
column 1 refers to the time series ID (701, 702, 703, …, 1400); column 2 provides the number of
observations of the in-sample set; column 3 provides the number of observations of the out-of-sample
set (equal to the required forecast horizon); column 4 indicates the category that the data come from
(micro, macro, industry, demographic, …); column 5 and 6 indicate the year and quarter of the first
observation, respectively; columns 7 onwards provide the data values. Note that the number of data
values equals to the sum of lengths of the in-sample and out-of-sample.
Part A: Select a suitable toolbox of forecasting models
Your toolbox should contain at least one (1) time series regression model, at least three (3) exponential
smoothing models and at least two (2) ARIMA models. More methods/models may be considered, up
MN50640: Business Statistics and Forecasting
Coursework (2017-2018)
to a maximum of ten (10) models in total. The selected models should be able to capture collectively
different underlying time series characteristics (level, trend, seasonality). A full justification of the
selected models should be provided.
Part B: Select and apply a suitable model selection strategy
Using only the in-sample data, propose a suitable strategy in order to select for each series individually
the most suitable forecasting model. Justify the selection of this model selection strategy over other
model selection strategies for forecasting. Apply the proposed model selection strategy to the data in
order to generate forecasts for the out-of-sample periods. You are strongly advised to consider
methodologies such as validation and/or cross-validation that could be applied across a range of
methods/models. The selection of the validation/cross-validation windows lengths and the number of
cross-validation steps should be justified.
Part C: Performance evaluation
Evaluate the forecasts produced in the previous step using at least three appropriate error measures.
Evaluation should be carried out across time series and across horizons. Justify the selection of these
error measures over other possible candidates. Compare the accuracy of the proposed selection
strategy with that of three (3) suitable benchmarks (for example, Naïve or Damped Exponential
Smoothing) when each of these is applied across all series. Was the application of the proposed model
selection strategy successful for this set of data? Critically discuss. For extra marks, consider the
decomposition of the analysis to different planning horizons (short-, medium- and long-term),
decomposition of the results with regards time series characteristics (trend and/or seasonality) as well
as decomposition for the different categories of data (micro, macro, industry, demographic, finance
or other).
Part D: Residuals diagnostics
Select the three time series in your set where 1201 < ID < 1230 (for example, if the last digit of your ID
is 9, then the three time series should be 1209, 1219 and 1229) and perform. residuals diagnostics for
the selected “optimal” and two more methods (one being exponential smoothing and the other a
regression model).
MARKING CRITERIA
Your report should have six (6) sections. The first (1st) section should be a very brief introduction, and
the last one (6th) should briefly summarise your main findings. The four sections in the middle, 2nd, 3rd,
4th and 5th should be devoted to Parts A, B, C and D (outlined above) respectively.
These sections are shown below with the associated marking weights:
1. Introduction (5%)
2. Part 2 – Select a suitable toolbox of forecasting models (20%)
3. Part 3 – Select and apply a suitable model selection strategy (25%)
4. Part 4 – Performance evaluation (30%)
5. Part 5 – Residuals diagnostics (15%)
6. Conclusions (5%)
You can also include a short (half-page) executive summary.
Please note that in all the above parts the quality of presentation, critical discussion and appropriate
references to the literature will be taken explicitly into account towards the mark to be allocated.
Your report should be a detailed description of everything you have done. With regards to the
references please follow the Harvard style. The word count should be 4,000 words. A small deviation
(±10%) is allowed, meaning that submissions within the range of 3,600-4,400 words will not be
penalised. If the assignment is judged to exceed the word limit excessively, then the standard
University penalty will apply. Please see here for what is included and what is excluded from the word
count: http://www.bath.ac.uk/internal/management/assessment/pdf/Section7.pdf
GUIDELINES
• All coursework must be submitted ELECTRONICALLY via Moodle before 15.00 pm on
Wednesday the 10th of January 2018. No paper or email submission will be accepted.
• You are advised to conduct your analysis using the R statistical software and take advantage
of the pre-implemented functions for the different forecasting methods. However, if you wish
to conduct part of your analysis using a different software (such as Excel), you are free to do
so. In such a case, you might find the write.table() function useful. This function allows
the export of data in comma separated value (csv) files that are subsequently easy to open
with alternative software (such as Excel or SPSS). The choice of the software to conduct your
analysis will not affect your final grade.
• The (quantitative) analysis should also be submitted as an accompanying file(s) (R scripts or
any Excel files used).
• Please do not e-mail any such files to us as we will have to disregard them. The only things
that we will take into account are the contents of your formal submission.
• Please use the provided cover page for the your submission.
• Please ensure that you read and follow the rules on plagiarism and unfair practice in your
University Handbook. This assignment must be your own individual work. Any suspicion of
copying from other student, the Internet or other sources will be considered by the unfair
practice board very seriously.
Best of luck – we are sure you will do well.
Fotios Petropoulos & Jooyoung Jeon November 15, 2017
Key references
1. Hyndman R.J., Koehler A.B., Snyder R.D. & Grose S. (2002) “A state space framework for automatic
forecasting using exponential smoothing methods”, International Journal of Forecasting, 18, 439-
454
2. Gardner E.S. (2006) “Exponential smoothing: the state of the art – Part II”, International Journal of
Forecasting, 22, 637-666
3. Hyndman R.J. and Khandakar Y. (2008) “Automatic time series forecasting: the forecast package
for R”, Journal of Statistical Software, 27(3), 1-22
4. Makridakis S. (1990) “Sliding simulation: a new approach to time series forecasting”, Management
Science, 36, 505–512
5. Tashman L.J. (2000) “Out-of-sample tests of forecasting accuracy: an analysis and review”,
International Journal of Forecasting, 16, 437-450
6. Petropoulos F., Makridakis S., Assimakopoulos V. & Nikolopoulos K. (2014) “‘Horses for Courses’ in
demand forecasting”, European Journal of Operational Research, 237(1), 152-163
7. Fildes R. & Petropoulos F. (2015) “Simple versus complex selection rules for forecasting many time
series”, Journal of Business Research, 68(8), 1692-1701
8. Hyndman R.J. & Koehler A.B. (2006) “Another look at measures of forecast accuracy”, International
Journal of Forecasting, 22, 679-688
9. Davydenko A. & Fildes R. (2013) “Measuring forecasting accuracy: The case of judgmental
adjustments to SKU-level demand forecasts”, International Journal of Forecasting, 29, 510-522