done Econ6037: Economic Forecasting
University of Hong Kong
Project #2 – Forecasting commodity prices (VAR + forecast evaluation)
Due date: Friday, April 09, 11:30p.m. (via the course website)
A note from the instructor
1. Please pay attention to the long instructions below. They are all here for good reasons!
2. This assignment is meant to be completed individually. Communication with, and hence
learning from, classmates is strongly encouraged. Caution, however, too much reliance on our
classmates for help diminishes the amount of our learning from the assignment. Each student
is expected to collect his/her own data, write or modify the R-scripts to suit the purpose of
the assignment, conduct his/her own analysis and write up this/her own report. Remember:
Give a man (your classmate) a fish, and you feed him for a day.
Teach a man (your classmate) to fish, and you feed him for a lifetime.
3. To enrich our understanding of the world, no students are allowed to work on the same data
set.
4. To some students, it might appear easier and more convenient to use Excel or to use R interactively for the computational part of this assignment. Here, aside from some very basic
data manipulation using Excel, students should use R to do most of the calculation as much
as possible. Try to write a short program/script of R for the task, with annotations so that
readers of the R script will know your programming logic. Points will be deducted if you do
not use R to generate the graphs and statistics.
5. Always try to write the report in a self-contained way and in a style that you would be happy
to show to your current or potential employer.
6. Start early. This assignment is very demanding, especially if you are not familiar with R!
7. Make sure your report is black-and-white printer friendly. For grading, I almost always print
out the reports using a black and white printer. Keep in mind that colors will not show on a
black and white printout.
Page 1 of 6
We are interested in forecasting commodity prices (call it yt) using data of monthly frequency. Before
you start, please indicate your choice of commodity in the corresponding “Choice” activity of our
course website. Note that no students are allowed to work on the same commodity. Data at monthly
frequency are available from various sources, Datastream, Bloomberg, IMF, etc.
The required statistical analysis of the project should consist of two parts: (1) Choice of best models
based on in-sample selection criteria, and (2) Evaluation of the out-of-sample performance of different
models, and consequently conclude whether in-sample selection criteria and out-of-sample criteria will
yield the same best model.
1. Choice of best models based on in-sample selection criteria (one within each class of models)
Imagine, as a job candidate, you are locked in a room for a test of your ability to develop
appropriate forecasting models. You are given the monthly data series of chosen commodity price
series and other relevant variables up to 2015, and your have to report to the selection panel your
best choice of model within each class of models and the reasons behind your decisions.
We will consider a range of models.
M1: No change model (i.e., use yt forecast of yt+h)
M2: Univariate model (trend + seasonality + cyclical components modelled as AR)
M3: Simple regression based model using variable x, possibly with distributed lags
M4: Simple regression based model using variable z, possibly with distributed lags
M5 Bivariate vector autoregression with y and x.
M6: Bivariate vector autoregression with y and z.
M7: Trivariate vector autoregression with y, x and z.
Aside from your chosen commodity (y), you are free to choose variables x and z. Of course, it
is important to tell the readers why these variables are likely useful and relevant. If you want
to, you can build models of ln yt
, models of ∆yt, models of ∆ ln yt
, etc. In that case, a clear
statement of decision and reasons is expected. Also, keep in mind that our ultimate objective is
to forecast yt (and hence our forecast evaluation), not whatever transformed variables you used
in the model building.
At the end, you should report the best model within the class of M2, and then the best model
with the class of M3, and so on.
2. Evaluation of the out-of-sample performance of different models
Imagine you are a member of the selection panel. You take all the models the candidate had just
submitted and evaluate whether the chosen model produces the most accurate out-of-sample
forecast performance from 2016 to present for different forecast horizons h = 1, 3, 6, 9, 12, all
based on a RECURSIVE SCHEME. Below, we illustrate with the “h =1” case. The extension
to the other horizons should be straightforward.
Denote the h-step-ahead forecast of model i as yˆ(i) t+h,t. The corresponding forecast error is then
e(i) t+h,t = yt+h h yˆ(i) t+h,t i = 1, 2, ..., 7; h = 1, 3, 6, 9, 12.
Page 2 of 6
Compare the performance of these models relative to “no change” model (i.e., M1) using the ratio
of MSPEs:
MSP E ratio(i) = MSP E(M i)
MSP E(M1) i = 2, 3, ..., 7.
We would also like to perform a statistical evaluation of the performance of these models.
✄ For each of the forecasting models (i = 1, 2, 3, 4, 5, 6, 7, 8), assess the forecastability via a
Mincer-Zarnowitz regression.
✄ For each of the forecasting models (i = 1, 2, 3, 4, 5, 6, 8), test statistically whether any of the
forecasts has the same MSFE as the random-walk-like forecast. That is, a statistical test of
H0 : E[(e(i) t+h,t)2
] = E[(e
(1)
t+h,t)2] vs H1 : E[(e(i) t+h,t)2] = E[(e
(1)
t+h,t)2]
for i = 2, 3, ..., 7, and h = 1, 3, 6, 9, 12.
When you are conducting these statistical tests, remember to state clearly the hypothesis being
tested and how the test is conducted.
Write all these in a report so that general readers will understand the significance of forecasting the
chosen commodity price, the logic behind the development of the forecasting models, and the out-ofsample evaluation. While we want a report readable to general readers (with basic but not sophisticated
training in Econometrics), students should also keep in mind that they should demonstrate adequate
understanding of what they have learned from class. That is, it is important to make clear statements
of DECISIONS at different juncture with the appropriate supporting evidence.
Upload a zip file containing the whole folder of your work related to this project to Assignment
corresponding to project #2. The zip file should include the report (pdf format), the R script, the
data file, and the Word file (or LyX file) and auxiliary files, etc. Note that the submitted files should
be adequate for the grader to replicate all the results (figures and tables) included in the report – with
a few clicks. To allow such easy replication, students may also want to include adequate annotations
in the R scripts.
Often, students are tempted to write a long report. Please avoid making the report too long. A good
report is usually concise, and yet precise. When you are writing up the report, you should assume a
reader from the industry (say, Economist Intelligence Unit). Always ask: “We know what we are doing
but do the readers know what we are doing?” “Is the report too long such that readers will find it
boring?” In your report, try to include the following sections:
1. An introduction. (One to two pages?)
✄ What we plan to do in the paper and why we want to do it.
✄ Why forecasting the chosen commodity price is important and significant.
✄ The potential use of the forecast of the chosen commodity price.
2. A brief description of the data. (One to two pages?)
✄ A brief description of the variables.
Page 3 of 6
✄ Data source: the URLs or tickers or acronyms from the database such as Bloomberg, Datastream; the definitions, the original source of the data, etc.
✄ Sample period, and data frequency.
✄ Reason(s) for the choice of the commodity.
✄ Brief reason(s) for choice of the predictors.
3. Choice of best models within each class of models, based on in-sample selection criteria (Three
to five pages?)
✄ A brief description of the modeling strategies.
✄ List of models considered.
✄ How we arrive at the chosen model, with supporting evidence.
4. Evaluation of the out-of-sample performance of different models (Three to five pages?)
✄ How the forecast comparison is done.
✄ How the tests are done.
✄ Our observations from the tables of statistics and plots.
✄ Major findings of forecast comparison
5. Concluding remarks (One to two page?)
✄ Major conclusion, policy implication (if any) and potential improvement of the analysis.
✄ Discuss what you would consider doing if you are free to consider alternative modelling
strategies.
✄ Take the best model to produce a forecast of 2021, and discuss the outlook of the chosen
commodity price.
6. Reference section (One page?)
The report should have less than 18 pages, with at least 12 pt fonts, at least 1.5 line spacing, and at
least 2 cm of margins on each side. Page numbering, figure numbering and table numbering should
be included. Some students feel obliged to fill up all 18 pages. Please don’t. A shorter report is
always preferred. It is about how to present the idea and analysis to the readers clearly. For the
same content and same clarity, readers always prefer shorter reports.
Page 4 of 6
R: R is a free software environment for statistical computing and graphics, available at
http://www.r-project.org/.
Bloomberg: Bloomberg is available from our computer lab on 10/F of KK Leung Building. Students are welcome to explore other reliable databases. Nonetheless, Bloomberg is preferred, and
familiarity with Bloomberg is a valuable assets in the business/research field.
DataStream: DataStream is available from our University Main Library. Familiarity with DataStream is a valuable assets in the business/research field.
Page 5 of 6
Objectives of this assignment:
✄ To practice how to forecast with simple time series models.
✄ To practice the forecast evaluation.
✄ Writing up the report: tighten up the logic of discussion (why we are doing this and that).
✄ Widen our horizon to see what happen in other commodities (students have to work on a diverse
set of countries).
✄ To see the advantages of different models and modelling strategies.
Grading rubrics (the following items may carry different weights):
Grading is mainly based on the report. The other materials are referred only when necessary. On
some items below, as an illustration, we highlight the points that are commonly deducted on common
mistakes.
✄ Cover page: title of the report, the name and student ID number, and date. (5 points deducted
if missing)
✄ Basic formatting: page numbering, equation number, table numbering, figure numbering; table
title, figure title. (5 to 10 points deducted if missing)
✄ Discussion associated with plots or tables. If you include a plot, make sure you discuss it. (up
to 50 points deducted if inadequate)
✄ Whether the R script and data file are adequate to regenerate the results used in the paper. (up
to 50 points deducted if inadequate)
✄ Data description / Data sources. (up to 10 points deducted if missing)
✄ Properly labeled tables and figures (Clear titles); whether notes to tables / figures are included.
(up to 10 points deducted if missing)
✄ Adequate guidance to readers in understanding the paper. (up to 50 points deducted if inadequate)
✄ Writing: Grammar, organization, transition from one paragraph to the next, etc. (up to 30
points deducted if inadequate)
✄ Proper citations and references. (up to 20 points deducted if inadequate)
✄ Motivation / Policy implications / Potential use of the analysis. (up to 20 points deducted if
inadequate)
✄ Are claims properly supported with evidence and statistical logic? (up to 50 points deducted if
inadequate)
✄ Discussion of the linkage of the paper to policy implications. (up to 20 points deducted if
inadequate)