COMP4660/8420 Assignment 1: Neural Networks

4TH ANU ANNUAL BIO-INSPIRED COMPUTING STUDENT CONFERENCE

http://cs.anu.edu.au/~tom/conf/ABCs2021/

also being used for

COMP4660/8420 Assignment 1: Neural Networks 

Submission Due: Sunday 25th April at 11:55pm

Context

Neural networks research prior to the deep learning boom has a lot to teach us still. The 

neural networks part of the course focuses on the research in this area from 1986 (back￾propagation algorithm published) into the 1990s and early 2000s. Later papers tend to be too 

esoteric to be useful for course based learning and I prefer you to read papers rather than just 

textbooks. 

The assignment will provide you some experience with taking two papers: one with a dataset, 

and another with a technique, and you will do a mix of data encoding / analysis on the dataset 

and implementing / applying a technique from the paper. Thus, each of your assignments will 

be unique, but you will still be able to usefully discuss with other students, as there will be 

other people using the dataset (but different techniques/papers) and other people using the 

same technique (but on different datasets).

To be possible as an assignment, we stick to simple (by modern standards) techniques, which 

fits well with the chronological focus from 1986 to early 2000s. Some of the datasets are from 

the same period, and some are from later papers.

Wattle will shortly have a way for you to select the technique, then the dataset.

I have uploaded a number of papers relating to the neural network lectures to Wattle, most of 

the technique papers will be one of these.

Perhaps it is worth noting that some of the techniques developed in that early neural 

networks period a likely be of use today. My example is the use of an input feature selection 

technique of mine from 1997 which has recently been implemented in the H2O AI package. In 

Google type: “Gedeon method” H2O and you get a surprising number of recent hits. I should 

also mention that I have no connections with H2O and found out about this when a Korean 

researcher emailed me to ask me about the function in the H2O package and I discovered that 

it was based on my paper J. Similarly, please also note that the CNN architecture dates to 

the 1980s (Fukushima’s Neocognitron in 1980 to LeCun’s coining of the term in 1989) but was 

not competitive given the computing power and datasets generally available at that time. Also 

please note that most/many deep learning methods have fully connected layers at the end 

which are simple neural networks as in the first part of the course.

Deliverables

1. A report (about 4-6 pages of text content with a MAXIMUM of 10 pages, including 

1

references, diagrams, graphs and tables). Remember to keep your report clear and 

concise. I say that the text content should be about 4-6 pages to signal that we will not 

be counting lines, but if it is 10 pages long and it is all diagrams then it is very clear 

that there is too little text. Conversely, if it is 10 pages long and there is just one 

diagram then there is too much text, but 6 pages with one diagram is fine, and so on.

2. PyTorch or Python source code file(s).

3. The dataset you used for your assignment, in original and preprocessed form, and a 

copy of the paper. (I know we provided it, but this makes it easier for marking if 

everything is in one spot.)

4. A copy of the technique paper (for consistency and to help the markers). Both the 

dataset and technique papers must be cited within your report, along with at least 4 

more papers.

Submission Method

Please submit your assignment via the EasyChair conference management system: 

https://www.easychair.org/conferences/?conf=abcs2021.

You will submit a pdf for the report, the name must start with your uNumber, and a zip 

folder of the other deliverables.

Objectives

The purpose for this assignment is for you to:

• Develop a good understanding of data management for artificial neural networks and 

enhance your skills in implementing neural network techniques in PyTorch / Python

• Enhance your approach to investigating a data set and setting up a problem for an 

artificial neural network to solve a classification/regression problem

• Develop a good understanding for reporting investigations in a conference style paper

• A small amount of experience in using Google scholar to find citing/cited papers

Task Description

Your task is to:

1. devise a classification or regression problem to investigate using the data set provided 

– this can be reproducing, extending or modifying the problem which is described in 

the paper related to that dataset;

2. implement a neural network in PyTorch / Python to solve the problem and implement 

a method to determine the performance of the neural network;

3. implement a technique from the literature (paper selected as described above) and 

determine its benefit or lack of benefit;

4. compare your results with results published in a research paper reporting results on 

the data set you chose (see below); and

5. write a report on your work

Data set

Choose a data set from the datasets provided. Wattle will have a way for you to select the 

combination of dataset and technique. You can modify the prediction task from that originally 

described in the paper. You should find it difficult to obtain near perfect results, as you will be 

2

marked on the steps taken to achieve better results and how they compare with your previous 

results, not on how high % accurate your results are. The paper related to the dataset may be 

of a similar age to the technique paper, or will be using unusual data so there will be little 

need to try to compete with the best modern techniques. Your report must describe which dataset you used, what modifications you made to the 

encodings if you needed to do so, and cite the paper associated with the data set to provide 

some context to your results. You must also cite the technique paper(s) you use. You should 

also cite some other papers which are somehow relevant. Other academic papers can 

generally be found by using Google scholar. Google scholar will have links to one or more 

paper repositories. You can get access if you do this from ANU campus as the library 

subscribes to most of the large document repositories. Some papers are in multiple 

repositories so if you cannot find a free copy then ask your tutor for advice. From off campus, 

if you log into the ANU Virtual Proxy server then usually you can access the same electronic 

resources that would be available on campus. You should also mention in your report any 

different topologies and analyses used in your experimentation.

Design of a Problem

Detail in your report what you want to model in the data set and explain what the inputs and 

outputs you will use to develop the neural network model.

Implementation

Choose an appropriate measure to report the results produced by your neural networks. You 

can use the measure used in the research papers that have the results of their predictions on 

the data set you have used and would like to compare your results with – if the paper did not 

do predictions, then the paper provides a context for your predictions instead. Remember to 

cite the papers in your report.

Report

The report must be in the style of an academic paper and must conform to the Springer 

Lecture Notes in Computer Science conference paper proceedings format, but with the 

margins changed to 2 cm and header/footer to 1 cm. The template for the report can be 

downloaded from Wattle by clicking the link named “AssignmentReportTemplate-LNCS￾Office2007.zip” and needs the margins to be modified. If you prefer to use Latex , there is a 

link on the ABCs conference website which takes you to Springer’s current pages. Any 

template from there will also need the margins to be modified. Use the Springer citation style 

as found in the template file you use.

Your report should have a meaningful title, which indicates what you have done. "COMP8420 

Assignment" is not a meaningful title. Your title and content should NOT mention the course:

we are modelling the assignment so that you are making a conference paper submission. Your 

u number should be showing in your email address. Your affiliation would be "Research 

School of Computer Science, Australian National University" etc. Please do not forget to 

include your name as the single author.

A suggested structure for the report:

• Abstract – A paragraph that summarises the work you did, the results you found and 

whether it was better, same or worse than a published research paper for the same 

3

dataset. An abstract is similar to an executive summary of the entire report.

• Introduction – A description of the motivation for the choice of the data set, the 

problem that you modelled using a neural network and an outline of the 

investigations that you carried out using the model. This section should also include a 

brief background to the problem and the methods used perform the analysis. 

Remember to use citations!

• Method – A description of the technique(s) you implemented and details of the 

investigations or tests you conducted using the technique. • Results and Discussion – Presentation of results from the investigations and detailed 

analysis of the results including comparison of your results with the results published 

in a research paper on the data set. Remember to use citations!

• Conclusion and Future Work – A statement on your findings and how your work can 

be extended or how might it be improved. Even if you have conducted a thorough 

investigation there is always work left to do. Outlining future work is VERY important 

as it shows that you have thought about the problem and have a deeper 

understanding then just stating a conclusion.

A comment: you should read papers in relevant areas of the literature (e.g. on similar topics)

to get more of a feel for how to write and layout academic papers. This forms part of the 

learning you should get out of advanced courses such as COMP4xxx/COMP8xxx given that 

the ANU is a research intensive University.

Peer Marking (optional)

Conference papers need reviewers, and conferences need a Program Committee. The plan is 

that all of you are the program committee, the program chair(s) will assign you some papers 

to review and you will either do them or not (the peer marking is optional). You will get 

marks for how well you do the reviews. How well means whether you gave appropriate 

criticism and helpful suggestions or not (a too critical or too uncritical review can be equally 

bad). I hope many of you will chose to do the peer marking. Any low marks can be redeemed 

in the final exam. There will be more details about the peer marking when we get closer to it, 

the reports need to be submitted before any peer marking is possible.

Marking

The assignment is worth 20% of overall marks. There is an assignment rubric in Wattle, which 

has the following components: Abstract, Task Selection, Task Complexity, Methodology, Data 

Inspection, Data Preparation, Findings, Evaluation, Discussion, Reproducibility, Writing 

Clarity, Writing Style, Presentation, Citations, Implementation and Code Style.