7CCSMPNN Support Vector Machines (SVMs)

Pattern Recognition, Neural Networks and Deep Learning

(7CCSMPNN)

Assignment: Support Vector Machines (SVMs)

This coursework is assessed. A type-written report needs to be submitted online

through KEATS by the deadline specified on the module’s KEATS webpage. In this

coursework, we consider a classification problem of 3 classes. A multi-class SVM-based

classifier formed by multiple SVMs is designed to deal with the classification problem.

Q1. Write down the first 7 digits of your student ID as s1s2s3s4s5s6s7. (5 Marks)

Q2. Find R1 which is the remainder of s1+s2+s3+s4+s5+s6+s7 4

. Table 1 shows the multi￾class methods to be used corresponding to the value of R1 obtained. (5 Marks)

R1 Method

0 One against one

1 One against all

2 Binary decision tree

3 Binary coded

Table 1: R1 and its corresponding multi-class method.

Q3. Create a linearly separable two-dimensional dataset of your own, which consists of

3 classes. List the dataset in the format as shown in Table 2. Each class should

contain at least 10 samples and all three classes have the same number of samples.

Note: This is your own created dataset. The chance of having the same dataset

in other submissions is slim. Do not share your dataset with others to avoid any

plagiarism/collusion issues. (10 Marks)

Sample of Class 1 Sample of Class 2 Sample of Class 3

... ... ... ... ... ...

Table 2: Samples of three classes.

Q4. Plot the dataset in Q3 to show that the samples are linearly separable. Explain

why your dataset is linearly separable. Hint: the Matlab built-in function

plot can be used and show some example hyperplanes which can linearly separable

the datasets. Identify which hyperplane is for which classes. (20 Marks)

1

Q5. According to the method obtained in Q2, draw a block diagram at SVM level to

show the structure of the multi-class classifier constructed by linear SVMs. Explain

the design (e.g., number of inputs, number of outputs, number f SVMs used, class

label assignment, etc.) and describe how this multi-class classifier works.

Remark: A blocking diagram is a diagram which is used to, say, show a concept or

a structure, etc. Here in this question, a diagram is used to show the structure of

the multi-class SVM classifier, i.e., how to put binary SVM classifiers together to

work as a multi-class SVM classifier. For example, Q5 of tutorial 8 is an example

of a block diagram at SVM level. Neural network diagram is a kind of diagram to

show its structure at neuron level. The block diagrams in lecture 9 are to show the

architecture of ensemble classifier, etc. (20 Marks)

Q6. According to your dataset in Q3 and the design of your multi-class classifier in Q5,

identify the support vectors of the linear SVMs by “inspection” and design their

hyperplanes by hand. Show the calculations and explain the details of your design.

(20 Marks)

Q7. Produce a test dataset by averaging the samples for each row in Table 2, i.e., (sam￾ple of class 1 + sample of class 2 + sample of class 3)/3. Summarise the results in

the form of Table 3, where N is the number of SVMs in your design and “Classi-

fication” is the class determined by your multi-class classifier. Explain how to get

the “Classification” column using one test sample. Show the calculations for one or

two samples to demonstrate how to get the contents in the table. (20 Marks)

Test Sample Output of SVM 1 · · · Output of SVM N Classification

... ... ... ... ... ... ... ... ... ...

Table 3: Summary of classification accuracy.

Marking: The learning outcomes of this assignment are that student understands the

fundamental principle and theory of support vector machine (SVM) classifier; is able to

design multi-class SVM classifier for linearly separable dataset and knows how to de￾termine the classification of test samples with the designed classifier. The assessment

will look into the knowledge and understanding on the topic. When answering the ques￾tions, show/explain/describe clearly the steps/design/concepts with reference to the equa￾tions/theory/algorithms (stated in the lecture slides). When making comments (if neces￾sary), provide statements with the support from the results obtained.

Purposes of Assignment: This assignment provides the overall classification idea from

samples to design to classification. It helps you to make clear the concept, working prin￾ciple, theory, classification of samples, design procedure and multiple-class classification

techniques for SVM.