Homework 13

1. In this problem you will implement a neural network to solve

a classification problem. To keep things simple, the data will

consist of covariates x(i) ∈ R

2 and a response yi ∈ {0, 1} for

i = 1, 2, . . . , N (notice y takes on only two possible values). The

classification problem involves fitting the model y ∼ f(x) over

functions f(x) that can be parameterized by our neural net,

which is described below. The attached file nn.txt contains

the samples. Each sample, corresponding to a row in the file,

gives the three values (x

Your neural network should consist of three layers, as discussed

in class. The input layer should contain X1 and X2 nodes,

which will be the two coordinates for each of the samples x

should contain m nodes, Zj

for j = 1, 2, . . . , m. And an output

layer consisting of nodes T1, T2. Then, the probability that the

class of x

Write a function that computes log L(α) (you will need to

pass the data to the function).

(e) Write a function that uses finite difference to compute the

stochastic gradient of log L(α) based on a single or small

number of samples (you can pick whether to use one sample

or a few).

(f) Set m = 4 and train your neural net by maximizing the

log L(α) using stochastic steepest ascent.

(g) Remember that a classifier in this case is a function F(x) :