1. Consider the Szeged Weather Data from the midterm project. The average temperature across all
years/months is about 12 degrees. Create a new binary variable that is 1 if the given month/year
is greater than or equal to 12, and 0 otherwise. For the questions below, you may compute any
gradients/hessians numerically, but you are to write your own optimization code unless told to do
otherwise.
(a) Writing out your own gradient descent algorithm, t a logistic regression model with your new
variable as the outcome and WindSpeed as the predictor. Check your answer using both the
optim function as well as glm.
(b) Now repeat (a), but using Newton’s method (i.e. use the Hessian as well as the gradient). Compare
the number of iterations it took to converge with the number from (a).
(c) We are now going to add on a penalty to the log-likelihood
‘ ( 0; 1) = ‘( 0; 1) 21:
Write an R function that takes 0 as its input and returns the minimizer of the above function
using gradient descent. Evaluate this function for several values of and plot ^ 1 as a function of
. How does in uence the estimate?
2. Consider the function
f(x;y) = (x 10)2 + 0:25(y x2)2:
Find the minimizer using both gradient descent as well as Newton’s method. In both cases, compute
the gradients/hessians analytically (i.e. don’t compute them numerically). Try a few di erent starting
values and compare how the approaches in terms how many iterations it takes them to converge.