STAT 440: Homework 8 Due: 4/2 at 10:00am
1. Consider the Szeged Weather Data from the midterm project. The average temperature across all
years/months is about 12 degrees. Create a new binary variable that is 1 if the given month/year
is greater than or equal to 12, and 0 otherwise. For the questions below, you may compute any
gradients/hessians numerically, but you are to write your own optimization code unless told to do
otherwise.
(a) Writing out your own gradient descent algorithm, fit a logistic regression model with your new
variable as the outcome and WindSpeed as the predictor. Check your answer using both the
optim function as well as glm.
(b) Now repeat (a), but using Newton’s method (i.e. use the Hessian as well as the gradient). Compare
the number of iterations it took to converge with the number from (a).
(c) We are now going to add on a penalty to the log-likelihood
`
λ(β0, β1) = ` (β0, β1) − λβ21 .
Write an R function that takes λ ≥ 0 as its input and returns the minimizer of the above function
using gradient descent. Evaluate this function for several values of λ and plot βˆ1 as a function of
λ. How does λ influence the estimate?
2. Consider the function
f(x, y) = (x − 10)2 + 0.25(y − x2)2.
Find the minimizer using both gradient descent as well as Newton’s method. In both cases, compute
the gradients/hessians analytically (i.e. don’t compute them numerically). Try a few different starting
values and compare how the approaches in terms how many iterations it takes them to converge.
3. Consider the anorexia dataset in the MASS package. This data considers two treatments for anorexia,
one based on Cognitive Behavorial Treatment (CBT) sessions from a professional therapist and another
based on family oriented treatment sessions (FT). The Treat variable is a factor with three levels: Cont
(control, no treatment), CBT, and FT. For each subject two weights (in lbs) were measured, one before
(Prewt) and one after (Postwt) treatment.
(a) Create a new variable, Outcome, that indicates if the post-weight is higher than the pre-weight,
thus denoting whether or not the treatment was successful. Design a figure that compares the
success of the three treatment groups graphically (e.g. a barplot).
(b) Using a uniform prior, compute and plot the posterior densities of the proportion of successes for
each treatment level.
(c) Use Monte Carlo with your posterior distributions to obtain confidence/credible intervals for the
difference in success proportions when comparing CBT to Cont and FT to Cont. What can you
conclude in terms of the data?
4. Consider the esoph dataset in the datasets package. This data considers cases of esophageal cancer in
a town in France. Each row of the data consists of an entire group of subjects and summarizes how
many cases (have esophageal cancer) and controls (no cancer) are in the group. The other variables
describe the group: agep denotes the age range of theg group, alcgp the daily alcohol consumption,
and tobgp the daily tobacco consumption.
(a) Create a new binary variable, Usage, that denotes 0 for little to no tobacco use (0–9 gm/day)
and medium to high use that includes the other categories. Again, graphically compare the
cass/controls against this new variable.
(b) Using a uniform prior, compute and plot the posterior densities of the proportion of cases for
Usage level.
(c) Use Monte Carlo with your posterior distributions to obtain confidence/credible intervals for the
difference in case proportions when comparing the two levels of usage. What can you conclude in
terms of the data?
STAT 440: Homework 8 Due: 4/2 at 10:00am
(d) Instead of the difference in proportions, repeat the previous part, but now comparing the difference
of the log odds. Now convert this interval into an interval for the odds ratio. Draw your conclusions
in terms of the odds ratio by giving a point estimate in addition to a confidence/credible interval
and interpreting the point estimate in the context of the problem.