CMSC 498L: Introduction to Deep Learning Released: Feb-04. Due Feb-11.
Assignment 1
Name: Enter Name Here UID: Enter UID Here
Instructions:
? Submit the assignment on ELMS.
? Assignments have to be formatted in LATEX. You can use overleaf for writing your assignments.
? Submit only the compiled PDF version of the assignment.
? Refer to policies (collaboration, late days, etc.) on the course website.
1 Probability
1. Density function. Let p be a Gaussian distribution with zero mean and variance of 0.1.
Compute the density of p at 0.
Sol:
1
Name: Enter Name Here UID: Enter UID Here
2. Conditional probability. A student is taking a one-hour-time-limit makeup examination.
Suppose the probability that the student will finish the exam in less than x hours is x/2,
?x ∈ [0, 1]. Given that the student is still working after 0.75 hour, what is the conditional
probability that the full hour will be used?
Sol:
— 2 —
Name: Enter Name Here UID: Enter UID Here
3. Bayes rule. Consider the probability distribution of you getting sick given the weather in
the table below.
Weather Sick?
sunny rainy cloudy snow
yes 0.144 0.02 0.016 0.02
no 0.576 0.08 0.064 0.08
Compute P( sick = yes | Weather = rainy ).
Sol:
— 3 —
Name: Enter Name Here UID: Enter UID Here
2 Calculus and Linear Algebra
For each of the following questions, we expect to see all the steps for reaching the solution.
1. Compute the derivative of the function f(z) with respect to z
i.e., df
dz
, where
f(z) = 1
1 + e z
Sol:
— 4 —
Name: Enter Name Here UID: Enter UID Here
2. Compute the derivative of the function f(w) with respect to wi
, where w, x ∈ RD and
f(w) = 1
1 + e wT x
Sol:
— 5 —
Name: Enter Name Here UID: Enter UID Here
3. Compute the derivative of the loss function J(w) with respect to w, where
J(w) = 12 Xmi=1
wT x(i) ) y(i)
Sol:
— 6 —
Name: Enter Name Here UID: Enter UID Here
4. Compute the derivative of the loss function J(w) with respect to w, where
J(w) = 12 "Xmi=1
wT x(i) ) y(i)2# + λkwk22
Sol:
— 7 —
Name: Enter Name Here UID: Enter UID Here
5. Compute the derivative of the loss function J(w) with respect to w, where
J(w) = Xmi=1
y(i)
log 1
1 + e wT x(i) + 1 1 y(i)
log 1 1 1
1 + e wT x(i)
Sol:
— 8 —
Name: Enter Name Here UID: Enter UID Here
6. Compute ?wf, where f(w) = tanh wT x.
Sol:
— 9 —
Name: Enter Name Here UID: Enter UID Here
7. Find the solution to the system of linear equations given by Ax=b, where
A = ??
2 1 11 3 31 2
?2 1 2
?? and b = ?? 8 11
?3 ?? .
Sol:
— 10 —
Name: Enter Name Here UID: Enter UID Here
8. Find the eigenvalues and associated eigenvectors of the matrix:
A = ??
7 0 03 9 92 3
18 0 08 ??
Sol:
— 11 —
Name: Enter Name Here UID: Enter UID Here
3 Activation functions
For each of the following activation functions, write their equations and their derivatives. Plot the
functions and derivatives, with x ∈ [[5, 5] and y ∈ [[10, 10] plot limits. (No need to submit the
code for plots.)
1. Relu
Sol:
— 12 —
Name: Enter Name Here UID: Enter UID Here
2. Tanh
Sol:
— 13 —
Name: Enter Name Here UID: Enter UID Here
3. Softmax
Sol:
— 14 —
Name: Enter Name Here UID: Enter UID Here
4. Sigmoid
Sol:
— 15 —
Name: Enter Name Here UID: Enter UID Here
5. Leaky ReLU
Sol:
— 16 —
Name: Enter Name Here UID: Enter UID Here
6. ELU (plot with α = 0.3)
Sol:
— 17 —
Name: Enter Name Here UID: Enter UID Here
7. Sinc
Sol:
— 18 —