Solutions for Data Science and Machine Learning, 1st Edition by Kroese (All Chapters included)
2 views 0 purchase
Module
Data Science and Machine Learning 1e Kroese
Institution
Data Science And Machine Learning 1e Kroese
Complete Solutions Manual for Data Science and Machine Learning, 1st Edition by Dirk P. Kroese; Zdravko Botev; Thomas Taimre; Radislav Vaisman ; ISBN13: 9781138492530....(Full Chapters are included and organized in reverse order from Chapter 9 to 1)...1 Importing, Summarizing, and Visualizing Data
...
to Accompany
Data Science and Machine Learning:
Mathematical and Statistical Methods
Dirk P. Kroese Zdravko I. Botev Thomas Taimre
Slava Vaisman Robert Salomone
** Immediate Download
** Swift Response
** All Chapters included
,CONTENTS
Preface 3
1 Importing, Summarizing, and Visualizing Data 5
2 Statistical Learning 17
3 Monte Carlo Methods 35
4 Unsupervised Learning 65
5 Regression 79
6 Kernel Methods 99
7 Classification 115
8 Tree Methods 139
9 Deep Learning 149
2
,Solutions Manual organized in reverse order, with the last chapter displayed first, to ensure that all
chapters are included in this document. (Complete Chapters included Ch9-1)
CHAPTER 9
D EEP L EARNING
1. Show that the softmax function
exp(z)
Softmax : z 7→ P .
k exp(zk )
satisfies the invariance property:
Softmax(z) = Softmax(z + c × 1), for any constant c.
Solution: Let w := Softmax(z + c × 1) and u := Softmax(z). For every i, we have
from the definition of the softmax function:
exp(zi + c)
wi = P
k exp(zk + c)
exp(c) exp(zi )
=P
k exp(c) exp(zk )
exp(zi )
=P
k exp(zk )
= ui .
This implies the identity.
2. Projection Pursuit is a network with one hidden layer that can be written as:
g(x) = S (ω> x),
where S is a univariate smoothing cubic spline. If we use squared-error loss with
τn = {yi , xi }ni=1 , we need to minimize the training loss:
n
1X 2
yi − S (ω> xi )
n i=1
with respect to ω and all cubic smoothing splines. This training of the network is
typically tackled iteratively in a manner similar to the EM algorithm. In particular,
we iterate (t = 1, 2, . . .) the following steps until convergence.
149
, 150
(a) Given the missing data ωt , compute the spline S t by training a cubic smoothing
spline on {yi , ω>t xi }. The smoothing coefficient of the spline may be determined
as part of this step.
(b) Given the spline function S t , compute the next projection vector ωt+1 via iter-
ative reweighted least squares:
ωt+1 = argmin (et − Xβ)> Σt (et − Xβ), (9.11)
β
where
yi − S t (ω>t xi )
et,i := ω>t xi + , i = 1, . . . , n
S t0 (ω>t xi )
is the adjusted response, and Σ1/2
t = diag(S t0 (ω>t x1 ), . . . , S t0 (ω>t xn )) is a diagonal
matrix.
Apply Taylor’s Theorem B.1 to the function S t and derive the iterative reweighted
least squares optimization program (9.11).
Solution: Using a linear approximation of S t around ω>t xi , we have:
n
X n
2 X 2
yi − S t (ω> xi ) ≈ yi − S t (ω>t xi ) − S t0 (ω>t xi )[ω − ωt ]> xi
i=1 i=1
n #2
yi − S t (ω>t xi )
X "
= [S t0 (ω>t xi )]2 ω>t xi + − xi ω .
>
i=1
S t0 (ω>t xi )
Hence, we obtain the iterative reweighted least squares.
3. Suppose that in the stochastic gradient descent method we wish to repeatedly draw
minibatches of size N from τn , where we assume that N × m = n for some large
integer m. Instead of repeatedly resampling from τn , an alternative is to reshuffle τn
via a random permutation Π and then advance sequentially through the reshuffled
training set to construct m non-overlapping minibatches. A single traversal of such
a reshuffled training set is called an epoch. The following pseudo-code describes the
procedure.
The benefits of buying summaries with Stuvia:
Guaranteed quality through customer reviews
Stuvia customers have reviewed more than 700,000 summaries. This how you know that you are buying the best documents.
Quick and easy check-out
You can quickly pay through credit card for the summaries. There is no membership needed.
Focus on what matters
Your fellow students write the study notes themselves, which is why the documents are always reliable and up-to-date. This ensures you quickly get to the core!
Frequently asked questions
What do I get when I buy this document?
You get a PDF, available immediately after your purchase. The purchased document is accessible anytime, anywhere and indefinitely through your profile.
Satisfaction guarantee: how does it work?
Our satisfaction guarantee ensures that you always find a study document that suits you well. You fill out a form, and our customer service team takes care of the rest.
Who am I buying these notes from?
Stuvia is a marketplace, so you are not buying this document from us, but from seller mizhouubcca. Stuvia facilitates payment to the seller.
Will I be stuck with a subscription?
No, you only buy these notes for £24.07. You're not tied to anything after your purchase.
