ML Prep

Squared loss and gradient

Problem

For x=(0,1,2)x=(0,1,2), y=(1,3,5)y=(1,3,5), and y^=b+wx\hat{y}=b+wx at (b,w)=(0,1)(b,w)=(0,1), use

J=12n∑i=1n(y^i−yi)2.J=\frac{1}{2n}\sum_{i=1}^{n}(\hat{y}_i-y_i)^2.

Compute the predictions, residuals, JJ, and the gradient with respect to (b,w)(b,w).

Reveal answer or reference solution

The predictions are (0,1,2)(0,1,2), so the residuals y^−y\hat{y}-y are (−1,−2,−3)(-1,-2,-3). Thus,

J=1+4+96=73.J=\frac{1+4+9}{6}=\frac{7}{3}.

The intercept gradient is

∂J∂b=mean⁡(y^−y)=−2,\frac{\partial J}{\partial b}=\operatorname{mean}(\hat{y}-y)=-2,

and the slope gradient is

∂J∂w=mean⁡(x(y^−y))=−83.\frac{\partial J}{\partial w}=\operatorname{mean}\bigl(x(\hat{y}-y)\bigr)=-\frac{8}{3}.

Therefore, ∇J=(−2,−83)\nabla J=(-2,-\frac{8}{3}).

Outcome

Previous attempts

Loading attempts saved in this browser…

    Review Screening Results