Backpropagate through tanh

Tutor me on https://mlprep.iwase.dev/item/ml-screen-neural-networks-standard-backprop/. This is a screening item. If window.mlPrepAgent is available, read attempts for item ml-screen-neural-networks-standard-backprop before tutoring. Inspect my attempt, keep the item ID, and do not reveal the full answer first. After a real attempt, append its record with recorded_by agent, include agent_session_id when available, and read it back.

Problem

A scalar network has $h=\tanh(wx)$ , $\hat{y}=vh$ , and $L=\frac{1}{2}(\hat{y}-y)^2$ . At $x=1$ , $w=0$ , $v=2$ , and $y=1$ , compute the forward pass, $\frac{\partial L}{\partial v}$ , and $\frac{\partial L}{\partial w}$ .

Reveal answer or reference solution

The forward pass gives $h=0$ , $\hat{y}=0$ , and $L=\frac{1}{2}$ . By the chain rule,

\frac{\partial L}{\partial v}=(\hat{y}-y)h=0.

Also,

\frac{\partial L}{\partial w}=(\hat{y}-y)v(1-h^2)x=(-1)(2)(1)(1)=-2.

The output weight has zero gradient because its input activation is zero.

Previous attempts

Loading attempts saved in this browser…

Review Screening Results

Backpropagate through tanh

Problem

Previous attempts

Appears in