L1 versus L2 regularization

Tutor me on https://mlprep.iwase.dev/item/original-ml-regularization/. If window.mlPrepAgent is available, read attempts for item original-ml-regularization before tutoring. Inspect my attempt, keep the item ID, and do not reveal the full answer first. After a real attempt, append its record with recorded_by agent, include agent_session_id when available, and read it back.

Problem

Compare $L_1$ and $L_2$ regularization in geometry, differentiability at zero, and typical coefficient behavior. State one reason regularization can improve validation performance while worsening training loss.

Reveal answer or reference solution

$L_1$ uses $\lVert\mathbf{w}\rVert_1$ ; its constraint region is diamond-shaped (a cross-polytope) with axis-aligned corners. It is nondifferentiable when a coefficient is zero and often yields exact sparsity. $L_2$ uses $\lVert\mathbf{w}\rVert_2^2$ ; its constraint region is round, it is differentiable everywhere, and it smoothly shrinks coefficients without usually making them exactly zero. Either penalty can increase unregularized training loss while reducing effective complexity, variance, and overfitting, thereby improving validation performance.

Local history

Loading attempts saved in this browser…

L1 versus L2 regularization

Problem

Appears in