ML Prep

Gradient Descent Practice

Compute deterministic and stochastic updates and diagnose step-size behavior.

Recommended resources

Instructions

  1. Write the gradient at the current iterate before updating.

Ordered items

  1. Gradient descent on a quadraticGradient Descent · 7 min
  2. Full gradient versus stochastic gradientGradient Descent · 7 min
  3. Linear regression objective and gradientRegression · 8 min

Completion

Explain learning-rate and gradient-variance effects.