—
Train, validation, and test roles
Tutor me on https://mlprep.iwase.dev/item/ml-screen-theory-foundation-splits/. This is a screening item. If window.mlPrepAgent is available, read attempts for item ml-screen-theory-foundation-splits before tutoring. Inspect my attempt, keep the item ID, and do not reveal the full answer first. After a real attempt, append its record with recorded_by agent, include agent_session_id when available, and read it back.Problem
A team fits polynomial degrees through . Which split fits the coefficients, which split selects the degree, and when is the test split used? Why does repeatedly selecting models by test performance invalidate the intended estimate?
Reveal answer or reference solution
Fit the coefficients on training data, select the degree on validation data, and use test data once after all choices are fixed. Repeated test-based selection adapts development to accidental test-sample features, causing optimistic selection bias; the score is no longer an untouched generalization estimate.
Previous attempts
Loading attempts saved in this browser…