Lab
Gradient Descent Simulator
Tune the learning rate and starting point on a loss landscape.
Training a model means minimizing a loss. Gradient descent takes repeated small steps downhill, opposite the gradient. The learning rate sets the step size: too small and it crawls, too large and it overshoots or diverges. Click the map to drop the starting point, or use the Start x and Start y sliders, which work with the keyboard.
The green dot is the minimum. Each step moves the point against the gradient, scaled by the learning rate. On the isotropic bowl a moderate rate glides straight in. The valley (x² + 10y²) is steep across and shallow along: the same rate that's fine on the bowl makes the path bounce across the ravine, the classic reason a single global learning rate is hard to choose.
Update rule: p ← p − η·∇f(p), where η is the learning rate and ∇f is the exact gradient of the surface shown.
Challenge
On the valley, nudge the learning rate up until the path starts zig-zagging, then diverges. Find the largest rate that still converges, then switch to the bowl and confirm it tolerates a much larger step. Why does the valley break first?