Lecture 17
Core loop: data → model → loss → gradient → update
🎥 This lecture is pre-recorded - I'm away at ICML this week, so there's no in-person class today. Watch at your own pace.
📅 Due today (Tue Jul 7): Chat 8. The CS686 project proposal is now due Thu Jul 9 (submit on LEARN under "Project Proposal").
💬 Questions? Post on Piazza - the TAs are available, and I'll follow up when I'm back.
By the end, this loop should feel readable:
for x, y in loader:
pred = model(x)
loss = loss_fn(pred, y)
loss.backward()
optimizer.step()
optimizer.zero_grad()Everything later - neural nets, LLMs, diffusion - scales up this idea.
The data is fixed.
The model has tunable parameters: \(w\) and \(b\).
Training means choosing parameters that make predictions good.
Forget ML for one slide.
We only need to make a loss small.
This is the smallest possible version of training.
The gradient points uphill, so step the other way.
\(x \leftarrow x - \eta \dfrac{dL}{dx}\)
direction that decreases loss
learning rate: how big a step?
Interactive demo — runs live in your browser (open the slides to try it).
Manual derivative
def loss(x):
return (x - 2) ** 2
def manual_grad(x):
return 2 * (x - 2)Automatic differentiation
x = nn.Parameter(torch.tensor(-4.0))
L = (x - 2) ** 2
L.backward()
print(x.grad) # -12For millions of parameters, we rely on autograd. But how does it know the answer?
Every operation records its own local derivative; the chain rule multiplies them backward.
Forward: \(u = x - 2\), then \(L = u^2\).
Chain rule: \(\tfrac{dL}{dx} = \tfrac{dL}{du}\cdot\tfrac{du}{dx}\)
Local: \(\tfrac{dL}{du} = 2u\) (since \(L=u^2\))
Local: \(\tfrac{du}{dx} = 1\) (since \(u=x-2\))
Multiply: \(\tfrac{dL}{dx} = 2u\cdot 1 = 2(x-2)\)
At \(x=-4\): \(\tfrac{dL}{dx} = 2(-6) = -12\) — exactly x.grad.
Interactive demo — runs live in your browser (open the slides to try it).
Value class) builds the graph \(x \to (x-2) \to \text{square} \to L\), runs L.backward(), and prints x.grad == -12. Editable real Python, executed in your browser via Pyodide.A model has many parameters, collected as \(\theta\). The loop does not change.
\(\theta \leftarrow \theta - \eta\,\nabla_\theta L(\theta)\)
optimizer = SGD(model.parameters(), lr=0.1) for x, y in loader: pred = model(x) loss = loss_fn(pred, y) loss.backward() optimizer.step() optimizer.zero_grad()model.parameters() → the parameters \(\theta\)
loss.backward() → the gradient \(\nabla_\theta L\)
optimizer.step() → update \(\theta \leftarrow \theta - \eta\nabla_\theta L\)
Prediction: \(\hat y = wx + b\) (\(w,b\) are the parameters)
Error: \(L = \frac{1}{m}\sum_i(\hat y_i-y_i)^2\)
Start with a bad line — big errors (dashed).
Measure the error, nudge \(w,b\) to shrink it.
Repeat — the errors keep shrinking.
Good fit: the errors are small.
model = nn.Linear(1, 1)loss_fn = nn.MSELoss()optimizer = torch.optim.SGD(model.parameters(), lr=0.1)pred = model(x)loss = loss_fn(pred, y)loss.backward()optimizer.step()optimizer.zero_grad()Interactive demo — runs live in your browser (open the slides to try it).
Now the target is a category, not a number.
\(\hat y = 217.3\)
a number
\(\hat y = \text{``spam''}\)
a class
But gradient descent needs a smooth output — so we predict a probability instead.
Compute a score:
\(z = w\cdot x + b\)
Squash it into a probability:
\(P(\text{spam}\mid x)=\sigma(z)=\frac{1}{1+e^{-z}}\)
Cross-entropy punishes being confidently wrong.
Recall \(\hat y = P(y=1\mid x) = \sigma(z)\) — the probability the model predicts.
\(L = -\big[y\log \hat y + (1-y)\log(1-\hat y)\big]\)
True label spam \(\Rightarrow y=1\), so \(L = -\log \hat y\).
\(\hat y=0.9\)
\(L=-\log 0.9 \approx 0.11\)
\(\hat y=0.1\)
\(L=-\log 0.1 \approx 2.30\)
Interactive demo — runs live in your browser (open the slides to try it).
One score per class; softmax turns scores into probabilities.
\(P(y=k\mid x)=\dfrac{e^{z_k}}{\sum_j e^{z_j}}\)
Next-token prediction is exactly this, over a vocabulary.
Interactive demo — runs live in your browser (open the slides to try it).
for x, y in loader:pred = model(x)loss = loss_fn(pred, y)loss.backward()optimizer.step()optimizer.zero_grad()This is the pattern to recognize for the rest of the module.
Full batch: few, expensive, smooth steps. Mini-batch: many, cheap, noisy steps — usually faster progress, and GPU-friendly.
updates the weights
chooses settings
final, one-time estimate
Never tune on the test set.
Same data, increasingly flexible models.
Training loss keeps dropping.
Validation loss starts rising — that growing gap is overfitting. Stop where validation bottoms out.
Interactive demo — runs live in your browser (open the slides to try it).
show the model more of the world
prefer simpler weights
\(L(\theta) + \lambda\lVert\theta\rVert^2\)
stop when validation turns up
Only the model and the loss change. The loop stays the same.
| Task | Model (architecture) | Loss |
|---|---|---|
| Regression (today) | Linear / MLP | squared error |
| Language model | Transformer | cross-entropy (next token) |
| Image generation | Diffusion U-Net | squared error (predict noise) |
forward → loss → backward → step, every time.
L18: if linear models use fixed features, how do neural nets learn their own?