Overfitting

We meet artificial intelligence (AI) and think she’s brilliant. She nails the training data—every twist, every turn. Then she stumbles on new stuff. Why? Because she memorized instead of learning. That’s overfitting.

Bias and variance

We want balance. Too much bias and she guesses wrong in the same way every time. Too much variance and she chases noise, like a student rewriting notes until the test is over. Overfitting is variance gone wild. Underfitting is bias dragging her down. Neither is pretty.

Regularization

Do one thing: add friction. Regularization is that friction. Weight decay, dropout, early stopping—all tricks that tell her not to cling too tightly to training examples. They act like a coach who says, “Yes, you can memorize the playbook, but you’d better learn to improvise.”

Validation sets

We keep a secret pile of data aside. Call it the validation set. She never sees it during training. We use it to check if she’s truly learning patterns, not just memorizing answers. When her scores rise on training but sink on validation, we know she’s drifting into overfitting. Time to pull her back.

Why it matters

Because in the real world, nobody cares if she masters the practice questions. They care if she gets the live ones right. Overfitting makes her fragile—great in the lab, brittle outside. Regularization and validation keep her honest.

A coder’s note

We’ve all done this. We polish code so long it only runs on our machine. Models overfit for the same reason. They just don’t know better until we teach them.