Overfitting vs Underfitting: Memorizing Noise vs Missing the Signal

Mon, 03 Aug 2026 03:37:23 +0900

Overview

Overfitting and underfitting describe the two ways a model can fail to generalize: one learns the training data too well, the other not well enough. Understanding which failure mode you’re in determines whether you should simplify or add regularization, or instead increase capacity and train longer. Overfitting traps a model in the noise of its training set, while underfitting leaves it unable to capture the underlying pattern at all.

Comparison Diagram

Comparison Table

Aspect	Overfitting	Underfitting
Underlying cause	Model too complex relative to the data, so it learns noise and idiosyncrasies	Model too simple to represent the true relationship in the data
Training error	Very low, often near zero	High, the model struggles even on data it was trained on
Validation/test error	High, much worse than training error	High, similar in magnitude to training error
Bias-variance profile	Low bias, high variance	High bias, low variance
Generalization to new data	Poor, predictions swing wildly on unseen inputs	Poor, predictions are consistently and systematically off
Learning curve signature	Training and validation loss diverge as training continues	Training and validation loss both plateau high and close together
Typical remedies	Regularization, more training data, dropout, early stopping, simpler model	Increase model capacity, add features, train longer, reduce regularization

Key Differences

Overfitting memorizes noise in the training set, while underfitting never learns the underlying pattern at all.
Overfitting shows near-zero training error but a wide train/validation gap, a sign of high variance; underfitting shows poor performance on both, a sign of high bias.
Overfitting is treated with regularization or more data; underfitting is treated by increasing model capacity.
Overfitting gets worse the longer an overly flexible model keeps training; underfitting persists regardless of duration, since it’s a structural limit.

When to Use Each

Overfitting

Bias-Variance on IT Comparison

Overfitting vs Underfitting: Memorizing Noise vs Missing the Signal

Overview

Comparison Diagram

Comparison Table

Key Differences

When to Use Each