Overview
Classification and regression are the two core types of supervised learning, distinguished by what kind of output they predict. Classification assigns inputs to a discrete class, while regression estimates a continuous value. Picking the wrong one for your target variable leads to mismatched loss functions, evaluation metrics, and model outputs.
Comparison Diagram
Comparison Table
| Aspect | Classification | Regression |
|---|---|---|
| Target variable type | Discrete, categorical labels from a finite set of classes | Continuous, ordered numeric values |
| Learning objective | Learn a decision boundary that separates classes | Learn a function mapping inputs to a continuous output |
| Typical loss function | Cross-entropy, log loss, or hinge loss | Mean squared error or mean absolute error |
| Model output format | Class label or probability distribution over classes | Single scalar value (or vector of scalars) |
| Common algorithms | Logistic regression, SVM, decision trees, kNN, softmax networks | Linear regression, ridge/lasso, decision trees, kNN, regression networks |
| Evaluation metrics | Accuracy, precision/recall, F1, ROC-AUC, confusion matrix | RMSE, MAE, R-squared, MAPE |
| Error interpretation | Prediction is simply right, wrong, or confused with another class | Prediction error has magnitude and direction, showing how far off it was |
Key Differences
- Classification predicts a discrete label from a fixed set of classes, while regression predicts a continuous value on a numeric scale.
- Classification models typically optimize cross-entropy loss to separate classes, while regression models optimize squared error to minimize distance from the true value.
- Classification is evaluated with metrics like accuracy/F1, while regression is evaluated with metrics like RMSE/R-squared.
- A classification error is simply right, wrong, or a class confusion, while a regression error carries a magnitude showing how far off the prediction was.
When to Use Each
Classification
- Spam Detection: Binary classification decides whether an email belongs to the spam or not-spam class.
- Image Recognition: Assigns an image to one of several known object categories.
- Medical Diagnosis: Predicts whether a patient falls into a disease-positive or disease-negative category.
Regression
- House Price Prediction: Estimates a continuous dollar value based on property features.
- Demand Forecasting: Predicts a continuous quantity like units sold next month.
- Temperature Prediction: Predicts a continuous numeric value like tomorrow’s high temperature.