Underfitting in Machine Learning: Summary

Underfitting occurs when a machine learning model is too simple to capture the true patterns in the data. As a result, it performs poorly on both the training set and unseen data. It is typically associated with high bias and low variance.

Core Idea

An underfitted model fails to learn enough from the training data. For example, fitting a straight line to clearly curved data will miss important structure and produce inaccurate predictions.

Underfitting vs. Overfitting

Underfitting: model is too simple; poor training and test performance.
Good fit: captures real patterns; performs well on both training and test data.
Overfitting: model is too complex; excellent training performance but poor test performance.

Main Causes of Underfitting

Model too simple for the problem
Poor feature selection or missing important variables
Too much regularization limiting model flexibility
Insufficient training or failure to converge
Improper preprocessing such as noisy, unscaled, or incorrectly encoded data
Wrong algorithm choice for the data complexity

How to Detect It

Low training accuracy and low validation accuracy
High training and validation loss
Learning curves where both errors remain high and close together
Residuals showing clear patterns instead of random noise
Performance similar to a simple baseline model

How to Fix Underfitting

Increase model complexity (e.g., deeper trees, more neurons, polynomial models)
Improve features through feature engineering and better transformations
Reduce regularization if it is too strong
Train longer or improve optimization settings
Improve data quality with cleaning, scaling, and correct encoding
Tune hyperparameters such as learning rate, depth, and network size

Model-Specific Examples

Linear models: underfit nonlinear relationships
Decision trees: underfit when too shallow
Neural networks: underfit when too small or over-regularized
SVMs: may underfit with restrictive kernels or poor parameter choices

Bias-Variance Relationship

Underfitting corresponds mainly to high bias and low variance. In contrast, overfitting is usually linked to low bias and high variance.

Key Takeaways

Underfitting means the model has not learned enough from the data.
It causes poor performance on both training and test sets.
It is usually fixed by making the model more expressive, improving features, reducing excessive constraints, or training more effectively.
The goal in machine learning is to balance simplicity and complexity for the best generalization.

What is underfitting in machine learning?

Resources for