The Complete Guide on Overfitting and Underfitting in Machine Learning

What is Overfitting?

When a model performs very well for training data but has poor performance with test data (new data), it is known as overfitting. In this case, the machine learning model learns the details and noise in the training data such that it negatively affects the performance of the model on test data. Overfitting can happen due to low bias and high variance.

Become an AI and ML Expert in 2024

Discover the Power of AI and ML With UsEXPLORE NOW

Reasons for Overfitting

• Data used for training is not cleaned and contains noise (garbage values) in it
• The model has a high variance
• The size of the training dataset used is not enough
• The model is too complex

Ways to Tackle Overfitting

• Using K-fold cross-validation
• Using Regularization techniques such as Lasso and Ridge
• Training model with sufficient data
• Adopting ensembling techniques

What is Underfitting?

When a model has not learned the patterns in the training data well and is unable to generalize well on the new data, it is known as underfitting. An underfit model has poor performance on the training data and will result in unreliable predictions. Underfitting occurs due to high bias and low variance.

Reasons for Underfitting

• Data used for training is not cleaned and contains noise (garbage values) in it
• The model has a high bias
• The size of the training dataset used is not enough
• The model is too simple

Ways to Tackle Underfitting

• Increase the number of features in the dataset
• Increase model complexity
• Reduce noise in the data
• Increase the duration of training the data

Now that you have understood what overfitting and underfitting are, let’s see what is a good fit model in this tutorial on overfitting and underfitting in machine learning. 

Your AI/ML Career is Just Around The Corner!

AI Engineer Master's ProgramExplore Program

What Is a Good Fit In Machine Learning?

To find the good fit model, you need to look at the performance of a machine learning model over time with the training data. As the algorithm learns over time, the error for the model on the training data reduces, as well as the error on the test dataset. If you train the model for too long, the model may learn the unnecessary details and the noise in the training set and hence lead to overfitting. In order to achieve a good fit, you need to stop training at a point where the error starts to increase.

Demo - Analyzing Goodness of Fit For IRIS Dataset

• Import the libraries

• Load the IRIS dataset

• Now, you will use K-Fold Cross-Validation with 20 folds (K=20) to evaluate the generalization efficiency of the model. Within each fold, you will estimate the train and test error using the training and testing datasets, respectively.

• Plot the mean absolute error (MAE) of the training phase and the MAE of the testing phase

Using the K-Fold Cross Validation method, you were able to significantly reduce the error in the testing dataset.

Become an AI and ML Expert in 2024

Discover the Power of AI and ML With UsEXPLORE NOW

Conclusion

Overfitting and Underfitting are two vital concepts that are related to the bias-variance trade-offs in machine learning. In this tutorial, you learned the basics of overfitting and underfitting in machine learning and how to avoid them. You also looked at the various reasons for their occurrence. 

If you are looking to learn the fundamentals of machine learning and get a comprehensive work-ready understanding of it, Simplilearn’s AI ML Course in partnership with Purdue & in collaboration with IBM. should be ideal for you. This 12-month long bootcamp program features comprehensive applied training in key concepts of Machine learning, Deep Learning with Keras and Tensorflow, Advanced deep learning and Computer Vision, Natural Language Processing and more.

Do you have any questions related to this tutorial on overfitting and underfitting in machine learning? In case you have questions, please put them in the comments section. We’ll help you answer them. To learn more, check the following video: Overfitting and Underfitting.

Happy learning!