Effects and results of dropout layer in reducing overfitting with convolutional Neural Networks (CNN)

Olivia Harris * and Michael Andrews

Paul G. Allen School of Computer Science, University of Washington, USA.
 
Research Article
World Journal of Advanced Engineering Technology and Sciences, 2024, 13(02), 836-853.
Article DOI: 10.30574/wjaets.2024.13.2.0584
Publication history: 
Received on 14 October 2024; revised on 27 November 2024; accepted on 12 December 2024
 
Abstract: 
Convolutional Neural Networks (CNNs) have proved to be more precise for most computer vision tasks like image classification, object detection, and facial recognition. In the process, though, CNNs are susceptible to overfitting, particularly where the model complexity is high but the training data are few. Overfitting diminishes the generalizability potential of a model to new data, and deep learning consequently demands regularization techniques. One of the most powerful and widely used regularization methods is dropout, in which a random set of neurons is dropped at each training iteration. It prevents neurons from co-adapting too strongly to specific features in the training data, making the network more robust and generalizable.
Here, we empirically validate the effect of the dropout layers used in the CNN model scenario. Particular interest to us is obtaining the dynamics of model training, generalization, and performance about changes in the dropout rate. Experimental and model comparisons are performed using standard image classification datasets under various dropout settings. In all our experiments, results indicate that models trained with dropout are achieved at the cost of reduced overfitting, enhanced validation accuracy, and better generalization over novel data.
The findings highlight the need to apply dropout in CNN architecture, particularly when dealing with small datasets. Our contribution highlights the trade-off in choosing an optimal dropout rate since high or low rates can lead to underfitting or insufficient regularization. Lastly, the current study reiterates the application of dropout as a leading method for enhancing the performance and stability of deep learning models in computer vision applications.
 
Keywords: 
Convolutional Neural Networks; Dropout; Overfitting; Image classification; Regularization; Generalization
 
Full text article in PDF: 

Click here

 
Copyright information: 
Copyright © 2024 Author(s) retain the copyright of this article. This article is published under the terms of the Creative Commons Attribution Liscense 4.0