DETERMINATION OF THE BEST OPTIMIZER FOR A NEURONETWORK IN THE DEVELOPMENT OF AUTOMATIC IMAGE TAGGING SYSTEMS

Abstract

Background. Choosing the best optimizer is an important step in developing efficient automatic image classification systems. In particular, for neural networks based on convolutional neural networks (CNNs), the choice between popular optimization methods such as Adam (Adaptive Moment Estimation) and SGD (Stochastic Gradient Descent, SGD) can significantly affect their performance. Comparing these optimizers allows us to determine the most suitable optimizer for solving specific machine learning problems.

Methods and tools. The TensorFlow library and the TensorBoard machine learning visualization interface were used to train and test the models. The neural networks were trained on two different datasets with significantly different characteristics. For each dataset, training was performed using two optimizers: Adam and SGD. When analyzing the results, such parameters as training and validation accuracy, as well as training and validation losses, were taken into account.

Results and Discussion. According to the results, the SGD optimizer showed better performance indicators compared to Adam. The initial accuracy for SGD was higher, and a significant excess of the accuracy growth rate was observed. In addition, the final accuracy of the model with the SGD optimizer was higher on both the training and validation datasets, which indicates a more efficient training process using this optimizer.

Conclusions. It was determined that the use of the SGD optimizer leads to better results compared to the Adam optimizer for automatic image classification tasks. This is confirmed by higher accuracy indicators and a faster training process.

Keywords: TensorFlow, Convolutional Neural Network, Adaptive Moment Estimation, Stochastic Gradient Descent, GPU, TPU.