Power law scaling of test error versus number of training images for deep convolutional neural networks

Vittorio Sala

doi:10.1117/12.2525811

21 June 2019 Power law scaling of test error versus number of training images for deep convolutional neural networks

Vittorio Sala

Proceedings Volume 11059, Multimodal Sensing: Technologies and Applications; 1105914 (2019) https://doi.org/10.1117/12.2525811
Event: SPIE Optical Metrology, 2019, Munich, Germany

Abstract

The highest accuracy in image classification is of utmost importance for the industrial application of algorithms based on convolutional neural networks. Empirically, it is sometimes possible to improve accuracy by increasing the size of the training set. In this work, the scaling of the test accuracy versus the size of the training set was studied for different networks. First, a network with a depth of few layers was initialized with random parameters and trained on subsets of images of variable size sampled from the MNIST dataset of handwritten digits and MNIST fashion dataset of clothes and accessories. The scaling of the accuracy versus the size of the training set may be described as the sum of two components: a power law and an offset independent on the size of the training set. Exponent of the power law appears to be the same in both dataset and independent on seeds, initial weights and number of convolutional filters. Then, the scaling of the accuracy versus the size of training set has been evaluated on a dataset of pictures of paintings, sacred icons and sculptures with the goal to correctly classify unknown pictures. The networks chosen are the ones implemented in the machine vision library Halcon 18.11, including two convolutional neural networks with unknown topology and Resnet50, pretrained on industrial images. The scaling of the accuracy versus the size of the training set seems to be compatible with the power law scaling observed on the few layers network trained on MNIST.

Citation Download Citation

Vittorio Sala "Power law scaling of test error versus number of training images for deep convolutional neural networks", Proc. SPIE 11059, Multimodal Sensing: Technologies and Applications, 1105914 (21 June 2019); https://doi.org/10.1117/12.2525811

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available