On the size of Convolutional Neural Networks and generalization performance

Maya Kabkab; Emily Hand; Rama Chellappa

doi:10.1109/ICPR.2016.7900188

On the size of Convolutional Neural Networks and generalization performance

Maya Kabkab, Emily Hand, Rama Chellappa

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

While Convolutional Neural Networks (CNNs) have recently achieved impressive results on many classification tasks, it is still unclear why they perform so well and how to properly design them. In this work, we investigate the effect of the convolutional depth of a CNN on its generalization performance for binary classification problems. We prove a sufficient condition - polynomial in the depth of the CNN - on the training database size to guarantee such performance. We empirically test our theory on the problem of gender classification and explore the effect of varying the CNN depth, as well as the training distribution and set size.

Original language	English (US)
Title of host publication	2016 23rd International Conference on Pattern Recognition, ICPR 2016
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	3572-3577
Number of pages	6
ISBN (Electronic)	9781509048472
DOIs	https://doi.org/10.1109/ICPR.2016.7900188
State	Published - Jan 1 2016
Externally published	Yes
Event	23rd International Conference on Pattern Recognition, ICPR 2016 - Cancun, Mexico Duration: Dec 4 2016 → Dec 8 2016

Publication series

Name	Proceedings - International Conference on Pattern Recognition
Volume	0
ISSN (Print)	1051-4651

Other

Other	23rd International Conference on Pattern Recognition, ICPR 2016
Country/Territory	Mexico
City	Cancun
Period	12/4/16 → 12/8/16

ASJC Scopus subject areas

Computer Vision and Pattern Recognition

Access to Document

10.1109/ICPR.2016.7900188

Cite this

Kabkab, M., Hand, E., & Chellappa, R. (2016). On the size of Convolutional Neural Networks and generalization performance. In 2016 23rd International Conference on Pattern Recognition, ICPR 2016 (pp. 3572-3577). Article 7900188 (Proceedings - International Conference on Pattern Recognition; Vol. 0). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICPR.2016.7900188

On the size of Convolutional Neural Networks and generalization performance. / Kabkab, Maya; Hand, Emily; Chellappa, Rama.
2016 23rd International Conference on Pattern Recognition, ICPR 2016. Institute of Electrical and Electronics Engineers Inc., 2016. p. 3572-3577 7900188 (Proceedings - International Conference on Pattern Recognition; Vol. 0).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Kabkab, M, Hand, E & Chellappa, R 2016, On the size of Convolutional Neural Networks and generalization performance. in 2016 23rd International Conference on Pattern Recognition, ICPR 2016., 7900188, Proceedings - International Conference on Pattern Recognition, vol. 0, Institute of Electrical and Electronics Engineers Inc., pp. 3572-3577, 23rd International Conference on Pattern Recognition, ICPR 2016, Cancun, Mexico, 12/4/16. https://doi.org/10.1109/ICPR.2016.7900188

@inproceedings{a05a882586ec4674a11d6f07153c3c0e,

title = "On the size of Convolutional Neural Networks and generalization performance",

abstract = "While Convolutional Neural Networks (CNNs) have recently achieved impressive results on many classification tasks, it is still unclear why they perform so well and how to properly design them. In this work, we investigate the effect of the convolutional depth of a CNN on its generalization performance for binary classification problems. We prove a sufficient condition - polynomial in the depth of the CNN - on the training database size to guarantee such performance. We empirically test our theory on the problem of gender classification and explore the effect of varying the CNN depth, as well as the training distribution and set size.",

author = "Maya Kabkab and Emily Hand and Rama Chellappa",

note = "Publisher Copyright: {\textcopyright} 2016 IEEE.; 23rd International Conference on Pattern Recognition, ICPR 2016 ; Conference date: 04-12-2016 Through 08-12-2016",

year = "2016",

month = jan,

day = "1",

doi = "10.1109/ICPR.2016.7900188",

language = "English (US)",

series = "Proceedings - International Conference on Pattern Recognition",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "3572--3577",

booktitle = "2016 23rd International Conference on Pattern Recognition, ICPR 2016",

}

TY - GEN

T1 - On the size of Convolutional Neural Networks and generalization performance

AU - Kabkab, Maya

AU - Hand, Emily

AU - Chellappa, Rama

PY - 2016/1/1

Y1 - 2016/1/1

N2 - While Convolutional Neural Networks (CNNs) have recently achieved impressive results on many classification tasks, it is still unclear why they perform so well and how to properly design them. In this work, we investigate the effect of the convolutional depth of a CNN on its generalization performance for binary classification problems. We prove a sufficient condition - polynomial in the depth of the CNN - on the training database size to guarantee such performance. We empirically test our theory on the problem of gender classification and explore the effect of varying the CNN depth, as well as the training distribution and set size.

AB - While Convolutional Neural Networks (CNNs) have recently achieved impressive results on many classification tasks, it is still unclear why they perform so well and how to properly design them. In this work, we investigate the effect of the convolutional depth of a CNN on its generalization performance for binary classification problems. We prove a sufficient condition - polynomial in the depth of the CNN - on the training database size to guarantee such performance. We empirically test our theory on the problem of gender classification and explore the effect of varying the CNN depth, as well as the training distribution and set size.

UR - http://www.scopus.com/inward/record.url?scp=85019140829&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85019140829&partnerID=8YFLogxK

U2 - 10.1109/ICPR.2016.7900188

DO - 10.1109/ICPR.2016.7900188

M3 - Conference contribution

AN - SCOPUS:85019140829

T3 - Proceedings - International Conference on Pattern Recognition

SP - 3572

EP - 3577

BT - 2016 23rd International Conference on Pattern Recognition, ICPR 2016

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 23rd International Conference on Pattern Recognition, ICPR 2016

Y2 - 4 December 2016 through 8 December 2016

ER -

On the size of Convolutional Neural Networks and generalization performance

Abstract

Publication series

Other

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this