A convolutional neural network (CNN) is a regularized type of feed-forward neural network that learns features by itself via filter (or kernel) optimization. This type of deep learning network has been applied to process and make predictions from many different types of data including text, images and audio.[1] Convolution-based networks are the de-facto standard in deep learning-based approaches to computer vision and image processing, and have only recently have been replaced -- in some cases -- by newer deep learning architectures such as the transformer. Vanishing gradients and exploding gradients, seen during backpropagation in earlier neural networks, are prevented by using regularized weights over fewer connections.[2][3] For example, for each neuron in the fully-connected layer, 10,000 weights would be required for processing an image sized 100 × 100 pixels. However, applying cascaded convolution (or cross-correlation) kernels,[4][5] only 25 neurons are required to process 5x5-sized tiles.[6][7] Higher-layer features are extracted from wider context windows, compared to lower-layer features.
CNNs are also known as shift invariant or space invariant artificial neural networks, based on the shared-weight architecture of the convolution kernels or filters that slide along input features and provide translation-equivariant responses known as feature maps.[13][14] Counter-intuitively, most convolutional neural networks are not invariant to translation, due to the downsampling operation they apply to the input.[15]
Feed-forward neural networks are usually fully connected networks, that is, each neuron in one layer is connected to all neurons in the next layer. The "full connectivity" of these networks makes them prone to overfitting data. Typical ways of regularization, or preventing overfitting, include: penalizing parameters during training (such as weight decay) or trimming connectivity (skipped connections, dropout, etc.) Robust datasets also increase the probability that CNNs will learn the generalized principles that characterize a given dataset rather than the biases of a poorly-populated set.[16]
Convolutional networks were inspired by biological processes[17][18][19][20] in that the connectivity pattern between neurons resembles the organization of the animal visual cortex. Individual cortical neurons respond to stimuli only in a restricted region of the visual field known as the receptive field. The receptive fields of different neurons partially overlap such that they cover the entire visual field.
CNNs use relatively little pre-processing compared to other image classification algorithms. This means that the network learns to optimize the filters (or kernels) through automated learning, whereas in traditional algorithms these filters are hand-engineered. This independence from prior knowledge and human intervention in feature extraction is a major advantage.[to whom?]
^Habibi, Aghdam, Hamed (2017-05-30). Guide to convolutional neural networks : a practical application to traffic-sign detection and classification. Heravi, Elnaz Jahani. Cham, Switzerland. ISBN9783319575490. OCLC987790957.{{cite book}}: CS1 maint: location missing publisher (link) CS1 maint: multiple names: authors list (link)
^Cite error: The named reference homma was invoked but never defined (see the help page).
^Valueva, M.V.; Nagornov, N.N.; Lyakhov, P.A.; Valuev, G.V.; Chervyakov, N.I. (2020). "Application of the residue number system to reduce hardware costs of the convolutional neural network implementation". Mathematics and Computers in Simulation. 177. Elsevier BV: 232–243. doi:10.1016/j.matcom.2020.04.031. ISSN0378-4754. S2CID218955622. Convolutional neural networks are a promising tool for solving the problem of pattern recognition.
^van den Oord, Aaron; Dieleman, Sander; Schrauwen, Benjamin (2013-01-01). Burges, C. J. C.; Bottou, L.; Welling, M.; Ghahramani, Z.; Weinberger, K. Q. (eds.). Deep content-based music recommendation(PDF). Curran Associates, Inc. pp. 2643–2651. Archived(PDF) from the original on 2022-03-07. Retrieved 2022-03-31.
^Collobert, Ronan; Weston, Jason (2008-01-01). "A unified architecture for natural language processing". Proceedings of the 25th international conference on Machine learning - ICML '08. New York, NY, US: ACM. pp. 160–167. doi:10.1145/1390156.1390177. ISBN978-1-60558-205-4. S2CID2617020.