The Neocognitron was designed to recognize handwritten digits. Much of its complexity stems from an attempt to make the recognition process robust against variations in different people's handwriting and against variations in the position of the digit presented to it.
The design of the Neocognitron was inspired by knowledge of the visual system derived from neurophysiological experiments such as those of Hubel and Wiesel ([15] and [16]). These experiments have discovered a great deal about the structure and function of the parts of the brain that derive their inputs primarily from the retina. In particular, it has been discovered that there is only a small number of types of cells receiving inputs directly from the retina and these cells have very restricted functions; that the local relationships between cells in the retina are preserved in the organization of the neural pathways; and that the visual system appears to be structured hierarchically, with information being passed from one level of the structure to the next. These discoveries are reflected in the design of the Neocognitron.
There are a number of features that make the Neocognitron different from other neural network architectures. These include its hierarchical structure, its training methodology, and its network dynamics. These features will be described in greater detail below.
The hierarchical structure of the Neocognitron is perhaps the most significant difference between it and other network architectures. It makes it possible to piece together the global picture in a number of steps, so that the amount of processing to be done by any one layer is limited. In addition, each of the units receives input from only a small fraction of the units in the previous layer, reducing the number of connections between units.