Geometry and Dynamics

Since this happens over fairly short times, it is natural to conjecture that the self-tuning phenomenon observed in the visual cortex of the cat occurs quiet generally and is related to learning and cognition. The situation may be described more colourfully in terms of the neurons being attracted to the data; if we describe the state of the neuron by the input to which it responds most strongly, we can imagine it as a point sitting in the same space as the input data. And when an input datum is presented to the sensorium, the neuron `moves' towards the datum, much like a dog being drawn irresistibly towards a rabbit when the latter sticks his head out of his hole. The neuron doesn't move physically inside the head of the cat, it changes its state so as to move in the state space. In the case of the edge detecting neurons, we draw a line and mark in a point to denote an angle of zero, another to denote an angle of 90 degrees. Then we put a datum in to the system and in the form of a vertical edge by lighting up the corresponding orientation, a point near the 90^o location. The neurons are not initially on the line, since there is no strongly tuned response to any orientation when the kitten first opens its eyes. So we may put the neurons initially at locations off the line representing the input states. The repeated flashing of data points at or near the 90^o location draws in the neurons however, until they are all close to the data presented. Similarly, if data points are presented near the 0^o location on the line, neurons respond by being attracted to the corresponding location. Neurons are attracted to data just as dogs are to rabbits.

There is some scope for enquiry concerning the dynamical laws governing the attraction of neurons to data or dogs to rabbits. If we have two rabbits alternately springing up in a field at different locations A and B, then if a pack of dogs is set loose at random initial positions in the field, what assumptions can we plausibly make concerning the motion? Well, one assumption is that the dogs can see the rabbits but not the other dogs. Presumably neurons can share a field of inputs, but neurons aren't data, and it seems unreasonable that they should be able to respond to each others states. On the other hand, there is a mechanism, lateral inhibition, whereby they can hear each other. If the strength of the output of one neuron is interpreted whimsically as the dog barking, then many groups of neurons have output passed between them, so that the output of one is allowed to inhibit the effects of the common input. This allows for more precise tuning, and it also may be interpreted in the geometrical for which I have a decided preference.

In dynamical terms then, the dogs move towards a rabbit when it stands up and presumably stop shortly after it goes down again. They cannot see each other, but they can see the rabbits, bark more frenziedly as they get closer, and can hear each others barking.

 

Figure 8.15: Neurons being attracted to data

It makes sense for the dogs to move larger distances when they are closer to the rabbits and smaller distances when they are further away, for the reasons discussed in chapter five. If one contemplates a dog which is attracted to each rabbit alternately, with the two rabbits, one at A and one at B, then the dog which simply jumps half way to the closest rabbit, winds up at neither but at the $\frac{1}{3}$ or $\frac{2}{3}$point along the line joining the rabbits. Moreover a pack of dogs does the same thing. The pack coalesces to a single dog very rapidly, all in the wrong place. Whether discussing dogs in a field or neurons in a state space modelled on a linear space, we are faced with arguments against letting the law of attraction be what is known in informed circles as a contraction mapping. (See any good book on linear analysis.)

It is natural to suppose that the inhibition from surrounding dogs operates by reducing the size of any jump they might make towards a rabbit. We therefore obtain a fairly simple set of functions determining the motion of a dog towards a rabbit or a neuron towards a datum. The options for the qualitative behaviour are not great, as a more sophisticated mathematical analysis will demonstrate. Here I argue only at the intuitive level, which carries reasonable conviction with most.

Other effects which may be expected to affect the dynamics of neurons when attracted to data, are found in biological neurons: habituation dims the response of neurons to repeated data eventually, lability of neurons, their propensity to respond by change of state is believed to be reduced after some time. I suppose in the more formal model that the neuron jump size towards the datum is reduced by a `fatigue' factor which is proportional to the number of data seen (or repeated) close to the present state of the neuron. This has the desirable effect of locking the neurons on close data and making them immune from the disturbing influence of remote data. A description of the formal system complete with rational function descriptions of the dynamics may be found in the paper by McKenzie and Alder cited in the Bibliography below.