There is an approach to the problem of which I shall have little to say, although it has its proponents and its merits. It is typified by CART, and it works roughly as follows.
Suppose we want to tell the gals from the guys again. We take the two dimensional weight and height representation for illustration.
We first see how to cut the space up into two sections by working out the best place to put a hyperplane (line) in the space so as to get the largest fraction of points correctly discriminated. This is just like the MLP with a single unit so far.
Then we look at the points that are wrong, and try to fix them up by further subdivision of the space. We repeat until we have everything right, or satisfy some other, less exacting, criterion. We wind up, typically, with a tree structure to decide what the category of a point is, going through a sequence of binary decisions.
This approximates what a Multi-Layer Perceptron with two hidden layers accomplishes, although the algorithms are generally faster and more intelligent.
The scope for generalising is obvious; for example we do not need the data points to be all in the same space, since we can classify on, for example, the dimension. We may have a mix of geometric and symbolic representations, and so on.
My reason for not expanding on this area is because I am not happy with either the range of possible representation systems, or the segmentation process. I think there are better ways to do it, and I shall indicate them later in this book.