Into ${\fam11\tenbbb R}^n$

Given a time series $u:{\fam11\tenbbb Z}\longrightarrow {\fam11\tenbbb R}$, it is sometimes convenient and feasible to turn the time series into a map from ${\fam11\tenbbb R}^n$ to ${\fam11\tenbbb R}^n$ which is iterated, with some noise added. All we do is to take a sequence of n consecutive time values of the series as input, and the shift to one time unit later as output. That is, we look at the map

$$% latex2html id marker 7350 \left( \begin{array} {c} u(t) \\... ...} u(t+1) \\ u(t) \\ \vdots \\ u(t-n+2) \end{array} \right)$$

This I shall call the delay map.

Similarly if we have a MA filter of order k from a time series u to a time series v we can write this as a linear map from ${\fam11\tenbbb R}^{k+1}$ to ${\fam11\tenbbb R}^k$. For example, if we have that

v(n) = b₀ u(n) + b₁ u(n-1)

we can write

$$% latex2html id marker 7352 \left( \begin{array} {c} v(n) \\... ...begin{array} {c} u(n) \\ u(n-1) \\ u(n-2) \end{array}\right)$$

Writing such a map as f, it is not hard to see that ARMA modeling of v becomes a modeling of f by matrix operations. This approach has the additional merit that if we have a time series of vectors, we can simply amalgamate the vectors in the same way, and the general theory doesn't trouble to distinguish between the cases where a time series is of vectors having dimension one and where it has any other dimension.

It is not particularly reasonable to expect the delay map to be linear or affine, with or without noise, but we can keep our fingers crossed and hope that it is. Chaotic systems arise rather easily when it isn't. Our position is not so much that linear and affine systems are the ones which arise in practice a great deal, it's that those are the ones we can figure out what to do something about.

If we take a point in the domain of the delay map and apply the map to this point, and then to the result of applying the map, and thus iterate the map on this point, we get a set of points in ${\fam11\tenbbb R}^n$. We may compute the mean and covariance matrix for this set in order to get a second order description of it, just as for a set of pixels in ${\fam11\tenbbb R}^2$. It is called the autocovariance matrix. It may turn out that the set of points lies on a subspace of dimension significantly less than n. Anything that can be said about the set of points obtained by parcelling up the time series in this way is a statement about the process which generated the time series, so we have a distinct interest in describing point sets in ${\fam11\tenbbb R}^n$.The mean and covariance matrix are rather inadequate descriptors for complicated shapes, and so it is possible to go to higher order. It is also possible to go to local descriptions which are locally low order and to piece them together. We shall have more to say about this later.

The simple minded trick of taking a time sequence of numbers and turning it into a time series of vectors by putting a moving window over the time series in the hope that scrutiny of the resulting point sets will be informative is also done in the old piecewise affine neural nets, where a simple perceptron then becomes a MA filter with a non-linear threshold stuck on the end. These are called time-delay neural nets. We can also feed the output back into the input through a subnet which procedure yields a recurrent neural net. The range of basic ideas here is really fairly limited.