Autoregressive Time Series

A time series f might have the property that the value of f at time n is related to the value at an earlier time. In particular if f were periodic with period T then we would have f(n) = f(n-T) or perhaps f(n) = -f(n-T/2). If f were built up as a finite sum of periodic functions with a variety of periods and phases, we could write

$$f(n+1) = a_0 f(n) + a_1 f(n-1) + \cdots a_{r-1} f(n-r+1)$$

for some suitable collection of r coefficients.

Definition 14028

A time series $f:{\fam11\tenbbb Z}\longrightarrow {\fam11\tenbbb R}$ is said to be autoregressive of order r iff

$$\exists a_0, a_1, \cdots a_{r-1} \in {\fam11\tenbbb R}: \for... ... n \in {\fam11\tenbbb Z}, f(n+1) = \sum_{i=0}^{r-1} a_i f(n-i)$$

As I have defined it, some series are autoregressive, while most are not. You choose some starting sequence of r values for f, and then operate upon them to produce the next value of f. Now you slide up one value and do the same again. A little thought shows that if the coefficients were of the wrong sort, the series could easily blow up; for example if r=1 and we have f(n+1) = 2 f(n), we get a geometric progression going bigger, while f(n+1) = 1/2 f(n) has it sinking to zero fast. The situation with more coefficients is more complicated and is considered by engineers under the title of stability of the filter.

To the engineer, a set of coefficients like the b_i above is just asking to be made into a filter. Is such a thing a filter? If I give you a starting segment you can go ahead and generate a time series, but what do I do if I have a whole time series as input? Well, nothing subtle. I can MA filter it and add it in, but that doesn't make for a separate kind of filter called Autoregressive (AR) filtering, although you might naively think it did.

What I can do is to consider a class of filters, called IIR filters , where if you input a time series, say u, you get output time series v and where

The IIR is short for Infinite Impulse Response and comes from the observation that if you choose r to be 1, input a time series u which is zero everywhere except at one time where it takes the value 1, make b₀ any non-zero value you wish, then unless the a_i are rather unlikely, the output time series will never become permanently zero (Although it can get pretty close to it). If you put such an input into a MA filter, then of course it rapidly goes to the average of the zero inputs, so is zero except for some finite period of time. Hence the term FIR filter, F for Finite, for what I have called a Moving Average filter, because my name tells you a lot more about what it is.

Much of the history of Filtering theory dominates the older treatments in the Engineering texts.

Given a sequence of real or complex numbers, some engineers feel an irresistible urge to make them the coefficients of a complex power series or Laurent series, and use the resulting complex function instead of the series. It is possible to satisfy oneself that there is a one to one correspondence between convergent infinite sequences and complex analytic functions. This leads to turning problems about time series into problems about properties of complex analytic or more generally meromorphic functions. (Meromorphic functions may go off to infinity at some discrete set of points while analytic ones don't.) Having suffered the agonies of mastering complex function theory, they don't see why you shouldn't suffer too; they may firmly believe that such suffering enobles and strengthens the character, although others may doubt this. I should undoubtedly be drummed out of the Mathematician's Union were I to cast doubts on this view. Besides I like Complex Function theory. So I won't.