STAT 804: Lecture 19 Notes
Processes with Periodic Components
Some of the series we have looked at have had clear annual cycles, returning to high levels in the same month every year. In our analysis of such processes we have tried to model the mean as a periodic function . Sometimes we have fitted specific periodic functions to - writing .
Another process we studied, that of sunspot numbers, also seems to show a clearly periodic component, though now the frequency or period of the oscillation is not so obvious. In this section of the course we investigate the notion of decomposing a general stationary time series into simple periodic components. We will take these components to be cosines and sines. We will be focussing on problems in which the period is not prespecified, that is problems more like the sunspot data than the annual cycle examples.
For a statistician, the simplest description of what we will do is to say that we will examine the correlation between our series and sines and cosines of various periods. We will use these correlations in several ways:
Periodic Functions
A periodic function f on the real line has the property that f(t+d)=f(t) for some d and all t. The smallest possible choice of d is the period of f. The frequency of f in cycles per time unit, is 1/d. The most famous periodic functions are the trigonometric functions and its relatives. This function has period and frequency cycles per time unit. Often, for trigonometric functions it is convenient to refer to as the frequency; the units now are radians per time point.
The achievement of Fourier was to recognize that essentially any function f with period 1 can be represented as a sum of functions or . The tactic is to suppose that
To discover the values of the coefficients we make use of the orthogonality properties:
and
Now multiply f(t) by say and integrate from 0 to 1. Expanding the integral using the supposed expression of f as a sum gives us
Similarly .
Mathematically the fact that we can derive a formula for the coefficients is far from proving that the resulting sum actually represents f; the key missing piece of the proof is that any function whose Fourier coefficients are all 0 is essentially the 0 function.
Correlation between functions
The integrals in the previous section can be thought of as analogous to covariances and variances. For instance a Riemann sum for
is
which is an average product. In fact it is possible to show that
So that the average product is just a ``sample'' covariance. It is also possible to evaluate the average product exactly to see that
exactly. When j=k this becomes a variance, equal to 1/2 so that the correlation is just the covariance times 2, which is 0 in any case when .
Interpreting all the integrals above, then, as covariances we see that all the sines are uncorrelated with each other and with all the cosines and all the cosines are uncorrelated with each other.
Notice particularly that the sine with frequency j and the cosine with frequency j are uncorrelated. This has an important implication for looking for components at frequency j cycles per time unit in a time series: if we want a certain frequency we have to consider both the cosine and the sine at that frequency. An alternative summary of what we need is to consider the trigonometric identity
When we look for a component with frequency we will allow ourselves to adjust the number , called the phase, in order to mazimize the correlation with our data. This is equivalent to adjusting the coefficients and to maximize a correlation with the right hand side of the trigonometric identity.
Complex Exponentials
Many of the identities in this subject are more easily derived using complex variables. In particular, the identity
where permits any series in sines and cosines to be rewritten in terms of exponentials. We can then often use tricks involving geometric sums to simplify our algebra.
For instance we can write
and
These permit us to rewrite the expansion (1) in the form
where for k;SPMgt;0, and for k;SPMlt;0. In fact
Fourier transforms
For functions which are not periodic we can proceed by a further approximation Suppose f is defined on the real line and fix a large value of T. Define
Then g is defined on [0,1] and
according to (1) above. Re-express the conclusion in terms of f to get
which simplifies to
You should recognize this sum as a Riemann sum for the integral
which then converges as to the expression
The function
is called the Fourier transform of f and we have derived a Fourier inversion formula. [WARNING: no proofs here! This integral will exist for, for example, f which are integrable over all the real line. ] This inversion formula expresses the function f as a linear combination of sines and cosines, though there are infinitely many frequencies involved.
Transforms of Stochastic Processes
We now seek to apply these ideas with the function f being our stochastic process X. We have several difficulties:
The discrete nature of X leads us to the study of a discrete approximation to the integral:
This object has real part
and imaginary part
so that apart from the means not being 0 we are studying the sample covariance with sines and cosines at frequency . We now study the statistical properties of these objects and then try to interpret them.
Suppose that X is a mean 0 stationary time series with autocovariance function C. We define the discrete Fourier transform of X as
Our choice to divide by the square root of T is motivated by the recognition that the sum of T terms typically has a standard deviation on the order of leading us to expect that will have a standard deviation which has a reasonable limit as .
We begin by computing moments of . Since is complex valued we have to think about what these moments should be. One way to think about this is to view as a vector with two components, the real and imaginary parts. This would give a mean and a 2 by 2 variance covariance matrix. Also of interest however will be the expected modulus squared of , namely
where is the complex conjugate of z. (If z=x+iy with x and y real then .)
Since the Xs have mean 0 we see that
(you should note that the expected value of a complex valued random variable is computed by finding the expected value of the real and imaginary parts). Then
The expected values are just C(s-t). We can gather together all the terms involving C(0), all those involving C(1) and so on to find
which simplifies to
As the coefficents of C(k) converges to 1 and we see (using C(k)=C(-k))
The right hand side of this expression is defined to be the spectral density, or power spectrum, of X:
There are a number of ways to look at spectral densities and the discrete Fourier transform:
For s=t the sum over k is simply T while for the sum can be done as a geometric series and seen to be 0. Thus the inside sum just picks out the term s=t giving as the inverse transform.
for any integer t we see that is, apart from a factor of sqrtT, a complex number whose real part is the sample covariance between X and and whose imaginary part is the sample covariance between X and . Consider computing the covariance between X and with a and b chosen to maximize the covariance subject to . The resulting coefficients are found by multiple regression of on the cosine and sine. Using the fact that we can check that the covariance is maximized by taking a and b proportional to the real and imaginary parts of respectively and that the covariance with this linear combination is . This calculation requires t to be a non-zero integer. In practice we usually apply these techniques to .
The leading term converges to 0 unless or . Restricting our attention to positive frequencies, when we get
If we move around and look at the size of when we hit a frequency at which there is a purely periodic component the modulus of will suddenly contain a component proportional to . This permits us to look for truly periodic components by looking for spikes or peaks in plots of the modulus squared of the DFT.