Some basic results in Fourier analysis

March 27, 2005


Given some function f on some interval, can we approximate it by a sum of sines and cosines? Actually, since computing with complex exponentials is easier, we will tackle a more general problem: let f be complex valued, and approximate it by the functions g_{n}(x)=ae^{2\\pi inx+i\\delta}, where a and \\delta are real numbers. We can always make a finite interval into [0,1] by a change of variable, so we deal only with functions on [0,1]. When we have to deal with extensions of this interval, we assume that the functions are periodic with period one -- we tile the whole real line with copies of the unit interval.

What amplitude a and phase \\delta bring g closest to f in some sense? The obvious distance between two functions f and g at some point x is |f(x)-g(x)|. If we then take this difference at all points and add it up, we get an integral

\\int |f(x)-g(x)| dx

Applying this to the question at hand, we get

D=\\int_{0}^{1} |f(x)-ae^{2\\pi inx+i\\delta}| dx

and to minimize with respect to a and \\delta we take the derivatives \\frac{dD}{da} and \\frac{dD}{d\\delta}. It turns out to be equivalent, and significantly less work, if we take instead the square of the distance:

= \\int_{0}^{1} |f(x)-ae^{2\\pi inx+i\\delta} |^{2} dx
= \\int_{0}^{1} (|f|^{2}(x)+a^{2}-f^{\\ast}ae^{2\\pi inx+i\\delta}-fae^{-2\\pi inx-i\\delta}) dx

We find the extrema from:

\\frac{dD}{da} = \\int_{0}^{1}(2a-f^{\\ast}(x)ae^{2\\pi inx+i\\delta}-f(x)ae^{-2\\pi inx-i\\delta})dx=0
\\frac{dD}{d\\delta} = \\int_{0}^{1}(if^{\\ast}(x)ae^{2\\pi inx+i\\delta}-if(x)ae^{-2\\pi inx-i\\delta})dx=0

dD/d\\delta=0 gives

\\int_{0}^{1}f^{\\ast}e^{2\\pi inx+i\\delta}dx=\\int_{0}^{1}fe^{-2\\pi inx-i\\delta}dx

Then dD/da=0 becomes

a\\cdot \\exp(i\\delta)=\\int_{0}^{1}f(x)e^{-2\\pi inx}dx

We call a\\cdot exp(i\\delta) the Fourier component of frequency n, and usually denote it \\hat{f}_{n}.

It is not at all obvious that we can actually reconstruct the function from these. It's possible that two different approximating functions might approximate the same feature, or that none of our functions will catch some features of the function. To resolve the first question, consider the overlap of two approximating functions of different frequencies \\hat{f}_{n}\\exp(2\\pi inx) and \\hat{f}_{m}\\exp(2\\pi imx), for m\

\\int_{0}^{1}\\hat{f}_{n}^{\\ast}\\hat{f}_{m}e^{2\\pi i(m-n)x}dx=\\frac{\\hat{f}_{n}^{\\ast}\\hat{f}_{m}}{2\\pi i(m-n)}[e^{2\\pi i(m-n)x}]_{x=0}^{x=1}

which equals 0 iff m-n is an integer. To avoid overlap, we need to work only with integer separated frequencies.

The simplest choice is to choose the frequencies as the integers themselves (or (b-a) times the integers if our function is defined on [a,b] instead of [0,1]). There are two reasons for this: first, trying to make the transition later to infinite domains will otherwise be a nightmare; and second, Fourier tools are ubiquitously used to analyze transport equations (cousins to Laplace's equation \\triangle u=0) and data believed to behave according to such equations. For such an equation in one dimension (d^{2}u/dx^{2}=0), if we take a constant solution u(x)=C, we would like it to have a simple Fourier series, namely one term. Unless we take the integers for our frequencies, the constant function (given by exp(0)) won't be among our approximating functions.

This becomes clearer if considered in this way: the natural "home" of sine, cosine, and complex exponential is not a section of the real line, but a circle. If we took non-integer frequencies, we enter precisely the same realm as non-integer powers in complex analysis: for rational frequencies, things will eventually even out, but it will take some work. For irrational frequencies, we diverge off forever.

Next, do the Fourier frequencies catch all the features of the function? The short answer is no, but they can catch enough. We need to introduce a more abstract viewpoint to make sense of this.

Characterization of Functions

Closeness is a geometric idea, so in order to discuss closeness of functions, we need to develop some geometric structure. The proper setting turns out to be a vector space.

L^{p} Spaces

Consider n point particles, each of mass M/n connected by springs. Let x_{n} represent distance between the nth and the (n+1)th mass. Then we can write down a vector (x_{1},x_{2},\\dots,x_{n}) for which the normal vector addition and scalar multiplication have a sensible physical meaning.

We can define several sensible norms on this system: the total displacement from leftmost to rightmost particle

\\|x\\|_1 = \\sum_{i=1}^{n}|x_{i}|

the square root of the potential energy of the system

\\|x\\|_{2} = \\sqrt{\\sum_{i=1}^{n}|x_{i}|^{2}}

Here we take the square root so that it satisfies \\|\\lambda x\\|_{2}=|\\lambda|\\|x\\|_{2} for \\lambda a scalar1. We could keep taking powers


for p some integer. Norms with very large p are rare in practice, except for p=\\infty, which we define as


The inner product also has a sensible interpretation.

\\langle x,y\\rangle=\\sum_{i=1}^{n}x_{i}^{\\ast}y_{i}

represents the overlap of the two vectors. This should be obvious for the inner product of the vector (1,1,\\dots,1) with the standard basis vectors (1,0,\\dots,0), (0,1,0,\\dots,0), \\dots,(0,\\dots,0,1). It is perhaps not so obvious if we take other than the standard basis vectors, but we can always make a change of basis to reduce one of our vectors to such a form and the interpretation remains valid.

Now take the limit as n\\rightarrow\\infty. After some hemming and hawing, our vector becomes a function. At the moment we say nothing about smoothness. In all our definitions above, the sums become integrals

\\|f\\|_{2}=\\sqrt{\\int|f|^{2}}, \\langle f,g\\rangle=\\int f^{\\ast}g, etc.

We recognize \\|\\cdot\\|_{2} as the measure we used in the introduction to find our approximation.

As a first criterion for what we must have to make the Fourier series work, consider that we need every term in the integral representation of D to be finite. Else we cannot pull the derivative through the integral. Recall that


so the criteria are

\\int|f|^{2}<\\infty and \\int|f|<\\infty

since |e^{i\\alpha}|=1 for all real \\alpha. We call the spaces of functions with \\|\\cdot\\|_{p} finite L^{p} (L stands for Lebesgue integrable). On finite domains, all we need is L^{2}, since by the Schwartz inequality


that is, \\|f\\|_{1}\\leq\\|f\\|_{2}.

Since our Fourier transform is a change of basis, it ought to preserve the "length" of our vector. It turns out that


and the equality holds when k\\rightarrow\\infty (called Bessel's inequality and Parseval's identity, respectively). The proof is unrevealing, and can be found in most standard analysis textbooks.

Roughness of Functions

It seems unlikely that an approximation by smooth functions like complex exponentials can reproduce points and jags in the original function. As it turns out, it can, if the points aren't too pointy. How do we measure "pointiness?"

Consider the family of functions f_{k}(x)=1-|2x-1|^{1/k}. The first few of these look like:

Point functions

At x=1/2, these functions are obviously not differentiable, that is


does not exist. However, take


where 0\\leq\\alpha\\leq1. If we put in a sufficiently small \\alpha, then the limit may exist.

The f_{k}'s only misbehave at x=1/2. There we have

\\lim_{\\varepsilon\\rightarrow0}\\frac{f_{k}(\\frac{1}{2}+\\varepsilon)-f_{k}(\\frac{1}{2})}{\\varepsilon^{\\alpha}} = \\lim_{\\varepsilon\\rightarrow0}\\frac{(2\\varepsilon)^{1/k}}{\\varepsilon^{\\alpha}}

which exists when \\alpha<1/k. To see more clearly what this means, consider the box [x,x+\\varepsilon]\\times[f(x),f(x+\\varepsilon)]:

Shrinking boxes on a function

How much must we change \\varepsilon to half the height of the box? If the function is differentiable, the answer (eventually, for a sufficiently small box) is to halve the horizontal dimensions of the box as well, since all differentiable can be well approximated as linear on sufficiently small regions. For our f_{k}, we have to shrink \\varepsilon faster and faster as k increases.

More generally, a function is said to be \\alpha-continuous if \\alpha is the largest exponent in the denominator for which the limit above exists on the whole interval under consideration. \\alpha=0 is discontinuous -- we can't make the limit exist. Differentiable functions have \\alpha=1. The f_{k}'s provide a family of example illustrating \\alpha's throughout the admissable range.

Note that the \\alpha depends on the domain which you are considering. f_{k}(x) has \\alpha=1 on the interval [1/4, 1/3], but \\alpha=1/k on [0,1].

Convergence of the Series

When is the series

\\sum_{n=-\\infty}^{\\infty}\\hat{f}_{n}e^{2\\pi inx}

equal to f? And, worse, do its partial sums converge to f? We are all familiar with power series, where Taylor's remainder theorem tells us exactly how bad life can be. Can we find a similar tool for Fourier series?

Convergence of the Infinite Series

First we show

\\sum_{n=-\\infty}^{\\infty}\\hat{f}_{n}e^{2\\pi inx}=f(x)

for all continuous functions on [0,1]. The proof is easy, but not obvious. Consider the function

P_{\\rho}(x)=\\sum_{n=-\\infty}^{\\infty}e^{2\\pi inx}\\rho^{|n|}

for 0\\leq\\rho\\leq1, called the Poisson kernel. The Poisson kernel has four useful properties:

  1. \\int_{0}^{1}P_{\\rho}(x)dx=1
  2. P_{\\rho}(x)=\\frac{1-\\rho^{2}}{1-2\\rho\\cos(2\\pi x)+\\rho^{2}}
  3. P_{\\rho}(x)>0 for all \\rho and x
  4. \\int_{|x|>\\delta}P_{\\rho}(x)dx\\leq\\frac{1-\\rho}{1-\\cos(2\\pi\\delta)}, for 0<\\delta<1

We prove each of these in turn.

Proof of 1. If we can show that the integral is finite, then we can apply Fubini's theorem to switch the order of the sum and the integral. Once we have, it immediately becomes

\\sum_{n\\in\\mathbb{Z}}\\rho^{|n|}\\int_{0}^{1}e^{2\\pi inx}dx

The integrals vanish for all n\
ot=0, and we are left with 1. To see that the integral is finite, consider

\\leq \\int_{0}^{1}\\sum_{n\\in\\mathbb{Z}}|e^{2\\pi inx}\\rho^{|n|}|dx
= \\int_{0}^{1}\\sum_{n=0}^{\\infty}\\rho^{|n|}dx-1
= \\int_{0}^{1}\\frac{1}{1-\\rho}dx-1<\\infty

Proof of 2. We rewrite the sum in P_{\\rho}(x) as two sums, one over the terms with n\\geq0 and one over n<0.

= \\sum_{n\\in\\mathbb{Z}} e^{2\\pi inx}\\rho^{|n|}
= \\sum_{n\\geq0}e^{2\\pi inx}\\rho^{n}+\\sum_{n<0}e^{-2\\pi inx}\\rho^{n}
= \\frac{1}{1-e^{2\\pi ix}\\rho}+\\frac{e^{-2\\pi ix}\\rho}{1-e^{-2\\pi ix}\\rho}
= \\frac{(1-e^{-2\\pi ix}\\rho)+e^{-2\\pi ix}\\rho(1-e^{2\\pi ix}\\rho)}{1-2\\rho\\cos(2\\pi x)+\\rho^{2}}
= \\frac{1-\\rho^{2}}{1-2\\rho\\cos(2\\pi x)+\\rho^{2}}

Proof of 3. We can rewrite P_{\\rho}(x) as

1+2\\sum_{n=1}^{\\infty}\\cos(2\\pi nx)\\rho^{n}

which is obviously positive.

Proof of 4. Observe that


where \\chi_{A}(x) is the characteristic function of the set A. Then

\\leq \\int_{0}^{1}\\frac{1-\\cos(2\\pi x)}{1-\\cos(2\\pi\\delta)}P_{\\rho}(x)dx
= \\frac{1}{1-\\cos(2\\pi\\delta)}\\int_{0}^{1}\\frac{(1-\\rho^{2})(1-\\cos(2\\pi x))}{1-2\\rho\\cos(2\\pi x)+\\rho^{2}}dx
= \\frac{1-\\rho}{1-\\cos(2\\pi\\delta)}

Now we will prove that

\\lim_{\\rho\\uparrow1}\\sum_{n\\in\\mathbb{Z}}\\hat{f}_{n}e^{2\\pi inx}\\rho^{|n|}=f(x)

for all x\\in[0,1]. All continuous functions on a finite interval are L^{1}, and it is easy to show that our sum, putting in the definition of \\hat{f}_{n}, is finite. Thus we can freely use Fubini's theorem.

\\sum_{n\\in\\mathbb{Z}}\\hat{f}_{n}e^{2\\pi inx}\\rho^{|n|}
= \\sum_{n\\in\\mathbb{Z}}\\rho^{|n|}\\int_{0}^{1}e^{2\\pi in(x-y)}f(y)dy
= \\int_{0}^{1}\\sum_{n\\in\\mathbb{Z}}\\rho^{|n|}e^{2\\pi in(x-y)}f(y)dx
= P_{\\rho}\\ast f(x)

where \\ast represents convolution2. To show that the series equals f at a point x, we consider |(P_{\\rho}\\ast f)(x)-f(x)|, which we can rewrite as the integral


Let \\varepsilon>0. Then there exists \\delta>0 such that |f(y)-f(x)|<\\varepsilon/2 whenever |y-x|<\\delta (since f is continuous). Also, |f(y)-f(x)|\\leq2\\|f\\|_{\\infty} from the triangle inequality. Then

< \\frac{\\varepsilon}{2}\\int_{|y-x|<\\delta}|P_{\\rho}(x-y)|dx+2\\|f\\|_{\\infty}\\int_{|y-x|\\geq\\delta}|P(x-y)|dx
\\leq \\frac{\\varepsilon}{2}+2\\|f\\|_{\\infty}\\frac{1-\\rho}{1-\\cos(2\\pi\\delta)}

Continuous functions are bounded, so \\|f\\|_{\\infty}<\\infty. When we take the limit \\rho\\uparrow1, the second term goes away, and our result is proved.

This seems wonderfully easy, but nowhere have we made any mention of partial sums, which are all that we can actually work with except in very special cases. The pathological behavior of the partial Fourier sums will occupy us for the rest of this article.

Convergence of Partial Sums - Preliminaries

The partial sums of Fourier series are difficult. We can establish a result for functions which are sufficiently smooth, but for overly rough creatures, there is no way to determine whether the series will behave itself.

As usual, we limit our attention to L^{2} functions, since we can't even begin to work with anything else. If we take one of the f_{k}'s defined before, we can hope to get some grasp of what happens. We approximate the function by a sine of frequency 1/2, but this doesn't catch the f_k's point. Indeed, in the neighborhood of the point, the sine is flat, and the function's behavior is essentially unchanged.

Now we have to fit a higher frequency in. Try twice the last frequency. We subtract off the approximated part of the function, and again we are left with a point, just a narrower one. At every additional frequency, our point gets narrower and narrower, and we get less of it with each frequency doubling. The trick is to find at what \\alpha our series cannot match the point fast enough to converge.

The answer turns out to be \\alpha=1/2, which we now set out to prove.

This problem turns out to be equivalent to finding for what \\alpha


This isn't obvious. How are they related? Our heuristic discussion above provides the proper insight. We divide the series into sections with 2^{N}<|n|<2^{N+1} (corresponding to all modes between frequency doublings). We have to take |n| since we were before talking about sines, and we must catch both complex exponentials corresponding to the sine.

Since we are concerned with \\alpha-Holder continuity, we rewrite the Fourier coefficients in the following manner:

= \\int_{0}^{1}f(x)e^{-2\\pi inx}dx
= \\int_{0}^{1}f(x)e^{-2\\pi inx}(\\frac{e^{2\\pi int}-1}{e^{2\\pi int}-1})dx
= \\frac{1}{e^{2\\pi int}-1}\\int_{0}^{1}f(x)(e^{-2\\pi in(x-t)}-e^{-2\\pi inx})dx
= \\frac{1}{e^{2\\pi int}-1}\\int_{0}^{1}e^{-2\\pi inx}(f(x+t)-f(x))dx since f is assumed periodic
= \\frac{t^{\\alpha}}{e^{2\\pi int}-1}\\int_{0}^{1}e^{-2\\pi inx}\\frac{f(x+t)-f(x)}{t^{\\alpha}}dx
= \\frac{t^{\\alpha}}{e^{2\\pi int}-1}\\widehat{\\frac{f(x+t)-f(x)}{t^{\\alpha}}}

where \\alpha is the appropriate \\alpha of our function. Now let t=1/4\\cdot2^{N} (since it is arbitrary). The denominator e^{2\\pi int}-1 turns out to be well behaved: since 2^{N}<|n|<2^{N+1}, we find that the exponential rotates over the left half of the unit circle in this range, and doesn't go near 1. Now consider the power contained in a such a range of frequencies:

\\leq C(\\frac{1}{2^{\\alpha N}})^{2}\\sum_{n}|\\widehat{\\frac{f(x+t)-f(x)}{t^{\\alpha}}}_{n}|^{2} for C some constant
\\leq \\frac{C}{2^{2\\alpha N}}\\|(\\frac{f(\\cdot+t)-f(\\cdot)}{t^{\\alpha}})(x)\\|_{2}^{2} by Bessel's inequality
\\leq \\frac{C}{2^{2\\alpha N}}

where we have absorbed the norm into C, since for our function it is finite. Now consider the sum

\\leq \\sum_{N\\in\\mathbb{Z}}\\sum_{2^{N}<|n|<2^{N+1}}2^{(N+1)2\\gamma}|\\hat{f}_{n}|^{2}
\\leq 2^{2\\gamma}\\sum_{N}2^{2N\\gamma}\\frac{C}{2^{2\\alpha N}}

If \\gamma<\\alpha, then we have a summable series, which tells us that n^{\\gamma}|\\hat{f}_{n}| is a square summable series. Then we have


We just showed that the second square root is finite. The first one is finite only if 2\\gamma>1, i.e., \\gamma>1/2. We must have \\alpha>\\gamma, which sets our lower limit on \\alpha. This is not to say series won't converge with lower \\alpha; such series are merely less predictable.

L^{2} is a complete space (see a standard theorem that goes under the name "Arzela-Ascoli" and is found in most analysis books). Summable series have Cauchy partial sums, so the series must converge to something. We will show in the next section that it converges to f.

Before we move on, we note that since

\\sum_{n}\\hat{f}_{n}e^{2\\pi inx}

is absolutely summable, we can invoke Weierstrass's M-test to say that the sum (as a function of x) is continuous.

Convergence of Partial Sums - Dirichlet Integrals

The proper tool to analyze partial Fourier sums is a cousin to the Poisson kernel we used earlier: the Dirichlet kernel, defined as

D_{n}(x)=\\sum_{k=-n}^{n}e^{2\\pi ikx}

The first few Dirichlet kernels look like

The first few Dirichlet kernels

The Dirichlet kernel has two properties of great importance:

  1. For t not an integer,
D_{n}(x)=\\frac{\\sin(\\pi(2n+1)x)}{\\sin(\\pi x)}

and D_{n}(x)=2n+1 if t is an integer.

  1. \\lim_{n\\rightarrow\\infty}D_{n}(x) is a \\delta function, i.e.,

The proofs are:

Proof of 1. We take

D_{n}(x)=\\sum_{k=-n}^{n}e^{2\\pi ikx}=1+2\\sum_{k=1}^{n}\\cos(2\\pi kx)

Now consider instead

\\sum_{k=1}^{n}e^{2\\pi ikx}
= e^{2\\pi ix}\\frac{1-e^{2\\pi inx}}{1-e^{2\\pi ix}}
= \\frac{\\sin(\\pi nx)}{\\sin(\\pi x)}e^{\\pi i(n+1)x}

Now we take the real part:

\\mathrm{Re}\\sum_{k=1}^{n}e^{2\\pi ikx}
=\\cos(\\pi(n+1)x)\\frac{\\sin(\\pi nx)}{\\sin(\\pi x)}
=\\sum_{k=1}^{n}\\cos(2\\pi kx)

We apply the identity

\\sin(\\pi nx)\\cos(\\pi(n+1)x)=\\frac{1}{2}(\\sin(\\pi(2n+1)x)-\\sin(\\pi x))

to get

\\mathrm{Re}\\sum_{k=1}^{n}e^{2\\pi ikx} = -\\frac{1}{2}+\\frac{\\sin(\\pi(2n+1)x)}{2\\sin(\\pi x)}

Combined with our formula for D_{n}(x) we get

D_{n}(x)=\\frac{\\sin(\\pi(2n+1)x)}{\\sin(\\pi x)}

for x not an integer. If x is an integer, then D_{n}(x)=2n+1 from simple inspection.

Proof of 2. First of all,


for all n, since all terms in the sum vanish under integration except the k=0 one. Now let f be a continuous function, and let \\varepsilon>0. There exists some \\delta>0 such that whenever |x-t|<\\delta, |f(x)-f(t)|<\\varepsilon.

It is enough to consider this identity at t=0 --- by a change of variable, we can always bring any other point to 0. Now take

= \\int_{0}^{1}|\\frac{\\sin(\\pi(2n+1)x)}{\\sin(\\pi x)}(f(x)-f(0))|dx
= \\varepsilon\\int_{|x|<\\delta}|\\frac{\\sin(\\pi(2n+1)x)}{\\sin(\\pi x)}|dx-2\\|f\\|_{\\infty}\\int_{|x|>\\delta}D_{n}(x)dx

In the second term, |f(x)-f(0)|\\leq2\\|f\\|_{\\infty}, and just as for the Poisson kernel (but now with n\\rightarrow\\infty instead of \\rho\\uparrow1),


Thus we have


Our result that


lets us rearrange integrals and infinite sums. Now consider

\\sum_{n\\in\\mathbb{Z}}\\int_{0}^{1}f(x)e^{2\\pi in(t-x)}dx=\\int_{0}^{1}f(x)D_{n}(t-x)dx=f(t)

by property 2 of the Dirichlet kernel. Finally we have the convergence as desired.

Further Directions

From here, the reader might find it interesting to explore the Gibbs phenomenon from the point of view of the Dirichlet kernel, the effect of the Fourier series on derivatives, and the transition to the Fourier transform.

Fred Ross
May 2005
Charlottesville, VA

  1. See almost any linear algebra book for the axioms that define a norm. ?

  2. The convolution of two functions f and g is defined by
    f\\ast g(t)=\\int f(x)g(t-x)dx.

Did you enjoy that? Try one of my books:
Nonfiction Fiction
Into the Sciences Monologue: A Comedy of Telepathy