Intuitions about positive definite functions

Question

I am looking for further intuitions about positive definite functions, and have several related questions on this matter. I know this isn't the most specific question, but I find that speaking intuitions can be very helpful and can actually at times encourage a deeper mathematical understanding.

A positive definite function $\phi(t)$ is a function such that for any $n\in \mathbb{N}$, any complex numbers $\xi_1,\xi_2,\cdots,\xi_n$, and any real numbers $t_1,t_2,\cdots,t_n$, $$\sum\limits_{i,j=1}^n \phi(t_i-t_j)\xi_i\overline{\xi_j} \geq 0.$$

Basically, this is saying that the matrices $A_{ij} = \phi(t_i-t_j)$ are positive definite.

However, why should we care about such matrices / what relationship do they have with the underlying function / what do they reveal about the properties of the underlying function? Why, for example, do we care about our function evaluated at 'time jumps' $\phi(t_i-t_j)$?

When a matrix is positive definite, it says something very specific about the way that it acts on a particular vector space. Namely, that it preserves "half-spaces" / it acts such that the angle between an input vector and its image will always be less than or equal to $\pi/2$. But it's unclear to me what, if anything, is the analogous statement for a positive definite function.

Moreover, I know that positive definiteness arises naturally in the theory of the Fourier transform, in that to be positive definite is is sufficient for our function to be the Fourier transform of a non-negative, real-valued, continuous function. However, with an eye towards building intuitions, why should we believe that a Fourier transform should have this property? What, if anything, does this property have to say about "frequency space"?

Answer 1

Honestly, the half-angle geometric intuition doesn't speak to me, because I don't find angles in infinite-dimensional spaces super intuitive.

Bochner's theorem states that all continuous function that are positive-definite must be the Fourier transform of some non-negative (real) measure. So positive definite functions are essentially the same thing as the functions with positive amplitudes on their oscillations.

Bochner's theorem is helpful to spot lots of other positive functions, for instance:

• $e^{-|x|}$, as the Fourier transform of the Cauchy distribution $\frac 1 {1+x^2}$. In fact, $\frac 1 {1+x^2}$ itself is also positive definite.

• $\cos(ax)$, as the Fourier transform of the average of two (shifted) Dirac deltas.

• Gaussians.

• The Shannon Wavelet.

• etc...

I think positive definite functions arise in signal processing. For instance, the $sinc$ function, which forms the basis for sampling and representing band-limited signals, is a positive function. One way to see this is to remember it's the Fourier transform of the rectangle function, which is the density of a positive measure.

Finally, to get some intuition about Bochner's theorem, look at the continuous version of things: Write the positive definiteness property as a double integral (instead of a double discrete sum), you get $$\int\overline{\xi(x)}\int \phi(x-t)\xi(t)dt dx \geq 0\tag1$$ The inner integral is the convolution of $\phi$ and $\xi$. The outer integral is the $L^2$ inner product of that convolution with $\xi$. By Parseval's theorem, it's equal to the inner product of the Fourier transforms of those functions. Because the Fourier transform of a convolution is the product of the Fourier transforms, we can rewrite $(1)$ as $$\int \overline{\widehat{\xi}}(\omega)\widehat{\phi}(\omega)\widehat{\xi}(\omega)d\omega = \int \widehat{\phi}(\omega)|\widehat{\xi}(\omega)|^2d\omega \geq 0$$ Since this must be true for all $\xi\in L^2$, you can see why having $\widehat{\phi}(\omega)\geq 0$ is needed.