I am reading this paper about density estimation (Appendix A), where the authors apply a Fourier transform to the estimated probability density (the $X_j$ are a sample of $N$ data points drawn independently from an unknown probability density function $f(x)$):

$$ \hat{f}(x) = \frac1N \sum^N_{j=1} K(x-X_j) $$

The authors now note that they apply the convolution theorem and the Fourier transform (where the Fourier transform of the function $g(x)$ is defined by $\Phi_g(t) = \int \exp\lbrace\mathrm{i}xt\rbrace g(x) \mathrm{d}x$), and obtain:

$$ \Phi_{\hat{f}}(t) = \kappa(t) \frac1N \sum^N_{t=1} \exp\lbrace\mathrm{i}tX_j\rbrace $$

where $\kappa(t)$ is the Fourier transform of the Kernel $K$.

Now, I am a bit stuck on how they obtained this form and how exactly they used the convolution theorem here. In particular, what happened to the argument $X_j$ in the kernel?

EDIT: Ok, I think I got it. Is this the right approach?

We apply the Fourier transform and obtain:

$$ \Phi_{\hat{f}}(t) = \frac1N \sum^N_{j=1} \int K(x-X_j) \exp\lbrace\mathrm{i}xt\rbrace\mathrm{d}x $$

We can easily rewrite this equation as an integral over a Dirac delta distribution:

$$ \Phi_{\hat{f}}(t) = \frac1N \sum^N_{j=1} \iint K(y) \delta(y-(x-X_j)) \exp\lbrace\mathrm{i}xt\rbrace\mathrm{d}x\mathrm{d}y $$

Integrating over $x$ and using the symmetry of $\delta$ we obtain:

$$ \Phi_{\hat{f}}(t) = \frac1N \sum^N_{j=1} \int K(y) \exp\lbrace\mathrm{i}(y+X_j)t\rbrace\mathrm{d}y $$

We see that we can move factor involving $X_j$ outside the integral, and are left with the result:

$$ \Phi_{\hat{f}}(t) = \frac1N \sum^N_{j=1} \exp\lbrace\mathrm{i}X_jt\rbrace \int K(y) \exp\lbrace\mathrm{i}yt\rbrace\mathrm{d}y $$

where the first factor is $\Delta(t) = \frac1N \sum^N_{j=1} \exp\lbrace\mathrm{i}X_jt\rbrace$, and the second factor is $\kappa(t) = \int K(y) \exp\lbrace\mathrm{i}yt\rbrace\mathrm{d}y$, exactly as in the paper.

I am still not sure in how far they applied the convolution theorem here?


It's a bit contrived, but observe that by the "sifting property" of Dirac-delta function:

$$ K(x-X_j) = K(x) * \delta(x-X_j) $$

where $*$ denotes convolution and $\delta$ is the Dirac-delta function.

Now you can apply Fourier convolution theorem and write:

$$ \mathcal{F}\{K(x-X_j)\}(t) = \kappa(t) e^{j t X_j} $$

where $\mathcal{F}\{g(x)\}(t)$ denotes the Fourier transform of a function $g(x)$.

Finally using the linearity property of the Fourier transform:

$$ \mathcal{F}\{\hat{f}(x)\}(t) = \frac{1}{N} \sum_{j=1}^N \mathcal{F}\{K(x-X_j)\}(t) = \kappa(t) \sum_{j=1}^N e^{j t X_j}. $$

  • 1
    $\begingroup$ Awesome, thanks a lot. Apparently this is known as the “sifting property” of the Dirac-delta. $\endgroup$ – mSSM Feb 9 '17 at 18:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.