If you are trying to use the FFT to estimate the frequency and/or amplitude of a sinusoid you run into a problem when the frequency of your signal doesn't fall exactly on a FFT bin. In this case, the result of the FFT causes the energy to leak into nearby FFT bins - often called scalloping loss. The common approach to reduce scalloping loss is to use windows, e.g. Hamming, Blackman etc. While using windows reduces the scalloping effect there are additional unwanted sideeffects, two of which are: Coherent energy loss (loss of amplitude as you have seen), and the increase of the effect FFT bin size (FFT frequency resolution - nominally sampling frequency/FFT size).
There are a couple of possible solutions to your problem:
First, use what is called a Flattop window. This is a special window that tries to maintain the amplitude i.e. it tries to avoid to coherent energy loss. See here for an article by Richard Lyons describing this. Note there are several slightly different versions of Flattop windows out there.
An alternative method, is two use the FFT bin with the peak amplitude and the amplitude of the adjacent FFT bins. To interpolate both the frequency and the amplitude estimates. Typically, before you can estimate the amplitude you'll need to know the frequency. The interpolation mechanism can vary from: linear, quadratic, spline and others.
A couple references are here:
Ref 1 - by Matt Donadio
Ref 2 - Eric Jacobsen
Note - a couple of problems also arise when:
- Noise is present
- There are nearby sinusoids / signals that leak energy into the FFT bins you are using for your frequency estimation.