Spectral centroid manipulations

Question

So, I've created a simple sound analysis application and one of the features I've implemented is the spectral centroid (as explained here https://en.wikipedia.org/wiki/Spectral_centroid). In order to get reliable results, we need to set a threshold on our spectral centroid data because the FFT always returns some energy in bins that really don't have any energy.

Now, I would like to be able to manipulate the spectral centroid (e.g push the values up or down, which to some extent resembles pitch shifting) and then use an inverse FFT operation to convert the values to waveform data. This way, we'd be able to hear the changes but I'm not sure how to do this.

The spectral centroid is calculated by using magnitudes, not raw FFT data. The magnitudes are calculated as given below

mag[i] = 10*log10(sqrt((pFFTLReal[i] * pFFTLReal[i] + pFFTLImg[i] * pFFTLImg[i])/fftNorm));//fftNorm is basically a correction factor that depends on a type of window (blackman, hamming etc)

So, given all this, how would I change the spectral centroid by, say, 10 percent and see that change in the generated (manipulated) waveform?

Is this even possible?

Please, if this is not clear enough, I'll try to provide some more information.

Thank you

Answer 1

It's been some time since I asked this question and after some work done on this subject, I think it's time to revisit it. It should be noted that while the spectral centroid, pretty much like any other spectral feature, is calculated using the FFT magnitudes, and not raw FFT data, we don't have to manipulate the magnitudes directly. Instead, we modify FFT data, which in turn, affects the magnitudes. This makes the process a lot easier than having to manipulate the magnitudes.

The spectral centroid is a single value (per FFT frame) so in the case of STFT we have as many centroid as we have STFT frames. The process of manipulating the spectral centroid is pretty straightforward. If, for instance, the spectral centroid is 600Hz and we wish to increase its value by 10% so that the new value is 660Hz, we would simply rearrange our frequency bins by a certain amount (a bin step). The actual calculation is qute simple

const binStep = frameLenHalf - frameLenHalf/(1+(float)freqDelta/100); 
   //frequency delta % (e.g. 10%, 20%, -10%, -20% etc)

    //for increasing frequency
    float *pCfftLReal = new float[frameLen];
    float *pCfftLImg =  new float[frameLen];
    //for increasing frequency
    for(j=frameLenHalf-1;j>binStep;j--){
            pCfftLReal[j] = pFFTReal0[j-binStep];
            pCfftLImg[j] = pFFTImg0[j-binStep];
                }
        }
    //for decreasing frequency
    for(int j=0; j<frameLenHalf-1-binStep; j++){
            pCfftLReal[j] = pFFTReal0[j+binStep];
            pCfftLImg[j] = pFFTImg0[j+binStep];
                } 

Needless to say, this process suffers from rounding errors (primarily calculating the binStep), but in general, the larger the bin resolution, the larger the errors. This can be alleviated by having a greater frame length (but for STFT this is usually undesirable). Other more complex approaches are also possible but I haven't investigated any of these.

Also, sometimes the results can be skewed if the amount of frequency shift exceeds the Nyqiust frequecy and we keep the original sample rate (upsampling would help here).

Before we can actually listen to our manipulated sound, an inverse FFT shoud be carried out on our modified data.

What I've described here is just a rough, naive approach to spectral centroid manipulation. It should also be said that this kind of manipulation doesn't retain the original harmonics ratio, given that the frequency delta shift is applied equally on every bin. Keeping the harmonics ratio intact would require calculating a new bin step for every frequency, which is only slightly more complex that the algo given above.

I've tried both approaches (with a constant bin step and a changing bin step) and perceptually, the first doesn't really change the original sound ( I guess this has to do with higher frequencies affecting timbre more than lower ones and the first approach changes the higher frequencies less than it does the lower ones).

I'm not done with this yet, so I will update this post as soon as I have something interesting to share.