0
$\begingroup$

I have a 4 channels audio file. As a pre-processing step, it goes through an STFT with $T$ frames and $M$ frequencies. The output is divided into magnitude and phase per channel, so for $C$ channels, we have a $T\times M\times 2\cdot C$ tensor.

I wish to focus on the channel with the highest energy per frame $T$. If the STFT had no overlap, I could simply take the highest energy channel per frame and concatenate to a single channel audio file. The $T$ frames, however, are produced by 50% overlapping windows.

In my case for example:

Audio length = 2880000

number of channels, $C$ = 4

Window Length = 1920

hop size = 960

number of FFT bins = 2048

number of frequencies, $M$ = 1024

Hence the output size:

$T = \frac{2880000}{960} + 1$ = 3001

How do I produce a single channel audio file, with the highest energy per window?

For reproduction: I am using librosa.core.stft for the STFT with a Hanning window.

Improve this question
$\endgroup$
14
  • $\begingroup$ your window length is 1920 samples and the FFT is 2048. does that mean that you are zero-padding with 128 zeros for each frame? and how big is $M$? is $M=1023$? or are you having a fewer number of bands? $\endgroup$
    – robert bristow-johnson
    Commented Apr 14, 2019 at 7:22
  • $\begingroup$ @robertbristow-johnson I have edited with M=1024. As the documentation of librosa.core.stft states, I am zero padding the window length to match the n_fft length. $\endgroup$
    – havakok
    Commented Apr 14, 2019 at 7:30
  • $\begingroup$ I am not sure what you mean by adding two adjacent FFT bins. I am using a predefined function on the whole audio file. I did not manually implement the FFT and I am not sure what is going on inside the implementation. $\endgroup$
    – havakok
    Commented Apr 14, 2019 at 7:36
  • $\begingroup$ you changed your $M$. it makes sense now. before you had 1024 positive frequency bins in the FFT result, but $M$=512. now $M$=1024 and it makes perfect sense now. $\endgroup$
    – robert bristow-johnson
    Commented Apr 14, 2019 at 7:37
  • $\begingroup$ Yep, I did. My bad.. $\endgroup$
    – havakok
    Commented Apr 14, 2019 at 7:38

1 Answer 1

Reset to default
1
+50
$\begingroup$

The obvious approach to this would be a crossfade between channels. This leads to problems, just as obvious, as you would get nasty comb filter effects at 2000Hz and above. So this is a no-go.

A solution just as simple would be to use just one channel, but apply a gain at each window, so that the energy is aligned with the one of the loudest channel at that point in time, i.e. the gain would be $$a_i = \frac{\max\limits_{c}(E_i^c)}{E_i^1}$$ for $c$ as channel index, $i$ as time index and assuming you use channel $1$. This way, you avoid any phase problems whatsoever and still have a signal, that suffices your energy requirement. Also, there is no non-linear processing and thus, no artifacts resulting from it, which would most certainly occur, if you follow some approach with time aligning the signals from different channels to avoid comb filter effects.

EDIT: I know that the overall SNR will not be improved by this approach, but I think the advantages are worth it, since SNR differences should be quite small, as distance between microphones is also small.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.