Beyond “mid-side”: decompose an audio stereo recording (coincident microphones) into Mid + 45° + 90°

Context: I have done in the past stereo recordings in XY position (coincident microphones):

from a source far from at least 20 meters (example: piano in a big reverberant building).

Since the microphones are coincident, there are IID (interaural intensity differences), but, sadly, few ITD (interaural timing differences). Thus the recording is less lively, much less "outside your head" (when listening with headphones), that it could have been with a spaced pair of microphones. That's the final problem that I'm trying to solve, since I can't redo the recordings: respatialize the sound into something closer to a spaced-pair recording. (I have other recordings with a spaced pair non-coicident mics, and I confirm it would have been better). See also Algorithms to re-spatialize a stereo recording audio signal?.

(Any coding idea to achieve this general goal is welcome).

The option I'm now considering to achieve this goal in this question is to:

• decompose the signal L[n], R[n] (left, right) into several layers
• apply different ITD (Interaural Timing Differences) on each layer
• mix the layers to get a new output signal

Example with Mid-Side:

Mid[n] = (L[n] + R[n]) / 2
Side[n] = (L[n] - R[n]) / 2

Out_L[n] = Mid[n] + Side[n + K1]    # K1 is a time-shifting parameter
Out_R[n] = Mid[n] - Side[n + K2]    # K2 is a time-shifting parameter


I tried this, but it does not really help to achieve the goal mentioned above.

Question: Is there a decomposition of a stereo signal that goes beyond than Mid (0°) + Side (90°) ?

Something like:

Layer1[n] =  Mid (0° to 30°)
Layer2[n] =  Diagonal (30° to 60°)
Layer3[n] =  Side (60° to 90°)


Note: The "Coincident Microphones" part of this answer of Algorithms to re-spatialize a stereo recording audio signal? could be useful, but I don't see exactly how to use this concretely in an applied algorithm.

• Have you made sure there's no time differences? I ask, because: you say "it's a reverb room"; for me, as someone who comes more from a wireless communications than an audio background, that tells me there's reflections, which means that the two microphones actually do perceive different path realizations, and there's a lot of inherent spatial information you might not realize you have – these amplitude differences don't come from nowhere, right, they happen because the paths observed by each microphone are different in their impulse response? – Marcus Müller Mar 5 at 11:13
• @MarcusMüller True, there are some, as a result of the different paths taken by the sound (reverberation on walls). But I don't see how I could enhance these time-differences to increase the sensation of space. One thing is nearly sure: the interaural timing differences is much bigger with a spaced-pair of mic (even of 30 centimenters), and it increases a lot the sensation of space. That's what I'd like to get from my actual XY recordings. – g6kxjv1ozn Mar 5 at 11:21
• As said, really not coming from an Audio background, so, before even trying to find literature, I'd start to play around a bit. A gut feeling of mine would be this: assuming the L- and R-microphone-observed RIRs are indeed different, you would get two different "optimal" equalizers if you tried to flatten the spectrum of each channel individually, right. So, what if you intentionally enhanced the notability of the differences in the L- and R-RIR? Try to best-effort estimate an equalizer for each of them, separately, but then apply it to the "wrong" channel. That would equalize the 1/2 – Marcus Müller Mar 5 at 11:23
• frequencies where indeed the spacing /orientation didn't lead to significant path differences correctly, but it would "worsen" the effect on those bandwidth where the different orientation did, in fact, lead to different phase and amplitude effects. If that leads to an audible change in "spacyness", I'd try to find the extremes in each EQ separately, and start trying to interpolate more smoothly between exactly these frequencies. – Marcus Müller Mar 5 at 11:26
• If I remember correctly @MarcusMüller, I think I already tried to change the equalization of each channel separately to increase the perceived differences between channels (if I understand your suggestion correctly). This changes the amplitude in the spectrum of L vs. R channels. But I think the real difference with a spaced-pair recording is the time-differences, this is what should be increased. – g6kxjv1ozn Mar 5 at 11:26

There is a generic class of algorithms called "upmixers" that indeed de-compose a stereo signal into multiple direction and/or it's constituent directional channels. They typically work by chopping the signal into frames, evaluating correlation/phase/directional differences in multiple frequency bands, steering the origial signal on rules based on the analysis and then splicing them all together again.

It's tricky business and typically involves quite bit of trade off between the effectiveness of the spatial decomposition and spectral and temporal artifacts.

However, I'm not convinced that this would help in your case. You have a far field recording done in a reverberant space: the energy of your direct sound component (which is the only thing localizable) is probably tiny as compared to the energy of the early reflections and reverb. Trying to pull out directional components out of the reverberant field is hard and chances are they are decorrelated anyway, so applying time delays to them is unlikely to make much of a difference.

If you just want a little more space in the recording, you could try to boost the difference channel a bit with a high boost shelf filter. No boost for low frequencies, some for the highs. You can play around with the cutoff frequency, but I would vary the cutoff from 500Hz to 2kHz and do try boost of 2-3 dB or thereabouts.

If you are aiming for "more natural over headphones", your best shot is to match the inter-channel correlation as a function of frequency to that of a human (or dummy) head in a diffuse field. At low frequencies the two ear signals are highly correlated, at high frequencies they tend be very uncorrelated and they are partially correlated in between. You can analyze the correlation of your recording and compare to a binaural target curve and then increase/decrease correlation at specific frequencies as needed. There are various ways of doing that, but the easiest way is to adjust the sum/difference balance: more sum = more correlation, more difference = less correlation.

• Thank you for your answer. If you just want a little more space in the recording, you could try to boost the difference channel a bit with a high boost shelf filter. No boost for low frequencies, some for the highs: yes you're right; I already did it, it improves the recording indeed. But it's still far from the other spaced-pair recordings I have which is 'another world' in terms of spatial sensation in the headphones. – g6kxjv1ozn Mar 5 at 14:00
• then increase/decrease correlation at specific frequencies as needed: good idea. How would you do this in specific frequencies band? – g6kxjv1ozn Mar 5 at 14:07
• Last question ;) There is a generic class of algorithms called "upmixers" that indeed de-compose a stereo signal into multiple direction and/or it's constituent directional channels: do you know an open-source toolbox or software that allows to try this concretely on my recordings (without having to re-code the whole algorithm)? – g6kxjv1ozn Mar 5 at 14:09
• Upmixers tend to be IP heavy and proprietary. I'm not aware of a good open source implementation although some of the original patents may have expired by now. – Hilmar Mar 5 at 22:43
• Re correlation: I would try to design a filter that's the inverse of your desired correlation boost and apply it to the difference channel. For starters you can try just splicing a few parametric biquads together. You probably need to re-EQ after this process, but as long as you EQ left and right identical, the correlation properties should stick. – Hilmar Mar 5 at 22:46