3
$\begingroup$

Context: I have done in the past stereo recordings in XY position (coincident microphones):

from a source far from at least 20 meters (example: piano in a big reverberant building).

Since the microphones are coincident, there are IID (interaural intensity differences), but, sadly, few ITD (interaural timing differences). Thus the recording is less lively, much less "outside your head" (when listening with headphones), that it could have been with a spaced pair of microphones. That's the final problem that I'm trying to solve, since I can't redo the recordings: respatialize the sound into something closer to a spaced-pair recording. (I have other recordings with a spaced pair non-coicident mics, and I confirm it would have been better). See also Algorithms to re-spatialize a stereo recording audio signal?.

(Any coding idea to achieve this general goal is welcome).

The option I'm now considering to achieve this goal in this question is to:

  • decompose the signal L[n], R[n] (left, right) into several layers
  • apply different ITD (Interaural Timing Differences) on each layer
  • mix the layers to get a new output signal

Example with Mid-Side:

Mid[n] = (L[n] + R[n]) / 2
Side[n] = (L[n] - R[n]) / 2

Out_L[n] = Mid[n] + Side[n + K1]    # K1 is a time-shifting parameter
Out_R[n] = Mid[n] - Side[n + K2]    # K2 is a time-shifting parameter

I tried this, but it does not really help to achieve the goal mentioned above.

Question: Is there a decomposition of a stereo signal that goes beyond than Mid (0°) + Side (90°) ?

Something like:

Layer1[n] =  Mid (0° to 30°)
Layer2[n] =  Diagonal (30° to 60°)
Layer3[n] =  Side (60° to 90°)

Note: The "Coincident Microphones" part of this answer of Algorithms to re-spatialize a stereo recording audio signal? could be useful, but I don't see exactly how to use this concretely in an applied algorithm.

Improve this question
$\endgroup$
9
  • $\begingroup$ Have you made sure there's no time differences? I ask, because: you say "it's a reverb room"; for me, as someone who comes more from a wireless communications than an audio background, that tells me there's reflections, which means that the two microphones actually do perceive different path realizations, and there's a lot of inherent spatial information you might not realize you have – these amplitude differences don't come from nowhere, right, they happen because the paths observed by each microphone are different in their impulse response? $\endgroup$
    – Marcus Müller
    Commented Mar 5, 2021 at 11:13
  • $\begingroup$ @MarcusMüller True, there are some, as a result of the different paths taken by the sound (reverberation on walls). But I don't see how I could enhance these time-differences to increase the sensation of space. One thing is nearly sure: the interaural timing differences is much bigger with a spaced-pair of mic (even of 30 centimenters), and it increases a lot the sensation of space. That's what I'd like to get from my actual XY recordings. $\endgroup$
    – g6kxjv1ozn
    Commented Mar 5, 2021 at 11:21
  • $\begingroup$ As said, really not coming from an Audio background, so, before even trying to find literature, I'd start to play around a bit. A gut feeling of mine would be this: assuming the L- and R-microphone-observed RIRs are indeed different, you would get two different "optimal" equalizers if you tried to flatten the spectrum of each channel individually, right. So, what if you intentionally enhanced the notability of the differences in the L- and R-RIR? Try to best-effort estimate an equalizer for each of them, separately, but then apply it to the "wrong" channel. That would equalize the 1/2 $\endgroup$
    – Marcus Müller
    Commented Mar 5, 2021 at 11:23
  • $\begingroup$ frequencies where indeed the spacing /orientation didn't lead to significant path differences correctly, but it would "worsen" the effect on those bandwidth where the different orientation did, in fact, lead to different phase and amplitude effects. If that leads to an audible change in "spacyness", I'd try to find the extremes in each EQ separately, and start trying to interpolate more smoothly between exactly these frequencies. $\endgroup$
    – Marcus Müller
    Commented Mar 5, 2021 at 11:26
  • $\begingroup$ If I remember correctly @MarcusMüller, I think I already tried to change the equalization of each channel separately to increase the perceived differences between channels (if I understand your suggestion correctly). This changes the amplitude in the spectrum of L vs. R channels. But I think the real difference with a spaced-pair recording is the time-differences, this is what should be increased. $\endgroup$
    – g6kxjv1ozn
    Commented Mar 5, 2021 at 11:26

2 Answers 2

Reset to default
3
$\begingroup$

There is a generic class of algorithms called "upmixers" that indeed de-compose a stereo signal into multiple direction and/or it's constituent directional channels. They typically work by chopping the signal into frames, evaluating correlation/phase/directional differences in multiple frequency bands, steering the origial signal on rules based on the analysis and then splicing them all together again.

It's tricky business and typically involves quite bit of trade off between the effectiveness of the spatial decomposition and spectral and temporal artifacts.

However, I'm not convinced that this would help in your case. You have a far field recording done in a reverberant space: the energy of your direct sound component (which is the only thing localizable) is probably tiny as compared to the energy of the early reflections and reverb. Trying to pull out directional components out of the reverberant field is hard and chances are they are decorrelated anyway, so applying time delays to them is unlikely to make much of a difference.

If you just want a little more space in the recording, you could try to boost the difference channel a bit with a high boost shelf filter. No boost for low frequencies, some for the highs. You can play around with the cutoff frequency, but I would vary the cutoff from 500Hz to 2kHz and do try boost of 2-3 dB or thereabouts.

If you are aiming for "more natural over headphones", your best shot is to match the inter-channel correlation as a function of frequency to that of a human (or dummy) head in a diffuse field. At low frequencies the two ear signals are highly correlated, at high frequencies they tend be very uncorrelated and they are partially correlated in between. You can analyze the correlation of your recording and compare to a binaural target curve and then increase/decrease correlation at specific frequencies as needed. There are various ways of doing that, but the easiest way is to adjust the sum/difference balance: more sum = more correlation, more difference = less correlation.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Thank you for your answer. If you just want a little more space in the recording, you could try to boost the difference channel a bit with a high boost shelf filter. No boost for low frequencies, some for the highs: yes you're right; I already did it, it improves the recording indeed. But it's still far from the other spaced-pair recordings I have which is 'another world' in terms of spatial sensation in the headphones. $\endgroup$
    – g6kxjv1ozn
    Commented Mar 5, 2021 at 14:00
  • $\begingroup$ then increase/decrease correlation at specific frequencies as needed: good idea. How would you do this in specific frequencies band? $\endgroup$
    – g6kxjv1ozn
    Commented Mar 5, 2021 at 14:07
  • $\begingroup$ Last question ;) There is a generic class of algorithms called "upmixers" that indeed de-compose a stereo signal into multiple direction and/or it's constituent directional channels: do you know an open-source toolbox or software that allows to try this concretely on my recordings (without having to re-code the whole algorithm)? $\endgroup$
    – g6kxjv1ozn
    Commented Mar 5, 2021 at 14:09
  • $\begingroup$ Upmixers tend to be IP heavy and proprietary. I'm not aware of a good open source implementation although some of the original patents may have expired by now. $\endgroup$
    – Hilmar
    Commented Mar 5, 2021 at 22:43
  • $\begingroup$ Re correlation: I would try to design a filter that's the inverse of your desired correlation boost and apply it to the difference channel. For starters you can try just splicing a few parametric biquads together. You probably need to re-EQ after this process, but as long as you EQ left and right identical, the correlation properties should stick. $\endgroup$
    – Hilmar
    Commented Mar 5, 2021 at 22:46
0
$\begingroup$

As is stated in the "Coincident Microphones" section of the linked answer, this approach won't work because the polar responses of the microphones are frequency dependent. This means that you cannot extract the directional information of each channel with a simple ("generic") formula that is valid only for low frequencies (well below the spatial aliasing of the system/microphone).

What you could try is, as already mentioned in comments, to decorrelate your two signals. This should increase the spatiousness somehow (I am not sure this will be achieved in some consistent way though). I haven't worked with decorrelation filters but I did manage to find two resources you could possibly have a look at. One is this short article (from Stanford University) and the other one is this paper which unfortunately you have to have an AES subscription to get access to (sorry for that). I am sure though that there must be more sources of information out there.

Another approach would be to try to use a bunch of all-pass filters to affect the phases of one or both the channels. This technique reminds a lot the diffuser part of a digital reverberation unit with the simplest one being the Schroeder all-pass design. You could watch this video on creating a digital reverberation unit for some more info on the diffuser.

Another possibility would be to try and add a simple delay to one of the two microphones in an attempt to "move them apart". As you mentioned, you already tried that with no good results. You could possibly try to do that in the frequency domain where you should "add a line" with constant slope to the phase of one of the two microphone signals. This shouldn't make any real difference with what you have already tried in the time domain but it will provide some extra options. For example, on top of the "line" you could add a small random number to each frequency's phase to make it somewhat random (conceptually you can think of it as an "imaginary" all-pass filter that moves only this frequency in time by a short amount).

One more option that the frequency domain approach provides is that you should be able to introduce inter-sample delays. To be honest it's been a while since the last time I worked on inter-sample delays and interpolations, but if I am not mistaken the designs I have seen did in one way or the other affect the magnitude of the signals. Thus, I am not sure that just by "scrambling" (in the way mentioned above) will provide completely meaningful results.

Please note that all above "solutions" constitute simple suggestions made on assumptions of what should work based on personal past experience. This means that all of the suggestions may provide a solution to some extend or none of them may provide any meaningful results. Nevertheless, it would be very instructive and beneficial to provide a solution (or more than one) if you reach one.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Sincerely, I do hope all above suggestions (along with the ones provided by other contributors in answers and/or comments) will lead to a solution to your problem. I would be delighted to find out what you did and what the results were. $\endgroup$
    – ZaellixA
    Commented Nov 26, 2021 at 11:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.