4
$\begingroup$

I have been studying the problem of room impulse responses (RIRs) interpolation for a couple of months. I am trying to use compressed sensing to reconstruct (at best) the sound field in the room with a better resolution, or (at least) get an RIR at some desired location where it was not measured before. "A better resolution" here means that by having only 20-30 measured RIRs in the room, I want to get the RIRs of, say, one million locations. That is, think of it as a better sampling of the sound field in the room, given the three dimensions of the room.
The problem is, in most RIR interpolation using CS papers, the domain of sparsity is not really described well. Most of the CS algorithms use some pre-defined domain of sparsity. When I think about it, RIRs are not sparse in time (generally around 3 seconds filled with data), neither are they sparse in frequency (an impulse utilizes all the frequencies from $-\infty$ to $\infty$). Does anyone have an idea of the sparsity of RIRs? Or, do you have any suggestions on how to tackle this problem using common and well-studied CS algorithms?
I want to give you my intuition. I believe that RIRs are sparse in some domain inspired by the spatial characteristics of the room. That is, one RIR is not sparse itself, and this is not what we are after. We are after the sparsity of the sound field in some domain. I do not know how to represent such a domain mathematically, but it can be thought of as follows: only a couple of samples of the sound field (a couple of RIRs) are enough to carry all the information of the sound field in the room (low entropy of the sound field), and hence projecting them onto that certain domain that inscribes this property can ease the task of CS. Then we can go back to the original domain (which is a 4D domain of 3 spatial dimensions and 1 temporal dimension) by taking the inverse of the transform that took us to the sparsity domain.
Any help is appreciated, especially if MATLAB is involved.

$\endgroup$
3
  • $\begingroup$ I think in this context the idea is to learn some dictionary which will yield sparse representation for many RIR's. $\endgroup$
    – Royi
    Dec 24, 2023 at 19:48
  • $\begingroup$ @Royi Yes fine, but how can I find that dictionary? Also, we need to enumerate over the possible representations in that dictionary, but then this will become computationally infeasible. Think about it in this way, we need a sparse representation of a sound field that is 4-dimensional. If we want to sample the dimensions of the room in mm, and the sound signal in 44.1kHz, then we will have something like 3000x3000x3000x44100 ≈ 1.2e15 for a 3x3x3 m^3 room and RIR's of 1 second each. This sounds like an impossible task for any commercial computer. $\endgroup$
    – Con
    Dec 25, 2023 at 17:43
  • $\begingroup$ I am not an expert about RIR. But probably I wouldn't look for a sparse representation per room as a 4D signal but per `$\left( x, y, z \right)$ over time. $\endgroup$
    – Royi
    Dec 26, 2023 at 18:18

1 Answer 1

4
$\begingroup$

That's tricky.

RIRs are NOT sparse in any obvious physical sense (time, frequency, etc). In fact they are insanely complicated with thousands of degrees of freedom. The amount of relevant physical parameters is very large: we have

  1. Position and orientation of the source
  2. Directional characteristic of the source
  3. Position and orientation of the receiver
  4. Directional characteristic of the receiver
  5. Geometry and acoustic properties of all surfaces including reflection, absorption, diffraction, resonance, diffusion, etc.

First of all you need to define which of this you want to interpolate. I think you want to the receiver location for a fixed source position in a fixed room, but it would help to specifically state that. And what about orientation? Natural sources and receivers are NOT omnidirectional so orientation matters.

I think your best chance to find any sparsity would be in the perceptual domain. In a reasonably well behaved room, when moving a relatively small distances from A to B, the fine structure of the RIR changes drastically (to the point of being mostly uncorrelated), but they often sound very similar. One potential way here would be to derive relevant perceptual attributes, interpolate those and and the resynthesize an RIR from that.

Another approach would be to use a parametric physical mode. The RIR can be broken down roughly in three areas

  1. Direct Sound
  2. Early reflections
  3. Late reverberation

The direct sound can often be calculated directly from the locations, orientations and polar patterns of source and receiver.

Early reflections are probably the most tricky one: You would need to identify the major reflections (Ceiling, floor, etc) in the measured RIRs and try then to interpolate "matching" reflections.

The late reverberation is fairly straight forward. You can use the frequency dependent decay envelop and just fill the fine structure with white noise. The thing here to interpolate are the reverb times and the direct/reflected energy ratio to the the reverb level correct.

Things get much more difficult in less "well-behaved" rooms. For example if you move from an area where there is direct line of sight to the source to one where the source is obstructed, the RIR changes fairly drastically over a relatively small distance. I don't think this can be interpolated with any reasonable means.

$\endgroup$
2
  • $\begingroup$ The interesting part is, people are publishing research papers that employ optimization to solve the problem simply by assuming some sparsity. I am not sure what difference it makes if an impulse is applied (hence RIRs are recorded as direct responses to it), or a sound source is applied (hence the output is a response to that specific sound source) then deconvolution is used to get the impulse response out of that. Is it the case that by applying a normal (hearable) sound we can assume sparsity in frequency (which is not the case with impulses and impulse responses)? $\endgroup$
    – Con
    Dec 28, 2023 at 1:58
  • $\begingroup$ Another problem is, how can I handle the horrendous high dimensionality of the problem? If I want to apply some 'mainstream' techniques such as trying what sinc interpolation might yield, I need over 300GB of memory, considering the numbers I mentioned in my comment on my question. This also hinders any application of machine learning methods or any type of interpolation over the dimensions of the room. $\endgroup$
    – Con
    Dec 28, 2023 at 2:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.