0
$\begingroup$

I am writing a sound library for my c++ hobby game engine. The sound library mixes music and sound effects into a single buffer that is sent to the audio hardware.

While mixing, any number of source sounds may appear or disappear between two output buffers causing a discontinuity at the border between the buffers.

The discontinuity can be heard as a disturbing high frequency click when the mix is played by the audio hardware.

I have solved this by doing internal bookkeeping of what sound sources are playing and neatly fading sound sources in and out. However this is more involved than it would first seem. The mixing code becomes difficult to read and maintain.

So I would prefer to instead simply handle any discontinuities in the final mix buffer before sending it to the sound card.

I have experimented with a wavelet transform solution which takes an area around the border between two buffers and adjusts any excessively large coefficients until they fit in with the overall distribution.

However that technique seems tricky and introduces a latency to the output because potentially quite old samples have to be updated if the discontinuity carries all the way down to low frequencies...

It would be preferable if the solution would only alter the new buffer and potentially carry over any residual errors at the end of the new buffer to be resolved in the next buffer.

Are there any known techniques for this?

Any help is greatly appreciated!

Improve this question
$\endgroup$

3 Answers 3

Reset to default
3
$\begingroup$

In 1998 I made a "zero level adjustment click removal" system that shifted the zero level to fix the discontinuity, and this "zero level" was gradually moved back to 0, by decreasing it by 1/2048 of its value every sample. It worked well for low-frequency sounds, but when a high-frequency sound ended abruptly at a large sample value, that created a low-frequency click. I ended up removing the system and simply adding more channels in order to fade out sounds. For this I used a one-pole lowpass filter on all volume envelopes, and when a sound was killed the volume envelope was set to zero and some time was allowed for the fadeout before making the channel available to other sounds. If too many channels were reserved for fading out, the one with the smallest amplitude value was freed to use.

But the thought kept bugging me that if the zero level adjustment would be done only to a lowpass filtered portion of the output, then it might sound better. Purely high-frequency sounds don't really need click removal.

Improve this answer
$\endgroup$
7
  • $\begingroup$ wow! Olli! your rep is shooting up like a diracian in the 2 weeks you've been around. when you first appeared, i nominated you for an initial infusion of 100 like i got, but now it looks like you won't need it. anyway, i don't understand why this is not a much simpler problem of windowing (or half windowing). a.k.a. fading or applying a decaying envelope. did these sound effects have a large DC component that you had to "gradually move back to 0" in 2048 samples? instead of ramping the amplitude, you just spliced a ramp to the audio itself? $\endgroup$
    – robert bristow-johnson
    Commented Apr 19, 2015 at 18:14
  • $\begingroup$ Thank you for your great answer Olli. I tried the solution you mentioned with shifting the zero level and I got the same result. C0 continuity was not enough to remove the clicks entirely. The system that I have now is similar to what you describe that you ended up with, only I don't set a strict limit on the number of channels, it is allowed to grow over the budget if needed to handle fade-outs and fade-ins... $\endgroup$
    – Daniel Peterson
    Commented Apr 19, 2015 at 18:24
  • $\begingroup$ Hoever, I am curios about the lowpass filter on the volume envelope. Does this mean that you faded high frequencies faster than low frequencies? $\endgroup$
    – Daniel Peterson
    Commented Apr 19, 2015 at 18:26
  • $\begingroup$ @robertbristow-johnson: Fading means you need to generate (compute) both the faded in and the faded out sound, and I was trying to avoid that. To Daniel it seems to be more of a problem arising from the architecture of his software. Yes it is equivalent to summing ramps to the audio. The nice thing was that you could combine the ramps from multiple channels into one big ramp. $\endgroup$
    – Olli Niemitalo
    Commented Apr 19, 2015 at 18:31
  • $\begingroup$ @DanielPeterson: No, it means I softened fast changes in the volume envelope, such as the abrupt muting of the sound when it ends. The thing generating the envelopes had no idea that they were to be filtered in the mixer. $\endgroup$
    – Olli Niemitalo
    Commented Apr 19, 2015 at 18:37
2
$\begingroup$

Using envelopes to fade in and fade out sounds when they start and end (or end early) is the common solution in game development both for audio as well as skeletal animation tracks believe it or not. From the sounds of it, the issue may be one of code architecture, and less of proper technique. Ollie's info is also good. I've seen things like some of what he's talking about in skeletal animation systems, but also in sound systems when you need to "hard evict" some sound instead of fading out because the number of sounds has hit a hard ceiling and you absolutely need to play a new sound.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ I see, I might have to scout for some good implementation then that I can copy... I will probably dive some more into this problem in the meantime though. Thank you for your answer! $\endgroup$
    – Daniel Peterson
    Commented Apr 19, 2015 at 18:31
  • 1
    $\begingroup$ it seems to me that your "hard evict" in the game audio context is the same as what we call "note stealing" in a synthesizer with a limited number of voices. someone hits a key and you gotta play the note, but there aren't enough voices left to play the note, so you have to pick on some voices (a note often contains multiple voices layered on top of each other) that are most nearly silent and bump them out to free resources for the new note. and you have to do that fast. $\endgroup$
    – robert bristow-johnson
    Commented Apr 19, 2015 at 18:42
1
$\begingroup$

It is not possible to do this trivially because any correction based on momentary values of the input signal will introduce unpleasant distortion. A first order filter have low-pass properties which is unacceptable. A higher order filter will introduce ringing.

Any solution based on signal analysis is moot because ultimately they will all be more complicated than the naive solution which is to do the bookkeeping mentioned in the question.

The simplest implementation of the bookkeeping solution that I have been able to come up with is to have a list of sounds sorted on priority and two counters. One counter for the number playing voices and one for the number of queued voices. The list is traversed once forwards from [0, totalVoiceCount] and once backwards from [end, totalVoiceCount]. In the forwards step, sounds are played, faded in and queued. In the backwards step, sounds are faded out, and played.

All other discontinuities (such as volume and pitch) are fixed by lowpass filtering the variables as mentioned in the answer by Olli.

Hopes this helps someone! :)

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.