2
$\begingroup$

I have implemented sample-rate conversion (SRC) from scratch in my hobby music player, and it worked fine. However, originally, the music player loaded entire audio files into memory and performed the sample-rate conversion on the entire audio signal during loading. I've now changed this to instead stream the audio file chunk by chunk (to reduce memory footprint), and tried to port my SRC implementation.

But I'm encountering a problem where it I hear clicks between each chunk when they're played back. Increasing the size of each chunk of the audio file I load at a time increases the duration between the clicks, so that's how I'm pretty confident that they appear between chunks. Other than these artifacts, the SRC works/sounds fine.

I'm wondering why this happens? My theory is that when performing SRC on the entire audio signal in one go (initial version) each sample in the upsampled signal was filtered using the next N samples in the upsampled signal (where N is the length of my filter). However, in this new version, the last few samples of each chunk are filtered with fewer and fewer samples (the filter length basically shrinks) as the next chunk isn't available yet (this also happened in the original version when reaching the end of the entire audio signal). This then creates a discontinuity when playing back the next chunk as the final filtered samples don't take the next chunk's samples into account. Does that make sense, and if so, do I solve this by simply ensuring this doesn't happen?

Edit To add a bit more info:

I'm using an FIR filter, that is a Sinc function with a Hamming window of length 64, and I apply that to each of the samples in my upsampled (zero-stuffed) signal. Then I decimate the signal afterwards

Here's how I perform the filtering currently:

byte_t* upsampled_audio_data = .. // Zero-stuffed upsampled signal
int32_t L = ... // Upsampling factor
int32_t M = ... // Decimation factor
for (uint8_t channel = 0; channel < audio_loader_data->channel_count; channel++)
{
    int32_t channel_offset = sample_count_upsampled * channel;
    for (int32_t i = 0; i < sample_count_upsampled; i++)
    {
        int32_t current_sample_index = channel_offset + i;
        // This is where I shrink the filter if there aren't enough remaining samples to use
        int32_t filter_length_for_sample = min(filter_length, sample_count_upsampled - i);
        // j_offset is used to skip zero-stuffed samples as they are 0 anyway
        int32_t j_offset = L - (i % L);
        float filtered_sample = 0.0f;
        // Assumes 16 bit per sample for now
        for (int32_t j = j_offset; j < filter_length_for_sample; j += L)
        {
            filtered_sample += (float)((int16_t*)upsampled_audio_data)[current_sample_index + j] * filter[j];
        }
        ((int16_t*)upsampled_audio_data_filtered)[current_sample_index] = filtered_sample;
    }
}
$\endgroup$
11
  • 1
    $\begingroup$ how do you specifically do your resampling? The resampler algorithms I know don't have your problem! It worries me that you talk about "chunks". That's not how you resample on the fly. $\endgroup$ May 5, 2022 at 20:40
  • $\begingroup$ @Fat32 basically, the "usual" (i.e., non-block based! That really seems to be the problem!) resamplers: The rational resampler implemented with polyphase filter decomposition (i.e. just a more efficient impl of upsample by $M$ -> anti-image filter -> anti-alias filter -> downsample by $N$), the MMSE interpolator (approximate the ideal infinitely long interpolation filter with a $\ell$-tap (e.g. $\ell=8$) filter for each of a set of fractional delays), the polyphase filterbank arbitrary resampler. $\endgroup$ May 5, 2022 at 21:00
  • 1
    $\begingroup$ fun fact, you can implement both the polyphase component filters in the rational resampler and the arbitrary PFB resampler using overlap-add (or -save) methods, if these filters become large enough to be a burden. $\endgroup$ May 5, 2022 at 21:01
  • $\begingroup$ So, I have a Sinc filter with a Hamming window of length 64, and I apply that to each of the samples in my upsampled (zero-stuffed) signal. Then I decimate the signal afterwards. I'm not very familiar with this stuff, but to me it seems the only difference between my two scenarios is that the filter "shrinks" towards the end of each chunk. Now this also happened in the original version, but only at the very end, so no "click" would be heard since playback stops. $\endgroup$
    – MulattoKid
    May 5, 2022 at 21:32
  • 2
    $\begingroup$ I also asked on Twitter (twitter.com/MuIattoKid/status/1522310846055165955), and it seems the consensus is I basically need to do what @hotpaw2 suggests below - I need to filter across chunks $\endgroup$
    – MulattoKid
    May 6, 2022 at 11:07

2 Answers 2

5
$\begingroup$

You just need to save the last $N-1$ samples from the previous frame and prepend them to the data from your current frame, where $N$ is the filter length.

An FIR filter is designed as

$$y[n] = \sum_{k=0}^{N-1}h[k]\cdot x[n-k]$$

That means that each output is a function of the current input sample and the previous $N-1$ input samples. That's why you need to carry them over from frame to frame. For the first frame, you can just use zeros.

There are various different ways how to do this in C/C++ (circular buffer, combined state and data buffer, frame + bulk buffer, etc) but if you are not overly worried about efficiency, each one will work fine.

$\endgroup$
1
  • $\begingroup$ Yup, makes sense - I was just confused yesterday when first making the change, but thinking about it now it makes complete sense why it didn't work and I heard "clicks"! $\endgroup$
    – MulattoKid
    May 6, 2022 at 16:50
3
$\begingroup$

If you delay your output by N samples (or more), you will always have N samples of input after the new (interpolated or output) sample point available for use by your filter and/or interpolator.

Or instead of considering it an output delay, you can consider the same thing to be an input fetch-ahead. Of blocks or samples.

$\endgroup$
1
  • $\begingroup$ Yeah, that's what I'm thinking of doing. The more I think about it, the more it makes sense $\endgroup$
    – MulattoKid
    May 6, 2022 at 5:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.