I am new to audio signal processing, but in one of my machine learning projects that I am working on, I need to down-sample my audio somewhere in the middle of my neural network. Therefore, I need a down-sampling method that can apply to a tensor with the shape (batch, len). (I am using PyTorch).

Since Max pooling and Average pooling are usually used to down-sample 2D images, I am thinking about if I can try my luck with them.

import numpy as np
import matplotlib.pyplot as plt
from scipy.signal import resample
import torch
from torch.nn import MaxPool1d, AvgPool1d, AdaptiveAvgPool1d

#-------------Generating Artifical Signal -----------------
fs = 44100
s = np.linspace(0,1,fs)
x = np.sin(2*np.pi*250*s, dtype=np.float32) + np.sin(2*np.pi*280*s, dtype=np.float32) \
    + np.sin(2*np.pi*480*s, dtype=np.float32) + np.sin(2*np.pi*1600*s, dtype=np.float32)
uprange = 800

#------------Comparing two resampling methods ------------
torch_downsample = AvgPool1d(2,2) # Will be used for downsampling
x_torch = torch.tensor(x)[None, None, :]
for i in range(6):
    uprange = uprange//2 # the number of samples being plotted out
    x = resample(x,len(x)//2) # downsample with fft resampling algorithm
    x_torch = torch_downsample(x_torch) #downsample with Pooling

    fig, ax = plt.subplots(1,2,figsize=(16,3)) # plot out the resampled waveforms
    fig.suptitle('Resampled {} time(s)'.format(i+1), fontsize=16)
    ax[0].plot(x[:uprange], '-b')
    ax[0].plot(x_torch.view(-1).numpy()[:uprange], '--r')

    z = np.fft.fft(x) # Analyzing the spectrum

    z = np.fft.fft(x_torch[0,0,:].numpy())
    ax[1].plot(abs(z)[:len(x)//2], '--r')
    print("len1 = {}, len2 = {}".format(x.shape, x_torch.shape))
    ax[0].legend(['fft_res', 'AvgPool'])
    ax[1].legend(['fft_res', 'AvgPool'])

My results

enter image description here

As you can see for the first 4 resamplings, the frequency components are still correct. But starting from the 5th resampling, there's a small defect at around 200Hz. The 6th resampling results in more defects.

My Questions

  1. Is there any method to prevent these defects?

  2. Also, am I doing downsampling correctly with Average pooling? I have no knowledge in audio signal processing, but I am trying hard to use my machine learning knowledge here.

Preferably, the solution can be implemented in PyTorch. (Don't tell me to use torchaudio, it messed up my whole PyTorch installation in an attempt to install it. I don't want to go through the trouble to trouble-shoot and reinstall everything)

  • $\begingroup$ It looks like you are seeing the results of aliasing from frequency content higher than half the final sampling rate. To avoid this you need to low pass filter the signal prior to decimation. $\endgroup$ Sep 15 '19 at 13:08
  • $\begingroup$ It seems there's no low pass filter in PyTorch. Is it easy to write a low pass filter from scratch? $\endgroup$ Sep 15 '19 at 14:13
  • $\begingroup$ Yes, there are many options- one is to install scipy.signal and use lfilter. Read the help docs on lfilter for more info. $\endgroup$ Sep 17 '19 at 12:26
  • $\begingroup$ Aren't Max pooling and Average pooling non-linear? So they would naturally cause distortion $\endgroup$
    – endolith
    Oct 9 '20 at 20:01

Read up on the sampling theorem, e.g. https://en.wikipedia.org/wiki/Nyquist%E2%80%93Shannon_sampling_theorem

If your feature vectors are still in the time domain and linearly derived from the original audio wave forms, you just need to low pass filter before down sampling.

You need to low-pass filter to avoid aliasing. There many different types of lowpass filters and the best choice depends on the specific requirements of your application (causality, latency, phase distortion, transients, size of transition band, MIPS & Memory, required aliasing rejection, pass band ripple tolerance, etc.)

As you lowpass filter, you lose information. If you need to down sample to a 500 Hz sampling rate all information above 250 Hz (in practice more like 220 Hz) is lost.

Lowpass filters are can be tricky to design depending on your requirements but the code to run them is very straight forward and can easily be done from scratch. Just make sure you use cascaded second order sections and Direct From I or Transposed Form II.

  • 2
    $\begingroup$ I voted + 1, but you should clarify that cascaded second-order sections apply for IIR filters, not FIR. $\endgroup$
    – Ben
    Oct 9 '20 at 17:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.