0
$\begingroup$

I am new to audio signal processing, but in one of my machine learning projects that I am working on, I need to down-sample my audio somewhere in the middle of my neural network. Therefore, I need a down-sampling method that can apply to a tensor with the shape (batch, len). (I am using PyTorch).

Since Max pooling and Average pooling are usually used to down-sample 2D images, I am thinking about if I can try my luck with them.

import numpy as np
import matplotlib.pyplot as plt
from scipy.signal import resample
import torch
from torch.nn import MaxPool1d, AvgPool1d, AdaptiveAvgPool1d

#-------------Generating Artifical Signal -----------------
fs = 44100
s = np.linspace(0,1,fs)
x = np.sin(2*np.pi*250*s, dtype=np.float32) + np.sin(2*np.pi*280*s, dtype=np.float32) \
    + np.sin(2*np.pi*480*s, dtype=np.float32) + np.sin(2*np.pi*1600*s, dtype=np.float32)
uprange = 800
plt.figure(figsize=(8,3))
plt.plot(x[:uprange])

#------------Comparing two resampling methods ------------
torch_downsample = AvgPool1d(2,2) # Will be used for downsampling
x_torch = torch.tensor(x)[None, None, :]
for i in range(6):
    uprange = uprange//2 # the number of samples being plotted out
    x = resample(x,len(x)//2) # downsample with fft resampling algorithm
    x_torch = torch_downsample(x_torch) #downsample with Pooling

    fig, ax = plt.subplots(1,2,figsize=(16,3)) # plot out the resampled waveforms
    fig.suptitle('Resampled {} time(s)'.format(i+1), fontsize=16)
    ax[0].plot(x[:uprange], '-b')
    ax[0].plot(x_torch.view(-1).numpy()[:uprange], '--r')

    z = np.fft.fft(x) # Analyzing the spectrum
    ax[1].plot(abs(z)[:len(x)//2])

    z = np.fft.fft(x_torch[0,0,:].numpy())
    ax[1].plot(abs(z)[:len(x)//2], '--r')
    print("len1 = {}, len2 = {}".format(x.shape, x_torch.shape))
    ax[0].legend(['fft_res', 'AvgPool'])
    ax[1].legend(['fft_res', 'AvgPool'])
    plt.show()

My results

enter image description here

As you can see for the first 4 resamplings, the frequency components are still correct. But starting from the 5th resampling, there's a small defect at around 200Hz. The 6th resampling results in more defects.

My Questions

  1. Is there any method to prevent these defects?

  2. Also, am I doing downsampling correctly with Average pooling? I have no knowledge in audio signal processing, but I am trying hard to use my machine learning knowledge here.

Preferably, the solution can be implemented in PyTorch. (Don't tell me to use torchaudio, it messed up my whole PyTorch installation in an attempt to install it. I don't want to go through the trouble to trouble-shoot and reinstall everything)

$\endgroup$
  • $\begingroup$ It looks like you are seeing the results of aliasing from frequency content higher than half the final sampling rate. To avoid this you need to low pass filter the signal prior to decimation. $\endgroup$ – Dan Boschen Sep 15 at 13:08
  • $\begingroup$ It seems there's no low pass filter in PyTorch. Is it easy to write a low pass filter from scratch? $\endgroup$ – Raven Cheuk Sep 15 at 14:13
  • $\begingroup$ Yes, there are many options- one is to install scipy.signal and use lfilter. Read the help docs on lfilter for more info. $\endgroup$ – Dan Boschen Sep 17 at 12:26
0
$\begingroup$

Read up on the sampling theorem, e.g. https://en.wikipedia.org/wiki/Nyquist%E2%80%93Shannon_sampling_theorem

If your feature vectors are still in the time domain and linearly derived from the original audio wave forms, you just need to low pass filter before down sampling.

You need to low-pass filter to avoid aliasing. There many different types of lowpass filters and the best choice depends on the specific requirements of your application (causality, latency, phase distortion, transients, size of transition band, MIPS & Memory, required aliasing rejection, pass band ripple tolerance, etc.)

As you lowpass filter, you lose information. If you need to down sample to a 500 Hz sampling rate all information above 250 Hz (in practice more like 220 Hz) is lost.

Lowpass filters are can be tricky to design depending on your requirements but the code to run them is very straight forward and can easily be done from scratch. Just make sure you use cascaded second order sections and Direct From I or Transposed Form II.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.