1
$\begingroup$

I am attempting to create a pitch correction algorithm. I started by performing a test. The test goes as such:

  1. Get WAV file
  2. Split it into bins of size n (512 in my case)
  3. Shift each bin by 2 semitones (using a high level pitch shifting algorithm)
  4. Aggregate all shifted bins together to recreate audio file

However, when I do this, a large amount of noise relative to the size of "n" is generated (spectral leakage maybe?) The smaller the size n, the larger the amount of noise

How do I implement the pitch shifting on a bin by bin basis while minimizing the noise, and how should I adjust for the phase shifting that the individual pitch shifts create?

Audio clips and code here (last clip in notebook is the one with all the added noise): https://colab.research.google.com/drive/1cpRhPpvXY_9XZidjOLKk_wW15EnkqLEX?usp=sharing

My code that attempted to fix the problem, but failed:

def win_taper(N, a):
    R = int(N * a / 2)
    r = np.arange(0, R) / float(R)
    win = np.r_[r, np.ones(N - 2*R), r[::-1]]
    stride = N - R - 1
    return win, stride

def pshift(key, x, f, G, overlap=0):

    notes = frequencies(key)

    N = len(x)
    y = np.zeros(N)

    win, stride = win_taper(G, overlap)

    for n in range(0, len(x) - G, stride):
      w = manipulate(x[n:n+G] * win, sr, f)
      y[n:n+G] += w
    return y

Tapered function taken from here: https://lcav.gitbook.io/dsp-labs/granular-synthesis/effect_description

$\endgroup$
  • $\begingroup$ Can I please ask if this was resolved? $\endgroup$ – A_A Jun 2 at 8:43
  • $\begingroup$ Nope not really. I started by using a sort of cross-fade between each window to negate the artifacts, but a wobbly sound was introduced that is undesired. I am back to square one. $\endgroup$ – niallmandal Jun 2 at 23:03
1
$\begingroup$

...when I do this, a large amount of noise relative to the size of "n" is generated (spectral leakage maybe?) The smaller the size n, the larger the amount of noise

No. This is not "spectral leakage". The flutter that you hear comes from the fact that you are pitch shifting each signal frame without taking into account the phase shifting caused by the pitch shifting of the earlier blocks.

To put it in a different way: When you work with an audio recording, the "zero" for time is the first sample and that is the starting point when evaluating phase. By pitch shifting block $n$, you alter its phase (relative to the first captured sample) by some $d\theta_n$ but when you pitch shift block $n+1$, you are not taking into account the $d\theta_n$ you just applied. Therefore, subsequent blocks are out of phase.

What you hear at play-back are those phase discontinuities.

How do I implement the pitch shifting on a bin by bin basis while minimizing the noise/leakage?

Exactly how you are going to take this into account depends on the algorithm used. If the algorithm is a linear phase algorithm, then you might get away easily by correcting the phase for a constant term. Another thing you can do is to process the audio in overlapping blocks (for example, one block from 0 to 99 and the next from 60 to 159, rather than 0..99 followed by 100..199), still as if they were independent of each other. The final result is produced by averaging over the overlapping region. This does not correct the "problem" but it softens the phase discontinuities. By tuning the overlap (and possibly using a low pass filter too), you can supress those "jumps" to the point that they are not too noticeable. In a real-time setting, the overlap can be implemented as preserving a proportion of the currently playing output buffer which you average with the right segment of the next processed frame as the final step.

But also, saying this, you might be able to find an "online" version of a phase-vocoder with all these considerations implemented.

Hope this helps.

| improve this answer | |
$\endgroup$
  • $\begingroup$ How exactly would correct the phase for a constant term? I come more from a programming background than DSP to be honest... $\endgroup$ – niallmandal May 26 at 22:12
  • $\begingroup$ Either that, or how would I shift the pitch of each window while also taking into account the phase shifting to get a continuous sound? $\endgroup$ – niallmandal May 27 at 4:11
  • $\begingroup$ @niallmandal The easiest approach is to apply the algorithm with overlap. Are you looking to apply pitch-shift in real time? $\endgroup$ – A_A May 28 at 16:08
  • $\begingroup$ no, not in real time. I plan to use this for offline purposes. I am having trouble in remove/supress the jumps in general $\endgroup$ – niallmandal Jun 2 at 23:04
  • $\begingroup$ @niallmandal If it is offline, why not send the whole recording through pitch shifting? $\endgroup$ – A_A Jun 3 at 9:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.