I am trying to understand the process of changing the pitch of a signal. After performing the FFT on a windowed signal, we have the frequency information for the signal. Why can we not simply change these frequencies to change the pitch of the signal? Instead, we must first use pitch detection methods to determine pitches that were present in the time-domain function. How does the FFT not give you this information? What is the frequency information given by the FFT if not the frequencies of the time-domain signal?
-
$\begingroup$ You seem to be referring to a specific algorithm or implementation of pitch correction. It would certainly be helpful if you provided a reference and more context. $\endgroup$– JazzmaniacDec 22, 2017 at 10:00
-
1$\begingroup$ Eventhough pitch of an instrument is related to the frequencies in it, they are not the same. Pitch is a psychoacoustic phenomena that has a complex dependence on frequency. Whereas frequency is a very simple thing that quantifies the number of repetition per second of a periodic waveform... $\endgroup$– Fat32Dec 23, 2017 at 13:47
-
$\begingroup$ Pitch detection is not always necessary, for example in the frequency domain it isn't required as the bins can be moved using multiplication to increase or decrease the overall pitch. However, to perform pitch correction on vocals, it's useful to preserve the formants. Without identifying the fundamental, it is difficult to preserve formants, leading to the chipmunk effect. (This answer is based on my current understanding, please correct me if I made a mistake) $\endgroup$– MysteryPancakeAug 9, 2020 at 15:59
-
$\begingroup$ hay Claire, did you get an answer to your question? $\endgroup$– robert bristow-johnsonAug 5, 2021 at 1:48
3 Answers
I don't know what algorithm you are talking about exactly, as you didn't state that in your question, but my idea of pitch correction consists of detecting the actual pitch and shifting it to the right one.
You ask why pitch detection is necessary. Well, to be able to write a program that automatically corrects the pitch of a recording (i.e. in opposition to a scenario where you are manually checking what frequency is the right one for each instant of time) you should first have an array containing all possible frequencies corresponding to musical notes. That is, a list containing all the values that would be correct if detected, so that they wouldn't have to be changed. If the singer has accurately hit a $\mathrm{C_3}$, then why would you auto-tune it?
Then, you should perform pitch detection using STFT. For each instant of time, there will be a dominant frequency. One (primitive) way to do this would be to find the frequency that shows the maximum amplitude. When you find that maximum (pitch detection), you should compare the frequency where it's located to the array of musical notes you created before. Then, assuming the singer is not that bad and the tone he's hitting is approximate to the desired one, one should take the detected frequency to the closest musical note (pitch shifting). For example, if the maximum is detected at $\mathrm{124 \ Hz}$, one would go for a $\mathrm{B_2}$ ($\mathrm{123.47 \ Hz}$) instead of a $\mathrm{C_3}$ ($\mathrm{130.81 \ Hz}$). I believe there are some things about logarithmic perception that should be taken into account here, but that's not relevant in this question. Take into account that there are pitch detection algorithms, I just used this easy method as an example to make it illustrative.
That's why detection has to be made. You have no way to automatically "round" the pitch to the right one if you don't know what actual pitch the singer is in.
-
$\begingroup$ This is one of the papers I am using: dave.ucsc.edu/physics195/thesis_2009/m_peimani.pdf. In it, pitch detection methods such as autocorrelation are discussed. My question is why would you need autocorrelation? Doesn't the STFT tell you the frequencies present in the original sample? This has already "detected" the frequencies for you. $\endgroup$– claireccDec 22, 2017 at 15:32
-
$\begingroup$ Well, the first paragraph of the 6th section "Pitch Correction" kind of tackles your question and is related to my answer as well. $\endgroup$– TenderoDec 22, 2017 at 15:35
-
$\begingroup$ You don't need autocorrelation, as well as you don't need STFT. They are just different methods to do the same thing: pitch detection. You can use time or frequency-domain methods, depending on your preference and what best suits your data and application. In theory, all the methods should give you the same result, which is a correctly detected pitch. $\endgroup$– TenderoDec 22, 2017 at 15:37
A DFT or FFT of a finite length segment (a window) of a longer signal does not give you the frequencies in that signal. It breaks that signal into a finite set of basis vectors which are orthogonal within the FFT length, which may or may not represent the frequency of a longer windowed sinusoid in any single FFT result bin. If the longer signal contains spectral frequencies not among those in the FFT basis vectors, it will not be found in the FFT result except as a large mix of frequencies (and phases) that are from slightly to very different from the real frequency of interest.
If you mess with frequencies that are different, from slightly, in nearby FFT result bins, to very different, covering the entire spectrum of the FFT result, from the real one (e.g. sinusoids in the longer original un-windowed source you care about), the you may or may not get the result you want.
Furthermore pitch is different from a sinusoidal frequency basis vector. It's a more complex psycho-acoustic phenomena that has to be separately estimated.
nice reference to that Michael Peimani paper. it's not bad for a thesis for a BS. i wouldn't consider it sufficient for an MS.
pitch correction is about pitch shifting to an amount of pitch shift based on the difference between what the desired (output) pitch is and what the actual (input) pitch is. often (as with Autotune or an even better pitch corrector than AT) the desired pitch is a function of the input pitch.
pitch shifting requires a parameter that is the pitch-shift amount (which is this difference between output and input pitch) but not necessarily (or inherently) the input (or output) pitches. in fact, one can conceptually shift the pitch of a sound that has no single input pitch. it could be a broadbanded segment of some filtered noise with many frequency components and the output would have all of those components moved up or down a given amount in the log-frequency scale.
frequency-domain pitch shifting (either phase vocoder or sinusoidal modeling) will not need to know the input pitch, but it still needs to know the principal pitch-shift amount; how many semitones ($\tfrac{1}{12}$ octave) up or down is the pitch being shifted.
time-domain pitch shifting (this would be called a variety of different names that sometime mean about the same thing) needs to know something about the input signal in order to splice the input signal to lengthen it (for up-shifting) or to shorten it (for down-shifting). that information is what comes from a pitch detector. the pitch detector returns essentially what the fundamental frequency is, which is the reciprocal of the period, and a time-domain pitch shifter will splice in or out an integer number of periods to eschew glitches resulting from the splice.