Analysis of PSOLA

I'm reading the original paper on PSOLA (Pitch-Synchronous Overlap-Add), and am struggling with equation (2), in which the authors analysed the effect of a simple pitch-and-time-scale modification on a stationary noise process $$n(m)$$, which is super-positioned on a periodic signal $$d(m)$$ with period $$P$$.

Here, the analysis window size can be assumed to be integer multiples of $$P$$, with hop size $$P$$, with the constant overlap assumption satisfied.

Denoting the pitch modification factor by $$\beta$$, the time-scale modification factor by $$\gamma := \beta$$, the original noise process by $$n(m)$$, the analysis window by $$h(m)$$ and the synthesis noise signal by $$\tilde{n}(m)$$, equation (2) states: \begin{align} \tilde{n}(m) &= \sum_s h(s\beta P - m)n(m - s \beta P + s P) \tag{2a} \\ &= \tilde{R}(\tau) = \frac{1}{\beta P} \sum_{k=0}^{\beta P - 1} E(\tilde{n}(k)\tilde{n}(k-\tau)) \tag{2b} \\ \tilde{R}(\tau) &= \frac{1}{\beta P} \sum_{m} \rho(\tau - m\beta P) R(\tau + m(1- \beta P)). \tag{2c} \end{align} where $$\tilde{R}(\tau)$$ and $$R(\tau)$$ denote the original and synthesized noises' auto-correlation, and $$\rho(\tau):=h(t)*h(-t)$$ is the analysis window auto-correlation.

Indeed all three of the equalities stated here baffled me. Specifically,

1. In (2a), why is the window $$h(.)$$ flipped such that it's not $$h(m - s\beta P)$$?
2. In (2a), why is the noise term $$n(m - s \beta P + sP)$$ and not just $$n(m - s\beta P)$$?
3. Given that we are modifying both the pitch and time by the same factor, why don't we simply have $$\tilde{n}(m)=n(m/\beta)$$ and indeed $$\tilde{d}(m)=d(m/\beta)$$?
4. In (2b), Why is the synthesized noise $$\tilde{n}(R)$$ equal to the autocorrelation $$\tilde{R}(\tau)$$?
5. How does (2c) come about?

Note that the paper provides the following footnote to explain this equation:

$$\tilde{n}(m)$$ is not a WSS (Wide sense stationary) process, but a WS (Wide-sense) cyclostationary process. This means that the autocorrelation sequence $$R(t_1, t_2) = E(\tilde{n}(t_1)\tilde{n}(t_2))$$ is not invariant on the diagonal of $$t_1-t_2$$ plane, but is nevertheless periodic=: $$R(t_1 + m\beta P, t_2 + m\beta P) = R(t_1, t_2)$$. However, there is a close relationship between WS stationary and WS cyclostationary processes: it can be demonstrated that if the time origin is subject to a random shift $$\theta(t)$$ that varies sufficiently slowly, then the resulting process $$\tilde{n}(t)=\tilde{n}(t - \theta{t}))$$ is WS stationary (Papoulis, 1984). The autocorrelation sequence of this process is equal to the "time-average autocorrelation" of process $$\tilde{n}(m)$$.

which is still cryptic to me.

Reference

Papoulise, A. (1984). Probability, Random Variables and Stochastic Processes, McGraw-Hill, NY

• Line (2b) looks like an error. I (erroneously) removed the leading = sign, but the original paper has it so I reverted my change. I can't see how it can be true that $\tilde{n}(m) = \tilde{R}(\tau)$.
– Peter K.
Nov 8, 2022 at 18:01

Attempts to answer you series of questions:

1. In (2a), why is the window $$h(.)$$ flipped such that it's not $$h(m - s\beta P)$$?

The paper (p455) states:

The windows $$h_m(n)$$ are of Hanning type$$\ldots$$

so they're symmetric and whether it's $$h(t)$$ or $$h(-t)$$ doesn't matter because $$h(t) = h(-t)$$.

1. In (2a), why is the noise term $$n(m - s \beta P + sP)$$ and not just $$n(m - s\beta P)$$?

I'm not exactly sure, but I suspect it's to do with Figure 2 of the paper.

I think the equation is showing the overall effect of going from analysis timescale to intermediate timescale to synthesized timescale.

1. Given that we are modifying both the pitch and time by the same factor, why don't we simply have $$\tilde{n}(m)=n(m/\beta)$$ and indeed $$\tilde{d}(m)=d(m/\beta)$$?

Because that only modifies the time, not the pitch.

1. In (2b), Why is the synthesized noise $$\tilde{n}(R)$$ equal to the autocorrelation $$\tilde{R}(\tau)$$?

I think that's an error. You can see in the original equation (2) that there is a comma at the end of the first line. I think the leading = sign on the second line shouldn't be there.

1. How does (2c) come about?

That's just substituting your (2a) into your equation (2b) (neglecting what I believe to be the erroneous leading =) and applying the expectation operator and the expression for $$\rho$$:

$$\rho(\tau) = h(t) \star h(-t)$$

• Thanks for attempting this. Your suggestion that (2c) arises from substituting (2a) into (2b) seems correct, although this raises further question on why it is $R(\tau + m(1- \beta P))$ and not simply $R(\tau)$. Further justification on why all the off-diagonal terms (in the multiplication of the two sums) are ignored is also needed given that the windows are overlapping. Likewise why $\rho(\tau - m \beta P)$ and not $\rho{\tau}$? Nov 9, 2022 at 2:10
• I'm also unsure about your answer to question (3). Having $\tilde{n}(m/\beta)$ is I suppose equivalent to a resampling of $n(m)$, which would change both pitch and tempo by the same factor. To preserve pitch but to scale up the tempo, one would naturally have $\tilde{x}(m) = \sum_s x(m - s\beta P)$, which I think is explained in section 1.4 of the paper. Nov 9, 2022 at 2:15
• @TimMak Regarding (2c) : This is because we're trying to find $\tilde{R}(\tau)$ in terms of $R(\tau)$. And the definition of $\tilde{n}(m)$ cyclically extends $n(m)$, which will mean that $\tilde{R}(\tau)$ will cyclically extend $R(\tau)$.
– Peter K.
Nov 9, 2022 at 20:22
• @TimMak Regarding (3) : Simple because that's not how they've defined the relationship between $\tilde{n}(m)$ and $n(m)$. There is that cyclical extension relationship between them, not just time scaling.
– Peter K.
Nov 9, 2022 at 20:25