1
$\begingroup$

I am trying to follow along a paper: The IVO Software Blizzard 2007 Entry: Improving Ivona Speech Synthesis System

This paper explains how automatically estimated pitchmarks (certain time periods in human speech) can be corrected if they are obviously wrong.

They write:

3.2. Statistical Pitchmark Correction Method

We observed also another problem which is referred to voice characteristic used in recording of ATR database. The voice is prone to pitchmark position errors, which is critical for speech concatenation using pitch synchronous methods (i.e. PS-OLA). Advanced pitchmark labeling algorithms produces up to 10 errors per 5 seconds of each utterance (error rate is dependent of voice characteristic and utterance as well). These errors mostly occure when an algorithm locates more or less glottal closure instants in given part of speech signal then in reality. Then we can finally observe:

  1. missing pitchmark in glottal closure instant,
  2. multiple pitchmarks near glottal closure instant. Having based on that observation we decided to implement simple method to correct these errors. We called it Statistical Pitchmark Correction Method. First we need to prepare vector V (contains one value per pitchmark) using formula:

enter image description here

enter image description here

where: ¢ti - pitch period duration of pitchmark i; ¢ti = ti+1 − ti

N - stands for window length 2 * N

Vi stands for pitch period duration of pitchmark i referenced to average pitch period duration (for window size 2 * N). Voice frequency doesn’t change dramatically in regular speech, so the value Vi should change smoothly in time and oscillate near 1. Basing on it we are able to detect pitchmark problems. Value of Vi close or bigger than 2 means there probably is a missing pitchmark, Vi value close to 0 means there probably are multiplied pitchmarks. Having used this simple method in Ivona Speech Synthesis we reduced concatenation errors and gained ”smoother” sounding speech.

Could anybody tell me how to convert this formula to a programming language like C#?

I am fighting with other challenges in that approach, and I would not like to make a mistake at implementing this formula. I am really not sure if I understand it correctly.

And I don't understand how I can correct the found errors using this formula.

$\endgroup$
3
  • 2
    $\begingroup$ I would recommend adding a link to the paper. $\endgroup$
    – Gillespie
    Sep 1, 2022 at 15:56
  • 1
    $\begingroup$ @Gillespie Thank you, I have added it. $\endgroup$
    – tmighty
    Sep 1, 2022 at 19:20
  • 1
    $\begingroup$ Too get more traction on this, I would further suggest that you ask a more specific question. Citing a paper and asking for code implementing it is a tall ask. Try to figure out what the heart of the paper is, and explain it to us with the relevant equations. Long quotes out of context are difficult to digest without reading the whole paper and starting from scratch ourselves, and most people don't have that much time to devote to your problem. $\endgroup$
    – Gillespie
    Sep 2, 2022 at 14:09

1 Answer 1

1
$\begingroup$

Understanding the paper

The paper you cite provides a method for detecting pitchmark errors, not correcting them. The detection method is very simple. What it's attempting to do is normalize the time differences between pitchmarks ($\Delta t_i$, also called the pitch period duration) by the average time difference of pitchmarks within a window. The formula for this calculation is:

$$ V_i = \frac{2N \cdot \Delta t_i}{\sum_{j = i-N}^{j = i+N-1} \Delta t_j } \tag{2} $$

(Note that there is an error in the formula as given in the paper: The sum in the denominator should be from $j = i - N$ to $j = $ i $ + N - 1$, not $j = $ j $ + N - 1$.)

Notice that the term $\frac{2N}{\sum_{j = i - N}^{j = i + N - 1} \Delta t_i} $ is just the reciprocal of the mean of the pitch period durations within a window around the current pitchmark, $i$. Thus, formula (2) is just normalizing the current pitch mark duration by this mean. If pitch period durations are similar to each other, $V_i$ should be close to 1. $V_i$ can be used to detect extra pitch marks ($V_i$ near 0), or missing pitch marks ($V_i$ near 2 or greater than 2).

Once you've used $V_i$ to detect missing or extra pitchmarks, I imagine the user would run a higher fidelity pitchmark detection near the site of the error. The paper doesn't tell you how to do this at all. It only says how to detect errors.

Code

Here's an example of MATLAB code that would implement formula (2) directly:

V = zeros(N_pitch, 1); 
for i = 1:N_pitch
    validIndices = max(i - N, 1):min(i + N-1, N_pitch);     

    deltaSum = 0;
    for k = 1:numel(validIndices) 
        deltaSum = deltaSum + deltaT(validIndices(k)); 
    end

    V(i) = numel(validIndices)*deltaT(i)/deltaSum; 
end

This is similar to a C implementation. Note that it accounts for edge conditions (for example, on the first pitch period duration $\Delta t_1$, we only average over non-negative indices in the window: $j = 1$ to $N$).

Here's a more efficient MATLAB implementation:

V = zeros(N_pitch, 1);
for i = 1:N_pitch
    validIndices = max(i - N, 1):min(i + N-1, N_pitch); 
    V(i) = deltaT(i)/mean(deltaT(validIndices)); 
end
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.