13
This is somehow a broad question. An interesting book is Digital Processing of speech signals by Rabiner & Schafer, in which you can find some good explanations to get started. It probably is out of date when looking into more advanced techniques, but I think it's good to understand the main characteristics of speech.
In short, every sound we produce is ...
11
The first equation you give is the difference equation for a lowpass FIR filter, or a linear filter with an impulse response that is finite in duration. I'll write it a bit differently (so that it is expressly discrete in time and causal):
$$
f_s[n] = 0.1 f[n-2] + 0.8 f[n-1] + 0.1 f[n]
$$
$f_s[n]$ is the smoothed version of the discrete-time input sequence ...
11
The way MFCC's are always used is by feeding them into the classifier. This can be done on a frame-by-frame basis (12x1 vector), or by concatenating (12xN) - same as a spectrogram.
Thus for DTW, you must perform the classification by calculating the distance between 12D vectors. It's a Dynamic Time Warping, so the difference must be calculated between ...
7
Yes, it's possible to analyse sound the way ears do.
For example, you could compute the DFT of a signal continuously using several Goertzel filters.
$$
y_k[n] = e^{j2\pi k/N} y_k[n-1] + x[n]
$$
where $k= 0,1,\ldots, N-1$, so that $y_0$ is the DC or zero frequency term.
Of course, this is an unstable filter, so some resetting or forgetting factor is ...
7
Why is each window/frame overlapping?
Windowing is a means to stationarize signals. Inside a small enough window, you can expect that the properties of the signal chunk do not vary too fast. And you now can use tools well-suited to stationary signals, like Fourier-based techniques.
You can imagine non-overlapping rectangular windows, each defining a frame....
6
The dimension of the frame's feature vector N depends on the frequency range and the number of frequency bins in the frame. For example for 16khz audio signal it's common to take 39 feature vectors created from 40 cepstrum values. For 8khz it's enough to have 20 cepstrum values.
Feature dimension is not equivalent to the frame length. Frame length could be ...
6
First of all there is some serious "cheating" in the MFCC reconstruction experiment you linked to: not only the MFCCs are used, but also the voiced/unvoiced bit and the pitch.
MFCC are not speaker independent. In fact, they are used for speaker identification/verification tasks!
The speaker "idiosyncracies" are both in their prosody (preserved by this ...
6
Researchers from the Johns Hopkins University have recently released a corpus of music, speech, and noise which, according to them, is suitable for training models for voice activity detection and music/speech discrimination.
See https://arxiv.org/pdf/1510.08484.pdf for details.
6
More overlap means you end up with more windows (of a given length) per second of audio. More windows (of a given length) requires more FFTs which requires more MACs or FLOPs which generally requires more processing power. In return, more window overlap provides greater time locality of information (e.g. on average, random transient events are likely ...
6
"Ok Google" is described in many publications by Google
Automatic Gain Control and Multi-style Training for Robust Small-Footprint Keyword Spotting with Deep Neural Networks
Convolutional Neural Networks for Small-Footprint Keyword Spotting
It is based on DNN specifically trained for keyphrase and runs really fast. It does not consume a lot of power even ...
5
From CMUSphinx FAQ:
There are various types of MFCC which differ by number of parameters,
but not really different for accuracy (it might be a few percent worse
or better).
The interpretation of MFCC (Roughtly introduced Alan V. Oppenheim and
Ronald W. Schafer. From Frequency to Quefrency: A History of the
Cepstrum. IEEE SIGNAL PROCESSING ...
5
What exactly can the vector/observation be? Can it be a relatively simple function, say approximation of the spectral envelope? Or is it usually a vector of R^n?
You won't go very far by summarizing the entire spectral envelope of a sound by a single real, so yes, observations are vectors - their dimensionality is in the 10 to 100 components bucket...
...
4
I highly doubt that something as complex as recognizing a word can be done directly in the time domain using features as basic as zero-crossing rate.
4
I learnt to use HTK based on this very basic tutorial
http://www.info2.uqam.ca/~boukadoum_m/DIC9315/Notes/Markov/HTK_basic_tutorial.pdf
It details how to make a yes/no recogniser. It should get you off to a good start at least. There is also some HTK code on the main HTK site that allows you to perform real time speech recognition.
To improve robustness, ...
4
Do we really need 10 samples of each word? or can we "combine" them into a single template? and if so, how?
You can use single template, 10 templates will make result more accurate.
Should we run the SVM directly on the MFCCs vectors? On the MFCC vector of vectors (for each word)? On the DTW values? or should we combine all the MFCC vectors into a single ...
4
It's better to copy first frame and last frame values to extend vector sequence beyond boundaries than to assign 0. This could be implemented just by adjusting indexes:
if (index1 < 0)
index1 = 0
if (index2 > N - 1)
index2 = N - 1
delta = v[index1] - v[index2]
4
By performing the windowing with overlap we are artificially increasing our time resolution (larger granularity of features in time). This is especially useful when frame duration is long (bad time resolution, very good frequency resolution), thus yielding kind of extra 'time resolution'.
Usually no one is using the rectangular window, but other types such ...
4
The standard method to remove stationary noise is spectral subtraction, where
the magnitudes of the short-time Fourier transform of the noisy signal are modified based on an SNR estimate in the respective frequency bin. The algorithm by Ephraim and Malah (see this paper) and its variants are used a lot. Note that the basic principle of spectral subtraction ...
4
You can take some big speech corpus like TEDLIUM and add the noise you like:
http://www-lium.univ-lemans.fr/en/content/ted-lium-corpus
The advantage of TEDLIUM is that it's a set of continuous recordings with speech timings, not just a collection of utterances.
4
Dynamic Time Warping is pretty well explained on this site. I'll use some of the diagrams from the PPT on that site to explain.
The idea is to divide the signals into segments (frames) and then compare frames sequentially through each signal. As illustrated below, motion from a segment in one signal to the next segment depends on the similarity to the ...
4
A common technique for computing the inverse fft is to invert the imaginary part of the input array, perform a forward fft and then invert the imaginary part of the output array. In the case of the Cepstrum, the input to the ifft is real-valued (due to the absolute value function) and also symmetric. In this case there is no imaginary part on the input to ...
3
DTW is not proportional to anything, including frame length. It's not a good idea to compare DTW values between two different pairs like you are doing.
The issue is that DTW is not just the number but also an alignment. If alignment does not match, you can not relate numbers. You only can relate numbers if you align to the same thing.
Normalization is ...
3
Regarding your existing steps:
Do you have some overlap between adjacent blocks? It is common in feature extraction systems to have some overlap between adjacent blocks, so that a short transient event can be correctly captured (if it is right at the end of a block, it will be right in the middle of the next one).
Your idea of discarding blocks is dangerous....
3
You should first appreciate why an HMM would be useful. The problem that you've described is classify a given utterance as either a yes or a no (it would be also useful to have a "neither" class). Because speech is a time-evolving quantity, it needs a classifier that is able to deal with sequences of feature vectors. Different people say "yes" or "no" with ...
3
I would guess that the most prevalent difference between accents (of the same language) is how the vowels are pronounced. Humans distinguish vowels mainly by the frequency of the formant, i.e. regions of high energy in the harmonic spectrum.
A good way to display this is the "formant map" which plots the frequency region of the second formant against the ...
3
The Baum-Welch algorithm uses the EM (Expectation Maximization) algorithm to estimate the model parameters $(T, E, \pi)$, where:
$T$: the transition probabilities
$E$: the emition probabilities
$\pi$: probability distribution on the states
Some years ago, I made the following quick-and-dirty implementation (may be fairly broken now), for the discrete ...
3
Assuming that there are no non-linear effects between the sound N source and the microphone (such as AGC), you might try to estimate the impulse response of the channel between the source for sound N and the microphone data. The better estimate you can make of that impulse response, the more of sound N that you can remove. It’s unlikely that the negative ...
3
ImageMagick is a great tool to edit multiple images in the command line. Thanks to your question, I just discovered SoX, or Sound eXchange:
SoX is a cross-platform (Windows, Linux, MacOS X, etc.) command line
utility that can convert various formats of computer audio files in to
other formats. It can also apply various effects to these sound files,
...
3
You may think of it as efficient way to apply Dirichlet Window Based interpolation in the Fourier Domain.
The advantage of applying the interpolation using Zero Padding in the Time Domain is very simple - it is simpler and more computationally efficient.
Only top voted, non community-wiki answers of a minimum length are eligible
