13
This is somehow a broad question. An interesting book is Digital Processing of speech signals by Rabiner & Schafer, in which you can find some good explanations to get started. It probably is out of date when looking into more advanced techniques, but I think it's good to understand the main characteristics of speech.
In short, every sound we produce is ...
11
The way MFCC's are always used is by feeding them into the classifier. This can be done on a frame-by-frame basis (12x1 vector), or by concatenating (12xN) - same as a spectrogram.
Thus for DTW, you must perform the classification by calculating the distance between 12D vectors. It's a Dynamic Time Warping, so the difference must be calculated between ...
7
Yes, it's possible to analyse sound the way ears do.
For example, you could compute the DFT of a signal continuously using several Goertzel filters.
$$
y_k[n] = e^{j2\pi k/N} y_k[n-1] + x[n]
$$
where $k= 0,1,\ldots, N-1$, so that $y_0$ is the DC or zero frequency term.
Of course, this is an unstable filter, so some resetting or forgetting factor is ...
7
Why is each window/frame overlapping?
Windowing is a means to stationarize signals. Inside a small enough window, you can expect that the properties of the signal chunk do not vary too fast. And you now can use tools well-suited to stationary signals, like Fourier-based techniques.
You can imagine non-overlapping rectangular windows, each defining a frame....
6
The dimension of the frame's feature vector N depends on the frequency range and the number of frequency bins in the frame. For example for 16khz audio signal it's common to take 39 feature vectors created from 40 cepstrum values. For 8khz it's enough to have 20 cepstrum values.
Feature dimension is not equivalent to the frame length. Frame length could be ...
6
First of all there is some serious "cheating" in the MFCC reconstruction experiment you linked to: not only the MFCCs are used, but also the voiced/unvoiced bit and the pitch.
MFCC are not speaker independent. In fact, they are used for speaker identification/verification tasks!
The speaker "idiosyncracies" are both in their prosody (preserved by this ...
6
Researchers from the Johns Hopkins University have recently released a corpus of music, speech, and noise which, according to them, is suitable for training models for voice activity detection and music/speech discrimination.
See https://arxiv.org/pdf/1510.08484.pdf for details.
6
More overlap means you end up with more windows (of a given length) per second of audio. More windows (of a given length) requires more FFTs which requires more MACs or FLOPs which generally requires more processing power. In return, more window overlap provides greater time locality of information (e.g. on average, random transient events are likely ...
6
"Ok Google" is described in many publications by Google
Automatic Gain Control and Multi-style Training for Robust Small-Footprint Keyword Spotting with Deep Neural Networks
Convolutional Neural Networks for Small-Footprint Keyword Spotting
It is based on DNN specifically trained for keyphrase and runs really fast. It does not consume a lot of power even ...
6
The i-vectors and x-vectors share the ability to represent speech utterance in a compact way (as a vector of fixed size, regardless of length of the utterance).
The extraction algorithms of i-vectors and x-vector are quite different. The x-vector concept is newer and the name of the method is similar to "i-vector" to suggests that this representation can be ...
5
From CMUSphinx FAQ:
There are various types of MFCC which differ by number of parameters,
but not really different for accuracy (it might be a few percent worse
or better).
The interpretation of MFCC (Roughtly introduced Alan V. Oppenheim and
Ronald W. Schafer. From Frequency to Quefrency: A History of the
Cepstrum. IEEE SIGNAL PROCESSING ...
5
What exactly can the vector/observation be? Can it be a relatively simple function, say approximation of the spectral envelope? Or is it usually a vector of R^n?
You won't go very far by summarizing the entire spectral envelope of a sound by a single real, so yes, observations are vectors - their dimensionality is in the 10 to 100 components bucket...
...
4
I highly doubt that something as complex as recognizing a word can be done directly in the time domain using features as basic as zero-crossing rate.
4
Do we really need 10 samples of each word? or can we "combine" them into a single template? and if so, how?
You can use single template, 10 templates will make result more accurate.
Should we run the SVM directly on the MFCCs vectors? On the MFCC vector of vectors (for each word)? On the DTW values? or should we combine all the MFCC vectors into a single ...
4
It's better to copy first frame and last frame values to extend vector sequence beyond boundaries than to assign 0. This could be implemented just by adjusting indexes:
if (index1 < 0)
index1 = 0
if (index2 > N - 1)
index2 = N - 1
delta = v[index1] - v[index2]
4
By performing the windowing with overlap we are artificially increasing our time resolution (larger granularity of features in time). This is especially useful when frame duration is long (bad time resolution, very good frequency resolution), thus yielding kind of extra 'time resolution'.
Usually no one is using the rectangular window, but other types such ...
4
The standard method to remove stationary noise is spectral subtraction, where
the magnitudes of the short-time Fourier transform of the noisy signal are modified based on an SNR estimate in the respective frequency bin. The algorithm by Ephraim and Malah (see this paper) and its variants are used a lot. Note that the basic principle of spectral subtraction ...
4
You can take some big speech corpus like TEDLIUM and add the noise you like:
http://www-lium.univ-lemans.fr/en/content/ted-lium-corpus
The advantage of TEDLIUM is that it's a set of continuous recordings with speech timings, not just a collection of utterances.
4
Dynamic Time Warping is pretty well explained on this site. I'll use some of the diagrams from the PPT on that site to explain.
The idea is to divide the signals into segments (frames) and then compare frames sequentially through each signal. As illustrated below, motion from a segment in one signal to the next segment depends on the similarity to the ...
4
A common technique for computing the inverse fft is to invert the imaginary part of the input array, perform a forward fft and then invert the imaginary part of the output array. In the case of the Cepstrum, the input to the ifft is real-valued (due to the absolute value function) and also symmetric. In this case there is no imaginary part on the input to ...
3
DTW is not proportional to anything, including frame length. It's not a good idea to compare DTW values between two different pairs like you are doing.
The issue is that DTW is not just the number but also an alignment. If alignment does not match, you can not relate numbers. You only can relate numbers if you align to the same thing.
Normalization is ...
3
The Baum-Welch algorithm uses the EM (Expectation Maximization) algorithm to estimate the model parameters $(T, E, \pi)$, where:
$T$: the transition probabilities
$E$: the emition probabilities
$\pi$: probability distribution on the states
Some years ago, I made the following quick-and-dirty implementation (may be fairly broken now), for the discrete ...
3
Assuming that there are no non-linear effects between the sound N source and the microphone (such as AGC), you might try to estimate the impulse response of the channel between the source for sound N and the microphone data. The better estimate you can make of that impulse response, the more of sound N that you can remove. It’s unlikely that the negative ...
3
ImageMagick is a great tool to edit multiple images in the command line. Thanks to your question, I just discovered SoX, or Sound eXchange:
SoX is a cross-platform (Windows, Linux, MacOS X, etc.) command line
utility that can convert various formats of computer audio files in to
other formats. It can also apply various effects to these sound files,
...
3
You may think of it as efficient way to apply Dirichlet Window Based interpolation in the Fourier Domain.
The advantage of applying the interpolation using Zero Padding in the Time Domain is very simple - it is simpler and more computationally efficient.
3
All recognition tasks (doesn't even have to be speech recognition) are reductions of a very high-dimensional signal (your speech recording's dimension is the number of audio samples!) to a low-dimensional signal.
As such, it is generally advisable to transform the input signal through an easy operation to a representation where the dimensionality can be ...
2
You want only objective criteria to seperate the signals or is it important that they have some sort of similarity when listened to by someone? That of course would have to restrict you to signals a bit longer (more than 1000 samples).
2
As has been mentioned in the comments, you can't just get a transcript without knowing what language is being spoken. However, you can decode the audio assuming it is of a given language. Thus I would propose the following system:
You run your speech recognizer N times on your audio, where N is the number of languages you are identifying, using the ...
2
When using an FFT, an evenly spaced sequence of events in one domain usually produces a strong component in the other domain at a location related to the spacing of the events in the first domain.
A voiced speech signal usually includes a lot of harmonics which are evenly spaced in the frequency domain. These evenly spaced events in the frequency domain ...
Only top voted, non community-wiki answers of a minimum length are eligible
