11
The way MFCC's are always used is by feeding them into the classifier. This can be done on a frame-by-frame basis (12x1 vector), or by concatenating (12xN) - same as a spectrogram.
Thus for DTW, you must perform the classification by calculating the distance between 12D vectors. It's a Dynamic Time Warping, so the difference must be calculated between ...
8
Why is each window/frame overlapping?
Windowing is a means to stationarize signals. Inside a small enough window, you can expect that the properties of the signal chunk do not vary too fast. And you now can use tools well-suited to stationary signals, like Fourier-based techniques.
You can imagine non-overlapping rectangular windows, each defining a frame....
8
The i-vectors and x-vectors share the ability to represent speech utterance in a compact way (as a vector of fixed size, regardless of length of the utterance).
The extraction algorithms of i-vectors and x-vector are quite different. The x-vector concept is newer and the name of the method is similar to "i-vector" to suggests that this representation can be ...
7
Yes, it's possible to analyse sound the way ears do.
For example, you could compute the DFT of a signal continuously using several Goertzel filters.
$$
y_k[n] = e^{j2\pi k/N} y_k[n-1] + x[n]
$$
where $k= 0,1,\ldots, N-1$, so that $y_0$ is the DC or zero frequency term.
Of course, this is an unstable filter, so some resetting or forgetting factor is ...
6
First of all there is some serious "cheating" in the MFCC reconstruction experiment you linked to: not only the MFCCs are used, but also the voiced/unvoiced bit and the pitch.
MFCC are not speaker independent. In fact, they are used for speaker identification/verification tasks!
The speaker "idiosyncracies" are both in their prosody (preserved by this ...
6
Researchers from the Johns Hopkins University have recently released a corpus of music, speech, and noise which, according to them, is suitable for training models for voice activity detection and music/speech discrimination.
See https://arxiv.org/pdf/1510.08484.pdf for details.
6
More overlap means you end up with more windows (of a given length) per second of audio. More windows (of a given length) requires more FFTs which requires more MACs or FLOPs which generally requires more processing power. In return, more window overlap provides greater time locality of information (e.g. on average, random transient events are likely ...
6
"Ok Google" is described in many publications by Google
Automatic Gain Control and Multi-style Training for Robust Small-Footprint Keyword Spotting with Deep Neural Networks
Convolutional Neural Networks for Small-Footprint Keyword Spotting
It is based on DNN specifically trained for keyphrase and runs really fast. It does not consume a lot of power even ...
5
What exactly can the vector/observation be? Can it be a relatively simple function, say approximation of the spectral envelope? Or is it usually a vector of R^n?
You won't go very far by summarizing the entire spectral envelope of a sound by a single real, so yes, observations are vectors - their dimensionality is in the 10 to 100 components bucket...
...
5
You may think of it as efficient way to apply Dirichlet Window Based interpolation in the Fourier Domain.
The advantage of applying the interpolation using Zero Padding in the Time Domain is very simple - it is simpler and more computationally efficient.
4
The standard method to remove stationary noise is spectral subtraction, where
the magnitudes of the short-time Fourier transform of the noisy signal are modified based on an SNR estimate in the respective frequency bin. The algorithm by Ephraim and Malah (see this paper) and its variants are used a lot. Note that the basic principle of spectral subtraction ...
4
By performing the windowing with overlap we are artificially increasing our time resolution (larger granularity of features in time). This is especially useful when frame duration is long (bad time resolution, very good frequency resolution), thus yielding kind of extra 'time resolution'.
Usually no one is using the rectangular window, but other types such ...
4
It's better to copy first frame and last frame values to extend vector sequence beyond boundaries than to assign 0. This could be implemented just by adjusting indexes:
if (index1 < 0)
index1 = 0
if (index2 > N - 1)
index2 = N - 1
delta = v[index1] - v[index2]
4
I highly doubt that something as complex as recognizing a word can be done directly in the time domain using features as basic as zero-crossing rate.
4
You can take some big speech corpus like TEDLIUM and add the noise you like:
http://www-lium.univ-lemans.fr/en/content/ted-lium-corpus
The advantage of TEDLIUM is that it's a set of continuous recordings with speech timings, not just a collection of utterances.
4
Dynamic Time Warping is pretty well explained on this site. I'll use some of the diagrams from the PPT on that site to explain.
The idea is to divide the signals into segments (frames) and then compare frames sequentially through each signal. As illustrated below, motion from a segment in one signal to the next segment depends on the similarity to the ...
4
A common technique for computing the inverse fft is to invert the imaginary part of the input array, perform a forward fft and then invert the imaginary part of the output array. In the case of the Cepstrum, the input to the ifft is real-valued (due to the absolute value function) and also symmetric. In this case there is no imaginary part on the input to ...
3
The Baum-Welch algorithm uses the EM (Expectation Maximization) algorithm to estimate the model parameters $(T, E, \pi)$, where:
$T$: the transition probabilities
$E$: the emition probabilities
$\pi$: probability distribution on the states
Some years ago, I made the following quick-and-dirty implementation (may be fairly broken now), for the discrete ...
3
Assuming that there are no non-linear effects between the sound N source and the microphone (such as AGC), you might try to estimate the impulse response of the channel between the source for sound N and the microphone data. The better estimate you can make of that impulse response, the more of sound N that you can remove. It’s unlikely that the negative ...
3
ImageMagick is a great tool to edit multiple images in the command line. Thanks to your question, I just discovered SoX, or Sound eXchange:
SoX is a cross-platform (Windows, Linux, MacOS X, etc.) command line
utility that can convert various formats of computer audio files in to
other formats. It can also apply various effects to these sound files,
...
3
All recognition tasks (doesn't even have to be speech recognition) are reductions of a very high-dimensional signal (your speech recording's dimension is the number of audio samples!) to a low-dimensional signal.
As such, it is generally advisable to transform the input signal through an easy operation to a representation where the dimensionality can be ...
2
This is not aliasing this is not any phenomenon, there are number of birds whose vocal fall in speech range even a frog vocal falls in speech range so essentially a training based speech detection is required to classify whether a given frame is speech or a non speech
one such implementation is shown here
Speech Discrimination based on multiscale spectro ...
2
The features are the same. You might encounter differences in some of the implementation details (window sizes, number of mel filters, number of extracted coefficients) reported in research papers, but these are not significant.
2
Both speech recognition and speaker recognition require some set of features to distinguish one speaker (or speech section) from another.
Suppose you have two speakers $S_1$ and $S_2$ and two words $w_1$ and $w_2$. Let's call $u_{ij}$ the utterance of speaker $i$ of word $j$. Then the MFCC's of the speech are $M(u_{ij})$.
One way to pose the speaker ...
2
You should split the signal up into overlapping chunks of 20 to 30 ms. It is certainly valid to take the complex cepstrum on the 4 second signal. But, since the frequency characteristics of speech are constantly varying the complex cepstrum on the 4 second signal will average all of these together and you won't know what happened when. If you take it in each ...
2
Dan Ellis has some neat Matlab scripts that allow you to pull audio features from files . . .
http://labrosa.ee.columbia.edu/matlab/
The dynamic time warp might be a good starting point for the task you describe.
2
You can try Prof. Rabiner's slides at http://www.ece.ucsb.edu/Faculty/Rabiner/ece259/ . They also have some matlab code examples which you can try.
2
There is no (known) magic solution, as the sound of the same word from very different speakers are often not actually similar to one another in term of any simple characteristics such as you list (frequency range, intensity range, pitch, etc.)
To often, humans can only guess at what word is said from an unknown speaker based on context. And you don't have ...
2
Zero crossing rate is not a good approach indeed, you can use any keyword spotting software to spot for the word, keyword spotting is based on HMM too and statistical hypothesis testing.
You can try keyword spotting with CMUSphinx
Only top voted, non community-wiki answers of a minimum length are eligible