# Tag Info

11

The way MFCC's are always used is by feeding them into the classifier. This can be done on a frame-by-frame basis (12x1 vector), or by concatenating (12xN) - same as a spectrogram. Thus for DTW, you must perform the classification by calculating the distance between 12D vectors. It's a Dynamic Time Warping, so the difference must be calculated between ...

9

Why is each window/frame overlapping? Windowing is a means to stationarize signals. Inside a small enough window, you can expect that the properties of the signal chunk do not vary too fast. And you now can use tools well-suited to stationary signals, like Fourier-based techniques. You can imagine non-overlapping rectangular windows, each defining a frame....

8

The i-vectors and x-vectors share the ability to represent speech utterance in a compact way (as a vector of fixed size, regardless of length of the utterance). The extraction algorithms of i-vectors and x-vector are quite different. The x-vector concept is newer and the name of the method is similar to "i-vector" to suggests that this representation can be ...

7

Yes, it's possible to analyse sound the way ears do. For example, you could compute the DFT of a signal continuously using several Goertzel filters. $$y_k[n] = e^{j2\pi k/N} y_k[n-1] + x[n]$$ where $k= 0,1,\ldots, N-1$, so that $y_0$ is the DC or zero frequency term. Of course, this is an unstable filter, so some resetting or forgetting factor is ...

7

More overlap means you end up with more windows (of a given length) per second of audio. More windows (of a given length) requires more FFTs which requires more MACs or FLOPs which generally requires more processing power. In return, more window overlap provides greater time locality of information (e.g. on average, random transient events are likely ...

6

First of all there is some serious "cheating" in the MFCC reconstruction experiment you linked to: not only the MFCCs are used, but also the voiced/unvoiced bit and the pitch. MFCC are not speaker independent. In fact, they are used for speaker identification/verification tasks! The speaker "idiosyncracies" are both in their prosody (preserved by this ...

6

Researchers from the Johns Hopkins University have recently released a corpus of music, speech, and noise which, according to them, is suitable for training models for voice activity detection and music/speech discrimination. See https://arxiv.org/pdf/1510.08484.pdf for details.

6

"Ok Google" is described in many publications by Google Automatic Gain Control and Multi-style Training for Robust Small-Footprint Keyword Spotting with Deep Neural Networks Convolutional Neural Networks for Small-Footprint Keyword Spotting It is based on DNN specifically trained for keyphrase and runs really fast. It does not consume a lot of power even ...

5

What exactly can the vector/observation be? Can it be a relatively simple function, say approximation of the spectral envelope? Or is it usually a vector of R^n? You won't go very far by summarizing the entire spectral envelope of a sound by a single real, so yes, observations are vectors - their dimensionality is in the 10 to 100 components bucket... ...

5

You may think of it as efficient way to apply Dirichlet Window Based interpolation in the Fourier Domain. The advantage of applying the interpolation using Zero Padding in the Time Domain is very simple - it is simpler and more computationally efficient.

4

The standard method to remove stationary noise is spectral subtraction, where the magnitudes of the short-time Fourier transform of the noisy signal are modified based on an SNR estimate in the respective frequency bin. The algorithm by Ephraim and Malah (see this paper) and its variants are used a lot. Note that the basic principle of spectral subtraction ...

4

By performing the windowing with overlap we are artificially increasing our time resolution (larger granularity of features in time). This is especially useful when frame duration is long (bad time resolution, very good frequency resolution), thus yielding kind of extra 'time resolution'. Usually no one is using the rectangular window, but other types such ...

4

It's better to copy first frame and last frame values to extend vector sequence beyond boundaries than to assign 0. This could be implemented just by adjusting indexes: if (index1 < 0) index1 = 0 if (index2 > N - 1) index2 = N - 1 delta = v[index1] - v[index2]

4

You can take some big speech corpus like TEDLIUM and add the noise you like: http://www-lium.univ-lemans.fr/en/content/ted-lium-corpus The advantage of TEDLIUM is that it's a set of continuous recordings with speech timings, not just a collection of utterances.

4

Dynamic Time Warping is pretty well explained on this site. I'll use some of the diagrams from the PPT on that site to explain. The idea is to divide the signals into segments (frames) and then compare frames sequentially through each signal. As illustrated below, motion from a segment in one signal to the next segment depends on the similarity to the ...

4

A common technique for computing the inverse fft is to invert the imaginary part of the input array, perform a forward fft and then invert the imaginary part of the output array. In the case of the Cepstrum, the input to the ifft is real-valued (due to the absolute value function) and also symmetric. In this case there is no imaginary part on the input to ...

3

The Baum-Welch algorithm uses the EM (Expectation Maximization) algorithm to estimate the model parameters $(T, E, \pi)$, where: $T$: the transition probabilities $E$: the emition probabilities $\pi$: probability distribution on the states Some years ago, I made the following quick-and-dirty implementation (may be fairly broken now), for the discrete ...

3

Assuming that there are no non-linear effects between the sound N source and the microphone (such as AGC), you might try to estimate the impulse response of the channel between the source for sound N and the microphone data. The better estimate you can make of that impulse response, the more of sound N that you can remove. It’s unlikely that the negative ...

3

ImageMagick is a great tool to edit multiple images in the command line. Thanks to your question, I just discovered SoX, or Sound eXchange: SoX is a cross-platform (Windows, Linux, MacOS X, etc.) command line utility that can convert various formats of computer audio files in to other formats. It can also apply various effects to these sound files, ...

3

All recognition tasks (doesn't even have to be speech recognition) are reductions of a very high-dimensional signal (your speech recording's dimension is the number of audio samples!) to a low-dimensional signal. As such, it is generally advisable to transform the input signal through an easy operation to a representation where the dimensionality can be ...

2

You should split the signal up into overlapping chunks of 20 to 30 ms. It is certainly valid to take the complex cepstrum on the 4 second signal. But, since the frequency characteristics of speech are constantly varying the complex cepstrum on the 4 second signal will average all of these together and you won't know what happened when. If you take it in each ...

2

Dan Ellis has some neat Matlab scripts that allow you to pull audio features from files . . . http://labrosa.ee.columbia.edu/matlab/ The dynamic time warp might be a good starting point for the task you describe.

2

Look up acoustic fingerprint or audio fingerprinting and dynamic-time warping for some basic technologies used to compare audio files.

2

To give more variants to Aaron's answer, speech recognition pipeline has multiple stages where you can cut the line between client and server. There are the following variants: Lossless audio (flac) Lossy audio (speex 8kb/s) MFCC features (about 6.3kb/s bitrate) Compressed MFCC features ETSI distributed speech recognition standard, 4.4 kb/s bitrate Phonetic ...

2

Its probably safe to assume that all of the major companies send enough information to reconstruct the audio. This is because having that audio for training is such a valuable resource. A certain percentage of the audio segments will be listened to and transcribed by a human annotator. Also features in these systems are more complicated than MFCCs. You ...

2

There is no (known) magic solution, as the sound of the same word from very different speakers are often not actually similar to one another in term of any simple characteristics such as you list (frequency range, intensity range, pitch, etc.) To often, humans can only guess at what word is said from an unknown speaker based on context. And you don't have ...

2

system using arduino Arduino performance is not enough for any serious speech recognition. You should better play with your mobile phone instead. With mobile you can build at least some interesting applications. Or try Raspberry Pi. From what I have read so far the entire process is the following You probably need to read more papers (including the ...

2

Will similar enough vectors be "put together" and only treated as 1? No, this is not how it is done. so for what p will P(o|p) be computed? Your assumption that recognition is performed by considering one by one all possible phone or word sequences and scoring them is wrong. For just a few seconds of speech the number of possible words would be ...

2

You compute the energy (before the log operation) as $$\max\left\{\sum_i s^2(i),\epsilon\right\}\tag{1}$$ with some small constant $\epsilon$ (they suggest $\epsilon=2e-22$ which gives $\ln\epsilon\approx -50$) to prevent the energy to become too small. The reason for this is that the logarithm of zero is $-\infty$ and gives you numerical trouble. So you ...

Only top voted, non community-wiki answers of a minimum length are eligible