12
This is somehow a broad question. An interesting book is Digital Processing of speech signals by Rabiner & Schafer, in which you can find some good explanations to get started. It probably is out of date when looking into more advanced techniques, but I think it's good to understand the main characteristics of speech.
In short, every sound we produce is ...
11
The first equation you give is the difference equation for a lowpass FIR filter, or a linear filter with an impulse response that is finite in duration. I'll write it a bit differently (so that it is expressly discrete in time and causal):
$$
f_s[n] = 0.1 f[n-2] + 0.8 f[n-1] + 0.1 f[n]
$$
$f_s[n]$ is the smoothed version of the discrete-time input sequence ...
10
The way MFCC's are always used is by feeding them into the classifier. This can be done on a frame-by-frame basis (12x1 vector), or by concatenating (12xN) - same as a spectrogram.
Thus for DTW, you must perform the classification by calculating the distance between 12D vectors. It's a Dynamic Time Warping, so the difference must be calculated between ...
8
A speech communication channel as used in telephony typically has a frequency response of 300 Hz to 3 kHz. Although this rejects a lot of the energy in normal speech, intelligibility is still quite good - the main problem seems to be that certain plosive consonants, e.g. "p" and "t", can be a little hard to discriminate without the higher frequency ...
7
Yes, it's possible to analyse sound the way ears do.
For example, you could compute the DFT of a signal continuously using several Goertzel filters.
$$
y_k[n] = e^{j2\pi k/N} y_k[n-1] + x[n]
$$
where $k= 0,1,\ldots, N-1$, so that $y_0$ is the DC or zero frequency term.
Of course, this is an unstable filter, so some resetting or forgetting factor is ...
7
Why is each window/frame overlapping?
Windowing is a means to stationarize signals. Inside a small enough window, you can expect that the properties of the signal chunk do not vary too fast. And you now can use tools well-suited to stationary signals, like Fourier-based techniques.
You can imagine non-overlapping rectangular windows, each defining a frame....
6
This kind of problem is usually solved using machine learning techniques.
Break down the signal into a sequence of 20ms or 50ms frames. Extract features on each frame. MFCC are generally good for this kind of application, though there are feature more specific to voice detection (4 Hz modulation energy - which is roughly the rate at which people speak ; ...
6
The dimension of the frame's feature vector N depends on the frequency range and the number of frequency bins in the frame. For example for 16khz audio signal it's common to take 39 feature vectors created from 40 cepstrum values. For 8khz it's enough to have 20 cepstrum values.
Feature dimension is not equivalent to the frame length. Frame length could be ...
6
First of all there is some serious "cheating" in the MFCC reconstruction experiment you linked to: not only the MFCCs are used, but also the voiced/unvoiced bit and the pitch.
MFCC are not speaker independent. In fact, they are used for speaker identification/verification tasks!
The speaker "idiosyncracies" are both in their prosody (preserved by this ...
6
More overlap means you end up with more windows (of a given length) per second of audio. More windows (of a given length) requires more FFTs which requires more MACs or FLOPs which generally requires more processing power. In return, more window overlap provides greater time locality of information (e.g. on average, random transient events are likely ...
6
"Ok Google" is described in many publications by Google
Automatic Gain Control and Multi-style Training for Robust Small-Footprint Keyword Spotting with Deep Neural Networks
Convolutional Neural Networks for Small-Footprint Keyword Spotting
It is based on DNN specifically trained for keyphrase and runs really fast. It does not consume a lot of power even ...
5
From CMUSphinx FAQ:
There are various types of MFCC which differ by number of parameters,
but not really different for accuracy (it might be a few percent worse
or better).
The interpretation of MFCC (Roughtly introduced Alan V. Oppenheim and
Ronald W. Schafer. From Frequency to Quefrency: A History of the
Cepstrum. IEEE SIGNAL PROCESSING ...
5
Pre-emphasis is a way of compensating for the rapid decaying spectrum of speech. The experiment is worth trying on real data - you will find that the DCT basis is better at extracting a set of decorrelated coefficients when the spectrum has been whitened by the pre-emphasis filter.
This justification does not really apply in the case of music where the ...
5
First of all, don't expect an answer like "Compute such and such well-known function on the data and if the result is above such or such value it's tiuuu otherwise it's tou". You won't get anything robust or reliable with approaches like that.
The good representations of sound which can be used for this kind of discrimination task are either the Mel ...
5
The audio feature extraction libraries given above could be a good start, with some caveats:
The features you need for this task are plain MFCCs, so most of those libraries are totally overkill since they also extract features which are more meaningful in a musical context.
Some of those libraries have been designed for offline analysis (processing a pre-...
5
Using a pass band filter matching the bandwidth of speech will help.
If you have several microphones (as is now the case on cellphones), there is a trove of ICA-like methods which can take advantage of that - but your question hints me that you have only one input.
What you want to do is "one-microphone source separation" (name taken from Roweis' ...
5
What exactly can the vector/observation be? Can it be a relatively simple function, say approximation of the spectral envelope? Or is it usually a vector of R^n?
You won't go very far by summarizing the entire spectral envelope of a sound by a single real, so yes, observations are vectors - their dimensionality is in the 10 to 100 components bucket...
...
5
Researchers from the Johns Hopkins University have recently released a corpus of music, speech, and noise which, according to them, is suitable for training models for voice activity detection and music/speech discrimination.
See https://arxiv.org/pdf/1510.08484.pdf for details.
5
any window (almost any) is better than a boxcar window, unless your spectrum is flat. speech signals are not flat.
I suspect that Hamming and Hann are commonly used because you can avoid scalloping loss for 50% overlap. One gets a sufficiently favorable tradeoff between time resolution and frequency domain dynamic (side lobe level) range. There is also ...
4
I learnt to use HTK based on this very basic tutorial
http://www.info2.uqam.ca/~boukadoum_m/DIC9315/Notes/Markov/HTK_basic_tutorial.pdf
It details how to make a yes/no recogniser. It should get you off to a good start at least. There is also some HTK code on the main HTK site that allows you to perform real time speech recognition.
To improve robustness, ...
4
This is not a complete answer, but detection of words from the speech itself is almost always dictionary based - hence one needs to know what words would sound like. If you don't know language, you don't quite know the relation to the sounds. Hence, speech/audio recognition don't quite work.
There can be many primary features to classify:
Reference:
...
4
There are a bunch of parameters that you can look at:
Overall energy
Short term spectrum: Speech has a fairly distinctive "pink-like" spectrum and noise (which is happening during the non-speech parts) tends to be white if it's electrically dominated or "red" (i.e. low frequency heavy) if it's acoustic background noise or microphone noise
Amplitude ...
4
Since the differences are only in the vowels, formant detection should do the trick. A vowel typically has periodic excitation, i.e you have a fundamental and harmonic frequencies. The sound gets generated in your throat by the vocal chords (glottis).
Your vocal tract (mouth, lips, tongue, upper throat, nasal cavities etc.) create specific resonances that ...
4
Do we really need 10 samples of each word? or can we "combine" them into a single template? and if so, how?
You can use single template, 10 templates will make result more accurate.
Should we run the SVM directly on the MFCCs vectors? On the MFCC vector of vectors (for each word)? On the DTW values? or should we combine all the MFCC vectors into a single ...
4
I highly doubt that something as complex as recognizing a word can be done directly in the time domain using features as basic as zero-crossing rate.
4
It's better to copy first frame and last frame values to extend vector sequence beyond boundaries than to assign 0. This could be implemented just by adjusting indexes:
if (index1 < 0)
index1 = 0
if (index2 > N - 1)
index2 = N - 1
delta = v[index1] - v[index2]
4
By performing the windowing with overlap we are artificially increasing our time resolution (larger granularity of features in time). This is especially useful when frame duration is long (bad time resolution, very good frequency resolution), thus yielding kind of extra 'time resolution'.
Usually no one is using the rectangular window, but other types such ...
4
The standard method to remove stationary noise is spectral subtraction, where
the magnitudes of the short-time Fourier transform of the noisy signal are modified based on an SNR estimate in the respective frequency bin. The algorithm by Ephraim and Malah (see this paper) and its variants are used a lot. Note that the basic principle of spectral subtraction ...
4
Dynamic Time Warping is pretty well explained on this site. I'll use some of the diagrams from the PPT on that site to explain.
The idea is to divide the signals into segments (frames) and then compare frames sequentially through each signal. As illustrated below, motion from a segment in one signal to the next segment depends on the similarity to the ...
3
Regarding your existing steps:
Do you have some overlap between adjacent blocks? It is common in feature extraction systems to have some overlap between adjacent blocks, so that a short transient event can be correctly captured (if it is right at the end of a block, it will be right in the middle of the next one).
Your idea of discarding blocks is dangerous....
Only top voted, non community-wiki answers of a minimum length are eligible
