Hot answers tagged

25

A first note: Most modern text-to-speech systems, like the one from AT&T you have linked to, use concatenative speech synthesis. This technique uses a large database of recordings of one person's voice uttering a long collection of sentences - selected so that the largest number of phoneme combinations are present. Synthesizing a sentence can be done ...


13

This is a well-studied problem, dating back from the mid 90s (DARPA/NIST broadcast transcription challenges). Search for "speech/music segmentation" or "audio segmentation" and you'll find thousands of research papers. There are two broad approaches to solve this problem: Supervised classification Train a speech/music classifier, using a standard machine ...


12

The task of taking a long contiguous audio recording and splitting it up in chunks in which only one speaker is speaking - without any prior knowledge about the voice characteristics of each speaker - is called "Speaker diarization". You can find links to research code on the wikipedia page. If you have prior recordings of each voice, and would rather do ...


11

This is only an approximation! While the sound of whistling has a very prominent first harmonic, the other harmonics are still present too; there's a non-harmonic noise component; and the first harmonic is more a narrow bump than a sharp line. A better description of the sound of whistling would be white noise filtered through a band-pass filter with a very ...


11

So essentially, @porten pointed out what chunks of sound are. Let's look at your code: CHUNK = 1024 # What is CHUNKS here ? The chunk is like a buffer, so therefore each buffer will contain 1024 samples, which you can then either keep or throw away. We use CHUNKS of data, instead of a continuous amount of audio because of processing power. (Let's assume ...


8

The sound wave is a pressure wave in the air, a mechanical vibration. If you have two rooms separated by a wall or door, there is little or no air passage through which the sound can propagate. Instead, the wall itself have to absorb and re-emit the mechanical vibration on the other side. Since doors and walls are massive objects, they are resonating on ...


7

Yes, it's possible to analyse sound the way ears do. For example, you could compute the DFT of a signal continuously using several Goertzel filters. $$ y_k[n] = e^{j2\pi k/N} y_k[n-1] + x[n] $$ where $k= 0,1,\ldots, N-1$, so that $y_0$ is the DC or zero frequency term. Of course, this is an unstable filter, so some resetting or forgetting factor is ...


6

If the signal is recorded using just one microphone, you can use methods such as spectral subtraction. This method is more suitable for "constant" noise, like the noise from a fan or an idle engine. Other methods rely on statistics and perceptual models of speech. If the signal is recorded with several microphones, you can use blind source separation for ...


6

"Ok Google" is described in many publications by Google Automatic Gain Control and Multi-style Training for Robust Small-Footprint Keyword Spotting with Deep Neural Networks Convolutional Neural Networks for Small-Footprint Keyword Spotting It is based on DNN specifically trained for keyphrase and runs really fast. It does not consume a lot of power even ...


5

Main difference is the frequency of the fundamental, which is about an octave higher for women with the split point around 160 Hz or so. A fundamental lower than 160 Hz is most likely a male and a fundamental higher is most likely a female. A good overview over a number of studies on the topic can be found at http://www2.ling.su.se/staff/hartmut/f0_m%26f.pdf


5

That's a basic property of room acoustics. In essence the wall is a mass (ignoring resonant structures for now). Transmission through walls works as follows: sound energy hits the wall -> wall starts to vibrate -> wall radiates sound into the adjacent room. The wall simply follows Newton's second law of motion: F = m*a, force = mass times acceleration. For a ...


5

When you whistle, your mouth acts as a Helmholtz resonator: the movement of the air past your lips is causing pressure disturbances that cause the mass of the air inside your mouth to oscillate on its own compliance (like a spring-bob oscillator). The frequency produced is determined by the volume of air and its density, as well as its bulk modulus (mass and ...


5

Depending on the actual recordings, the algorithm complexity could range from dead easy to really complex... I'll take the studio recording case first, so I can assume : - (Almost) no noise coming from outside (cars, trucks, bus...) - Nobody slamming the door in the middle of the recording - Voice samples are recorded at optimal level independently of who ...


4

What is intended to do with the loop for is to record a determined number of seconds. This number is specified by RECORD_SECONDS. In order to do it, it is necessary know how many samples we have to take. In your example RECORD_SECONDS = 5 , so we want to record 5 seconds. Also, the variable RATE says how many samples are taken per second. Its unit is Hz = ...


4

I am afraid there's not much you can do. The voice part seems to have gone through the equivalent of a low pass filter with a cut off of around 1000 Hz. Basically, all of the speech components above 1000Hz are gone. The filtering action may not have been an intentional filter, but may have been due to the improper biasing of the tape during recording. If ...


3

Here is a picture of me whistling into the FFT Spectrum Analyzer app on my android. Center-line is 1.2k Hz. Notice the overwhelming strength of the first harmonic. Also note that while there is a second harmonic, it is about 20 to 40 times weaker than the first harmonic.


3

You can use something like MorphVox. Here is a demonstration. The process is called voice morphing or conversion. If you are interested in the technical aspects, a recent article you can study is Voice Conversion Using Dynamic Kernel Partial Least Squares Regression.


3

I don't think filtering is the way to go. The spectra of the different signals (the desired voice, the screamy children and the cars) are probably overlapping, so there is no real way to get rid of all that noise without spoiling your signal. Actually, the approach would depend in how loud the noise is in comparison to the signal. If the speaker is near the ...


3

Usually a data stream is broken up into chunks of data, where each chunk is some number of samples. In practice an analog to digital converter (ADC) has a buffer that you read data from it. So the buffer is also sometimes called a chunk of data. It's usually faster in practice to read many samples in a chunk, then process that chunk, instead of reading the ...


3

I just came across one article related to the compand command that might be useful (just in case someone looking for help sees this article). From the article: To test this, and my understanding of compand, I added a very simple filter to remove the quiet section of the audio by decreasing their volume dramatically: ffmpeg -i in.mp3 -filter_complex \ ...


3

The noise sounds very stationary, so I guess spectral subtraction should work well. Note however that most implementations usually have quite a few parameters to tweak, and spectral subtraction can sound either very good or completely useless depending on whether the parameters are chosen well for the given problem. If you search for Matlab implementations ...


3

How we should go about from converting Google Text to Speech's m3 file to Justin Bieber's voice? Ask Mr. Bieber to repeat what your MP3 says. I think letting one speech synthesizer first speak something and then hoping to convert it is a bad approach (as you can see if you consider how complicated that would be – first, you need to recognize in some low-...


2

Songify seems use Prosodic Modification (Pitch, Time) for monophonic signals: First you need get the pitch contour and pitch duration from musics you want to follow (use one pitch track for monophonic signal like YIN, AMDF, Auto-Correlation) Split your imput signal in voiced/unvoiced Apply Pitch Scale Modification in voiced parts to match your extracted ...


2

When using an FFT, an evenly spaced sequence of events in one domain usually produces a strong component in the other domain at a location related to the spacing of the events in the first domain. A voiced speech signal usually includes a lot of harmonics which are evenly spaced in the frequency domain. These evenly spaced events in the frequency domain ...


2

This is not a simple problem. Almost certainly any "home-brewed" system you create looking for "frequency patterns" is going to have horrible performance, unless it develops complexity comparable to full-fledged speech recognizer. The reason for this is that recognizing speech, while it seems like a simple task to our ears, is in fact quite complex, and ...


2

There are many ways to estimate the instant of glottal closure (GCI). Here (PDF link) is a recent journal article that reviews many of them. The freely-available VOICEBOX toolbox for Matlab includes a couple GCI estimation techniques. For most approaches, the fundamental idea is the same: the speech signal is can be viewed as an excitation signal ...


2

Using the standard deviation of time lags is not a bad idea - the problem is that for very noisy signals such as consonants you won't really get a pattern with peaks. Your suggestion would be more useful in the context of musical instruments sound (for example to measure the inharmonicity of a sound, from violin to piano to bell...) You can look at the ...


2

There exist DSP techniques to estimate anatomical differences in head resonance (vowel formant centers, etc.) and glottal fold characteristics (e.g. Tenor vs Bass, etc.) used in music, speech and audiology research. However there may or may not be sufficient statistical difference is these areas between the two possible vocalization sources, or the mic ...


2

This is not aliasing this is not any phenomenon, there are number of birds whose vocal fall in speech range even a frog vocal falls in speech range so essentially a training based speech detection is required to classify whether a given frame is speech or a non speech one such implementation is shown here Speech Discrimination based on multiscale spectro ...


2

You may want to try a two-pronged approach. First, have a standard pre-packaged CN to inject (like jan and Jason R suggest) at the start of the conversation/exchange. As the conversation progresses, gather low-volume data, perhaps as part of your noise reduction algorithm I assume you already have, by sampling the lowest volume signal coming in between ...


Only top voted, non community-wiki answers of a minimum length are eligible