Hot answers tagged

26

A first note: Most modern text-to-speech systems, like the one from AT&T you have linked to, use concatenative speech synthesis. This technique uses a large database of recordings of one person's voice uttering a long collection of sentences - selected so that the largest number of phoneme combinations are present. Synthesizing a sentence can be done ...


14

This is a well-studied problem, dating back from the mid 90s (DARPA/NIST broadcast transcription challenges). Search for "speech/music segmentation" or "audio segmentation" and you'll find thousands of research papers. There are two broad approaches to solve this problem: Supervised classification Train a speech/music classifier, using a standard machine ...


11

So essentially, @porten pointed out what chunks of sound are. Let's look at your code: CHUNK = 1024 # What is CHUNKS here ? The chunk is like a buffer, so therefore each buffer will contain 1024 samples, which you can then either keep or throw away. We use CHUNKS of data, instead of a continuous amount of audio because of processing power. (Let's assume ...


11

What you want to do is called "Super Resolution" in the field of imaging. This is an ill posed problem. Hence in order to solve it you need some prior / model about your audio data. For example you may look at the model in Speech Super Resolution Using Parallel WaveNet.


9

While I don't know about tools for upsampling using AI, I am challenging the assumption that the main problem with your sample is lost high frequncies due to resampling from 44kHz to 22kHz, so "just" upsampling is unlikely to solve your root problem. The possibility of AI upsampling itself is sensible, though: While standard (mathematical) ...


7

Yes, it's possible to analyse sound the way ears do. For example, you could compute the DFT of a signal continuously using several Goertzel filters. $$ y_k[n] = e^{j2\pi k/N} y_k[n-1] + x[n] $$ where $k= 0,1,\ldots, N-1$, so that $y_0$ is the DC or zero frequency term. Of course, this is an unstable filter, so some resetting or forgetting factor is ...


6

"Ok Google" is described in many publications by Google Automatic Gain Control and Multi-style Training for Robust Small-Footprint Keyword Spotting with Deep Neural Networks Convolutional Neural Networks for Small-Footprint Keyword Spotting It is based on DNN specifically trained for keyphrase and runs really fast. It does not consume a lot of power even ...


5

Depending on the actual recordings, the algorithm complexity could range from dead easy to really complex... I'll take the studio recording case first, so I can assume : - (Almost) no noise coming from outside (cars, trucks, bus...) - Nobody slamming the door in the middle of the recording - Voice samples are recorded at optimal level independently of who ...


4

You can use something like MorphVox. Here is a demonstration. The process is called voice morphing or conversion. If you are interested in the technical aspects, a recent article you can study is Voice Conversion Using Dynamic Kernel Partial Least Squares Regression.


4

What is intended to do with the loop for is to record a determined number of seconds. This number is specified by RECORD_SECONDS. In order to do it, it is necessary know how many samples we have to take. In your example RECORD_SECONDS = 5 , so we want to record 5 seconds. Also, the variable RATE says how many samples are taken per second. Its unit is Hz = ...


4

I am afraid there's not much you can do. The voice part seems to have gone through the equivalent of a low pass filter with a cut off of around 1000 Hz. Basically, all of the speech components above 1000Hz are gone. The filtering action may not have been an intentional filter, but may have been due to the improper biasing of the tape during recording. If ...


4

As pointed in some of the comments, there are several issues involved in your problem that you should consider to go for an optimal solution: Quality of the record system. Features of the microphone, filter of the preamps, etc. What is the frequency response of the system (mic + preamp + adc)? https://blog.faberacoustical.com/2009/ios/iphone/iphone-...


3

Usually a data stream is broken up into chunks of data, where each chunk is some number of samples. In practice an analog to digital converter (ADC) has a buffer that you read data from it. So the buffer is also sometimes called a chunk of data. It's usually faster in practice to read many samples in a chunk, then process that chunk, instead of reading the ...


3

I don't think filtering is the way to go. The spectra of the different signals (the desired voice, the screamy children and the cars) are probably overlapping, so there is no real way to get rid of all that noise without spoiling your signal. Actually, the approach would depend in how loud the noise is in comparison to the signal. If the speaker is near the ...


3

Here is a picture of me whistling into the FFT Spectrum Analyzer app on my android. Center-line is 1.2k Hz. Notice the overwhelming strength of the first harmonic. Also note that while there is a second harmonic, it is about 20 to 40 times weaker than the first harmonic.


3

The noise sounds very stationary, so I guess spectral subtraction should work well. Note however that most implementations usually have quite a few parameters to tweak, and spectral subtraction can sound either very good or completely useless depending on whether the parameters are chosen well for the given problem. If you search for Matlab implementations ...


3

How we should go about from converting Google Text to Speech's m3 file to Justin Bieber's voice? Ask Mr. Bieber to repeat what your MP3 says. I think letting one speech synthesizer first speak something and then hoping to convert it is a bad approach (as you can see if you consider how complicated that would be – first, you need to recognize in some low-...


2

When using an FFT, an evenly spaced sequence of events in one domain usually produces a strong component in the other domain at a location related to the spacing of the events in the first domain. A voiced speech signal usually includes a lot of harmonics which are evenly spaced in the frequency domain. These evenly spaced events in the frequency domain ...


2

Songify seems use Prosodic Modification (Pitch, Time) for monophonic signals: First you need get the pitch contour and pitch duration from musics you want to follow (use one pitch track for monophonic signal like YIN, AMDF, Auto-Correlation) Split your imput signal in voiced/unvoiced Apply Pitch Scale Modification in voiced parts to match your extracted ...


2

This is not a simple problem. Almost certainly any "home-brewed" system you create looking for "frequency patterns" is going to have horrible performance, unless it develops complexity comparable to full-fledged speech recognizer. The reason for this is that recognizing speech, while it seems like a simple task to our ears, is in fact quite complex, and ...


2

There are many ways to estimate the instant of glottal closure (GCI). Here (PDF link) is a recent journal article that reviews many of them. The freely-available VOICEBOX toolbox for Matlab includes a couple GCI estimation techniques. For most approaches, the fundamental idea is the same: the speech signal is can be viewed as an excitation signal ...


2

Using the standard deviation of time lags is not a bad idea - the problem is that for very noisy signals such as consonants you won't really get a pattern with peaks. Your suggestion would be more useful in the context of musical instruments sound (for example to measure the inharmonicity of a sound, from violin to piano to bell...) You can look at the ...


2

There exist DSP techniques to estimate anatomical differences in head resonance (vowel formant centers, etc.) and glottal fold characteristics (e.g. Tenor vs Bass, etc.) used in music, speech and audiology research. However there may or may not be sufficient statistical difference is these areas between the two possible vocalization sources, or the mic ...


2

This is not aliasing this is not any phenomenon, there are number of birds whose vocal fall in speech range even a frog vocal falls in speech range so essentially a training based speech detection is required to classify whether a given frame is speech or a non speech one such implementation is shown here Speech Discrimination based on multiscale spectro ...


2

You may want to try a two-pronged approach. First, have a standard pre-packaged CN to inject (like jan and Jason R suggest) at the start of the conversation/exchange. As the conversation progresses, gather low-volume data, perhaps as part of your noise reduction algorithm I assume you already have, by sampling the lowest volume signal coming in between ...


2

Dan Ellis has some neat Matlab scripts that allow you to pull audio features from files . . . http://labrosa.ee.columbia.edu/matlab/ The dynamic time warp might be a good starting point for the task you describe.


2

This is an addition to porten's answer. You are not exactly right with your description of rate. The rate is in Hertz, meaning in your case 44100 Hz - or 44100 samples per second. So you are basicly reading out that many digitized values from your device. So if you want to record 5 seconds, you will have to save $5\cdot 44100$ samples. Because, as porten ...


2

Are you being asked to write a simple vocoder? Your question is somewhat vague and confusing, but it seems like you are being asked to determine parameter thresholds for deciding when a speech frame is voiced (has fundamental periodic component and harmonics), unvoiced or frictional (noise generated), and silent (do not encode). It looks like you would ...


2

Some recent ideas (2-3 years old, I haven't followed recently): Design an elaborate generative model of the spectrogram of music signals, combining a non-negative mixture of a small set of background frames (models the background music) + a harmonic comb filtered by a non-negative mixture of smooth basis functions (the source filter model of the lead ...


2

There is no (known) magic solution, as the sound of the same word from very different speakers are often not actually similar to one another in term of any simple characteristics such as you list (frequency range, intensity range, pitch, etc.) To often, humans can only guess at what word is said from an unknown speaker based on context. And you don't have ...


Only top voted, non community-wiki answers of a minimum length are eligible