I would like to train a deep neural network to perform lip syncing given an audio input like in this article.

I want to train my neural network only on Obama speeches like done in the article and want to apply it to other people voices. Does the structure of the audio signal in the time domain (after a RMS normalization) change much from one person to another if they say the same thing ?


In short, yes.

One thing that you'll find is that the human ear is actually relatively insensitive to phase, so you can take an audio sample, apply an allpass filter that doesn't affect the signal's magnitude spectrum but distorts its phase, and it won't make it sound much different to you. However, the time-domain waveform could look completely different. I'm not sure what techniques the researchers used in the link you gave, but I would guess that it isn't likely to be based upon the time-domain structure of the audio waveforms.

| improve this answer | |
  • $\begingroup$ Ok I understand better now, thank you. In fact they applied it to other Obama speeches so they didn't have this problem. Would a voice changer software be enough to solve this problem ? $\endgroup$ – Tiffany Sep 12 '17 at 12:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.