Say there are 3 people in the room. Each of them have a unique voice timbre. I want to recognize people by their timbre and know who's speaking.


The task of taking a long contiguous audio recording and splitting it up in chunks in which only one speaker is speaking - without any prior knowledge about the voice characteristics of each speaker - is called "Speaker diarization". You can find links to research code on the wikipedia page.

If you have prior recordings of each voice, and would rather do classification, this is a slightly different problem (Speaker recognition or Speaker identification). Software tools for that are available here (note that general purposes speech recognition packages like Sphinx or HTK are flexible enough to be coaxed into doing that).

| improve this answer | |
  • $\begingroup$ Wow! Thanks for such an extensive answer. I'm interested in doing this live, basically detecting when and who is speaking and maybe writing that to a separate file, or just writing to log when and who spoke. $\endgroup$ – Dmitry Aug 15 '12 at 15:58
  • $\begingroup$ Did you find any library which works in real time. I came across IBM Watson Speech Recognition API, but it is not open source. $\endgroup$ – subtleseeker Mar 26 '19 at 8:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.