Does anyone know how does the "Songify" app algorithm work?

There are 2 inputs:

  • speech (or amateur singing), and
  • underlying music.

Somehow the speech is altered and mixed into the music. Which DSP algorithms are needed to achieve this effect?

  • Pitch-shifter?
  • Time-stretcher?
  • Fundamental (F0) frequency detection?
  • Auto-tune?
  • Else?

What would be the steps?


Check the Melodify SDK, does the same thing.

  • 5
    $\begingroup$ Can you please expand a little on what the linked-to page says? Generally link-only answers on StackExchange are not very highly regarded. $\endgroup$
    – Peter K.
    Jul 29 '13 at 16:13

Songify seems use Prosodic Modification (Pitch, Time) for monophonic signals:

  • First you need get the pitch contour and pitch duration from musics you want to follow (use one pitch track for monophonic signal like YIN, AMDF, Auto-Correlation)

  • Split your imput signal in voiced/unvoiced

  • Apply Pitch Scale Modification in voiced parts to match your extracted pitch contour (you can use PSOLA technique here)

  • Apply Time stretch to match your extracted pitch duration (you can use PSOLA technique here)


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.