I want to design a web-app for my cousin who is 2 years of age in which i have implemented a functionality in which when an image is clicked some sound gets played and the user has to make the same sound which gets recorded.

For eg-If i click on image of "Apple" the sound made is "A for Apple".Now the user has to say those words which get recorded. Now I want to calculate the percentage of accuracy with which the user spoke.I want to know how can i know the accuracy percentage.
I read some research papers and did some online research and I have come to know across is that i will have to use DTW or MFCC and other research papers it was asked to use both.
I want to know what resources should i refer to and what parameters of sound signal should i use for the purpose of knowing the accuracy

  • $\begingroup$ That is a super cool idea! $\endgroup$ – Marcus Müller Jul 11 at 10:07
  • 1
    $\begingroup$ I think you shouldn't be reading papers right now, though. Get a textbook about speech processing – there's much established knowledge there! (and a textbook is better to read than the original papers - it's knowledge, sorted, condensed and explained, whereas papers tend to present something new. If you read a new paper in a field that is somewhat mature, it will inherently be a bit "exotic", because all the "common knowledge" things have already been tried out.) $\endgroup$ – Marcus Müller Jul 11 at 10:09
  • $\begingroup$ Be a bit careful though: you'll end up training a very complex neural network (your cousin) that can trivially overfit to what is presented as correct answer :) $\endgroup$ – Marcus Müller Jul 11 at 10:11
  • $\begingroup$ I know it's not helpful, but on iOS you could make such app in SwiftUI fairly easily. If you want to make a very simple classifier then MFCC's + DTW is the simplest way to go. Alternatively, you can use ready speech recognition model/API to get the exact translation. $\endgroup$ – jojek Jul 11 at 14:22
  • $\begingroup$ A person who is 2 years of age has a different opinion on what "A for Apple" should sound like than an adult. The classifier would have to get a bit creative so that it correctly interprets the 2yo sounds. How this enrichment is done is going to be interesting in this application... $\endgroup$ – A_A Jul 14 at 8:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.