2
$\begingroup$

Dears, I am new in signal processing and I am dealing with an Android application that should do some audio files (having the same "small" duration) comparison in real time. My main focus is about comparing two small audio files (about 10 seconds each one) which are related to a hand clap for example (or other sounds related to one short event like ball falling). In other terms, I should save a reference audio file (or a significant representation you may suggest) and then compare (in real time) a newly recorded audio file with the one already saved. For instance, if my audio of reference is a hand clap so I decide that the 2 audios are similar if the second one includes a clap even if there are some noises or the 2 sounds intensity is not the same.

I already checked musicg lib for doing this kind of comparison however, I have to convert my recorded audio files to .wav which may add some slowness in my real time application. I am wondering if there is another alternative that fits better small audio files (about 10 seconds) having the same duration ?

Thanks in advance for your suggestions and help.

Improve this question
$\endgroup$
8
  • 1
    $\begingroup$ It's not clear what you want -- you want a ball bouncing and a hand clapping to be classified as a "hand clap"? Or you want a ball bouncing to be "not a hand clap", or you want a ball bouncing to be a "ball bouncing", distinct from a "hand clap"? $\endgroup$
    – TimWescott
    Commented Dec 28, 2020 at 3:13
  • $\begingroup$ Hand clapping or ball bouncing are just examples of which kind of sounds I want to analyse. It can be other sounds as well. The most important, when I am running my program I save the reference audio (or its representation) which is defined as hand clapping for example and I have to listen to MIC and detect the presence of the reference sound (hand clapping for this case). Hope it is clearer now ? $\endgroup$
    – Newdevos
    Commented Dec 28, 2020 at 10:15
  • $\begingroup$ Hand clapping or ball bouncing are just examples of which kind of sounds I want to analyse but not both at the same time. It can be other sounds as well. The most important, when I am running my program: i) I save the reference audio (or its representation) which can be defined as hand clapping for example then, ii) I have to listen to MIC and detect the presence of the reference sound (hand clapping for this case). iii) Once I get a similar sound to hand lapping I notify the user. Hope it is clearer now ? $\endgroup$
    – Newdevos
    Commented Dec 28, 2020 at 10:28
  • $\begingroup$ Yes, it is. Now, this is Stackexchange, where they want the whole question in the question, and the answers in the answers. Could you edit your question with the above clarification? $\endgroup$
    – TimWescott
    Commented Dec 28, 2020 at 16:10
  • $\begingroup$ Sorry for the catechism, but I think you're asking for more than is possible: You want to take a reference file with a recording of a specific hand clapping in a specific room into a specific microphone under specific conditions (or a specific ball bouncing, or a specific dog barking etc.). Then you want an application that identifies any hand clapping, or any ball bouncing, or any dog barking. Yes? $\endgroup$
    – TimWescott
    Commented Dec 28, 2020 at 16:15

2 Answers 2

Reset to default
-1
$\begingroup$

A hand clap has a very high intensity and so I believe that you can detect the time location using an intensity threshold. For comparison purposes you can do a direct comparison of FFT, using for example mean square error or other metric.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Hi Filipe, I am following your recommandation. In fact, I implemented and applied FFT on both audios (having same size) but I am wondering how to compare them. You were pointing Mean Square Error. So I want to get a clear formula that I can impelemnt in java. Now, the result of FFT is an array of Complex objects(having imaginary and real attributes). Would be possible please to explicit how to perform it for this kind of FFT result ? Thanks in advance. $\endgroup$
    – Newdevos
    Commented May 19, 2021 at 20:28
  • $\begingroup$ From the FFT you just need the magnitudes, so you must calculate ABS value. To estimate the difference between spectrums you can calculate SQRT( SUM( (x1-y1)^2 + (x2-y2)^2 + ... ) ) $\endgroup$
    – Filipe Pinto
    Commented May 20, 2021 at 11:12
  • 1
    $\begingroup$ Thanks Filipe. Assuming I have 2 arrays of complex numbers, X and Y as result of FFT. First, I have to compute the absolute value of each complex number in the 2 arrays. Second I apply this formula: SQRT( SUM( (x1-y1)^2 + (x2-y2)^2 + ... ) ). Hope my understanding is right ? $\endgroup$
    – Newdevos
    Commented May 20, 2021 at 13:54
-1
$\begingroup$

What you're interested in sounds like a generic version of a matched filter. I'm not going to be able to speak to the software aspect of how you implement the file handling and "always-on" listening ability of this problem, but from a signal processing perspective, that's what you want to look into. It's worth noting that your filter is unlikely to match any hand clapping or any ball bouncing based on the reference waveform, but the magnitude of the matched filter response should give you a pretty reliable measure of how close the input is. You can use more advanced techniques if you get a lot of false alarms, like requiring the filter response to be a certain SNR by taking constant ambient noise measurements and comparing the amplitude to those.

EDIT: I re-read your question and am convinced this is the correct approach, but I suggest against trying to find a file-level solution to this issue. What you want to do is open the "reference" file and time-reverse the audio data. Then as mic input comes in, you convolve it with the reversed reference waveform (this is matched filtering). A strong response (impulse) from that convolution output indicates a match. In terms of tools you don't even need a powerful audio library, just something that can get the audio in 1-D vector form and then a time-reverse and 1-D convolution (common operations in matrix libraries).

Improve this answer
$\endgroup$
28
  • $\begingroup$ Thanks for this suggestion. I have to read more about this topic to understand it better in order to implement it. $\endgroup$
    – Newdevos
    Commented Dec 28, 2020 at 22:08
  • $\begingroup$ Good way to learn this quick would be to experiment with basic code. Take the data from your audio file and time-reverse it to create the matched filter. Convolve the original data with that filter and you should see a large impulse in the output $\endgroup$
    – Keegs
    Commented Dec 29, 2020 at 14:53
  • $\begingroup$ Please I want to be sure that the meaning of "get the audio in 1-D vector" is equivalent to sampling the original audio and extract a raw audio data. In fact, in Android I did not find a clear code/API tha does this unless one API. This API ensures "the extraction of demuxed, typically encoded, media data from a data source". Then I will get the 1-D vector as suggested. Please tell me if the result matches with your recommandation of "get the audio in 1-D vector" ?? Thanks in advance. Kid regards. $\endgroup$
    – Newdevos
    Commented Dec 30, 2020 at 15:54
  • $\begingroup$ Sampling for me is related to this explanation $\endgroup$
    – Newdevos
    Commented Dec 30, 2020 at 16:06
  • $\begingroup$ Yes you want the raw samples in a vector where each element of the vector is one amplitude value separated in time by the sampling period. You will likely want to sum the channels if the audio is in stereo or any other multi-channel format. I am very surprised that no Android library exists to import a sound file into a numerical vector/array. $\endgroup$
    – Keegs
    Commented Dec 30, 2020 at 19:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.