1
$\begingroup$

We are looking for writing applications which would recognize Bird calls and Frog calls using Sound processing.

EDIT: I want to identify the species of bird / frog and do not want to differentiate whether the call is from a bird or a frog. There are about 1300+ bird species in India and about 300+ frog species which we are looking to identify based on the call.

I have a background of iOS application development and C / C++ programming but have never worked on Digital Signal Processing.

We do have few researchers who would collect multiple calls of birds and frogs and categorize it as per species. These calls can be used as reference purpose (?) to identify the bird or frog species by recording the call and comparing this call in user's device.

I do not know the technical feasibility of it, but I would like to know:

  • Are there any open source solutions which can be extended / used to achieve this?
  • Is this really possible to achieve with some 80% to 90% accuracy?
  • If open source solution does not exist, how to go ahead with the implementation?

I understand that the scope of this question is too broad and wide which should be ideally avoided in StackExchange. But would like to have your opinion.

Improve this question
$\endgroup$
10
  • 1
    $\begingroup$ Do you want to achieve a binary classification such as : Frog or Bird, or do you really want to predict the bird type using the sound? $\endgroup$
    – Tolga Birdal
    Commented Oct 21, 2014 at 11:34
  • 2
    $\begingroup$ This reminds me very much of an xkcd comic: xkcd.com/1425 Your situation isn't (quite) as bad, but an accurate system that separates frogs from birds and identifies species will require a good bit of work. $\endgroup$
    – JRE
    Commented Oct 23, 2014 at 13:00
  • $\begingroup$ @tbirdal - I would like to classify not between frog or a bird, but, I would like to classify individual species within frog or a bird based on the call. $\endgroup$
    – Raj Pawan Gumdal
    Commented Oct 25, 2014 at 3:53
  • 2
    $\begingroup$ 300 frog species? xkcd has a point, that's a significant research project. "Some pointers to get started" would be hiring 2 or 3 PhD's. $\endgroup$
    – MSalters
    Commented Oct 25, 2014 at 17:59
  • 1
    $\begingroup$ Yes, I was referring to PhD's on the technical side. And that's because it is a research project - with a bit of luck they can tell you how feasible it is in about a year. You'd want someone with a good background in audio and another person with a background in pattern recognition. $\endgroup$
    – MSalters
    Commented Oct 28, 2014 at 11:27

6 Answers 6

Reset to default
2
$\begingroup$

See the article below for one of the methods we use for detecting andn classifying dolphin whistles. These methods may have some sucess with birds, but I can promise that they won't separate 300 different species. All methods are available in the open source PAMGUARD software (sourceforge or pamguard.org)

J Acoust Soc Am. 2013 Sep;134(3):2427-37. Automatic detection and classification of odontocete whistles. Gillespie D, Caillat M, Gordon J, White P.

Abstract

Methods for the fully automatic detection and species classification of odontocete whistles are described. The detector applies a number of noise cancellation techniques to a spectrogram of sound data and then searches for connected regions of data which rise above a pre-determined threshold. When tested on a dataset of recordings which had been carefully annotated by a human operator, the detector was able to detect (recall) 79.6% of human identified sounds that had a signal-to-noise ratio above 10 dB, with 88% of the detections being valid. A significant problem with automatic detectors is that they tend to partially detect whistles or break whistles into several parts. A classifier has been developed specifically to work with fragmented whistle detections. By accumulating statistics over many whistle fragments, correct classification rates of over 94% have been achieved for four species. The success rate is, however, heavily dependent on the number of species included in the classifier mix, with the mean correct classification rate dropping to 58.5% when 12 species were included.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Yes PamGuard is something which I too came across. It definitely seems promising but I could not figure out the platform used for source code and also whether it can be integrated in an iOS development environment. I can see lot of Matlab stuff, but I am not sure if it is for classifier or does it play any role in the actual runtime detection of the call? $\endgroup$
    – Raj Pawan Gumdal
    Commented Jan 10, 2015 at 16:18
1
$\begingroup$

Basically it is a classification problem. You can want to classify the incoming voice.

There are many ways to tackle it. One of them would be to extract features ( by features, I mean frequency, pitch etc) and label it accordingly(if it is of bird or frog).

Then shortlist the features, which will help you differentiate between these two classes(bird or frog) and make a dictionary.

In your dictionary , it will be like this

Class
features Pitch Frequency etc etc

  1. Frog (P) = 5 (F)= 10

    1. Bird (P) = 1 (F) = 20

Whenever, your new sound comes in, you compute it features, and based on which feature it closely corresponds to (or there are many methods to do it) you choose the class.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks for your insight, how do we extract features for each species uniquely? Do you have any pointers for the same? $\endgroup$
    – Raj Pawan Gumdal
    Commented Oct 25, 2014 at 3:57
  • $\begingroup$ For this case, you have to know before hand that A voice belongs to A specie and so ON. so based on the it, you can find its features(pitch,etc). Before I go on I would like to add that I know some PhDs who are working on similar problem. So this problem if you really want to make a working system out of it would require a PhD atleast as MSalters suggested $\endgroup$
    – Omer
    Commented Oct 27, 2014 at 14:44
  • $\begingroup$ We do have special naturalists who get the calls of species from the wild and classify it per species. I guess we do not need a PhD graduate for that, once we have multiple calls of each species I guess we now have to proceed ahead with feature extraction as you say. However, my question still remains, how do we identify these unique features and extract it? Do we have any workflow for the same? $\endgroup$
    – Raj Pawan Gumdal
    Commented Oct 28, 2014 at 11:22
1
$\begingroup$

I suggest going with Mel frequency cepstral coefficients because they encode timbral information. Extract MFCC (13 values) from the recording and clasiffy it with calculating distance. You need to have two MFCCs characteristic for your two types of sound calculated and hard-coded. In my case, it worked this way for about 7 different sounds, so for 2 it wouldn't be a problem at all...

Improve this answer
$\endgroup$
2
  • $\begingroup$ You'd probably want to combine coefficients from a few time offsets. $\endgroup$
    – MSalters
    Commented Oct 24, 2014 at 13:32
  • $\begingroup$ Just to be clear, I would like to identify the particular species of a frog or a bird based on the call. $\endgroup$
    – Raj Pawan Gumdal
    Commented Oct 25, 2014 at 3:58
1
$\begingroup$

This is off-the-shelf technology for us but we're not aware of Open Source equivalents. We're not bird of frog experts so we need experts to label the database, but after that we can turn that into a representative model.

I think the other answers have some problems. For instance, the MFCC cefficients summarize energy at each frequency band. Trivially, if you move further away, energies decrease across the spectrum. This would mean you end up recognizing primarily how far away your sound source was. "Frequency" isn't a practical description of complex sounds like birds or frogs, even though you can classify use it to classify sirens. Calculating distance metrics, as both previous answers suggest fails due to anisotropy in feature space.

Improve this answer
$\endgroup$
1
$\begingroup$

The build environment for PAMGuard is described on the pamguard.org website at http://www.pamguard.org/15_SourceCode.html

Improve this answer
$\endgroup$
0
$\begingroup$

Ok, coming from a computer vision background, I would say this is a "fine-grained" problem. In such a case pure MFCC might not be ideal due to significant similarity between the sounds/pitches etc. Lately, there has been a study by Inria and submitted to the well-known Clef challenge found here. Here the problem is not exactly the same, but their database they includes multiple species. Their work is again based on MFCC, but with a clever pruning and indexing strategy. As it is based on MFCC, you would find it an easy read.

The description of this challenge is also here: clef challenge

Also, in musical analysis community this is known as a problem of granularity. There are different features or methods of extraction in order to capture the grain / granularity, as many of the audio features are already provide a coarse representation. One such work is here, which use the fine-grained features for audio retrieval. As you only have 1000-1500 classes, you would be fine with a moderate classification-retrieval algorithm. Larger scales of course demand the indexing schemes, which may not be relevant for your case.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.