Polyphonic detection/Mulit-pitch detection/Chord recognition

Question

I build application witch will be transform piano sound to musical notes. I found some pitch detection algorithms with implementations (mainly based on autocorrelation or fft), but chord recognition, of course is not working. What is best method for instrument as piano(both hands + chords)? Is there any implementation of this problem?

Answer 1

As @jan pointed out, you're probably asking for a little too much, especially if you're looking for a ready implementation. Doing a quick search on Google, I came across several papers that may be a helpful start.

In this paper called Multi-pitch Detection Algorithm Using Constrained Gaussian Mixture, the authors use the Expectation Maximization algorithm to solve a Gaussian Mixture Model to detect multiple tones. This is most likely not a very computationally efficient algorithm, since mixture models are normally very hard to solve, and have to be primarily done offline (no real-time version).

If you're looking for a moderately robust, but potentially fast algorithm, look at this Stack Overflow answer. The author recommends MUSIC and ESPRIT algorithms.

The best thing to do is to give something a try and come back to us with specific DSP and algorithms questions and sites like Stack Overflow for specific implementation questions.

Answer 2

Here is a bibliography including some recent research papers on possible solutions to polyphonic pitch detection/recognition/estimation problems: http://www.cs.tut.fi/~klap/iiro/

The ISMIR/MIREX conference also publishes many research papers on this topic.

Answer 3

monophonic pitch detection is hard enough. polyphonic is a whole dimension harder. dunno how Melodyne does it, but they're probably the top of the pile these days.

there aren't a lot of papers published on pitch detection, these days. back in the olden days it was about speech processing and there are a few algs for monophonic. then the issue became that of measuring the fundamental frequency $f_0$ (or the period, $\tau_0$) of a periodic or quasi-periodic signal. ultimately the algs that were any good were based on Average Magnitude Difference Function (AMDF) or autocorrelation. there's a relationship between the two (or at least between autocorrelation and the average magnitude-squared difference function). one has peaks exactly where the other has minimums.

i've never really fiddled with polyphonic pitch detection, because most of what i had worked on was meant to be used in real time. perhaps, if you can identify a note (perhaps the note with the most amplitude) out of the mix, you can use a comb filter to filter out that note and continue to identify the strongest note of the residual that is left. all this is predicated on the notion that all of the notes are harmonic or quasi-periodic. some tones, that humans perceive as having a pitch, are not harmonic. bells, for instance. in that case, i dunno how you would use a comb filter to isolate the note, since the harmonics might not line up with the teeth of the comb.

Answer 4

I built a console chord-recogniser for the jack audio API using a good and fast library, which does the magic.

It can be found here https://github.com/relascope/chord-rec

The madmom library looks also interesting.

Answer 5

Have you looked at this paper?

https://arxiv.org/pdf/1804.02918&ved=2ahUKEwj46ZyDrajjAhXkIzQIHdt5AqYQFjACegQIAhAB&usg=AOvVaw3OoDyPWKVPW7fDitFpn_cH&cshid=1562693079932