For multi-pitch detection, I mean for example pressing several piano keys at the same time. For signal pitch detection (pressing one key at a time), after FFT, mostly the frequency with the largest magnitude is desired frequency. However, for multi-pitch detection, let's just say press two piano keys, the two frequencies with the first two largest magnitudes are not the frequencies of keys I pressed.
First of all:
mostly the frequency with the largest magnitude is desired frequency
This is a very naive and flawed approach, as what you are measuring here is very different from the perceptual notion of pitch. Not to mention that you hit the limited resolution of FFT...
Some of the challenges one has to face in multi-pitch detection:
Octaves are difficult to tell apart. Play two Cs one octave apart, and the harmonics of the highest note reinforces the harmonics of the lowest note, rather than create new peaks in the spectrum, to the point that one could hear one single "C" note with a different timbre. When mixing several notes together, some of their partials can reinforce or cancel each other.
Pitch detection can no longer be seen as a problem of period estimation. The most successful monophonic pitch detection algorithms are time-domain and based on metrics like the average magnitude (or square) difference function, and the underlying assumption is that the signal has a salient periodicity - it is almost similar to itself when translated in time. These methods cannot be applied to polyphonic signals, as the period of a mixture of periodic signals is the LCD of the components' periods. Knowing this LCD does not give away the factors...
The fact that the number of notes is not known in advance increases the size of the search space, and makes it an ill-posed problem. You are no longer ranking hypothesis like "this is a C3" vs "this is a C#3", but instead, "this is a C3 and a G4" vs "this is just a C3" vs "this is a C2 and a C3". For example, take a single piano note. One could describe its spectra as a "single C3 piano note". Or one could describe its spectra as "a C3, a C4, a G4, etc... all played together by an instrument producing purely sinusoidal notes". Distinguishing among these two options requires prior knowledge of the spectral shape of a piano note.
Intervals in Western music are usually close to having their pitch frequencies related by low integer ratios. Therefore, the composite of 2 notes can create an overtone sequence with many more than 2 possible groupings that contain tones that are roughly equally spaced in frequency (a problem for Harmonic Product Spectrum and other related methods.)
Because of equal temperament tuning, the interval notes are not always related by exact low integer ratios... they are only very close. Close enough for different overtones from different base pitch notes to beat. Which means any single short FFT frame might be in the null of beat note of two overtones beating. This further compounds the problem of multiple pitch candidates appearing and disappearing, for the multiple possible overtone sequences.
Instead of your frequency search space being points related by simple sequences of ratios of 2,3,4,5,etc. for a single pitch, the search space will need to contain sequences of all the fractional ratios of all the overtones of all the notes in all the possible interval chords and intervals. So many the sequences usually end up all overlapping, given any windowing noise (FFT length limited fattened spectral humps), instead of being distinct peaks in the pitch search space.