Let's say you have a system with 1 transmitter and 5 receivers. We'll use the MUSIC algorithm to determine DOA of incoming signals.

If we have 1 signal (at 0° for example), it's easy to calculate RMSE vs SNR. Simply take the peak of the MUSIC spectrum (an angle) and find the difference between the actual target angle.

However, what if we have 2 signals? One at 0° and a second at 10° for example. You can no longer look at the MUSIC spectrum and simply pick the 2 greatest points, since the absolute greatest points may be at 0.0° and 0.1°, even if there's a clear (lower) peak at 10°.

What would be the/an appropriate way to measure the error when multiple signals are present? Or perhaps another metric (besides RMSE) is needed, such as probability of observing 2 peaks, etc.

  • There's actually algorithms like Root MUSIC that try to find the extrema of the pseudospectrum function, and you could do that. Nir is right, you need to consider individual targets, and not individual values. – Marcus Müller Jul 26 '16 at 18:51

This problem is addressed by the Point Process Theory. A currently used metric to measure the performance of algorithms is the Optimal SubPattern Assignment, or OSPA metric. Normally the OSPA is adopted to compare tracking algorithm, with some dependence in time, but it might fit to your problem as well. Here you have some references:

"A Metric for Performance Evaluation of Multi-Target Tracking Algorithms", from Ristic et al.,

and

"A Consistent Metric for Performance Evaluation of Multi-Object Filters", from Schuhmacher et al.

The way to do it is to find peaks and not maximum values. Like the function findpeaks in MATLAB (local peaks) you can define resolution (distance between peaks and more...)

Your Answer

 
discard

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.