It is my understanding that the least squares algorithm (e.g., in equalization) minimizes the received signal error. However, minimizing the received signal error does not necessarily equate to minimizing the BER.

I was wondering why those two are not always equivalent? And how to properly gauge the performance of the least squares algorithm if BER is not the right metric?

EDIT: Simulation description

random bits --> frame bits --> NRZL --> modulator --> AWGN --> demodulator (integer multiple oversampled) --> LS EQ --> soft bits --> BER comparison

  • modulator: FSK and BPSK (tried both)
  • AWGN: Eb/No of 7dB - 20dB
  • frame structure: 16-bit training sequence followed by 496 random bits
  • Optionally add multipath delay as well (1-2 bauds).
  • Taps on LS EQ 3-11 bauds.
  • 4
    $\begingroup$ My suggestion is to think very carefully about what you mean by "received signal error". Try to put it in mathematical terms (equations). $\endgroup$
    – MBaz
    Jan 16, 2018 at 2:09
  • $\begingroup$ Can you expand on this please? $\endgroup$ Jan 16, 2018 at 16:39
  • $\begingroup$ @BigBrownBear00: MBaz is trying to get you to think about what you believe that an equalizer does. Different equalizer structures optimize against different criteria, but they in general try to force some observable attribute of your signal toward a desired value. Depending on what that attribute is, and the characteristics of your channel, this may or may not result in the best BER performance. Least squares is just an optimization technique used to implement an equalizer; it doesn't actually dictate what criterion you're actually optmizing. $\endgroup$
    – Jason R
    Jan 18, 2018 at 13:57
  • $\begingroup$ @JasonR I understand that, but my intuition leads me to believe that minimizing the received signal error will also minimize the bit error rate. I am having a hard time thinking of a scenario where that is not true.. $\endgroup$ Jan 18, 2018 at 14:30
  • 1
    $\begingroup$ @BigBrownBear00: My point is, though, that the equalizer doesn't know the true value of the symbols, unless the data is known a priori. This can be useful for training the equalizer, but it doesn't accomplish the goal of transferring information. You have to use some criterion to decide what the error signal is. For instance, a decision-directed equalizer uses the Euclidean distance to the nearest valid symbol in the constellation as the error metric. You can design your equalizer to minimize this, but that doesn't guarantee minimum BER, as some of the decisions that it uses will be wrong. $\endgroup$
    – Jason R
    Jan 18, 2018 at 14:53

1 Answer 1


I will try to answer on your question with this: Least Square equalizer is minimizing the intersymbol interference of your signal (ISI), between the symbols, which does not translate directly to minimization of the bit error rate. When I am saying 'directly' that means that there is no exact linear dependency between the error reduction in the equalizer, and the bit error rate. However, they generally tend to go into the same direction.

The reason for that is the nature of equalizer, as the nonlinear estimator.

The chain of processing that you have shown consist of processing modules, and some of them are nonlinear, which interferes with the bit error rate minimization. In other words, bit error rate minimization is not a criterion that is used by the equalizer.

Below is the mass of papers that are talking about it, and it is not a simple matter.

Please note the first one: "Approximate Minimum Bit-Error Rate Equalization for Binary Signaling" which actually argues exactly the same line of inquiry that you are advocating: "Although most linear and decision-feedback equalizers are designed to minimize a mean-squared error (MSE) performance metric, the equalizer that directly minimizes biterror rate (BER) may significantly outperform the minimum MSE equalizer".

This means that the digital bit error based criterion would be a better choice for reducing the bit error rate than the one that is typically used.




  • $\begingroup$ Thanks for the thoughtful response. You clearly understood the nature of my question and provided helpful resources. $\endgroup$ Jan 20, 2018 at 0:16
  • $\begingroup$ I am glad that it helped you. $\endgroup$
    – VladP
    Jan 22, 2018 at 15:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.