How to choose a fixed adaptation step for decision feedback equalizer

Question

How do I choose $\mu$ - a value of a step size for adaptation in a decision feedback equalizer (DFE) with adaptive reference control (ARC)? For a regular adaptive FIR filter an adaptation step depends on a variance of input, but DFE gets symbol values as input. Another problem is that DFE employs a sign-error LMS adaptation algorithm, while ARC - a simple LMS. So do $\mu$ value for $ARC$ and $h_i(k+1)$ can be the same?

The algorithm for the DFE and ARC is the following (based on this M.Sc. thesis by Mirna Hage as a reference).

With $x^{in}_k$ being a $k^{th}$ sample:

\begin{align} x_k &= x^{in}_k - \sum_{i=1}^N h_i(k) \cdot a_{k-i}\\ a_k &= \begin{cases} +3 & \text{if} & x_k > 2 \cdot ARC_{k-1}\\ +1 & \text{if} & 0 \leq x_k \leq 2 \cdot ARC_{k-1}\\ -1 & \text{if} & -2 \cdot ARC_{k-1} \leq x_k < 0\\ -3 & \text{if} & x_k < 2 \cdot ARC_{k-1} \end{cases}\\ e_k &= x_k - ARC_{k-1} \cdot a_k\\ ARC_k &= ARC_{k-1} + \mu\cdot e_k \cdot a_k\\ h_i(k+1) &= h_i(k) + \mu\cdot \operatorname{sgn}(e_k) \cdot a_{k-i} \end{align}

where

• $N$: DFE length
• $x_k$: DFE output
• $ARC_k$: Signal level estimate
• $a_k$: Symbol estimate for a 4-level PA
• $e_k$: estimate error
• $h(k)$: DFE weights.

I tried $\mu=0,125 * 2/N$, but it seems to be wrong in some cases.

Answer 1

I'm leaving the answer here, if somebody ever stumbles upon the same question.

About LMS: Both DFE and ARC employ Least Mean Squares (LMS) adaptive algorithms: DFE is an adaptive filter and ARC can be considered as an additional tap of that filter. Hence the adaptation step size should be selected accordingly.

The general update equation for the LMS algorithm is the following:

$$ \mathbf{w}(n+1) = \mathbf{w}(n) + \mu\cdot e(n) \cdot \mathbf{x}(n), $$

where $\mathbf{w}(n)$, $e(n)$ and $\mathbf{x}(n)$ are respectively filter weights, an error and tap values at $n^{th}$ sample and $\mu$ can be a constant scalar.

With $p$ being the filter order, the bounds for $\mu$ can be defined as

$$ 0 < \mu < \frac{2}{(p+1)E\{|x(n)|^2\}}, $$

where $E\{|x(n)|^2\}$ represents the power in $x(n)$ and can be estimated using an average:

$$ \hat{E}\{|x(n)|^2\} = \frac{1}{N}\sum_{k=0}^{N-1} |x(n-k)|^2. $$

Then $\mu$ value can be chosen as a fraction of $\mu_{max}$, e.g. $0.125 \cdot \mu_{max}$.

However, in the sign-error LMS, which is an LMS algorithm with reduced complexity with the following update equation:

$$ \mathbf{w}(n+1) = \mathbf{w}(n) + \mu\cdot sign(e(n)) \cdot \mathbf{x}(n), $$

replacing an error value with its sign changes the magnitude of the correction. If $e(n)>1$, then the magnitude of the correction is smaller compared to the standard LMS. Hence the $\mu$ value can be chosen to be larger.

To the questions:

1. For example in a question, the input signal is a sequence of evenly (presumably) distributed PAM4 symbols, i.e. "-3", "-1", "1" and "3", which gives a power equal 5. This gives $\mu_{max} = \frac{2}{(p+1)\cdot 5}$.

2. DFE and ARC employ different LMS algorithms: sign-error LMS and standard LMS respectively. This calls for separate $\mu$ values for DFE and ARC. Hence the recommended adjustment would be to use sign-error LMS for both functions. Not only does this simplifies the $\mu$ value selection, it also simplifies the further implementation.

Considering the need to simplify the hardware implementation of the algorithms, $\mu$ can be chosen to be a power of two. For this particular example, $\mu$ equal to $\mu_{max}$ rounded to one significant binary digit happened to be a good choice.

References: A good reference on the LMS algorithms and step size selection is Chapter 9 "Adaptive Filtering" in Statistical Digital Signal Processing and Modeling by M.H. Hayes. Another good option is a Least_mean_squares_filter entry on Wikipedia.