In a formulation of the variable stepsize normalized least mean squares (VSS-NLMS)-algorithm, i found an expression that I do not know yet, and searching the internet did so far not yield results i can make sense of.

The algorithm is expressed as

$$\mathbf{\hat{h}}(k) = \mathbf{\hat{h}}(k-1) + \mu_\text{NPVSS}(k)\mathbf{s}(k)e(k),$$

where $$\mathbf{\hat{h}}(k)$$ is the current estimate of the desired filter impulse response, $$\mathbf{s}(k)$$ the source signal and $$e(k)=x(k)-\hat{x}(k)$$ the error difference between filter output and actual output (using the real $$\mathbf{h}(k)$$). The stepsize for this algorithm is defined as $$\mu_\text{NPVSS}(k) = \frac{1}{\mathbf{s}^\text{T}(k)\mathbf{s}(k)} \left[ 1-\frac{\sigma_b}{\sigma_e(k)} \right],$$

with $$\sigma_b$$ and $$\sigma_e$$ denoting the square root of the variance/power of a noise process $$b(k)$$ uncorrelated to the source signal and of the error difference, respectively.

For a practical implementation it is suggested to compute the fraction $$[\mathbf{s}^\text{T}(k)\mathbf{s}(k)]^{-1}$$ as $$[\delta + \mathbf{s}^\text{T}(k)\mathbf{s}(k)]^{-1}$$ instead.

In this context, the regularization parameter is defined as $$\delta = \text{cst} \cdot \sigma_s^2.$$

I am not too familiar with the concept of regularization, so I have no clue what the term $$\text{cst}$$ means in this algorithmic formulation.

What does it mean? Any pointers or literature recommendations would be very much appreciated.

The standard normalized step-size LMS algorithm computes the current step-size according to

$$\mu = \frac{c}{s_k^T \cdot s_k}$$

where $$c$$ is a suitable scale factor and $$s_k^T \cdot s_k$$ is the total energy of the current tap inputs. The algorithm aims to adjust step size according to input signal power; when input has large power then decrease the step-size so that filter coefficient update will not deivate much. But if the signal goes to zero or very small levels (silence durations, or small variance sections), then the division will possibly be very large and algorithm may produce unbounded (very large) outputs or even diverge...

To prevent this from happening, a small nonzero factor; the regularisation factor, is added to the denominator

$$\mu = \frac{c}{s_k^T \cdot s_k + \delta}$$

to limit the maximum of the fraction to $$c/\delta$$ when input power goes to zero.

• Thank you for your quick response! How can $\delta$ be determined? Do I simply choose an arbitrary constant, maybe based on the desired maximum stepsize $\mu_\text{max}= c/\delta$, or are there any other aspects to consider? Jul 24 '19 at 15:38
• fundamentally depends on your input character and based on experience. It should be large enough to prevent unbounded outputs (wrt some criteria) and small enough to permit responsive normalization... Jul 24 '19 at 20:38