Taken from Adaptive Filter Theory (2014) written by Haykin page 110 :

$$y(n) = \sum_{k=0}^{\infty} w_k^*u(n-k), \quad n=0,1,2,...$$

where $u$ and $w$ are complex values. My question is why use complex conjugate of $w_k$? The answer found in the book says "..., in complex terminology, the term $w_k^*u(n-k)$ represents the scalar version of an inner product of the filter coefficient $w_k$ and the filter input $u(n-k)$ ". I still don't understand, can you elaborate more on this answer?


2 Answers 2


Turns out that convolution and correlation are closely related. For real signals (and finite energy signals):

Convolution: $\qquad y[n] \triangleq h[n]*x[n] = \sum\limits_{m=-\infty}^{\infty} h[n-m] \, x[m]$

Correlation: $\qquad R_{yx}[n] \triangleq \sum\limits_{m=-\infty}^{\infty} y[n+m] \, x[m] = y[-n]*x[m]$

Now, in metric spaces, we like to use this notation:

$$ R_{xy}[n] \triangleq \Big\langle x[m], y[n+m] \Big\rangle = \sum\limits_{m=-\infty}^{\infty}x[m] y[n+m] $$

The $\langle \mathbf{x},\mathbf{y} \rangle$ is the Inner product of the vectors $\mathbf{x}$ and $\mathbf{y}$ where $\mathbf{x} =\{x[n]\}$ and $\mathbf{y} =\{y[n]\}$. Then we also like to define the norm of a vector as

$$\begin{align} \| \mathbf{x} \| &\triangleq \sqrt{\big\langle \mathbf{x},\mathbf{x} \big\rangle} \\ &= \sqrt{\sum\limits_{m=-\infty}^{\infty}x[m] x[m]} \\ &= \sqrt{\sum\limits_{m=-\infty}^{\infty}x^2[m]} \\ \end{align}$$

and that looks a lot like the Euclidian length of a vector with an infinite number of dimensions. All this works very well for the case where the elements $x[n]$ of the vector $\mathbf{x}$ are all real. The norm $\| \mathbf{x} \|$ is always real and non-negative.

So, if we generalize and allow the elements of $\mathbf{x}$ to be complex-valued, then if the same definition of norm is to be used,

$$\| \mathbf{x} \| \triangleq \sqrt{\big\langle \mathbf{x},\mathbf{x} \big\rangle}$$

then the definition of the inner product needs to be modified a little:

$$\Big\langle \mathbf{x},\mathbf{y} \Big\rangle = \sum\limits_{m=-\infty}^{\infty}x[m] y^*[m]$$

Then if $\mathbf{x}$ has complex-valued elements, the norm comes out as:

$$\begin{align} \| \mathbf{x} \| &\triangleq \sqrt{\big\langle \mathbf{x},\mathbf{x} \big\rangle} \\ &= \sqrt{\sum\limits_{m=-\infty}^{\infty}x[m] x^*[m]} \\ &= \sqrt{\sum\limits_{m=-\infty}^{\infty}\Big|x[m]\Big|^2} \\ \end{align}$$

So, evidently, Haykin is just walking that definition of inner product back to the definition of convolution.


The use of the conjugate in the formation of the adaptive filter isn't necessary. However, if you do not write the output using a conjugate then it is quite easy to forget that the variables you are dealing with are complex. If you write $$h(n)=\sum_{k=0}^{\infty}w_k(n)u(n-k) $$ then it isn't clear that you are dealing with complex quantities.

As Robert has already pointed out, the definition of correlation needs to be updated to handle complex data if you are used to only seeing it defined for real data.

Another reason for using the conjugate like this, is to simplify the taking of derivatives for finding the solution to the adaptive filter. Assume we have a real-valued objective function $J(w)$ that we are trying to minimize - usually this is the mean squared error i.e. $E[e^*(n)e(n)]$. Taking the derivative of this quantity wrt $w$ is not so straightforward.

The common technique is write the objective function as a function of $w$ and $w^*$ - that is, treat $w$ and $w^*$ as independent variables. Now we have $$ J(w) = F(w,w^*)$$

To find the minimum we take the derivatives wrt $w$ and $w^*$ and set them to zero, so we wish to solve $$\frac{\partial F(w,w^*)}{\partial w } = \frac{\partial F(w,w^*)}{\partial w^* }= 0$$

However, if you do the analysis you will find that $$\frac{\partial F(w,w^*)}{\partial w } =0 \iff \frac{\partial F(w,w^*)}{\partial w^* }= 0 $$

So that you only need to solve one of these equations.

For complete details you can look at:

  • "A complex gradient operator and its application in adaptive array theory", Brandwood 1983, Communications, Radar and Signal Processing, IEE Proceedings F
  • "The Complex Gradient Operator and the CR-Calculus" Kreutz-Delgado here
  • "Complex gradient and Hessian", van den Bos, 1994, Vision, Image and Signal Processing, IEE Proceedings

For the theory of Adaptive Filters, I much prefer the presentation in "Fundamentals of Adaptive Filtering" by Ali Sayed. He presents a unified derivation of LMS, NLMS, RLS, APA and Lattice filters.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.