8 votes
Accepted

What's the Difference Between LMS and Gradient Descent Adaptation?

The LMS algorithm is based on the idea of gradient descent to search for the optimal (minimum error) condition, with a cost function equal to the mean squared error at the filter output. However, it ...
Jason R's user avatar
  • 24.6k
5 votes

Adaptive filtering: Optimum filter length and delay

In order to be able to choose an optimal value for the delay $\Delta$ it's important to understand how the system works. The purpose of the delay is to decorrelate the desired signal $s(n)$ and the ...
Matt L.'s user avatar
  • 89.5k
4 votes
Accepted

Gradient descent algorithm not converging

Your step size is too large. The upper limit $2/\lambda_{max}$ for the step size $\mu$ is valid if the update is defined as $$\mathbf{w}_{k+1}=\mathbf{w}_{k}-\frac{\mu}{2}\nabla J(\mathbf{w}_k)$$ The ...
Matt L.'s user avatar
  • 89.5k
3 votes

Best DSP algoritms for ultrasonic background noise cancellation

One way to do this is to look at modeling your signal: $$ x[n] = x_h[n] + x_n[n] $$ where $x_h$ is the hissing sound and $x_n$ is the noise. If you can say that $x_n$ is modeled as: $$ x_n[n] = \sum_{...
Peter K.'s user avatar
  • 25.7k
3 votes
Accepted

Filtered-X LMS algorithm and built-in MATLAB implementation

I think you are mixing the estimated, real and modelled responses. You can kinda safely assume that the real transfer functions $h_{N_{p}}$ and $h_{N_{s}}$ are never known. This, of course, is in the ...
ZaellixA's user avatar
  • 1,254
2 votes

Regarding the choice of cost function in adaptive control - squared error vs absolute error

This is an interesting question since both squared error and absolute error are convex functions, so they are both going to give the optimal solution when minimized. My intuition is that the $\ell_2$-...
orchi_d's user avatar
  • 567
2 votes
Accepted

Control theory: how do you initialize input for a model predictive controller?

This algorithm in general tries to solve an optimization problem each step, defined as, $$ \begin{aligned} & \underset{\textbf{u}}{\text{minimize}} & & \sum_{i\ =\ 0}^{N-1}\left[x^T\!(k+1+...
fibonatic's user avatar
  • 974
2 votes
Accepted

What is usual independence assumptions on adaptive filters

I think there is an error in your referenced independence assumption. $w(k)$ should be the update part $\Delta w(k)$ i.e the $w(k+1)=w(k)+\mu \Delta w(k) =w(k)+\mu \frac{x(k)*e(k)}{c+x(k)^Hx(k)}$ ...
Claes Rolen's user avatar
2 votes
Accepted

Are the RLS filter and Kalman filter gradient methods?

Recursive least squares (RLS) filters don't use gradient descent. As their name suggests, they use a least-squares fit to determine the optimum coefficients at each time step. Via clever formulation ...
Jason R's user avatar
  • 24.6k
2 votes

Applying Photoshop's "Shadow / Highlight" Correction Using Standard Image Processing Algorithms

This Mathematica code substitutes a "gamma" operation for whatever Photoshop's "Amount" parameter controls, but it achieves roughly the same result. ...
user6552's user avatar
  • 121
2 votes

How do I implement an adaptive thresholding algorithm for underwater sonar

Your question has received quite few contributions, probably because of a lacking content. During a recent conference , I came across the PhD thesis: Détection en Environnement non Gaussien (Detection ...
Laurent Duval's user avatar
2 votes
Accepted

RLS Algorithm (Memoryless)

In its bare classical form the RLS algorithm, recursively (for every new iteration), solves the classical problem of least squares; by computing the optimal FIR transversal filter coefficients $w[n]$ ...
Fat32's user avatar
  • 28.1k
2 votes
Accepted

Stochastic approximation algorithm

The issue is possibly that the input signal you have chosen is not persistently exciting. This means that the signal doesn't "excite" enough modes of the filter in order to be able to accurately ...
Peter K.'s user avatar
  • 25.7k
2 votes

Is a neural network an adaptive filter?

An adaptive filter is a special case of a neural network (NN). They have in common that they multiply an input x[n] with weights w[n], the result y[n]=x[n]w[n] is compared to the target t[n] (e.g. the ...
Aaron Verweg's user avatar
2 votes

Is a neural network an adaptive filter?

or is it called a neural network because it is "fancy"? Machine neural networks are called such because they deliberately emulate the functioning of biological neural networks, in an ...
TimWescott's user avatar
  • 12.6k
2 votes
Accepted

Convergence of the RLS Algorithm for a Forgetting Factor $ \lambda < 1 $

The convergence itself depends on the eigen values of the empirical correlation matrix (See remark below). By setting $ \lambda \leq 1 $ we allow the filter to adapt in the non stationary cases. We ...
Royi's user avatar
  • 19.5k
1 vote

Algorithm for "adaptive phase rotation" in iZotope RX 8

This is an interesting topic. I assume that the goal can be expressed as designing a time-variant allpass filter (phase distortion) that minimize amplitude peaks while retaining the frequency ...
Knut Inge's user avatar
  • 3,350
1 vote

LMS Adaptive Filter for system identification

In general for a standard LMS you can only ensure convergence if the stepsize $µ < 1 / (2p\sigma^2)$ . With p being the filter order and $\sigma^2$ the variance of the input signal x. Therefore if ...
Don's user avatar
  • 165
1 vote

Approximate a Known System with Adaptive Filter and an Unknown System in a Series

The problem with your diagram is that the calculation of the error isn't done on the output of the adaptive filter. The adaptive filter minimizes the error based on the idea the error is a function ...
Royi's user avatar
  • 19.5k
1 vote

How to differentiate two different signals from their combined signal

If the recording is anechoic than this is simple enough: assuming that the distance between the microphones is $d$ than you can estimate the front source as $$x_f(t) = m_f(t-d/c_0) - g \cdot m_r(t)$$ ...
Hilmar's user avatar
  • 44.1k
1 vote
Accepted

How to choose a fixed adaptation step for decision feedback equalizer

I'm leaving the answer here, if somebody ever stumbles upon the same question. About LMS: Both DFE and ARC employ Least Mean Squares (LMS) adaptive algorithms: DFE is an adaptive filter and ARC can be ...
megasplash's user avatar
1 vote
Accepted

Unknown symbol/expression in text about adaptive filters (cst)

The standard normalized step-size LMS algorithm computes the current step-size according to $$ \mu = \frac{c}{s_k^T \cdot s_k} $$ where $c$ is a suitable scale factor and $s_k^T \cdot s_k$ is the ...
Fat32's user avatar
  • 28.1k
1 vote

Is CMA equalization applicable for OFDM

No. OFDM isn't constant modulus (i.e. constant envelope) in time domain, if you look at it as one system. It's quite the opposite; it's known for its high PAPR (which you probably know!). This is the ...
Marcus Müller's user avatar
1 vote

Block LMS with overlapping blocks

There is no hard rule regarding convergence speed of the block-LMS vs sample-by-sample LMS. It really depends on the scenario. On top of my head is the following two (stationary) scenarios: A very ...
M.Halimeh's user avatar
1 vote

Is document image binarization a closed research field

First, in science, a field is rarely closed, sometimes asleep only. Resistance to low-contrast, real-time, badly scanned, composite documents/writers or from aging medium seem to remain challenges, ...
Laurent Duval's user avatar
1 vote

Stochastic approximation algorithm

To do system identification using a driving function, it is necessary that the driving function $x[n]$ be broadbanded, meaning that the driving function has a Fourier Transform of non-zero value over ...
robert bristow-johnson's user avatar
1 vote
Accepted

Regression vector size for prediction, reconstruction and filtration with adaptive filters

Suppose that you are adapting $w$ to minimize $\text{E}(y[n]-w[n]*u[n])^2$ where $$y[n]=h[n]*u[n]+\nu[n]$$ $y[n]$ and $u[n]$ are known and $\nu[n]$ is an additive noise component. With a long enough ...
Hooman's user avatar
  • 321
1 vote

Upsampled input to an Adaptive filter?

Prior to upsampling, you have a white signal meaning every single frequency in the Nyquist bandwidth from $-\pi$ to $\pi$ is represented. This is a requirement to obtain an impulse (because the ...
hops's user avatar
  • 1,422
1 vote

How the Gain Term $ K \left( n \right) $ Is Derived? Why Is It Called Gain?

In all adaptive signal processing schemes, be it a Least Mean Squares (LMS), Recursive Least Squares (RLS) or a Kalman Filter, The fundamental concept is the update of some parameter: such as the ...
Fat32's user avatar
  • 28.1k
1 vote

What's the Difference Between LMS and Gradient Descent Adaptation?

I can add that LMS algorithm has a sample-based update.
Abdolvakil Fazli's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible