Skip to main content
OverflowAI is here! AI power for your Stack Overflow for Teams knowledge community. Learn more
20 votes
Accepted

FIR Filter Design: Window vs Parks McClellan and Least Squares

I agree that the windowing filter design method is not one of the most important design methods anymore, and it might indeed be the case that it is overrepresented in traditional textbooks, probably ...
Matt L.'s user avatar
  • 90.3k
10 votes
Accepted

High Dynamic Range FIR Filters

Like @MattL. and @aconcernedcitizen say, the issue is numerical. Python's scipy.signal.firls uses internally the solver ...
Olli Niemitalo's user avatar
9 votes
Accepted

Jacobian Computation in Least Squares IIR Filter Design

The Jacobian is not computed numerically but analytically and then just evaluated. The frequency response of the IIR filter is $$H(e^{j\omega})=\frac{b_0+b_1e^{-j\omega}+\ldots+b_Me^{-jM\omega}}{1+...
Matt L.'s user avatar
  • 90.3k
8 votes
Accepted

What's the Difference Between LMS and Gradient Descent Adaptation?

The LMS algorithm is based on the idea of gradient descent to search for the optimal (minimum error) condition, with a cost function equal to the mean squared error at the filter output. However, it ...
Jason R's user avatar
  • 24.6k
7 votes

FIR Filter Design: Window vs Parks McClellan and Least Squares

I'll show here one benefit of a windowed design and a trick to get the same benefit from Parks–McClellan. For half-band, quarter-band etc. filters windowing retains the time-domain zeros of the ...
Olli Niemitalo's user avatar
7 votes

FIR Filter Design: Window vs Parks McClellan and Least Squares

Windowed Sinc filters can be adaptively generated on the fly on processors barely powerful enough to run the associated FIR filter. Windowed Sinc filters can be generated in finite bounded time. The ...
hotpaw2's user avatar
  • 35.4k
7 votes
Accepted

Sequential Form of the Least Squares Estimator for Linear Least Squares Model

Slope from all samples obtained To summarize the question's problem, you want to calculate the slope based on all samples obtained thus far, and as new samples are obtained, update the slope without ...
Olli Niemitalo's user avatar
7 votes

High Dynamic Range FIR Filters

The problem lies in the formulation of the desired response, and especially in the "don't care" region, which is extremely wide for the chosen filter length. Even though I can't give any ...
Matt L.'s user avatar
  • 90.3k
6 votes

Least Mean Squares (LMS) Filter Weight Update

To expand on what Peter K. has said, if the signals being used by the filter are stationary, then the filter weights or coefficients can be determined and the filter operates as it was designed ...
Michael_RW's user avatar
6 votes

Zero Phase Filter: Determining Initial Conditions for Forward Backward Filtering

For anyone who is interested, i coincidentally found a paper describing the method implemented in matlab's filtfilt.m. A link to the paper is attached. At least to my understanding matlab's filtfilt.m ...
user967493's user avatar
6 votes

Least Mean Squares (LMS) Filter Weight Update

That really depends on context, but generally adaptive implies that the calculations are done on-line / on the fly. In some applications, the filter is updated for a while, then the adaptation is ...
Peter K.'s user avatar
  • 25.8k
6 votes
Accepted

Estimation of the Amplitude of a Sine / Cosine Wave and Its Independence of the SNR / Amplitude of the Wave

One may have a look at the CRLB of estimating the parameters of a sine wave. The model for signal is given by: $$ x \left[ n \right] = A \cos \left( \omega n + \phi \right) + w \left[ n \right], \; n =...
Royi's user avatar
  • 19.7k
6 votes
Accepted

FIR filter design with nonlinear phase from measured amplitude and phase responses

The algorithm gives you the best least squares approximation possible for a causal filter with the specified filter order and the given desired frequency response. The problem with your specification ...
Matt L.'s user avatar
  • 90.3k
5 votes
Accepted

Quadratic Programming with Linear Equality Constraints

Let's solve a more general problem (Least Squares with Linear Equality Constraints): $$ \begin{alignat*}{3} \arg \min_{x} & \quad & \frac{1}{2} \left\| A x - b \right\|_{2}^{2} \\ \text{...
Royi's user avatar
  • 19.7k
4 votes

What Is the Relationship Between a Kalman Filter and Polynomial Regression?

I suggest this reference regarding the comparison between least-squares and Kalman filters : Fundamentals of Kalman Filtering: A Practical Approach by P. Zarchan & H. Mussof Especially Chapter 3 ...
Florian D's user avatar
4 votes

What Is the Relationship Between a Kalman Filter and Polynomial Regression?

A lot has been said already, allow me to add some comments: Kalman filters are an application of Bayesian probability theory, which means that "a priori information" or "prior uncertainty" can (and ...
Bart Van Hove's user avatar
4 votes
Accepted

Difference between Leaking Factor and Forgetting Factor

The play similar role in those algorithms - the ability to forget the past and adapt to current reality. In the LMS, the classic implementation has $ \alpha = 1 $. Namely the optimal weights at any ...
Royi's user avatar
  • 19.7k
4 votes

Frequency-domain deconvolution: "Direct" filtering vs "Wiener" filtering

The direct approach is noise sensitive and the second one (which is also known as $H_1$ estimator) is somehow noise resistant. $H_1$ estimator assumes that there is no noise at the input signal, $B(\...
ZR Han's user avatar
  • 3,248
3 votes

FIR Filter Design: Window vs Parks McClellan and Least Squares

Because of the comment, I obviously must have seen this question nearly 5 years ago, but I don't remember it really. But one advantage that windowed-sinc has over P-McC or LS for a brick-wall ...
robert bristow-johnson's user avatar
3 votes
Accepted

MATLAB: Implementing Least Squares Estimator for a Given Model

The equation you're trying to solve is $$ \mathbf{y}=\mathbf{X}\mathbf{h}, $$ where $\mathbf{h}$ is your unknown. The matrix $\mathbf{X}$ is going to have a time-shifted structure that reflects the ...
David's user avatar
  • 2,871
3 votes
Accepted

How to apply least-squares estimation for sparse coefficient estimation?

I am not entirely sure what Matlab's LASSO routine does so I started with Ordinary Least Squares (OLS) and worked backwards. From an OLS perspective X1 as you have it won't work. You've got a ...
Chad Sexington's user avatar
3 votes

Estimating Damping Factor (Q) from Noisy Measurements

Our system has the impulse response (why we changed to imaginary instead of real? :) ): $$h(t)=\mathbb{Im}(Ae(iwt)e(-bt))=Ae^{-bt}sin(wt)$$ With the following structure (ref.) (discretized): $$H(s)=\...
Brethlosze's user avatar
  • 1,430
3 votes
Accepted

How to Use the DFT (FFT) to Solve a Least Squares Regularization Problem (Inverse Problem)?

The question really depends on $ f \left( \cdot \right) $. Yet in order to show how to use FFT we can even use 1D signals. Let's rewrite the problem: $$ \hat{x} = \arg \min_{x} \frac{1}{2} \left\| K ...
Royi's user avatar
  • 19.7k
3 votes
Accepted

Least Squares Linear Phase FIR Filter Design

I think that the term "constraints" is not a very fortunate choice in this context, but what is meant is the number of frequency points that are specified: $$H(\omega_i)\stackrel{!}{=}D(\omega_i),\...
Matt L.'s user avatar
  • 90.3k
3 votes

Solving LASSO ($ {L}_{1} $ Regularized Least Squares) with Gradient Descent

It can easily solved by the Gradient Descent Framework with one adjustment in order to take care of the $ {L}_{1} $ norm term. Since the $ {L}_{1} $ norm isn't smooth you need to use the concept of ...
Royi's user avatar
  • 19.7k
3 votes
Accepted

What Is the Definition of Linear Predictive Coefficients When the Optimal Value Aren't Unique?

Constant zero sequence is the only finite sequence for which $a_i$ are not uniquely defined. Indeed, $a_i$ are a solution to the equation $$ \begin{bmatrix} R_0 && R_1 && \ldots &...
seed's user avatar
  • 151
3 votes
Accepted

Design of FIR Filters with Arbitrary Magnitude and Phase Responses

For the sake of clarity, let me point out that you do not approximate $(1)$ by $(2)$, but the other way around: you approximate a desired frequency response, i.e., your specification, by the filter's ...
Matt L.'s user avatar
  • 90.3k
3 votes

Sequential Form of the Least Squares Estimator for Linear Least Squares Model

There are really great answers. I will try to give the Sequential Least Squares approach which generalizes to any Linear Model. Sequential Least Squares Model We're after solving the Linear Least ...
Royi's user avatar
  • 19.7k

Only top scored, non community-wiki answers of a minimum length are eligible