Questions tagged [least-squares]
The least-squares tag has no usage guidance.
125
questions
5
votes
1
answer
362
views
FIR filter design with nonlinear phase from measured amplitude and phase responses
I am having trouble when design FIR filter fitting to the complex data (i.e., amplitude and phase responses from measurements). I did try to use Matt. L's lslevin method here since this method is to ...
- 89
1
vote
1
answer
69
views
Equalization with FSK
What are the downsides of using an LMS (or least squares) equalizer with FSK signals? What are the tradeoffs on doing equalization pre-demoduation vs post-demodulation?
Most literature I can find ...
- 521
0
votes
1
answer
71
views
Numerical issues in scipy's Savitzky Golay filter coefficients for large polynomial order
Consider the design of a Savitzky-Golay filter of window length 101 and (high) polynomial order 20. Using scipy version 1.10.1, the filter coefficients can be obtained in python as:
...
- 375
0
votes
0
answers
34
views
Least squares filter with non-linear phase and independent weights for phase and magnitude
Intro
My question is related to a previous one linked here.
I am interested in non-linear phase FIR filters with a specific desired phase response.
After I tried the options in the linked question I ...
- 281
3
votes
1
answer
102
views
Fitting high order rational function to frequency response measurement data
I need to model measurement data of a frequency response with physically meaningful band-pass filters.
All Measurements happening in die range of 20Hz to 20000Hz
I have to work with python.
The ...
- 31
0
votes
1
answer
117
views
Solving inverse problem using black box implementation of the kernel
My question is related to Solving regularized least squares problem using black-box computation of $\mathbf{A}\mathbf{x}$ and $\mathbf{A}^T\mathbf{x}$.
In case, the problem is formulated as:
\begin{...
- 326
0
votes
0
answers
47
views
Jointly determining the weighting matrix and target vector in weighted least square
I have the same weighted least square form weighted least square composed of diagonal weighting matrix $ W $, IDFT matrix $F({\omega})$, desired response vector $D({\omega})$ and FIR Filter ...
3
votes
1
answer
117
views
Solving regularized least squares problem using black-box computation of $\mathbf{A}\mathbf{x}$ and $\mathbf{A}^T\mathbf{x}$
Let $\mathbf{A} \in \mathbb{R}^{n \times n}$. I'm working in a problem where I have a black-box algorithmic solution to compute the products $\mathbf{A}\mathbf{x}$ and $\mathbf{A}^T \mathbf{x}$ given ...
- 95
4
votes
1
answer
153
views
The Different Solutions for Filter Coefficients Estimation for Periodic Convolution and Full Convolution
As a continuation of the question Least Squares Solution Using the DFT vs Wiener Hopf Equations raised by Dan Boschen.
The question is, given the model:
$$ \boldsymbol{y} = \boldsymbol{h} * \...
- 19.3k
7
votes
1
answer
196
views
Noise leakage problem with least square estimation in the frequency-distance domain
I have data $d$ recorded from an antenna of sensors. These data are composed of a Gaussian noise $n$ and a signal $s$ which I try to estimate. This signal propagates on the antenna with frequency ...
- 85
0
votes
1
answer
105
views
FIR filter for flipping the phase in negative phase
I have built up a FIR filter based on least square approach.
I deploy the IDFT matrix $$F_{inv}$$ in above document page 7 and the desired response for compensated is H_F, the objective is
$$
min_{...
1
vote
0
answers
63
views
How do I apply Least Squares to determine the coefficients of a linear time-invariant filter having the desired output?
I'm trying to solve the following problem:
Consider a linear time-invariant filter with linear phase:
$$H(e^{j\omega}) = a_0 + \sum_{i = 1}^{N} 2a_k\cos(k\omega)$$
Determine the coefficients of this ...
7
votes
2
answers
567
views
Estimation of the Amplitude of a Sine / Cosine Wave and Its Independence of the SNR / Amplitude of the Wave
Consider a sinusoid in AWGN:
$$Y = A \cos(\omega t+\phi)+n $$
Assume the frequency and phase are known. To estimate $A$ we can use least squares (which in this case is equivalent to the DFT):
$\hat{A} ...
- 384
0
votes
0
answers
37
views
Textbooks which derive largest usable rate for standard LMS filter?
Suppose $x$ is sampled from standard 0-centered Gaussian in d-dimensions, and I apply the following iteration.
$$w \leftarrow w-\alpha x \langle w, x \rangle \tag 1$$
What is largest $\alpha$ such ...
- 211
6
votes
2
answers
364
views
Constrained Least Squares Filter Design
I would like to design a complex FIR filter, $h$, for a known signal that produces a desired output: $d$ = $s*h$ (where $s$ is my signal and $d$ is the desired filter output). Let $S$ be the ...
- 1,642
2
votes
2
answers
152
views
What procedure to find the parameters of a given filter prototype to fit a desired frequency response?
To model the frequency response of a system, I am looking for a method to fit the response of cascaded bi-quad filters to the response of that system using an optimization algorithm. The $S$-domain ...
- 43
11
votes
2
answers
835
views
High Dynamic Range FIR Filters
Related to this question on using the Windowing method for FIR filter design versus the optimized algorithms such as least squares (firls in MATLAB, Octave and ...
- 48.8k
2
votes
1
answer
106
views
How to find point of inflexion of a digital signal?
Let's say I would like to find a point of inflexion of a digital signal which has been gathered via sampling of a continuous time domain signal with sampling period $100\,\mu s$. The digital signal ...
- 375
5
votes
2
answers
376
views
Noise Variance and Sampling in the Linear Regression Context
I am new to signal processing, so please bear with me. My question applies to any general problem of estimation from noisy measurements but I will like to understand this through a problem given here.
...
- 207
0
votes
0
answers
82
views
Impacted of a conjugated filter
What is the effect of using a conjugated FIR filter? I have a use case where a weight vector, x, can be estimated using the least squares approximation:
$$Ax=b$$ $$ A^HAx = A^Hb$$ $$ x = (A^HA)^{-1} ...
- 521
2
votes
2
answers
66
views
Methods for time series estimation in time domain
I am trying to estimate the clean form of a time series, $u(t)$ that is corrupted by additive White Gaussian noise $w(t)$ at a particular SNR. The received signal is:
$$y(t) = u(t) + w(t)$$
My first ...
- 291
0
votes
0
answers
43
views
compressed sensing versus Lomb-Scargle
Say I have a signal that's the sum of only a few sine/cosine waves, and some noise, which has been measured at random times. I would like to find the frequencies of the waves. With this goal in-mind, ...
- 101
1
vote
2
answers
362
views
Projection Matrix derivation to constrained optimization problem with Lagrange multiplier
I am trying to derive famous Projection operator as a constrained minimization problem for the least square problem. The question is as follows:
Find $x$ minimizing $ (y-x)^T(y-x)$ subject to $x = H\...
- 13
4
votes
1
answer
270
views
Frequency-domain deconvolution: "Direct" filtering vs "Wiener" filtering
Can someone help with clarifying the difference between two approaches
to frequency domain "deconvolution:
For the frequency domain problem:
We want to find a filter $F(\omega)$ which will ...
- 141
1
vote
0
answers
19
views
Apply window taper for selected window or to full data block in algebraic problem with known frequencies?
First, I'd like to introduce a problem. I try to solve a set of linear equations in LS (Least Squares) sense, i.e. Ax=y (LS) => A^TA x = A^T y.
x is a vector I'm looking for, but y is a measured ...
1
vote
0
answers
600
views
Recursive Least Square For Filtering
I just started doing research on Recursive Least Square for filtering noises such as sensors and dc motors noises. The only thing I've seen on the internet was Theoretical information about RLS but ...
- 11
6
votes
1
answer
526
views
Image Restoration by Solving Constrained Least squares in Frequency Domain (Frequency Domain Filtering)
I am trying to implement the constrained least squares filtering as described in Rafael C. Gonzalez, Richard E. Woods - Digital Image Processing 3rd Edition Section 5.9. The equation (...
- 163
2
votes
2
answers
123
views
Why should the last point be excluded when performing a least-squares fit of a periodic discrete time signal?
I fitted the function: f(t)=A_o+A_1 cos(wt)+B_1 sin(wt) to the following periodic discrete signal:
...
- 21
1
vote
1
answer
94
views
The cost function of LS algorithm
I'm trying to understand the derivation of LS algorithm for channel estimation in OFDM.
Could anyone explain why do we need complex conj transpose in the below equation?
- 45
6
votes
2
answers
146
views
Determine the Signal Curve from Parameters of a Power Curve by Noisy Measurement
I have a class of signals described by function:
$$
f(inc,d,t)=inc\cdot t^d
$$
where inc and d have a finite set of values like 1, 2, 3, i.e.
$$
inc, d\in \left \{1,2,3 \right \}
$$
and
$$
0\leq t<...
- 63
4
votes
1
answer
148
views
Questions on the Generalized Tikhonov Regularization
My first question is about the quadratic functional that is used in generalized Tikhonov regularization:
$$\Psi(f)=\frac{1}{2}\|f\|^2_\Gamma=f^T\Gamma f.$$
In the above equation what does $\Gamma$ ...
- 269
3
votes
1
answer
65
views
Variational Regularization Method in Image Processing
I would like to understand better variational regularisation methods in image processing. In particular, the formulas in this image:
Why formula (3.13)? In the notes that I read I cannot find ...
- 269
2
votes
1
answer
105
views
Difference equation system identification
Suppose I have a difference equation modeling a vehicle like this:
$$d[k+1]=d[k]+a\cdot u[k]+b,\tag1\label{eq}$$
where $d[k]$ is total distance traveled at time $k$, $u[k]$ is engine input at time $k$ ...
- 325
0
votes
0
answers
53
views
Unable to estimate for AR model using OLS, Yule Walker and MLE
I am learning estimation methods following the book of Steven Kay, "Fundamentals of Statistical Processing, Volume I: Estimation Theory "
Theory says that if the measurement noise is ...
- 291
1
vote
1
answer
60
views
How to find the value of regularization parameter/s when having one or multiple priors
I am working on a image reconstruction framework, specifically interferometry. In simple words we have data that is pertubated by noise, say
$$
V^o_i = V_i + \sigma_i
$$
Where $V^o$ is a complex ...
2
votes
1
answer
2k
views
Building a FIR filter from an arbitrary frequency-magnitude response curve (eg. Least Squares fitting) [duplicate]
I'm trying to create a FIR filter from a magnitude equation, where my starting equation provides the magnitude (amplitude) between 0 and 1 for any given frequency in Hz.
I posted the magnitude ...
- 447
0
votes
2
answers
88
views
Simultaneously minimize and maximize two cost functions
I want to solve the following optimization problem, in which I have two cost functions $J_1 = \mathbf{w}^T R_1 \mathbf{w}$ and $J_2 = \mathbf{w}^T R_2 \mathbf{w}$, where $R_1$ and $R_2$ are covariance ...
- 331
6
votes
2
answers
646
views
Quadratic Programming with Linear Equality Constraints
I need to solve an equality constrained minimization problem as give below
$$\min_{\textbf{w}} \mathbf{w}^TR\mathbf{w} $$
such that
$$X\mathbf{w} = \mathbf{1}$$
where $R\in \mathbb{R}^{n\times n}$ is ...
- 331
0
votes
1
answer
61
views
Least Squares Filter Design: Deriving the Objective Function
I'm following the derivation in this paper A Comb Filter Design Using Fractional-Sample Delay to obtain the objective function for the least-squares filter design.
N-order FIR filter:
$H(z) = \sum_{...
- 113
2
votes
0
answers
26
views
Solving Sparse Model with given Dictionary Using LASSO
I was trying to solve a problem where the basis matrix contains the components of $\sin(nx)$, $\cos(nx)$, $\sinh(nx)$ and $\cosh(nx)$.
Say the $n$ varies from 1 to 100. While solving the lasso linear ...
- 301
1
vote
0
answers
33
views
How to find the solution of nonlinear least square (NLS) equation for freqeuncy estimation
I am struggling to understand the theory of parametric methods for power spectral estimations of line spectra.
And I want to find the solution of nonlinear least square (NLS) method for estimating ...
- 11
1
vote
2
answers
1k
views
Solve Undetermined Linear System Using NumPy's `lstsq()` Function
I would like to understand what I am doing wrong here.
I am trying to perform polynomial regression by minimizing the least squares, $||Au-y||^2$, where $y$ is the given data and $A$ is the matrix ...
- 31
5
votes
1
answer
344
views
Use Recursive Least Square as an "Actual" Filter?
I am currently studying recursive least square (RLS) and as far as I understand, the setup is (for example) that we are given a process with uncertain parameters/noise and we want to adapt parameters ...
- 150
1
vote
1
answer
82
views
Why is the concept of a "state covariance matrix" necessary in estimation?
I'm currently taking a course in optimal estimation (and it's still very early in the course).
Much of our work is based around the idea of a measurement model
$y=Hx + v$
This model assumes our ...
- 793
5
votes
1
answer
356
views
Jacobian Computation in Least Squares IIR Filter Design
Long time lurker and first time poster - but unfortunately I haven't had any joy untangling this on my own.
I've been studying Mathias Lang's thesis, Algorithms for the Constrained Design of Digital ...
- 73
4
votes
2
answers
334
views
System Identification with a Limited Bandwidth Input Signal and Region of Interest
Given a FIR filter $h[n]$. Its action can described as:
$$
\mathbf{y} = \mathbf{H} \mathbf{x} \\
\mathbf{y} = \mathbf{X} \mathbf{h}
$$
where $\mathbf{H}$ and $\mathbf{X}$ is a Toeplitz matrix.
If $h$...
- 473
1
vote
0
answers
138
views
Difference Between a 1st Order SG Filter And a Least Squares Moving Average
I have been studying SG filters and i recently found another filter which seem to be commonly used in financial data smoothing which is the least-squares moving average, this filter is also called ...
- 225
4
votes
1
answer
841
views
Learning the Coefficients of Auto Regressive (AR) Model Using Least Mean Squares (LMS) Filter for Signal Prediction
I want to do two things.
Estimating Coefficients of AR model using LMS
Using Coefficients found in step 1 and predict future samples of a signal using AR equation.
I don't have a desired signal so I ...
- 87
1
vote
1
answer
181
views
Range Estimation Based on Distance Differences
It is a range estimation problem, as shown in the figure.
The blue circles are the anchor node' positions, which are produced by one moving anchor (the moving direction is shown with black arrow). ...
- 25
0
votes
0
answers
156
views
Non Linear Recursive / Online Least Squares
I am searching for a recursive or online non linear least squares algorithm. I want to spread the computation out as new data is sampled like in the linear Recursive Least Squares or the LMS.
...
- 1