Skip to main content
Share Your Experience: Take the 2024 Developer Survey

Questions tagged [least-squares]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1 vote
0 answers
31 views

Python: tune Least Squares Support Vector Machine (LS-SVM) With Gride Search Optimization

Iam looking for LSSVM with GrideSearch optimization in python, but could not find it. Scikit learn has SVM with Gride search but not for LSSVM. Please could you help Thank you
novin's user avatar
  • 11
4 votes
1 answer
61 views

What are some good questions for a graduate level signal processing course?

I am currently taking an graduate-level Advanced Signal Processing class and I have a midterm soon. However, the midterm is not only open-book but it is also open-internet and untimed. Now I have no ...
Sherlock Rozman's user avatar
2 votes
2 answers
76 views

Fit Data Samples with a Robust Fit

I have a data from a sensor which the connection model of $x$ and $y$ is known: For instance, in the case above, the model is linear. The issue is how to handle outliers. Specifically when there are ...
Royi's user avatar
  • 19.7k
1 vote
0 answers
50 views

IIR filter design for "data-aided" least-squares filtering

Consider a discrete-time signal consisting of a desired signal component and noise, as follows. $$y_k=d_k + n_k, \qquad k\in\{0,1,\dots,K-1\}$$ This signal is applied to a linear time-invariant (LTI) ...
rhz's user avatar
  • 447
0 votes
0 answers
36 views

How can I implement simple robust regression?

I have many datasets like this: to which I am trying to fit lines: $$ y = a + bx $$ However as you can see there are some outliers in each dataset. The number varies from data set to dataset. I am ...
Andy's user avatar
  • 1,783
5 votes
1 answer
449 views

FIR filter design with nonlinear phase from measured amplitude and phase responses

I am having trouble when design FIR filter fitting to the complex data (i.e., amplitude and phase responses from measurements). I did try to use Matt. L's lslevin method here since this method is to ...
user190055's user avatar
1 vote
1 answer
95 views

Equalization with FSK

What are the downsides of using an LMS (or least squares) equalizer with FSK signals? What are the tradeoffs on doing equalization pre-demoduation vs post-demodulation? Most literature I can find ...
BigBrownBear00's user avatar
1 vote
1 answer
98 views

Numerical issues in scipy's Savitzky Golay filter coefficients for large polynomial order

Consider the design of a Savitzky-Golay filter of window length 101 and (high) polynomial order 20. Using scipy version 1.10.1, the filter coefficients can be obtained in python as: ...
rhz's user avatar
  • 447
0 votes
0 answers
41 views

Least squares filter with non-linear phase and independent weights for phase and magnitude

Intro My question is related to a previous one linked here. I am interested in non-linear phase FIR filters with a specific desired phase response. After I tried the options in the linked question I ...
Yair M's user avatar
  • 305
3 votes
1 answer
126 views

Fitting high order rational function to frequency response measurement data

I need to model measurement data of a frequency response with physically meaningful band-pass filters. All Measurements happening in die range of 20Hz to 20000Hz I have to work with python. The ...
LorenzNew's user avatar
0 votes
1 answer
139 views

Solving inverse problem using black box implementation of the kernel

My question is related to Solving regularized least squares problem using black-box computation of $\mathbf{A}\mathbf{x}$ and $\mathbf{A}^T\mathbf{x}$. In case, the problem is formulated as: \begin{...
Eric Johnson's user avatar
0 votes
0 answers
48 views

Jointly determining the weighting matrix and target vector in weighted least square

I have the same weighted least square form weighted least square composed of diagonal weighting matrix $ W $, IDFT matrix $F({\omega})$, desired response vector $D({\omega})$ and FIR Filter ...
simon Liao's user avatar
3 votes
1 answer
124 views

Solving regularized least squares problem using black-box computation of $\mathbf{A}\mathbf{x}$ and $\mathbf{A}^T\mathbf{x}$

Let $\mathbf{A} \in \mathbb{R}^{n \times n}$. I'm working in a problem where I have a black-box algorithmic solution to compute the products $\mathbf{A}\mathbf{x}$ and $\mathbf{A}^T \mathbf{x}$ given ...
mlbj's user avatar
  • 95
4 votes
1 answer
171 views

The Different Solutions for Filter Coefficients Estimation for Periodic Convolution and Full Convolution

As a continuation of the question Least Squares Solution Using the DFT vs Wiener Hopf Equations raised by Dan Boschen. The question is, given the model: $$ \boldsymbol{y} = \boldsymbol{h} * \...
Royi's user avatar
  • 19.7k
7 votes
1 answer
209 views

Noise leakage problem with least square estimation in the frequency-distance domain

I have data $d$ recorded from an antenna of sensors. These data are composed of a Gaussian noise $n$ and a signal $s$ which I try to estimate. This signal propagates on the antenna with frequency ...
User327201's user avatar
0 votes
1 answer
142 views

FIR filter for flipping the phase in negative phase

I have built up a FIR filter based on least square approach. I deploy the IDFT matrix $$F_{inv}$$ in above document page 7 and the desired response for compensated is H_F, the objective is $$ min_{...
simon Liao's user avatar
1 vote
0 answers
64 views

How do I apply Least Squares to determine the coefficients of a linear time-invariant filter having the desired output?

I'm trying to solve the following problem: Consider a linear time-invariant filter with linear phase: $$H(e^{j\omega}) = a_0 + \sum_{i = 1}^{N} 2a_k\cos(k\omega)$$ Determine the coefficients of this ...
Bertrando Del Poggetto's user avatar
7 votes
2 answers
663 views

Estimation of the Amplitude of a Sine / Cosine Wave and Its Independence of the SNR / Amplitude of the Wave

Consider a sinusoid in AWGN: $$Y = A \cos(\omega t+\phi)+n $$ Assume the frequency and phase are known. To estimate $A$ we can use least squares (which in this case is equivalent to the DFT): $\hat{A} ...
student1's user avatar
  • 394
0 votes
0 answers
39 views

Textbooks which derive largest usable rate for standard LMS filter?

Suppose $x$ is sampled from standard 0-centered Gaussian in d-dimensions, and I apply the following iteration. $$w \leftarrow w-\alpha x \langle w, x \rangle \tag 1$$ What is largest $\alpha$ such ...
Yaroslav Bulatov's user avatar
6 votes
2 answers
433 views

Constrained Least Squares Filter Design

I would like to design a complex FIR filter, $h$, for a known signal that produces a desired output: $d$ = $s*h$ (where $s$ is my signal and $d$ is the desired filter output). Let $S$ be the ...
Gillespie's user avatar
  • 1,797
2 votes
2 answers
182 views

What procedure to find the parameters of a given filter prototype to fit a desired frequency response?

To model the frequency response of a system, I am looking for a method to fit the response of cascaded bi-quad filters to the response of that system using an optimization algorithm. The $S$-domain ...
papaya's user avatar
  • 43
11 votes
2 answers
941 views

High Dynamic Range FIR Filters

Related to this question on using the Windowing method for FIR filter design versus the optimized algorithms such as least squares (firls in MATLAB, Octave and ...
Dan Boschen's user avatar
  • 51.9k
2 votes
1 answer
130 views

How to find point of inflexion of a digital signal?

Let's say I would like to find a point of inflexion of a digital signal which has been gathered via sampling of a continuous time domain signal with sampling period $100\,\mu s$. The digital signal ...
Steve's user avatar
  • 397
5 votes
2 answers
543 views

Noise Variance and Sampling in the Linear Regression Context

I am new to signal processing, so please bear with me. My question applies to any general problem of estimation from noisy measurements but I will like to understand this through a problem given here. ...
Zero's user avatar
  • 207
0 votes
0 answers
82 views

Impacted of a conjugated filter

What is the effect of using a conjugated FIR filter? I have a use case where a weight vector, x, can be estimated using the least squares approximation: $$Ax=b$$ $$ A^HAx = A^Hb$$ $$ x = (A^HA)^{-1} ...
BigBrownBear00's user avatar
2 votes
2 answers
73 views

Methods for time series estimation in time domain

I am trying to estimate the clean form of a time series, $u(t)$ that is corrupted by additive White Gaussian noise $w(t)$ at a particular SNR. The received signal is: $$y(t) = u(t) + w(t)$$ My first ...
Sm1's user avatar
  • 291
0 votes
0 answers
47 views

compressed sensing versus Lomb-Scargle

Say I have a signal that's the sum of only a few sine/cosine waves, and some noise, which has been measured at random times. I would like to find the frequencies of the waves. With this goal in-mind, ...
user3433489's user avatar
1 vote
2 answers
461 views

Projection Matrix derivation to constrained optimization problem with Lagrange multiplier

I am trying to derive famous Projection operator as a constrained minimization problem for the least square problem. The question is as follows: Find $x$ minimizing $ (y-x)^T(y-x)$ subject to $x = H\...
Osman Coskun's user avatar
4 votes
1 answer
307 views

Frequency-domain deconvolution: "Direct" filtering vs "Wiener" filtering

Can someone help with clarifying the difference between two approaches to frequency domain "deconvolution: For the frequency domain problem: We want to find a filter $F(\omega)$ which will ...
telemeister's user avatar
1 vote
0 answers
19 views

Apply window taper for selected window or to full data block in algebraic problem with known frequencies?

First, I'd like to introduce a problem. I try to solve a set of linear equations in LS (Least Squares) sense, i.e. Ax=y (LS) => A^TA x = A^T y. x is a vector I'm looking for, but y is a measured ...
Adam Ciesielski's user avatar
1 vote
0 answers
656 views

Recursive Least Square For Filtering

I just started doing research on Recursive Least Square for filtering noises such as sensors and dc motors noises. The only thing I've seen on the internet was Theoretical information about RLS but ...
tosbağa's user avatar
6 votes
1 answer
610 views

Image Restoration by Solving Constrained Least squares in Frequency Domain (Frequency Domain Filtering)

I am trying to implement the constrained least squares filtering as described in Rafael C. Gonzalez, Richard E. Woods - Digital Image Processing 3rd Edition Section 5.9. The equation (...
Maciek Woźniak's user avatar
2 votes
2 answers
130 views

Why should the last point be excluded when performing a least-squares fit of a periodic discrete time signal?

I fitted the function: f(t)=A_o+A_1 cos(wt)+B_1 sin(wt) to the following periodic discrete signal: ...
user53910's user avatar
1 vote
1 answer
102 views

The cost function of LS algorithm

I'm trying to understand the derivation of LS algorithm for channel estimation in OFDM. Could anyone explain why do we need complex conj transpose in the below equation?
dcs's user avatar
  • 45
6 votes
2 answers
150 views

Determine the Signal Curve from Parameters of a Power Curve by Noisy Measurement

I have a class of signals described by function: $$ f(inc,d,t)=inc\cdot t^d $$ where inc and d have a finite set of values like 1, 2, 3, i.e. $$ inc, d\in \left \{1,2,3 \right \} $$ and $$ 0\leq t<...
PSz's user avatar
  • 63
4 votes
1 answer
174 views

Questions on the Generalized Tikhonov Regularization

My first question is about the quadratic functional that is used in generalized Tikhonov regularization: $$\Psi(f)=\frac{1}{2}\|f\|^2_\Gamma=f^T\Gamma f.$$ In the above equation what does $\Gamma$ ...
Novak Djokovic's user avatar
3 votes
1 answer
67 views

Variational Regularization Method in Image Processing

I would like to understand better variational regularisation methods in image processing. In particular, the formulas in this image: Why formula (3.13)? In the notes that I read I cannot find ...
Novak Djokovic's user avatar
2 votes
1 answer
110 views

Difference equation system identification

Suppose I have a difference equation modeling a vehicle like this: $$d[k+1]=d[k]+a\cdot u[k]+b,\tag1\label{eq}$$ where $d[k]$ is total distance traveled at time $k$, $u[k]$ is engine input at time $k$ ...
Halleff's user avatar
  • 347
0 votes
0 answers
60 views

Unable to estimate for AR model using OLS, Yule Walker and MLE

I am learning estimation methods following the book of Steven Kay, "Fundamentals of Statistical Processing, Volume I: Estimation Theory " Theory says that if the measurement noise is ...
Sm1's user avatar
  • 291
1 vote
1 answer
63 views

How to find the value of regularization parameter/s when having one or multiple priors

I am working on a image reconstruction framework, specifically interferometry. In simple words we have data that is pertubated by noise, say $$ V^o_i = V_i + \sigma_i $$ Where $V^o$ is a complex ...
Miguel Cárcamo's user avatar
3 votes
1 answer
2k views

Building a FIR filter from an arbitrary frequency-magnitude response curve (eg. Least Squares fitting) [duplicate]

I'm trying to create a FIR filter from a magnitude equation, where my starting equation provides the magnitude (amplitude) between 0 and 1 for any given frequency in Hz. I posted the magnitude ...
mike's user avatar
  • 533
0 votes
2 answers
95 views

Simultaneously minimize and maximize two cost functions

I want to solve the following optimization problem, in which I have two cost functions $J_1 = \mathbf{w}^T R_1 \mathbf{w}$ and $J_2 = \mathbf{w}^T R_2 \mathbf{w}$, where $R_1$ and $R_2$ are covariance ...
user5045's user avatar
  • 331
6 votes
2 answers
778 views

Quadratic Programming with Linear Equality Constraints

I need to solve an equality constrained minimization problem as give below $$\min_{\textbf{w}} \mathbf{w}^TR\mathbf{w} $$ such that $$X\mathbf{w} = \mathbf{1}$$ where $R\in \mathbb{R}^{n\times n}$ is ...
user5045's user avatar
  • 331
0 votes
1 answer
71 views

Least Squares Filter Design: Deriving the Objective Function

I'm following the derivation in this paper A Comb Filter Design Using Fractional-Sample Delay to obtain the objective function for the least-squares filter design. N-order FIR filter: $H(z) = \sum_{...
Sndn's user avatar
  • 113
2 votes
0 answers
27 views

Solving Sparse Model with given Dictionary Using LASSO

I was trying to solve a problem where the basis matrix contains the components of $\sin(nx)$, $\cos(nx)$, $\sinh(nx)$ and $\cosh(nx)$. Say the $n$ varies from 1 to 100. While solving the lasso linear ...
Debasish Jana's user avatar
1 vote
0 answers
34 views

How to find the solution of nonlinear least square (NLS) equation for freqeuncy estimation

I am struggling to understand the theory of parametric methods for power spectral estimations of line spectra. And I want to find the solution of nonlinear least square (NLS) method for estimating ...
JH Lee's user avatar
  • 11
1 vote
2 answers
1k views

Solve Undetermined Linear System Using NumPy's `lstsq()` Function

I would like to understand what I am doing wrong here. I am trying to perform polynomial regression by minimizing the least squares, $||Au-y||^2$, where $y$ is the given data and $A$ is the matrix ...
reloh100's user avatar
5 votes
1 answer
452 views

Use Recursive Least Square as an "Actual" Filter?

I am currently studying recursive least square (RLS) and as far as I understand, the setup is (for example) that we are given a process with uncertain parameters/noise and we want to adapt parameters ...
SampleTime's user avatar
1 vote
1 answer
98 views

Why is the concept of a "state covariance matrix" necessary in estimation?

I'm currently taking a course in optimal estimation (and it's still very early in the course). Much of our work is based around the idea of a measurement model $y=Hx + v$ This model assumes our ...
Izzo's user avatar
  • 872
5 votes
1 answer
388 views

Jacobian Computation in Least Squares IIR Filter Design

Long time lurker and first time poster - but unfortunately I haven't had any joy untangling this on my own. I've been studying Mathias Lang's thesis, Algorithms for the Constrained Design of Digital ...
Nick M's user avatar
  • 73