Questions tagged [least-squares]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
21
votes
3answers
4k views

FIR Filter Design: Window vs Parks McClellan and Least Squares

Are there any advantages to use a window approach over Parks-McClellan (further abbreviated here as PMcC) or Least Squares algorithms for FIR filter design of a low pass filter? Assume with today's ...
16
votes
1answer
1k views

What Does Make an Error Surface Convex? Is It Determined by the Covarinace Matrix or the Hessian?

I am currently learning about least-squares (and other) estimations for regression, and from what I am also reading in some adaptive algorithm literatures, often times the phrase "... and since the ...
14
votes
1answer
713 views

How to Improve Least Means Squares (LMS) / NLMS Filter Performance?

Are there ways to increase computational performance of a normalized least squares (NLMS) filter? Multidelay block frequency-domain (MDF) filters have been proposed to do this, but they also take away ...
8
votes
2answers
1k views

Is the Kalman Filter a Best Linear Unbiased Estimator (BLUE) for Heteroscedastic Noise?

According to the Gauss-Markov Theorem, a ordinary least squares estimator is BLUE if the noise entering a system is uncorrelated with zero mean and is homoscedastic (has a constant finite variance). I ...
7
votes
2answers
807 views

Zero Phase Filter: Determining Initial Conditions for Forward Backward Filtering

Is anybody familiar with Gustafson's algorithm for minimizing transients in forward backward filtering [1]? I'm trying to implement it and my first guess was to check Matlab's filtfilt.m, since they ...
5
votes
1answer
4k views

What's the Difference Between LMS and Gradient Descent Adaptation?

I found algorithms that seems the same to me, but they are described with different names (in field of adaptive filtering). For example: LMS - least-mean-squares seems to be GD - stochastic gradient ...
4
votes
2answers
3k views

Estimate the Transfer Function of an Unknown System

Suppose you have a system, H, that you want to estimate its transfer function. You have a finite number of complex input samples, x, and noisy complex (magnitude and phase) output samples, y: In ...
4
votes
6answers
355 views

The Least Norm Solution of Under Determined Linear System

Suppose I have a matrix $$A= \begin{pmatrix} 1 & 0 & 1 & 0\\ 0 & 1 & 1& 0\\ \end{pmatrix} $$ where the variables are channel information like assume $X_1$, $X_2$, $X_3$ ...
4
votes
3answers
6k views

How to Remove the Periodic Oscillations from a Signal

The task that I have is to remove the annual and semiannual oscillation from a set of temperature measurements, taken over several years, by means of least squares method. I found the method ...
4
votes
1answer
259 views

Direct Correlation (DC) Time Delay Estimation: Variance Keeps Decreasing for Increasing SNR?

I'm trying to reproduce the results from this paper "Discrete time techniques for Time Delay Estimation" doi:10.1109/78.193195, for both the correlation and Least Squares. I've generated a random (...
4
votes
1answer
531 views

What Is the Difference between RLS, LMS and Wiener Filter? When Is One Preferred Over Another?

I'm dealing with a channel equalization problem where the channel is modeled as a WSS process. I understand LMS utilities a Wiener-like approach, ie it converges to the optimal (wiener) solution. I ...
4
votes
1answer
1k views

Constant Modulus Algorithm and the Gradient Operation

CMA is a blind channel equalization algorithm with the details presented above. I am facing 3 difficulties and shall appreciate help Q1: Does $H$ and the bar over $\bar{y_k}$ represent the Transpose ...
3
votes
2answers
10k views

Difference Between Equiripple & Least Squares Design for FIR Digital Filters

For an efficient and optimized digital FIR filter design, there are two methods available broadly, Equiripple filter design & Least Squares filter design. A general method for designing a filter ...
3
votes
1answer
86 views

Jacobian Computation in Least Squares IIR Filter Design

Long time lurker and first time poster - but unfortunately I haven't had any joy untangling this on my own. I've been studying Mathias Lang's thesis, Algorithms for the Constrained Design of Digital ...
3
votes
2answers
256 views

How to Combine / Fuse 2 Least Squares Estimates?

Say you want to compute the least squares estimate of $w$ from a data-set: $$ \begin{bmatrix}d_1 \\d_2 \\\vdots\\d_N \end{bmatrix} =\begin{bmatrix} x_1 \\x_2 \\ \vdots \\x_N\end{bmatrix}w + \begin{...
3
votes
3answers
386 views

Force Linear Phase for a FIR Filter Synthesized Using Berchin's FDLS?

As a follow-on to this post, how would you force linear phase for a FIR filter you synthesized using Berchin's FDLS?
3
votes
1answer
2k views

Python: Least Squares Support Vector Machine (LS-SVM)

I'm looking for a Python package for a LS-SVM or a way to tune a normal SVM from scikit-learn to a Least-Squares Support Vector Machine for a classification problem. The goal of a SVM is to maximize ...
3
votes
1answer
2k views

Use MATLAB to Restore a Signal from a Given Degraded Signal Using Tikhonov Regularization

Anyone could share how to develop an application in MATLAB to restore the signal from a given degraded signal using Tikhonov regularization i.e restoring the signal $f$ via solving $$ \min || g - f ...
3
votes
1answer
197 views

Least Angle Regression (LARS) without Matrix Inversion

Sorry if this is too damned long. I did what I could to abbreviate it. The question is about Least Angle Regression (LARS). I'm new to numerical work with matrices. I believe I have a way to ...
3
votes
0answers
226 views

Least Squares Approximation for FIR Filter Design

Following this paper , I am trying to make a least-squares algorithm in MATLAB, but for type I (I know about firls()). ...
2
votes
3answers
250 views

Least Mean Squares (LMS) Filter Weight Update

I have a general question regarding Least mean squares adaptive filters. Using the example of noise cancellation, I understand that if you have a set of reference signals (S) and corrupted signals (S+...
2
votes
4answers
303 views

Sequential Form of the Least Squares Estimator for Linear Least Squares Model

I'm currently working on a project in which I need to find the tilt of a surface. Let's assume I'm only concerned with a single dimension tilt (i.e. slope) to begin. I currently have the ability to ...
2
votes
2answers
160 views

Least Square Error Estimation: Conditions for $ (A^TA)^{ -1} = A^{-1}(A^T)^{-1} $?

I am new to linear algebra and have this simple question... in least sqaure estimation...the best estimation of the equation $Ax = b$ is $x_{Estimated} = A (A^t A )^{-1} A^t b$...the projection of $b$ ...
2
votes
2answers
239 views

Questions on Weighted Least Square Estimation

This is a page from the book linear algebra,geodesy and gps by Gilbert Strang.... the page explains about the justification of the inverse of the of the co variance matrix of measurement vector $b$ in ...
2
votes
1answer
105 views

What is the Technique to Find Variance of Estimation Error

Given an $n$-vector $y$ (responses) and a design matrix $X$, I wish to fit them with a simple linear regression model $$y=X\beta+e,$$ or, $y_t = x_t'\beta_0 + e_t$ where $e\sim\mathcal{N}(0, \sigma^...
2
votes
1answer
314 views

How to Use the DFT (FFT) to Solve a Least Squares Regularization Problem (Inverse Problem)?

Let $X$ and $K$ be an image and a Point Spread Function (PSF), respectively. The blur image $B$ is obtained as follows $$B = X * K$$ I want to solve the following general regularization problem $$...
2
votes
2answers
166 views

Finding the Best Gaussian Smoothing Kernel to Minimize the Discrepancy Between Two Images

Suppose we have two grayscale images, $A$ and $B$. $A$ and $B$ very strongly resemble each other, such that the mean of the absolute difference $\lvert A - B\rvert$ is fairly small. Suppose further ...
2
votes
0answers
290 views

Motion Artifact Cancellation from ECG Signal Using 3 Axis Accelerometer Data

I have a ECG signal measured as a test subject is performing 3 different activities - sitting, walking(2mph) and jogging(5mph) and also the corresponding 3-axis accelerometer(X,Y and Z) signals. I can ...
1
vote
3answers
148 views

Complex Least Squares Approximation

In the case of frequency domain FIR filter design, the error function given by : $$E(\omega)=H(e^{j\omega})-D(e^{j\omega}) \tag{1}$$ is a linear function with respect to the unknown filter ...
1
vote
2answers
62 views

What Is the Definition of Linear Predictive Coefficients When the Optimal Value Aren't Unique?

Linear predictive coefficients of signal y are defined as the best $k$ coefficients $a_i, i = 1, \ldots, k$, that will approximate $y_n$ by $-\sum_{i=1}^k{a_iy_{n-i}}$. (Best approximation is that ...
1
vote
2answers
105 views

MATLAB: Implementing Least Squares Estimator for a Given Model

The formula to estimate $\mathbf{h}$ is then $$\hat{\mathbf{h}} = (X^T X)^{-1} X^T \vec{y}\tag{2}$$ I think this can be implemented in Matlab using ...
1
vote
1answer
101 views

Using MATLAB Function `mpiir_l2()`: Results Are Not IIR but FIR Filters - Why and How to Avoid This?

I'm using mpiir_l2() from user Matt L's PhD thesis to design IIR filters. I set the number of numerator and denominator coefficients both to the same value (between ...
1
vote
2answers
324 views

Least Squares Fitting to Inverse Exponential Function

I have a time series of measurements that resembles the shape of an exponential function. The samples are a bit noisy and sometimes there is a weak sine like ripple signal ontop of it. Simplified the ...
1
vote
1answer
244 views

Least Squares Linear Phase FIR Filter Design

In explication ''the geometric interpretation of least squares'' Typically, the number of frequency constraints is much greater than the number of design variables (filter coefficients). In these ...
1
vote
2answers
110 views

Solving a Linear Mean Square Estimation the Easy Way

I have an exercise which is quite trivial. However I got stuck and I'm not sure if this the end-result. I assume there has to be a way to get this result much quicker. Given are two randomly ...
1
vote
1answer
129 views

Obtaining Correct (Least Squares Sense) Affine Transform Parameters Between Two Images

I have two images that I want to compute the affine motion model parameters. The model that I use is $$x' = a_1x+a_2y+a_3$$ $$y' = a_4x+a_5y+a_6$$ To calculate those 6 parameters, I picked 6 points (...
1
vote
1answer
159 views

RLS Algorithm (Memoryless)

I have been studying the adaptive filters lately and now when I am at RLS (Recursive Least Squar) Algorithm I came across the term used in the weighting function of the RLS called forgetting factor ($\...
1
vote
1answer
80 views

Deriving the Matrix Inversion Lemma for RLS Equations vs the Woodbury Derivation

Can any one help me in deriving the matrix inversion lemma rule for RLS algorithm? I don't know how to start with. Many books have just stated but they haven't derived it.
1
vote
3answers
626 views

Least Squares and Auto and Cross Correlation

I am trying to understand why auto and cross correlation helps find the best fit line in least squares. I have an equation as stated here: $Ax=b$ -- I have not exact solution, so I use the least ...
1
vote
2answers
286 views

Conceptual Question on Least Squares Estimation Method

What is meant by Least square plus Gaussian method of estimating the parameters of an unknown system? I am aware of least squares in general but some works refer it as least square+gaussian method. ...
1
vote
1answer
94 views

Equivalence of Maximum Likelihood (ML) and Discrete Fourier Transfrom (DFT) Peak Finding for Single Tone Estimation

My understanding is Maximum Likelihood and DFT Peak Finding for a single tone produce the same results assuming the ML is restricted to the same frequencies as the DFT. I was wondering if there was ...
1
vote
1answer
83 views

Adaptive LMS Algorithm MATLAB

I'm having some trouble implementing my LMS Adaptive Filter in MATLAB to separate wideband and narrowband signals from a voice signal. I'm using a delayed version of my input as a reference as well ...
1
vote
4answers
224 views

Building a Signal Model for Samples from a Sensor

I have a signal I'm getting from a sensor (shown below in the photo of an excel plot) and I need to process it to get it to look something like the red line I've overdrawn on it. Sort of a moving ...
1
vote
0answers
26 views

Difference Between a 1st Order SG Filter And a Least Squares Moving Average

I have been studying SG filters and i recently found another filter which seem to be commonly used in financial data smoothing which is the least-squares moving average, this filter is also called ...
1
vote
1answer
103 views

Sequential Non Linear Least Squares Problem

I have the the following non-linear function, $$s(x;A_k,\mu_k,\sigma_k)=\sum_{k=1}^2 A_k \exp\left(\frac{-(x-\mu_k)^2}{\sigma_k^2}\right)$$ with unknown (but deterministic) parameters $A_k,\mu_k,\...
1
vote
0answers
75 views

Solve for Transfer Function Coefficients Embedded in a Non Linear System

Given a complex input signal $x$ and real input signal $v$, a 4th order (for simplicity) transfer function $H(z)$ is first applied to $v$ to obtain $w$, which in the time domain is represented by the ...
1
vote
0answers
36 views

Digital Filter Design Accuracy [duplicate]

Having only dealt with digital filter scarcely, the question dawned to me when I used the firls function in matlab to design an equalizer with a certain gain response. In general, can we prescribe ...
1
vote
0answers
123 views

Instability Problem in Normalized Least Mean Squares (NLMS) Adaptive Algorithm

I have been used NLMS algorithm to equalize 4x4 MIMO signals, but the bit-error-rate (BER) after equalization is unstable with iterations. I don't know if it is the normal behavior of the adaptive ...
1
vote
0answers
61 views

Solving an Array Signal Processing Estimation Problem based on the Rayleigh Quotient

The Rayleigh quotient for a covariance matrix $\mathbf{C}$ and a non-zero steering vector $\mathbf{a}$ is given by $$ R(\mathbf{C},\mathbf{a}) := \frac{\mathbf{a}^H\mathbf{C}\mathbf{a}}{\mathbf{a}^H\...
1
vote
0answers
84 views

How to Approximate / Fit a Function and a Lower Bound on It?

I am dealing with a complicated optimization problem with two parameters $N, K$. Setting $K =$ constant and finding the minimum for different $N$'s lead to a set of function values. I can do this for $...