Questions tagged [least-squares]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1
vote
0answers
16 views

Solving Sparse Model with given Dictionary Using LASSO

I was trying to solve a problem where the basis matrix contains the components of $\sin(nx)$, $\cos(nx)$, $\sinh(nx)$ and $\cosh(nx)$. Say the $n$ varies from 1 to 100. While solving the lasso linear ...
1
vote
0answers
25 views

How to find the solution of nonlinear least square (NLS) equation for freqeuncy estimation

I am struggling to understand the theory of parametric methods for power spectral estimations of line spectra. And I want to find the solution of nonlinear least square (NLS) method for estimating ...
0
votes
1answer
83 views

numpy.linalg.lstsq underdetermined case

I would like to understand what I am doing wrong here. I am trying to perform polynomial regression by minimizing the least squares, ||Au-y||^2, where y is the given data and A is the matrix where the ...
4
votes
1answer
100 views

Use recursive least square as an “actual” filter?

I am currently studying recursive least square (RLS) and as far as I understand, the setup is (for example) that we are given a process with uncertain parameters/noise and we want to adapt parameters ...
1
vote
1answer
38 views

Why is the concept of a “state covariance matrix” necessary in estimation?

I'm currently taking a course in optimal estimation (and it's still very early in the course). Much of our work is based around the idea of a measurement model $y=Hx + v$ This model assumes our ...
3
votes
1answer
107 views

Jacobian Computation in Least Squares IIR Filter Design

Long time lurker and first time poster - but unfortunately I haven't had any joy untangling this on my own. I've been studying Mathias Lang's thesis, Algorithms for the Constrained Design of Digital ...
1
vote
1answer
127 views

System Identification with a Limited Bandwidth Input Signal and Region of Interest

Given a FIR filter $h[n]$. Its action can described as: $$ \mathbf{y} = \mathbf{H} \mathbf{x} \\ \mathbf{y} = \mathbf{X} \mathbf{h} $$ where $\mathbf{H}$ and $\mathbf{X}$ is a Toeplitz matrix. If $h$...
1
vote
0answers
39 views

Difference Between a 1st Order SG Filter And a Least Squares Moving Average

I have been studying SG filters and i recently found another filter which seem to be commonly used in financial data smoothing which is the least-squares moving average, this filter is also called ...
0
votes
0answers
130 views

How to find Coefficients of Autoregressive (AR) model using least means square (LMS) Algorithm without having future signal?

I want to do two things. Estimating Coefficients of AR model using LMS Using Coefficients found in step 1 and predict future samples of a signal using AR equation. I don't have a desired signal so I ...
0
votes
1answer
115 views

Range Estimation Based on Distance Differences

It is a range estimation problem, as shown in the figure. The blue circles are the anchor node' positions, which are produced by one moving anchor (the moving direction is shown with black arrow). ...
0
votes
0answers
61 views

Non Linear Recursive / Online Least Squares

I am searching for a recursive or online non linear least squares algorithm. I want to spread the computation out as new data is sampled like in the linear Recursive Least Squares or the LMS. ...
1
vote
1answer
122 views

Equivalence of Maximum Likelihood (ML) and Discrete Fourier Transfrom (DFT) Peak Finding for Single Tone Estimation

My understanding is Maximum Likelihood and DFT Peak Finding for a single tone produce the same results assuming the ML is restricted to the same frequencies as the DFT. I was wondering if there was ...
1
vote
2answers
126 views

Solving a Linear Mean Square Estimation the Easy Way

I have an exercise which is quite trivial. However I got stuck and I'm not sure if this the end-result. I assume there has to be a way to get this result much quicker. Given are two randomly ...
0
votes
1answer
54 views

Linear Difference Equation and Method of Least Squares

I'm reading the book "Fault-Diagnosis Systems" by Isermann in the par. 9.2.1a. The author explains how to estimate the parameter of a linear difference equation using Least Squares. We start with a ...
4
votes
4answers
537 views

Sequential Form of the Least Squares Estimator for Linear Least Squares Model

I'm currently working on a project in which I need to find the tilt of a surface. Let's assume I'm only concerned with a single dimension tilt (i.e. slope) to begin. I currently have the ability to ...
0
votes
1answer
34 views

Defining Model in Kalman for Getting Position Using

I am stuck at modeling a system model, i.e. getting my state vector and input vector for navigating just using navaid and ins (tactical). My guess is that position is my only state vector and INS ...
1
vote
1answer
220 views

Adaptive LMS Algorithm MATLAB

I'm having some trouble implementing my LMS Adaptive Filter in MATLAB to separate wideband and narrowband signals from a voice signal. I'm using a delayed version of my input as a reference as well ...
1
vote
1answer
127 views

Using MATLAB Function `mpiir_l2()`: Results Are Not IIR but FIR Filters - Why and How to Avoid This?

I'm using mpiir_l2() from user Matt L's PhD thesis to design IIR filters. I set the number of numerator and denominator coefficients both to the same value (between ...
0
votes
1answer
97 views

Equalizers with low data rate systems

In case of using equalizers with ZigBee systems, what type of fading does the channel show, flat or frequency selective, fast or slow? Is the inter-symbol interference (ISI) problem encountered in ...
0
votes
1answer
374 views

Recursive Least Square Adaptive Linear Equalizer

For the adaptive filter to work properly, a desired signal d(n) needs to be provided. The output from the equalizer y(n) is subtracted from d(n) to produce an error signal, which is used to adjust the ...
0
votes
1answer
395 views

Design of FIR Filters with Arbitrary Magnitude and Phase Responses

I found many information in this thesis "Algorithms for the Constrained Design of Digital Filters with Arbitrary Magnitude and Phase Responses",i want to understand it. This question is a ...
2
votes
1answer
130 views

Sequential Non Linear Least Squares Problem

I have the the following non-linear function, $$s(x;A_k,\mu_k,\sigma_k)=\sum_{k=1}^2 A_k \exp\left(\frac{-(x-\mu_k)^2}{\sigma_k^2}\right)$$ with unknown (but deterministic) parameters $A_k,\mu_k,\...
2
votes
3answers
204 views

Complex Least Squares Approximation

In the case of frequency domain FIR filter design, the error function given by : $$E(\omega)=H(e^{j\omega})-D(e^{j\omega}) \tag{1}$$ is a linear function with respect to the unknown filter ...
0
votes
1answer
90 views

Why Isn't the ML Estimator (MLE) in MIMO Spatial Multiplexing Obtained by the Least Squares Solution?

In the simplest scenario of MIMO spatial multiplexing: $$\mathbf{y} = H\mathbf{s} + \mathbf{n}$$ where: $\mathbf{s}=[s_0,s_1,...s_{M-1}] \\\mathbf{y}=[y_0,y_1,...y_{N-1}]$ $\mathbf{n}=[n_0,n_1,......
1
vote
2answers
81 views

Question About Kailath's Paper - An Innovations Approach to Least Squares Estimation Part I: Linear Filtering in Additive White Noise

I'm reading the paper at the link below and I was following it for about 2 pages until I hit a road block on the bottom of page 648 where the author says: putting together 9-11, we obtain and ...
1
vote
2answers
62 views

What Is the Definition of Linear Predictive Coefficients When the Optimal Value Aren't Unique?

Linear predictive coefficients of signal y are defined as the best $k$ coefficients $a_i, i = 1, \ldots, k$, that will approximate $y_n$ by $-\sum_{i=1}^k{a_iy_{n-i}}$. (Best approximation is that ...
1
vote
2answers
586 views

Why Is Non Linear Least Squares Method from MATLAB and Alglib Gives Different Results on the Same Data?

i'm trying to rewrite my Matalab prototype for some DSP to C++ and encountering a displeasing problem. I'm trying to fit data to a function $y = a * (\pi / 2 + arctg(b * x))$. In Matlab it works well ...
3
votes
2answers
332 views

Finding the Best Gaussian Smoothing Kernel to Minimize the Discrepancy Between Two Images

Suppose we have two grayscale images, $A$ and $B$. $A$ and $B$ very strongly resemble each other, such that the mean of the absolute difference $\lvert A - B\rvert$ is fairly small. Suppose further ...
0
votes
3answers
124 views

Least Squares with Blocks / Updates

I have a continuous-time system that I want to fit via least squares. I just send $N$ digital samples $x[n]$ through the system and receive (via analog signal chain, ADC etc) $N$ digital samples $y[n]$...
1
vote
2answers
287 views

Solving LASSO ($ {L}_{1} $ Regularized Least Squares) with Gradient Descent

To the best of my knowledge, state of the art methods for optimizing the LASSO objective function include the LARS algorithm and proximal gradient methods. I was wondering however, if the LASSO ...
4
votes
1answer
3k views

Python: Least Squares Support Vector Machine (LS-SVM)

I'm looking for a Python package for a LS-SVM or a way to tune a normal SVM from scikit-learn to a Least-Squares Support Vector Machine for a classification problem. The goal of a SVM is to maximize ...
1
vote
1answer
291 views

Obtaining Correct (Least Squares Sense) Affine Transform Parameters Between Two Images

I have two images that I want to compute the affine motion model parameters. The model that I use is $$x' = a_1x+a_2y+a_3$$ $$y' = a_4x+a_5y+a_6$$ To calculate those 6 parameters, I picked 6 points (...
1
vote
2answers
694 views

Least Squares Fitting to Inverse Exponential Function

I have a time series of measurements that resembles the shape of an exponential function. The samples are a bit noisy and sometimes there is a weak sine like ripple signal ontop of it. Simplified the ...
0
votes
1answer
116 views

Received Signal Error vs. BER

It is my understanding that the least squares algorithm (e.g., in equalization) minimizes the received signal error. However, minimizing the received signal error does not necessarily equate to ...
1
vote
1answer
331 views

Least Squares Linear Phase FIR Filter Design

In explication ''the geometric interpretation of least squares'' Typically, the number of frequency constraints is much greater than the number of design variables (filter coefficients). In these ...
1
vote
1answer
215 views

RLS Algorithm (Memoryless)

I have been studying the adaptive filters lately and now when I am at RLS (Recursive Least Squar) Algorithm I came across the term used in the weighting function of the RLS called forgetting factor ($\...
-1
votes
1answer
158 views

Properties of Optimization Techniques in Filter Design

I have this design: Can you tell me about the type of filter and the algorithm just viewing this design, or should there be other information?
3
votes
0answers
285 views

Least Squares Approximation for FIR Filter Design

Following this paper , I am trying to make a least-squares algorithm in MATLAB, but for type I (I know about firls()). ...
3
votes
1answer
439 views

How to Use the DFT (FFT) to Solve a Least Squares Regularization Problem (Inverse Problem)?

Let $X$ and $K$ be an image and a Point Spread Function (PSF), respectively. The blur image $B$ is obtained as follows $$B = X * K$$ I want to solve the following general regularization problem $$...
4
votes
1answer
313 views

Direct Correlation (DC) Time Delay Estimation: Variance Keeps Decreasing for Increasing SNR?

I'm trying to reproduce the results from this paper "Discrete time techniques for Time Delay Estimation" doi:10.1109/78.193195, for both the correlation and Least Squares. I've generated a random (...
1
vote
0answers
90 views

Solve for Transfer Function Coefficients Embedded in a Non Linear System

Given a complex input signal $x$ and real input signal $v$, a 4th order (for simplicity) transfer function $H(z)$ is first applied to $v$ to obtain $w$, which in the time domain is represented by the ...
0
votes
1answer
342 views

Estimate a Transfer Function from ARX Models vs. ARIMAX?

There is diffrent models which can be used to create a dynamical model by using least squares. Those models are following: ARX ARMAX ARIMAX OE BJ But if my goal with creating a dynamical model is to ...
1
vote
1answer
408 views

Difference between Leaking Factor and Forgetting Factor

I am using a Recursive least square adaptive filter to process electromyography signals and it is working decently so far. I decided to implement an LMS adaptive filter as a noise cancellation, so ...
1
vote
0answers
38 views

Digital Filter Design Accuracy [duplicate]

Having only dealt with digital filter scarcely, the question dawned to me when I used the firls function in matlab to design an equalizer with a certain gain response. In general, can we prescribe ...
6
votes
6answers
424 views

The Least Norm Solution of Under Determined Linear System

Suppose I have a matrix $$A= \begin{pmatrix} 1 & 0 & 1 & 0\\ 0 & 1 & 1& 0\\ \end{pmatrix} $$ where the variables are channel information like assume $X_1$, $X_2$, $X_3$ ...
2
votes
1answer
1k views

Maximum Likelihood Estimator (MLE), MMSE and LS - Are All of Them Regressor, Estimator and Predictor?

Can all three criteria ML, MMSE, and LS be called regressor, estimator, and predictor ? If not, an intuitive explanation of why they can't be, would be good.
0
votes
1answer
70 views

$ \frac{1}{f^2} $ Weighting for Least Squares FIR

I am currently modifying the firls.m from Octave to allow all types of FIRs, differentiators, and Hilbert transformers, while taking a peak at Matlab's. I see there ...
0
votes
1answer
134 views

Least Squares FIR for Type II

Following this paper (see the Matlab example at the end), I am trying to make a least-squares algorithm in Octave, but for type II (I know about firls). For type I ...
0
votes
2answers
397 views

How to Add Group Delay?

I was confused about the value of group delay, I am not sure whether my opinion is correct. For example, if I want to add a group delay of $0.25$ time units to a zero-phase signal, does it mean that I ...
1
vote
2answers
134 views

MATLAB: Implementing Least Squares Estimator for a Given Model

The formula to estimate $\mathbf{h}$ is then $$\hat{\mathbf{h}} = (X^T X)^{-1} X^T \vec{y}\tag{2}$$ I think this can be implemented in Matlab using ...