Questions tagged [least-squares]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
7
votes
2answers
288 views

High Dynamic Range FIR Filters

Related to this question on using the Windowing method for FIR filter design versus the optimized algorithms such as least squares (firls in MATLAB, Octave and ...
2
votes
1answer
54 views

How to find point of inflexion of a digital signal?

Let's say I would like to find a point of inflexion of a digital signal which has been gathered via sampling of a continuous time domain signal with sampling period $100\,\mu s$. The digital signal ...
2
votes
0answers
62 views

Noise variance and sampling

I am new to signal processing, so please bear with me. My question applies to any general problem of estimation from noisy measurements but I will like to understand this through a problem given here. ...
0
votes
0answers
64 views

Impacted of a conjugated filter

What is the effect of using a conjugated FIR filter? I have a use case where a weight vector, x, can be estimated using the least squares approximation: $$Ax=b$$ $$ A^HAx = A^Hb$$ $$ x = (A^HA)^{-1} ...
2
votes
2answers
54 views

Methods for time series estimation in time domain

I am trying to estimate the clean form of a time series, $u(t)$ that is corrupted by additive White Gaussian noise $w(t)$ at a particular SNR. The received signal is: $$y(t) = u(t) + w(t)$$ My first ...
0
votes
0answers
16 views

compressed sensing versus Lomb-Scargle

Say I have a signal that's the sum of only a few sine/cosine waves, and some noise, which has been measured at random times. I would like to find the frequencies of the waves. With this goal in-mind, ...
1
vote
2answers
145 views

Projection Matrix derivation to constrained optimization problem with Lagrange multiplier

I am trying to derive famous Projection operator as a constrained minimization problem for the least square problem. The question is as follows: Find $x$ minimizing $ (y-x)^T(y-x)$ subject to $x = H\...
4
votes
1answer
139 views

Frequency-domain deconvolution: "Direct" filtering vs "Wiener" filtering

Can someone help with clarifying the difference between two approaches to frequency domain "deconvolution: For the frequency domain problem: We want to find a filter $F(\omega)$ which will ...
1
vote
0answers
13 views

Apply window taper for selected window or to full data block in algebraic problem with known frequencies?

First, I'd like to introduce a problem. I try to solve a set of linear equations in LS (Least Squares) sense, i.e. Ax=y (LS) => A^TA x = A^T y. x is a vector I'm looking for, but y is a measured ...
1
vote
0answers
198 views

Recursive Least Square For Filtering

I just started doing research on Recursive Least Square for filtering noises such as sensors and dc motors noises. The only thing I've seen on the internet was Theoretical information about RLS but ...
5
votes
1answer
231 views

Image Restoration by Solving Constrained Least squares in Frequency Domain (Frequency Domain Filtering)

I am trying to implement the constrained least squares filtering as described in Rafael C. Gonzalez, Richard E. Woods - Digital Image Processing 3rd Edition Section 5.9. The equation (...
2
votes
2answers
99 views

Why should the last point be excluded when performing a least-squares fit of a periodic discrete time signal?

I fitted the function: f(t)=A_o+A_1 cos(wt)+B_1 sin(wt) to the following periodic discrete signal: ...
1
vote
1answer
55 views

The cost function of LS algorithm

I'm trying to understand the derivation of LS algorithm for channel estimation in OFDM. Could anyone explain why do we need complex conj transpose in the below equation?
5
votes
2answers
111 views

Determine the Signal Curve from Parameters of a Power Curve by Noisy Measurement

I have a class of signals described by function: $$ f(inc,d,t)=inc\cdot t^d $$ where inc and d have a finite set of values like 1, 2, 3, i.e. $$ inc, d\in \left \{1,2,3 \right \} $$ and $$ 0\leq t<...
4
votes
1answer
72 views

Questions on the Generalized Tikhonov Regularization

My first question is about the quadratic functional that is used in generalized Tikhonov regularization: $$\Psi(f)=\frac{1}{2}\|f\|^2_\Gamma=f^T\Gamma f.$$ In the above equation what does $\Gamma$ ...
3
votes
1answer
51 views

Variational Regularization Method in Image Processing

I would like to understand better variational regularisation methods in image processing. In particular, the formulas in this image: Why formula (3.13)? In the notes that I read I cannot find ...
2
votes
1answer
79 views

Difference equation system identification

Suppose I have a difference equation modeling a vehicle like this: $$d[k+1]=d[k]+a\cdot u[k]+b,\tag1\label{eq}$$ where $d[k]$ is total distance traveled at time $k$, $u[k]$ is engine input at time $k$ ...
0
votes
0answers
36 views

Unable to estimate for AR model using OLS, Yule Walker and MLE

I am learning estimation methods following the book of Steven Kay, "Fundamentals of Statistical Processing, Volume I: Estimation Theory " Theory says that if the measurement noise is ...
1
vote
1answer
49 views

How to find the value of regularization parameter/s when having one or multiple priors

I am working on a image reconstruction framework, specifically interferometry. In simple words we have data that is pertubated by noise, say $$ V^o_i = V_i + \sigma_i $$ Where $V^o$ is a complex ...
1
vote
1answer
634 views

Building a FIR filter from an arbitrary frequency-magnitude response curve (eg. Least Squares fitting) [duplicate]

I'm trying to create a FIR filter from a magnitude equation, where my starting equation provides the magnitude (amplitude) between 0 and 1 for any given frequency in Hz. I posted the magnitude ...
0
votes
2answers
39 views

Simultaneously minimize and maximize two cost functions

I want to solve the following optimization problem, in which I have two cost functions $J_1 = \mathbf{w}^T R_1 \mathbf{w}$ and $J_2 = \mathbf{w}^T R_2 \mathbf{w}$, where $R_1$ and $R_2$ are covariance ...
5
votes
2answers
149 views

Quadratic Programming with Linear Equality Constraints

I need to solve an equality constrained minimization problem as give below $$\min_{\textbf{w}} \mathbf{w}^TR\mathbf{w} $$ such that $$X\mathbf{w} = \mathbf{1}$$ where $R\in \mathbb{R}^{n\times n}$ is ...
0
votes
1answer
33 views

Least Squares Filter Design: Deriving the Objective Function

I'm following the derivation in this paper A Comb Filter Design Using Fractional-Sample Delay to obtain the objective function for the least-squares filter design. N-order FIR filter: $H(z) = \sum_{...
1
vote
0answers
18 views

Solving Sparse Model with given Dictionary Using LASSO

I was trying to solve a problem where the basis matrix contains the components of $\sin(nx)$, $\cos(nx)$, $\sinh(nx)$ and $\cosh(nx)$. Say the $n$ varies from 1 to 100. While solving the lasso linear ...
1
vote
0answers
28 views

How to find the solution of nonlinear least square (NLS) equation for freqeuncy estimation

I am struggling to understand the theory of parametric methods for power spectral estimations of line spectra. And I want to find the solution of nonlinear least square (NLS) method for estimating ...
1
vote
2answers
395 views

Solve Undetermined Linear System Using NumPy's `lstsq()` Function

I would like to understand what I am doing wrong here. I am trying to perform polynomial regression by minimizing the least squares, $||Au-y||^2$, where $y$ is the given data and $A$ is the matrix ...
5
votes
1answer
178 views

Use Recursive Least Square as an "Actual" Filter?

I am currently studying recursive least square (RLS) and as far as I understand, the setup is (for example) that we are given a process with uncertain parameters/noise and we want to adapt parameters ...
1
vote
1answer
60 views

Why is the concept of a "state covariance matrix" necessary in estimation?

I'm currently taking a course in optimal estimation (and it's still very early in the course). Much of our work is based around the idea of a measurement model $y=Hx + v$ This model assumes our ...
3
votes
1answer
197 views

Jacobian Computation in Least Squares IIR Filter Design

Long time lurker and first time poster - but unfortunately I haven't had any joy untangling this on my own. I've been studying Mathias Lang's thesis, Algorithms for the Constrained Design of Digital ...
2
votes
2answers
244 views

System Identification with a Limited Bandwidth Input Signal and Region of Interest

Given a FIR filter $h[n]$. Its action can described as: $$ \mathbf{y} = \mathbf{H} \mathbf{x} \\ \mathbf{y} = \mathbf{X} \mathbf{h} $$ where $\mathbf{H}$ and $\mathbf{X}$ is a Toeplitz matrix. If $h$...
1
vote
0answers
65 views

Difference Between a 1st Order SG Filter And a Least Squares Moving Average

I have been studying SG filters and i recently found another filter which seem to be commonly used in financial data smoothing which is the least-squares moving average, this filter is also called ...
0
votes
0answers
446 views

How to find Coefficients of Autoregressive (AR) model using least means square (LMS) Algorithm without having future signal?

I want to do two things. Estimating Coefficients of AR model using LMS Using Coefficients found in step 1 and predict future samples of a signal using AR equation. I don't have a desired signal so I ...
0
votes
1answer
141 views

Range Estimation Based on Distance Differences

It is a range estimation problem, as shown in the figure. The blue circles are the anchor node' positions, which are produced by one moving anchor (the moving direction is shown with black arrow). ...
0
votes
0answers
112 views

Non Linear Recursive / Online Least Squares

I am searching for a recursive or online non linear least squares algorithm. I want to spread the computation out as new data is sampled like in the linear Recursive Least Squares or the LMS. ...
4
votes
1answer
220 views

Equivalence of Maximum Likelihood (ML) and Discrete Fourier Transfrom (DFT) Peak Finding for Single Tone Estimation

My understanding is Maximum Likelihood and DFT Peak Finding for a single tone produce the same results assuming the ML is restricted to the same frequencies as the DFT. I was wondering if there was ...
1
vote
2answers
190 views

Solving a Linear Mean Square Estimation the Easy Way

I have an exercise which is quite trivial. However I got stuck and I'm not sure if this the end-result. I assume there has to be a way to get this result much quicker. Given are two randomly ...
3
votes
1answer
105 views

Linear Difference Equation and Method of Least Squares

I'm reading the book "Fault-Diagnosis Systems" by Isermann in the par. 9.2.1a. The author explains how to estimate the parameter of a linear difference equation using Least Squares. We start with a ...
6
votes
4answers
1k views

Sequential Form of the Least Squares Estimator for Linear Least Squares Model

I'm currently working on a project in which I need to find the tilt of a surface. Let's assume I'm only concerned with a single dimension tilt (i.e. slope) to begin. I currently have the ability to ...
0
votes
1answer
38 views

Defining Model in Kalman for Getting Position Using

I am stuck at modeling a system model, i.e. getting my state vector and input vector for navigating just using navaid and ins (tactical). My guess is that position is my only state vector and INS ...
2
votes
1answer
640 views

Adaptive LMS Algorithm MATLAB

I'm having some trouble implementing my LMS Adaptive Filter in MATLAB to separate wideband and narrowband signals from a voice signal. I'm using a delayed version of my input as a reference as well ...
0
votes
1answer
101 views

Equalizers with low data rate systems

In case of using equalizers with ZigBee systems, what type of fading does the channel show, flat or frequency selective, fast or slow? Is the inter-symbol interference (ISI) problem encountered in ...
0
votes
1answer
754 views

Recursive Least Square Adaptive Linear Equalizer

For the adaptive filter to work properly, a desired signal d(n) needs to be provided. The output from the equalizer y(n) is subtracted from d(n) to produce an error signal, which is used to adjust the ...
2
votes
1answer
684 views

Design of FIR Filters with Arbitrary Magnitude and Phase Responses

I found many information in this thesis "Algorithms for the Constrained Design of Digital Filters with Arbitrary Magnitude and Phase Responses",i want to understand it. This question is a ...
3
votes
1answer
162 views

Sequential Non Linear Least Squares Problem

I have the the following non-linear function, $$s(x;A_k,\mu_k,\sigma_k)=\sum_{k=1}^2 A_k \exp\left(\frac{-(x-\mu_k)^2}{\sigma_k^2}\right)$$ with unknown (but deterministic) parameters $A_k,\mu_k,\...
4
votes
3answers
396 views

Complex Least Squares Approximation

In the case of frequency domain FIR filter design, the error function given by : $$E(\omega)=H(e^{j\omega})-D(e^{j\omega}) \tag{1}$$ is a linear function with respect to the unknown filter ...
3
votes
1answer
152 views

Why Isn't the ML Estimator (MLE) in MIMO Spatial Multiplexing Obtained by the Least Squares Solution?

In the simplest scenario of MIMO spatial multiplexing: $$\mathbf{y} = H\mathbf{s} + \mathbf{n}$$ where: $\mathbf{s}=[s_0,s_1,...s_{M-1}] \\\mathbf{y}=[y_0,y_1,...y_{N-1}]$ $\mathbf{n}=[n_0,n_1,......
3
votes
2answers
112 views

Question About Kailath's Paper - An Innovations Approach to Least Squares Estimation Part I: Linear Filtering in Additive White Noise

I'm reading the paper at the link below and I was following it for about 2 pages until I hit a road block on the bottom of page 648 where the author says: putting together 9-11, we obtain and ...
1
vote
2answers
74 views

What Is the Definition of Linear Predictive Coefficients When the Optimal Value Aren't Unique?

Linear predictive coefficients of signal y are defined as the best $k$ coefficients $a_i, i = 1, \ldots, k$, that will approximate $y_n$ by $-\sum_{i=1}^k{a_iy_{n-i}}$. (Best approximation is that ...
4
votes
2answers
1k views

Why Is Non Linear Least Squares Method from MATLAB and Alglib Gives Different Results on the Same Data?

i'm trying to rewrite my Matalab prototype for some DSP to C++ and encountering a displeasing problem. I'm trying to fit data to a function $y = a * (\pi / 2 + arctg(b * x))$. In Matlab it works well ...
5
votes
2answers
600 views

Finding the Best Gaussian Smoothing Kernel to Minimize the Discrepancy Between Two Images

Suppose we have two grayscale images, $A$ and $B$. $A$ and $B$ very strongly resemble each other, such that the mean of the absolute difference $\lvert A - B\rvert$ is fairly small. Suppose further ...