7 votes

Curve Fit of Step Function with Boundary on the 2nd Derivative

Hmmmmmmmmm, interesting question. Since you want to use the second derivative as your criteria, it would seem that you would want to have the maximum second derivative absolutie value for as short of ...
Cedron Dawg's user avatar
  • 7,522
7 votes

Derivative with respect to complex conjugate

That's a trick which you will also find in a DSP context, that's why I choose to provide an answer here. It is related to the Wirtinger derivative, and you can find more details about it in this ...
Matt L.'s user avatar
  • 88.8k
6 votes

Best Metric to Compare Sparsity of Vectors

I am sorry I cannot comment your answer due to my low reputation. Gini and your suggested sparsity ratio ($l_1(x)/l_2(x)$) both give me the same value for $\lambda$. But The problem I still see is ...
N8_Coder's user avatar
  • 223
6 votes
Accepted

Why use parametric based estimation methods - confusion regarding terms

Hi: I'll try to answer as briefly as possible and only with respect to statistics. not dsp. In statistics, if you have a nice pdf such as the normal distribution, then maximizing the likelihood is ...
mark leeds's user avatar
  • 1,107
6 votes

Why Do Most of The Papers Use the Frobenius Norm for Denoising?

The Frobenius Norm has multiple equivalent definitions – the useful for error measure is probably this one: $$\left\|M\right\|_\mathrm F = \sqrt{\sum_{p\in M}\left\lvert p\right\rvert^2}$$ That's a ...
Marcus Müller's user avatar
6 votes
Accepted

Why Does the Median Filter Minimize the Absolute Value Error $L_1$ Cost Function?

Given a set of values $ {\left\{ {s}_{i} \right\}}_{i = 1}^{N} $, we're basically after: $$ \arg \min_{x} \sum_{i = 1}^{N} \left| {s}_{i} - x \right| $$ One should notice that $ \frac{\mathrm{d} \left ...
Royi's user avatar
  • 19.3k
5 votes

Adaptive filtering: Optimum filter length and delay

In order to be able to choose an optimal value for the delay $\Delta$ it's important to understand how the system works. The purpose of the delay is to decorrelate the desired signal $s(n)$ and the ...
Matt L.'s user avatar
  • 88.8k
5 votes
Accepted

Lagrange Multipliers Optimization - Complex Functions

I found the following in Charles Therrien's "Discrete Random Signals and Statistical Signal Processing" in one of the Appendicies. Say you have the function $Q(a)$ you wish to minimize such that $C(a)...
David's user avatar
  • 2,841
5 votes

Derivative with respect to complex conjugate

I felt I needed to write an additional answer to try to clear my mind about the question. Here is the try, step by step. Caveat: for simplicity, I used the same notation $C$ of a function of reals $u$...
Laurent Duval's user avatar
5 votes

How to Formulate a Constraint Which Ensures All Variables Have the Same Sign

The solution from the blog you linked goes as following (Coordinating Variable Signs by Paul Rubin, Web Archive): Someone asked me today (or yesterday, depending on whose time zone you go by) how to ...
Royi's user avatar
  • 19.3k
5 votes
Accepted

Quadratic Programming with Linear Equality Constraints

Let's solve a more general problem (Least Squares with Linear Equality Constraints): $$ \begin{alignat*}{3} \arg \min_{x} & \quad & \frac{1}{2} \left\| A x - b \right\|_{2}^{2} \\ \text{...
Royi's user avatar
  • 19.3k
4 votes

What Is the Best First Order IIR (AR Filter) Approximation to a Moving Average Filter (FIR Filter)?

Based in experimental tests with k in range (2 to 100) the best fit (sum squared error) gives a relation of alfa = 1/k^0.865 ...
chiliwili69's user avatar
4 votes

Derivative with respect to complex conjugate

I have found an alternate answer which is very simple and comprehensive, so thinking to share with all. In order to differentiate an expression $f(z)$ with respect to a complex $z$, the Cauchy-...
Sal's user avatar
  • 163
4 votes
Accepted

On the Measurement Matrix Used for Compressing Sensing

It is indeed possible to formulate this setting in terms of matrix-vector products. First, let us re-formulate your $x$ (notice throughout that I use bold letters for vectors and matrices): $$x = \...
Thomas Arildsen's user avatar
4 votes
Accepted

How to Solve Non Blind Image Deblurring with Total Variation Prior Using ADMM?

Formulation of the Problem I am solving the problem under the following assumptions: The blurring operator is Linear and Spatially Invariant (Hence applied by convolution). The blurring operator is ...
Royi's user avatar
  • 19.3k
4 votes

Optimization of square matrix multiplied with another matrix to have the final result a unitary matrix

Could it be that you are indeed looking for the closest orthogonal matrix $Y$? Then, there is a solution which involves computing the square root of $ D^TD$ . If $E=(D^TD)^{1/2}$ were invertible, the ...
Laurent Duval's user avatar
4 votes
Accepted

Adding Variance \ Weights Information When Solving a Basis Pursuit Denoising Problem (BPDN)

Your formulation: $$ \arg \min_{\boldsymbol{x}} \frac{1}{2} {\left\| A \boldsymbol{x} - \boldsymbol{y} \right\|}_{2}^{2} + \lambda {\left\| \boldsymbol{x} \right\|}_{1} $$ Has 2 elements: The ...
Royi's user avatar
  • 19.3k
4 votes

Why expected value is optimal?

Nothing is "optimal" in an absolute sense. Something is or is not optimal according to a defined, precise criterion. Say you have a real random variable $X$. You want to find a number $m$ ...
MBaz's user avatar
  • 14.9k
4 votes
Accepted

Gradient descent algorithm not converging

Your step size is too large. The upper limit $2/\lambda_{max}$ for the step size $\mu$ is valid if the update is defined as $$\mathbf{w}_{k+1}=\mathbf{w}_{k}-\frac{\mu}{2}\nabla J(\mathbf{w}_k)$$ The ...
Matt L.'s user avatar
  • 88.8k
3 votes
Accepted

Fitting an IIR filter to a complex transfer function

As I mentioned in a comment, I think that the equation error method is a very good starting point for designing an IIR filter in the frequency domain with prescribed magnitude and phase responses. For ...
Matt L.'s user avatar
  • 88.8k
3 votes
Accepted

How to Use the DFT (FFT) to Solve a Least Squares Regularization Problem (Inverse Problem)?

The question really depends on $ f \left( \cdot \right) $. Yet in order to show how to use FFT we can even use 1D signals. Let's rewrite the problem: $$ \hat{x} = \arg \min_{x} \frac{1}{2} \left\| K ...
Royi's user avatar
  • 19.3k
3 votes

Best Metric to Compare Sparsity of Vectors

Norms like $\ell_p$, $p \ge 1$, or quasi-norms ($0<p< 1$) are all $1$-homogeneous: $\ell_p(\lambda x) = |\lambda|\ell_p( x)$. Which is not the case for the $\ell_0$ count measure, which is scale ...
Laurent Duval's user avatar
3 votes
Accepted

Filtering performance on Poisson noise with quadratic data-fidelity

For large intensities / large "bins", i.e. "areas for which events are counted and accumulated", Poisson processes lead to nearly Gaussian distributed individual values -- basically, without trying to ...
Marcus Müller's user avatar
3 votes

What Is the Best First Order IIR (AR Filter) Approximation to a Moving Average Filter (FIR Filter)?

I stumbled upon this old question and I would like to share my solution. As mentioned in other answers, there is no analytical solution, but the function to be minimized behaves nicely and the optimal ...
Matt L.'s user avatar
  • 88.8k
3 votes
Accepted

Reference Code for Positive Basis Pursuit Denoising

SparseLab should be able to solve the positivity-constrained problem: SparseLab (Stanford) - Seeking Sparse Solutions to Linear System of Equations. See Donoho & Tanner, "Precise ...
Thomas Arildsen's user avatar
3 votes

Curve Fit of Step Function with Boundary on the 2nd Derivative

You can make a discretized regularized linear equation system. $\bf d$ is the original signal $\bf v$ is what we add to the signal, (the additive change) $\bf v+d$ is the result. So there are two ...
mathreadler's user avatar
3 votes
Accepted

Difference Between Iteratively Reweighted Least Squares (IRLS) and Sequential Quadratic Programming?

SQP is a method for solving smooth (objective and constraint functions are at least twice differentiable) constrained nonlinear optimization problems. It solves a series of quadratic programming ...
Brian Borchers's user avatar
3 votes

Solving LASSO ($ {L}_{1} $ Regularized Least Squares) with Gradient Descent

It can easily solved by the Gradient Descent Framework with one adjustment in order to take care of the $ {L}_{1} $ norm term. Since the $ {L}_{1} $ norm isn't smooth you need to use the concept of ...
Royi's user avatar
  • 19.3k
3 votes
Accepted

Fastest Available Algorithm to Blur an Image (Low Pass Filter)

The fastest blur would be Box Blur. You can implement it using Running Sum. I think Intel FilterBoxBorder works in that manner. If you'd like you can do a few ...
Royi's user avatar
  • 19.3k

Only top scored, non community-wiki answers of a minimum length are eligible