# Tag Info

### Curve Fit of Step Function with Boundary on the 2nd Derivative

Hmmmmmmmmm, interesting question. Since you want to use the second derivative as your criteria, it would seem that you would want to have the maximum second derivative absolutie value for as short of ...

### Derivative with respect to complex conjugate

That's a trick which you will also find in a DSP context, that's why I choose to provide an answer here. It is related to the Wirtinger derivative, and you can find more details about it in this ...

### Best Metric to Compare Sparsity of Vectors

I am sorry I cannot comment your answer due to my low reputation. Gini and your suggested sparsity ratio ($l_1(x)/l_2(x)$) both give me the same value for $\lambda$. But The problem I still see is ...
Accepted

### Why use parametric based estimation methods - confusion regarding terms

Hi: I'll try to answer as briefly as possible and only with respect to statistics. not dsp. In statistics, if you have a nice pdf such as the normal distribution, then maximizing the likelihood is ...

### Why Do Most of The Papers Use the Frobenius Norm for Denoising?

The Frobenius Norm has multiple equivalent definitions – the useful for error measure is probably this one: $$\left\|M\right\|_\mathrm F = \sqrt{\sum_{p\in M}\left\lvert p\right\rvert^2}$$ That's a ...
Accepted

### Derivative with respect to complex conjugate

I felt I needed to write an additional answer to try to clear my mind about the question. Here is the try, step by step. Caveat: for simplicity, I used the same notation $C$ of a function of reals $u$...

### How to Formulate a Constraint Which Ensures All Variables Have the Same Sign

The solution from the blog you linked goes as following (Coordinating Variable Signs by Paul Rubin, Web Archive): Someone asked me today (or yesterday, depending on whose time zone you go by) how to ...
Accepted

Accepted

### How to Solve Non Blind Image Deblurring with Total Variation Prior Using ADMM?

Formulation of the Problem I am solving the problem under the following assumptions: The blurring operator is Linear and Spatially Invariant (Hence applied by convolution). The blurring operator is ...

### Optimization of square matrix multiplied with another matrix to have the final result a unitary matrix

Could it be that you are indeed looking for the closest orthogonal matrix $Y$? Then, there is a solution which involves computing the square root of $D^TD$ . If $E=(D^TD)^{1/2}$ were invertible, the ...
Accepted

### Adding Variance \ Weights Information When Solving a Basis Pursuit Denoising Problem (BPDN)

Your formulation: $$\arg \min_{\boldsymbol{x}} \frac{1}{2} {\left\| A \boldsymbol{x} - \boldsymbol{y} \right\|}_{2}^{2} + \lambda {\left\| \boldsymbol{x} \right\|}_{1}$$ Has 2 elements: The ...

### Why expected value is optimal?

Nothing is "optimal" in an absolute sense. Something is or is not optimal according to a defined, precise criterion. Say you have a real random variable $X$. You want to find a number $m$ ...
Accepted

### Gradient descent algorithm not converging

Your step size is too large. The upper limit $2/\lambda_{max}$ for the step size $\mu$ is valid if the update is defined as $$\mathbf{w}_{k+1}=\mathbf{w}_{k}-\frac{\mu}{2}\nabla J(\mathbf{w}_k)$$ The ...
Accepted

### Fitting an IIR filter to a complex transfer function

As I mentioned in a comment, I think that the equation error method is a very good starting point for designing an IIR filter in the frequency domain with prescribed magnitude and phase responses. For ...
Accepted

### How to Use the DFT (FFT) to Solve a Least Squares Regularization Problem (Inverse Problem)?

The question really depends on $f \left( \cdot \right)$. Yet in order to show how to use FFT we can even use 1D signals. Let's rewrite the problem:  \hat{x} = \arg \min_{x} \frac{1}{2} \left\| K ...

### Best Metric to Compare Sparsity of Vectors

Norms like $\ell_p$, $p \ge 1$, or quasi-norms ($0<p< 1$) are all $1$-homogeneous: $\ell_p(\lambda x) = |\lambda|\ell_p( x)$. Which is not the case for the $\ell_0$ count measure, which is scale ...
Accepted

### Filtering performance on Poisson noise with quadratic data-fidelity

For large intensities / large "bins", i.e. "areas for which events are counted and accumulated", Poisson processes lead to nearly Gaussian distributed individual values -- basically, without trying to ...

### What Is the Best First Order IIR (AR Filter) Approximation to a Moving Average Filter (FIR Filter)?

I stumbled upon this old question and I would like to share my solution. As mentioned in other answers, there is no analytical solution, but the function to be minimized behaves nicely and the optimal ...
Accepted

### Reference Code for Positive Basis Pursuit Denoising

SparseLab should be able to solve the positivity-constrained problem: SparseLab (Stanford) - Seeking Sparse Solutions to Linear System of Equations. See Donoho & Tanner, "Precise ...

### Curve Fit of Step Function with Boundary on the 2nd Derivative

You can make a discretized regularized linear equation system. $\bf d$ is the original signal $\bf v$ is what we add to the signal, (the additive change) $\bf v+d$ is the result. So there are two ...
Accepted

### Difference Between Iteratively Reweighted Least Squares (IRLS) and Sequential Quadratic Programming?

SQP is a method for solving smooth (objective and constraint functions are at least twice differentiable) constrained nonlinear optimization problems. It solves a series of quadratic programming ...
### Solving LASSO (${L}_{1}$ Regularized Least Squares) with Gradient Descent
It can easily solved by the Gradient Descent Framework with one adjustment in order to take care of the ${L}_{1}$ norm term. Since the ${L}_{1}$ norm isn't smooth you need to use the concept of ...