Skip to main content
Share Your Experience: Take the 2024 Developer Survey
7 votes

Curve Fit of Step Function with Boundary on the 2nd Derivative

Hmmmmmmmmm, interesting question. Since you want to use the second derivative as your criteria, it would seem that you would want to have the maximum second derivative absolutie value for as short of ...
Cedron Dawg's user avatar
  • 7,570
7 votes

Derivative with respect to complex conjugate

That's a trick which you will also find in a DSP context, that's why I choose to provide an answer here. It is related to the Wirtinger derivative, and you can find more details about it in this ...
Matt L.'s user avatar
  • 90.4k
6 votes

Best Metric to Compare Sparsity of Vectors

I am sorry I cannot comment your answer due to my low reputation. Gini and your suggested sparsity ratio ($l_1(x)/l_2(x)$) both give me the same value for $\lambda$. But The problem I still see is ...
N8_Coder's user avatar
  • 223
6 votes

Why Do Most of The Papers Use the Frobenius Norm for Denoising?

The Frobenius Norm has multiple equivalent definitions – the useful for error measure is probably this one: $$\left\|M\right\|_\mathrm F = \sqrt{\sum_{p\in M}\left\lvert p\right\rvert^2}$$ That's a ...
Marcus Müller's user avatar
6 votes
Accepted

Why use parametric based estimation methods - confusion regarding terms

Hi: I'll try to answer as briefly as possible and only with respect to statistics. not dsp. In statistics, if you have a nice pdf such as the normal distribution, then maximizing the likelihood is ...
mark leeds's user avatar
  • 1,117
6 votes
Accepted

Why Does the Median Filter Minimize the Absolute Value Error $L_1$ Cost Function?

Given a set of values $ {\left\{ {s}_{i} \right\}}_{i = 1}^{N} $, we're basically after: $$ \arg \min_{x} \sum_{i = 1}^{N} \left| {s}_{i} - x \right| $$ One should notice that $ \frac{\mathrm{d} \left ...
Royi's user avatar
  • 19.7k
5 votes

Adaptive filtering: Optimum filter length and delay

In order to be able to choose an optimal value for the delay $\Delta$ it's important to understand how the system works. The purpose of the delay is to decorrelate the desired signal $s(n)$ and the ...
Matt L.'s user avatar
  • 90.4k
5 votes

Derivative with respect to complex conjugate

I felt I needed to write an additional answer to try to clear my mind about the question. Here is the try, step by step. Caveat: for simplicity, I used the same notation $C$ of a function of reals $u$...
Laurent Duval's user avatar
5 votes

How to Formulate a Constraint Which Ensures All Variables Have the Same Sign

The solution from the blog you linked goes as following (Coordinating Variable Signs by Paul Rubin, Web Archive): Someone asked me today (or yesterday, depending on whose time zone you go by) how to ...
Royi's user avatar
  • 19.7k
5 votes
Accepted

Quadratic Programming with Linear Equality Constraints

Let's solve a more general problem (Least Squares with Linear Equality Constraints): $$ \begin{alignat*}{3} \arg \min_{x} & \quad & \frac{1}{2} \left\| A x - b \right\|_{2}^{2} \\ \text{...
Royi's user avatar
  • 19.7k
5 votes

Why expected value is optimal?

Nothing is "optimal" in an absolute sense. Something is or is not optimal according to a defined, precise criterion. Say you have a real random variable $X$. You want to find a number $m$ ...
MBaz's user avatar
  • 15.3k
4 votes

What Is the Best First Order IIR (AR Filter) Approximation to a Moving Average Filter (FIR Filter)?

Based in experimental tests with k in range (2 to 100) the best fit (sum squared error) gives a relation of alfa = 1/k^0.865 ...
chiliwili69's user avatar
4 votes

Derivative with respect to complex conjugate

I have found an alternate answer which is very simple and comprehensive, so thinking to share with all. In order to differentiate an expression $f(z)$ with respect to a complex $z$, the Cauchy-...
Sal's user avatar
  • 163
4 votes
Accepted

On the Measurement Matrix Used for Compressing Sensing

It is indeed possible to formulate this setting in terms of matrix-vector products. First, let us re-formulate your $x$ (notice throughout that I use bold letters for vectors and matrices): $$x = \...
Thomas Arildsen's user avatar
4 votes
Accepted

How to Solve Non Blind Image Deblurring with Total Variation Prior Using ADMM?

Formulation of the Problem I am solving the problem under the following assumptions: The blurring operator is Linear and Spatially Invariant (Hence applied by convolution). The blurring operator is ...
Royi's user avatar
  • 19.7k
4 votes

Optimization of square matrix multiplied with another matrix to have the final result a unitary matrix

Could it be that you are indeed looking for the closest orthogonal matrix $Y$? Then, there is a solution which involves computing the square root of $ D^TD$ . If $E=(D^TD)^{1/2}$ were invertible, the ...
Laurent Duval's user avatar
4 votes
Accepted

Adding Variance \ Weights Information When Solving a Basis Pursuit Denoising Problem (BPDN)

Your formulation: $$ \arg \min_{\boldsymbol{x}} \frac{1}{2} {\left\| A \boldsymbol{x} - \boldsymbol{y} \right\|}_{2}^{2} + \lambda {\left\| \boldsymbol{x} \right\|}_{1} $$ Has 2 elements: The ...
Royi's user avatar
  • 19.7k
4 votes
Accepted

Gradient descent algorithm not converging

Your step size is too large. The upper limit $2/\lambda_{max}$ for the step size $\mu$ is valid if the update is defined as $$\mathbf{w}_{k+1}=\mathbf{w}_{k}-\frac{\mu}{2}\nabla J(\mathbf{w}_k)$$ The ...
Matt L.'s user avatar
  • 90.4k
3 votes
Accepted

How to Use the DFT (FFT) to Solve a Least Squares Regularization Problem (Inverse Problem)?

The question really depends on $ f \left( \cdot \right) $. Yet in order to show how to use FFT we can even use 1D signals. Let's rewrite the problem: $$ \hat{x} = \arg \min_{x} \frac{1}{2} \left\| K ...
Royi's user avatar
  • 19.7k
3 votes
Accepted

Reference Code for Positive Basis Pursuit Denoising

SparseLab should be able to solve the positivity-constrained problem: SparseLab (Stanford) - Seeking Sparse Solutions to Linear System of Equations. See Donoho & Tanner, "Precise ...
Thomas Arildsen's user avatar
3 votes

Best Metric to Compare Sparsity of Vectors

Norms like $\ell_p$, $p \ge 1$, or quasi-norms ($0<p< 1$) are all $1$-homogeneous: $\ell_p(\lambda x) = |\lambda|\ell_p( x)$. Which is not the case for the $\ell_0$ count measure, which is scale ...
Laurent Duval's user avatar
3 votes

What Is the Best First Order IIR (AR Filter) Approximation to a Moving Average Filter (FIR Filter)?

I stumbled upon this old question and I would like to share my solution. As mentioned in other answers, there is no analytical solution, but the function to be minimized behaves nicely and the optimal ...
Matt L.'s user avatar
  • 90.4k
3 votes

Curve Fit of Step Function with Boundary on the 2nd Derivative

You can make a discretized regularized linear equation system. $\bf d$ is the original signal $\bf v$ is what we add to the signal, (the additive change) $\bf v+d$ is the result. So there are two ...
mathreadler's user avatar
3 votes
Accepted

Difference Between Iteratively Reweighted Least Squares (IRLS) and Sequential Quadratic Programming?

SQP is a method for solving smooth (objective and constraint functions are at least twice differentiable) constrained nonlinear optimization problems. It solves a series of quadratic programming ...
Brian Borchers's user avatar
3 votes

Solving LASSO ($ {L}_{1} $ Regularized Least Squares) with Gradient Descent

It can easily solved by the Gradient Descent Framework with one adjustment in order to take care of the $ {L}_{1} $ norm term. Since the $ {L}_{1} $ norm isn't smooth you need to use the concept of ...
Royi's user avatar
  • 19.7k
3 votes
Accepted

Fastest Available Algorithm to Blur an Image (Low Pass Filter)

The fastest blur would be Box Blur. You can implement it using Running Sum. I think Intel FilterBoxBorder works in that manner. If you'd like you can do a few ...
Royi's user avatar
  • 19.7k
3 votes

Fastest Available Algorithm to Blur an Image (Low Pass Filter)

I have implemented a fast 5x5 Gaussian-blur in C++ and compared the performance to OpenCV on Raspberry Pi 3B+ running 32bit Raspbian OS. The function uses all the 4 cores of the Raspberry Pi and works ...
zana zakaryaie nejad's user avatar
3 votes
Accepted

Why does it seem most of people will optimize the downlink rate,not the uplink rate?

This simply reflects the load asymmetry of real world applications. On most networks, traffic is dominated by content streaming, which is a download activity. Netflix alone accounts for 15% of the ...
Hilmar's user avatar
  • 45.4k
3 votes
Accepted

Resources on Solving Convex Optimization Problems in the Compressed Sensing Field

There are few options: Stephen Boyd, Lieven Vandenberghe - Convex Optimization. This is the classic in this field. Very well written book. Also have a look on other papers of Boyd on similar subjects ...
Royi's user avatar
  • 19.7k

Only top scored, non community-wiki answers of a minimum length are eligible