7
votes
Curve Fit of Step Function with Boundary on the 2nd Derivative
Hmmmmmmmmm, interesting question.
Since you want to use the second derivative as your criteria, it would seem that you would want to have the maximum second derivative absolutie value for as short of ...
- 7,522
7
votes
Derivative with respect to complex conjugate
That's a trick which you will also find in a DSP context, that's why I choose to provide an answer here. It is related to the Wirtinger derivative, and you can find more details about it in this ...
- 88.8k
6
votes
Best Metric to Compare Sparsity of Vectors
I am sorry I cannot comment your answer due to my low reputation. Gini and your suggested sparsity ratio ($l_1(x)/l_2(x)$) both give me the same value for $\lambda$. But
The problem I still see is ...
- 223
6
votes
Accepted
Why use parametric based estimation methods - confusion regarding terms
Hi: I'll try to answer as briefly as possible and only with respect to statistics. not dsp.
In statistics, if you have a nice pdf such as the normal distribution, then maximizing the likelihood is ...
- 1,107
6
votes
Why Do Most of The Papers Use the Frobenius Norm for Denoising?
The Frobenius Norm has multiple equivalent definitions – the useful for error measure is probably this one:
$$\left\|M\right\|_\mathrm F = \sqrt{\sum_{p\in M}\left\lvert p\right\rvert^2}$$
That's a ...
- 29.6k
6
votes
Accepted
Why Does the Median Filter Minimize the Absolute Value Error $L_1$ Cost Function?
Given a set of values $ {\left\{ {s}_{i} \right\}}_{i = 1}^{N} $, we're basically after:
$$ \arg \min_{x} \sum_{i = 1}^{N} \left| {s}_{i} - x \right| $$
One should notice that $ \frac{\mathrm{d} \left ...
- 19.3k
5
votes
Adaptive filtering: Optimum filter length and delay
In order to be able to choose an optimal value for the delay $\Delta$ it's important to understand how the system works. The purpose of the delay is to decorrelate the desired signal $s(n)$ and the ...
- 88.8k
5
votes
Accepted
Lagrange Multipliers Optimization - Complex Functions
I found the following in Charles Therrien's "Discrete Random Signals and Statistical Signal Processing" in one of the Appendicies.
Say you have the function $Q(a)$ you wish to minimize such that $C(a)...
- 2,841
5
votes
Derivative with respect to complex conjugate
I felt I needed to write an additional answer to try to clear my mind about the question. Here is the try, step by step. Caveat: for simplicity, I used the same notation $C$ of a function of reals $u$...
- 31.7k
5
votes
How to Formulate a Constraint Which Ensures All Variables Have the Same Sign
The solution from the blog you linked goes as following (Coordinating Variable Signs by Paul Rubin, Web Archive):
Someone asked me today (or yesterday, depending on whose time zone you
go by) how to ...
- 19.3k
5
votes
Accepted
Quadratic Programming with Linear Equality Constraints
Let's solve a more general problem (Least Squares with Linear Equality Constraints):
$$
\begin{alignat*}{3}
\arg \min_{x} & \quad & \frac{1}{2} \left\| A x - b \right\|_{2}^{2} \\
\text{...
- 19.3k
4
votes
What Is the Best First Order IIR (AR Filter) Approximation to a Moving Average Filter (FIR Filter)?
Based in experimental tests with k in range (2 to 100) the best fit (sum squared error) gives a relation of alfa = 1/k^0.865
...
- 41
4
votes
Derivative with respect to complex conjugate
I have found an alternate answer which is very simple and comprehensive, so thinking to share with all.
In order to differentiate an expression $f(z)$ with respect to a complex $z$, the Cauchy-...
- 163
4
votes
Accepted
On the Measurement Matrix Used for Compressing Sensing
It is indeed possible to formulate this setting in terms of matrix-vector products. First, let us re-formulate your $x$ (notice throughout that I use bold letters for vectors and matrices):
$$x = \...
- 1,322
4
votes
Accepted
How to Solve Non Blind Image Deblurring with Total Variation Prior Using ADMM?
Formulation of the Problem
I am solving the problem under the following assumptions:
The blurring operator is Linear and Spatially Invariant (Hence applied by convolution).
The blurring operator is ...
- 19.3k
4
votes
Optimization of square matrix multiplied with another matrix to have the final result a unitary matrix
Could it be that you are indeed looking for the closest orthogonal matrix $Y$? Then, there is a solution which involves computing the square root of $ D^TD$ . If $E=(D^TD)^{1/2}$ were invertible, the ...
- 31.7k
4
votes
Accepted
Adding Variance \ Weights Information When Solving a Basis Pursuit Denoising Problem (BPDN)
Your formulation:
$$ \arg \min_{\boldsymbol{x}} \frac{1}{2} {\left\| A \boldsymbol{x} - \boldsymbol{y} \right\|}_{2}^{2} + \lambda {\left\| \boldsymbol{x} \right\|}_{1} $$
Has 2 elements:
The ...
- 19.3k
4
votes
Why expected value is optimal?
Nothing is "optimal" in an absolute sense. Something is or is not optimal according to a defined, precise criterion.
Say you have a real random variable $X$. You want to find a number $m$ ...
- 14.9k
4
votes
Accepted
Gradient descent algorithm not converging
Your step size is too large. The upper limit $2/\lambda_{max}$ for the step size $\mu$ is valid if the update is defined as
$$\mathbf{w}_{k+1}=\mathbf{w}_{k}-\frac{\mu}{2}\nabla J(\mathbf{w}_k)$$
The ...
- 88.8k
3
votes
Accepted
Fitting an IIR filter to a complex transfer function
As I mentioned in a comment, I think that the equation error method is a very good starting point for designing an IIR filter in the frequency domain with prescribed magnitude and phase responses. For ...
- 88.8k
3
votes
Accepted
How to Use the DFT (FFT) to Solve a Least Squares Regularization Problem (Inverse Problem)?
The question really depends on $ f \left( \cdot \right) $.
Yet in order to show how to use FFT we can even use 1D signals.
Let's rewrite the problem:
$$ \hat{x} = \arg \min_{x} \frac{1}{2} \left\| K ...
- 19.3k
3
votes
Best Metric to Compare Sparsity of Vectors
Norms like $\ell_p$, $p \ge 1$, or quasi-norms ($0<p< 1$) are all $1$-homogeneous: $\ell_p(\lambda x) = |\lambda|\ell_p( x)$. Which is not the case for the $\ell_0$ count measure, which is scale ...
- 31.7k
3
votes
Accepted
Filtering performance on Poisson noise with quadratic data-fidelity
For large intensities / large "bins", i.e. "areas for which events are counted and accumulated", Poisson processes lead to nearly Gaussian distributed individual values -- basically, without trying to ...
- 29.6k
3
votes
What Is the Best First Order IIR (AR Filter) Approximation to a Moving Average Filter (FIR Filter)?
I stumbled upon this old question and I would like to share my solution. As mentioned in other answers, there is no analytical solution, but the function to be minimized behaves nicely and the optimal ...
- 88.8k
3
votes
Accepted
Reference Code for Positive Basis Pursuit Denoising
SparseLab should be able to solve the positivity-constrained problem: SparseLab (Stanford) - Seeking Sparse Solutions to Linear System of Equations.
See Donoho & Tanner, "Precise ...
- 1,322
3
votes
Curve Fit of Step Function with Boundary on the 2nd Derivative
You can make a discretized regularized linear equation system.
$\bf d$ is the original signal
$\bf v$ is what we add to the signal, (the additive change)
$\bf v+d$ is the result.
So there are two ...
- 453
3
votes
Accepted
Difference Between Iteratively Reweighted Least Squares (IRLS) and Sequential Quadratic Programming?
SQP is a method for solving smooth (objective and constraint functions are at least twice differentiable) constrained nonlinear optimization problems. It solves a series of quadratic programming ...
- 156
3
votes
Solving LASSO ($ {L}_{1} $ Regularized Least Squares) with Gradient Descent
It can easily solved by the Gradient Descent Framework with one adjustment in order to take care of the $ {L}_{1} $ norm term.
Since the $ {L}_{1} $ norm isn't smooth you need to use the concept of ...
- 19.3k
3
votes
Accepted
Fastest Available Algorithm to Blur an Image (Low Pass Filter)
The fastest blur would be Box Blur.
You can implement it using Running Sum.
I think Intel FilterBoxBorder works in that manner.
If you'd like you can do a few ...
- 19.3k
Only top scored, non community-wiki answers of a minimum length are eligible