2
$\begingroup$

The signal $y(n)$ can be modeled in the time-domain as

$$y(n) = \sum_{i=0}^{L-1}h(i) x(n-i) + v(t) \tag{1}$$

where $\mathbf{h} = [h_1,h_2,\ldots,h_L]^T$ is the coefficient of length $L-1$, and $v(t)$ is the additive White Gaussian noise with variance $\sigma^2_v$.

In the Eq(20) found in the paper, http://www.eurasip.org/Proceedings/Eusipco/Eusipco2008/papers/1569101936.pdf

the eq has the term ${||h||}^p_p$ and the Authors mention that here $0<p \le 1$.

Consider an array $h = [1,0.2,0,0.5]$. What is meant by ${||h||}^p_p$ Is it the difference between the elements of $h$? What would be the meaning when p =0, p=1, and p=2?

The Authors in Eq(24)present the derivative of the cost function computed in Eq(20). The derivative is taken with respect to $h$ parameter. In that, they use the term $p\lambda \tilde{h}$ where $\tilde{h}$ is the sparse coefficient. I think here $p$ may not be zero.

$\endgroup$
2
$\begingroup$

The symbol ${||h||}_p$ refers to the p-norm of the vector. It is calculated as

$${||h||}_p=\sqrt[p]{\left|h_1\right|^p+\left|h_2\right|^p+...+\left|h_n\right|^p}$$

When they write ${||h||}^p_p$, it is just that multiplied by itself $p$ times, so that the root disappears. Mathematically:

$${||h||}^p_p=\sum\limits_{i=1}^n \left|h_i\right|^p$$

$\endgroup$
  • $\begingroup$ Thank you for your answer. I have one last doubt which is how does this term get included in the cost function $J()$? I saw in the paper that the prior of $h$ is the Laplace pdf taken as the prior. If so, then how only the term $\lambda ||\mathbf{h}||$ comes in the expression, is unclear to me. Would it be possible to clarify this as well? $\endgroup$ – Ria George Apr 21 '17 at 5:06
  • $\begingroup$ Well, that's a whole different question. It would be cool if you wrote another one asking that instead of doing it in a comment, as it seems rather laborious to answer that. $\endgroup$ – Tendero Apr 21 '17 at 10:34
  • $\begingroup$ Sorry to sound noisy but I have a doubt on what the meaning of this expression be for $l_1$ norm, ${||\mathbf{h}||}^2_{l_1}$? Based on your answer, I am not sure if this is the proper interpretation of the term : Step 1 is the evaluation of the $l_1$ norm which would be for this term, ${||\mathbf{h}||}_{l_1} = |h_1| + |h_2| +...+|h_n|$ and then what is the square term? Can you please help? $\endgroup$ – Ria George Apr 27 '17 at 17:49
  • 1
    $\begingroup$ ${||\mathbf{h}||}^2_1 = (|h_1| + |h_2| +...+|h_n|)^2$. $\endgroup$ – Tendero Apr 27 '17 at 17:53
1
$\begingroup$

To add on @Tendero, the expression $\sum_k x_k^p$ is sometimes called the "power $p$-norm" when $p\ge 1$. Most often, you can see mentions of the "squared $\ell_2$ norm" or "$\ell_2$ norm squared". For $p=1$, the exponent does not modify the computation, so it is just called the $\ell_1$ norm.

The use of the power is often more convenient mathematically and computationally: having a $p$-root ($\sqrt[p]{\cdot}$) can be cumbersome in computing derivatives to find extrema.

When $p\ge 1$, it satisfies all the norm axioms. But when $0<p<1$, the triangle inequality is not satisfied anymore, so it should not be called a norm. The correct denomination is a quasi-norm, with a modulus of concavity modulus $K$ such that

$$\ell_p(x+y) \le K(\ell_p(x) +\ell_p(y))\,. $$

When $p=0$, this is not a norm nor a quasi-norm anymore. It can be called cardinality function, sparsity, count index.

In signal processing where sparsity is considered useful, $\ell_0$ is usually the target to minimize: number of non-zero samples, number of taps for a filter.

However, it is quite intractable too (not differentiable). Under some theoretical conditions, the minimization of an $\ell_0$ penalty can be replaced by an $\ell_1$ penalty, the "last" convex $\ell_p^p$ term.

However, more and more works address non-convex penalties ($p<1$) that better approximate $\ell_0$.

Finally, in a Bayesian context, the prior of a Laplacian distribution can be encapsulated in an $\ell_1$ penalty, as the Gaussian distribution can be encapsulated in an $\ell_2$ penalty, see for instance Why is Laplace prior producing sparse solutions?.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.