I meet a confusing thing in image processing recently....

Assume the image $x \in \mathbb{R}^n$, with its derivative (difference) matrix: $D^+ = \begin{bmatrix} D_h \\ Dv \end{bmatrix} \in \mathbb{R}^{2n\times n}$ ($+$ means forward difference), also equal to $\nabla$. Therefore, it is natural to define the divergence: $\triangle = \nabla \cdot \nabla$.

I have seen some papers use $div = \triangle = D_h^-D_h^+ + D_v^-D_v^+ \in \mathbb{R}^{n\times n}$, where $-$ denotes the backward difference.

Here is my question: assume I want to calculate the $\frac{\partial\|\nabla x -p\|_2^2}{\partial x}$ where $p\in \mathbb{R}^{2n\times 1}$ is a vector not related to $x$, what is the result? I have seen some authors use $\nabla\cdot (\nabla x -p)$.

However, if writing the $\nabla$ as matrix form $D$ as I have introduced before, $D^T$ is exactly adjoint of gradient, not backward difference. Hence $-\triangle x$ would appear! So what is the right formula? Could anyone tell me?

  • $\begingroup$ What is p, and please specify dimensions of all the matrices and vectors $\endgroup$ – Dsp guy sam Apr 21 '20 at 6:36
  • $\begingroup$ @Dspguysam Thanks for your advise. I have edited the question. $\endgroup$ – stander Qiu Apr 21 '20 at 6:46
  • $\begingroup$ $\nabla x$ will be a vector of dimension 2n and you are subtracting that to a matrix $p$ of dimension 2nxn? Did I get that right? $\endgroup$ – Dsp guy sam Apr 21 '20 at 6:59
  • $\begingroup$ @Dspguysam Yes, you are right. I have already marked the dimension of $p$. $\endgroup$ – stander Qiu Apr 21 '20 at 8:15
  • $\begingroup$ a matrix times a vector will result in a vector, and you can only subtract or add vectors of same dimension in a space, therefore $p$ should be a vector, not a matrix, and the dimension of "vector" $p$ should be the same as number of rows of $\nabla$, $p$ should be $\mathbb{R}^{2n}$ not $\mathbb{R}^{2n*n}$ $\endgroup$ – Dsp guy sam Apr 21 '20 at 8:29

Consider the expansion of the term below $$\|\nabla x -p\|_2^2 = (\nabla x -p)^T(\nabla x -p)$$ $$\|\nabla x -p\|_2^2 = (x^T\nabla^T -p^T)(\nabla x -p)$$ $$\|\nabla x -p\|_2^2 = (x^T\nabla^T\nabla x - x^T\nabla^T p - p^T\nabla x +p^Tp)$$

Now consider the following basic definition:

$$\frac{\partial(A x)}{\partial x} = A^T$$ $$\frac{\partial(x^TA)}{\partial x} = A$$

Now applying the above two definitions together with the expansion of the objective above to differentiate the objective we have

$$\frac{\partial\|\nabla x -p\|_2^2}{\partial x} = 2\nabla^T\nabla x - 2\nabla^Tp$$

$$\frac{\partial\|\nabla x -p\|_2^2}{\partial x} = 2\nabla^T(\nabla x - p)$$

since $\nabla^T = \nabla$, therefore we have

$$\frac{\partial\|\nabla x -p\|_2^2}{\partial x} = 2\nabla(\nabla x - p)$$ the constant 2 is just a constant, so the result we have is consistent with the ones that authors are using, its simply a consequence of vector differentiation

  • $\begingroup$ The question is here: where is the conclusion of $\nabla^T = \nabla$? When we express it in matrix form, for example, let $D$ represents the forward difference, $D^T \neq D$ sinc $D_h^T \neq D_h$ $\endgroup$ – stander Qiu Apr 21 '20 at 8:37
  • $\begingroup$ $D^T$ is the adjoint of gradient. If you implement it in programming language, $D_h^T = -D^-_h$ $\endgroup$ – stander Qiu Apr 21 '20 at 8:39
  • $\begingroup$ For this to be a valid definition $\triangle = \nabla \cdot \nabla$, firstly $\nabla$should be a square matrix and is $\triangle$ should be positive semidefinite, in that case $\nabla^T = \nabla$ $\endgroup$ – Dsp guy sam Apr 21 '20 at 8:44
  • $\begingroup$ I agree. In mathematics this is valid. But how about changing the $\nabla$ to $D$? The things goes different. $\endgroup$ – stander Qiu Apr 21 '20 at 8:51
  • $\begingroup$ this is all maths. According to the definition you provide D is not a square matrix, then it can never be substituted as $\nabla$ , which has to be a square matrix, based on the definitions you provide as I explained above, If you want give me an objective function in terms of D that you want to differentiate with respect to x. We will have no problem doing it. Currently the objective is in terms of $\nabla$ for which the result is consistent. D and $\nabla$ have to have equal dimesnions first to be equated! $\endgroup$ – Dsp guy sam Apr 21 '20 at 8:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.