I meet a confusing thing in image processing recently....

Assume the image $x \in \mathbb{R}^n$, with its derivative (difference) matrix: $D^+ = \begin{bmatrix} D_h \\ Dv \end{bmatrix} \in \mathbb{R}^{2n\times n}$ ($+$ means forward difference), also equal to $\nabla$. Therefore, it is natural to define the divergence: $\triangle = \nabla \cdot \nabla$.

I have seen some papers use $div = \triangle = D_h^-D_h^+ + D_v^-D_v^+ \in \mathbb{R}^{n\times n}$, where $-$ denotes the backward difference.

Here is my question: assume I want to calculate the $\frac{\partial\|\nabla x -p\|_2^2}{\partial x}$ where $p\in \mathbb{R}^{2n\times 1}$ is a vector not related to $x$, what is the result? I have seen some authors use $\nabla\cdot (\nabla x -p)$.

However, if writing the $\nabla$ as matrix form $D$ as I have introduced before, $D^T$ is exactly adjoint of gradient, not backward difference. Hence $-\triangle x$ would appear! So what is the right formula? Could anyone tell me?

  • $\begingroup$ What is p, and please specify dimensions of all the matrices and vectors $\endgroup$ Apr 21, 2020 at 6:36
  • $\begingroup$ @Dspguysam Thanks for your advise. I have edited the question. $\endgroup$ Apr 21, 2020 at 6:46
  • $\begingroup$ $\nabla x$ will be a vector of dimension 2n and you are subtracting that to a matrix $p$ of dimension 2nxn? Did I get that right? $\endgroup$ Apr 21, 2020 at 6:59
  • $\begingroup$ @Dspguysam Yes, you are right. I have already marked the dimension of $p$. $\endgroup$ Apr 21, 2020 at 8:15
  • $\begingroup$ a matrix times a vector will result in a vector, and you can only subtract or add vectors of same dimension in a space, therefore $p$ should be a vector, not a matrix, and the dimension of "vector" $p$ should be the same as number of rows of $\nabla$, $p$ should be $\mathbb{R}^{2n}$ not $\mathbb{R}^{2n*n}$ $\endgroup$ Apr 21, 2020 at 8:29

1 Answer 1


Consider the expansion of the term below $$\|\nabla x -p\|_2^2 = (\nabla x -p)^T(\nabla x -p)$$ $$\|\nabla x -p\|_2^2 = (x^T\nabla^T -p^T)(\nabla x -p)$$ $$\|\nabla x -p\|_2^2 = (x^T\nabla^T\nabla x - x^T\nabla^T p - p^T\nabla x +p^Tp)$$

Now consider the following basic definition:

$$\frac{\partial(A x)}{\partial x} = A^T$$ $$\frac{\partial(x^TA)}{\partial x} = A$$

Now applying the above two definitions together with the expansion of the objective above to differentiate the objective we have

$$\frac{\partial\|\nabla x -p\|_2^2}{\partial x} = 2\nabla^T\nabla x - 2\nabla^Tp$$

$$\frac{\partial\|\nabla x -p\|_2^2}{\partial x} = 2\nabla^T(\nabla x - p)$$

since $\nabla^T = \nabla$, therefore we have

$$\frac{\partial\|\nabla x -p\|_2^2}{\partial x} = 2\nabla(\nabla x - p)$$ the constant 2 is just a constant, so the result we have is consistent with the ones that authors are using, its simply a consequence of vector differentiation

  • $\begingroup$ The question is here: where is the conclusion of $\nabla^T = \nabla$? When we express it in matrix form, for example, let $D$ represents the forward difference, $D^T \neq D$ sinc $D_h^T \neq D_h$ $\endgroup$ Apr 21, 2020 at 8:37
  • $\begingroup$ $D^T$ is the adjoint of gradient. If you implement it in programming language, $D_h^T = -D^-_h$ $\endgroup$ Apr 21, 2020 at 8:39
  • $\begingroup$ For this to be a valid definition $\triangle = \nabla \cdot \nabla$, firstly $\nabla$should be a square matrix and is $\triangle$ should be positive semidefinite, in that case $\nabla^T = \nabla$ $\endgroup$ Apr 21, 2020 at 8:44
  • $\begingroup$ I agree. In mathematics this is valid. But how about changing the $\nabla$ to $D$? The things goes different. $\endgroup$ Apr 21, 2020 at 8:51
  • $\begingroup$ this is all maths. According to the definition you provide D is not a square matrix, then it can never be substituted as $\nabla$ , which has to be a square matrix, based on the definitions you provide as I explained above, If you want give me an objective function in terms of D that you want to differentiate with respect to x. We will have no problem doing it. Currently the objective is in terms of $\nabla$ for which the result is consistent. D and $\nabla$ have to have equal dimesnions first to be equated! $\endgroup$ Apr 21, 2020 at 8:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.