# Calculate the Derivative of Gradient Field of an Image

I meet a confusing thing in image processing recently....

Assume the image $$x \in \mathbb{R}^n$$, with its derivative (difference) matrix: $$D^+ = \begin{bmatrix} D_h \\ Dv \end{bmatrix} \in \mathbb{R}^{2n\times n}$$ ($$+$$ means forward difference), also equal to $$\nabla$$. Therefore, it is natural to define the divergence: $$\triangle = \nabla \cdot \nabla$$.

I have seen some papers use $$div = \triangle = D_h^-D_h^+ + D_v^-D_v^+ \in \mathbb{R}^{n\times n}$$, where $$-$$ denotes the backward difference.

Here is my question: assume I want to calculate the $$\frac{\partial\|\nabla x -p\|_2^2}{\partial x}$$ where $$p\in \mathbb{R}^{2n\times 1}$$ is a vector not related to $$x$$, what is the result? I have seen some authors use $$\nabla\cdot (\nabla x -p)$$.

However, if writing the $$\nabla$$ as matrix form $$D$$ as I have introduced before, $$D^T$$ is exactly adjoint of gradient, not backward difference. Hence $$-\triangle x$$ would appear! So what is the right formula? Could anyone tell me?

• What is p, and please specify dimensions of all the matrices and vectors Apr 21, 2020 at 6:36
• @Dspguysam Thanks for your advise. I have edited the question. Apr 21, 2020 at 6:46
• $\nabla x$ will be a vector of dimension 2n and you are subtracting that to a matrix $p$ of dimension 2nxn? Did I get that right? Apr 21, 2020 at 6:59
• @Dspguysam Yes, you are right. I have already marked the dimension of $p$. Apr 21, 2020 at 8:15
• a matrix times a vector will result in a vector, and you can only subtract or add vectors of same dimension in a space, therefore $p$ should be a vector, not a matrix, and the dimension of "vector" $p$ should be the same as number of rows of $\nabla$, $p$ should be $\mathbb{R}^{2n}$ not $\mathbb{R}^{2n*n}$ Apr 21, 2020 at 8:29

Consider the expansion of the term below $$\|\nabla x -p\|_2^2 = (\nabla x -p)^T(\nabla x -p)$$ $$\|\nabla x -p\|_2^2 = (x^T\nabla^T -p^T)(\nabla x -p)$$ $$\|\nabla x -p\|_2^2 = (x^T\nabla^T\nabla x - x^T\nabla^T p - p^T\nabla x +p^Tp)$$

Now consider the following basic definition:

$$\frac{\partial(A x)}{\partial x} = A^T$$ $$\frac{\partial(x^TA)}{\partial x} = A$$

Now applying the above two definitions together with the expansion of the objective above to differentiate the objective we have

$$\frac{\partial\|\nabla x -p\|_2^2}{\partial x} = 2\nabla^T\nabla x - 2\nabla^Tp$$

$$\frac{\partial\|\nabla x -p\|_2^2}{\partial x} = 2\nabla^T(\nabla x - p)$$

since $$\nabla^T = \nabla$$, therefore we have

$$\frac{\partial\|\nabla x -p\|_2^2}{\partial x} = 2\nabla(\nabla x - p)$$ the constant 2 is just a constant, so the result we have is consistent with the ones that authors are using, its simply a consequence of vector differentiation

• The question is here: where is the conclusion of $\nabla^T = \nabla$? When we express it in matrix form, for example, let $D$ represents the forward difference, $D^T \neq D$ sinc $D_h^T \neq D_h$ Apr 21, 2020 at 8:37
• $D^T$ is the adjoint of gradient. If you implement it in programming language, $D_h^T = -D^-_h$ Apr 21, 2020 at 8:39
• For this to be a valid definition $\triangle = \nabla \cdot \nabla$, firstly $\nabla$should be a square matrix and is $\triangle$ should be positive semidefinite, in that case $\nabla^T = \nabla$ Apr 21, 2020 at 8:44
• I agree. In mathematics this is valid. But how about changing the $\nabla$ to $D$? The things goes different. Apr 21, 2020 at 8:51
• this is all maths. According to the definition you provide D is not a square matrix, then it can never be substituted as $\nabla$ , which has to be a square matrix, based on the definitions you provide as I explained above, If you want give me an objective function in terms of D that you want to differentiate with respect to x. We will have no problem doing it. Currently the objective is in terms of $\nabla$ for which the result is consistent. D and $\nabla$ have to have equal dimesnions first to be equated! Apr 21, 2020 at 8:56