I have to find the gradient of the following term with respect to $X_{1}$:

$\|\Phi\circ(X_{1}-X_{2})-u\|_F^2$ ,

where $u\in\mathbb{R}^{n}$; $X_{1}, X_{2}\in\mathbb{R}^{N\times J}$ and $\Phi\in\mathbb{R}^{n\times NJ}$. The $\circ$ operation is defined as follows:

$\Phi\circ X=\Sigma_{i=1}^{J}\Phi_{i} x_{i}$,

where $x_i$ are the columns of $X$ and $\Phi_{i}$ are $n\times N$ submatrices of $\Phi$.

I know that $\frac{\partial}{\partial X}\|X\|_{F}^{2}=2X$, but the $\circ$ operation is making it rather complicated. Can somebody help?

P.S. Also, would changing the Frobenius norm to l2 norm make any difference here (since $(\Phi\circ(X_{1}-X_{2})-u)$ is a column vector)?

  • $\begingroup$ Use linearity to simplify terms that depend on X1 then use the chain rule. $\endgroup$ – Mark Borgerding Nov 7 '14 at 13:01
  • $\begingroup$ Yes, I got the answer. I've posted it. $\endgroup$ – nemo Nov 8 '14 at 15:50

This is the result of taking the derivative of $\|\Phi\circ(X_{1}-X_{2})-u\|_F^2$ w.r.t. $X_{1}$:

$[\Phi^{1'}(\Phi\circ(X_{1}-X_{2})-u), ...,\Phi^{J'}(\Phi\circ(X_{1}-X_{2})-u)]$ .

This can be got by first finding the expression for the norm, and then taking the matrix derivative (w.r.t. $X_{1}$) to get an $N \times J$ matrix. This matrix is then represented in terms of the sub matrices $\Phi^{i}$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.