I'm using NMF (Non-negative Matrix Factorization) on a Blind Source Separation application and using sparsity, decorrelation and smoothness regularization on the Frobenius Norm Cost Function using ALS algorithm.

So, the cost function is:

$$D_F(\boldsymbol Y||\boldsymbol A\boldsymbol X)=\frac{1}{2}||\boldsymbol Y-\boldsymbol A\boldsymbol X||_2^2+\alpha_{sA}||A||_1+\alpha_{sX}||X||_1+\alpha_{dA}trace(\boldsymbol A\boldsymbol 1_{J\times J}\boldsymbol A^T)+\alpha_{dX}trace(\boldsymbol X^T\boldsymbol 1_{J\times J}\boldsymbol X)+\frac{\alpha_{rA}}{2}||\boldsymbol A\boldsymbol L_A||^2_2+\frac{\alpha_{rX}}{2}||\boldsymbol L_X\boldsymbol X||^2_2$$

Minimized with respect to $\boldsymbol X$ and $\boldsymbol A$ where $\boldsymbol L_x$ and $\boldsymbol L_A$ are:

$$\boldsymbol L_x=\boldsymbol L_A=\begin{bmatrix} 1&-1&0&\dots&0\\ 0&1&-1&\dots&0\\ \vdots &\vdots &\vdots &\ddots &0\\ 0&0&\dots&1&-1 \end{bmatrix}$$

$\boldsymbol A$ and $\boldsymbol X$ are updated by:

$$\boldsymbol A\rightarrow(\boldsymbol Y\boldsymbol X^T-\alpha_{sA}\boldsymbol 1_{J\times J})(\boldsymbol X\boldsymbol X^T+\alpha_{dA}\boldsymbol 1_{J\times J}+\alpha_{rA}\boldsymbol L_A\boldsymbol L_A^T)^{-1}$$

$$\boldsymbol X\rightarrow(\boldsymbol A^T\boldsymbol A+\alpha_{dX}\boldsymbol 1_{J\times J}+\alpha_{rX}\boldsymbol L_A\boldsymbol L_A^T)^{-1}(\boldsymbol A^T\boldsymbol Y-\alpha_{sX}\boldsymbol 1_{J\times J})$$

I'm already using multi-start initialization of the matrices $\boldsymbol A$ and $\boldsymbol X$ and, at each iteration, if one of the matrices with is inverted on the equations above has zero determinant, $\boldsymbol A$ and $\boldsymbol X$ are reinitialized.

Then the algoritm stills get a source with all zeros sometimes, that doesn't happens when $\alpha_s$, $\alpha_d$ and $\alpha_r$ are low enough, or for low number of iterations, but this way, the algorithm doesn't does it's job.

How can I prevent the apperence of all zero sources?

  • $\begingroup$ What are you minimizing $D_F(\boldsymbol Y||\boldsymbol A\boldsymbol X)$ with respect to? Only $\boldsymbol X$? $\endgroup$
    – mhdadk
    Commented Jan 5, 2023 at 17:21
  • $\begingroup$ @mhdadk I just edited the question, it's minimizing with respect to both $\boldsymbol X$ and $\boldsymbol A$ $\endgroup$ Commented Jan 6, 2023 at 0:56
  • 1
    $\begingroup$ I'd say that's a lot of factors in your cost function. It's hard to choose parameters (e.g., the $\alpha$'s in your equation) for complicated cost functions appropriately. You might be better off starting with one regularizer until you have a good feel for it's behavior, and then adding more terms one at a time. $\endgroup$
    – Gillespie
    Commented Jan 6, 2023 at 2:19


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.