# NMF for BSS, prevent zero valued sources

I'm using NMF (Non-negative Matrix Factorization) on a Blind Source Separation application and using sparsity, decorrelation and smoothness regularization on the Frobenius Norm Cost Function using ALS algorithm.

So, the cost function is:

$$D_F(\boldsymbol Y||\boldsymbol A\boldsymbol X)=\frac{1}{2}||\boldsymbol Y-\boldsymbol A\boldsymbol X||_2^2+\alpha_{sA}||A||_1+\alpha_{sX}||X||_1+\alpha_{dA}trace(\boldsymbol A\boldsymbol 1_{J\times J}\boldsymbol A^T)+\alpha_{dX}trace(\boldsymbol X^T\boldsymbol 1_{J\times J}\boldsymbol X)+\frac{\alpha_{rA}}{2}||\boldsymbol A\boldsymbol L_A||^2_2+\frac{\alpha_{rX}}{2}||\boldsymbol L_X\boldsymbol X||^2_2$$

Minimized with respect to $$\boldsymbol X$$ and $$\boldsymbol A$$ where $$\boldsymbol L_x$$ and $$\boldsymbol L_A$$ are:

$$\boldsymbol L_x=\boldsymbol L_A=\begin{bmatrix} 1&-1&0&\dots&0\\ 0&1&-1&\dots&0\\ \vdots &\vdots &\vdots &\ddots &0\\ 0&0&\dots&1&-1 \end{bmatrix}$$

$$\boldsymbol A$$ and $$\boldsymbol X$$ are updated by:

$$\boldsymbol A\rightarrow(\boldsymbol Y\boldsymbol X^T-\alpha_{sA}\boldsymbol 1_{J\times J})(\boldsymbol X\boldsymbol X^T+\alpha_{dA}\boldsymbol 1_{J\times J}+\alpha_{rA}\boldsymbol L_A\boldsymbol L_A^T)^{-1}$$

$$\boldsymbol X\rightarrow(\boldsymbol A^T\boldsymbol A+\alpha_{dX}\boldsymbol 1_{J\times J}+\alpha_{rX}\boldsymbol L_A\boldsymbol L_A^T)^{-1}(\boldsymbol A^T\boldsymbol Y-\alpha_{sX}\boldsymbol 1_{J\times J})$$

I'm already using multi-start initialization of the matrices $$\boldsymbol A$$ and $$\boldsymbol X$$ and, at each iteration, if one of the matrices with is inverted on the equations above has zero determinant, $$\boldsymbol A$$ and $$\boldsymbol X$$ are reinitialized.

Then the algoritm stills get a source with all zeros sometimes, that doesn't happens when $$\alpha_s$$, $$\alpha_d$$ and $$\alpha_r$$ are low enough, or for low number of iterations, but this way, the algorithm doesn't does it's job.

How can I prevent the apperence of all zero sources?

• What are you minimizing $D_F(\boldsymbol Y||\boldsymbol A\boldsymbol X)$ with respect to? Only $\boldsymbol X$? Jan 5 at 17:21
• @mhdadk I just edited the question, it's minimizing with respect to both $\boldsymbol X$ and $\boldsymbol A$ Jan 6 at 0:56
• I'd say that's a lot of factors in your cost function. It's hard to choose parameters (e.g., the $\alpha$'s in your equation) for complicated cost functions appropriately. You might be better off starting with one regularizer until you have a good feel for it's behavior, and then adding more terms one at a time. Jan 6 at 2:19