I'm using NMF (Non-negative Matrix Factorization) on a Blind Source Separation application and using sparsity, decorrelation and smoothness regularization on the Frobenius Norm Cost Function using ALS algorithm.
So, the cost function is:
$$D_F(\boldsymbol Y||\boldsymbol A\boldsymbol X)=\frac{1}{2}||\boldsymbol Y-\boldsymbol A\boldsymbol X||_2^2+\alpha_{sA}||A||_1+\alpha_{sX}||X||_1+\alpha_{dA}trace(\boldsymbol A\boldsymbol 1_{J\times J}\boldsymbol A^T)+\alpha_{dX}trace(\boldsymbol X^T\boldsymbol 1_{J\times J}\boldsymbol X)+\frac{\alpha_{rA}}{2}||\boldsymbol A\boldsymbol L_A||^2_2+\frac{\alpha_{rX}}{2}||\boldsymbol L_X\boldsymbol X||^2_2$$
Minimized with respect to $\boldsymbol X$ and $\boldsymbol A$ where $\boldsymbol L_x$ and $\boldsymbol L_A$ are:
$$\boldsymbol L_x=\boldsymbol L_A=\begin{bmatrix} 1&-1&0&\dots&0\\ 0&1&-1&\dots&0\\ \vdots &\vdots &\vdots &\ddots &0\\ 0&0&\dots&1&-1 \end{bmatrix}$$
$\boldsymbol A$ and $\boldsymbol X$ are updated by:
$$\boldsymbol A\rightarrow(\boldsymbol Y\boldsymbol X^T-\alpha_{sA}\boldsymbol 1_{J\times J})(\boldsymbol X\boldsymbol X^T+\alpha_{dA}\boldsymbol 1_{J\times J}+\alpha_{rA}\boldsymbol L_A\boldsymbol L_A^T)^{-1}$$
$$\boldsymbol X\rightarrow(\boldsymbol A^T\boldsymbol A+\alpha_{dX}\boldsymbol 1_{J\times J}+\alpha_{rX}\boldsymbol L_A\boldsymbol L_A^T)^{-1}(\boldsymbol A^T\boldsymbol Y-\alpha_{sX}\boldsymbol 1_{J\times J})$$
I'm already using multi-start initialization of the matrices $\boldsymbol A$ and $\boldsymbol X$ and, at each iteration, if one of the matrices with is inverted on the equations above has zero determinant, $\boldsymbol A$ and $\boldsymbol X$ are reinitialized.
Then the algoritm stills get a source with all zeros sometimes, that doesn't happens when $\alpha_s$, $\alpha_d$ and $\alpha_r$ are low enough, or for low number of iterations, but this way, the algorithm doesn't does it's job.
How can I prevent the apperence of all zero sources?
