# Apply Low Pass (Smoothing) Filter on a Set of Matrices and Reject Outliers

Given a set of a $3$ by $3$ matrices ${H}_{i}$.
Each matrix is an Homography matrix.
They are used to stabilize Video Stream.
Yet some of them are outliers which creates "Jumps" in the "Stabilized Video Stream".
I would like to apply Low Pass filter on matrices in a manner which "Low Pass" the homography (Meaning, the Mapping).
Moreover, I'd like to reject Outliers, Matrices which compared to other rotates / scales the frame in a completely different way.
The problem I don't know any metrics which deals with that property of a $3$ by $3$ matrix.

Any idea?

I would build a tensor of those matrices and use low rank or some thresholding methods on it.

You may have a look for Tensor SVD.

The method used was a parametric representation of the matrix $$H$$.

For intuition, think of rotation matrix in $$\mathbb{R}^{2}$$, namely $$H \in \mathbb{R}^{2 \times 2}$$.
Since we have series of those (In case of video, a series in time) we can define $$H \left( t \right)$$.

As a rotation matrix it can be parameterized by a single parameter $$\theta$$ which is the rotation value. Since we are dealing with a time series then we have $$\theta \left( t \right)$$.

Now, this is just a simple Time Series and regular methods of smoothing and outlier rejection can be applied.

Another idea would be to find the best base to approximate all matrices (Which with respect to that base the can be diagonalized) and then work on the "Singular Values / Eigen Values".

You are trying to detect and reject abrupt changes in the 3x3 matrix, right?

Think of the 3x3 matrix as a 9 element vector space. Apply smoothing on each of the 9 dimensions. If the original 9 elements are not similar to the smoothed vector, then reject it and use the smoothed estimate (minus the contribution from the rejected one?).

As far as how to judge similarity, I suppose the max distance would be appropriate if you expect individual elements to get corrupted. The Euclidean distance would be better if the entire 3x3 matrix gets corrupted.

• I couldn't get what element by element smoothing would achieve. I think you could have 2 matrices which would rotate an image similarly yet are completely different in their numbers.
– Royi
Jul 18 '12 at 11:27
• I assumed the matrices would be normalized in magnitude. Is that true? If so, I don't see how two matrices that perform the same rotation could be completely different. Jul 19 '12 at 14:32
• That's the essence of it. How different are two Hommography matrices. You suggest to filter the matrices item by item?
– Royi
Jul 23 '12 at 6:03
• I think I could reconstruct the parameters of the Homography (Rotation, Translation, Scaling) in a non linear way and then have the parameters filtered and reconstruct the matrices.
– Royi
Sep 23 '12 at 8:02