So basically I have a set of multidimensional data that I need to determine the covariance of between dimensions in real-time. Each point that comes in is a vector. I have gotten the mean and variance per dimension using this answer: Determining the mean and standard deviation in real time. But I also need to find the covariance between dimensions, resulting in a covariance matrix. Each vector is 3 dimensional and is a point in space.

Does anyone know how to do this and does this?

To explain further, I want the covariance matrix so I can then use the eigenvalues and eigenvectors to characterize a point cloud. The point clouds are segments of a scanned environment using the Kinect. I need to make sure the segments are labelled consistently over frames. I will do this by matching the mean and covariance eigenvectors/eigenvalues of segments between frames. I am working with unity.


Or what would be equally useful would be how to get the covariance from just the variances and means that I already have.

  • $\begingroup$ sounds like a Kalman filter problem to me... $\endgroup$
    – Jason S
    Jun 30, 2016 at 6:03

1 Answer 1


To make the Online variance algorithm work for vectors, change one line:

M2 += np.outer( delta, delta2 )  # deltas 3-vecs, M2 3 x 3

# outer( x, y ) is pairs x[i] * y[j] --
# [[ x0 y0, x0 y1, x0 y2 ],
#  [ x1 y0, x1 y1, x1 y2 ],
#  [ x2 y0, x2 y1, x2 y2 ]]

A small python class for this is under gist.github.com/denis-bz . It can covariance points with uniform weights (FIR), or up-weight newer data (IIR), or both.

Is there a tutorial on your application with 3d point clouds ?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.