# Background averaging not working

I'm working on a problem where I'm trying to use background averaging to see what the frame looks like in a beehive behind the bees. I've been trying to use standard averaging (I take an image every 1 or 4 seconds over the course of my 3 hour video) as well as other techniques like mog and mog2. The problem is that there is an extremely high density of bees in the centre that is tightly packed and barely moving, so while I end up with a great view of the background around the middle, the centre turns into a bit of a smear like this: Does anyone have any advice on what I could do to improve things? I spoke to a researcher who suggested something called "sparse coding" as one possible options and I'm curious if OpenCV has anything like that or if anyone knows of any other techniques I could try.

I would try techniques related to low-rank approximations of your data $X$. For instance, the paper GoDec: Randomized Low-rank & Sparse Matrix Decomposition in Noisy Case attempts to decompose $$X = L + S + G$$ with a low-rank part ($\mathrm{rank}(L) \le r$), a sparse part ($\mathrm{card}(S) \le k$) and a residual $G$.

It has been employed for background modeling, with backgrounds related to the low-rank part, plus sparse en independent objects. Some video demos are:

There are alternatives in the field of robust matrix factorization, and I understand it as an instance of sparse coding.

Why not high pass filter each image before averaging, cutting out the lower frequency (bigger bee than comb)? Then you should mostly be integrating comb when you average/sum the images

Hey It seems that average really isnt the solution here as at most time a bee captures the background,

If you say that mixture of Gaussians doesnt work , I would have tried 1. To use visualization to understand a bit more about what I'm trying to find. Ill plot the time value of a background pixel and look at a bunch of them

e.g. if your image is built like a [ rowscolumntimes ] matrix called timeSeries

plot ( timeSeries(i,j,:) ,'.');

1. One very simple idea I used is median filtering which works great to remove bright outliers. However in your case you might need to use a smaller percentile on each pixel

something like:

bg_img(i,j) = prctile ( timeSerties(i,j,:) , 30 ) ;