I'm working on a custom video codec which uses the standard mean squared error to find blocks.

As explained by this youtube video you choose a search area size to find a block in. Each pixel of the search area is turned into a block, and then the search block is compared to the master block.

These difference for each pixel are multiplied by themselves and added up for the entire block, and you get the total mean squared error. If the error is within a threshold you "found" the block. If the error is too high you need to do some more compression work.

So I implemented this mean squared algorithm, but there are certain videos I'm not finding blocks very well with.

For example this test I put together, of a 3d camera pan across a desk, (warning the test is 5.4MB and will take a while to load, once it does load you can play the video by "scrolling" the page).

Using a reference frame every 4 frames I was only able to find roughly 50% of the blocks using a 16x16 block size, 65% using an 8x8 block size, and 77% using a 4x4 block size.

I don't get it as the animation is very simple and beside a small shift in 3d perspective the blocks are just moving along a single vector.

This makes the file size HUGE for such a simple video. Right now it's clocking in at 5.8 MB for 64 frames. This is why it takes so long to download.

Are there other methods based on signal processing which modern video codecs use to find blocks? Like comparing the dct coefficents of each block to reduce noise errors?

On a side note the low quality of the video is not from the codec (mostly), but it's because I did a low quality render from my 3d program. Over the weekend I plan to do a full quality render.

Edit, one thing I've thought of is to test if the block is getting brighter/darker due to lighting conditions. If it's getting brighter every pixel error will be offset by how much brighter the light got. This offset can be stored and used to bring the overall error of the block back down to tolerable numbers.

I'm just wondering if this is normally done or if there are better methods to tackle this problem.


1 Answer 1


I assume you're talking about finding blocks in successive frames.

What you can do, in order to define and limit the search area, is to define the direction of movement between frames.
You can do something like optical flow and then per pixel define the search window in the next frame.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.