Fastest Available Algorithm to Blur an Image (Low Pass Filter)

Iam working with a camera that produces ugly artifacts:

by using ANY blur filter on the camera's output the visual quality improves drastically:

The above image was created using OpenCV's cv::medianBlur with a kernel size of 3.
I identified cv::medianBlur to be the fastest smooth/blur method in OpenCV. However for my needs it is still too slow since it uses up to 80% of the whole processing time including encoding (ffmpeg MPEG4). I already tryed to use cv::UMAT but uploading each Image to the GPU and downloading the result again is taking even more time. So using OpenCL / Cuda isn't an option either!?

Therefore Iam looking for the fastest way to blur/smooth an image. However there are so many algorithms in so many librarys(OpenCV/IPP/swscale) to implement and test them all would take to much time. So do you have any suggestions which algorithms I can take a look at, or could offer a really good performance?

Here are some test results for 75.000 iterations of a 640x360 image:

+------------------------------+--------+----------+
|          Algorithm           | Kernel | Time(ms) |
+------------------------------+--------+----------+
| cv::medianBlur               | 3x3    | 18492    |
| cv::medianBlur ocl           | 3x3    | 54596    |
| ippiFilterMedianCross_8u_C3R | 3x3    | 15755    |
| cv::blur                     | 3x3    | >100000  |
| cv::GaussianBlur             | 3x3    | >100000  |
| cv::filter2d                 | 3x3    | >100000  |
+------------------------------+--------+----------+

• these artifacts look like JPEG compression was overdone. There's a load of papers on reducing JPEG artifacts – I'd recommend googling for something like "JPEG artifact reduction OpenCV" Jul 16 '18 at 8:54
• oh, and the idea with GPU computing is always that you hide the copy overhead by a) copying up a whole bunch of images and computing over them at once b) already copying over the next bunch while the current is still being processed and starting the next computation before copying down the results of the previous one. Pipelining! Jul 16 '18 at 8:57
• And: is 640x360 really your target image size, or are you aiming for larger images? Jul 16 '18 at 8:59
• Yes this is true, but the camera streams the video as MJPEG, therefore I have no infulence on the compression just decompression. To remove the artifacts there might be a lot of great algorithms, but they are even slower than a simple blur. Jul 16 '18 at 9:00
• My hypothesis is that running on your GPU, your memory bandwidth will become the limit, and you can apply relatively complex repair algorithms. The problem is just that your images are so small that the whole overhead per image is large. Put them in a three-dimensional array and copy over 75,000 of them at once! Jul 16 '18 at 9:01

The fastest blur would be Box Blur.
You can implement it using Running Sum.
I think Intel FilterBoxBorder works in that manner.

If you'd like you can do a few passes of it to approximate the Gaussian Blur.

You can also use IIR Filter Coefficients to blur the image quite easily.

You may have a look at my project Fast Gaussian Blur.

• Is there any existing optimized implementation for Box Blur? For me it looks like it could use some SSE instructions and cache optimization. There is also the method ippiFilterBox "software.intel.com/en-us/ipp-dev-reference-filterbox" which is around 2 times slower than the ippiFilterMedianCross. Jul 16 '18 at 4:55
• Pay attention that you don't compare apples to apples. The Median Filter is working on 8 / 16 Bit images. The Box Blur works on 64 Float. Try FilterBoxBorder on the same image type and it should be faster.
– Royi
Jul 16 '18 at 5:57
• I used ippiFilterBoxBorder_8u_CR3 for my test. and recyceld the buffer. That was 2 times as slow ippiFilterMedianCross Jul 16 '18 at 7:28
• Input Format is RGB888 / CV_8UC3 / 8u_CR3 Jul 16 '18 at 9:16
• It seems something is awkward about this. Running Sum should require less operation than median even with the tricks for median known for Discrete Image values (8 / 16 Bit). Which version of IPP do you use? Could you share the code on IPP forum? It seems you might found an issue here.
– Royi
Jul 16 '18 at 12:19

I have implemented a fast 5x5 Gaussian-blur in C++ and compared the performance to OpenCV on Raspberry Pi 3B+ running 32bit Raspbian OS. The function uses all the 4 cores of the Raspberry Pi and works 2-3 times faster than OpenCV. The boost is even more on 64bit OS. Here is the link to code with documentation: https://github.com/zanazakaryaie/fastGaussianBlur

• Does your implementation support any size of Gaussian Blur? Nov 12 '21 at 6:11
• No, it doesn't. I needed a 5x5 kernel back then Nov 12 '21 at 16:38
• It makes sense to say something about it on your post. Nov 18 '21 at 11:31