Iam working with a camera that produces ugly artifacts: enter image description here

by using ANY blur filter on the camera's output the visual quality improves drastically: enter image description here

The above image was created using OpenCV's cv::medianBlur with a kernel size of 3.
I identified cv::medianBlur to be the fastest smooth/blur method in OpenCV. However for my needs it is still too slow since it uses up to 80% of the whole processing time including encoding (ffmpeg MPEG4). I already tryed to use cv::UMAT but uploading each Image to the GPU and downloading the result again is taking even more time. So using OpenCL / Cuda isn't an option either!?

Therefore Iam looking for the fastest way to blur/smooth an image. However there are so many algorithms in so many librarys(OpenCV/IPP/swscale) to implement and test them all would take to much time. So do you have any suggestions which algorithms I can take a look at, or could offer a really good performance?

Here are some test results for 75.000 iterations of a 640x360 image:

|          Algorithm           | Kernel | Time(ms) |
| cv::medianBlur               | 3x3    | 18492    |
| cv::medianBlur ocl           | 3x3    | 54596    |
| ippiFilterMedianCross_8u_C3R | 3x3    | 15755    |
| cv::blur                     | 3x3    | >100000  |
| cv::GaussianBlur             | 3x3    | >100000  |
| cv::filter2d                 | 3x3    | >100000  |
  • $\begingroup$ these artifacts look like JPEG compression was overdone. There's a load of papers on reducing JPEG artifacts – I'd recommend googling for something like "JPEG artifact reduction OpenCV" $\endgroup$ – Marcus Müller Jul 16 '18 at 8:54
  • 1
    $\begingroup$ oh, and the idea with GPU computing is always that you hide the copy overhead by a) copying up a whole bunch of images and computing over them at once b) already copying over the next bunch while the current is still being processed and starting the next computation before copying down the results of the previous one. Pipelining! $\endgroup$ – Marcus Müller Jul 16 '18 at 8:57
  • $\begingroup$ And: is 640x360 really your target image size, or are you aiming for larger images? $\endgroup$ – Marcus Müller Jul 16 '18 at 8:59
  • $\begingroup$ Yes this is true, but the camera streams the video as MJPEG, therefore I have no infulence on the compression just decompression. To remove the artifacts there might be a lot of great algorithms, but they are even slower than a simple blur. $\endgroup$ – Crigges Jul 16 '18 at 9:00
  • $\begingroup$ My hypothesis is that running on your GPU, your memory bandwidth will become the limit, and you can apply relatively complex repair algorithms. The problem is just that your images are so small that the whole overhead per image is large. Put them in a three-dimensional array and copy over 75,000 of them at once! $\endgroup$ – Marcus Müller Jul 16 '18 at 9:01

The fastest blur would be Box Blur.
You can implement it using Running Sum.
I think Intel FilterBoxBorder works in that manner.

If you'd like you can do a few passes of it to approximate the Gaussian Blur.

You can also use IIR Filter Coefficients to blur the image quite easily.

You may have a look at my project Fast Gaussian Blur.

  • $\begingroup$ Is there any existing optimized implementation for Box Blur? For me it looks like it could use some SSE instructions and cache optimization. There is also the method ippiFilterBox "software.intel.com/en-us/ipp-dev-reference-filterbox" which is around 2 times slower than the ippiFilterMedianCross. $\endgroup$ – Crigges Jul 16 '18 at 4:55
  • $\begingroup$ Pay attention that you don't compare apples to apples. The Median Filter is working on 8 / 16 Bit images. The Box Blur works on 64 Float. Try FilterBoxBorder on the same image type and it should be faster. $\endgroup$ – Royi Jul 16 '18 at 5:57
  • $\begingroup$ I used ippiFilterBoxBorder_8u_CR3 for my test. and recyceld the buffer. That was 2 times as slow ippiFilterMedianCross $\endgroup$ – Crigges Jul 16 '18 at 7:28
  • 1
    $\begingroup$ Input Format is RGB888 / CV_8UC3 / 8u_CR3 $\endgroup$ – Crigges Jul 16 '18 at 9:16
  • 1
    $\begingroup$ It seems something is awkward about this. Running Sum should require less operation than median even with the tricks for median known for Discrete Image values (8 / 16 Bit). Which version of IPP do you use? Could you share the code on IPP forum? It seems you might found an issue here. $\endgroup$ – Royi Jul 16 '18 at 12:19

I have implemented a fast Gaussian-blur in C++ and compared the performance to OpenCV on Raspberry Pi 3B+ running 32bit Raspbian OS. The function uses all the 4 cores of the Raspberry Pi and works 2-3 times faster than OpenCV. The boost is even more on 64bit OS. Here is the link to code with documentation: https://github.com/zanazakaryaie/fastGaussianBlur


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.