# Pyramid vs Scale-space in Focus Stacking

I'm working on a focus stacking program, I wrote one a few years ago which used a very simple method, basically taking a DoG approach, but only operating on the single full resolution image. The resulting images had a soft focus look, kind of dreamy.

This time I researched further and found papers describing image pyramids, essentially doing the same thing as before except on multiple spatial frequencies, then recombining them with the maximum components which I imagine will solve the soft focus look.

But learning about pyramid based approaches led me to scale-space representation and multi-resolution analysis, and I can't tell if I should pursue one of those methods for possibly better results? (And MRA led me to wavelet transforms which I also don't see how I would implement.)

Furthermore, I don't really understand how either would even be implemented for focus stacking, but my impression of scale-space is that in practice I would just end up building a pyramid anyway, but that doesn't seem right. So am I missing something about that?

And does it seem like there is anything to be gained by using a newer or more sophisticated approach than image pyramids for focus stacking?

(I used Python last time but am using Apple's CoreImage framework this time, so I might be a bit more limited now, maybe MRA isn't even an option.)

• Just to understand a little better: would the approach provided in the page Extended Depth of Field suit your needs? Dec 22, 2015 at 16:19
• Yes, thank you! The first paper should help me to understand how to implement wavelet transforms, and just knowing the phrase “extended depth of field” will help me expand my search. I guess I'm going to try the pyramid based approach first just because of how easy it is to implement, and if I'm not happy with the results I'll look into these more sophisticated methods. Thanks again! Dec 23, 2015 at 7:17

Your question relates to techniques with different names. The first one is related to micrographs taken at different (aligned) focal positions combined into a focus image. This Extended Depth of Field is based on complex wavelet-based methods.

To answer your second question on difference between MRA, pyramids, etc.: most pyramids and wavelet-based methods for 2D image processing are instance of multiscale/multirate//multiorientations decompositions based on certain scale-space discretizations (a review paper on these issues is A Panorama on Multiscale Geometric Representations, Signal Processing, 2011). With scale-space in general, you do not need to downsample, and end up with a stack, not a pyramid.

Since one selects maximum coefficients across scale, band, or orientation, my opinion is that a complex pyramid, a complex (or dual-tree) wavelet transformation, with a little redundancy and some shift or rotation invariance could be useful, at the cost of more painful implementation. Standard real MRA wavelets are limited in orientations, full scale-space might be quite involved.

Maximum coefficient selection is probably a poor's man saliency detection, which is another track to follow based on the specificity of your images.

More general terms for your search are multifocus image fusion, multiscale or multi-resolution images fusion, even multichannel. A few references:

You can as well find inspiration in multiscale versions of High-dynamic-range (HDR) imaging, for instance:

• I'm implementing multifocus image fusion at the moment, are there any recent developments worth noting? Sep 23, 2018 at 11:26
• I am afraid I did not check that recently. Be happy to exchange with your experience Sep 23, 2018 at 12:14