6
$\begingroup$

I am working on a project that employs a linear array sensor that provides data from the same object at two different energies. Collected in time, I end up with two images (16-bit sensor values, MxM image), call one HIGH the other LOW. My engineering background is Material Science and metallurgy, so I am coming up to speed on DSP, image analysis, and all of the bits that bring all of that together.

I have been researching "image fusion" and have been working from Gonzalez's fabulous Digital Image Processing (3rd ed.) text. I have developed an understanding of DWT and have successfully written code to replicate the 3-scale DWT transforms in the text (major accomplishment for me).

Now, I am trying to understand how to "fuse" the DWT coefficients from the LOW and HIGH images. From the IEEE Xplore database, I pulled several papers on the topic; one of the more useful is by Zhiyu Chen, et.al. "A Combinational Approach to the Fusion, De-noising and Enhancement of Dual-Energy X-Ray Luggage Images".

Succinctly, they recommend "averaging the corresponding approximation coefficients of L and H" (sigma)... and "summing the corresponding detail coefficients of L and H" (psi). But, that is all of what is said on the matter.

I have several other sources, but they must presume some obvious bit of understanding that I am lacking!

My question is, in order to successfully fuse the HIGH and LOW images: what does this mean? In the case of a 3-scale DWT, do I average or sum the 3-rd scale approximation/detail coefficients? Do I somehow average across each scale, despite the fact that the size of the coefficient matrices are different by powers of 2?

Improve this question
$\endgroup$
6
  • $\begingroup$ Are the LOW and HIGH images overlapping? In other words, does each "pixel" value in the HIGH image and the LOW image correspond to measuring the same thing? $\endgroup$
    – Mustafa
    Jul 31, 2013 at 17:19
  • $\begingroup$ @Mustafa Yes, each pixel in the LOW image is in 1-to-1 correlation to the HIGH image. Just to be explicit: pixel m,n in LOW corresponds to m,n in HIGH $\endgroup$
    – jjwebster
    Jul 31, 2013 at 18:32
  • $\begingroup$ You won't have to average across scale. If the paper explicitly mentioned averaging the approximation coefficients and summing the detail coefficients.. why not try that? What I mean is, for a given scale, take the approximation coefficients from LOW and average with the approximation coefficients from HIGH. For that same scale, take the detail coefficients from LOW and add to the detail coefficient from HIGH. Intuitively it makes sense, because the approximation coefficients are responsible for gross structure so you want to preserve energy by averaging. $\endgroup$
    – Mustafa
    Jul 31, 2013 at 20:41
  • $\begingroup$ It looks like Chen's PhD thesis is available online. Particularly, see Appendix A.2 ("Wavelet-based image fusion") on page 174. Perhaps it will help. $\endgroup$
    – Peter K. ♦
    Jul 31, 2013 at 20:46
  • $\begingroup$ @Mustafa I think I'm confused about the "order of operation". Help me understand: I perform the first decomposition on each image, yielding the approximation and the H, V, and D coefficients. I average the approximation from each, and sum the detail from each. Then, I perform the second decomposition... using which approximation? The "original" approximation from each? At the end of N-scales, I use the final approximation and reconstruct back through the tree, using the averages/sums from each scale? I've tried that, but the results were not what I expect (could be my algo, though). $\endgroup$
    – jjwebster
    Jul 31, 2013 at 23:13

1 Answer 1

Reset to default
2
$\begingroup$

If I remember my wavelet transform correctly, this is what happens. Let's imagine that we are working with only the LOW image at the moment. I perform the first-scale decomposition which gives me the LOW_A0 (approximation) and the three detail coefficient images which are LOW_H0, LOW_V0 and LOW_D0. Then, to perform the second scale decomposition, you take the approximation image LOW_A0 and obtain LOW_A0_A1 (approximation) and the corresponding three detail coefficient images LOW_A0_H1, LOW_A0_V1 and LOW_A0_D1. Finally, in the last scale decomposition, you will split LOW_A0_A1 into its corresponding approximation coefficients LOW_A0_A1_A2 and the final detail coefficient images LOW_A0_A1_H2, LOW_A0_A1_V2 and LOW_A0_A1_D2.

So at the end you are left with the following: 1) The lowest approximation coefficients LOW_A0_A1_A2 2) The detail coefficients at each level which are 9 in number (LOW_H0, LOW_V0, LOW_D0, LOW_A0_H1, LOW_A0_V1, LOW_A0_D1, LOW_A0_A1_H2, LOW_A0_A1_V2 and LOW_A0_A1_D2).

Now, also consider that in parallel you have also performed the wavelet decomposition for the HIGH image and you have the same end stage products which are: 1) The lowest approximation coefficients HIGH_A0_A1_A2 2) The detail coefficients at each level which are 9 in number (HIGH_H0, HIGH_V0, HIGH_D0, HIGH_A0_H1, HIGH_A0_V1, HIGH_A0_D1, HIGH_A0_A1_H2, HIGH_A0_A1_V2 and HIGH_A0_A1_D2).

You want to now reconstruct a FUSED image which contains information from both the LOW and the HIGH images. At this stage, you would apply the fusion algorithm. Average the lowest approximation coefficients k*LOW_A0_A1_A2 + (1-k)*HIGH_A0_A1_A2, where k= 0.5 for equal weighting (you may want to weight one more than the other). Then, sum the detail coefficients at each of the corresponding LOW and HIGH detail coefficients. For example, LOW_H0 + HIGH_H0.

Now take these approximation coefficient and detail coefficients and do the wavelet synthesis to get the final result. Does this make sense? I haven't read the paper/thesis you pointed out to though.

Improve this answer
$\endgroup$
2
  • $\begingroup$ This makes perfect sense. I am working on this presently and will report my findings. I now understand the fusion process. It's amazing how simple it is once you read the answer. You know, like the Theory of Relativity -- so obvious once Einstein works out the details for you. $\endgroup$
    – jjwebster
    Aug 1, 2013 at 17:34
  • $\begingroup$ This approach works (that is, I am able to reconstruct an image that seems correct). I believe my error previously was having the wrong order of decomposition coefficients: I believe I was using the N-scale coefficients in place of the 1-scale, the (N-1)-scale in place of the 2-scale, etc. I've accepted your answer and appreciate the help. $\endgroup$
    – jjwebster
    Aug 1, 2013 at 22:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.