# Where and how do an Image and a complex number meet?

I am trying to implement a Gabor Filter bank. Gabor Filter uses a complex function. How do the filters like Gabor Filter operate on an Image?

Does a complex number represent an entire Image or only a pixel?

Is an image a 2D array of complex numbers?

• Can you please clarify what exactly is your question by adding a little bit more detail of what you are trying to do and maybe what you have done so far (?). – A_A May 29 '16 at 6:22
• @A_A, I am trying to implement a Gabor Filter bank. Gabor Filter uses a complex function. How do the filters like Gabor Filter operate on an Image? – user18425 May 29 '16 at 11:50

No it's not a complex number but a real number per pixel for a gray scale image and 3 real numbers per pixel for a standard RGB image which is supposed to be a matrix of N by M pixels, stored in an appropriate storage class in a given technology and programming language, such as an array type in C.

Gray scale digital images are represented as 2D arrays of numbers, each representing the intensity of the image at the given pixel, traditionaly in 8 bit integers between [0,255] and also in [0,1] for floating point implementations.

Color images are treated in a number of ways. Most fundamental approach is to use the RGB convention to represent each pixel with its optical components such as Red, Green and Blue colors. These values are again mostly either 8 bits in [0,255] or floating values in [0,1].

In the old days, color images were also treated via palettes that I don't want to mention here. There are also other represantations for images such as HSV,YUV,YIQ,YPbPr,YCmCr (analog and digital) codecs, commercial and broadcasting applications, but are not much related with your question.

Note that the RGB triad could also be appended with a fourth byte, referred to as the alpha channel which can be utilized by some applications

Also newer imaging standards allow more than 8-bits per color component.

• Then, where and how do an Image and a complex number meet? – user18425 May 29 '16 at 0:00
• They don't... May be you are confusing it with Fourier Transform of an image (or any signal) which is in general a complex Function of the frequency. – Fat32 May 29 '16 at 0:06
• Then, how do the filters like Gabor Filter operate on an Image? Gabor Filter uses a complex function. – user18425 May 29 '16 at 0:57
• A filter can have real as well as complex valued coefficients, and threfore you can also generate complex valued signals if you process signals via complex coefficient filters but the physical insight is that the brightness (being a scalar quantitiy) at a pixel is represented with a real variable conventionally. Intermediate steps may use complex numbers but the end result will eventually be real. – Fat32 May 29 '16 at 1:21

Lets say you have two filters, $h_e(x)$ and $h_o(x)$, and when you apply them to an image you get two outputs,

$$g_e(x) = (f \ast h_e)(x)$$ $$g_o(x) = (f \ast h_o)(x)$$

Now say these filters have some special mathematical properties like $h_e(x)$ responds to even features and $h_o(x)$ responds to odd features, and the amplitude of their combined response is $\sqrt{g_e(x)^2 + g_o(x)^2}$ which describes the local image strength.

In this case it is easier just to combine the filters into a single complex valued filter. That is,

$$h(x) = h_e(x) + ih_o(x)$$

Then you can do the one filtering operation

$$g(x) = (f \ast h)(x)$$

from which you get

$$g_e = \Re(g(x))$$ $$g_o = \Im(g(x))$$ $$A = \sqrt{g_e(x)^2 + g_o(x)^2} = |g(x)|$$ $$\phi = \rm{atan2}(g_o(x),g_e(x)) = \arg(g(x))$$

So you see, putting two filters into one complex-valued filter encodes the relationship between them, especially in terms of amplitude (their combined energy) and phase (the ratio of their responses). It says that taking the absolute value of the complex filter response is a meaningful quantity, as is the argument.

Some other representations use more imaginary numbers, such as quaternions, to include more than two filters. Some have matrix or vector valued filters. All of this is about saying there is some kind of relationship between the filter outputs and their combined amplitude.

• @LaurentDuval Sorry, you mean I should say this mainly applies to quadrature filters? – geometrikal May 30 '16 at 3:16

You can have images that can be expressed with complex-valued pixels, see for instance Blind deconvolution of noisy complex-valued image. Some images acquired through Fourier-like imaging systems can be complex too. An instance is medical MRI imaging, but you can find that in seismic data too.

Many processing methods like the Fourier transform, complex wavelets or Laplacian pyramids can turn a real image into complex-valued images. One can use Complex Derivative Filters or phase congruency to detect edges or infer texture parameters. Optimization algorithms are designed for such complex instances: A Majorize-Minimize Memory Gradient Method for Complex-Valued Inverse Problems.

If you want to play with maths, you can encode a natural image into a single complex number by constructing digits in base $256$ and putting half of the pixels in the real part and the second half in the imaginary part because $\mathbb{C}$ is bigger than $[0,255]^{M\times N}$. But this is practically useless.

So:

1. An whole image can be represented by a single complex number, but useless
2. Many instances of images can have complex values at each pixel.