Concept of Orthogonality and Orthonormality

Please explain me the concept of Orthogonality and Orthonormality. I understand that the mathematical definition of orthogonality between two vectors is that their dot product is zero. It means that there is no correlation between the two vectors. And in case of Orthonormality the vector length would be 1.

I am unable to understand the concepts clearly. Please explain me the concept of Orthogonality and Orthonormality in terms of Image Processing.

• There is nothing to understand beyond what you have already stated in your first paragraph. What parts of the last three sentences in your first paragraph are you not understanding clearly? Would a statement like "orthogonal" means that the vectors are at right angles to each other make it more palatable? – Dilip Sarwate Nov 6 '13 at 2:54
• @PremnathD : Why do you think that orthonormality or orthogonality are related to image processing? – hotpaw2 Nov 6 '13 at 8:46
• In transforms, I came across orthogonal property of a transform helps in image denoising and compression. Recently while studying wavelets I came across orthonormal – Premnath D Nov 6 '13 at 16:01

Orthogonal means that the inner product is zero. For example, in the case of using dot product as your inner product, two perpendicular vectors are orthogonal. Orthonormal means these vectors have been normalized so that their length is 1.

Orthogonal vectors are useful for creating a basis for a space. This is because every point in the space can be represented as a (linear) combination of the vectors. So for example in 3D space, x=[0,0,1] y=[0,1,0] and z=[1,0,0] form an orthornormal basis. No component of x can be represented with a component of the other vectors. This is because they are linearly independent

This is a different type of independence to statistical dependence however.

Correlation is a different notion. There are different ways of representing the correlation of two vectors (random variables X and Y).

$corr(X,Y) = \frac{cov(X,Y)}{\sigma_x \sigma_y} = \frac{E[X - \mu_x]E[Y - \mu_y]}{\sigma_x \sigma_y}$

Covariance is a measure of how two variables change together.

Another measure of correlation is cross-correlation. This is measure of the similarity of two functions (or vectors).

These concepts ares used in many different ways in image processing.

For example cross-correlation is used in template matching. When looking for a small image inside a much larger image, the small template image is 'slided' over the large image and the cross-correlation is computed for each position. Locations with a high cross-correlation are likely to contain the image we are looking for.

The concept of Linearly independent component vectors is used in principal component analysis (PCA). PCA takes a cloud of points in space and calculates a set of orthogonal basis vectors to represent them.

Independent component analysis (ICA) is used for separating mixed signals. ie: separating a speaker at a noisy cocktail party. ICA uses properties of the signal correlation to separate the signals.