Global vector descriptor

Question

Usually, algorithms as SIFT, SURF and many others provides a set of $k$ keypoints and the associated descriptor in $d$ dimensions (for example, in SIFT each descriptor has $d$=128 dimensions).

So, in order to describe an image we need a matrix $k \times d$ ($k$ descriptor vectors, each one in $d$ dimensions). So far so good.

My question is: how can we describe an image through a single vector?

This could be really useful since we could save a lot of space and because certain algorithms (like LSH) requires a vector as input/query.

In some papers (for example this, section 6.5) this approach is described as "global descriptors".

Up to know, I found only this paper but it doesn't seem so accurate (and it's from 2009, not so new).

APPLICATION:

I'm trying to implement a memoization optimization to the following application: given an image (taken with a smartphone) containing a movie poster, return the internet movie database's score of that movie.

The optimization (so what I'm trying to implement) is about "remembering" (memoize) the input image and the relative result (IMD score) and for each new input image, search "the most similar one" and return the already computed result (avoid the database query) if a certain threshold is reached.

Two "similar images" in this context are two images pictures containing the same poster but from different prospective and/or places.

The actual approach is solving an approximate nearest neighbor through LSH in order to find the most similar image, but in order to do that I need to represent a query (an image) through a vector, and not through a matrix.

Answer 1

There are lots of different ways. You could simply put all the image pixel values into a vector with dimension = width x height. :) What you really want to know is how can we describe an image using a lower-dimensional representation and still caption all the important information.

The same approaches that describe an image patch, e.g. SIFT, apply. Just think of the entire image as the patch. However, the method very much depends on the information in the image and the purpose of the vector. For example, you could use some histograms of the gradients, subsample the frequency spectrum, take a certain number of wavelet coefficients, perform edge detection, use moments of the image, divide it into super-pixels and so on. You might need the representation to have different invariances such brightness, rotation, contrast, phase or scale. Basically any method that gives you a condensed version of the information in an image can be put into a vector.

Research into the right method to use for your particular application is probably in the literature somewhere. As for saving a lot of space, this is what compression is all about. e.g. JPEG discards the smallest coefficients of the DCT of blocks in the image to reduce space.