3
$\begingroup$

Given a set of images $ \left\{ \boldsymbol{x}_{i} \right\}_{i = 1}^{N} $ how could one cluster them in an unsupervised manner? What are the useful features / tools to do so?

For instance, will extracting the images mean and variance be a useful method?

Update:

Let's take an example! I have MNIST database. It contains lots of images with the size 28x28. I want to make a plot like this. What should I use to get all the x-axis points and y-axis points?

enter image description here

Improve this question
$\endgroup$
13
  • 1
    $\begingroup$ both mean and variance have unambiguous definition. Could you elaborate on what is unclear about that? $\endgroup$
    – Marcus Müller
    Dec 28, 2021 at 0:01
  • $\begingroup$ “It's sounds too easy (and very wrong) to just use some few basic MATLAB commands for compute the mean and variance in an image.” Yes, it’s easy. That doesn’t make it wrong. The mean and standard deviation (MATLAB functions mean and std) are trivial statistics. They are also unlikely to be very helpful in distinguishing images. $\endgroup$
    – Cris Luengo
    Dec 28, 2021 at 3:55
  • $\begingroup$ @MarcusMüller I have seen lots of scatter plots where each dot inside the scatter plot is an image. I wonder how they compute that dot that represent one single image. I assume it some kind of average/std, but I'm not sure. Think...image classification with scatter plots. $\endgroup$
    – euraad
    Dec 28, 2021 at 9:39
  • 1
    $\begingroup$ @MrYui that is t-Distributed Stochastic Neighbor Embedding (t-SNE). Check this blog and this paper. $\endgroup$
    – AlexTP
    Dec 28, 2021 at 20:13
  • 1
    $\begingroup$ @Royi not to disagree too much, but the post-close edit completely changed the direction and content of the question. Before, it was literally just "how do I calculate mean and variance", then it became the overly broad "what's this figure and how can I make one"; even in its current form I'd say, "hey, that's a nice figure, I bet it came from some publication; did you have a chance to look into what t-SNE is? Maybe you have a question regarding that!". $\endgroup$
    – Marcus Müller
    Dec 29, 2021 at 17:31

1 Answer 1

Reset to default
2
$\begingroup$

Unsupervised clustering of image data is tricky thing and requires adjusting the method to the content of the images set.

Assuming we're dealing with the MNIST data set we can do some nice things using known tools.
First, let's assume we're after 2 features, namely we're after a dimensionality reduction from 784 features / dimensions to 2.

The first approach you suggested, using Mean and Variance isn't fruitful for images.
This can be shown form the following image:

enter image description here

As we can see, the mean and variance features doesn't create separation between different classes of the data.

The other algorithm in your question, the t-SNE, originally developed for visualization is very capable in tasks like this.
It can reduce the dimensionality while keeping the classes information:

enter image description here

The idea is finding a method to embed data which is similar in high dimension closely in low dimension and keep data which is not similar far away.
Indeed in the figure above one could see that in most cases same digits are grouped together.

After the t-SNE we saw few other similar dimensionality reduction algorithm which are as capable (Even more) like UMAP.

The code is available at my StackExchange Codes Signal Processing Q80767 GitHub Repository (Look at the SignalProcessing\Q80767 folder).

Improve this answer
$\endgroup$
5
  • 1
    $\begingroup$ Nice answer! Also good to see you sharing code on GitHub. $\endgroup$
    – Peter K. ♦
    Dec 30, 2021 at 10:20
  • 2
    $\begingroup$ @PeterK., Thank you! I think it was you who made me do that. I saw it once on one of your answers, that you share the code on GitHub, and thought it is a great way to conserve knowledge. The best engineering is a copied engineering :-). $\endgroup$
    – Royi
    Dec 30, 2021 at 10:45
  • $\begingroup$ Very good! I like it! Is UMAP better than T-sne? Who is fastest? Who is easiest? $\endgroup$
    – euraad
    Jan 6, 2022 at 20:35
  • $\begingroup$ Can you show with LDA as well? $\endgroup$
    – euraad
    Jan 6, 2022 at 20:46
  • $\begingroup$ @Royi Yes! Here you go and thank you for your answer. Very helpful to see the difference between PCA and t-SNE. dsp.stackexchange.com/questions/80949/… $\endgroup$
    – euraad
    Jan 8, 2022 at 10:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.