2
$\begingroup$

My requirements are these:

  • Fast classification of vectors of length 100 to one of 30000 classes
  • Iterative learning (can improve the model after it was first learnt)
  • Preferably available implementations in Matlab and Java

Thanks

$\endgroup$
5
  • $\begingroup$ I haven't tried anything yet. I was thinking to use SVM, but I guess that 30k classes would take too long even with one vs all. I don't yet know much about trees and forests but I have a feeling that one of these might fit. As I don't know many classification algorithms, I'm currently looking for directions to start from. $\endgroup$ Commented Nov 25, 2013 at 17:59
  • $\begingroup$ I would suggest you to try "random forest". $\endgroup$
    – shn
    Commented Dec 13, 2013 at 13:09
  • $\begingroup$ afaik Random Forest is intended for binary classification. Is there any multiclass extension that would deal with such an amount of classes? $\endgroup$ Commented Dec 13, 2013 at 17:21
  • $\begingroup$ Could you give some more details about the data? Do you have "good" separation of classes? Linear? What do you mean by iterative learning? Do you consider a change in the dimensionality as Iterative Learning? The fastest classification algorithm I know is the Euclidian distance, followed by the Mahalanobis distance. Do they fit your requirements? Maybe you get compliance to the "iterativity" by using a Kohonen Map classifier, but then the requirement of "fast classification" might cause troubles. $\endgroup$ Commented Feb 5, 2017 at 23:26
  • $\begingroup$ Do you have the class labels? Are you looking for a supervised approach? Or can you also work with unsupervised one? $\endgroup$ Commented Feb 5, 2017 at 23:53

3 Answers 3

1
$\begingroup$

I would suggest using Deep Learning and especially convolutional neural networks. This might be a generic approach to large scale classification tasks. One such example is identifying faces. As an example, 10.000 classes have been tackled by Sun et. al. in the following : http://ieeexplore.ieee.org/document/6909640/

And iterative learning is of course supported.

$\endgroup$
0
$\begingroup$

Well, it depends on the nature of the classes. Personally, I prefer the SVM because it's easy to work with and it works (Not that fast)! You can also use the k-means method (but it really slooooooooooooooooow especially with this number of classes). Another classifier worth looking into is the Adaboost, they say it's fast :)

Have fun...!

$\endgroup$
4
  • $\begingroup$ thanks, but none of what you've proposed addresses the first two requirements... $\endgroup$ Commented Nov 26, 2013 at 12:31
  • 1
    $\begingroup$ also, k-means is not a classification algorithm, rather an algorithm for automatic clustering. $\endgroup$ Commented Nov 26, 2013 at 12:46
  • 1
    $\begingroup$ K-meanks is originally developed for clustering but it can be adapted for classification (google it...). $\endgroup$
    – Muhammad
    Commented Nov 26, 2013 at 15:47
  • 1
    $\begingroup$ @Muhammad K-means is a clustering method, so it is not intended to be used for classification, even if you consider "clusters" as "classes" (which are actually quite different). You will always get better results with a classification method (where learning is supervised) compared to a clustering method (unsupervised) that you desperately want to use for classification. I know some poeple use clustering for classification, but this is really a bad practice. However, if you are talking about "supervised clustering" this is a quite different thing (it is a clustering with some constraints). $\endgroup$
    – shn
    Commented Dec 13, 2013 at 13:05
0
$\begingroup$

Have you considered multiple sigmoid regression? There is an analytical simplification of it's partial derivative that makes iterative algorithms simple. You can implement them yourself pretty easily.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.