2
$\begingroup$

My requirements are these:

  • Fast classification of vectors of length 100 to one of 30000 classes
  • Iterative learning (can improve the model after it was first learnt)
  • Preferably available implementations in Matlab and Java

Thanks

$\endgroup$
  • $\begingroup$ I haven't tried anything yet. I was thinking to use SVM, but I guess that 30k classes would take too long even with one vs all. I don't yet know much about trees and forests but I have a feeling that one of these might fit. As I don't know many classification algorithms, I'm currently looking for directions to start from. $\endgroup$ – Michael Litvin Nov 25 '13 at 17:59
  • $\begingroup$ I would suggest you to try "random forest". $\endgroup$ – shn Dec 13 '13 at 13:09
  • $\begingroup$ afaik Random Forest is intended for binary classification. Is there any multiclass extension that would deal with such an amount of classes? $\endgroup$ – Michael Litvin Dec 13 '13 at 17:21
  • $\begingroup$ Could you give some more details about the data? Do you have "good" separation of classes? Linear? What do you mean by iterative learning? Do you consider a change in the dimensionality as Iterative Learning? The fastest classification algorithm I know is the Euclidian distance, followed by the Mahalanobis distance. Do they fit your requirements? Maybe you get compliance to the "iterativity" by using a Kohonen Map classifier, but then the requirement of "fast classification" might cause troubles. $\endgroup$ – luciano kruk Feb 5 '17 at 23:26
  • $\begingroup$ Do you have the class labels? Are you looking for a supervised approach? Or can you also work with unsupervised one? $\endgroup$ – Tolga Birdal Feb 5 '17 at 23:53
1
$\begingroup$

I would suggest using Deep Learning and especially convolutional neural networks. This might be a generic approach to large scale classification tasks. One such example is identifying faces. As an example, 10.000 classes have been tackled by Sun et. al. in the following : http://ieeexplore.ieee.org/document/6909640/

And iterative learning is of course supported.

$\endgroup$
0
$\begingroup$

Well, it depends on the nature of the classes. Personally, I prefer the SVM because it's easy to work with and it works (Not that fast)! You can also use the k-means method (but it really slooooooooooooooooow especially with this number of classes). Another classifier worth looking into is the Adaboost, they say it's fast :)

Have fun...!

$\endgroup$
  • $\begingroup$ thanks, but none of what you've proposed addresses the first two requirements... $\endgroup$ – Michael Litvin Nov 26 '13 at 12:31
  • 1
    $\begingroup$ also, k-means is not a classification algorithm, rather an algorithm for automatic clustering. $\endgroup$ – Michael Litvin Nov 26 '13 at 12:46
  • 1
    $\begingroup$ K-meanks is originally developed for clustering but it can be adapted for classification (google it...). $\endgroup$ – Muhammad Nov 26 '13 at 15:47
  • 1
    $\begingroup$ @Muhammad K-means is a clustering method, so it is not intended to be used for classification, even if you consider "clusters" as "classes" (which are actually quite different). You will always get better results with a classification method (where learning is supervised) compared to a clustering method (unsupervised) that you desperately want to use for classification. I know some poeple use clustering for classification, but this is really a bad practice. However, if you are talking about "supervised clustering" this is a quite different thing (it is a clustering with some constraints). $\endgroup$ – shn Dec 13 '13 at 13:05
0
$\begingroup$

Have you considered multiple sigmoid regression? There is an analytical simplification of it's partial derivative that makes iterative algorithms simple. You can implement them yourself pretty easily.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.