I am running kNN on a very small data set for binary classification. Each class has 100 samples. I am getting the best performances for $k=1$ and $k=3$. Can I deduce information about my data set from kNN performing very well for small values of $k$?


1 Answer 1


If the result is consistent with a large test set than it means your training data is dense and well define the degrees of freedom of the problem.

If the training set was small it means there is a good separation between your classes in space.

  • $\begingroup$ Thank you for your answer. Would it also mean by any chance that the data is overfitting? $\endgroup$
    – prax1telis
    Commented Apr 3, 2017 at 18:22
  • $\begingroup$ It might be. But KNN was built for cases your training label data is large and hence catches all the space geometry. So make sure you have really large size Training Set and Test Set. What happens when you set K = 5, how bad things become? $\endgroup$
    – Royi
    Commented Apr 4, 2017 at 4:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.