I am running kNN on a very small data set for binary classification. Each class has 100 samples. I am getting the best performances for $k=1$ and $k=3$. Can I deduce information about my data set from kNN performing very well for small values of $k$?


If the result is consistent with a large test set than it means your training data is dense and well define the degrees of freedom of the problem.

If the training set was small it means there is a good separation between your classes in space.

| improve this answer | |
  • $\begingroup$ Thank you for your answer. Would it also mean by any chance that the data is overfitting? $\endgroup$ – prax1telis Apr 3 '17 at 18:22
  • $\begingroup$ It might be. But KNN was built for cases your training label data is large and hence catches all the space geometry. So make sure you have really large size Training Set and Test Set. What happens when you set K = 5, how bad things become? $\endgroup$ – Royi Apr 4 '17 at 4:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.