1
$\begingroup$

I am new to Matlab. For me the task is to perform data classification on some data set. The given data set consists of numeric values. One thing that bothers me is I have to perform all KNN, ANN, and SVM. In performing KNN, I faced a problem of how to find the value of K. If it is user defined? From which value we have to start and when to stop it?

$\endgroup$
1
$\begingroup$

Basically too large k may tend to underfit, while smaller k may tend to overfit. You may try to run the classification with different k within the preset range with cross-validation. Observe and find out the k that gives you the least cross validation sum-of-square error.

Another possible metric besides the sum-of-square error is called Silhouette applied to interpretation and validation of clusters of data. The technique provides a succinct graphical representation of how well each object lies within its cluster.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Hello Lenon...Can you help em some more further ? Can you let me know briefly about Cross Validation and how to calculate it for 2d Data set ?. I have seen about CV in web but those formula and notations makes me confused. So, please let me know when to apply CV and how to apply ? $\endgroup$ – Satya Achanta Feb 16 '14 at 1:27
  • $\begingroup$ You split your data into training set and testing set, use training set to train the model, then use the model to test how well the testing set can be classified. Just count the number of mis-classifications, the lower the better. $\endgroup$ – lennon310 Feb 16 '14 at 1:31
  • $\begingroup$ en.wikipedia.org/wiki/Cross-validation_(statistics) $\endgroup$ – lennon310 Feb 16 '14 at 1:32
  • $\begingroup$ Thank you lennon. I have some more questions,but, I will google them . Even I am not aware of model clearly. I mean , " what the model means " ? . I will google it. Thnk you $\endgroup$ – Satya Achanta Feb 16 '14 at 1:37
  • $\begingroup$ model is your knn result with training data as the input. Thanks $\endgroup$ – lennon310 Feb 16 '14 at 1:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.