knn

$k$ Nearest Neighbors

$k$ NN Find the $k$ training instances that are the closest to the

test instance, and classify the test instance as the majority class of the $k$ nearest training instances.

• Choosing $k$
  • Too small: overfitting, susceptible to noise
  • Too big: misclassify, instances far away are included
• Majority voting vs. distance-weighted, or taking average.