We propose a novel online kernel classifier algorithm that converges to the Hard Margin SVM solution. The same update rule is used to both add and remove support vectors from the current classifier. Experiments suggest that this algorithm matches the SVM accuracies after a single pass over the training examples. This algorithm is attractive when one seeks a competitive classifier with large datasets and limited computing resources.

@inproceedings{bordes-bottou-2005,
  author = {Bordes, Antoine and Bottou, {L\'{e}on}},
  title = {The Huller: a simple and efficient online SVM},
  year = {2005},
  booktitle = {Machine Learning: ECML 2005},
  pages = {505-512},
  series = {Lecture Notes in Artificial Intelligence, LNAI~3720},
  publisher = {Springer Verlag},
  url = {http://leon.bottou.org/papers/bordes-bottou-2005},
}

• This underrated algorithm is the precursor of LaSVM (Bordes & al., 2005).
• Sebastian Nowozin at TU-Berlin proposes a free implementation of the Huller in C.