Abstract: This paper proposes an online solver of the dual formulation of support vector machines for structured output spaces. We apply it to sequence labelling using the exact and greedy inference schemes. In both cases, the per-sequence training time is the same as a perceptron based on the same inference procedure, up to a small multiplicative constant. Comparing the two inference schemes, the greedy version is much faster. It is also amenable to higher order Markov assumptions and performs similarly on test. In comparison to existing algorithms, both versions match the accuracies of batch solvers that use exact inference after a single pass over the training examples.

@inproceedings{bordes-usunier-bottou-2008,
  author = {Bordes, Antoine and Usunier, Nicolas and Bottou, L\'{e}on},
  title = {Sequence Labelling SVMs Trained in One Pass},
  booktitle = {Machine Learning and Knowledge Discovery in Databases: ECML PKDD 2008},
  year = {2008},
  editor = {Daelemans, Walter and Goethals, Bart and Morik, Katharina},
  series = {Lecture Notes in Computer Science, LNCS~5211},
  pages = {146-161},
  publisher = {Springer},
  url = {http://leon.bottou.org/papers/bordes-usunier-bottou-2008},
}

Antoine Bordes provides an implementation of the LaRank algorithm which was in fact written for this paper. It also contains a special case for linear kernels.