User Tools

Site Tools

Large Scale Kernel Machines

Léon Bottou, Olivier Chapelle, Dennis DeCoste, Jason Weston.


Pervasive and networked computers have dramatically reduced the cost of collecting and distributing large datasets. In this context, machine learning algorithms that scale poorly could simply become irrelevant. We need learning algorithms that scale linearly with the volume of the data while maintaining enough statistical efficiency to outperform algorithms that simply process a random subset of the data.

This volume offers researchers and engineers practical solutions for training kernel machines from large scale datasets, with detailed descriptions of algorithms and experiments carried out on realistically large datasets. At the same time it offers researchers information that can address the relative lack of theoretical grounding for many useful algorithms.

After a detailed description of state-of-the-art support vector machine technology, an introduction of the essential concepts discussed in the volume, and a comparison of primal and dual optimization techniques, the book progresses from well-understood techniques to more novel and controversial approaches. Many contributors have made their code and data available online for further experimentation. Topics covered include fast implementations of known algorithms, approximations that are amenable to theoretical guarantees, and algorithms that perform well in practice but are difficult to analyze theoretically.

Large Scale Kernel Machines, Edited by Léon Bottou, Olivier Chapelle, Dennis DeCoste, and Jason Weston, Neural Information Processing Series, MIT Press, Cambridge, MA., 2007.

MIT Press page Amazon Barnes & Nobles

  editor = {Bottou, L\'{e}on and Chapelle, Olivier and {DeCoste}, Dennis and Weston, Jason},
  title = {Large Scale Kernel Machines},
  publisher = {MIT Press},
  address = {Cambridge, MA.},
  year = {2007},
  url = {},



  • Series Foreword
  • Preface (pdf)
  1. Support Vector Machine Solvers – Bottou and Lin (pdf, techreport)
  2. Training a Support Vector Machine in the Primal – Chapelle
  3. Fast Kernel Learning with Sparse Inverted Index – Haffner and Kanthak
  4. Large-Scale Learning with String Kernels – Sonnenburg, Rätsch and Rieck
  5. Large-Scale Parallel SVM Implementation – Durdanovic, Cosatto and Graf
  6. A Distributed Sequential Solver for Large-Scale SVMs – Yom-Tov
  7. Newton Methods for Fast Semisupervised Linear SVMs – Sindhwani and Keerthi
  8. The Improved Fast Gauss Transform with Applications to Machine Learning – Raykar and Duraiswami
  9. Approximation Methods for Gaussian Process Regression – Quinonero-Candela, Rasmussen and Williams
  10. Brisk Kernel Independent Component Analysis – Jegelka and Gretton
  11. Building SVMs with Reduced Classifier Complexity – Keerthi, Chapelle and DeCoste
  12. Trading Convexity for Scalability – Collobert, Sinz, Weston, and Bottou
  13. Training Invariant SVMs Using Selective Sampling – Loosli, Bottou and Canu (techreport)
  14. Scaling Learning Algorithms toward AI – Bengio and LeCun
  • References (bibtex)
  • Contributors
  • Index (pdf.)
papers/lskm-2007.txt · Last modified: 2011/10/12 20:42 by leonb

Page Tools