User Tools

Site Tools

Stochastic Algorithms for One Pass Learning


The goal of the presentation is to describe practical stochastic gradient algorithms that process each training example only once, yet asymptotically match the performance of the true empirical optimum. This statement needs, of course, to be made more precise. To achieve this, we'll review the works of Nevel'son and Has'minskij (1972), Fabian (1973, 1978), Murata & Amari (1998), Bottou & LeCun (2004), Polyak & Juditsky (1992), Wei Xu (2010), and Bach & Moulines (2011). We will then show how these ideas lead to practical algorithms and new challenges.


Léon Bottou and Yann LeCun: Large Scale Online Learning, Advances in Neural Information Processing Systems 16 (NIPS 2003), Edited by Sebastian Thrun, Lawrence Saul and Bernhard Schölkopf, MIT Press, Cambridge, MA, 2004.


Léon Bottou and Yann LeCun: On-line Learning for Very Large Datasets, Applied Stochastic Models in Business and Industry, 21(2):137-151, 2005.


talks/onepass.txt · Last modified: 2021/03/23 10:05 by leonb

Page Tools