====== Online Learning and Stochastic Approximations ====== 

{{talk-mlss.png?200 }}
This four part lecture was given at the 
[[http://www.mlss.cc|Machine Learning Summer School]] 
held in [[http://www.kyb.tuebingen.mpg.de/mlss04/mlss03|Tübingen in 2003]]
organized by [[http://ml.typepad.com/about.html|Olivier Bousquet]],
[[http://www.kyb.mpg.de/~bs|Bernhard Schölkopf]] and
[[http://www.ipsi.fraunhofer.de/mine/en/people/luxburg|Ulrike von Luxburg]].
The lecture discusses Stochastic Approximations and in particular [[wp>Stochastic_gradient_descent|Stochastic Gradient Descent]] applied to online learning algorithms.

\\


  * See [[http://leon.bottou.org/slides/mlss/part1.djvu|part 1: Framework (djvu 157KB)]] [[http://leon.bottou.org/slides/mlss/part1.pdf|(pdf 2.1MB)]].
  * See [[http://leon.bottou.org/slides/mlss/part2.djvu|part 2: Cookbook (djvu 459KB)]] [[http://leon.bottou.org/slides/mlss/part2.pdf|(pdf 2.7MB)]].
  * See [[http://leon.bottou.org/slides/mlss/part3.djvu|part 3: Convergence and Martingales (djvu 38KB)]] [[http://leon.bottou.org/slides/mlss/part3.pdf|(pdf 1.3MB)]].
  * See [[http://leon.bottou.org/slides/mlss/part4.djvu|part 4: Optimal online algorithms (djvu 55KB)]] [[http://leon.bottou.org/slides/mlss/part4.pdf|(pdf 1.4MB)]].

  * See the corresponding [[:papers:bottou-mlss-2004|book chapter]].