leon.bottou.org

User Tools

Site Tools

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
research:largescale [2012/12/24 11:52]
leonb [Learning with Stochastic Gradient Descent] 		research:largescale [2013/02/25 09:57] (current)
leonb [Papers]
Line 49: Line 49:
 ===== Approximate Optimization ===== ===== Approximate Optimization =====
  
 +{{ wall2.png}}
 Large-scale machine learning was first approached as an engineering problem. For instance, to leverage a Large-scale machine learning was first approached as an engineering problem. For instance, to leverage a
 larger training set, we can use a parallel computer to run a known machine learning algorithm  larger training set, we can use a parallel computer to run a known machine learning algorithm 
Line 60: Line 61:
 takes into account the effect of approximate  takes into account the effect of approximate 
 optimization on learning algorithms. optimization on learning algorithms.
 +
 The analysis shows distinct tradeoffs for the  The analysis shows distinct tradeoffs for the 
 case of small-scale and large-scale learning problems. case of small-scale and large-scale learning problems.
Line 68: Line 70:
 complexity of the underlying optimization  complexity of the underlying optimization 
 algorithms in non-trivial ways. algorithms in non-trivial ways.
- 
- 
 For instance, [[:research:stochastic|Stochastic Gradient Descent (SGD)]] algorithms For instance, [[:research:stochastic|Stochastic Gradient Descent (SGD)]] algorithms
-appear to be mediocre optimization algorithms +appear to be mediocre optimization algorithms and yet are shown to  
-and yet are shown to perform extremely well on large-scale learning problems.+[[:projects/sgd|perform extremely well]] on large-scale learning problems. 
  
  
Line 79: Line 80:
   * NIPS 2007 tutorial "[[:talks/largescale|Large Scale Learning]]".   * NIPS 2007 tutorial "[[:talks/largescale|Large Scale Learning]]".
  
 +===== Related =====
 +
 +   * [[:research:stochastic|Stochastic gradient learning algorithms]]
 ===== Papers ===== ===== Papers =====
  
Line 89: Line 93:
 </box> </box>
  
-===== See also =====+<box 99% orange> 
 +Léon Bottou and Yann LeCun:  **On-line Learning for Very Large Datasets**,  //Applied Stochastic Models in Business and Industry//, 21(2):137-151, 2005.
  
-  * [[stochastic|Learning with Stochastic Gradient Descent]].+[[:papers/bottou-lecun-2004a|more...]] 
 +</box>
  
 +<box 99% orange>
 +Léon Bottou:  **Online Algorithms and Stochastic Approximations**,  //Online Learning and Neural Networks//, Edited by David Saad, Cambridge University Press, Cambridge, UK, 1998.
  
 +[[:papers/bottou-98x|more...]]
 +</box>
 +
 +<box 99% blue>
 +Léon Bottou:  //**Une Approche théorique de l'Apprentissage Connexionniste: Applications à la Reconnaissance de la Parole**//, Orsay, France, 1991.
 +
 +[[:papers/bottou-91a|more...]]
 +</box>
  
research/largescale.1356367973.txt.gz · Last modified: 2012/12/24 11:52 by leonb

Page Tools