This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
research:largescale [2012/12/24 11:43] leonb [Approximate Optimization] |
research:largescale [2013/02/25 09:57] (current) leonb [Papers] |
||
|---|---|---|---|
| Line 49: | Line 49: | ||
| ===== Approximate Optimization ===== | ===== Approximate Optimization ===== | ||
| + | {{ wall2.png}} | ||
| Large-scale machine learning was first approached as an engineering problem. For instance, to leverage a | Large-scale machine learning was first approached as an engineering problem. For instance, to leverage a | ||
| larger training set, we can use a parallel computer to run a known machine learning algorithm | larger training set, we can use a parallel computer to run a known machine learning algorithm | ||
| Line 54: | Line 55: | ||
| objective function. Such approaches rely on the appealing | objective function. Such approaches rely on the appealing | ||
| assumption that one can decouple the statistical aspects from the computational aspects of the machine | assumption that one can decouple the statistical aspects from the computational aspects of the machine | ||
| - | learning problem. My [[: | + | learning problem. |
| - | that this assumption is incorrect, and that giving it up leads to considerably | + | |
| - | more effective learning algorithms. | + | |
| - | The [[: | + | This work shows that this assumption is incorrect, and that giving it up leads to considerably |
| - | develops a theoretical framework | + | more effective learning algorithms. A new theoretical framework |
| - | that takes into account the effect of approximate | + | takes into account the effect of approximate |
| optimization on learning algorithms. | optimization on learning algorithms. | ||
| + | |||
| The analysis shows distinct tradeoffs for the | The analysis shows distinct tradeoffs for the | ||
| case of small-scale and large-scale learning problems. | case of small-scale and large-scale learning problems. | ||
| Line 70: | Line 70: | ||
| complexity of the underlying optimization | complexity of the underlying optimization | ||
| algorithms in non-trivial ways. | algorithms in non-trivial ways. | ||
| - | For instance, | + | For instance, [[: |
| - | [[: | + | appear to be mediocre optimization algorithms and yet are shown to |
| - | is shown to perform | + | [[: |
| + | |||
| + | |||
| + | |||
| + | ===== Tutorials ===== | ||
| + | |||
| + | * NIPS 2007 tutorial " | ||
| + | |||
| + | ===== Related ===== | ||
| + | |||
| + | * [[: | ||
| + | ===== Papers ===== | ||
| - | ===== Stochastic Gradient for Large-Scale Learning | + | <box 99% orange> |
| + | Léon Bottou and Olivier Bousquet: | ||
| + | //Advances in Neural Information Processing Systems//, 20, | ||
| + | MIT Press, Cambridge, MA, 2008. | ||
| - | [[stochastic]] | + | [[: |
| + | </ | ||
| + | <box 99% orange> | ||
| + | Léon Bottou and Yann LeCun: | ||
| + | [[: | ||
| + | </ | ||
| + | <box 99% orange> | ||
| + | Léon Bottou: | ||
| + | [[: | ||
| + | </ | ||
| - | ===== Active Learning ===== | + | <box 99% blue> |
| + | Léon Bottou: | ||
| - | One simple way to handle large-scale learning problems is to chose examples wisely. | + | [[: |
| - | This idea was explored in our work on [[lasvm|Active and Online Support Vector Machines]]. | + | </ |
| - | But there is still much work to do about active learning as a way to handle very large data repositories. | + | |