This shows you the differences between two versions of the page.
Next revision | Previous revision | ||
papers:bordes-bottou-gallinari-2009 [2009/07/24 16:39] leonb created |
papers:bordes-bottou-gallinari-2009 [2017/11/29 10:27] (current) leonb [Errata] |
||
---|---|---|---|
Line 9: | Line 9: | ||
descent but requires less iterations | descent but requires less iterations | ||
to achieve the same accuracy. | to achieve the same accuracy. | ||
- | This algorithm won the ``Wild Track'' | + | This algorithm won the "Wild Track" |
PASCAL Large Scale Learning Challenge. | PASCAL Large Scale Learning Challenge. | ||
+ | < | ||
+ | // | ||
+ | Please see section [[#Errata]] below. | ||
+ | < | ||
<box 99% orange> | <box 99% orange> | ||
- | Antoine Bordes, Léon Bottou and Patrick Gallinari: | + | Antoine Bordes, Léon Bottou and Patrick Gallinari: |
- | < | + | [[http:// |
+ | [[http:// | ||
+ | < | ||
+ | [[http:// | ||
[[http:// | [[http:// | ||
- | [[http:// | + | [[http:// |
</ | </ | ||
Line 27: | Line 34: | ||
year = {2009}, | year = {2009}, | ||
volume = {10}, | volume = {10}, | ||
- | pages = {to appear}, | + | pages = {1737--1754}, |
+ | month = {July}, | ||
url = {http:// | url = {http:// | ||
} | } | ||
+ | |||
+ | ==== Implementation ==== | ||
+ | |||
+ | The complete source code of | ||
+ | [[http:// | ||
+ | is available on | ||
+ | [[http:// | ||
+ | This source code comes with a script that replicates the | ||
+ | experiments discussed in this paper. | ||
+ | |||
+ | |||
+ | ==== Appendix ==== | ||
+ | |||
+ | The appendix contains a derivation of upper and lower bounds | ||
+ | on the asymptotic convergence speed of stochastic gradient algorithm. | ||
+ | The constants are exact in the case of second order stochastic gradient. | ||
+ | |||
+ | |||
+ | ==== Errata ==== | ||
+ | |||
+ | The SGDQN algorithm as described in this paper contains a subtle flaw | ||
+ | described in a subsequent [[: | ||
+ | |||
+ | There is a missing 1/2 factor in the bounds of theorem 1. | ||
+ | |||
+ | \[ | ||
+ | | ||
+ | | ||
+ | ~\leq~ \mathbb{E}_{\sigma}\big[\: | ||
+ | {\frac{1}{2}} \frac{{\mathrm tr}(\mathbf{HBGB})}{2\lambda_{\min}-1}\, | ||
+ | \] | ||
+ | |||
+ | The version of the paper found on this site contains the correct theorem and proof. | ||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||