This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision | ||
papers:bordes-bottou-gallinari-2009 [2009/08/07 10:27] leonb |
papers:bordes-bottou-gallinari-2009 [2017/11/29 10:27] (current) leonb [Errata] |
||
---|---|---|---|
Line 12: | Line 12: | ||
PASCAL Large Scale Learning Challenge. | PASCAL Large Scale Learning Challenge. | ||
- | //Note//: | + | < |
- | The appendix contains a derivation of upper and lower bounds | + | //Errata//: |
- | on the asymptotic convergence speed of stochastic gradient algorithm. | + | Please see section [[#Errata]] below. |
- | This result is exact in the case of second order stochastic gradient. | + | < |
<box 99% orange> | <box 99% orange> | ||
Line 22: | Line 21: | ||
[[http:// | [[http:// | ||
+ | [[http:// | ||
+ | < | ||
[[http:// | [[http:// | ||
[[http:// | [[http:// | ||
Line 46: | Line 47: | ||
This source code comes with a script that replicates the | This source code comes with a script that replicates the | ||
experiments discussed in this paper. | experiments discussed in this paper. | ||
+ | |||
+ | |||
+ | ==== Appendix ==== | ||
+ | |||
+ | The appendix contains a derivation of upper and lower bounds | ||
+ | on the asymptotic convergence speed of stochastic gradient algorithm. | ||
+ | The constants are exact in the case of second order stochastic gradient. | ||
+ | |||
+ | |||
+ | ==== Errata ==== | ||
+ | |||
+ | The SGDQN algorithm as described in this paper contains a subtle flaw | ||
+ | described in a subsequent [[: | ||
+ | |||
+ | There is a missing 1/2 factor in the bounds of theorem 1. | ||
+ | |||
+ | \[ | ||
+ | | ||
+ | | ||
+ | ~\leq~ \mathbb{E}_{\sigma}\big[\: | ||
+ | {\frac{1}{2}} \frac{{\mathrm tr}(\mathbf{HBGB})}{2\lambda_{\min}-1}\, | ||
+ | \] | ||
+ | |||
+ | The version of the paper found on this site contains the correct theorem and proof. | ||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||