Erratum: SGDQN is Less Careful than Expected

Abstract: The SGDQN algorithm described in (Bordes et al., 2009) contains a subtle flaw that prevents it from reaching its design goals. Yet the flawed SGDQN algorithm has worked well enough to be a winner of the first Pascal Large Scale Learning Challenge (Sonnenburg et al., 2008). This document clarifies the situation, proposes a corrected algorithm, and evaluates its performance.

Antoine Bordes, Léon Bottou, Patrick Gallinari, Jonathan Chang and S. Alex Smith: Erratum: SGDQN is Less Careful than Expected, Journal of Machine Learning Research, 11:2229–2240, Aug 2010.

JMLR Link    jmlr-2010.djvu jmlr-2010.pdf jmlr-2010.ps.gz 

@article{bordes-2010,
  author = {Bordes, Antoine and Bottou, L\'{e}on and Gallinari, Patrick 
            and Chang, Jonathan and Smith, S. Alex},
  title = {Erratum: SGDQN is Less Careful than Expected},
  journal = {Journal of Machine Learning Research},
  year = {2010},
  volume = {11},
  pages = {2229--2240},
  month = {Aug},
  url = {http://leon.bottou.org/papers/bordes-2010},
}

Remark

This paper started as an small erratum for (Bordes et al., 2009) and eventually morphed into a full contribution subjected to the normal review process.