Differences

This shows you the differences between two versions of the page.

--- papers:tr-bottou-2011 [2016/09/12 09:34]
leonb [Notes]
+++ papers:tr-bottou-2011 [2016/09/12 09:44] (current)
leonb [Notes]
@@ Line 31: / Line 31: @@
 ===== Notes =====
-This documents cite the work of Vincent Etter (2009) carried out during his NEC Labs internship. Vincent's master report is now available on [[http://vincent.etter.io/publications/etter2009master.pdf|his home page]] ({{vincentetter.pdf|local copy}}). Section 5 is an exploration of that were extensively discussed between Ronan Collobert, Jason Weston and I. We had the hope to discover relevant recursive sentence representation in an unsupervised manner. Alas we found that the shape of the structure of a recursive network has very little impact on its representation abilities, something that was confirmed by [[https://arxiv.org/pdf/1301.2811.pdf|Scheible and Schütze]] on a sentiment classification task. Even a left-to-right tree (which amounts to using a recurrent neural network in fact) worked essentially as well, something that was later confirmed by [[http://arxiv.org/pdf/1506.01057v1.pdf|Li et al.]] on a variety of tasks.  I still had hopes to make it work when I wrote this tech report in 2010. I now believe that structure discovery needs a completely different approach on the cost functions (2016).
+This documents cite the work of Vincent Etter (2009) carried out during his NEC Labs internship. Vincent's master report is now available on [[http://vincent.etter.io/publications/etter2009master.pdf|his home page]] ({{vincentetter.pdf|local copy}}). Section 5 is an exploration of that were extensively discussed between Ronan Collobert, Jason Weston and I. We had the hope to discover relevant recursive sentence representation in an unsupervised manner. Alas, we found that the shape of the structure of a recursive network has very little impact on its representation abilities, something that was clearly confirmed by [[https://arxiv.org/pdf/1301.2811.pdf|Scheible and Schütze (2013)]] on a sentiment classification task. Even a left-to-right tree (which amounts to using a recurrent neural network in fact) works essentially as well, something that was cleanly confirmed by [[http://arxiv.org/pdf/1506.01057v1.pdf|Li et al. (2015)]] on a broad collection of NLP tasks.  I still had hopes to make it work when I wrote this tech report in 2010. However these two works have convinced me that structure discovery won't happen without a new idea.

User Tools

Site Tools

Differences

Page Tools