leon.bottou.org

User Tools

Site Tools

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
projects:neuristique [2009/07/29 11:27]
leonb 		projects:neuristique [2023/09/14 20:23] (current)
leonb
Line 45: Line 45:
 and produced the latest descendant of SN: [[:projects:lush|Lush]]. and produced the latest descendant of SN: [[:projects:lush|Lush]].
 Almost twenty years after the initial version of SN,  Almost twenty years after the initial version of SN, 
-[[:projects:lush|Lush]] is still going strong!  +[[:projects:lush|Lush]] was still going strong!  
-===== TL/Prevision =====+The lack of GPU support makes it less relevant to the current developments. 
 +===== TL/Prevision and Early Data Mining =====
  
 After writing my [[:papers:bottou-91a|thesis]] and working with [[wp>Vapnik]] in [[wp>AT&T_Bell_Laboratories]], After writing my [[:papers:bottou-91a|thesis]] and working with [[wp>Vapnik]] in [[wp>AT&T_Bell_Laboratories]],
-it was clear that [[:research|Machine Learning]] was a scientific revolution in the making. +it was clear that [[:research|Machine Learning]] was a //scientific revolution// in the making. 
-We had practical successes and we were quickly acquiring a theoretical understanding. +The TL/Prevision project started when we understood one of the business aspects of that revolution.
-What were the business aspects of this revolution?+
  
-A few years earlier in [[wp>Ecole Poytechnique]],  +A few years earlier in [[wp>École Polytechnique]], [[https://www.iddri.org/en/about-iddri/team/claude-henry|Claude Henry]] 
-[[http://www.enseignement.polytechnique.fr/economie/membres/pageperso.php?id=448|Claude Henry]]+
 gave a wonderful class about micro-economical models of public economy. gave a wonderful class about micro-economical models of public economy.
-For instance, when you build a toll bridge, the cost must be paid before a single car passes the bridge. +Consider for instance the problem of funding a toll bridge. 
-Once the bridge is build, allowing an extra car costs nearly nothing. +Since letting an extra car cross the bridge costs nearly nothing once the bridge is built, 
-It seems reasonable to allow any extra car provided that the driver pays more than this small marginal cost.  +it seems reasonable to allow any car whose driver pays more than this small marginal cost.  
-On the other hand, if everybody pays a toll price equal to this marginal costs,  +On the other hand, if everybody pays a toll equal to this small marginal cost,  
-that will never pay for the fixed cost of building the bridge.  +the cost of building the bridge will not be recovered. 
-The best way to recover the fixed cost would be to guess how much each driver  +The best way to recover this initial cost would be to guess how much each  
-is willing to pay for the benefit of passing the bridge and charging that. +driver is willing to pay for the benefit of passing the bridge. 
-Therefore one designs complicated toll price structure based on a priori discrimination: +Therefore it is common practice to design complicated price  
-commercial vehicles pay more, peak hours are more expensive, etc.+structures based on a priori discrimination: commercial vehicles pay more,  
 +peak hours are more expensive, etc.
  
 {{tlp.png?400 }} {{tlp.png?400 }}
Line 71: Line 71:
 between the number of billable units and the total cost of the infrastructure. between the number of billable units and the total cost of the infrastructure.
 This applies to much more than building bridges. This applies to much more than building bridges.
-The nonconvexity appears whenever an activity benefits from economy of scale. +The nonconvexity appears whenever an activity benefits from economies of scale. 
-Airline pricing is an extreme example. +Businesses as different as airlines, software development, telecommunications,  
-Yet things as different as software pricing, telecommunication pricing obey similar rules. +or internet advertisement have in common that markets cannot determine a simple  
-Internet advertisement is a minor variant of that phenomenon:  +relation between the asking price and the level of service. 
-advertisers name their price and internet companies decide the level of service,  +In order to deploy other mechanisms,  
-that is, they decide where and when to show the advertisement. +businesses must rely on a priori knowledge and on statistical  
-All these businesses have in common that markets cannot  +data collected during their operation. 
-determine a simple relation between asked price and level of services. +This is in fact the fundamental driver for [[wp>Data Mining]] 
-In order to deploy other pricing mechanisms,  +and this is a good target for [[wp>Machine Learning]].
-these businesses must rely on statistical data collected during their operation. +
-I am still convinced that this is the fundamental driver of the [[wp>Data Mining]] industry.+
  
- +The TL/Prevision project took shape when I realized this connection.
-The TL/Prevision project took shape when I realized that this +
-was an powerful business case for machine learning.+
 The idea was to develop an automated machine learning package  The idea was to develop an automated machine learning package 
 able to collect data from the businesses'[[wp>SQL]] [[wp>databases]] able to collect data from the businesses'[[wp>SQL]] [[wp>databases]]
Line 91: Line 87:
 The internal algorithms were based on [[:research:local|Locally Regularized Learning Systems]] The internal algorithms were based on [[:research:local|Locally Regularized Learning Systems]]
 and optimized using [[http://books.nips.cc/papers/files/nips06/0059.pdf|Hoeffding races]] and optimized using [[http://books.nips.cc/papers/files/nips06/0059.pdf|Hoeffding races]]
-(in fact TL/Prevision performs [[http://hunch.net/?p=268|Bennet races]])+(in fact TL/Prevision performs [[http://hunch.net/?p=268|Bennett races]])
 The software package was implemented by Neuristique using The software package was implemented by Neuristique using
 seed funds from [[http://www.oseo.fr|ANVAR]]  seed funds from [[http://www.oseo.fr|ANVAR]] 
Line 107: Line 103:
 Every small disagreement took existential proportions.   Every small disagreement took existential proportions.  
 That was exhausting. I left Neuristique in 2000. That was exhausting. I left Neuristique in 2000.
-Xavier Driancourt (http://www.orlogia.com) ran it  +Xavier Driancourt ran it profitably until 2003 when we decided that 
-profitably until 2003 when we decided that +its business activity no longer required the relatively costly legal structure of the company.
-its business activity no longer required the +
-relatively costly legal structure of the company. +
-Xavier still runs [[http://www.orlogia.com|Orlogia]], a consulting business.+
  
 Despite these disappointements, the legacy of Neuristique is very positive: Despite these disappointements, the legacy of Neuristique is very positive:
Line 117: Line 110:
    * Numerous machine learning algorithms have been developed with SN, including [[http://yann.lecun.com/exdb/lenet/|convolutionnal neural networks]] for image recognition or signal processing, and early implementations of [[wp>Support Vector Machines]].    * Numerous machine learning algorithms have been developed with SN, including [[http://yann.lecun.com/exdb/lenet/|convolutionnal neural networks]] for image recognition or signal processing, and early implementations of [[wp>Support Vector Machines]].
  
-   * A SN based handwriting recognition system was used by many banks across the world to read checks. Some ATM machines made by NCR use compiled SN code running on embedded DSP boards. A version of this check reader was embedded in large check reading engines sold by NCR and OrboGraph. According to some estimates, this system processes 10 to 20 percent of all the checks written in the US. +   * A SN based handwriting recognition system was used by many banks across the world to read checks. Some ATM machines made by NCR use compiled SN code running on embedded DSP boards. A version of this check reader was embedded in large check reading engines sold by NCR and OrboGraph. Over more than ten years, this system has processed about 10% of all the checks written in the US. 
  
-   * The first prototype of the DjVu image and document compression system, including the first decoder and the first foreground/background segmenter. +   * The first prototype of the DjVu image and document compression system was written using SN, including the first decoder and the first foreground/background segmentation algorithms. 
  
    * The TL/Prevision technology was licensed to [[kxen|KXEN Inc.]] and helped defining their initial view of the business.    * The TL/Prevision technology was licensed to [[kxen|KXEN Inc.]] and helped defining their initial view of the business.
Line 125: Line 118:
 ===== Acolytes ===== ===== Acolytes =====
  
-Here is a non exhaustive list of the Neuristique((please email me if you think your name is missing)) collaborators.+Here is a non exhaustive list of the Neuristique((please email me if you think your name is missing)) direct collaborators:
  
-  * Jean Bourrely ([[http://cat.inist.fr/?aModele=afficheN&cpsidt=150046|thesis]])+  * [[http://cat.inist.fr/?aModele=afficheN&cpsidt=150046|Jean Bourrely]] 
 +  * [[https://fr.linkedin.com/in/michel-de-bollivier-b44580|Michel de Bollivier]]
   * Michel Diricq   * Michel Diricq
-  * Xavier Driancourt ([[http://www.orlogia.com]]) +  * [[https://fr.linkedin.com/in/xavier-driancourt-8593701|Xavier Driancourt]] 
-  * Christian and Marie-Pia Ignace +  * [[https://fr.linkedin.com/in/christian-ignace-16247524|Christian]] and [[https://fr.linkedin.com/in/mariepiaignace|Marie-Pia Ignace]] 
-  * Yann LeCun ([[http://yann.lecun.com]]) +  * [[http://yann.lecun.com|Yann LeCun]] 
-  * Steve Suddarth ([[http://www.informatik.uni-trier.de/~ley/db/indices/a-tree/s/Suddarth:Steven_C=.html|pubs]])+  * [[http://transparentsky.net/|Steve Suddarth]] 
 + 
 +Neuristique technologies also benefited from the contributions of many more people: 
 + 
 +  * [[https://lipn.univ-paris13.fr/~bennani|Younes Bennani]] 
 +  * [[https://www.researchgate.net/profile/Carlos_Mejia4|Carlos Mejia]]) 
 +  * ... 
  
-See also [[http://www.neuristique.fr]]. 
projects/neuristique.1248881222.txt.gz · Last modified: 2009/07/29 11:27 by leonb

Page Tools