Locally Regularized Learning Systems

After joining AT&T Bell Laboratories in 1991, I worked with Vladimir Vapnik on the links between learning technique and the widely used regularization techniques. I applied local regularization methods to optical character recognition and bettered all previous systems on our USPS benchmark with a 3.3% error rate. Alas the algorithm was quite slow. Three months later, Patrice Simard took the record, achieving 2.5% error with Tangent Distances. I do not remember which was the slowest algorithm.