Subsampled TDNNs
Overview
During the first years of my thesis, my main thema was the construction of speech recognition systems using neural networks. Kevin Lang and Geoff Hinton had published a tech report describing Time-Delay Neural Networks (TDNN). Alex Waibel and his team then demonstrated their efficiency at discriminating the japanese phonems /b/, /d/, /g/. But their approach was very costly. Training took weeks on their Alliant super-computer.
Using Stochastic Gradient Descent, I proposed a new and computationally efficient variant of Time-Delay Neural Networks. I was able to run speaker-independent word recognition systems on a regular workstation instead of a super-computer. This was later extended to continuous speech recognition system by combining a time-delay neural network and a Viterbi decoder. The combination was trained globally using a discriminant algorithm.
See Also
Publications
Léon Bottou: Reconnaissance de la parole par reseaux connexionnistes, Proceedings of Neuro Nimes 88, 197-218, Nimes, France, 1988.
Léon Bottou, Françoise. Fogelman Soulié, Pascal Blanchet and Jean Sylvain Lienard: Experiments with Time Delay Networks and Dynamic Time Warping for Speaker Independent Isolated Digit Recognition, Proceedings of EuroSpeech 89, 2:537-540, Paris, France, 1989.
M.D. Bedworth, L. Bottou, J. S. Bridle, F. Fallside, L. Flynn, F. Fogelman Soulié, K.M. Ponting and R.W. Prager: Comparison of neural and conventional classifiers on a speech recognition problem, Proceedings of IEE 1st International Conference on Artificial Neural Networks, London, 1989.
Xavier Driancourt and Léon Bottou: TDNN-Extracted features, Proceedings of Neuro Nimes 90, EC2, Nimes, France, 1990.
Léon Bottou, Françoise Fogelman Soulié, Pascal Blanchet and Jean Sylvain Lienard: Speaker independent isolated digit recognition: Multilayer perceptron vs Dynamic Time Warping, Neural Networks, 3:453-465, 1990.
Léon Bottou: Une Approche théorique de l’Apprentissage Connexionniste: Applications à la Reconnaissance de la Parole, Orsay, France, 1991.
Xavier Driancourt, Léon Bottou and Patrick Gallinari: Learning Vector Quantization, Multi Layer Perceptron and Dynamic Time Warping: Comparison and Cooperation, Proceedings of the International Joint Conference on Neural Networks, Seattle, 1991.
