Special ML Seminar
The talk is based on the following two talks: Talk 1 " End-2-end neural models for automatic speech recognition" 1) QuartzNet: deep convolutional acoustic model which achieves state-of-the-art accuracy on LibriSpeech and Wall Street Journal, while having 3x-5x fewer parameters than competing models, 2) ASR post-processing model which "translates" ASR output into grammatically and semantically correct text. The Transformer-based encoder-decoder model demonstrates significant improvement in word error rate over the baseline acoustic model with greedy decoding. It outperforms baseline 6-gram language model re-scoring and approaches the performance of re-scoring with Transformer-XL neural language model. Talk 2 "NovoGrad: a stochastic gradient method with layer-wise gradient normalization" Abstract: We propose NovoGrad, a first-order stochastic gradient method with layer-wise gradient normalization via second moment estimators and with decoupled weight decay. The method requires half as much memory as Adam/AdamW. We evaluated NovoGrad on the diverse set of problems, including image classification, speech recognition, neural machine translation and language modeling. On these problems, NovoGrad performed equal to or better than SGD and Adam/AdamW. Bio: Boris Ginzburg is Principal Engineer in NVIDIA, working on deep learning applications for speech and language processing. He joined NVIDIA in 2015. Before that he worked in Intel on HW for deep learning, CPU, and wireless networking. Boris has Ph.D. in Applied Math from Technion.
Date: Sun 10 Nov 2019
Start Time: 14:00
End Time: 15:00
1003 | Electrical Eng. Building