Skip to content
CaMLSysUniversity of Cambridge
Home People Publications Projects Contact

← All publications

ShrinkML: End-to-End ASR Model Compression Using Reinforcement Learning

  • Łukasz DudziakSamsung AI Center, Cambridge
  • Mohamed S. AbdelfattahSamsung AI Center, Cambridge
  • Ravichander VipperlaSamsung AI Center, Cambridge
  • Stefanos LaskaridisSamsung AI Center, Cambridge
  • Nicholas D. LaneSamsung AI Center, CambridgeUniversity of Oxford

Interspeech, 2019

Abstract

End-to-end automatic speech recognition (ASR) models are increasingly large and complex to achieve the best possible accuracy. In this paper, we build an AutoML system that uses reinforcement learning (RL) to optimize the per-layer compression ratios when applied to a state-of-the-art attention based end-to-end ASR model composed of several LSTM layers. We use singular value decomposition (SVD) low-rank matrix factorization as the compression method. For our RL-based AutoML system, we focus on practical considerations such as the choice of the reward/punishment functions, the formation of an effective search space, and the creation of a representative but small data set for quick evaluation between search steps. Finally, we present accuracy results on LibriSpeech of the model compressed by our AutoML system, and we compare it to manually-compressed models. Our results show that in the absence of retraining our RL-based search is an effective and practical method to compress a production-grade ASR system. When retraining is possible, we show that our AutoML system can select better highly-compressed seed models compared to manually hand-crafted rank selection, thus allowing for more compression than previously possible.

Materials

  • PDF
BibTeX
@inproceedings{dudziak2019shrinkml,
  title = {{ShrinkML: End-to-End ASR Model Compression Using Reinforcement Learning}},
  author = {Dudziak, Łukasz and Abdelfattah, Mohamed S. and Vipperla, Ravichander and Laskaridis, Stefanos and Lane, Nicholas D.},
  year = {2019},
  booktitle = {Interspeech},
  pages = {2235--2239},
  publisher = {ISCA},
  doi = {10.21437/interspeech.2019-2811},
  url = {https://doi.org/10.21437/interspeech.2019-2811}
}
Cambridge Machine Learning Systems Group
Department of Computer Science and Technology
University of Cambridge
ContactGitHub