Skip to content
CaMLSysUniversity of Cambridge
Home People Publications Projects Contact

← All publications

Deterministic Binary Filters for Convolutional Neural Networks

  • Vincent W.-S. TsengCornell University
  • Sourav BhattacharyaNokia Bell Labs
  • Javier Fernández MarquésUniversity of Oxford
  • Milad AlizadehUniversity of Oxford
  • Catherine TongUniversity of Oxford
  • Nicholas D. LaneUniversity of OxfordNokia Bell Labs

International Joint Conference on Artificial Intelligence (IJCAI), 2018

Abstract

We propose Deterministic Binary Filters, an approach to Convolutional Neural Networks that learns weighting coefficients of predefined orthogonal binary basis instead of the conventional approach of learning directly the convolutional filters. This approach results in model architectures with significantly fewer parameters (4x to 16x) and smaller model sizes (32x due to the use of binary rather than floating point precision). We show our deterministic filter design can be integrated into well-known network architectures (such as ResNet and SqueezeNet) with as little as 2% loss of accuracy (under datasets like CIFAR-10). Under ImageNet, they result in 3x model size reduction compared to sub-megabyte binary networks while reaching comparable accuracy levels.

Materials

  • Publisher
BibTeX
@inproceedings{tseng2018deterministic,
  title = {{Deterministic Binary Filters for Convolutional Neural Networks}},
  author = {Tseng, Vincent W.-S. and Bhattacharya, Sourav and Fernández Marqués, Javier and Alizadeh, Milad and Tong, Catherine and Lane, Nicholas D.},
  year = {2018},
  booktitle = {International Joint Conference on Artificial Intelligence (IJCAI)},
  pages = {2739--2747},
  doi = {10.24963/ijcai.2018/380},
  url = {https://www.ijcai.org/proceedings/2018/380}
}
Cambridge Machine Learning Systems Group
Department of Computer Science and Technology
University of Cambridge
ContactGitHub