Skip to content
CaMLSysUniversity of Cambridge
Home People Publications Projects Contact

← All publications

Ubiquitous memory augmentation via mobile multimodal embedding system

  • Dongqi CaiBeijing University of Posts and TelecommunicationsUniversity of Cambridge
  • Shangguang WangBeijing University of Posts and Telecommunications
  • Chen PengBeijing University of Posts and Telecommunications
  • Zeling ZhangBeijing University of Posts and Telecommunications
  • Zhenyan LuBeijing University of Posts and TelecommunicationsPengcheng Laboratory
  • Tao QiBeijing University of Posts and Telecommunications
  • Nicholas D. LaneUniversity of CambridgeFlower Labs
  • Mengwei XuBeijing University of Posts and Telecommunications

Nature Communications, 2025

Abstract

Forgetting is inevitable in human memory. Recently, multimodal embedding models have been proposed to vectorize multimodal reality into a unified embedding space. Once generated, these embeddings allow mobile users to quickly retrieve relevant information, effectively augmenting their memory. However, as the model’s capacity increases, its resource consumption also rises. The resulting slow throughput and significant computational resource requirements hinder its deployment on mobile devices. In this paper, we present Reminisce, an efficient on-device multimodal embedding system that enables high-throughput embedding and precise retrieval on resource-constrained mobile devices. The core design draws inspiration from the memory functions of the human brain, utilizing coarse-grained embeddings to identify likely candidates, which are then refined through query-driven fine-grained retrieval. A series of algorithm-hardware orchestrated optimizations automatically navigates this process and strengthen the embedding quality. Experiments show that Reminisce provides high-quality embedding representation with high throughput while operating silently in the background with negligible memory usage and reduced energy consumption.

Materials

  • Publisher
BibTeX
@article{cai2025ubiquitousmemory,
  title = {{Ubiquitous memory augmentation via mobile multimodal embedding system}},
  author = {Cai, Dongqi and Wang, Shangguang and Peng, Chen and Zhang, Zeling and Lu, Zhenyan and Qi, Tao and Lane, Nicholas D. and Xu, Mengwei},
  year = {2025},
  journal = {Nature Communications},
  volume = {16},
  number = {1},
  doi = {10.1038/s41467-025-60802-5},
  url = {https://doi.org/10.1038/s41467-025-60802-5}
}
Cambridge Machine Learning Systems Group
Department of Computer Science and Technology
University of Cambridge
ContactGitHub