Skip to content
CaMLSysUniversity of Cambridge
Home People Publications Projects Contact

← All publications

FedAnchor: Enhancing Federated Semi-Supervised Learning with Label Contrastive Loss for Unlabeled Clients

  • Xinchi QiuUniversity of Cambridge
  • Yan GaoUniversity of Cambridge
  • Lorenzo SaniUniversity of Cambridge
  • Heng PanUniversity of Cambridge
  • Wanru ZhaoUniversity of Cambridge
  • Pedro P. B. GusmaoUniversity of Surrey
  • Mina AlibeigiZenseact
  • Alex IacobUniversity of Cambridge
  • Nicholas D. LaneUniversity of Cambridge

Preprint arXiv, 2024

Abstract

Federated learning (FL) is a distributed learning paradigm that facilitates collaborative training of a shared global model across devices while keeping data localized. The deployment of FL in numerous real-world applications faces delays, primarily due to the prevalent reliance on supervised tasks. Generating detailed labels at edge devices, if feasible, is demanding, given resource constraints and the imperative for continuous data updates. In addressing these challenges, solutions such as federated semi-supervised learning (FSSL), which relies on unlabeled clients’ data and a limited amount of labeled data on the server, become pivotal. In this paper, we propose FedAnchor, an innovative FSSL method that introduces a unique double-head structure, called anchor head, paired with the classification head trained exclusively on labeled anchor data on the server. The anchor head is empowered with a newly designed label contrastive loss based on the cosine similarity metric. Our approach mitigates the confirmation bias and overfitting issues associated with pseudo-labeling techniques based on high-confidence model prediction samples. Extensive experiments on CIFAR10/100 and SVHN datasets demonstrate that our method outperforms the state-of-the-art method by a significant margin in terms of convergence rate and model accuracy.

Materials

  • PDF
BibTeX
@misc{qiu2024fedanchor,
  title = {{FedAnchor: Enhancing Federated Semi-Supervised Learning with Label Contrastive Loss for Unlabeled Clients}},
  author = {Qiu, Xinchi and Gao, Yan and Sani, Lorenzo and Pan, Heng and Zhao, Wanru and Gusmao, Pedro P. B. and Alibeigi, Mina and Iacob, Alex and Lane, Nicholas D.},
  year = {2024},
  url = {https://arxiv.org/abs/2402.10191},
  eprint = {2402.10191},
  archiveprefix = {arXiv}
}
Cambridge Machine Learning Systems Group
Department of Computer Science and Technology
University of Cambridge
ContactGitHub