Skip to content
CaMLSysUniversity of Cambridge
Home People Publications Projects Contact

← All publications

Sheaf HyperNetworks for Personalized Federated Learning

  • Bao NguyenUniversity of Cambridge
  • Lorenzo SaniUniversity of CambridgeFlower Labs
  • Xinchi QiuUniversity of Cambridge
  • Pietro LiòUniversity of Cambridge
  • Nicholas D. LaneUniversity of CambridgeFlower Labs

Preprint arXiv, 2024

Abstract

Graph hypernetworks (GHNs), constructed by combining graph neural networks (GNNs) with hypernetworks (HNs), leverage relational data across various domains such as neural architecture search, molecular property prediction and federated learning. Despite GNNs and HNs being individually successful, we show that GHNs present problems compromising their performance, such as over-smoothing and heterophily. Moreover, we cannot apply GHNs directly to personalized federated learning (PFL) scenarios, where a priori client relation graph may be absent, private, or inaccessible. To mitigate these limitations in the context of PFL, we propose a novel class of HNs, sheaf hypernetworks (SHNs), which combine cellular sheaf theory with HNs to improve parameter sharing for PFL. We thoroughly evaluate SHNs across diverse PFL tasks, including multi-class classification, traffic and weather forecasting. Additionally, we provide a methodology for constructing client relation graphs in scenarios where such graphs are unavailable. We show that SHNs consistently outperform existing PFL solutions in complex non-IID scenarios. While the baselines’ performance fluctuates depending on the task, SHNs show improvements of up to 2.7% in accuracy and 5.3% in lower mean squared error over the best-performing baseline.

Materials

  • PDF
BibTeX
@misc{nguyen2024sheafhypernetworks,
  title = {{Sheaf HyperNetworks for Personalized Federated Learning}},
  author = {Nguyen, Bao and Sani, Lorenzo and Qiu, Xinchi and Liò, Pietro and Lane, Nicholas D.},
  year = {2024},
  url = {https://arxiv.org/abs/2405.20882},
  eprint = {2405.20882},
  archiveprefix = {arXiv}
}
Cambridge Machine Learning Systems Group
Department of Computer Science and Technology
University of Cambridge
ContactGitHub