Skip to content
CaMLSysUniversity of Cambridge
Home People Publications Projects Contact

← All publications

Response-Conditioned Parallel-to-Sequential Orchestration for Multi-Agent Systems

  • Nurbek TastanMBZUAIUniversity of Cambridge
  • Alex IacobUniversity of CambridgeFlower Labs
  • Lorenzo SaniUniversity of CambridgeFlower Labs
  • Meghdad KurmanjiUniversity of Cambridge
  • Nicholas D. LaneUniversity of CambridgeFlower Labs
  • Samuel HorvathMBZUAI
  • Karthik NandakumarMBZUAIMichigan State University

Preprint arXiv, 2026

Abstract

Multi-agent systems can solve complex tasks through collaboration between multiple Large Language Model agents. Existing collaboration frameworks typically operate in either a parallel or a sequential mode. In the parallel mode, agents respond independently to queries followed by aggregation of responses. In contrast, sequential systems allow agents to communicate via a directed topology and refine one another step by step. However, both modes are inadequate for achieving the desired objectives of minimizing communication and latency while simultaneously maximizing the accuracy of the final response. In this work, we introduce a hybrid paradigm called Nexa, a trainable response-conditioned policy that bridges the gap between the two modes. Nexa begins with a parallel execution stage, embeds the resulting responses into a shared semantic space, and then predicts a sparse directed acyclic communication graph. If the graph is empty, the system remains purely parallel; if it is non-empty, the system performs one sequential message propagation. The policy is a lightweight transformer model, and the method avoids the need for external LLM judges or reward models, as well as hand-crafted test-time topology search. We formalize this hybrid execution problem, show that the resulting graph is acyclic by construction, and that the framework strictly subsumes pure parallel execution, and present a training procedure based on policy-gradient optimization. Results demonstrate that the response-conditioned policy learned by Nexa under one setting can be reused when the number of agents, the task, or the underlying agent changes, thus emphasizing the generalizability of the learned communication policy.

Materials

  • PDF
BibTeX
@misc{tastan2026responseconditionedorchestration,
  title = {{Response-Conditioned Parallel-to-Sequential Orchestration for Multi-Agent Systems}},
  author = {Tastan, Nurbek and Iacob, Alex and Sani, Lorenzo and Kurmanji, Meghdad and Lane, Nicholas D. and Horvath, Samuel and Nandakumar, Karthik},
  year = {2026},
  url = {https://arxiv.org/abs/2605.15573},
  eprint = {2605.15573},
  archiveprefix = {arXiv}
}
Cambridge Machine Learning Systems Group
Department of Computer Science and Technology
University of Cambridge
ContactGitHub