A 2026 study by Żywot, Chen, Yuan, Søgaard, and de Rijke showed that a minimal multi-agent system — one orchestrator model dispatching to specialized tool-using sub-agents, all instantiated from small Qwen3 language models — can match or outperform a substantially larger single model with direct tool access on agentic benchmarks, with explicit reasoning ('thinking') helping at the orchestrator and hurting in sub-agents. A second version of the study (April 2026) extended its evaluation to GPQA, AIME, MuSiQue, and Humanity's Last Exam. The open problem is twofold: (i) design and evaluation of dynamic thinking policies that adaptively enable or disable explicit reasoning per agent role and task, seeking to preserve thinking's decomposition benefits while avoiding its documented coordination failures; and (ii) whether the scale-versus-orchestration and scale-versus-tool trade-offs, and the dominance of orchestrator capacity, generalize beyond the Qwen3 family, beyond one fixed tool stack, and to separately instantiated agents.

Background

Large language models (LLMs) are increasingly deployed as agents: systems that interleave natural-language reasoning with external actions such as web search, code execution, and file inspection. Performance on such tasks has traditionally been assumed to grow with parameter count ('model scale'), but recent work shows larger models do not reliably improve on multi-step tool-use tasks, and argues that small models with the right scaffolding are competitive agents (Belcak et al., 2025). Two architectural levers compete with raw scale. First, multi-agent orchestration: a supervising 'orchestrator' model decomposes a task and dispatches sub-tasks to specialized sub-agents (each with its own system prompt and tool, e.g. a web searcher, a coder, and a file inspector), connected through structured memory — a compact shared workspace rather than full conversation histories; model-and-tool orchestration systems of this kind include ToolOrchestra (Su et al., 2025). Second, explicit 'thinking' modes: chain-of-thought-style reasoning traces that hybrid models such as Qwen3 (Yang et al., 2025) can switch on or off. Żywot et al. (2026) ran controlled experiments on a minimal orchestrator-plus-sub-agents system built entirely from small open-source Qwen3 models (1.7B to 32B parameters, 'B' denoting billions), evaluated on GAIA (a benchmark of tool-requiring general-assistant tasks; Mialon et al., 2023), GPQA (graduate-level science questions designed to resist web search; Rein et al., 2023), AIME (problems from the American Invitational Mathematics Examination), MuSiQue (compositional multi-hop question answering; Trivedi et al., 2021), and Humanity's Last Exam, a set of expert-level academic questions (Phan et al., 2025). Their findings: an 8B-parameter multi-agent system with orchestrator-level thinking matches a 32B single-agent model with direct tool access on GAIA and substantially exceeds it on AIME; tool access helps some benchmarks and hurts others in single-agent use; explicit reasoning at the orchestrator yields the largest gains while reasoning in sub-agents is neutral or harmful; and overall performance is driven by orchestrator capacity far more than sub-agent capacity. A related study of hierarchical search agents independently found the same capacity asymmetry — scaling the delegation backbone matters far more than scaling the executor — across Qwen3 and DeepSeek-V4 backbones (Cai et al., 2026). These conclusions rest, however, on one model family, one fixed set of tool implementations and retrieval pipelines, static per-run thinking configurations, and (in the study's first version) a single shared model instance dynamically re-roled by prompts to save GPU memory. Work on adaptive reasoning exists only outside multi-agent orchestration: turn-adaptive thinking budgets for single multi-turn agents (Jali et al., 2026) and reinforcement-learned reasoning-mode switching for single models (Fang et al., 2026), plus a theoretical information-bottleneck analysis of when multi-agent systems beat single-agent ones (Yu et al., 2026). Whether the orchestration-versus-scaling trade-offs generalize and whether explicit reasoning can be allocated dynamically across agent roles remain open.

Problem Statement

The source study established its trade-offs under one model family (Qwen3), one fixed tool stack, and static thinking configurations. The open problem is to establish the external validity and controllability of these orchestration-versus-scaling gains. Its two linked questions share one scientific objective — determining whether small-model orchestration beating large single models is a general, controllable phenomenon rather than an artifact of one experimental setup: (1) Dynamic thinking policies: design and evaluate policies that decide at run time, from signals such as task difficulty, model uncertainty, agent role, or conversation state, whether explicit reasoning is enabled at the orchestrator, at sub-agents, both, or neither; and determine whether such policies preserve the task-decomposition benefits of orchestrator reasoning while avoiding the coordination failures observed under static configurations (tool-call thrashing, query corruption, premature closure). (2) Generality across backbones and tooling: determine whether the scale-versus-orchestration and scale-versus-tool trade-offs, and the dominance of orchestrator capacity over sub-agent capacity, persist across model families beyond Qwen3, across alternative tool implementations and retrieval pipelines (tool design, retrieval quality, search-index freshness, sandboxed coding environments), and under fully separate agent instances rather than shared-backbone prompt-based role assignment.

References

  1. Agata Żywot, Xinyi Chen, Yifei Yuan, Anders Søgaard, and Maarten de Rijke. Can Small Agents Collaborate to Beat a Single Large Language Model? (v2, April 2026; version 1, January 2026, titled 'Can Small Agent Collaboration Beat a Single Big LLM?'). arXiv:2601.11327, 2026. DOI: 10.48550/arXiv.2601.11327. https://arxiv.org/abs/2601.11327
  2. Grégoire Mialon, Clémentine Fourrier, Craig Swift, Thomas Wolf, Yann LeCun, and Thomas Scialom. GAIA: a benchmark for General AI Assistants. arXiv:2311.12983, 2023. DOI: 10.48550/arXiv.2311.12983. https://arxiv.org/abs/2311.12983
  3. David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. GPQA: A Graduate-Level Google-Proof Q&A Benchmark. arXiv:2311.12022, 2023. DOI: 10.48550/arXiv.2311.12022. https://arxiv.org/abs/2311.12022
  4. Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. MuSiQue: Multihop Questions via Single-hop Question Composition. arXiv:2108.00573, 2021. DOI: 10.48550/arXiv.2108.00573. https://arxiv.org/abs/2108.00573
  5. Long Phan, Alice Gatti, Ziwen Han, Nathaniel Li, et al. Humanity's Last Exam. arXiv:2501.14249, 2025 (published as 'A benchmark of expert-level academic questions to assess AI capabilities', Nature 649:1139-1146, 2026, DOI: 10.1038/s41586-025-09962-4). DOI: 10.48550/arXiv.2501.14249. https://arxiv.org/abs/2501.14249
  6. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, et al. Qwen3 Technical Report. arXiv:2505.09388, 2025. DOI: 10.48550/arXiv.2505.09388. https://arxiv.org/abs/2505.09388
  7. Peter Belcak, Greg Heinrich, Shizhe Diao, Yonggan Fu, Xin Dong, Saurav Muralidharan, Yingyan Celine Lin, and Pavlo Molchanov. Small Language Models are the Future of Agentic AI. arXiv:2506.02153, 2025. DOI: 10.48550/arXiv.2506.02153. https://arxiv.org/abs/2506.02153
  8. Hongjin Su, Shizhe Diao, Ximing Lu, Mingjie Liu, Jiacheng Xu, Xin Dong, Yonggan Fu, Peter Belcak, Hanrong Ye, Hongxu Yin, Yi Dong, Evelina Bakhturina, Tao Yu, Yejin Choi, Jan Kautz, and Pavlo Molchanov. ToolOrchestra: Elevating Intelligence via Efficient Model and Tool Orchestration. arXiv:2511.21689, 2025. DOI: 10.48550/arXiv.2511.21689. https://arxiv.org/abs/2511.21689
  9. Qinnan Cai, Yibo Zhao, and Xiang Li. Think Big, Search Small: Where Capacity Matters in Hierarchical Search Agents? arXiv:2607.07548, 2026. DOI: 10.48550/arXiv.2607.07548. https://arxiv.org/abs/2607.07548
  10. Neharika Jali, Anupam Nayak, and Gauri Joshi. Not All Turns Are Equally Hard: Adaptive Thinking Budgets For Efficient Multi-Turn Reasoning in Agents. arXiv:2604.05164, 2026. DOI: 10.48550/arXiv.2604.05164. https://arxiv.org/abs/2604.05164
  11. Yiyang Fang, Pei Fu, Jinjie Li, Jian Liang, Wenke Huang, Ruijie Luo, Shaojie Zhang, Jian Luan, Yi R. Fung, and Mang Ye. Switch-Reasoner: Learn When to Think in Multitask Mixtures via Reinforcement Learning. arXiv:2607.08572, 2026. DOI: 10.48550/arXiv.2607.08572. https://arxiv.org/abs/2607.08572
  12. Wendi Yu, Lianhao Zhou, Xiangjue Dong, Sai Sudarshan Barath, Declan Staunton, Byung-Jun Yoon, Xiaoning Qian, James Caverlee, and Shuiwang Ji. When Do Multi-Agent Systems Help? An Information Bottleneck Perspective. arXiv:2607.16133, 2026. DOI: 10.48550/arXiv.2607.16133. https://arxiv.org/abs/2607.16133
  13. Matthew Sharp, O. Bilgin, Iason Gabriel, and Lewis Hammond. Agentic Inequality. arXiv:2510.16853, 2025. DOI: 10.48550/arXiv.2510.16853. https://arxiv.org/abs/2510.16853