SHIRITAS

論文 / arXiv:2611.12856

WorkflowArena: Benchmarking Executable Workflow Orchestration in LLM-Based Multi-Agent Systems

PLAIN SUMMARY / やさしい要約

複数のAIに仕事を分担させるとき、大事なのは「何を誰に任せ、どう順番に進めるか」です。この研究は、その段取りの良さを別の能力として捉え、測るための課題集(ベンチマーク)を提案します。各課題では、最初に渡される作業場所と、いくつかの部品が組み合わさった最終成果物が決まっています。たとえるなら、料理で「材料と台所」を渡して、完成した料理だけでなく、下ごしらえを誰に任せたかも一緒に見るような考え方です。

AIが専門用語を使わずに書いた解説です。内容の正確さは、下の原文の要旨で確認してください。

ABSTRACT / 要旨(原文)

Large language model agents are increasingly deployed as multi-agent systems rather than as a standalone model operating within a single context. Recent agent harnesses allow the model to instantiate worker agents and to author the executable workflow that coordinates them, shifting the design of system structure from developers to the model at run time. The quality of this organization therefore constitutes a distinct capability, yet existing benchmarks either measure end-to-end task completion or evaluate coordination under structures specified in advance. We formalize this capability as workflow orchestration and introduce WorkflowArena, a benchmark of 145 long-horizon tasks spanning 8 professional domains, from software engineering and formal mathematics to legal analysis and 3D production. Each task specifies an initial workspace and an integrated deliverable composed of interdependent components, which is assessed by a hidden verifier. Agents determine autonomously whether and how to delegate, and all delegation is recorded, enabling joint evaluation of deliverable quality and the organization of the underlying work. Experiments with nine models show that no single configuration performs best across all domains and that systems exhibit distinct delegation strategies: at high effort, Claude models in Claude Code invoke workflows less frequently but with larger worker groups, whereas GPT models in Codex delegate more frequently in smaller units. When the executing model is held fixed on a challenging subset of tasks, workflows authored by Claude Opus 5.5 substantially improve weaker executors, indicating that workflow organization is itself a measurable determinant of task performance.

ここに表示しているのは原論文の要旨です。AIによる要約や解釈は含みません。

この論文を取り上げた記事