論文 / arXiv:2610.20597
Co-Evolving Acting and Forecasting for Language Agents
PLAIN SUMMARY / やさしい要約
長い手順が必要な作業をAIに学ばせると、試行に時間や計算がかかりがちです。この研究は、ゴール時の点数だけでなく、うまくいった手順から「この先に続く動き方」を学ぶ仕組みを提案します。たとえば、迷路を解けた人の足取りを見て、途中の地点ごとに“次にどう進むか”も練習するようなイメージです。実験では、いくつかの評価環境で既存手法より良い結果や学習の効率向上が報告されています。
AIが専門用語を使わずに書いた解説です。内容の正確さは、下の原文の要旨で確認してください。
ABSTRACT / 要旨(原文)
Training language agents is bottlenecked by the high cost of long-horizon rollouts, yet standard agentic reinforcement learning (RL) learns only from a terminal scalar reward, leaving the supervision in intermediate steps largely unused. Prior work exploits this supervision through value targets that estimate per-step success probabilities or observation targets that fit environment response. Both are hard to construct well: (1) value targets come from the same single scalar signal for each step in the trajectory, and (2) observation targets, such as web pages and terminal logs, are long and noisy and reach only one step ahead. We propose \name, which adds a third kind of multi-step action targets. On successful on-policy trajectories, \name trains the policy backbone to generate the action sequences that follows each history. Each action is also supervised when forecast from the preceding histories, forming a temporal ensemble that the shared backbone absorbs during training, and this enables acting and forecasting to co-evolve. The policy still acts one step at a time at inference, with no overhead. Across five benchmarks (ALFWorld, ScienceWorld, WebShop, $τ^2$-bench, and AppWorld), \name (1) achieves stronger aggregate performance than GRPO and RL baselines with predictive auxiliary targets across simulated environments, while improving overall performance on both realistic applications; (2) delivers up to $1.9\times$ and $2.0\times$ higher sample efficiency than ECHO and GRPO, respectively, on ScienceWorld; and (3) achieves these gains with a single policy backbone, without additional models, separate pretraining, or inference overhead. The codebase is available at https://github.com/ulab-uiuc/CoPE.
ここに表示しているのは原論文の要旨です。AIによる要約や解釈は含みません。
