論文 / arXiv:2610.35866
Auto-Research Arena: Benchmarking and Advancing Auto-Research in Language Modeling
PLAIN SUMMARY / やさしい要約
AIが研究を自動で進めるようになると、「どれが良い進め方か」を比べる物差しが必要になります。この研究は、自動研究のやり方を同じ条件で比べられる仕組み「Auto-Research Arena」を提案します。まず合格ライン(品質や使える計算資源)を決め、合格した案だけを次の勝負(どれだけ良くなったか)で比べる考え方です。たとえるなら、試合に出られる資格審査と、試合の得点を分けて運用するようなものです。
AIが専門用語を使わずに書いた解説です。内容の正確さは、下の原文の要旨で確認してください。
ABSTRACT / 要旨(原文)
As autonomous research systems become more capable, benchmarks must both evaluate their performance and support meaningful research progress. We introduce Auto-Research Arena, a multi-target benchmark for evaluating autonomous research systems and discovering ideas in language modeling. Arena supports diverse research objectives within a shared experimental environment by separating admissibility from improvement: model quality and resource constraints determine whether a result qualifies, while the selected target measures progress among admissible results. We instantiate Arena in language model pretraining with nine targets spanning model, data, training, and inference efficiency, and evaluate seven research harnesses using two foundation models. Arena clearly differentiates existing harnesses across research objectives, with several sophisticated methods outperformed by our simple sequential search baseline, indicating substantial headroom in current auto-research systems. Beyond evaluation, Arena reveals a substantially broader frontier of model and system ideas aligned with active research directions. Reusing these discoveries in a new quality-driven search yields lower training loss under matched data and FLOPs and lower held-out BPB at the same model size. Together, Arena provides a common framework for benchmarking autonomous research and expanding its scope in language modeling. The code is available at https://github.com/Auto-Research-Arena/auto-research-arena
ここに表示しているのは原論文の要旨です。AIによる要約や解釈は含みません。
