SHIRITAS

論文 / arXiv:2610.30132

LongZero: Enhancing Long-Context Reasoning in LLMs from Zero Data

PLAIN SUMMARY / やさしい要約

長い文章を読んで考える力をAIに付けるには、ふつうは練習問題(データ)がたくさん必要です。この研究は、外から問題を集めなくても、AIが自分で「答えと根拠つきの問題」を作り、それを長い文章の形に引き延ばして練習する方法を提案します。たとえるなら、先に解答と解説の要点を自分で作ってから、あとで長い教材に仕立て直して勉強するようなものです。2つのAIモデルと5つのテストで比べたところ、外部データを使って訓練する方法より良い結果だったと述べています。

AIが専門用語を使わずに書いた解説です。内容の正確さは、下の原文の要旨で確認してください。

ABSTRACT / 要旨(原文)

Reinforcement learning (RL) is an effective way to strengthen the long-context reasoning of large language models (LLMs), but it needs training data that are long, diverse, challenging, and verifiable. Human annotation of such data is expensive, naturally occurring data are too scarce to sustain RL, and existing synthesis methods depend on external corpora or stronger teacher models, which become harder to obtain as the trained models get stronger. Recently, RL from zero data, in which a model learns from tasks it proposes itself, has achieved excellent performance, but existing methods target short-context tasks. Therefore, we propose $\textbf{LongZero}$, a method that lets an LLM improve its own long-context reasoning without any external data. Instead of writing questions and answers over existing documents, LongZero first has the model synthesize the question and answer with its supporting evidence in a compact structured form, and only then separately expands the evidence into a long multi-document context. LongZero combines these self-written multi-hop reasoning tasks with self-written multi-round retrieval tasks and trains the model with GRPO. We evaluate LongZero on two LLMs across five long-context reasoning benchmarks and compare it with RL on data synthesized from external corpora. Experimental results show that LongZero can generalize to real tasks, and outperforms both baselines, suggesting a viable and promising path to improving long-context reasoning of LLMs.

ここに表示しているのは原論文の要旨です。AIによる要約や解釈は含みません。

この論文を取り上げた記事