論文 / arXiv:2610.26809
Safety Mid-Training: Internalizing LLM Safety as a Foundational Capability
PLAIN SUMMARY / やさしい要約
大規模言語モデルは、あとから別の勉強をさせると「危ない答え方」を思い出してしまうことがあります。そこでこの研究は、安全な考え方を後付けではなく、最初からの基礎体力として身につけさせる“中間の学習段階”を提案します。安全に関する問いと、役に立つ作業の問いを両方使い、考え方の筋道も一緒に練習させます。料理で言えば、盛り付け(事後調整)だけでなく、包丁の使い方や火加減(基礎)を途中でしっかり教える、という発想に近いです。
AIが専門用語を使わずに書いた解説です。内容の正確さは、下の原文の要旨で確認してください。
ABSTRACT / 要旨(原文)
Safety alignment for large language models (LLMs) is primarily performed during post-training, while recent work has also begun incorporating safety data into pre-training. Yet safety can degrade when later training no longer includes safety data, and gains in safety often come at the cost of general capability. This tension suggests that safety and general reasoning should not be treated as separate training goals. We instead view safety as an LLM capability that can be learned through a reasoning structure shared with general tasks. Both require the model to understand intent or task goals, apply relevant knowledge, and reason about potential consequences. We introduce safety mid-training, a stage between pre-training and post-training that jointly develops safety and general reasoning through this shared structure. Given safety and utility prompts, the base model generates structured documents and reasoning traces that are then used for mid-training. Safety mid-training reduces average attack success rate (ASR) by 72% to 81% relative to Qwen3-1.7B, 4B, and 8B base models, while also improving general capability. The benefits persist through subsequent post-training. After general-only SFT, mid-trained models largely retain their safety, achieving lower ASR than the baselines without sacrificing general capability. They also retain safety under general-only GRPO without safety rewards, while subsequent RL reaches a better safety-utility frontier when initialized from the mid-trained models. Together, these results suggest that learning safety and general reasoning within a shared mid-training stage can reduce the safety-utility trade-off and make safety more robust to subsequent training.
ここに表示しているのは原論文の要旨です。AIによる要約や解釈は含みません。
