論文 / arXiv:2610.12410
Predicting Alignment Generalization with Value Representations
PLAIN SUMMARY / やさしい要約
大規模言語モデルに「ある望ましい振る舞い」を教えると、教えていない別の振る舞いにも影響が出ることがあります。この研究は、その影響がどこにどう広がるかを前もって予測する課題を提案します。そして、予測に使う情報として「モデル内部の反応」と「文章で書いた説明」のどちらが役に立つかを比べました。たとえば、料理で一つの味付けを変えたとき、別の料理の味にも影響が出るかを、材料表だけで当てるのがよいのか、実際に試食した反応で当てるのがよいのかを比べるような話です。
AIが専門用語を使わずに書いた解説です。内容の正確さは、下の原文の要旨で確認してください。
ABSTRACT / 要旨(原文)
LLM developers post-train their models to exhibit prosocial values and behavioral traits, which are enumerated in an alignment target. However, while recent post-training developments have yielded models that score highly on alignment evaluations, training models on sets of narrow behaviors still influences their behavior across unseen contexts and environments in unexpected ways. In this paper, we establish the task of alignment generalization prediction, i.e., predicting how fine-tuning a model to follow a given value changes its behavior across a wide range of held-out values. We conduct a large-scale analysis of alignment generalization effects across 66 values found in modern alignment targets, and benchmark representational techniques on the alignment generalization prediction task. We find that representations based on model activations when applying values in context significantly outperform methods based on textual descriptions of the values. Specifically, the best activations-based methods achieve correlations of 0.45 with our generalization matrix, compared with 0.05 from description-based baselines. We then show the applicability of representations that predict alignment generalization toward downstream tasks by using them to measure how similar the values in a multi-value alignment target are, which we find is significantly correlated with model robustness. Finally, we show initial evidence towards a shared, model-independent value space, which we use to develop the first taxonomy of LLM values grounded in empirical generalization dynamics. Our work demonstrates the importance of studying value generalization in LLMs and its application toward the more empirical design and training of model behavior.
ここに表示しているのは原論文の要旨です。AIによる要約や解釈は含みません。
