SHIRITAS

論文 / arXiv:2611.08438

Understanding AI from the Inside: The State of Mechanistic Interpretability for AI Safety

PLAIN SUMMARY / やさしい要約

AIが何をするかだけでなく、なぜそうするのかを「中身」から理解しようとする研究分野があります。この記事の元になる論文は、その分野がAIの安全性にどう関係するかをまとめ、「仕組みを理解すると本当に安全になるのか」を正面から問い直します。さらに、研究の主張をどう評価するかの共通の見取り図も提案しています。たとえば、車の運転が上手いか(振る舞い)だけでなく、エンジンの中がどう動いているか(内部)も見る、という対比で整理します。

AIが専門用語を使わずに書いた解説です。内容の正確さは、下の原文の要旨で確認してください。

ABSTRACT / 要旨(原文)

Neural networks are likely to be among the most consequential artefacts in human history. As these systems become more capable and autonomous, their potential to cause societal harm also grows, making AI safety one of the defining challenges of our time. Yet we continue to build increasingly powerful systems without fully understanding how they work, how they will behave in unfamiliar situations, or how to reliably control them. Mechanistic interpretability offers a path towards understanding these systems by studying their internal representations and computations, with the aim of evaluating, predicting, and controlling behaviour. We survey mechanistic interpretability for AI safety in order to answer the question: Does understanding how models work actually make them safer? We place mechanistic interpretability in the broader context of interpretability research and compare its mechanistic evidence with behavioural evaluation. We introduce the foundations of mechanistic interpretability, including features, polysemanticity, transformer computation, and circuits, and survey methods for feature extraction and analysis, circuit discovery and causal verification, explaining intermediate representations in natural language, learning dynamics, and training-data attribution. Beyond surveying existing methods, we propose a unified framework for evaluating mechanistic interpretability claims and characterise six research orientations by their goals, assumptions, and trade-offs. Finally, we examine how mechanistic understanding can support AI safety, including knowledge editing, steering, and monitoring, and assess its limitations, failure modes, open problems, and relationship to behavioural safety measures. We also release a repository of code notebooks with worked examples that reproduce key findings from surveyed methods, allowing readers to build intuition through hands-on experimentation.

ここに表示しているのは原論文の要旨です。AIによる要約や解釈は含みません。

この論文を取り上げた記事