SHIRITAS

論文 / arXiv:2610.37657

Heads of Security: Learned Attention Gates Improve Prompt Injection Robustness

PLAIN SUMMARY / やさしい要約

外部から渡される文章には、役に立つ情報と、モデルをだますための命令が混ざることがあります。HoSは、文章のどこに注意を向けるかを小さな「ゲート」で調整し、だましの命令には反応しにくくする方法だと説明されています。たとえば、同じ紙に「材料の説明」と「危ない指示」が書かれていても、読むべき部分を読み分ける“しおり”のような仕組みを想定するとイメージしやすいでしょう。論文では、いくつかの攻撃テストで成功率が下がったり、攻撃側の計算が増えたりしたと報告しています。

AIが専門用語を使わずに書いた解説です。内容の正確さは、下の原文の要旨で確認してください。

ABSTRACT / 要旨(原文)

We introduce Heads of Security (HoS), a prompt injection defense that enforces instruction-data separation via a small set of learned per-head attention gates. HoS learns how each head should redistribute attention between trusted and untrusted tokens, enabling models to reason over untrusted content without following injected instructions. We evaluate HoS on Llama-3.3-70B-Instruct and GPT-OSS-20B. We compare it against SecAlign++, a state-of-the-art finetuning-based defense, and ASIDE, a defense that, like ours, builds instruction-data separation into the model architecture. We show that HoS offers greater robustness to adaptive attacks, better performance on standard utility benchmarks, and matches top performance on standard agentic prompt injection benchmarks. More specifically, the black-box RL-Hammer attack reaches 97% ASR against GPT-OSS-20B and 85% against SecAlign, but only 10% against HoS. Claudini, a state-of-the-art white-box attack, strengthened against HoS through autoresearch, requires 3 times more FLOPs to reach the same ASR on HoS as it does on GPT-OSS-20B or SecAlign. HoS preserves base-model utility across standard multiple-choice benchmarks and prompting setups, whereas SecAlign and ASIDE degrade in few-shot settings. On agentic benchmarks with static attacks, HoS and SecAlign perform similarly and substantially improve the security-utility trade-off over the base model. ASIDE struggles with tool use, achieving only 10% utility. These results show that prompt injection robustness can be achieved and improved through targeted attention interventions.

ここに表示しているのは原論文の要旨です。AIによる要約や解釈は含みません。

この論文を取り上げた記事