SHIRITAS

論文 / arXiv:2611.26545

The Geometry of Lies: Knowledge and Persona Directions for Detecting Deception in Large Language Models

PLAIN SUMMARY / やさしい要約

大きな言語モデルが「嘘」をつくときの手がかりを、モデル内部の“向き”のような特徴として取り出して見分けようとした研究です。役を演じさせて得た情報から、正直・不正直の違いを表す方向(persona direction)を作り、別の方法(knowledge direction)とも比べます。結果は、場面によってはうまくいく一方で、同じ言い回しのまま「嘘」と「正直な否認」を区別するのは簡単ではないことも示します。たとえるなら、話し方の“口癖”を追うと見分けられる時もあるけれど、口癖が同じだと別の手がかりが要る、という話です。

AIが専門用語を使わずに書いた解説です。内容の正確さは、下の原文の要旨で確認してください。

ABSTRACT / 要旨(原文)

Linear probes are the cheapest LLM lie detectors but often fail to transfer across kinds of lie. Label-free directions from honest and deceptive role-play are untested at 70B scale and on lies about knowledge, where a lie and an honest denial can be identical. We widen a published persona bank with roles that deny knowledge they have, plus honest counterparts. At each model's middle layer the resulting persona direction ranks lies across Liars' Bench better than single-dataset probes on three of its four models; it fails on Qwen, even when Qwen stays in role far more often. On Llama-3.3-70B-Instruct it beats directions from a random persona bank, but its gain over the published direction is within the range of random widenings. On lies about knowledge, persona directions mainly track a reply's wording: where lies and honest denials share their words, our three main contrasts stay inside the random-direction band on Llama and Qwen and reach only its edge on Gemma, though on Mistral one clears it. A knowledge direction contrasts questions the model can answer with ones nobody can, each followed by the same scripted denial. It separates lies from identical honest denials on two models and narrowly on a third, but not on the fourth; its other results partly depend on the benchmark's wording. On the Gender-Secret fine-tune it clears the band by a few hundredths on Llama and fails on Mistral. The sum of the two directions ranks lies better than a supervised probe at the same depth on all four models (on one, both near chance) but is level with the persona direction on Llama's average and behind that probe on held-out lies. More varied role-play questions do not help the persona direction on Liars' Bench.

ここに表示しているのは原論文の要旨です。AIによる要約や解釈は含みません。

この論文を取り上げた記事