論文 / arXiv:2610.40649
Rethinking Neural Thickets Without Weight Perturbations
PLAIN SUMMARY / やさしい要約
この研究は「少し設定を揺らすと賢い答えが出るのは、モデルの中に“専門家”が近くにいるからだ」という説明を見直します。モデル本体はいじらず、出力に小さなランダムなズレを足して複数の答えを作り、少しの確認問題で良いものを選んで、最後にまとめる方法を提案します。たとえるなら、同じ人に言い方を少し変えて何度か答えてもらい、ミニテストで良さそうな答えを残して多数決するイメージです。言語だけでなく画像でも、環境が変わる状況で役立つ可能性があると述べています。
AIが専門用語を使わずに書いた解説です。内容の正確さは、下の原文の要旨で確認してください。
ABSTRACT / 要旨(原文)
The Neural Thickets hypothesis posits that pretrained language models lie in dense neighborhoods of task-specific experts in weight space, which random Gaussian perturbations can uncover. We reproduce the reported gains, but show that they do not require this geometric explanation. Perturbing a single decoder block already recovers much of the improvement, while keeping the model entirely frozen and adding a random bias to its output logits matches or exceeds full-weight perturbations on most reasoning tasks. This suggests an alternative explanation based on output diversity, validation-based selection, and voting. We formalize this view as Perturb-Select-Vote (PSV), a general procedure that generates perturbed predictors, selects high-performing members on a small validation set, and aggregates their predictions. This interpretation connects Neural Thickets to established inference-time methods, including sampling, self-consistency, and verifier-guided selection. A first-order analysis further shows that small weight perturbations act primarily through their induced logit shifts. We extend this analysis to vision, where logit PSV substantially improves performance under domain shift, including on Earth observation, while full-weight perturbations provide little additional benefit. Together, these results suggest that the gains attributed to nearby weight-space experts can instead be obtained through output diversity, selection, and voting. Logit PSV realizes these benefits with a fully frozen model, providing a simple, training-free alternative to full-weight perturbation ensembles across language and vision.
ここに表示しているのは原論文の要旨です。AIによる要約や解釈は含みません。
