SHIRITAS

論文 / arXiv:2611.06858

CompressionBench: How can your coding agents compress LLMs?

PLAIN SUMMARY / やさしい要約

この研究は、AIが自分で大きな言語モデルを「小さくして使える形にする」作業をどこまで進められるかを確かめます。材料(元のモデル)と制約(保存容量・時間・動かす環境)を渡し、AIに圧縮後のデータと実行できるプログラムまで作らせてチェックします。たとえるなら、引っ越しで荷物を減らしつつ、必要な道具がちゃんと使える状態かまで確認するようなものです。結果として、うまく通る提出が一定数あり、どこで判断が難しくなるかも観察されています。

AIが専門用語を使わずに書いた解説です。内容の正確さは、下の原文の要旨で確認してください。

ABSTRACT / 要旨(原文)

Recursive self-improvement requires AI systems to improve the models and infrastructure used by subsequent systems. LLM compression offers a concrete efficiency objective: retain a pretrained model's behavior within a smaller model storage budget. We study whether coding agents can carry out this step autonomously. We formulate compression as a terminal task in which an agent receives a source checkpoint, storage limit, inference environment, and time budget, then constructs compressed weights and executable inference code. The agent controls its algorithm and experiments; perplexity feedback guides search, and independent verification checks the final model. Across six agents and six model--size settings, 29 of 36 submissions pass verification. Selecting the lowest-perplexity agent design per task yields an average 3.4% perplexity increase over BF16 and 77.33% downstream accuracy, compared with 79.56% for BF16. Recorded experiments reveal how agents achieve these outcomes and where improvement breaks down. Agents lower PPL in 21 of 24 revisions at nearly fixed storage; complete-model feedback reverses 17 of 79 prior local rankings; and generation and inference tests expose failures that likelihood alone misses. The results identify representation design, experimental judgment, and capability validation as central to agent-driven model compression, a measurable component of efficiency-oriented self-improvement.

ここに表示しているのは原論文の要旨です。AIによる要約や解釈は含みません。

この論文を取り上げた記事