論文 / arXiv:2611.12872
HiVe-Mem: Interleaving Horizontal Memory Expansion and Vertical Evidence Composition for Multimodal Agents
PLAIN SUMMARY / やさしい要約
会話や画像のやりとりが長くなると、昔の大事な情報を見つけて使うのが難しくなります。この研究は、過去の記録を「関係でつながったメモ帳」として整理し、必要なところを広げて探す方法を提案します。さらに、要約や画像など複数の形の手がかりから、質問に合わせて使う分だけ選ぶ仕組みも入れます。結果として、いくつかのテストで成績が良くなり、読む量(トークン)も減ったと述べています。たとえば、索引で見つけた1ページだけでなく、前後の関連ページも一緒に開いて確認するようなイメージです。
AIが専門用語を使わずに書いた解説です。内容の正確さは、下の原文の要旨で確認してください。
ABSTRACT / 要旨(原文)
As multimodal agents accumulate increasingly long interaction histories, effectively retrieving and utilizing past information becomes critical for long-term interaction. Existing memory systems typically rely on semantic retrieval over compressed memories and expose predefined representations, which may miss relevant memories and overlook heterogeneous evidence needs across queries. We introduce \textbf{\hivemem}, a multimodal memory framework that interleaves \textbf{\textsc{H}}or\textbf{\textsc{i}}zontal Memory Expansion with \textbf{\textsc{Ve}}rtical Evidence Composition. For horizontal memory expansion, \hivemem organizes memory episodes into a relational graph using textual and visual anchors, expanding semantically retrieved seeds with related neighbors to recover complementary information. For vertical evidence composition, each retrieved memory retains multiple evidence forms, including compact summaries, captions, grounded visual views, raw images, and interaction history. A lightweight trainable selector independently composes an evidence subset for each retrieved memory, enabling query-adaptive utilization of heterogeneous multimodal information. This two-dimensional design jointly broadens memory coverage and adapts the evidence contributed by individual memories. Experiments across three multimodal memory benchmarks show that \hivemem consistently improves task performance across base models while reducing token consumption by about 90\% compared with the most efficient competing multimodal baselines.
ここに表示しているのは原論文の要旨です。AIによる要約や解釈は含みません。
