論文 / arXiv:2610.10538
Never Look Back: Understanding Persistence in 3D Object Memory from Egocentric Videos
PLAIN SUMMARY / やさしい要約
人が一日を過ごす様子を撮った動画から、「あとで必要になるかもしれない物の情報」を覚えておく仕組みを扱った研究です。提案手法は、物の場所だけでなく、見た順番や短い説明も一緒に記録します。いったん視界から消えた物や、触らなかった物も記録に残します。たとえるなら、動画を見ながら“物の置き場所メモ”を作っておき、あとで動画を開き直さずに探し物の質問に答えるイメージです。
AIが専門用語を使わずに書いた解説です。内容の正確さは、下の原文の要旨で確認してください。
ABSTRACT / 要旨(原文)
As we move through the world and carry out everyday tasks, we encounter objects that may become relevant only later. We are capable of recalling where we left something or what was inside a container, even without knowing we would need it later. Here, we study how an embodied assistant can build a similar memory from egocentric videos, by observing a person's day-to-day activities. We present Ledger, a persistent 3D object memory that combines object locations, their histories, and contextual descriptions. It associates observations across the recording and retains objects after they leave the view, including those the person never touches. It clusters each object's observations by resting locations and records a move only after repeated evidence, reducing the effect of localization noise. Short descriptions preserve details such as an object's contents or supporting surface. It saves these records to later answer spatial questions without having to access the original images or video. Our memory raises HD-EPIC accuracy from 29.7% to 42.6%, UCS-Bench accuracy from 33.8% to 38.5% and localizes Ego4D objects with a 0.99 m median error on returned predictions. Our analyses identify complementary roles for temporal persistence, contextual descriptions, and retrieval. Our study on 100 stitched streams of multiple scenes each further exposes failures in both retrieval and construction. Per-scene construction partially recovers the performance lost across scene changes compared to that of single scene streams.
ここに表示しているのは原論文の要旨です。AIによる要約や解釈は含みません。
