Object Permanence in AI: Crucial for Autonomous Agents and Robotics

Just like infants learn to grasp object permanence over time, AI models continue to evolve

AI does not have a world model. That is a problem because agents could lose track of the state of an object in the world once it becomes invisible. Let’s say a bicycle on the road. This problem of losing the state of an object can commonly be observed in realizations of autonomous agents in robotics, mobility, or cognitive applications. One solution to this problem is object permanence. Object permanence could help us train better agents to operate autonomously in uncontrolled environments like factories and in-space assembly.

image source : https://www.simplypsychology.org/object-permanence.html

Object permanence originates from studies in developmental psychology where it commonly refers to the understanding that objects continue to exist even when they are no longer visible. For infants, it serves as a precursor to language acquisition and the development of searching behaviors.

In the context of autonomous agents, object permanence poses unique challenges. that should be considered when building applications in that space.

Invisible / Visible Displacements

As per this writing (early 2024), most existing research and implementation focus on tracking visible displacements where frames are associated with bounded regions across subsequent frames, comparing color, shape, or location. This tracking is relatively straightforward—objects move within the field of view, and their trajectories are observable.