Learning Visual Planning Models from Partially Observed Images. (arXiv:2211.15666v1 [cs.LG])

There has been increasing attention on planning model learning in classical
planning. Most existing approaches, however, focus on learning planning models
from structured data in symbolic representations. It is often difficult to
obtain such structured data in real-world scenarios. Although a number of
approaches have been developed for learning planning models from fully observed
unstructured data (e.g., images), in many scenarios raw observations are often
incomplete. In this paper, we provide a novel framework, aType{Recplan}, for
learning a transition model from partially observed raw image traces. More
specifically, by considering the preceding and subsequent images in a trace, we
learn the latent state representations of raw observations and then build a
transition model based on such representations. Additionally, we propose a
neural-network-based approach to learn a heuristic model that estimates the
distance toward a given goal observation. Based on the learned transition model
and heuristic model, we implement a classical planner for images. We exhibit
empirically that our approach is more effective than a state-of-the-art
approach of learning visual planning models in the environment with incomplete
observations.