Alchemy is a new meta-learning environment rich enough to contain interesting
abstractions, yet simple enough to make fine-grained analysis tractable.
Further, Alchemy provides an optional symbolic interface that enables meta-RL
research without a large compute budget. In this work, we take the first steps
toward using Symbolic Alchemy to identify design choices that enable deep-RL
agents to learn various types of abstraction. Then, using a variety of
behavioral and introspective analyses we investigate how our trained agents use
and represent abstract task variables, and find intriguing connections to the
neuroscience of abstraction. We conclude by discussing the next steps for using
meta-RL and Alchemy to better understand the representation of abstract
variables in the brain.