Bootstrapped Meta-Learning. (arXiv:2109.04504v1 [cs.LG])

Meta-learning empowers artificial intelligence to increase its efficiency by
learning how to learn. Unlocking this potential involves overcoming a
challenging meta-optimisation problem that often exhibits ill-conditioning, and
myopic meta-objectives. We propose an algorithm that tackles these issues by
letting the meta-learner teach itself. The algorithm first bootstraps a target
from the meta-learner, then optimises the meta-learner by minimising the
distance to that target under a chosen (pseudo-)metric. Focusing on
meta-learning with gradients, we establish conditions that guarantee
performance improvements and show that the improvement is related to the target
distance. Thus, by controlling curvature, the distance measure can be used to
ease meta-optimization, for instance by reducing ill-conditioning. Further, the
bootstrapping mechanism can extend the effective meta-learning horizon without
requiring backpropagation through all updates. The algorithm is versatile and
easy to implement. We achieve a new state-of-the art for model-free agents on
the Atari ALE benchmark, improve upon MAML in few-shot learning, and
demonstrate how our approach opens up new possibilities by meta-learning
efficient exploration in a Q-learning agent.



Related post