Sparsely Changing Latent States for Prediction and Planning in Partially Observable Domains

• DOI: 10.48550/arxiv.2110.15949
• Publication Date: 2021-01-01
• Authors: Christian Gumbsch; Martin V. Butz; Georg Martius
• Keywords: Spurious relationship; Generative model; ENCODE

A common approach to prediction and planning in partially observable domains is use recurrent neural networks (RNNs), which ideally develop maintain a latent memory about hidden, task-relevant factors. We hypothesize that many of these hidden factors the physical world are constant over time, changing only sparsely. To study this hypothesis, we propose Gated $L_0$ Regularized Dynamics (GateL0RD), novel architecture incorporates inductive bias stable, sparsely states. The implemented by means internal gating function penalty on norm state changes. demonstrate GateL0RD can compete with or outperform state-of-the-art RNNs variety control tasks. tends encode underlying generative environment, ignores spurious temporal dependencies, generalizes better, improving sampling efficiency overall performance model-based reinforcement learning Moreover, show developing states be easily interpreted, step towards better explainability RNNs.