Memory-Consistent Neural Networks for Imitation Learning

• DOI: 10.48550/arxiv.2310.06171
• Publication Date: 2023-01-01
• Authors: Kaustubh Sridhar; Souradeep Dutta; Dinesh Jayaraman; James Weimer; Insup Lee
• Keywords: FOS: Computer and information sciences; Computer Science - Machine Learning; Computer Science - Robotics; Artificial Intelligence (cs.AI); Computer Science - Artificial Intelligence; Robotics (cs.RO); Machine Learning (cs.LG)

Imitation learning considerably simplifies policy synthesis compared to alternative approaches by exploiting access expert demonstrations. For such imitation policies, errors away from the training samples are particularly critical. Even rare slip-ups in action outputs can compound quickly over time, since they lead unfamiliar future states where is still more likely err, eventually causing task failures. We revisit simple supervised ``behavior cloning'' for conveniently nothing than pre-recorded demonstrations, but carefully design model class counter compounding error phenomenon. Our ``memory-consistent neural network'' (MCNN) hard-constrained stay within clearly specified permissible regions anchored prototypical ``memory'' samples. provide a guaranteed upper bound sub-optimality gap induced MCNN policies. Using MCNNs on 10 tasks, with MLP, Transformer, and Diffusion backbones, spanning dexterous robotic manipulation driving, proprioceptive inputs visual inputs, varying sizes types of demonstration data, we find large consistent gains performance, validating that better-suited vanilla deep networks applications. Website: https://sites.google.com/view/mcnn-imitation