Biologically Inspired Sleep Algorithm for Reducing Catastrophic Forgetting in Neural Networks

• DOI: 10.1609/aaai.v34i10.7239
• Publication Date: 2020-06-29T17:45:51Z
• Authors: Timothy Tadros; Giri Krishnan; Ramyaa Ramyaa; Maxim Bazhenov
• Keywords: Sleep

Artificial neural networks (ANNs) are known to suffer from catastrophic forgetting: when learning multiple tasks, they perform well on the most recently learned task while failing previously tasks. In biological networks, sleep is play a role in memory consolidation and incremental learning. Motivated by processes that be involved generation we developed an algorithm implements sleep-like phase ANNs. framework, demonstrate able recover older tasks were otherwise forgotten. We show creates unique representations of each class inputs neurons relevant previous fire during sleep, simulating replay memories.