A bio-inspired recurrent neural network with self-adaptive neurons and PCM synapses for solving reinforcement learning tasks

One of the main challenges in artificial intelligence is the realization of systems capable of learning from their own experience and adapting themselves to a constantly changing environment. In nature, neurobiological systems modify the morphology of the synaptic connections in response to the past experience in order to optimize the interactions with the surrounding world. The introduction of experience-driven mechanisms in artificial systems would thus enable resilience and reinforcement learning in neural networks. Here, we present a novel brain-inspired recurrent neural network (RNN) with PCM synapses capable of advanced tasks such as maze navigation by reinforcement learning. We experimentally demonstrate that the multilevel synaptic capability provided by PCM devices mimics biology and allows a self-optimization of the navigation task. The autonomous agent can rely on PCM-based plasticity and neuronal spike-frequency adaptation to explore the environment and become its own expert teacher via a penalty/reward scheme. From these results, PCM-based local edge computing appears a key concept to enable learning and autonomous navigation in agents such as robots and cars.