A central goal in systems neuroscience is to understand the functions performed by neural circuits. Previous top-down models addressed this question by comparing the behaviour of an ideal model circuit, optimised to perform a given function, with neural recordings. However, this requires guessing in advance what function is being performed, which may not be possible for many neural systems. To address this, we propose an inverse reinforcement learning (RL) framework for inferring the function performed by a neural network from data. We assume that the responses of each neuron in a network are optimised so as to drive the network towards ‘rewarded’ states, that are desirable for performing a given function. We then show how one can use inverse RL to infer the reward function optimised by the network from observing its responses. This inferred reward function can be used to predict how the neural network should adapt its dynamics to perform the same function when the external environment or network structure changes. This could lead to theoretical predictions about how neural network dynamics adapt to deal with cell death and/or varying sensory stimulus statistics.
The authors would like to thank Ulisse Ferrari for useful discussions and feedback.
Chalk MJ, Tkačik G, Marre O. Inferring the function performed by a recurrent neural network. PLoS ONE. 2021;16(4 April). doi:10.1371/journal.pone.0248940
Chalk, M. J., Tkačik, G., & Marre, O. (2021). Inferring the function performed by a recurrent neural network. PLoS ONE. https://doi.org/10.1371/journal.pone.0248940
Chalk, Matthew J, Gašper Tkačik, and Olivier Marre. “Inferring the Function Performed by a Recurrent Neural Network.” PLoS ONE, 2021. https://doi.org/10.1371/journal.pone.0248940.
M. J. Chalk, G. Tkačik, and O. Marre, “Inferring the function performed by a recurrent neural network,” PLoS ONE, vol. 16, no. 4 April. 2021.
Chalk MJ, Tkačik G, Marre O. 2021. Inferring the function performed by a recurrent neural network. PLoS ONE. 16(4 April), e0248940.
Chalk, Matthew J., et al. “Inferring the Function Performed by a Recurrent Neural Network.” PLoS ONE, vol. 16, no. 4 April, e0248940, 2021, doi:10.1371/journal.pone.0248940.
All files available under the following license(s):
Creative Commons Attribution 4.0 International Public License (CC-BY 4.0):
2021_pone_Chalk.pdf 2.77 MB