Measuring and Controlling Solution Degeneracy Across Task-Trained Recurrent Neural Networks

A Kempner team introduces a new framework for measuring and controlling solution degeneracy in task-trained recurrent neural networks. By studying more than 3,400 RNNs on four neuroscience-related tasks, they show that factors such as task difficulty, learned features, network size, and regularization influence the different internal strategies networks develop, even when they achieve the same level of performance.