Optimal Information Storage and the Distribution of Synaptic Weights

• DOI: 10.1016/j.neuron.2004.08.023
• Publication Date: 2005-02-17T03:36:05Z
• Authors: Nicolas Brunel; Vincent Hakim; Philippe Isope; Jean-Pierre Nadal; Boris Barbour
• Keywords: MESH: Rats; [SDV.NEU.NB]Life Sciences [q-bio]/Neurons and Cognition [q-bio.NC]/Neurobiology; Neuroscience(all); Models, Neurological; -; Synaptic Transmission; MESH: Synapses; MESH: Afferent Pathways; MESH: Neural Networks (Computer); Purkinje Cells; 03 medical and health sciences; MESH: Purkinje Cells; MESH: Models, Neurological; MESH: Synaptic Transmission; Animals; Learning; MESH: Animals; [PHYS.COND.CM-DS-NN]Physics [physics]/Condensed Matter [cond-mat]/Disordered Systems and Neural Networks [cond-mat.dis-nn]; MESH: Excitatory Postsynaptic Potentials; Afferent Pathways; 0303 health sciences; [SDV.NEU.NB] Life Sciences [q-bio]/Neurons and Cognition [q-bio.NC]/Neurobiology; Excitatory Postsynaptic Potentials; Reproducibility of Results; MESH: Neural Inhibition; Neural Inhibition; [PHYS.COND.CM-DS-NN] Physics [physics]/Condensed Matter [cond-mat]/Disordered Systems and Neural Networks [cond-mat.dis-nn]; Rats; MESH: Reproducibility of Results; MESH: Nerve Net; Synapses; MESH: Learning; Neural Networks, Computer; Nerve Net

It is widely believed that synaptic modifications underlie learning and memory. However, few studies have examined what can be deduced about the learning process from the distribution of synaptic weights. We analyze the perceptron, a prototypical feedforward neural network, and obtain the optimal synaptic weight distribution for a perceptron with excitatory synapses. It contains more than 50% silent synapses, and this fraction increases with storage reliability: silent synapses are therefore a necessary byproduct of optimizing learning and reliability. Exploiting the classical analogy between the perceptron and the cerebellar Purkinje cell, we fitted the optimal weight distribution to that measured for granule cell-Purkinje cell synapses. The two distributions agreed well, suggesting that the Purkinje cell can learn up to 5 kilobytes of information, in the form of 40,000 input-output associations.