There’s an excellent review on normalization in the January 2012 edition of Nature Reviews Neuroscience by Carandini and Heeger. The theory and mathematics of normalization have stayed consistent since the seminal papers Heeger (1992) and Carandini and Heeger (1994). The response of a given neuron is divided by the summed output of a normalization pool, whose tuning determines exactly what form of normalization takes place. This covers phenomena such as contrast gain control, surround suppression and so-called self-normalization. The figure below illustrates the classic case of contrast normalization in the LGN.
Normalization seems to take place at every level of the sensory hierarchy. In the visual system, in particular, new levels of normalization have been found pretty much in every area that has been looked at, whether it’s the retina, V1, MT, MST, V4 or IT. Recent research has implicated normalization in the integration of signals across sensory modalities, as in the case of vestibular and visual inputs in MST, as well as in attentional modulation.
The most interesting aspect of the review for me is the discussion of the mechanisms underlying normalization. Originally, it was thought that normalization occurred through shunting inhibition. In this scheme, the normalization pool acts on a neuron by modulating the input resistance of a neuron. By opening channels with a reversal potential near the resting potential of a neuron, in particular GABA type A receptors, the conductance of the neuron is manipulated without co-occuring excitatory or inhibitory input. This theory explains, among other things, the change in the apparent gain and temporal profile of responses as a function of contrast in LGN neurons.
This theory has been controversial, however. Chris Koch showed that under some circumstances the divisive effect of normalization actually translates into a subtractive effect after taking into account the spiking nonlinearity of the neuron. However, it seems that if the input is noisy the divisive effect is recovered. In V1, conductance does increase with contrast, but it is not invariant with orientation. Blocking GABA-A in V1 does not seem to remove contrast suppression nor cross-orientation suppression.That being said, it does seem that at least in some systems, GABA-A is the culprit, in particular in the olfactory system of fruitflies.
I was surprised that there is no mention of such research in the sensory systems of rodents. I did a quick search on normalization in mouse vision and could not find a single relevant paper. With all the genetic, imaging and pharmalogical tools available for these systems, it seems like a pretty low-hanging fruit waiting to be picked. Insect models are neat but their relevance to mammals can be limited; on the other hand, the tools available in higher mammals are fairly crude. This could be a really nice research project for someone starting a post-doc soon (hint, hint).
There’s other theories of normalization as well. One idea is that the relevant mechanism is indeed a conductance increase, but this is brought about by a simultaneous increase in excitation and inhibition. Another idea is that synaptic depression can account for some, but not all aspects of normalization. Membrane fluctuations brought about by spontaneous activity interact with the spiking nonlinearity; a decrease in spontaneous activity could decrease the rate at which fluctuations cross the threshold, resulting in a gain decrease.
There’s a ton of interesting references in the review, I printed up a good deal of them and I am including them as supplementary references below.
Carandini, M., & Heeger, D. (2012). Normalization as a canonical neural computation Nature Reviews Neuroscience DOI: 10.1038/nrn3136
Heeger, D. (2009). Normalization of cell responses in cat striate cortex Visual Neuroscience, 9 (02) DOI: 10.1017/S0952523800009640
Carandini, M., & Heeger, D. (1994). Summation and division by neurons in primate visual cortex Science, 264 (5163), 1333-1336 DOI: 10.1126/science.8191289
Olsen, S., Bhandawat, V., & Wilson, R. (2010). Divisive Normalization in Olfactory Population Codes Neuron, 66 (2), 287-299 DOI: 10.1016/j.neuron.2010.04.009
Busse, L., Wade, A., & Carandini, M. (2009). Representation of Concurrent Stimuli by Population Activity in Visual Cortex Neuron, 64 (6), 931-942 DOI: 10.1016/j.neuron.2009.11.004
Alitto, H., & Dan, Y. (2010). Function of inhibition in visual cortical processing Current Opinion in Neurobiology, 20 (3), 340-346 DOI: 10.1016/j.conb.2010.02.012
Katzner, S., Busse, L., & Carandini, M. (2011). GABAA Inhibition Controls Response Gain in Visual Cortex Journal of Neuroscience, 31 (16), 5931-5941 DOI: 10.1523/JNEUROSCI.5753-10.2011
Olsen, S., & Wilson, R. (2008). Lateral presynaptic inhibition mediates gain control in an olfactory circuit Nature, 452 (7190), 956-960 DOI: 10.1038/nature06864
Silver, R. (2010). Neuronal arithmetic Nature Reviews Neuroscience, 11 (7), 474-489 DOI: 10.1038/nrn2864
Hao, J., Wang, X., Dan, Y., Poo, M., & Zhang, X. (2009). An arithmetic rule for spatial summation of excitatory and inhibitory inputs in pyramidal neurons Proceedings of the National Academy of Sciences, 106 (51), 21906-21911 DOI: 10.1073/pnas.0912022106
Rothman, J., Cathala, L., Steuber, V., & Silver, R. (2009). Synaptic depression enables neuronal gain control Nature, 457 (7232), 1015-1018 DOI: 10.1038/nature07604
Chance, F., Abbott, L., & Reyes, A. (2002). Gain Modulation from Background Synaptic Input Neuron, 35 (4), 773-782 DOI: 10.1016/S0896-6273(02)00820-6
Bock, D., Lee, W., Kerlin, A., Andermann, M., Hood, G., Wetzel, A., Yurgenson, S., Soucy, E., Kim, H., & Reid, R. (2011). Network anatomy and in vivo physiology of visual cortical neurons Nature, 471 (7337), 177-182 DOI: 10.1038/nature09802
Haider, B., Krause, M., Duque, A., Yu, Y., Touryan, J., Mazer, J., & McCormick, D. (2010). Synaptic and Network Mechanisms of Sparse and Reliable Visual Cortical Activity during Nonclassical Receptive Field Stimulation Neuron, 65 (1), 107-121 DOI: 10.1016/j.neuron.2009.12.005
Murphy, B., & Miller, K. (2009). Balanced Amplification: A New Mechanism of Selective Amplification of Neural Activity Patterns Neuron, 61 (4), 635-648 DOI: 10.1016/j.neuron.2009.02.005
One response to “Normalization as a canonical neural computation”
[…] bugging people for a long time. For example, how is divisive normalization implemented in cortex? I mentioned a couple of posts ago that this question could presumably be attacked by current methods (optogenetics and pharmacology) […]