Normalization as a canonical neural computation

There’s an excellent review on normalization in the January 2012 edition of Nature Reviews Neuroscience by Carandini and Heeger. The theory and mathematics of normalization have stayed consistent since the seminal papers Heeger (1992) and  Carandini and Heeger (1994). The response of a given neuron is divided by the summed output of a normalization pool, whose tuning determines exactly what form of normalization takes place. This covers phenomena such as contrast gain control, surround suppression and so-called self-normalization. The figure below illustrates the classic case of contrast normalization in the LGN.

Normalization seems to take place at every level of the sensory hierarchy. In the visual system, in particular, new levels of normalization have been found pretty much in every area that has been looked at, whether it’s the retina, V1, MT, MST, V4 or IT. Recent research has implicated normalization in the integration of signals across sensory modalities, as in the case of vestibular and visual inputs in MST, as well as in attentional modulation.

The most interesting aspect of the review for me is the discussion of the mechanisms underlying normalization. Originally, it was thought that normalization occurred through shunting inhibition. In this scheme, the normalization pool acts on a neuron by modulating the input resistance of a neuron. By opening channels with a reversal potential near the resting potential of a neuron, in particular GABA type A receptors, the conductance of the neuron is manipulated without co-occuring excitatory or inhibitory input. This theory explains, among other things, the change in the apparent gain and temporal profile of responses as a function of contrast in LGN neurons.

This theory has been controversial, however. Chris Koch showed that under some circumstances the divisive effect of normalization actually translates into a subtractive effect after taking into account the spiking nonlinearity of the neuron. However, it seems that if the input is noisy the divisive effect is recovered. In V1, conductance does increase with contrast, but it is not invariant with orientation. Blocking GABA-A in V1 does not seem to remove contrast suppression nor cross-orientation suppression.That being said, it does seem that at least in some systems, GABA-A is the culprit, in particular in the olfactory system of fruitflies.

I was surprised that there is no mention of such research in the sensory systems of rodents. I did a quick search on normalization in mouse vision and could not find a single relevant paper. With all the genetic, imaging and pharmalogical tools available for these systems, it seems like a pretty low-hanging fruit waiting to be picked. Insect models are neat but their relevance to mammals can be limited; on the other hand, the tools available in higher mammals are fairly crude. This could be a really nice research project for someone starting a post-doc soon (hint, hint).

There’s other theories of normalization as well. One idea is that the relevant mechanism is indeed a conductance increase, but this is brought about by a simultaneous increase in excitation and inhibition. Another idea is that synaptic depression can account for some, but not all aspects of normalization. Membrane fluctuations brought about by spontaneous activity interact with the spiking nonlinearity; a decrease in spontaneous activity could decrease the rate at which fluctuations cross the threshold, resulting in a gain decrease.

There’s a ton of interesting references in the review, I printed up a good deal of them and I am including them as supplementary references below.

Carandini, M., & Heeger, D. (2012). Normalization as a canonical neural computation Nature Reviews Neuroscience DOI: 10.1038/nrn3136

Heeger, D. (2009). Normalization of cell responses in cat striate cortex Visual Neuroscience, 9 (02) DOI: 10.1017/S0952523800009640

Carandini, M., & Heeger, D. (1994). Summation and division by neurons in primate visual cortex Science, 264 (5163), 1333-1336 DOI: 10.1126/science.8191289

Supplementary references:

Olsen, S., Bhandawat, V., & Wilson, R. (2010). Divisive Normalization in Olfactory Population Codes Neuron, 66 (2), 287-299 DOI: 10.1016/j.neuron.2010.04.009

Busse, L., Wade, A., & Carandini, M. (2009). Representation of Concurrent Stimuli by Population Activity in Visual Cortex Neuron, 64 (6), 931-942 DOI: 10.1016/j.neuron.2009.11.004

Alitto, H., & Dan, Y. (2010). Function of inhibition in visual cortical processing Current Opinion in Neurobiology, 20 (3), 340-346 DOI: 10.1016/j.conb.2010.02.012

Katzner, S., Busse, L., & Carandini, M. (2011). GABAA Inhibition Controls Response Gain in Visual Cortex Journal of Neuroscience, 31 (16), 5931-5941 DOI: 10.1523/JNEUROSCI.5753-10.2011

Olsen, S., & Wilson, R. (2008). Lateral presynaptic inhibition mediates gain control in an olfactory circuit Nature, 452 (7190), 956-960 DOI: 10.1038/nature06864

Silver, R. (2010). Neuronal arithmetic Nature Reviews Neuroscience, 11 (7), 474-489 DOI: 10.1038/nrn2864

Hao, J., Wang, X., Dan, Y., Poo, M., & Zhang, X. (2009). An arithmetic rule for spatial summation of excitatory and inhibitory inputs in pyramidal neurons Proceedings of the National Academy of Sciences, 106 (51), 21906-21911 DOI: 10.1073/pnas.0912022106

Rothman, J., Cathala, L., Steuber, V., & Silver, R. (2009). Synaptic depression enables neuronal gain control Nature, 457 (7232), 1015-1018 DOI: 10.1038/nature07604

Chance, F., Abbott, L., & Reyes, A. (2002). Gain Modulation from Background Synaptic Input Neuron, 35 (4), 773-782 DOI: 10.1016/S0896-6273(02)00820-6

Bock, D., Lee, W., Kerlin, A., Andermann, M., Hood, G., Wetzel, A., Yurgenson, S., Soucy, E., Kim, H., & Reid, R. (2011). Network anatomy and in vivo physiology of visual cortical neurons Nature, 471 (7337), 177-182 DOI: 10.1038/nature09802

Haider, B., Krause, M., Duque, A., Yu, Y., Touryan, J., Mazer, J., & McCormick, D. (2010). Synaptic and Network Mechanisms of Sparse and Reliable Visual Cortical Activity during Nonclassical Receptive Field Stimulation Neuron, 65 (1), 107-121 DOI: 10.1016/j.neuron.2009.12.005

Murphy, B., & Miller, K. (2009). Balanced Amplification: A New Mechanism of Selective Amplification of Neural Activity Patterns Neuron, 61 (4), 635-648 DOI: 10.1016/j.neuron.2009.02.005

About these ads

One thought on “Normalization as a canonical neural computation

Leave a comment

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s