Early on, we decided to set up a cluster of computers that could be used by anybody working on the Utah array data. Each computer in this cluster would be indistinguishable from the next; it would be accessed almost exclusively remotely; it would have local access to all the array data; and would have a decent CPU and a bucket of RAM. Getting all the pieces working well together was a challenge, and I hope that you can learn a thing or two from my experiences.

Hardware and OS setup

Here’s a picture of the array:

At the bottom, you can see the 4 identical computers. Each has an i7 920 Intel CPU, 12 GB of RAM (6×2 GB, upgradeable to 24 GB), a very small internal hard drive, an on-board video card, and that’s pretty much it. Drone 1 (on the left) is connected to an external housing that can contain up to 4 hard drives (the small case on top of the computer with the blue lights); currently it’s housing 3x 2TB drives for a total of 6 TB. This external stack is connected to drone 1 through eSATA, which is a type of connection that allows essentially the same access speed as if the hard drives were connected internally (up to 3GB/s bandwidth I think).

To give the other computers access to the data in the housing, it was essential to get them all on the same fast network. The MNI’s internal network is capped at 100 Mbits/s (~12MBytes/s), so I decided to connect them through their own fast internal network. This was done through the Gigabit (1 Gbit/s, or ~120 MBytes/s) router sitting on top of the third computer. Due to some internal policies from the computer service at McGill, it was also necessary to connect each of these computers individually to the McGill network. Hence an extra Gigabit Ethernet card was installed in each computer.

I installed Ubuntu on drone 1, as well as the relevant scientific software: Matlab, R, RKWard, etc. I also installed NX, which is a wonderful remote desktop server that allows ones to access these Linux computers externally from your platform of choice, whether Windows, Mac, or Linux. It compresses X11 traffic and is remarkably efficient; one can quite comfortably work from home on these computers despite small bandwidth. I used no-ip.com to provide a simple internet address for the drones as opposed to a hard-to-remember IP.

Once everything was installed on drone 1, I cloned the drive three times to derive the 3 other computers, saving much time.

To give access to the data from the other computers, I used sshfs, which allows one to automatically mount a remote hard drive through ssh in a 100% automagic manner. Using symlinks, I  arranged it so that every computer could access the data from the same (~/ArrayData) location. Thus, from the perspective of a remote user, all drones have the same software and access the same data. A similar system was used for documents (ie. m files).

Automated fetching and backup pipeline

Next, I sought to automate the process of getting the data from the recording computers. Two computers are used to record the data: a Mac which does the presentation through homebrew Psychtoolbox-based software, and a PC which records .PLX files from the Plexon MAP box. We agreed to use a consistent naming scheme for experiments: a letter (representing which physical hard drive the data will eventually be put in), three digits (for the recording day), and another letter (or letters) for each experiment on this recording day.

SFTP servers were installed on both the Mac and PC to allow external access to the data from the drones. I then wrote a Python script which fetches the data from a particular experiment (say, c012b) from both the PC and Mac and reads metadata in .mat files from the Mac and the .plx file from the PC and spits out usable files. I used the PLX reader modified from OpenElectrophy that I discussed earlier, as well as the MatIO library from reading .mat files in Python that is part of SciPy.

Using this script is pretty simple; one simply needs to type in the command sudo python fetchAndConvert.py -e c012a to fetch and convert experiment c012a. It’s also possible to fetch and convert an entire day with the same software.

The software first creates a fixed directory structure; for experiment c012a it first creates the root folder ~/c/c012/c012a. Under this folder it creates several sub folders, include /plx, which contains the original plx file, /mat, which contains plx data after it’s been processed in a format that matlab can read, and /ml, which contains the data from Mac.

Each of the mat/expName_chxxx files are sequences of 16-bit integers describing the 10kHz wideband which can be read directly in Baudline, as previously discussed, and in Matlab using fread.

The python file also creates some notes.txt files which are derived from the metadata from the Mac, and describe the experimental parameters. These notes can be visualized in a plain text editor, or in the Experiment Notes app. This is a very simple app made through PyQT that shows notes, allows one to view sorted spikes (more on this later) and so forth.The data is directly loaded from the directory structure and notes.txt files, rather than from a database. Looking back at this, this wasn’t the smartest idea; it takes a while for the app to traverse the directory structure, and it would be easier and more flexible to use a database (MySQL or SQLite for example) to store the information rather than flat files.

Using a fixed directory structure makes it straightforward to automate backing up tasks. I have a cron job set up on drone 1 that starts the Unison synching app twice a week. The data and analysis files are backed up to a remote Terastation.

Automated spike sorting

Once the data is fetched and reformatted in a way that Matlab can understand it, the raw wideband data is filtered and spikes are detected and sorted with the help of Wave_clus. Kyler Brown heavily modifed the Wave_clus codebase and created a secondary GUI to enable this. The user first opens the GUI and selects a .mat file in the mat directory of the experiment to be sorted:

Pressing the sort button starts the sorting process. This can take a long time, hence an option was added that sends an email to the relevant people whenever this first (automated) sorting phase is finished. Obviously, it would be feasible to notify the user through other means, like text messaging, if desired.

Once the file is automatically sorted, by selecting the review option the user can tweak sorting through an interface ripped from wave_clus. The interface takes screenshots whenever the user views a channel and places them in the mat folder; these can then be accessed through the Experiment Notes app.

Once the sorting has been reviewed, the user returns to the original GUI and selects another option to use the currently selected experiment as a template for sorting other experiments on the same day.

LFPs are also derived automatically in a separate step after spike sorting using the software I wrote for Zanos, Mineault and Pack (2011).

Conclusion

Dealing with large amounts of neurophysiological data is a challenge. I’ve shared with you how we dealt with this by building a cluster. I’m pretty happy with the chosen solution, although alternative setups would be workable. One possibility is that instead of setting up a cluster, a single computer dedicated to fetching, storing data is created. This is a workable arrangement if a fast enough link is available between the analysis computers and the storage computer. Once the data is sorted (for spikes) or downsampled (for LFPs), the amount of data that needs to be transferred between computers is sufficiently small that it can be comfortable to work with over a standard 100 Mbit/s network. I do think however that it’s not a good idea to use an entirely decentralized system where sorting is done on the analysis computers; too much data needs to be moved around and sharing the data between analyzers is a pain.

Creating scripts to automate fetching and preprocessing I think was a very good idea; it guarantees that the data structure is the same for every experiment. This makes it easier to automate other aspects of the preprocessing like spike sorting.

People use different methods to keep track of the experiments they perform; this can span from experimental notebooks to Excel worksheets to more sophisticated solutions like that enabled by OpenElectrophy. The Experiment Notes app allows central tracking of the experiments performed. OpenElectrophy has a similar functionality, and it would probably have been less trouble to simply use this ready made solution than rolling out our own.

As far as spike sorting goes, it should be possible to use to use proprietary software like Plexon’s spike sorter rather than a homebrew solution. There are several issues with this: it requires the use of Windows; it costs a lot of money; everything is GUI-driven and it seems like it would be a pain to automate; I don’t have the type of fine-grained control over filters and processing that I have with a homebrew solution. I like the algorithm implemented in wave_clus; it’s semi-supervised and manual adjustments are rarely necessary.

It’s also possible to assume that model weights are non-negative, in addition to being sparse. Such non-negative sparse priors are less frequently used that straight sparseness priors, but they can be useful from time to time. You might have a model where model parameters cannot logically be negative, on physical grounds. Or you might want to express a database of images as a sum of features.

Perhaps surprisingly, it’s pretty easy to solve for such problems. Patrik Hoyer shows in his non-negative sparse coding paper how to solve for model weights using a simple iterative method here. He has software available which implements non-negative sparse coding, non-negative matrix factorization, and associated algorithms.

YALL1 (Your ALgorithms for L1) is a Matlab package which can solve many problems related to L1. This includes what it calls the L1/L2+ problem, such that .

The idea is to find out the expected deviance of an oracle (EDO), which knows the internal decision process of the observer with perfect accuracy, minus the value of the internal noise. As it turns out, the only thing that matters for the EDO is the distribution of the internal variable (in the case logistic regression, 1/(1+exp(-eta))) across trials. This distribution can be estimated from observer-observer consistency data (repeated trials). I considered three cases: the internal variable has a beta distribution or one of two discrete distribution which I conjecture give lower and higher bounds for the EDO. I could never prove that these conjectured bounds are real bounds but it’s supported by some simulations.

Here’s the appendix (the references are broken, and it will make more sense in the context of our classification image paper).

To have any chance of detecting single units, it’s therefore quite important to denoise and optimally preprocess the signal. Here’s a rundown of how I do it. It took a lot of tinkering to get everything right, but I’m pretty satisfied with the results.

Of course, it’s important to avoid electrical interference, minimize line noise, use a Faraday cage, patch up some foil around leaky connections, etc. If you have the raw signals available, as opposed to only the waveform snippets, then you can do some more denoising offline. We sample every channel at 10 kHz. Here’s an example of one such channel, with a single good-sized spike on the right-hand side, as well as its time-changing power spectrum below.

As you can see, there’s only a little bit of line noise around 60Hz. More importantly, there’s no visible line noise harmonics around 180 Hz, 300 Hz, etc. The 60 Hz noise you can get rid of pretty easily by high-pass filtering, but the harmonics overlap the power spectrum of the spikes, so ideally you want to keep them to a minimum.

To detect spikes efficiently, it’s best to start by re-referencing the wideband channels. This will get rid of a lot of common noise, reward artifacts, etc. Take a look at the raw cross-correlations of the 96 channels:

Clearly most channels are highly correlated; that’s normal, since the electrodes only cover about 4 mm of cortex, and low frequency are strongly spatially correlated. Now let’s look at the cross-correlations after high-pass filtering the wideband data (using diff in Matlab):

Now we see a much different pattern. High-frequency information tends to be correlated in groups of 16 electrodes. The way the array is wired up, electrodes 1-16 are not necessarily close together on the array. However, they get sent together from the array to the amplifier on the same wire. That means that the correlation pattern that we’re seeing is probably common electrical noise.

Thus, I reference each electrode against the truncated mean of the group of 16 electrodes that it belongs to. I use the truncated mean (taking the mean of samples minus the largest and smallest 2) rather than the mean so that non-common high frequency information (ie spikes) don’t bleed across electrodes.

Next, I upsample the wideband data to 40kHz with cubic splines (csapi). I band pass filter with this snippet of code:

Wn = [500,6000]/40000*2;
[ha,hb] = ellip(2,.1,40,Wn);
upSampData = filtfilt(ha,hb,upSampData);

It’s pretty important, I’ve found, to use a filter which has a slow rolloff on either side of a passband, otherwise you get artifacts. After that, it’s pretty straightforward to detect spikes by thresholding. I use the median absolute deviation (mad) estimator for the noise power, and set the threshold to 4.4 times the standard deviation.

This gives me a bank of spikes which are aligned at their peak. Initially, I ran WaveClus directly on these snippets. However, it can be helpful to redo detection based on optimal filter based on the bank of spikes detected in the first round.

In theory optimal detection is straightforward. Assuming the noise of the signal is white, it simply involves convolving the signal with the template one wants to find. However it’s a bad idea to whiten the signal after upsampling.

So what I do instead is this. I take the mean spike from the first round, and filter it twice with a pseudo-noise-whitening matrix. I say pseudo, because if you take noise snippets and compute the covariance of the noise, you’ll find that it’s very poorly conditioned. If E is the noise covariance matrix, I add 5*eye(128)+.2*diag(((1:128)-56).^2) before taking the eigendecomposition to form a whitening matrix. 56 is the time sample where the peak of the waveform is located. Adding a quadratic term to the diagonal of the covariance matrix ensures that the whitening filter only acts on the area of the spike that contains the peak. Here’s the spike on the left, the (pseudo) whitening matrix in the middle, and the spike after double-whitening. You can see that unlike with a true whitening filter, the double-whitened spike acts locally.

Redetection can be quite helpful for spikes which who are just on the edge of the threshold. Here’s the distribution of the (negative of) the band-pass signal for one electrode. You can see that there’s a pedestal like feature on the right which indicates that there’s some spikes there. If you look only at the local maxima of the signal (right plot), however, you’ll notice that a threshold (shown in blue) can’t separate this activity from the noise very well.

Once you optimally filter, however, you get the distributions below. Notice that the optimal filter moves the spikes out of the background noise distribution, so that it can quite clearly be separated using a simple threshold. After this, all that is left to do is sort the spike snippets in WaveClus.

Specifically, we can assume that the prior in a GLM is given by , where the precision Q is given by:

Then the hyperparameters are selected by optimizing the evidence of the model via trust-region Newton, following a Laplace approximation of the posterior (Bishop, Chapters 3-5). One application of this is in models where the parameters are organized into two or more dimensions (say, x and y or space and time). Then, it’s natural to add a penalty for the smoothness of the parameters along one dimension and a second for the other dimension. What you get is a form of Automatic Smoothness Determination (ASD). Here’s an example of applying this in a logistic regression model where the filter is a 2D Gabor:

The precision is sufficiently flexible to accommodate interesting model structure. For example, in the poster that Theo and I presented at SFN, I used it to track functional connectivity as a function of time. Functional connectivity can be estimated by using the firing patterns of other neurons as inputs to a GLM whose output is the target neuron. Each functional connection is defined by 3 parameters, corresponding to Laguerre basis functions. Only a handful of possible connections will turn out to be significant; therefore, it’s natural to impose a penalty on the magnitude of each group of 3 parameters. Now if in addition we let these parameters change over time, it’s also possible to add one smoothness penalty per group of 3 parameters. In total, we had 28 cells, hence 27 inputs for a given cell; thus, through were used to impose group sparseness, and through were used for temporal smoothness.

You can download the software here.

Of course, the description of neurons using Poisson GLMs abstracts away most biophysical facts about neuronal integration. Somewhere between the Poisson GLM and highly detailed compartmental models of neurons lies the integrate-and-fire (IF) neuron model. IF models assume that the neuron integrates neuronal input until is reaches a threshold, after which it is reset. This can account for certain aspects of neuronal data not accounted for by Poisson GLMs. Among them, there is the fact that neurons frequently show nonlinear phase shifts as their drive is increased. Typically, with a stronger stimulus the neuronal response will not only be stronger but also of shorter latency. This is explained in IF models by the fact that neurons will take less time to reach threshold with stronger inputs.

IF models are a bitch to fit, however. It would be really nice if you could get IF-like effects within a much-easier-to-fit GLM. Paninski (2004) has some hints that this is possible. The idea is to leverage the fact that it possible to integrate spike history effects in a GLM. The traditional way of doing this is to add regressors to the design matrix containing the time-lagged spikes. This allows one to model both the absolute and relative refractory period of neurons, among other things. The result is similar to an autoregressive model with exogenous inputs.

It’s possible to extend this spike-history GLM so that it not only modifies the gain of the neuron after firing but also its temporal filter. In the IF model, right after a spike, the neuron is emptied of its input. This is equivalent to saying that following a spike, its time filter is truncated so that it only integrates over time periods following the latest spike. It’s easy to modify the design matrix of a GLM so that following a spike, inputs before this critical time are set to 0.

Now there are some problems in applying this idea directly. In the traditional IF model, the temporal filter is an exponential. In a typical GLM, the shape of the temporal filter is determined by the data; it will peak after some latency, and it might be bimodal. With a naive application of the IF-GLM, the entire input would be wiped out after a spike, and the cell would only reach full gain after an amount of time equal to its latency. Thus, at the very least, an IF-GLM should account for the latency of the neuron.

Furthermore, since there are many stages of temporal integration between the sensory input and the neuron, the nonlinear integration effects of IF neurons will be smeared out. It might be better if instead of assuming that the temporal filter is emptied after a spike, it is simply changed, and let the data decide what exactly those changes are. Such a GLM with a spike-history-dependent temporal filter is very similar to the Spike Response Model (SRM) of Gerstner. Here’s a very recent NIPS paper examining the link between an integrate-and-fire model and a GLM.

While I haven’t seen any applications of IF-GLMs and SRM-GLMs in the literature as of yet, it will be interesting to see if models which better account for nonlinear temporal integration can explain some of the variance discrepancy in visual cortex.

1. At what frequency should a display (say a CRT) refresh so that it doesn’t appear to flicker (fusion)?
2. At what frequency should a movie or video game be displayed so that it doesn’t appear choppy (smoothness) ?

These are two very different questions. Let’s look at them in turn.

Fusion

The first question is straightforward. It’s been known for more than a hundred years that light flickering at high frequency appears stable. It’s an important phenomenon in everyday life, as many display and lighting technologies work by displaying very brief flashes of light several times a second. This includes the incadescent light bulb, fluorescent lights and cathode ray tubes (CRTs, the bulky glass tubes in old TVs).

It’s easy enough to measure the critical flicker fusion frequency in human observers by asking them to report their sensations when viewing a stimulus flickering on a CRT display with fast phosphors.

Image via Scholarpedia

The critical fusion frequency depends on the luminance of the stimulus and its size, as shown in the graph above (Hecht and Smith, 1936). For a large high luminance stimulus covering the fovea, like a full screen white field on a CRT, flicker fusion occurs at about 60 Hz.

It’s interesting to note that there are cells in the LGN of primates (lateral geniculate nucleus, a relay between the eye and the brain through which the visual signal is forwarded) which respond to higher temporal frequencies than 60 Hz and are more sensitive than human observers to flicker (Spekreijse et al. 1971). That means that the signal is available but somehow it isn’t available to consciousness. Cells in the visual cortex appear to discard the high-frequency information. It is possible, however, that such signals reach the brain through various other means (blindsight or miscellaneous projections to non-visual areas). Indeed, the 120 Hz flicker of older fluorescent lights has been found to cause cognitive deficits and headaches (Veitch and McColl 1995).

It’s also interesting to note that many insects have much faster visual systems than us. They have poor visual acuity, however, because they have so few photoreceptors — hence their vision is much more focused on the analysis of motion than us. Ruck (1961) finds a critical fusion frequency of more than 200 Hz in the housefly, and cites a figure of 300 Hz for the honeybee. Indeed, you need special hardware to do visual stimulation for insect vision research (oscilloscopes were used in the old days, although this might have changed).

Smoothness

Flicker fusion is not very relevant when you see a movie or play a video game. A typical movie is shot at 24 Hz, yet that doesn’t mean that it will feel flickery when played back. Flicker fusion would result if you flashed each image with a fast display like a CRT. You can get around flicker fusion in a movie by presenting it at a higher frame rate, repeating each frame several times, or simply presenting it on a display with built-in persistence like an LCD screen.

Each frame in a movie is only slightly different than the last — this is what enables compression algorithms like MPEG4. This is a very different different situation than in the flicker fusion setup, where each frame is black, then white, then black, etc. The modulations in luminance are much smaller in a movie. Hence fusion occurs at much lower framerates than in the flicker fusion scenario, as is visible in the graph of flicker fusion frequency versus illuminance above.

Image via Wikipedia

Importantly, however, flicker can occur from a different source. If you look at the image of a DLP projector, and move your gaze outside the screen, you will notice fringes of color. This also occurs if you wave your hand in front of the projector, as shown above. In a single-chip DLP projectors, the DLP chip produces black and white images, and the colors red, green and blue are created by a spinning color wheel. Wikipedia states that the wheel of common projectors can rotate at 4 or 5x the frame rate of the signal, so 240 or 300 Hz. This is a very high rate, yet clearly you are able to discriminate such large temporal frequencies.

Between each flash of the projector’s image, your eyes can move up to a third of a degree of visual angle (assuming a peak saccadic velocity of 300 degrees per second, 300 Hz framerate, and 3 colors). Hence, slightly offset images in red, green and blue are captured on the retina, which you perceive as fringes of color.

What’s happening is that temporal frequencies are translated to spatial frequencies through motion. So even if you can’t discriminate flicker through its temporal aspect, adding motion will allow it to be discovered through its spatial signature. In fact, the visual system appears to exploit this effect to enhance its sensitivity to high spatial frequencies — fixational eye movements shift the spatio-temporal frequency spectrum of natural scenes towards frequencies neurons are more sensitive to (Rucci et al., SFN 2011)

Temporal aliasing is the main reason why videogames can look choppy at frame rates of 24 or 30 Hz. Let’s say a line moves from the left of the screen to the right. Its path can be represented as a two-dimensional image where one dimension represents its x position and the other represents time. In the graph shown below at the left, the line moves slowly, and its retinal image, which integrates over several frames, appear smooth with a bit of motion blur.

On the right, the same line is now moving faster. Now the retinal image shows a faded trail of the path of the line; this will be interpreted as choppy when viewed by a human observer. The problem is that a video game video rendering pipeline generates instantaneous snapshots of the video game world at a set sampling frequency (say 30Hz). But the Nyquist-Shannon sampling theorem implies that if stuff happens in the virtual world at higher temporal frequencies than half of the sampling rate (the Nyquist frequency), the result will be temporal aliasing, the creation of  artifacts caused by high temporal frequencies interactions with a slow sampler.

Image via Wikipedia

Note that aliasing has little to do with the visual system; it’s a signal processing effect, and it occurs in many modalities other than time. In the spatial domain, aliasing can causes artifacts like Moire patterns. The image above shows the corresponding effect when a brick wall with spatial frequencies above the sampling frequency is sampled without first eliminating high frequencies through an analog filter — note the interference pattern at the bottom right.

The solution is to sample the video game world at a higher temporal frequency, then downsample before display. In the image domain, the corresponding trick is to render the image at, say 4x the target display rate, then pool the image in regions of 4×4 before display. This is known as anti-aliasing.

Cinema cameras apply their own analog anti-aliasing by integrating photons over the entire period the camera shutter is open (usually 1/24th of a second). This means that frames per second in video games and video cameras are not comparable.

It’s possible to fake spatio-temporal anti-aliasing by combining the current image with a spatially blurred version of previous frames. This was frequently done in the old days of Flash, when fast objects were replaced with spatially blurred versions of these objects to simulate smooth motion with low frame rates. From a signal processing standpoint, what is done is using filters to match the spatio-temporal frequency content of the aliased image sequence and an ideally sampled version of this same sequence.

From Claypool, Claypool and Damaa (2006)

So then what’s the minimum frame rate at which a video game should be rendered to ensure that it doesn’t suffer from jitter or choppiness? There’s no maximal rendering framerate above which aliasing effects are guaranteed to be eliminated. Once again, it has little to do with the human visual system and everything to do with the Nyquist-Shannon sampling theorem. You can always cook up a hypothetical situation which demands arbitrarily large numbers of frames per second to be viewed without artifacts. And the relevant question is really whether the choppiness bothers you or not, rather than whether it’s visible. Claypool, Claypool and Damaa (2006) report that performance saturates in a first-person shooter game at about 30 fps (above; notice that confidence intervals overlap between 30 and 60 fps). So, say it loud, say it proud: 30 fps is good enough for me!

Hecht S, & Smith EL (1936). Intermittent stimulation by light : VI. area and the relation between critical frequency and intensity. The Journal of general physiology, 19 (6), 979-89 PMID: 19872977
Spekreijse H, van Norren D, & van den Berg TJ (1971). Flicker responses in monkey lateral geniculate nucleus and human perception of flicker. Proceedings of the National Academy of Sciences of the United States of America, 68 (11), 2802-5 PMID: 5001396
Veitch, J., & McColl, S. (1995). Modulation of fluorescent light: Flicker rate and light source effects on visual performance and visual comfort. Lighting Research and Technology, 27 (4), 243-256 DOI: 10.1177/14771535950270040301
Ruck, P. (1961). Photoreceptor Cell Response and Flicker Fusion Frequency in the Compound Eye of the Fly, Lucilia sericata (Meigen). Biological Bulletin, 120 (3) DOI: 10.2307/1539540
M. Rucci, J. D. Victor, A. Casile, X. Kuang (2011). Fixational eye movements enhance sensitivity to high spatial frequencies in the retina and LGN. SFN 2011
Mark Claypool, Kajal Claypool, & Feissal Damaab (2006). The Effects of Frame Rate and Resolution on Users Playing First Person Shooter Games. Proceedings of SPIE

One was a continuation of the work on receptive field models of V2. Last year the same subject was tackled and Tony confided that he was disappointed to see little difference between V2 and V1 receptive fields. This year was more conclusive, showing the presence of discrete nonlinear subunits in V2 cells not visible in V1. From a modeling persective, the most interesting aspect I thought was the use of a convolutional neural-net-type to look at tiled subunit structures. My current best try at the Neural Prediction Challenge in V1 is based on this very principle, and of course state-of-the-art V4 modeling is based on this as well. As a student from the Gallant lab told me (sorry, can’t remember the name), well nobody has any better ideas anyways!

The other two show kind of an interesting turnabout in Tony’s world philosophy. The man that hated natural images has started using natural textures to look at population coding in V2. Two results stood out:

1. V2 is much more sensitive than V1 to natural textures
2. V2 encodes natural textures along perceptual lines

Posters are available from the above links.

Gilbert had some interesting ideas in the early 90′s about reorganization of receptive fields following removal of input. The idea is that if you have a scotoma on the retina, or any other type of permanent deinnervation, within minutes or hours receptive fields reorganize towards the location. This increases the amount of information transmitted in this new situation.

He extended this idea to artificial scotoma, which occur when an area of the visual field has zero contrast whereas other areas have high contrast. Say a square of gray against a grating background. The idea was that this virtual lesion would trigger rapid reorganization of RFs in V1, much like a real lesion would.

DeAngelis then did some further experiments on the subject and showed that the claimed reorganization could be explained by a simple change in gain which masquaraded as an increase in the size of V1 RFs towards the non-lesioned area.

Our idea was to extend this research in area V4. The idea goes, even if the only changes in neurons in V1 are gain-related, that might translate into effective RF shape downstream.

Here’s a little cartoon of the idea. Sorry I only have access to Paint from the hotel computer. As an adapting stimulus causes spatially coherent changes in the gain of V1 cells, a neuron downstream which integrates over large area of space sees its receptive field shift. I think there’s some interesting consequences for neural coding: gain changes in downstream areas can bubble up and cause much less trivial changes upstream in a hierarchical scheme.

Unfortunately this idea didn’t really pan out for spikes. We did find changes in receptive field gain but not in RF position. I had big plans to roll out MCMC-based RF tracking methodology but since I didn’t see any changes at all with slightly cruder methods (maximum likelihood in highly parametrized generalized linear models), I kind of gave up on the idea, and so focused on LFPs instead (more later).

Talking to Curtis Baker and Greg DeAngelis and others, though, I think I figured out the problem. I got so worked up on the methodology (I’m a computational guy, remember, that’s what I like) that I didn’t tweak the experimental paradigm enough to get good results. Initially we had natural images as the adaptation stimulus, and that showed promising results. Then we tried better controlled sparse orientation adaptation stimuli, because I wanted to be able to correlate the content of the images with the changes in the RFs, and I think it just didn’t drive the cells. Plus the adaptation period was probably way too short.

Another problem I ran into is that the Utah array recordings we used weren’t as stable as they needed to be. Even losing 10% of spikes towards the end of the recording can easily mess up the analysis of such a dataset. So I had to reject most cells from analysis and that only left about 10 cells per recording day which didn’t really give the inference power I was looking for. If anybody has any insights on stabilizing Utah arrays somehow I’d be very interested.

So instead I mostly analyzed LFP receptive fields. You will notice that there are 2 different components to the LFP receptive fields estimated through reverse correlation. There’s an initial component which is very broadly tuned for space and a later component, either of the same or different polarity, that is much more tightly tuned in space. Both of these components have smoothyl changing retinotopy on the array.

As it turned out the broad component would see some similar changes in peak RF position and size across the entire array, while the tightly tuned component would see much more local changes. What I think is happening is that the tightly spatially tuned component is integrating over a small chunk of cortex, say 250 microns, while the other component is integrating over a larger area. That brings a little twist to the Ringach/Shapley LFP cortical integration story that I’d like to follow through.

The next steps are pretty obvious:

• change the content of the adapting image to trigger change as strongly and rapidly as possible
• increase the adaptation period
• modify the mapping stimulus to map receptive fields as efficiently as possible, possibly using online infomax methods or cruder trial and error
• continue work on estimating a well-defined MUA to increase the number of effective electrodes I can work with
• change spike detection and tracking algorithms to account for nonstationary recordings

Thanks to everyone who went to see the poster, your feedback was invaluable.