CSHL projects: variance estimation and N-AFC psychophysics

CSHL computational neuroscience: vision is over, and we presented projects using some of the new methods and ideas we explored in class. Earlier in the class, Geoff Boynton did a presentation on psychophysics, and illustrated how to estimate a threshold through Maximum Likelihood in a N-AFC task, where N >= 2. I was curious to understand the properties of such an estimator, so I simulated the full posterior using mcmc (specifically, the delayed rejection adaptive metropolis method) . Turns out the posterior is relatively well-behaved, and 4AFC is quite a bit more efficient than 2AFC, all else being equal. 3AFC is a little trickier, with the criterion of the observer determining whether the procedure is more or less efficient than 2AFC. Here’s the analysis.

My main project was determining how observers can estimate a distribution of inputs that are presented sequentially in time and optimally adapt to this distribution. As it turns out, optimal estimation of variance using a causal nonlinear filter is a tricky problem. I used a boneheaded particle filter to approximate the filter and studied its properties, which are consistent with DeWeese and Zador 1998. Turns out the filter is highly nonlinear, acts asymetrically for increases and decreases in variance, and is not invariant to a uniform scaling of the input. You can see the simulations here.

I then looked at how the discriminability of a stimulus should change as a function of time since the last step increase or decrease in variance, and as it turns out, d’ can change nonmonotically after a change in the variance, and even ignoring the asymmetry in the nonlinear variance estimator, d’ is modulated very differently depending on whether it’s measured after an increase or decrease in variance. This could have some bearing on the results of Snippe et al. (2004). Here are the results.

I spent most of my time doing psychophysical experiments to verify that these effects exist but I failed miserably. C’est la vie. Ideally one would have a more direct method than d’ to find the observer’s internal estimate of the variance of a stimulus. Davis Glasser mentioned that some people have looked at discrimination thresholds for changes in variance as a function of time, this might be one way of doing it.

One response to “CSHL projects: variance estimation and N-AFC psychophysics”

  1. Hey Patrick. It turns out that someone has thought about estimating d-prime from a 4-response yes/no experiment using the method I described at the course. Somehow, my colleague John Palmer knew about this old crusty reference:

    Dorfman, D. D., and E. J. Alf. 1969. Maximum likelihood estimation of parameters of signal-detection theory and determination of confidence intervals—Rating-method data. Journal of Mathematical Psychology, 6, 487-496.

Leave a comment