Approximate log determinant of huge matrices

Log determinants frequently need to be computed as part of Bayesian inference in models with Gaussian priors or likelihoods. They can be quite troublesome to compute; done through the Cholesky decomposition, it’s an operation. It’s pretty much infeasible to compute the exact log det for arbitrary matrices > 10,000 x 10,000. Models with that many … More Approximate log determinant of huge matrices

Verifying analytical Hessians

When using optimization functions which require Hessians, it’s easy to mess up the math and end up with incorrect second derivatives. While Matlab’s optimization toolbox offers the DerivativeCheck option for checking gradients, it doesn’t work on Hessians. This nice package available on Matlab Central will compute Hessians numerically, so you can easily double check your … More Verifying analytical Hessians

Using the binomial GLM instead of the Poisson for spike data

Cortical spike trains roughly follow Poisson statistics. When it comes to modeling the spike rate as a function of a stimulus, in a receptive field estimation context, for example, it’s thus natural to use the Poisson GLM. In the Poisson GLM with canonical link, the rate is assumed to be generated by a weighted sum … More Using the binomial GLM instead of the Poisson for spike data

Adaptive Metropolis-Hastings – a plug-and-play MCMC sampler

Gibbs sampling is great but convergence is slow when parameters are correlated. If the covariance structure is known, you can reparametrize to get better mixing. Alternatively you can keep the same parametrization but switch to Metropolis-Hastings with a Gaussian proposal distribution whose covariance is similar to the model parameters. But what if you don’t know … More Adaptive Metropolis-Hastings – a plug-and-play MCMC sampler