# Generalized linear models with Gaussian priors in Matlab

I have posted a set of functions to fit GLMs with Gaussian priors on Matlab Central. GLMs are a class of models in which an internal variable is contructed linearly from regressors, ie, $\mathbf{r} = \mathbf{Xw}$, and the observed variable is generated from the internal variable through a non-Gaussian noise process. GLMs include standard linear regression, Poisson regression, and logistic regression models as special cases. These three models are very useful in the estimation of neuronal receptive fields. The variable $\mathbf{w}$ can be fitted by maximizing the likelihood of the data with respect to the model parameters; the built-in Matlab function glmfit does this . This approach is prone to overfitting, however.

Using a prior on $\mathbf{w}$ and finding the maximum a posteriori estimate of $\mathbf{w}$ alleviates overfitting. Gaussian priors of the form $N(\mathbf{w}|0;(\lambda \mathbf{A})^{-1})$ allow the expression of prior knowledge that the weights are of small magnitude (A = I) or that the weights are smooth (A = D, a second order finite difference operator), for example. The set of functions allow the determination of the hyperparameter $\lambda$ through k-fold cross-validation.

1. […] of a revival as a technique for estimating model hyperparameters. I just posted an update on my GLM with quadratic penalties package to use this […]

2. […] function in PMTK to do this step. For the M step, I added a new “weights” option in my GLM toolbox. I filled in the rest by following chapter 13 of Bishop as well as Rabiner […]

3. […] to infer the parameters of this model using any software for GLMs with quadratic penalties, including my own Matlab package. I loaded up a 60s PSTH of neural data (y, 5ms bins) from one of our experiments, which looks like […]