Whiten a matrix: Matlab & Python code

Whitening a matrix is a useful preprocessing step in data analysis. The goal is to transform matrix X into matrix Y such that Y has identity covariance matrix. This is straightforward enough, but in case you are too lazy to write such a function here’s how you can do it in Matlab:

function [X] = whiten(X,fudgefactor)
X = bsxfun(@minus, X, mean(X));
A = X'*X;
[V,D] = eig(A);
X = X*V*diag(1./(diag(D)+fudgefactor).^(1/2))*V';
end

You can read about bsxfun here if you are unfamiliar with this function. Because the results of whitening can be noisy, a fudge factor is used so that eigenvectors associated with small eigenvalues do not get overamplified. Thus the whitening is only approximate. Here’s an image patch whitened this way:

This accentuates high frequencies in the image. At the bottom you can see the covariance matrix of the set of natural image patches before and after whitening.

Here’s an explanation for why this works as well as a reference you may cite.

What about in Python? ali_m very nicely translated my code to Python on StackOverflow, which I’m reprinting here for your ease of use:

import numpy as np

def whiten(X,fudge=1E-18):

   # the matrix X should be observations-by-components

   # get the covariance matrix
   Xcov = np.dot(X.T,X)

   # eigenvalue decomposition of the covariance matrix
   d, V = np.linalg.eigh(Xcov)

   # a fudge factor can be used so that eigenvectors associated with
   # small eigenvalues do not get overamplified.
   D = np.diag(1. / np.sqrt(d+fudge))

   # whitening matrix
   W = np.dot(np.dot(V, D), V.T)

   # multiply by the whitening matrix
   X_white = np.dot(X, W)

   return X_white, W

8 comments

  1. Thank you very much!

    I have a question. I’m using python and when i do the “diag(1./(diag(D)+fudgefactor).^(1/2))” the resultant array of that specific computation contains complex numbers, so i get an error when i try to multiply by X*V…*V’ (the rest of the equation). I don’t understand why you don’t get the same error? Thank you very much!

  2. Hi, Thanks for the nice description.
    I am not sure, but I think there is a problem in the above code. To compute covariance matrix you are using X’ * X, so my understanding is that you are considering each row as a sample and columns are features or variable. So, for the mean removal, based on the covariance definition, we need to remove mean from each sample and put samples around the origin in the feature space. As I know, mean(X) without dimension definition and in defaults calculate means of columns. So, in the above code instead of mean removal from rows/samples we are removing means from columns. To wrap up, I think it has to be, mean(X,2) or it has to be X*X’.

Leave a comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s