Memory management in gnumpy

I’m working on a project using gnumpy, which is a wrapper for CUDAmat, a toolbox for GPU computation in Python. I was having memory management issues, where the script would crash after the memory on the graphics card would fill up. I kept trying to delete the variables via del but this seemed to have no effect.

Digging into the internals of gnumpy, once del is called it does not actually delete the object; rather, it marks it as overwritable. I assume this is because allocating memory on the GPU is expensive, so rather than deleting the object it sets it up for recycling; clever. However, it will only reuse an existing deleted array if the new array is of the same size. I was using batches of data of unequal sizes, and pushing them on the GPU as I went along; after a few batches, the GPU was filled up with arrays marked as overwritable.

There are three possible solutions:

  • Make the batches of equal size. This is the most efficient, if least flexible solution – it does not involve allocating and deallocating memory
  • Call gnumpy.free_reuse_cache – this deallocates memory and triggers garbage collection. This is slow
  • Deallocate the memory associated with a given array manually. This may be done as follows:
import gnumpy as gpu
def force_deallocate(thesize):
        gpu.__memoryInUse -= thesize*4
        del gc.garbage[:]
    except IndexError:
        #Can't deallocate if it don't exist, can ya?

#assuming you have an array called a
thesize = a.size
del a #delete variable
force_deallocate(thesize) #deallocate the corresponding memory

2 responses to “Memory management in gnumpy”

  1. Thanks for sharing, Patrick! (I faced similar problems… now I know the reasons and how to tackle them)

Leave a comment

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s