I’m working on a project using gnumpy, which is a wrapper for CUDAmat, a toolbox for GPU computation in Python. I was having memory management issues, where the script would crash after the memory on the graphics card would fill up. I kept trying to delete the variables via del but this seemed to have no effect.
Digging into the internals of gnumpy, once del is called it does not actually delete the object; rather, it marks it as overwritable. I assume this is because allocating memory on the GPU is expensive, so rather than deleting the object it sets it up for recycling; clever. However, it will only reuse an existing deleted array if the new array is of the same size. I was using batches of data of unequal sizes, and pushing them on the GPU as I went along; after a few batches, the GPU was filled up with arrays marked as overwritable.
There are three possible solutions:
- Make the batches of equal size. This is the most efficient, if least flexible solution – it does not involve allocating and deallocating memory
- Call gnumpy.free_reuse_cache – this deallocates memory and triggers garbage collection. This is slow
- Deallocate the memory associated with a given array manually. This may be done as follows:
import gnumpy as gpu def force_deallocate(thesize): try: gpu._cmsForReuse[thesize].pop() gpu.__memoryInUse -= thesize*4 del gc.garbage[:] except IndexError: #Can't deallocate if it don't exist, can ya? pass #assuming you have an array called a thesize = a.size del a #delete variable force_deallocate(thesize) #deallocate the corresponding memory