Page 1 of 1

Sab never frees memory allocated to the cache - idles with > 700MB in use

Posted: April 30th, 2010, 1:23 pm
by psychos
Version: 0.5.2 rc1
OS: Ubuntu Server 10.04 (Lucid Lynx)
Install-type: Ubuntu PPA repo (http://ppa.launchpad.net/jcfp/ppa/ubuntu)
Firewall Software: Linux (same machine)
Are you using IPV6? yes
Is the issue reproducible? yes

I have Sab set to have a 512MB memory cache. Once memory has been allocated into the cache and used, Sab never frees the memory, even completely idle after finishing a download. The only way to release the memory is to restart the daemon.

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
psychos   2515  0.0 17.9 1031128 729032 ?      Sl   Apr26  62:52 /usr/bin/python -OO /usr/bin/sabnzbdplus --daemon --server 10.0.0.1:8080

...I have > 700MB (RSS) used, even though the daemon is idle. If I restart the daemon, the memory is freed:

psychos  20342  0.1  1.0 326660 44116 ?        Sl   14:05   0:00 /usr/bin/python -OO /usr/bin/sabnzbdplus --daemon --server 10.0.0.1:8080

As I understand it, Sab caches articles in memory until it has a full file to write (or runs out of memory and has to start storing articles on disk.) If this is the case, memory should be freed once a file is written out.  It might also make sense to flush out to disk and free memory if Sab has been paused for a while (perhaps 15+ minutes, or perhaps configurable.)

At the very least, though, when Sab is fully done downloading/processing and is just idle, the cache memory should be freed.

(Edited to follow bug report posting guidelines.)

Re: Sab never frees memory allocated to the cache - idles with > 700MB in use

Posted: April 30th, 2010, 5:06 pm
by shypike
It's probably the fault of Python's garbage collector that's notoriously
reluctant to give memory back.
The odd thing is that on Windows the memory is returned.

Re: Sab never frees memory allocated to the cache - idles with > 700MB in use

Posted: May 1st, 2010, 1:39 am
by psychos
Yeah, I looked through the pertinent code, and nothing seems to be amiss that the GC shouldn't handle. I worked out a somewhat hackish patch to free the memory; I'm going to take a better look at the code in general and make a cleaner patch to submit.

Re: Sab never frees memory allocated to the cache - idles with > 700MB in use

Posted: May 1st, 2010, 3:50 am
by shypike
psychos wrote: I'm going to take a better look at the code in general and make a cleaner patch to submit.
Would be appreciated!