Sab never frees memory allocated to the cache - idles with > 700MB in use

Report & discuss bugs found in SABnzbd
Forum rules
Help us help you:
  • Are you using the latest stable version of SABnzbd? Downloads page.
  • Tell us what system you run SABnzbd on.
  • Adhere to the forum rules.
  • Do you experience problems during downloading?
    Check your connection in Status and Interface settings window.
    Use Test Server in Config > Servers.
    We will probably ask you to do a test using only basic settings.
  • Do you experience problems during repair or unpacking?
    Enable +Debug logging in the Status and Interface settings window and share the relevant parts of the log here using [ code ] sections.
Post Reply
psychos
Newbie
Newbie
Posts: 2
Joined: April 30th, 2010, 12:59 pm

Sab never frees memory allocated to the cache - idles with > 700MB in use

Post by psychos »

Version: 0.5.2 rc1
OS: Ubuntu Server 10.04 (Lucid Lynx)
Install-type: Ubuntu PPA repo (http://ppa.launchpad.net/jcfp/ppa/ubuntu)
Firewall Software: Linux (same machine)
Are you using IPV6? yes
Is the issue reproducible? yes

I have Sab set to have a 512MB memory cache. Once memory has been allocated into the cache and used, Sab never frees the memory, even completely idle after finishing a download. The only way to release the memory is to restart the daemon.

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
psychos   2515  0.0 17.9 1031128 729032 ?      Sl   Apr26  62:52 /usr/bin/python -OO /usr/bin/sabnzbdplus --daemon --server 10.0.0.1:8080

...I have > 700MB (RSS) used, even though the daemon is idle. If I restart the daemon, the memory is freed:

psychos  20342  0.1  1.0 326660 44116 ?        Sl   14:05   0:00 /usr/bin/python -OO /usr/bin/sabnzbdplus --daemon --server 10.0.0.1:8080

As I understand it, Sab caches articles in memory until it has a full file to write (or runs out of memory and has to start storing articles on disk.) If this is the case, memory should be freed once a file is written out.  It might also make sense to flush out to disk and free memory if Sab has been paused for a while (perhaps 15+ minutes, or perhaps configurable.)

At the very least, though, when Sab is fully done downloading/processing and is just idle, the cache memory should be freed.

(Edited to follow bug report posting guidelines.)
Last edited by psychos on April 30th, 2010, 4:56 pm, edited 1 time in total.
User avatar
shypike
Administrator
Administrator
Posts: 19774
Joined: January 18th, 2008, 12:49 pm

Re: Sab never frees memory allocated to the cache - idles with > 700MB in use

Post by shypike »

It's probably the fault of Python's garbage collector that's notoriously
reluctant to give memory back.
The odd thing is that on Windows the memory is returned.
psychos
Newbie
Newbie
Posts: 2
Joined: April 30th, 2010, 12:59 pm

Re: Sab never frees memory allocated to the cache - idles with > 700MB in use

Post by psychos »

Yeah, I looked through the pertinent code, and nothing seems to be amiss that the GC shouldn't handle. I worked out a somewhat hackish patch to free the memory; I'm going to take a better look at the code in general and make a cleaner patch to submit.
User avatar
shypike
Administrator
Administrator
Posts: 19774
Joined: January 18th, 2008, 12:49 pm

Re: Sab never frees memory allocated to the cache - idles with > 700MB in use

Post by shypike »

psychos wrote: I'm going to take a better look at the code in general and make a cleaner patch to submit.
Would be appreciated!
Post Reply