Is it a bad idea to add a lot to your download queue? [high memory usage]

[tantalus]

I've been way behind on my usenet downloads and my download queue was getting pretty large before I noticed the memory consumption of SAB getting very high.  I had about 250 items (4-8 gigs per item) in the queue and the real memory usage was shown to be over 2 GB. Is this possibly due to it calculating an ETA for each item at all times? It seems other programs I've used before have not had quite the same issue with demanding large amounts of memory. But maybe I've never managed to amass quite the same length of queue before.

I'm on a hackintosh running snow leopard (10.6.2) with a (overclocked to 3.2ghz) q6600 and 8GB of ram.

Also I'm curious why the SAB program seems to top out at 100% CPU utilisation.  With 4 cores to play with, it seems that it might be better if the threads were allowed to use more than this if necessary.  The program seems to bog down at times, but maybe this is all related top my imprudent use of a long queue.

Thanks for reading and for any response.

[shypike]

A large queue takes up memory, because all admin is kept in memory.
What may help a bit is to restart SABnzbd, because after a restart
it won't immediately load everything back in memory.
With a system like your's this queue should not be a problem.
However: don't use the Classic skin, use smpl or Plush.

The programming language we use doesn't support multi-core.

[tantalus]

Thanks for the info.

I was using plush and tried restarting.  It was using 2 GB even fresh after a restart and idling with no connections.  I'll just have to be aware of the limitation I suppose.

[villain]

i tried moving over to the 0.5.x version but ended up going back to 0.4.x because of this (problem?)

im running everything on a macbook with 2gb of ram. it seems when you add ~160 nzbs to the queue and its all loaded into memory everything nearly stops responding

so the idea is to load everything into memory, shutdown sabnzbd, then start it backup and only the items being downloaded are loaded into memory?

[shypike]

0.4.12 isn't any better when memory use is concerned,
in fact it's worse. Especially because paging isn't supported.
The "remedy" is the same for 0.4.12.
So far my testing hasn't revealed any real problems with large queues,
except for the memory usage.
In a future release we'll tackle the memory usage.

[villain]

ok, that's strange then. i can run 0.4.12 happily on this macbook and everything seems to work fine. with 0.5.x when i add 20 nzbs to the queue it looks like it loads all of them into memory. add another 20 and it also adds those into memory. the more i add the more memory it uses.

when using 0.4.12 it only seems to add the parts of the nzb into memory for which it's downloading, not all of the nzbs in the queue