SAB slow, barely responsive, downloads horrible

[MountainDew]

I've been using SAB on a VM for about 4 months now and it's worked great up until yesterday. The webui is slow, sometimes barely responsive. In fact, It's not even showing the completed downloads at the bottom half of the screen. Not even after I restart sab or the VM. And the download speed is not at the level I set. If it's set to 300KB, I might get 60KB, 180KB, it's all over the place. My connection is good so the download speed should be consistent.
Any time I try to click on links on the webui, it takes 5-10 secs to respond. Again, this is even after a shutdown of sab, restart of sab, and a restart of the vm. Sab is only taking up 200MB of ram and the VM has 4GB (win7 ultimate with 4 cores).
I suppose I can allocate more ram but, considering it worked fine in the past, why would it be an issue now?
The only real difference between the past and now is I have a huge queue built: about 2TB.
I'm trying to download a current log but it's hanging at the moment. If anyone can provide insight on this, I'd appreciate it.

[shypike]

A 2TB queue is quite a strain, especially when you also have very large NZBs.
Which skin are you using?

[MountainDew]

A 2TB queue is quite a strain, especially when you also have very large NZBs.
Which skin are you using?

smpl - black
Should I try to remove some of my queue?

[shypike]

Should I try to remove some of my queue?

You can at least try it.
If you remove jobs "without data", you can later re-add them using the Status->Repair page.
For smpl and Plush you should also limit visible size to at most 20.
Classic is completely useless with very large queues.

[MountainDew]

Should I try to remove some of my queue?

You can at least try it.

I had about 2TB in the queue and once I got it around 1.8TB, the webui was much faster and more importantly, the downloads resumed.
FYI, I was only viewing 5 in the queue at a time.

[shypike]

Some basic administration must be kept in memory for each job.
I'll do some checks to see whether this explodes sometimes.
Most of the queue admin should reside on disk, only to be loaded when needed.
Did you see excessive memory usage before performance became acceptable again?

[MountainDew]

Did you see excessive memory usage before performance became acceptable again?

The SAB.exe was using over 200MB of ram; the PC ram was over 3gig. Since then, the exe has been under 200MB and the PC memory has been under 3 gigs.

[shypike]

Did you see excessive memory usage before performance became acceptable again?

The SAB.exe was using over 200MB of ram; the PC ram was over 3gig. Since then, the exe has been under 200MB and the PC memory has been under 3 gigs.

200M is not excessive if you have an active download and an article cache.
For a paused queue, it's not extra-ordinary either.
However, it may very well be that SABnzbd parses the complete queue in a clumsy way
each time the web-ui asks for a refresh.
I will look into this, but it will take a while.

[MountainDew]

Just to let you know, when I had the 2TB queue and the memory was at 200MB, the downloads were idle. Even when I tried to resume the queue, nothing happened. Again, after getting to about 1.6TB, downloads resumed.
If you need clarification, let me know.

[shypike]

Clear.
It will take me some time to setup a proper testcase and do the investigation,
as I'm now busy with other issues.