Page 2 of 3

Re: is there a way to optimize and/or speed up SABnzbd+?

Posted: March 14th, 2008, 6:25 pm
by DeXeS
Don't forget that downloading highspeed, unraring, unzipping, par checking just are all very heavy processes. No matter what unrar/download/par/unzip program you use...

Re: is there a way to optimize and/or speed up SABnzbd+?

Posted: March 14th, 2008, 6:37 pm
by steve51184
yeah that's why i want it to download the files THEN par/unrar and not at the same time as this is what's using the cpu and slowing things down

Re: is there a way to optimize and/or speed up SABnzbd+?

Posted: March 15th, 2008, 5:15 am
by neilt0
25mbit + 1.5GHz + 1GB RAM is not a good config to run SAB. I'd addd more RAM and/or use a faster CPU or put up with the fact that the machine can't cope.

That's the spec I used to use to download (with 17mbps throughput available to me, it was struggling) and I'm now using a 2GHz Core 2 Duo with 2GB RAM to get max throughput with SAB.

Re: is there a way to optimize and/or speed up SABnzbd+?

Posted: March 15th, 2008, 12:23 pm
by jcfp
As a linux user, finding out what limits the performance is quite easy. Just run 'top -c -d3' while downloading and see. 0% Idle cpu? -> cpu problem; lots of swapping? -> memory problem; neither? -> disk problem. Most likely you'll find the limited amount of cpu power is to blame. Either that or the hard disk. A gigabyte of memory should be more than enough, I have that in my system and download from usenet at much higher speeds than 25 mbit.
steve51184 wrote: yeah that's why i want it to download the files THEN par/unrar and not at the same time as this is what's using the cpu and slowing things down
That won't help you. While download speed on itself might be higher, the overall process of downloading + unpacking will take at least as much time, if not longer. So doing this is basically useless unless you pay per minute for your connection.

Re: is there a way to optimize and/or speed up SABnzbd+?

Posted: March 15th, 2008, 4:51 pm
by DeXeS
jcfp wrote:
steve51184 wrote: yeah that's why i want it to download the files THEN par/unrar and not at the same time as this is what's using the cpu and slowing things down
That won't help you. While download speed on itself might be higher, the overall process of downloading + unpacking will take at least as much time, if not longer. So doing this is basically useless unless you pay per minute for your connection.
I think he wants this because he probably uses sabnzbd+ on his desktop... not on a server or dedicated computer.
You can automatic parcheck and unrar/unzip once a while with AutoUnpack. but you run unix so that program won't work but there are probably unix like programs out. but if you do this suggestion, you don't have the functionality of sabnzbd+ anymore... only using the downloading part of sabnzbd+

Re: is there a way to optimize and/or speed up SABnzbd+?

Posted: March 15th, 2008, 6:19 pm
by jcfp
If he's concerned about other processes slowing down, he might try 'cpulimit' on the unrar and par2 processes... and if necessary bandwidth limit sabnzbd so that eats less cpu as well. Or just use sabnzbd's scheduling option and let it have all cpu when the system would otherwise just sit idle in the middle of the night. Die Qual der Wahl ;)

Re: is there a way to optimize and/or speed up SABnzbd+?

Posted: March 15th, 2008, 6:31 pm
by steve51184
DeXeS wrote:
jcfp wrote:
steve51184 wrote: yeah that's why i want it to download the files THEN par/unrar and not at the same time as this is what's using the cpu and slowing things down
That won't help you. While download speed on itself might be higher, the overall process of downloading + unpacking will take at least as much time, if not longer. So doing this is basically useless unless you pay per minute for your connection.
I think he wants this because he probably uses sabnzbd+ on his desktop... not on a server or dedicated computer.
You can automatic parcheck and unrar/unzip once a while with AutoUnpack. but you run unix so that program won't work but there are probably unix like programs out. but if you do this suggestion, you don't have the functionality of sabnzbd+ anymore... only using the downloading part of sabnzbd+
this is for a dedicated server with other sites on the server is it's very important to keep the load down when using this script
jcfp wrote: If he's concerned about other processes slowing down, he might try 'cpulimit' on the unrar and par2 processes... and if necessary bandwidth limit sabnzbd so that eats less cpu as well. Or just use sabnzbd's scheduling option and let it have all cpu when the system would otherwise just sit idle in the middle of the night. Die Qual der Wahl ;)
cpulimit is exactly what i need and i've installed it but i now need to know how to integrate it into the script so the par2/unrar commands work with this and not the "nice" command (or both is that's easier)

Re: is there a way to optimize and/or speed up SABnzbd+?

Posted: March 16th, 2008, 3:28 am
by shypike
I'll take this up with the NAS specialist in the team.

Re: is there a way to optimize and/or speed up SABnzbd+?

Posted: March 16th, 2008, 3:33 am
by steve51184
shypike wrote: I'll take this up with the NAS specialist in the team.
thank you very much :)

Re: is there a way to optimize and/or speed up SABnzbd+?

Posted: March 25th, 2008, 1:30 pm
by steve51184
shypike wrote: I'll take this up with the NAS specialist in the team.
been 9 days how's the progress?

Re: is there a way to optimize and/or speed up SABnzbd+?

Posted: March 27th, 2008, 4:09 am
by shypike
None, been too busy with other things.
It's on "the list", it will take some time before this is resolved. There so many other features and bugs to do.
One of team members is looking at the possibility to stop downloading during processing.

Re: is there a way to optimize and/or speed up SABnzbd+?

Posted: April 3rd, 2008, 12:57 pm
by shypike
I have been looking at cpulimit, but I don't quite know how to integrate it.
It works on an already running process, not like nice.

Also, when I apply it externally to a running SABnzbd process, SABnzbd crashes.

I'm afraid this goes beyond my Unix knowledge.

Re: is there a way to optimize and/or speed up SABnzbd+?

Posted: April 3rd, 2008, 5:41 pm
by steve51184
would it not be better to make cpulimit work with par2/unrar rather then SABnzbd?

Re: is there a way to optimize and/or speed up SABnzbd+?

Posted: April 4th, 2008, 1:57 am
by shypike
I still don't know how to use it. It does not work like nice.
I wonder if I start unrar, whether SABnzbd will be able to start cpulimit afterwards.
Also older cpulimit's releases are quite buggy (judging by the releases notes) and
most standard Linux distributions have old releases.

Re: is there a way to optimize and/or speed up SABnzbd+?

Posted: April 4th, 2008, 5:10 am
by jcfp
shypike wrote:I still don't know how to use it. It does not work like nice.
I wonder if I start unrar, whether SABnzbd will be able to start cpulimit afterwards.
Should be possible, as long as sabnzbd is able to figure out what the relevant pid is. Then run cpulimit on that pid, and include the -z option so it will automatically exit if either no program is running with the given pid or after that program terminates:

Code: Select all

$ pidof ls
6569
$ cpulimit -l 5 -z -p 6569
Warning: cannot renice.
To work better you should run this program as root, or adjust RLIMIT_NICE.
For example in /etc/security/limits.conf add a line with: * - nice -10

Process 6569 detected
Process 6569 dead!
$ 
The other option would be to start cpulimit before it is actually needed, targeting programs by name rather pid, and not using the -z option so it will just keep waiting for victims. Such an approach imho is not advisable, since (1) it might also affect processes not started by sabnzbd, and (2) cpulimit will keep running until it is actively kill'ed, meaning sabnzbd would have to watch over that (possible trouble if sabnzbd crashes keeping leftover cpulimit processes running?).