Python high load when downloading news (on Synology)

[gg99]

Hello

I have the SABnzbd.py-0.5.6 version on a Synology DS211 server. ( DSM3.0)
It runs but the python process is very high when downloading something

Here is a log of the transfer,
There are lot of
"2011-02-21 19:54:50,011::INFO::[__init__:702] /var/packages/sab2/target/sab/cache/SABnzbd_article_ui9EOU removed"
is it normal stuff ?

Thanks for your help

[shypike]

SABnzbd is a bit of a  CPU and memory hog.
The logging message is normal.
These are articles that have been assembled into files
and are no longer needed.
BTW: this should really be a debug logging message and not an info one.
Normally you should set logging to Warnings/Errors only.
Logging takes resources too.

[gg99]

ok
in my case cpu load is 90% for python process on a 1.6GHz CPU

lowering the trace output does not  make any difference

[shypike]

Doesn't sound unlikely, especially if you have SSL on.
Do you have many connections open to the Usenet server(s).
More isn't always better.

[gg99]

I don't use SSL  and my connection number is limited to 20.
Today I tried with the 0.6.0 beta 2 and it doesn't give any improvements

I will try with just few connections and maybe also limit bandwidth and tell you the results

[shypike]

Just checking.
Do you know if your Synology package has a compiled C yenc module?
If not, SABnzbd will do yEnc decoding in Python, which is rather expensive.

[gg99]

Yes the Synology has a yenc module

So I made some tests : reducing connection number has no impact
But reducing bandwidth helps a lot
My full bandwidth is 2MBytes

- With no limit , and no other bandwidth usage : cpu load is over 90%
- Reducing it to 1024 give a CPU load of 15% and 30% for 1536

But what may be interested is :
with Sabnzbd bandwith limit set to 1384 and another internet usage ( such as video on youtube ), the CPU load goes directly to 95% and the reached SABnzbd bandwith is just over 1MB
I set the limit to 1024, and the CPU load goes to 10 % !!!

It seems that a function loops ( or something like this ) , and loads the CPU if the limit is not reached ...

[shypike]

It seems that a function loops ( or something like this ) , and loads the CPU if the limit is not reached ...

Interesting.
There is indeed a loop involved.
But I did not expect this non-linear behaviour.
I'll make a note of this for the next time re-examine the download engine.