Page 1 of 1

[1.2.1 Beta1] 100% CPU Usage while idle

Posted: February 15th, 2017, 4:13 am
by PsVita
Hi,

I run a service that allows people to host various apps via a one-click install system. I noticed today that on one of my servers, that is running 1.2.1 Beta1, around 50% of all instances are running at 100% CPU.

I've enabled debug log but sadly nothing shows up in the logs for these instances. I also tried strace-ing the processes but it didn't give me enough info to find out what's going on.

Code: Select all

futex(0x17c4230, FUTEX_WAKE_PRIVATE, 1) = 1
select(0, NULL, NULL, NULL, {3, 0})     = 0 (Timeout)
select(0, NULL, NULL, NULL, {3, 0})     = 0 (Timeout)
select(0, NULL, NULL, NULL, {3, 0})     = 0 (Timeout)
select(0, NULL, NULL, NULL, {3, 0})     = 0 (Timeout)
futex(0x17c4230, FUTEX_WAIT_PRIVATE, 0, NULL) = -1 EAGAIN (Resource temporarily unavailable)
futex(0x17c4230, FUTEX_WAIT_PRIVATE, 0, NULL) = 0
futex(0x17c4230, FUTEX_WAIT_PRIVATE, 0, NULL) = 0
futex(0x17c4230, FUTEX_WAIT_PRIVATE, 0, NULL) = -1 EAGAIN (Resource temporarily unavailable)
futex(0x17c4230, FUTEX_WAIT_PRIVATE, 0, NULL) = 0
select(0, NULL, NULL, NULL, {3, 0})     = 0 (Timeout)
select(0, NULL, NULL, NULL, {3, 0})     = 0 (Timeout)
futex(0x17c4230, FUTEX_WAIT_PRIVATE, 0, NULL) = 0
select(0, NULL, NULL, NULL, {3, 0})     = 0 (Timeout)
select(0, NULL, NULL, NULL, {3, 0})     = 0 (Timeout)
futex(0x17c4230, FUTEX_WAIT_PRIVATE, 0, NULL) = -1 EAGAIN (Resource temporarily unavailable)
futex(0x17c4230, FUTEX_WAIT_PRIVATE, 0, NULL) = -1 EAGAIN (Resource temporarily unavailable)
futex(0x7f9018004e30, FUTEX_WAKE_PRIVATE, 1) = 1
futex(0x17c4230, FUTEX_WAKE_PRIVATE, 1) = 1
futex(0x7f9018004e30, FUTEX_WAKE_PRIVATE, 1) = 1
futex(0x17c4230, FUTEX_WAKE_PRIVATE, 1) = 1
select(0, NULL, NULL, NULL, {3, 0})     = 0 (Timeout)
select(0, NULL, NULL, NULL, {3, 0})     = 0 (Timeout)
select(0, NULL, NULL, NULL, {3, 0})     = 0 (Timeout)
select(0, NULL, NULL, NULL, {3, 0})     = 0 (Timeout)
futex(0x17c4230, FUTEX_WAIT_PRIVATE, 0, NULL) = -1 EAGAIN (Resource temporarily unavailable)
futex(0x17c4230, FUTEX_WAIT_PRIVATE, 0, NULL) = 0
select(0, NULL, NULL, NULL, {3, 0})     = 0 (Timeout)
futex(0x17c4230, FUTEX_WAIT_PRIVATE, 0, NULL) = -1 EAGAIN (Resource temporarily unavailable)
select(0, NULL, NULL, NULL, {3, 0})     = 0 (Timeout)
select(0, NULL, NULL, NULL, {3, 0})     = 0 (Timeout)
select(0, NULL, NULL, NULL, {3, 0})     = 0 (Timeout)
select(0, NULL, NULL, NULL, {3, 0})     = 0 (Timeout)
futex(0x17c4230, FUTEX_WAIT_PRIVATE, 0, NULL) = -1 EAGAIN (Resource temporarily unavailable)
select(0, NULL, NULL, NULL, {3, 0})     = 0 (Timeout)
select(0, NULL, NULL, NULL, {3, 0}q)     = 0 (Timeout)
futex(0x7f9018004e30, FUTEX_WAKE_PRIVATE, 1) = 1
futex(0x17c4230, FUTEX_WAKE_PRIVATE, 1) = 1
futex(0x7f9018004e30, FUTEX_WAKE_PRIVATE, 1) = 1
futex(0x17c4230, FUTEX_WAKE_PRIVATE, 1) = 1
Could somebody help me enabling python tracing for Sabnzbd so I can see where the activity is coming from?

Re: [1.2.1 Beta1] 100% CPU Usage while idle

Posted: February 15th, 2017, 4:36 am
by safihre
I don't know anything about that unfortunately.
We haven't heard this before, not while running idle..

Re: [1.2.1 Beta1] 100% CPU Usage while idle

Posted: February 15th, 2017, 5:48 am
by PsVita
Do you have any other tips for me to dig deeper?

Edit:

Found a way to trace Python programs and it seems to be somewhere in the growl related notification code.

It spends a lot of time doing:

Code: Select all

notifier.py(226): 		while not recv_data.endswith("\r\n\r\n"):
notifier.py(227): 			recv_data += s.recv(1024)
... over and over again.
Once I turned growl off (no valid settings were supplied anyway) the CPU usage went down. It seems that my code automatically deploys with Growl enabled even if no settings are specified, this might be the issue. This probably used to be the default for Sab back in the day when I coded up support.

Is there anything that recently changed growl related?

Re: [1.2.1 Beta1] 100% CPU Usage while idle

Posted: February 15th, 2017, 6:58 am
by safihre
Not that we know off. I don't use growl ever.. But it was enabled for a long time by default (by mistake). I think only in 1.0.0 we fixed that.
Maybe refresh your empty default config for the packages with the new version?

Re: [1.2.1 Beta1] 100% CPU Usage while idle

Posted: February 15th, 2017, 7:51 am
by PsVita
Will do that, thanks for the help.