3.2.0RC2 constantly crashes after postprocessor (docker hotio)

Questions and bug reports for Beta releases should be posted here.
Forum rules
Help us help you:
  • Tell us what system you run SABnzbd on.
  • Adhere to the forum rules.
  • Do you experience problems during downloading?
    Check your connection in Status and Interface settings window.
    Use Test Server in Config > Servers.
    We will probably ask you to do a test using only basic settings.
  • Do you experience problems during repair or unpacking?
    Enable +Debug logging in the Status and Interface settings window and share the relevant parts of the log here using [ code ] sections.
User avatar
sander
Release Testers
Release Testers
Posts: 8811
Joined: January 22nd, 2008, 2:22 pm

Re: 3.2.0RC2 constantly crashes after postprocessor

Post by sander »

Also, in that hotio docker:

Code: Select all

bash-5.1# find / -name libc.so.6 -print
bash-5.1#
No libc.so.6?!

An no other library that offers malloc_trim:

Code: Select all

bash-5.1# grep -irn malloc_trim /usr/lib/
bash-5.1#
What kind of stripped down OS is this?

Maybe @jcfp can shine his light on this?
Puzzled
Full Member
Full Member
Posts: 160
Joined: September 2nd, 2017, 3:02 am

Re: 3.2.0RC2 constantly crashes after postprocessor

Post by Puzzled »

I get lots of hits for Symbol not found: malloc_trim and Docker so it's probably a common thing. There should probably be a check before calling it, or do it in a try.
User avatar
sander
Release Testers
Release Testers
Posts: 8811
Joined: January 22nd, 2008, 2:22 pm

Re: 3.2.0RC2 constantly crashes after postprocessor (docker hotio)

Post by sander »

Puzzled wrote: February 18th, 2021, 8:46 am I get lots of hits for Symbol not found: malloc_trim and Docker so it's probably a common thing.
Oh what OS / docker image?
Puzzled
Full Member
Full Member
Posts: 160
Joined: September 2nd, 2017, 3:02 am

Re: 3.2.0RC2 constantly crashes after postprocessor (docker hotio)

Post by Puzzled »

sander wrote: February 18th, 2021, 8:48 am Oh what OS / docker image?
Usually Alpine Linux but several images.
User avatar
sander
Release Testers
Release Testers
Posts: 8811
Joined: January 22nd, 2008, 2:22 pm

Re: 3.2.0RC2 constantly crashes after postprocessor (docker hotio)

Post by sander »

Indeed, Alpine seems to have no libc.so.6 & thus reproducible:

Code: Select all

docker run -it --rm alpine /bin/ash

apk add python3

 # python3
Python 3.8.7 (default, Dec 26 2020, 08:45:55)
[GCC 10.2.1 20201203] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ctypes
>>> ctypes.CDLL("libc.so.6").malloc_trim(0)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.8/ctypes/__init__.py", line 386, in __getattr__
    func = self.__getitem__(name)
  File "/usr/lib/python3.8/ctypes/__init__.py", line 391, in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))
AttributeError: Symbol not found: malloc_trim
>>>
See ... easy to detect.

After
LIBC = ctypes.CDLL("libc.so.6")

SAB should check if malloc_trim() works ok. If not, set LIBC to None to avoid it calling
User avatar
sander
Release Testers
Release Testers
Posts: 8811
Joined: January 22nd, 2008, 2:22 pm

Re: 3.2.0RC2 constantly crashes after postprocessor (docker hotio)

Post by sander »

sabnzbd/__init__.py:38:KERNEL32 = LIBC = None
sabnzbd/__init__.py:61: LIBC = ctypes.CDLL("libc.so.6")

Set LIBC to None if a malloc_trim() results in a Traceback.

Then the "if" should avoid calling it:

sabnzbd/postproc.py : 1044: if sabnzbd.LIBC:
sabnzbd/postproc.py : 1045: sabnzbd.LIBC.malloc_trim(0)

I tried, but so far it didn't work. Probably PEBKAC
Puzzled
Full Member
Full Member
Posts: 160
Joined: September 2nd, 2017, 3:02 am

Re: 3.2.0RC2 constantly crashes after postprocessor (docker hotio)

Post by Puzzled »

What did you try?

How about

Code: Select all

try:
    LIBC.malloc_trim()
except:
    LIBC=None
in __init__.py?
User avatar
sander
Release Testers
Release Testers
Posts: 8811
Joined: January 22nd, 2008, 2:22 pm

Re: 3.2.0RC2 constantly crashes after postprocessor (docker hotio)

Post by sander »

Yes, I did that. But within the hotio docker, with a SAB restart ... so not sure I did the right things / the changes survived the sab/docker restart.


EDIT:

Ah, got it working;



Code: Select all

bash-5.1# ./SABnzbd.py -s 0.0.0.0:8081
SJ: going in
SJ: no libc no malloc
2021-02-18 14:55:04,796::INFO::[SABnzbd:1149] --------------------------------
2021-02-18 14:55:04,797::INFO::[SABnzbd:1150] SABnzbd.py-3.2.0RC2
Puzzled
Full Member
Full Member
Posts: 160
Joined: September 2nd, 2017, 3:02 am

Re: 3.2.0RC2 constantly crashes after postprocessor (docker hotio)

Post by Puzzled »

Do you use docker commit after doing the changes?
User avatar
jcfp
Release Testers
Release Testers
Posts: 986
Joined: February 7th, 2008, 12:45 pm

Re: 3.2.0RC2 constantly crashes after postprocessor

Post by jcfp »

sander wrote: February 18th, 2021, 8:43 amWhat kind of stripped down OS is this?
Probably using some alternative c standard lib without memory management stuff (dietlibc or similar, as typically found in embedded systems). Checking at startup like puzzled proposed should be an easy fix.
User avatar
sander
Release Testers
Release Testers
Posts: 8811
Joined: January 22nd, 2008, 2:22 pm

Re: 3.2.0RC2 constantly crashes after postprocessor (docker hotio)

Post by sander »

I'll send a PR
User avatar
sander
Release Testers
Release Testers
Posts: 8811
Joined: January 22nd, 2008, 2:22 pm

Re: 3.2.0RC2 constantly crashes after postprocessor (docker hotio)

Post by sander »

User avatar
sander
Release Testers
Release Testers
Posts: 8811
Joined: January 22nd, 2008, 2:22 pm

Re: 3.2.0RC2 constantly crashes after postprocessor

Post by sander »

jcfp wrote: February 18th, 2021, 10:52 am
sander wrote: February 18th, 2021, 8:43 amWhat kind of stripped down OS is this?
Probably using some alternative c standard lib without memory management stuff (dietlibc or similar, as typically found in embedded systems). Checking at startup like puzzled proposed should be an easy fix.
Ah: https://pkgs.alpinelinux.org/contents?b ... &repo=main

After installing that, there is a /lib/libc.so.6 , but it's just a link to /lib/libc.musl-x86_64.so.1, and so no malloc_trim() in it.

Anyway: Alpine uses musl (not glibc), which has no malloc_trim().
fringe09
Newbie
Newbie
Posts: 10
Joined: August 20th, 2020, 7:40 pm

Re: 3.2.0RC2 constantly crashes after postprocessor (docker hotio)

Post by fringe09 »

Issue resolved! Pulled the latest nightly image and it no longer crashes. Thank you for fixing it quickly.
User avatar
sander
Release Testers
Release Testers
Posts: 8811
Joined: January 22nd, 2008, 2:22 pm

Re: 3.2.0RC2 constantly crashes after postprocessor (docker hotio)

Post by sander »

fringe09 wrote: February 18th, 2021, 4:10 pm Issue resolved! Pulled the latest nightly image and it no longer crashes. Thank you for fixing it quickly.
Ah, that's nice: hotio nightly does "Every commit to develop branch".

Cool.

Thanks for reporting. TIL ... not each Linux system has libc.
Post Reply