Page 2 of 2
Re: 3.2.0RC2 constantly crashes after postprocessor
Posted: February 18th, 2021, 8:43 am
by sander
Also, in that hotio docker:
Code: Select all
bash-5.1# find / -name libc.so.6 -print
bash-5.1#
No libc.so.6?!
An no other library that offers malloc_trim:
Code: Select all
bash-5.1# grep -irn malloc_trim /usr/lib/
bash-5.1#
What kind of stripped down OS is this?
Maybe @jcfp can shine his light on this?
Re: 3.2.0RC2 constantly crashes after postprocessor
Posted: February 18th, 2021, 8:46 am
by Puzzled
I get lots of hits for Symbol not found: malloc_trim and Docker so it's probably a common thing. There should probably be a check before calling it, or do it in a try.
Re: 3.2.0RC2 constantly crashes after postprocessor (docker hotio)
Posted: February 18th, 2021, 8:48 am
by sander
Puzzled wrote: ↑February 18th, 2021, 8:46 am
I get lots of hits for Symbol not found: malloc_trim and Docker so it's probably a common thing.
Oh what OS / docker image?
Re: 3.2.0RC2 constantly crashes after postprocessor (docker hotio)
Posted: February 18th, 2021, 8:54 am
by Puzzled
sander wrote: ↑February 18th, 2021, 8:48 am
Oh what OS / docker image?
Usually Alpine Linux but several images.
Re: 3.2.0RC2 constantly crashes after postprocessor (docker hotio)
Posted: February 18th, 2021, 9:17 am
by sander
Indeed, Alpine seems to have no libc.so.6 & thus reproducible:
Code: Select all
docker run -it --rm alpine /bin/ash
apk add python3
# python3
Python 3.8.7 (default, Dec 26 2020, 08:45:55)
[GCC 10.2.1 20201203] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ctypes
>>> ctypes.CDLL("libc.so.6").malloc_trim(0)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.8/ctypes/__init__.py", line 386, in __getattr__
func = self.__getitem__(name)
File "/usr/lib/python3.8/ctypes/__init__.py", line 391, in __getitem__
func = self._FuncPtr((name_or_ordinal, self))
AttributeError: Symbol not found: malloc_trim
>>>
See ... easy to detect.
After
LIBC = ctypes.CDLL("libc.so.6")
SAB should check if malloc_trim() works ok. If not, set LIBC to None to avoid it calling
Re: 3.2.0RC2 constantly crashes after postprocessor (docker hotio)
Posted: February 18th, 2021, 9:19 am
by sander
sabnzbd/__init__.py:38:KERNEL32 = LIBC = None
sabnzbd/__init__.py:61: LIBC = ctypes.CDLL("libc.so.6")
Set LIBC to None if a malloc_trim() results in a Traceback.
Then the "if" should avoid calling it:
sabnzbd/postproc.py : 1044: if sabnzbd.LIBC:
sabnzbd/postproc.py : 1045: sabnzbd.LIBC.malloc_trim(0)
I tried, but so far it didn't work. Probably PEBKAC
Re: 3.2.0RC2 constantly crashes after postprocessor (docker hotio)
Posted: February 18th, 2021, 9:24 am
by Puzzled
What did you try?
How about
Code: Select all
try:
LIBC.malloc_trim()
except:
LIBC=None
in __init__.py?
Re: 3.2.0RC2 constantly crashes after postprocessor (docker hotio)
Posted: February 18th, 2021, 9:28 am
by sander
Yes, I did that. But within the hotio docker, with a SAB restart ... so not sure I did the right things / the changes survived the sab/docker restart.
EDIT:
Ah, got it working;
Code: Select all
bash-5.1# ./SABnzbd.py -s 0.0.0.0:8081
SJ: going in
SJ: no libc no malloc
2021-02-18 14:55:04,796::INFO::[SABnzbd:1149] --------------------------------
2021-02-18 14:55:04,797::INFO::[SABnzbd:1150] SABnzbd.py-3.2.0RC2
Re: 3.2.0RC2 constantly crashes after postprocessor (docker hotio)
Posted: February 18th, 2021, 9:34 am
by Puzzled
Do you use docker commit after doing the changes?
Re: 3.2.0RC2 constantly crashes after postprocessor
Posted: February 18th, 2021, 10:52 am
by jcfp
sander wrote: ↑February 18th, 2021, 8:43 amWhat kind of stripped down OS is this?
Probably using some alternative c standard lib without memory management stuff (dietlibc or similar, as typically found in embedded systems). Checking at startup like puzzled proposed should be an easy fix.
Re: 3.2.0RC2 constantly crashes after postprocessor (docker hotio)
Posted: February 18th, 2021, 11:01 am
by sander
I'll send a PR
Re: 3.2.0RC2 constantly crashes after postprocessor (docker hotio)
Posted: February 18th, 2021, 2:15 pm
by sander
Re: 3.2.0RC2 constantly crashes after postprocessor
Posted: February 18th, 2021, 2:51 pm
by sander
jcfp wrote: ↑February 18th, 2021, 10:52 am
sander wrote: ↑February 18th, 2021, 8:43 amWhat kind of stripped down OS is this?
Probably using some alternative c standard lib without memory management stuff (dietlibc or similar, as typically found in embedded systems). Checking at startup like puzzled proposed should be an easy fix.
Ah:
https://pkgs.alpinelinux.org/contents?b ... &repo=main
After installing that, there is a /lib/libc.so.6 , but it's just a link to /lib/libc.musl-x86_64.so.1, and so no malloc_trim() in it.
Anyway: Alpine uses musl (not glibc), which has no malloc_trim().
Re: 3.2.0RC2 constantly crashes after postprocessor (docker hotio)
Posted: February 18th, 2021, 4:10 pm
by fringe09
Issue resolved! Pulled the latest nightly image and it no longer crashes. Thank you for fixing it quickly.
Re: 3.2.0RC2 constantly crashes after postprocessor (docker hotio)
Posted: February 18th, 2021, 4:23 pm
by sander
fringe09 wrote: ↑February 18th, 2021, 4:10 pm
Issue resolved! Pulled the latest nightly image and it no longer crashes. Thank you for fixing it quickly.
Ah, that's nice: hotio nightly does "Every commit to develop branch".
Cool.
Thanks for reporting. TIL ... not each Linux system has libc.