[SOLVED] Error startup: CoreFoundation/CoreFoundation.h:

[dirkonline]

Hi,

I am running SAB on a QNAP 431+ (ARM)

It starts etc but apparently, YENC is not working and there is this weird error during startup in the CLI I just cannot figure out:

This is error I get on the CLI:

CoreFoundation/CoreFoundation.h: No such file or directory
compilation terminated.

[~] # /etc/init.d/sabnzbd.sh restart
armv7l
armv7l
Shutting down SABnzbdplus at Sun Feb 12 10:03:56 CET 2017...
Waiting for SABnzbdplus to shut down: 1, Done (Sun Feb 12 10:03:57 CET 2017)
armv7l
Performing SABnzbdplus prestartup checks at Sun Feb 12 10:03:57 CET 2017...
Checking for git... Found!
HEAD is now at 80c8ad5 remove -o from 7zip. Reverses #1149 Fixes #1175
Already up-to-date.
Checking environment
Checking if all configured directories exist....... Done
Creating Symbolic links for required utils. Done
Done
Already up-to-date.
Starting SABnzbdplus/share/CACHEDEV1_DATA/.qpkg/SABnzbdplus/Repository/lib/python/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_7882f2abxb4bfcf60.c:210:43: fatal error: CoreFoundation/CoreFoundation.h: No such file or directory
compilation terminated.
Done (Sun Feb 12 10:04:22 CET 2017)

This is my +Info log from startup:

Code: Select all

2017-02-12 10:04:19,314::INFO::[SABnzbd:1178] --------------------------------
2017-02-12 10:04:19,315::INFO::[SABnzbd:1179] SABnzbd.py-1.2.0 (rev=<HASH>2901752775f08b7)
2017-02-12 10:04:19,315::INFO::[SABnzbd:1180] Full executable path = /share/CACHEDEV1_DATA/.qpkg/SABnzbdplus/sabnzbd/SABnzbd.py
2017-02-12 10:04:19,315::INFO::[SABnzbd:1192] Platform = posix
2017-02-12 10:04:19,316::INFO::[SABnzbd:1193] Python-version = 2.7.12 (default, Dec 26 2016, 19:28:10) 
[GCC 5.4.0]
2017-02-12 10:04:19,316::INFO::[SABnzbd:1194] Arguments = SABnzbd.py -f //share/CACHEDEV1_DATA/.qpkg/SABnzbdplus/Config/sabnzbd.ini --browser 0 --daemon --pid /tmp
2017-02-12 10:04:19,317::INFO::[SABnzbd:1196] Preferred encoding = UTF-8
2017-02-12 10:04:19,317::INFO::[SABnzbd:1244] Read INI file //share/CACHEDEV1_DATA/.qpkg/SABnzbdplus/Config/sabnzbd.ini
2017-02-12 10:04:19,320::INFO::[__init__:994] Loading data for rss_data.sab from //share/CACHEDEV1_DATA/.qpkg/SABnzbdplus/Config/<USERNAME>/rss_data.sab
2017-02-12 10:04:19,321::INFO::[__init__:994] Loading data for totals10.sab from //share/CACHEDEV1_DATA/.qpkg/SABnzbdplus/Config/<USERNAME>/totals10.sab
2017-02-12 10:04:19,322::INFO::[postproc:91] Loading postproc queue
2017-02-12 10:04:19,322::INFO::[__init__:994] Loading data for postproc2.sab from //share/CACHEDEV1_DATA/.qpkg/SABnzbdplus/Config/<USERNAME>/postproc2.sab
2017-02-12 10:04:19,323::INFO::[__init__:994] Loading data for queue10.sab from //share/CACHEDEV1_DATA/.qpkg/SABnzbdplus/Config/<USERNAME>/queue10.sab
2017-02-12 10:04:19,326::INFO::[__init__:994] Loading data for watched_data2.sab from //share/CACHEDEV1_DATA/.qpkg/SABnzbdplus/Config/<USERNAME>/watched_data2.sab
2017-02-12 10:04:19,327::INFO::[__init__:994] Loading data for Rating.sab from //share/CACHEDEV1_DATA/.qpkg/SABnzbdplus/Config/<USERNAME>/Rating.sab
2017-02-12 10:04:19,327::INFO::[__init__:997] //share/CACHEDEV1_DATA/.qpkg/SABnzbdplus/Config/<USERNAME>/Rating.sab missing
2017-02-12 10:04:19,328::INFO::[scheduler:190] Setting schedule for midnight BPS reset
2017-02-12 10:04:19,329::INFO::[__init__:354] All processes started
2017-02-12 10:04:19,330::INFO::[SABnzbd:304] Web dir is /share/CACHEDEV1_DATA/.qpkg/SABnzbdplus/sabnzbd/interfaces/Glitter
2017-02-12 10:04:19,331::INFO::[SABnzbd:304] Web dir is /share/CACHEDEV1_DATA/.qpkg/SABnzbdplus/sabnzbd/interfaces/Config
2017-02-12 10:04:19,623::INFO::[SABnzbd:432] _yenc module... found!
2017-02-12 10:04:19,624::INFO::[SABnzbd:437] Cryptography module... found!
2017-02-12 10:04:19,624::INFO::[SABnzbd:442] par2 binary... found (/share/CACHEDEV1_DATA/.qpkg/SABnzbdplus/x31/bin-utils/par2)
2017-02-12 10:04:19,625::INFO::[SABnzbd:447] par2cmdline binary... found (/share/CACHEDEV1_DATA/.qpkg/SABnzbdplus/x31/bin-utils/par2)
2017-02-12 10:04:19,625::INFO::[SABnzbd:450] UNRAR binary... found (/share/CACHEDEV1_DATA/.qpkg/SABnzbdplus/x31/bin-utils/unrar)
2017-02-12 10:04:19,625::INFO::[SABnzbd:456] unzip binary... found (/usr/bin/unzip)
2017-02-12 10:04:19,626::INFO::[SABnzbd:461] 7za binary... found (/share/CACHEDEV1_DATA/.qpkg/SABnzbdplus/x31/bin-utils/7za)
2017-02-12 10:04:19,626::INFO::[SABnzbd:467] nice binary... found (/share/CACHEDEV1_DATA/.qpkg/SABnzbdplus/x31/bin-utils/nice)
2017-02-12 10:04:19,627::INFO::[SABnzbd:471] ionice binary... found (/share/CACHEDEV1_DATA/.qpkg/SABnzbdplus/x31/bin-utils/ionice)
2017-02-12 10:04:19,627::INFO::[SABnzbd:1299] SSL version OpenSSL 1.0.2j  26 Sep 2016
2017-02-12 10:04:19,627::INFO::[SABnzbd:1300] SSL supported protocols ['TLS v1.2', 'TLS v1.1', 'TLS v1']
2017-02-12 10:04:19,632::INFO::[SABnzbd:1439] Starting web-interface on 0.0.0.0:8800
2017-02-12 10:04:19,633::INFO::[_cplogging:219] [12/Feb/2017:10:04:19] ENGINE Bus STARTING
2017-02-12 10:04:19,642::INFO::[_cplogging:219] [12/Feb/2017:10:04:19] ENGINE Started monitor thread '_TimeoutMonitor'.
2017-02-12 10:04:19,977::INFO::[_cplogging:219] [12/Feb/2017:10:04:19] ENGINE Serving on http://0.0.0.0:8800
2017-02-12 10:04:19,977::INFO::[_cplogging:219] [12/Feb/2017:10:04:19] ENGINE Bus STARTED
2017-02-12 10:04:19,978::INFO::[zconfig:64] No Bonjour/ZeroConfig support installed
2017-02-12 10:04:19,978::INFO::[SABnzbd:1477] Starting SABnzbd.py-1.2.0
2017-02-12 10:04:19,981::INFO::[dirscanner:316] Dirscanner starting up
2017-02-12 10:04:19,983::INFO::[urlgrabber:72] URLGrabber starting up

I did check if the dev tools are installed but they are:

[~] # opkg install python-dev --force-overwrite
Package python-dev (2.7.12-2) installed in root is up to date.

Just to make sure the info is here:

Version: 1.2.0 [1be50eb]
Uptime: 15m
Config File: //share/CACHEDEV1_DATA/.qpkg/SABnzbdplus/Config/sabnzbd.ini
Used cache: Cached 0 articles (0 B)
Parameters: SABnzbd.py -f //share/CACHEDEV1_DATA/.qpkg/SABnzbdplus/Config/sabnzbd.ini --browser 0 --daemon --pid /tmp
Python Version: 2.7.12 (default, Dec 26 2016, 19:28:10) [GCC 5.4.0] [UTF-8]
OpenSSL: OpenSSL 1.0.2j 26 Sep 2016 [TLS v1.2, TLS v1.1, TLS v1]

Any help would be greatly appreciated

[safihre]

This error is caused by the cryptography package not being installed correctly.
If possible, run a
pip install cryptography --upgrade

[safihre]

And for yEnc, can you run this?
pip install http://www.golug.it/pub/yenc/yenc-0.4.0.tar.gz

Are you running the latest clinton-hall package?
http://bit.ly/2jPntF9

[dirkonline]

I am running the latest package by Clinton, yes.

I am about to run the commands now, hold on for a second please:

[dirkonline]

This error is caused by the cryptography package not being installed correctly.
If possible, run a
pip install cryptography --upgrade

[~] # pip install cryptography --upgrade
Collecting cryptography
Downloading cryptography-1.7.2.tar.gz (420kB)
100% |████████████████████████████████| 430kB 805kB/s
Collecting idna>=2.0 (from cryptography)
Downloading idna-2.2-py2.py3-none-any.whl (55kB)
100% |████████████████████████████████| 61kB 2.5MB/s
Collecting pyasn1>=0.1.8 (from cryptography)
Downloading pyasn1-0.2.2-py2.py3-none-any.whl (51kB)
100% |████████████████████████████████| 61kB 2.8MB/s
Requirement already up-to-date: six>=1.4.1 in /share/CACHEDEV1_DATA/.qpkg/Entware-ng/lib/python2.7/site-packages (from cryptography)
Collecting setuptools>=11.3 (from cryptography)
Downloading setuptools-34.1.1-py2.py3-none-any.whl (389kB)
100% |████████████████████████████████| 399kB 868kB/s
Requirement already up-to-date: enum34 in /share/CACHEDEV1_DATA/.qpkg/Entware-ng/lib/python2.7/site-packages (from cryptography)
Collecting ipaddress (from cryptography)
Downloading ipaddress-1.0.18-py2-none-any.whl
Collecting cffi>=1.4.1 (from cryptography)
Downloading cffi-1.9.1.tar.gz (407kB)
100% |████████████████████████████████| 409kB 813kB/s
Collecting appdirs>=1.4.0 (from setuptools>=11.3->cryptography)
Downloading appdirs-1.4.0-py2.py3-none-any.whl
Collecting packaging>=16.8 (from setuptools>=11.3->cryptography)
Downloading packaging-16.8-py2.py3-none-any.whl
Collecting pycparser (from cffi>=1.4.1->cryptography)
Downloading pycparser-2.17.tar.gz (231kB)
100% |████████████████████████████████| 235kB 1.4MB/s
Collecting pyparsing (from packaging>=16.8->setuptools>=11.3->cryptography)
Downloading pyparsing-2.1.10-py2.py3-none-any.whl (56kB)
100% |████████████████████████████████| 61kB 2.4MB/s
Installing collected packages: idna, pyasn1, appdirs, pyparsing, packaging, setuptools, ipaddress, pycparser, cffi, cryptography
Found existing installation: idna 2.1
Uninstalling idna-2.1:
Successfully uninstalled idna-2.1
Found existing installation: pyasn1 0.1.9
Uninstalling pyasn1-0.1.9:
Successfully uninstalled pyasn1-0.1.9
Found existing installation: setuptools 27.2.0
Uninstalling setuptools-27.2.0:
Successfully uninstalled setuptools-27.2.0
Found existing installation: ipaddress 1.0.17
Uninstalling ipaddress-1.0.17:
Successfully uninstalled ipaddress-1.0.17
Found existing installation: pycparser 2.14
Uninstalling pycparser-2.14:
Successfully uninstalled pycparser-2.14
Running setup.py install for pycparser ... done
Found existing installation: cffi 1.8.3
Uninstalling cffi-1.8.3:
Successfully uninstalled cffi-1.8.3
Running setup.py install for cffi ... error
Complete output from command /opt/bin/python2.7 -u -c "import setuptools, tokenize;__file__='/share/CACHEDEV1_DATA/.qpkg/Entware-ng/tmp/pip-build-cvx8Oh/cffi/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /opt/tmp/pip-dkyjAt-record/install-record.txt --single-version-externally-managed --compile:
running install
running build
running build_py
creating build
creating build/lib.linux-armv7l-2.7
creating build/lib.linux-armv7l-2.7/cffi
copying cffi/vengine_cpy.py -> build/lib.linux-armv7l-2.7/cffi
copying cffi/model.py -> build/lib.linux-armv7l-2.7/cffi
copying cffi/commontypes.py -> build/lib.linux-armv7l-2.7/cffi
copying cffi/backend_ctypes.py -> build/lib.linux-armv7l-2.7/cffi
copying cffi/cparser.py -> build/lib.linux-armv7l-2.7/cffi
copying cffi/ffiplatform.py -> build/lib.linux-armv7l-2.7/cffi
copying cffi/vengine_gen.py -> build/lib.linux-armv7l-2.7/cffi
copying cffi/api.py -> build/lib.linux-armv7l-2.7/cffi
copying cffi/lock.py -> build/lib.linux-armv7l-2.7/cffi
copying cffi/__init__.py -> build/lib.linux-armv7l-2.7/cffi
copying cffi/setuptools_ext.py -> build/lib.linux-armv7l-2.7/cffi
copying cffi/cffi_opcode.py -> build/lib.linux-armv7l-2.7/cffi
copying cffi/verifier.py -> build/lib.linux-armv7l-2.7/cffi
copying cffi/recompiler.py -> build/lib.linux-armv7l-2.7/cffi
copying cffi/_cffi_include.h -> build/lib.linux-armv7l-2.7/cffi
copying cffi/parse_c_type.h -> build/lib.linux-armv7l-2.7/cffi
copying cffi/_embedding.h -> build/lib.linux-armv7l-2.7/cffi
warning: build_py: byte-compiling is disabled, skipping.

running build_ext
building '_cffi_backend' extension
creating build/temp.linux-armv7l-2.7
creating build/temp.linux-armv7l-2.7/c
arm-openwrt-linux-gnueabi-gcc -fno-strict-aliasing -O2 -pipe -march=armv7-a -mtune=cortex-a9 -fno-caller-saves -fhonour-copts -Wno-error=unused-but-set-variable -Wno-error=unused-result -mfloat-abi=soft -DNDEBUG -fno-inline -DNDEBUG -O2 -pipe -march=armv7-a -mtune=cortex-a9 -fno-caller-saves -fhonour-copts -Wno-error=unused-but-set-variable -Wno-error=unused-result -mfloat-abi=soft -fPIC -DUSE__THREAD -I/usr/include/ffi -I/usr/include/libffi -I/opt/include/python2.7 -c c/_cffi_backend.c -o build/temp.linux-armv7l-2.7/c/_cffi_backend.o
c/_cffi_backend.c:15:17: fatal error: ffi.h: No such file or directory
compilation terminated.
error: command 'arm-openwrt-linux-gnueabi-gcc' failed with exit status 1

----------------------------------------
Rolling back uninstall of cffi
Command "/opt/bin/python2.7 -u -c "import setuptools, tokenize;__file__='/share/CACHEDEV1_DATA/.qpkg/Entware-ng/tmp/pip-build-cvx8Oh/cffi/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /opt/tmp/pip-dkyjAt-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /share/CACHEDEV1_DATA/.qpkg/Entware-ng/tmp/pip-build-cvx8Oh/cffi/
[~] #

[dirkonline]

And for yEnc, can you run this?
pip install http://www.golug.it/pub/yenc/yenc-0.4.0.tar.gz

Are you running the latest clinton-hall package?
http://bit.ly/2jPntF9

[~] # pip install http://www.golug.it/pub/yenc/yenc-0.4.0.tar.gz
Collecting http://www.golug.it/pub/yenc/yenc-0.4.0.tar.gz
Downloading http://www.golug.it/pub/yenc/yenc-0.4.0.tar.gz (63kB)
100% |████████████████████████████████| 71kB 553kB/s
Requirement already satisfied (use --upgrade to upgrade): yenc==0.4.0 from http://www.golug.it/pub/yenc/yenc-0.4.0.tar.gz in /share/CACHEDEV1_DATA/.qpkg/Entware-ng/lib/python2.7/site-packages

On the qnap forum, a kind poster mentioned that my sab was using the wrong tools folder:

I can't see any real errors there. :S

However, looks like your SAB is using the x31 utilities directory, not the arm directory, so now we'll run this again, but have the links point to x31. Here's the abridged version:
CODE: SELECT ALL
ent_python_path="$(getcfg "Entware-ng" Install_Path -f /etc/config/qpkg.conf)/lib/python2.7/site-packages"
sab_python_path="$(getcfg "SABnzbdplus" Install_Path -f /etc/config/qpkg.conf)/x31/python"
ln -fs "${ent_python_path}/yenc.py" "$sab_python_path"
ln -fs "${ent_python_path}/_yenc.so" "$sab_python_path"
/etc/init.d/sabnzbd.sh restart

You won't need to stop SAB first. Also, won't need your logs as they look fine. You're now only looking for that error on startup (whatever it means )

But, I'll have to call it a night. I'll be back a few hours from now when I'm better rested. D

Good luck!

I did that, but to no avail:

[~] # ent_python_path="$(getcfg "Entware-ng" Install_Path -f /etc/config/qpkg.co nf)/lib/python2.7/site-packages"
[~] # sab_python_path="$(getcfg "SABnzbdplus" Install_Path -f /etc/config/qpkg.c onf)/x31/python"
[~] # ln -fs "${ent_python_path}/yenc.py" "$sab_python_path"
[~] # ln -fs "${ent_python_path}/_yenc.so" "$sab_python_path"
[~] # /etc/init.d/sabnzbd.sh restart
armv7l
armv7l
Shutting down SABnzbdplus at Sun Feb 12 09:56:46 CET 2017...
Waiting for SABnzbdplus to shut down: 1, Done (Sun Feb 12 09:56:47 CET 2017)
armv7l
Performing SABnzbdplus prestartup checks at Sun Feb 12 09:56:47 CET 2017...
Checking for git... Found!
HEAD is now at 80c8ad5 remove -o from 7zip. Reverses #1149 Fixes #1175
Already up-to-date.
Checking environment
Checking if all configured directories exist....... Done
Creating Symbolic links for required utils. Done
Done
Already up-to-date.
Starting SABnzbdplus/share/CACHEDEV1_DATA/.qpkg/SABnzbdplus/Repository/lib/pytho n/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_7882f2abxb4bfcf60. c:210:43: fatal error: CoreFoundation/CoreFoundation.h: No such file or director y
compilation terminated.
Done (Sun Feb 12 09:57:12 CET 2017)
[~] # echo $?
0
[~] # opkg install python-dev --force-overwrite
Package python-dev (2.7.12-2) installed in root is up to date.

I installed SAB anew following these after uninstalling it all:
https://forum.qnap.com/viewtopic.php?f=133&t=129696

The only changes I made afterwards with help of the OP are mentioned above

[~] # ent_python_path="$(getcfg "Entware-ng" Install_Path -f /etc/config/qpkg.co nf)/lib/python2.7/site-packages"
[~] # sab_python_path="$(getcfg "SABnzbdplus" Install_Path -f /etc/config/qpkg.c onf)/x31/python"
[~] # ln -fs "${ent_python_path}/yenc.py" "$sab_python_path"
[~] # ln -fs "${ent_python_path}/_yenc.so" "$sab_python_path"
[~] # /etc/init.d/sabnzbd.sh restart

[sander]

I think you have stumbled into further root causes, more deep down:

Running setup.py install for cffi ... error

c/_cffi_backend.c:15:17: fatal error: ffi.h: No such file or directory

On Ubuntu I would try:

Code: Select all

sudo apt-get install build-essential libssl-dev libffi-dev python-dev

as libffi-dev provides ffi.h. See http://packages.ubuntu.com/trusty/amd64 ... v/filelist

[dirkonline]

I think you have stumbled into further root causes, more deep down:

Running setup.py install for cffi ... error

c/_cffi_backend.c:15:17: fatal error: ffi.h: No such file or directory

On Ubuntu I would try:

Code: Select all

sudo apt-get install build-essential libssl-dev libffi-dev python-dev

Cheers!
hmm I did that based on the Entware commands:

[~] # opkg install python-dev --force-overwrite
Package python-dev (2.7.12-2) installed in root is up to date.

Maybe, the OP on the Qnap Forum has already stumbled on the root cause with my install not looking in the right tool folders but I am frankly unable to sort that out with the knowledge I have.

[sander]

I have no QNAP, but does this work?

Code: Select all

opkg install libffi-dev --force-overwrite

[dirkonline]

I have no QNAP, but does this work?

Code: Select all

opkg install libffi-dev --force-overwrite

I am afraid not, the package is called differently it seems. I bet this is just something small. The original installation went without any errors at all. Thank you kindly for taking the time to help me out here.

[safihre]

It's a bit strange. Because we used to use pyOpenSsl, which has the cryptography package included.
We didn't hear problems with that on the QNAP so I am very surprised that this suddenly wouldn't work.

[safihre]

However, it seems your pip python and sabnzbd python might not be the same or working from the same base directory.
Otherwise yEnc would be detected. Since it's clearly installed!

[dirkonline]

hmm is the path defined in the sab start script= I think it was, yes. So, now I wonder where it SHOULD be pointing.

DAEMON="/opt/bin/python"
DAEMON_PACKAGE="$QPKG_DIR/Repository/lib/python"
DAEMON_OPTS=" SABnzbd.py -f /${QPKG_DIR}/Config/sabnzbd.ini --browser 0 --daemon --pid /tmp" #watch out that --pid only sets the path to the pidfile, not the name

[sander]

Dirk (and Safihre), if we cannot solve, maybe we can SABnzbd detect it more elegantly?

Can you run this on your QNAP:

Code: Select all

python -c "import cryptography; print dir(cryptography); print cryptography.__version__"

Example output:

Code: Select all

$ python -c "import cryptography; print dir(cryptography); print cryptography.__version__"
['__about__', '__all__', '__author__', '__builtins__', '__copyright__', '__doc__', '__email__', '__file__', '__license__', '__name__', '__package__', '__path__', '__summary__', '__title__', '__uri__', '__version__', 'absolute_import', 'division', 'print_function', 'sys', 'warnings']
1.5

If not sure if this gives an error message, but I hope so .... Reason: Safihre tells me the package 'cryptography' is optional for SABnzbd, so if python can detect a problem with the installation, SAB can and can ignore the package.

[dirkonline]

Dirk (and Safihre), if we cannot solve, maybe we can SABnzbd detect it more elegantly?

Can you run this on your QNAP:

Code: Select all

python -c "import cryptography; print dir(cryptography); print cryptography.__version__"

Example output:

Code: Select all

$ python -c "import cryptography; print dir(cryptography); print cryptography.__version__"
['__about__', '__all__', '__author__', '__builtins__', '__copyright__', '__doc__', '__email__', '__file__', '__license__', '__name__', '__package__', '__path__', '__summary__', '__title__', '__uri__', '__version__', 'absolute_import', 'division', 'print_function', 'sys', 'warnings']
1.5

If not sure if this gives an error message, but I hope so .... Reason: Safihre tells me the package 'cryptography' is optional for SABnzbd, so if python can detect a problem with the installation, SAB can and can ignore the package.

['__about__', '__all__', '__author__', '__builtins__', '__copyright__', '__doc__ ', '__email__', '__file__', '__license__', '__name__', '__package__', '__path__' , '__summary__', '__title__', '__uri__', '__version__', 'absolute_import', 'divi sion', 'print_function', 'sys', 'warnings']
1.5.1

There is an output, not sure if that helps though