Issues with external processing, mismatched Python

[randoomiser]

In the sabnzbd/newsunpack.py file (line 141-142), the execution of external Python files can be inadvertently run using a different interpreter than the one that started SABnzbd. Firstly, the insertion of the 'python' command is hard-coded for WIN32 systems, which may result in the executable not being found if not on the user's PATH; changing the hardcoded 'python' string to be sys.executable would solve this issue.

Secondly, for systems that would execute the .py file directly, the files use the shebang "/usr/bin/env python" to operate using the default Python interpreter on the user's path. This causes a problem in several situations, as the default Python interpreter can easily differ from that running SABnzbd. This is particularly notable when the system-level Python differs by version - eg system is Python 3.x vs a locally-compiled 2.x installation.

My suggestion is to adjust the code so all Python scripts run using the same interpreter that started SABnzbd, changing line 141-142 like so:

Code: Select all

    if extern_proc.endswith('.py') and (sabnzbd.WIN32 or not os.access(extern_proc, os.X_OK)):
        command.insert(0, 'python')

to

Code: Select all

    if extern_proc.endswith('.py'):
        command.insert(0, sys.executable)

This would resolve both issues by ensuring consistency between running interpreters.

[shypike]

This will not work for Windows and OSX binaries.
These are distributed with partial libraries so that some script might not work with the "embedded" Python.
On Windows you simply need to install a Python distro, you cannot use the embedded distro.
On OSX the binary might not use the system default Python, which your script should use.

Can you explain why it is a problem that SABnzbd and script use different interpreters?

[randoomiser]

Okay, well in my specific case, the system executable of "python" is Python 2.4, whilst I've compiled my own Python 2.7 to run SABNzbd. The post-processing scripts from SickBeard are no longer Python 2.4 compatible and hence the issue. The same issue for would apply for anyone with a system "python" as Python 3.x, and who has installed their own Python 2.x to run SABnzbd.

I'll override my PATH environment variable locally to the SABnzbd process to resolve the issue. However, perhaps it's worth documenting this so users are aware scripts are called with the system-level Python, rather than the one that's running SABnzbd. I was getting SyntaxErrors for the scripts using Python 2.6+ syntax and was confused until going to the source.

[shypike]

I'll override my PATH environment variable locally to the SABnzbd process to resolve the issue. However, perhaps it's worth documenting this so users are aware scripts are called with the system-level Python, rather than the one that's running SABnzbd. I was getting SyntaxErrors for the scripts using Python 2.6+ syntax and was confused until going to the source.

I still don't quite understand it.
In my view the script should be run with either the system default Python or whatever is specified in line 1 of the script (the shebang).
It would be an issue if SABnzbd pushed environment variables to the script that are different from
what the script would normally get. Is this the case?

[randoomiser]

My interpretation of running post-processing Python scripts associated with SABnzbd is that they'd be called using the same Python interpreter, rather than using what could be a different installation (and in my case, is). How this operates wasn't obvious until reading the code, but now that I know how this works inside SABnzbd, I can manipulate my PATH to ensure the expected interpreter is always used.

Separate to the above, though somewhat related, I've noted that the Python script execution will always overrides the shebang. Because of the injection of the 'python' string as the first command argument (sabnzbd/newsunpack.py, line 142), this overrides the shebang and calls the script explicitly uses the default Python interpreter on the PATH. In a situation where someone wants/needs the shebang to point to a specific interpreter, this will be overridden. Not so much a problem in my case, but a potential problem for someone expecting the shebang to be the interpreter used.

[shypike]

The override for Py files was done to allow running Python scripts without the X flag.
This is needed when the scripts are on a CIFS server.
SABnzbd should check for the X flag before overriding.

[shypike]

I checked the code again.
"python" is forced when a .py file has no X-bit flag.
Therefore, when you have #!/bin/program as first line AND the X-bit has been set,
SABnzbd will use /bin/program and not python.

Code: Select all

    if extern_proc.endswith('.py') and (sabnzbd.WIN32 or not os.access(extern_proc, os.X_OK)):
        command.insert(0, 'python')

To me, this is the correct solution.
On Unix and OSX, the shebang will be used as long as the X-bit has been set.
If it isn't set or the platform is Windows, "python" is used.