Fat32 Post Processing Script

[dominicglenn]

Hey guys, I'm having a problem with a/all post processing scripts. I'm using the very latest beta version of sabnzbd on a mac.

I actually use the mac as a download server and my pc to watch stuff but the only way that I can get files off the mac is by using my fat32 formatted ipod (the network is rather slow for file transfer). So I've set up a post processing script 'splitter.py' to automatically split whatever first parameter as a file into separate 4000000000 byte chunks (under 4gb but not by much).

I've tested the script myself and it works perfectly on the mac, however every time that sab finishes downloading a file and goes to use that script it simply reports that it cannot run the script giving no description as to the problem. I've set the permissions of the script to 777 and it should be able to run but it just isn't and it's annoying me a lot.

Do you guys know why this isn't working possibly?

[rollingeyeball]

Python post processing scripts need to be called by something else. I'm not sure what you use on mac but on windows it's a batch file.
I think, anyway.

[switch]

Python scripts should be fine as long as they are executable. I am not sure if setting it to 0777 covers it, but try doing chmod -x on the file.

[dominicglenn]

Setting chmod +x doesn't seem to make a difference.

[shypike]

The chmod +x is required and so is this initial line:

Code: Select all

#!/bin/python

Otherwise program a little shell script around it.

[dominicglenn]

I've currently got it chmoded to 777 and to +x and I'm still getting the same error, simply saying

'Cannot run script /Users/dominicsantos/Newsgroups/Scripts/splitter.py'

Here is the script:

Code: Select all

#!/bin/python

import os, sys

def main():
    import sys

    chunkSize = 4000000000
    readChunk = 5000000

    filename = sys.argv[1]
    fileSize = os.path.getsize(filename)

    if fileSize <= chunkSize:
        exit(0)

    try:
        input = open(filename, 'rb')
    except (OSError, IOError), e:
        exit(1)

    bytesRead = 0
    end = 1
    endString = ''
    buffer = ''

    while bytesRead < fileSize:
        endString = "." + str(end).zfill(3)

        try:
            output = open(filename + endString, 'wb')
        except (OSError, IOError), e:
            exit(1)

        end += 1
        read = 0

        while read < chunkSize:
            buffer = input.read(readChunk)
            output.write(buffer)

            read += len(buffer)
            bytesRead += len(buffer)

            if len(buffer) == 0:
                break

        output.close()

    input.close()
    os.remove(filename)

if __name__=="__main__":
    main()

[shypike]

Have you tried a simple shell script?
Like the sample script provided in SABnzbd's program folder.

[dominicglenn]

May sound a bit silly but I don't know where that is, I've looked inside the package contents of the application, and in the 'Library/Application Support/SABnzbd' folder too.

[shypike]

Just test with this little script:

Code: Select all

#!/bin/sh
echo Test Script
echo $1 $2 $3 $4 $5 $6 $7 $8

chmod +x
And put it in the scripts folder.
I assume the script is on the Mac and not on the iPod.

[dominicglenn]

The scripts are on the mac, and when I created that script file and ran it it did so successfully.

I realised that the header on the python script file was pointing to something that wasn't there, so I made it point to '#!/usr/bin/python' which is where the python interpreter is (I double checked this).

After that it did manage to run the script but told me that it could not find the module 'os' to import it (the script I wrote imports both 'os' and 'sys'). I found a work-around that would allow me to not have to use the 'os' module and removed it. Now when I run the script the only error I get is:

Code: Select all

'import site' failed; use -v for traceback

I have run the script successfully from terminal at every stage of this and it has worked exactly as it should not complaining about 'import site' or anything, it works fine from terminal by typing:

Code: Select all

'python splitter.py'

'/usr/bin/python splitter.py'

'./splitter.py'

The only thing I currently think is that it must be running the python that comes within the SABnzbd package but I've told it specifically to run the other one, the one on my mac.

[shypike]

The only thing I can think of is that the environment variables are not correct.
In the sense that the set of Python-related environment variables
is different when you use the shell compared to when SABnzbd launches your script.
This would make sense if the resident Python and SABnzbd's Python
are different.
In that case you would need a wrapper script that sets these variables correctly.
PYTHONPATH is the major variable, there could be others.

[rAf]

You can use a wrapper. Create a splitter.sh script that call your python script. This code (not tested) should work don't forget to chmod +x your splitter.sh

Code: Select all

#!/bin/sh
`python splitter.py $1 $2 $3 $4 $5 $6 $7`

[dominicglenn]

I've tried to use that wrapper script but it is still giving me problems. Firstly I had to specify exactly where my script was, simply putting 'splitter.py' in the wrapper script caused python to not be able to find it because it was looking in the wrong namespace. The current wrapper script I'm using looks like this:

Code: Select all

#!/bin/sh

`python /Users/dominicsantos/Newsgroups/Scripts/splitter.py "$1"`

That is still giving me errors though, firstly I had to edit my 'splitter.py' script because I need the 'os' module in order to delete the original file after splitting it, now that I've re-added that module it is giving me this error:

Code: Select all

'import site' failed; use -v for traceback
Traceback (most recent call last):
  File "/Users/dominicsantos/Newsgroups/Scripts/splitter.py", line 3, in <module>
    import os, sys
ImportError: No module named os

EDIT: As usual I can run both my splitter.py and splitter_wrapper.sh from the terminal with no problems what so ever and they function as they should.

[shypike]

The wrapper script should set the PYTHONPATH variable correct first.
Otherwise it won't help!

Open a normal shell and look at the environment variables.
Copy anything having to do with Python to the wrapper script.

[rAf]

@shypke : any particular reason that the need_shell is set to False in build_command (newsunpack.py) ?