Truncated NZB filename in post processing script

Come up with a useful post-processing script? Share it here!
Post Reply
copico
Newbie
Newbie
Posts: 8
Joined: May 12th, 2011, 3:45 pm

Truncated NZB filename in post processing script

Post by copico »

In SABnzbd's (v1.0.2) history list GUI, I see that the NZB "source" field has the correct filename:

Code: Select all

Source
auto.Mop.Bear.US.S08E06.American.Aftermarket.720p.HDTV.x264-DHD.nzb
But in my post processing script, I see the NZB filename is truncated:

Code: Select all

Python:

print('argv: %s\n' %(', '.join(sys.argv)))
path, nzb, job, idx, cat, group, status = sys.argv[1:8]
print('path: %s' %(path))
print('nzb: %s' %(nzb))
print('job: %s' %(job))
print('idx: %s' %(idx))
print('cat: %s' %(cat))
print('group: %s' %(group))
print('status: %s\n\n' %(status))

Code: Select all

Output:

argv: e:\prj\scripts\sabnzbd\process.py, f:\down\@new\auto.Mop.Bear.US.S08E06.American.Aftermarket.720p.HDTV.x264-DHD, auto.Mop.Bear.US.S08E06.American.A.nzb, auto.Mop.Bear.US.S08E06.American.Aftermarket.720p.HDTV.x264-DHD, , *, alt.binaries.teevee, 0

path: f:\down\@new\auto.Mop.Bear.US.S08E06.American.Aftermarket.720p.HDTV.x264-DHD
nzb: auto.Mop.Bear.US.S08E06.American.A.nzb
job: auto.Mop.Bear.US.S08E06.American.Aftermarket.720p.HDTV.x264-DHD
idx: 
cat: *
group: alt.binaries.teevee
status: 0
Any idea what I'm doing wrong?
Last edited by copico on June 7th, 2016, 5:09 pm, edited 1 time in total.
User avatar
shypike
Administrator
Administrator
Posts: 19774
Joined: January 18th, 2008, 12:49 pm

Re: Truncated NZB filename in post processing script

Post by shypike »

Truncated?
You mean the ".nzb" part has been removed?
copico
Newbie
Newbie
Posts: 8
Joined: May 12th, 2011, 3:45 pm

Re: Truncated NZB filename in post processing script

Post by copico »

shypike wrote:Truncated?
You mean the ".nzb" part has been removed?
The part before the .nzb in the filename is truncated. For example, in the above, this is the real filename of the nzb:
auto.Mop.Bear.US.S08E06.American.Aftermarket.720p.HDTV.x264-DHD.nzb
But during the post-processing the script gets it as:
auto.Mop.Bear.US.S08E06.American.A.nzb
(The .nzb extension is still there)

The "job name" parameter is correct though:
auto.Mop.Bear.US.S08E06.American.Aftermarket.720p.HDTV.x264-DHD
Last edited by copico on June 7th, 2016, 5:10 pm, edited 1 time in total.
User avatar
sander
Release Testers
Release Testers
Posts: 8811
Joined: January 22nd, 2008, 2:22 pm

Re: Truncated NZB filename in post processing script

Post by sander »

I tried you python script, with SABnzbd 1.0.2, on Ubuntu, and ... no truncating. See below.

So ... isn't there a special chacter in your NZB filename? Can rename an existing NZB file to something long with only plain characters in it, and then try again?

Code: Select all

argv: /home/sander/sab-post-proc/just-print-it.py, /home/sander/Downloads/complete/more-Tales-of-the-City-in-the-2000s-yEnc-blalbla-blalala-dafaddf-adfsads---asdfasdf--asdfaaaa---dddd.mp3, more-Tales-of-the-City-in-the-2000s-yEnc-blalbla-blalala-dafaddf-adfsads---asdfasdf--asdfaaaa---dddd.mp3.nzb, more-Tales-of-the-City-in-the-2000s-yEnc-blalbla-blalala-dafaddf-adfsads---asdfasdf--asdfaaaa---dddd.mp3, , *, alt.binaries.sounds.radio.bbc, 0

path: /home/sander/Downloads/complete/more-Tales-of-the-City-in-the-2000s-yEnc-blalbla-blalala-dafaddf-adfsads---asdfasdf--asdfaaaa---dddd.mp3
nzb: more-Tales-of-the-City-in-the-2000s-yEnc-blalbla-blalala-dafaddf-adfsads---asdfasdf--asdfaaaa---dddd.mp3.nzb
job: more-Tales-of-the-City-in-the-2000s-yEnc-blalbla-blalala-dafaddf-adfsads---asdfasdf--asdfaaaa---dddd.mp3
idx: 
cat: *
group: alt.binaries.sounds.radio.bbc
status: 0
copico
Newbie
Newbie
Posts: 8
Joined: May 12th, 2011, 3:45 pm

Re: Truncated NZB filename in post processing script

Post by copico »

sander wrote:I tried you python script, with SABnzbd 1.0.2, on Ubuntu, and ... no truncating. See below.

So ... isn't there a special chacter in your NZB filename? Can rename an existing NZB file to something long with only plain characters in it, and then try again?
Thanks for testing! I wonder if it's a Windows thing... I'm using Windows 7 x64.

I just tried it with alpha characters only, and it also gets truncated:

Code: Select all

argv: e:\prj\scripts\sabnzbd\process.py, f:\down\@new\testaverylongfilenamewithonlyatozcharactersandnospecialcharacters, testaverylongfilenamewithonlyatozc.nzb, testaverylongfilenamewithonlyatozcharactersandnospecialcharacters, , *, alt.binaries.sounds.mp3.jazz, 0 (len: 8)

path: f:\down\@new\testaverylongfilenamewithonlyatozcharactersandnospecialcharacters
nzb: testaverylongfilenamewithonlyatozc.nzb
job: testaverylongfilenamewithonlyatozcharactersandnospecialcharacters
idx: 
cat: *
group: alt.binaries.sounds.mp3.jazz
status: 0
The others usually have a dot "." or "-" or "_" in the name, but no special character that I can see. The files are mostly from Sonarr.


Not sure if it's useful as a hint, but I noticed the truncated length are all the same:

Code: Select all

testaverylongfilenamewithonlyatozc.nzb
auto.Dame.of.Chromes.S06E06.720p.H.nzb
auto.Meadliest.Patch.S12E10.Into.t.nzb
auto.64.Hours.In.X.And.Y.S11E03.72.nzb
auto.LINDEN_HAS_TALLEN_(2016)_720p.nzb
(34 + ext)
Last edited by copico on June 7th, 2016, 5:14 pm, edited 1 time in total.
copico
Newbie
Newbie
Posts: 8
Joined: May 12th, 2011, 3:45 pm

Re: Truncated NZB filename in post processing script

Post by copico »

Something else odd... I just noticed SABnzbd itself creates two NZB backups, one with the long filename and one with the shortened filename:

auto.Dame.of.Chromes.S06E06.720p.HDTV.x264-TOPKEK.nzb.gz
auto.Dame.of.Chromes.S06E06.720p.H.nzb.gz

auto.64.Hours.In.X.And.Y.S11E03.720p.HDTV.x264-C4TV.nzb.gz
auto.64.Hours.In.X.And.Y.S11E03.72.nzb.gz

The content of the decompressed .nzb inside are identical, only the filenames are different.
Last edited by copico on June 7th, 2016, 5:15 pm, edited 1 time in total.
copico
Newbie
Newbie
Posts: 8
Joined: May 12th, 2011, 3:45 pm

Re: Truncated NZB filename in post processing script

Post by copico »

Could it have something to do with this function in SABnzbd?

Code: Select all

def trim_win_path(path):
    """ Make sure Windows path stays below 70 by trimming last part """
    if sabnzbd.WIN32 and len(path) > 69:
        path, folder = os.path.split(path)
        maxlen = 69 - len(path)
        if len(folder) > maxlen:
            folder = folder[:maxlen]
        path = os.path.join(path, folder).rstrip('. ')
    return path
I couldn't follow the entire code of how it's used exactly, but if I truncate "incomplete_directory\filename" down to 70 characters, it does
seem to match what I'm seeing above...

Code: Select all

\\?\F:\p2p\SABnzbd\_data\incomplete\testaverylongfilenamewithonlyatozcharactersandnospecialcharacters.nzb
\\?\F:\p2p\SABnzbd\_data\incomplete\testaverylongfilenamewithonlyatozc
0........1.........2.........3.........4.........5.........6.........7
1234567890123456789012345678901234567890123456789012345678901234567890
I guess the .nzb filename passed to the post-processing script is derived from this result?

It looks like Windows itself has a 255 character path limit. Is 70 a limit for something else?
User avatar
sander
Release Testers
Release Testers
Posts: 8811
Joined: January 22nd, 2008, 2:22 pm

Re: Truncated NZB filename in post processing script

Post by sander »

Interesting hypothesis.

In postproc.py there is this code:

Code: Select all

            # Run the user script
            script_path = make_script_path(script)
            if (all_ok or not cfg.safe_postproc()) and (not nzb_list) and script_path:
                ...
                script_log, script_ret = external_processing(short_path(script_path, False), short_path(workdir_complete, False), nzo.filename,
                                                             dirname, cat, nzo.group, job_result,
                                                             nzo.nzo_info.get('failure', ''))
                script_line = get_last_line(script_log)
The documentation http://wiki.sabnzbd.org/user-scripts says:
1 The final directory of the job (full path)
2 The original name of the NZB file
3 Clean version of the job name (no path info and ".nzb" removed)
4 Indexer's report number (if supported)
5 User-defined category
6 Group that the NZB was posted in e.g. alt.binaries.x
7 Status of post processing. 0 = OK, 1=failed verification, 2=failed unpack, 3=1+21, -1=failed download
8 URL to be called when job failed (if provided by the server, it is always sent, so check parameter 7!) The URL is provided by some indexers as the X-DNZB-Failure header.2
So that looks the same, doesn't it?

And (if so): parameter 2, "The original name of the NZB", is generated by fileshort_path(workdir_complete, False)

So, indeed based on the workdir (which could be shorter on Windows), and not on the real original NZB name. If the above is true, that would be the explanation.

Regarding "It looks like Windows itself has a 255 character path limit. Is 70 a limit for something else?" ... I don't know. I believe there is a lot of legacy in that code.
copico
Newbie
Newbie
Posts: 8
Joined: May 12th, 2011, 3:45 pm

Re: Truncated NZB filename in post processing script

Post by copico »

I think the first argument passed to external_processing() is the script to be executed (arg #0), so the nzb filename should be in "nzo.filename".

But it looks like the problem has something to do with the "Check before download" (pre-check) option in SABnzbd. If I disable that option, I do get the correct filename in the script:

Code: Select all

argv: e:\prj\scripts\sabnzbd\process.py, f:\down\@new\test2ofaverylongfilenametestwillthisworkletstryitandsee, test2ofaverylongfilenametestwillthisworkletstryitandsee.nzb, test2ofaverylongfilenametestwillthisworkletstryitandsee, , *, alt.binaries.sounds.mp3.jazz, 0 (len: 8)

path: f:\down\@new\test2ofaverylongfilenametestwillthisworkletstryitandsee
nzb: test2ofaverylongfilenametestwillthisworkletstryitandsee.nzb
job: test2ofaverylongfilenametestwillthisworkletstryitandsee
idx: 
cat: *
group: alt.binaries.sounds.mp3.jazz
status: 0

This part in nzbqueue.py is maybe where the problem is?

Code: Select all

res, nzo_ids = ProcessSingleFile(nzo.work_name + '.nzb', nzb_path, keep=True, reuse=True)
Post Reply