Post-Processing Script errors ('userid' and 'rsstoken')

Get help with all aspects of SABnzbd
Forum rules
Help us help you:
  • Are you using the latest stable version of SABnzbd? Downloads page.
  • Tell us what system you run SABnzbd on.
  • Adhere to the forum rules.
  • Do you experience problems during downloading?
    Check your connection in Status and Interface settings window.
    Use Test Server in Config > Servers.
    We will probably ask you to do a test using only basic settings.
  • Do you experience problems during repair or unpacking?
    Enable +Debug logging in the Status and Interface settings window and share the relevant parts of the log here using [ code ] sections.
Howitzer
Newbie
Newbie
Posts: 28
Joined: September 15th, 2016, 8:55 pm

Post-Processing Script errors ('userid' and 'rsstoken')

Post by Howitzer »

I am seeing these errors in my post processing script log periodically.

Code: Select all

'userid' is not recognized as an internal or external command,
operable program or batch file.
'rsstoken' is not recognized as an internal or external command,
operable program or batch file.
This only happens with NZBs from one indexer.

The exact same nzb file when dropped into the Watched folder does not cause these errors.

The exact same release from a different indexer does not cause these errors.

I think the problem has to do with the Post Processing Fail URL value being supplied by this indexer.

The command line being passed to execute my post proc script looks normal up to the last parameter...

https://xxxxxx.xxx/failed/?guid xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx & userid=xxxxx & rsstoken=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

*edited... further tests have shown that the GUID value is sent with a space not an equals sign, this I can fix by combining parm8 and parm9.

This is why my batch file is trying to interpret the 'userid' and 'rsstoken' as commands. Windows Command Process treat & as a seperator between multiple commands to be executed.

Imagine if the Fail URL ended with " & DEL *.*" or " & FORMAT C:\" instead of " & userid=xxxxx"

Can SAB do anything to protect against parameter values that include spaces which would cause a downstream process to handle them incorrectly?
Last edited by Howitzer on July 23rd, 2021, 11:31 am, edited 3 times in total.
User avatar
sander
Release Testers
Release Testers
Posts: 8808
Joined: January 22nd, 2008, 2:22 pm

Re: Post-Processing Script errors ('userid' and 'rsstoken')

Post by sander »

in your batch file, put quotes around it
Howitzer
Newbie
Newbie
Posts: 28
Joined: September 15th, 2016, 8:55 pm

Re: Post-Processing Script errors ('userid' and 'rsstoken')

Post by Howitzer »

I can't... that's not how these parameters work... I can put quotes around the parameters I receive from SAB but if I receive:

parm1 parm2 parm3 parm4 parm5 parm6 parm7 parm8 parm9partofParm8 & Cmd2partofParm8 & Cmd3partofParm8

There is nothing I can do with Cmd2 and Cmd3 as they are not treated as parameters they are part of a distinct separate command.

https://sourcedaddy.com/windows-7/multi ... -line.html

If a value that is supposed to be a URL contains spaces then shouldn't SAB be doing something to ensure it is a valid URL? URLs cannot contain a space, so the spaces that are creating a problem, while the fault of the source indexer site, should probably be escaped into %20 to make the value a technically valid URL.

If this was done then the parm8 value sent would be:

https://xxxxxx.xxx/failed/?guid%20xxxxx ... xxxxxxxxxx
User avatar
sander
Release Testers
Release Testers
Posts: 8808
Joined: January 22nd, 2008, 2:22 pm

Re: Post-Processing Script errors ('userid' and 'rsstoken')

Post by sander »

Checking

We're talking about https://sabnzbd.org/wiki/scripts/post-p ... ng-scripts parameter 8 "URL to be called when job failed (if provided by the server, it is always sent, so check parameter 7!). The URL is provided by some indexers as the X-DNZB-Failure header", right?

And you say passed spaces in the URL, right?
If so, in the NZB, what's the exact URL in the X-DNZB-Failure ? With or without spaces?

And finally: what are you going the do with parameter 8?
Howitzer
Newbie
Newbie
Posts: 28
Joined: September 15th, 2016, 8:55 pm

Re: Post-Processing Script errors ('userid' and 'rsstoken')

Post by Howitzer »

Yes that's the parameter.

Well it looks like it... my postproc script in SAB is a batch file, the first two lines are @ECHO OFF and ECHO %CmdCmdLine% so I can see exactly what I got. It outputs up to the first &

%8 is the url but ends at ?guid
%9 is the value

I combined those as %8=%9 so I had the right value (or so I thought) in case I needed it.

When I removed @ECHO OFF I noticed the other bits coming from SAB and because of the & CMD.exe treats it as a 2nd and 3rd command to be executed, not as a parameter of the first command.

I really don't do anything with parameter 8, I'd be happy to ignore it, but the way the end of the URL (as provided by the indexer I assume) is coming out of SAB is not as a parameter because of the space and the &.
User avatar
sander
Release Testers
Release Testers
Posts: 8808
Joined: January 22nd, 2008, 2:22 pm

Re: Post-Processing Script errors ('userid' and 'rsstoken')

Post by sander »

Check the url in the nzb itself

And I dont understand why you cant ignore the atuff beyond parameter 7
Howitzer
Newbie
Newbie
Posts: 28
Joined: September 15th, 2016, 8:55 pm

Re: Post-Processing Script errors ('userid' and 'rsstoken')

Post by Howitzer »

The post proc fail URL is not in the nzb file itself.

I can ignore the parameters.. that's not the problem. The problem is that some text which should be part of parameter 8 has spaces and a & character. When you pass CMD.EXE a string that contains a space followed by a & character that is treated as the end of the command, all subsequent text up to the next space followed by & is treated as a second command to be executed, same thing for a third command etc.

https://sourcedaddy.com/windows-7/multi ... -line.html

If you read the above link it explains how CMD.EXE treats the values being sent by SAB. There are more complex versions of this using two && so the subsequent commands only run if the previous one is successful (no error) or || so the subsequent commands only run if the previous one fails with an error.

In this case however this is being triggered completely unintentionally because parameter 8 has embedded spaces and since it is supposed to be a URL those spaces should be escaped to a value of %20 when they are present.
Last edited by Howitzer on July 25th, 2021, 8:26 am, edited 3 times in total.
User avatar
safihre
Administrator
Administrator
Posts: 5335
Joined: April 30th, 2015, 7:35 am
Contact:

Re: Post-Processing Script errors ('userid' and 'rsstoken')

Post by safihre »

Could you send me the URL at [email protected] so I can try it?
If you like our support, check our special newsserver deal or donate at: https://sabnzbd.org/donate
Howitzer
Newbie
Newbie
Posts: 28
Joined: September 15th, 2016, 8:55 pm

Re: Post-Processing Script errors ('userid' and 'rsstoken')

Post by Howitzer »

Hey @safihre

Any luck on this? The Indexer in question has come back to me and we have confirmed that the x-dnzb-failure header value they are supplying is a valid URL with no spaces. So the spaces are somehow being added by a process in SAB before being passed to the Post-Proc. and creating this error/problem.
User avatar
safihre
Administrator
Administrator
Posts: 5335
Joined: April 30th, 2015, 7:35 am
Contact:

Re: Post-Processing Script errors ('userid' and 'rsstoken')

Post by safihre »

I can confirm that SABnzbd is sending the information correctly, using debugger I can see that we send the full URL.
Additionally if I use a Python script and print the arguments I get this:

Code: Select all

[
    'C:\\Users\\saf\\Documents\\GitHub\\tests\\test.py', 
    "C:\\Users\\saf\\Videos\\Complete\\[05_31]_-_'Great_Thinkers.epub'.1", 
    "[05_31]_-_'Great_Thinkers.epub'.nzb", "[05_31]_-_'Great_Thinkers.epub'", 
    '*', 
    'alt.binaries.e-book', 
    '0', 
    'https://drunkenslug.com/failed/?guid=96ba25cf260095526cdd82f68f96af7a32a228a9&userid=1234&rsstoken=xxxx1de177d1f3ec310117'
]
If you like our support, check our special newsserver deal or donate at: https://sabnzbd.org/donate
User avatar
sander
Release Testers
Release Testers
Posts: 8808
Joined: January 22nd, 2008, 2:22 pm

Re: Post-Processing Script errors ('userid' and 'rsstoken')

Post by sander »

I can confirm what Safihre says: if NZB comes in via URL, the fail-URL is presented correctly to the postprocessing script as the last parameter. No spaces in there.

Python3 PP-script to prove / verify:

Code: Select all

#!/usr/bin/env python3
import sys
for i in sys.argv:
        print(i)
print("---------------------DONE-------------")
sys.exit(0)
Output:

Code: Select all

/home/sander/.sabnzbd/sab-scripts/print_all_input_pars.py
/media/zeegat/[05_31]_-_'blabla.epub'
[05_31]_-_"blabla.epub".nzb
[05_31]_-_'blabla.epub'

*
alt.binaries.e-book
0
https://drunkenslug.com/failed/?guid=333&userid=444&rsstoken=adsfadsf
---------------------DONE-------------
So ... works for me!
Howitzer
Newbie
Newbie
Posts: 28
Joined: September 15th, 2016, 8:55 pm

Re: Post-Processing Script errors ('userid' and 'rsstoken')

Post by Howitzer »

Well as I said in the first post, I am not using a Python script so what happens with a Python script doesn't really mean anything. Python scripts are handle by a different process than Windows batch scripts.

Create a batch file with one line and use it to post proc to see the issue.

ECHO %CmdCmdLine%

The http headers from the indexer do not contain the spaces. the value passed to CMD.EXE do contain them, the only thing in between is the SAB processing.

Perhaps the issue is that CMD.EXE sees the & and since the URL is not enclosed in quotes it is treating the & as a delimiter? If that is the case then SAB would need to add the quote to make the url safe to be passed as a parameter to CMD.exe.
User avatar
safihre
Administrator
Administrator
Posts: 5335
Joined: April 30th, 2015, 7:35 am
Contact:

Re: Post-Processing Script errors ('userid' and 'rsstoken')

Post by safihre »

It seems to indeed be the interpretation of cmd.exe that's the problem:
https://stackoverflow.com/a/27960888/5235502

SABnzbd doesn't do anything with the arguments, it is just passed to the Python core function which should take care of all this. But unfortunately it doesn't.

Can you confirm that if the parameter is in quotes, it works as expected? That would be easy to implement.
If you like our support, check our special newsserver deal or donate at: https://sabnzbd.org/donate
Howitzer
Newbie
Newbie
Posts: 28
Joined: September 15th, 2016, 8:55 pm

Re: Post-Processing Script errors ('userid' and 'rsstoken')

Post by Howitzer »

Good link! That definitely seems to be the issue.

I tried to simulate from a CMD window...

With quotes worked in that the fail parm was presented as one string parameter but CMD still threw an error:

& was unexpected at this time.
User avatar
safihre
Administrator
Administrator
Posts: 5335
Joined: April 30th, 2015, 7:35 am
Contact:

Re: Post-Processing Script errors ('userid' and 'rsstoken')

Post by safihre »

So it would require special escaping (as mentioned in Stackoverflow) for & and others if cmd.exe is used..
Because the escaping would not be needed for python or other scripts.
Not sure if I want to go down that road..
If you like our support, check our special newsserver deal or donate at: https://sabnzbd.org/donate
Post Reply