[BASH] Pre-processing script (or queue-filter scripts)

Come up with a useful post-processing script? Share it here!

[BASH] Pre-processing script (or queue-filter scripts)

Postby Mar2zz » April 14th, 2011, 4:05 pm

It took some effort to find out how this works, so I thougt to share the knowledge how this works. Sabnzbd 6.x can do preprocessing, what means it can do stuff to the nzb before it enters the queue.

Sab starts the scripts and feeds it parameters like this:
Running pre-queue script ['/home/username/.ppdir/pre-queue.sh', 'ubuntu 11.04', '3', '', '', '', '1579198044', 'alt.binaries.boneless alt.binaries.misc', '', '', '', '']
All parameters are translated into $x (where x is the number of paramater, like $1 here is ubuntu 11.04, the name of the nzb-file)
Here are all parameters and their numbers: http://wiki.sabnzbd.org/user-pre-queue-script

Now you can use this input to alter things before it enters the queue. Like setting a category. When downloading a movie from a 'normal' indexsite like binsearch, sab doesn't know by itself its a movie. But when the group is alt.binaries.movies, it will be a movie 99% of the times something is posted in that group. The same goes for ebooks, tvshows, playstation 3 etc. So if parameter 7 ($7) contains movie, you'll want to change categorie to movie. In 5.x this couldn't be automated. But now it can. An example script, this script will set category to specified if a word is found in the group (like alt.binaries.movies will set category to movies, because movie is found):

Code: Select all
#!/usr/bin/env bash
#
# NZB set category Script by Mar2zz v0.1

# input (this is what sabnzbd feeds to this script)
# All parameters (except 1) can be empty, meaning a default value.
# 1 : Name of the NZB (no path, no ".nzb")
# 2 : PP (0, 1, 2 or 3)
# 3 : Category
# 4 : Script (no path)
# 5 : Priority (-100, -1, 0 or 1 meaning Default, Low, Normal, High)
# 6 : Size of the download (in bytes)
# 7 : Group list (separated by spaces)
# 8 : Show name
# 9 : Season (1..99)
# 10 : Episode (1..99)
# 11: Episode name

# outputparams (what this script tells sabnzbd)
# The script writes the results to the console, each parameter on a separate line.
# Each parameter (except 1) can be an empty line, meaning the original value.
# 1 : 0=Refuse, 1=Accept
# 2 : Name of the NZB (no path, no ".nzb")
# 3 : PP (0, 1, 2 or 3)
# 4 : Category
# 5 : Script (basename)
# 6 : Priority (-100 -2, -1, 0 or 1, meaning Default, Paused, Low, Normal, High )
# 7 : Group to be used (in case your provider doesn't carry all groups and there are multiple groups in the NZB)


### if a word between *'s is found in the groups the category will be changed to the one that's set.
case $7 in
   *movie*)
      cat=movies
      ;;
   *book*)
      cat=ebooks
      ;;
   *teevee*)
      cat=tvshows
      ;;
   *multimedia*)
      cat=tvshows
      ;;
   *)
      cat=$3
      ;;
esac


### now tell sab what to do with nzb (note, this must be echoed in specific order, every command on a new line)
echo "1"    # tell sab to use this .nzb
echo $1    # keep original name for nzb
echo $2    # keep original pp
echo "$cat"    #set category to new if one of the parameters was found


That's it. Make this script executable and set it as queue-filter script in sab. Expand it with other 'trigger' words to automate more categorys (e.g. anime or lossless).

The same stuff can be done to nzb names. Just echo $1 and do something to it. echo $1 | sed 's/.par//' will always remove .par from nzbnames (sometimes that happens).

It can be done to all 11 parameters, so it's possible to cover everything. Very nice feature! Thank you devs! sab > {insert random nzb-program} (always has been)

edit:
Also, here is code that notifies XBMC when an nzb enters the queue, similar to what sickbeard and couchpotato can do.
Code: Select all
### If you want to be notified on snatched downloads in XBMC, please specify your xbmc credentials:
### Note: curl needs to be installed for this (sudo apt-get install curl)
xbmc_notify () {
xbmc_host=xbmc # hostname or ipadres
xbmc_port=8080
xbmc_user=xbmc # if user is not set delete '$xbmc_user:$pass@' in url below
xbmc_pass=xbmc # if password is not set delete ':$pass@' in url below

if which curl > /dev/null
   then
   queued=$(echo $1 | sed -e 's/ /%20/g')
   curl -s "http://$xbmc_user:$xbmc_pass@$xbmc_host:$xbmc_port/xbmcCmds/xbmcHttp/?command=ExecBuiltIn(Notification(Snatched,$queued))" | xargs echo > /dev/null
fi
}
xbmc_notify # comment this if you do not want to use xbmc-notify
Last edited by Mar2zz on September 23rd, 2011, 7:58 pm, edited 2 times in total.
User avatar
Mar2zz
Jr. Member
Jr. Member
 
Posts: 85
Joined: February 4th, 2011, 9:30 am

Re: [BASH] Pre-processing script (or queue-filter scripts)

Postby Mar2zz » June 8th, 2011, 8:50 am

A script that pauses all nzb's coming from sickbeard (can also be used to set low priority) (someone asked for in PM, thought to share it here):
Code: Select all
#!/usr/bin/env bash


### Set to paused if it's a file from sickbeard
case $3
   TV)    # <- change this to your Sickbeard category
      priority="-2" # -2 means paused, change to 0 if you want to set it to low (low means: goes to end of queue automatic)
      ;;
   *)
      priority="-100" # -100 means Default
      ;;
esac


### now tell sab what to do with nzb (note, this must be echoed in specific order, every command on a new line)
echo "1"       # 1 tell sab to use this .nzb (1) or not (0)
echo $1       # 2 set nzbname
echo $2       # 3 set pp
echo         # 4 set category
echo         # 5 set script
echo "$priority"   # 6 Set priority
echo         # 7 Used group
User avatar
Mar2zz
Jr. Member
Jr. Member
 
Posts: 85
Joined: February 4th, 2011, 9:30 am

Re: [BASH] Pre-processing script (or queue-filter scripts)

Postby avalonai2 » July 12th, 2011, 2:36 pm

Is there a script like this for windows? This looks like a fantastic feature.
avalonai2
Newbie
Newbie
 
Posts: 1
Joined: July 11th, 2011, 11:08 am

Re: [BASH] Pre-processing script (or queue-filter scripts)

Postby hrtrulz » July 25th, 2011, 10:05 am

I would like a version of this script to run on widows as well
hrtrulz
Newbie
Newbie
 
Posts: 2
Joined: July 25th, 2011, 10:03 am

Re: [BASH] Pre-processing script (or queue-filter scripts)

Postby ldh830114 » September 6th, 2011, 6:03 am

Mar2zz wrote:A script that pauses all nzb's coming from sickbeard (can also be used to set low priority) (someone asked for in PM, thought to share it here):
Code: Select all
#!/usr/bin/env bash


### Set to paused if it's a file from sickbeard
case $3
   TV)    # <- change this to your Sickbeard category
      priority="-2" # -2 means paused, change to 0 if you want to set it to low (low means: goes to end of queue automatic)
      ;;
   *)
      priority="-100" # -100 means Default
      ;;
esac


### now tell sab what to do with nzb (note, this must be echoed in specific order, every command on a new line)
echo "1"       # 1 tell sab to use this .nzb (1) or not (0)
echo $1       # 2 set nzbname
echo $2       # 3 set pp
echo         # 4 set category
echo         # 5 set script
echo "$priority"   # 6 Set priority
echo         # 7 Used group



Please I would also like a windows script this would solve all my problems
ldh830114
Newbie
Newbie
 
Posts: 3
Joined: September 5th, 2011, 3:49 am

Re: [BASH] Pre-processing script (or queue-filter scripts)

Postby akan88 » September 21st, 2011, 6:19 am

I have tried adapting the script to change to pause when size is over a threshold - however I can't get it to work. Can anyone see anythign glaringly obvious?

Code: Select all
PRIORITY=$5
SIZE=$6
IF [ $SIZE > 2791000000 ]
THEN
        PRIORITY="-2"
FI

### now tell sab what to do with nzb (note, this must be echoed in specific order, every command on a new line)
echo "1"       # 1 tell sab to use this .nzb (1) or not (0)
echo $1       # 2 set nzbname
echo $2       # 3 set pp
echo         # 4 set category
echo         # 5 set script
echo "$PRIORITY"   # 6 Set priority

akan88
Newbie
Newbie
 
Posts: 4
Joined: September 21st, 2011, 6:16 am

Re: [BASH] Pre-processing script (or queue-filter scripts)

Postby Mar2zz » September 22nd, 2011, 9:06 am

use -gt instead of >, or see this for more options (note the difference between string and integer comparisons)
http://tldp.org/LDP/abs/html/comparison ... OMPARISON1
User avatar
Mar2zz
Jr. Member
Jr. Member
 
Posts: 85
Joined: February 4th, 2011, 9:30 am

Re: [BASH] Pre-processing script (or queue-filter scripts)

Postby akan88 » September 22nd, 2011, 3:03 pm

Tried the -gt, no dice. I confirmed by echoing a string within the if statement to a file. The file logs the fact the if statement evaluation works. In the sabnzbd logs I see

A bunch of lines stating that the nzb within the incomplete folder is created then

2011-09-22 20:00:53,184::INFO::[newsunpack:1395] Running pre-queue script ['/home/me/sabpreprocess.sh', 'Some NZB, '', '', '', '-100', '8941791754', 'alt.binaries.X.Z', '', '', '', '']
2011-09-22 20:00:53,201::INFO::[newsunpack:1415] Pre-Q accepts Some NZB

And then a bunch of lines about the directory within the incomplete fold being removed

Help??!?!?!?
akan88
Newbie
Newbie
 
Posts: 4
Joined: September 21st, 2011, 6:16 am

Re: [BASH] Pre-processing script (or queue-filter scripts)

Postby Mar2zz » September 22nd, 2011, 4:04 pm

hmm, maybe because you echo blanks? Try this:
Code: Select all
### return things to sab (changed or unchanged)
echo 1         # 0=Refuse, 1=Accept
echo $1   # Name of the NZB (no path, no ".nzb")
echo $2         # PP (0, 1, 2 or 3)
echo $3      # Category
echo $4         # Script (basename)
echo $PRIORITY      # Priority (-100 -2, -1, 0 or 1, meaning Default, Paused, Low, Normal, High )
echo $6         # Group to be used (in case your provider doesn't carry all groups and there are multiple groups in the NZB)
User avatar
Mar2zz
Jr. Member
Jr. Member
 
Posts: 85
Joined: February 4th, 2011, 9:30 am

Re: [BASH] Pre-processing script (or queue-filter scripts)

Postby akan88 » September 23rd, 2011, 7:12 am

Nope - didn't work

Also - the documentation in the header in the sample above says:

# outputparams (what this script tells sabnzbd)
# The script writes the results to the console, each parameter on a separate line.
# Each parameter (except 1) can be an empty line, meaning the original value.
akan88
Newbie
Newbie
 
Posts: 4
Joined: September 21st, 2011, 6:16 am

Re: [BASH] Pre-processing script (or queue-filter scripts)

Postby Mar2zz » September 23rd, 2011, 11:22 am

Ok, I tested in my own file and this works for me:
Code: Select all
if [ "$6" -gt "100" ]
   then
   priority="-2"
fi

### return things to sab (changed or unchanged)
echo 1             # 0=Refuse, 1=Accept
echo $nzb          # Name of the NZB (no path, no ".nzb")
echo $2            # PP (0, 1, 2 or 3)
echo $3            # Category
echo $4            # Script (basename)
echo $priority     # Priority (-100 -2, -1, 0 or 1, meaning Default, Paused, Low, Normal, High )
echo $6            # Group to be used (in case your provider doesn't carry all groups and there are multiple groups in the NZB)
User avatar
Mar2zz
Jr. Member
Jr. Member
 
Posts: 85
Joined: February 4th, 2011, 9:30 am

Re: [BASH] Pre-processing script (or queue-filter scripts)

Postby akan88 » September 23rd, 2011, 4:15 pm

Mar2zz

Thanks for all your help.

Multiple issues but finally fixed it. Didn't mention this was on a QNap box and bash wasn't the default.

Had to set bash. if...then should be lower case!. Your -gt also helped.

I have it working now - yay!

Quick tip for those struggling and I should have done, was to execute the script with dummy arguments and check the output.

Thanks again Mar2zz!
akan88
Newbie
Newbie
 
Posts: 4
Joined: September 21st, 2011, 6:16 am

Re: [BASH] Pre-processing script (or queue-filter scripts)

Postby Mar2zz » September 23rd, 2011, 4:59 pm

You are welcome :)

The easiest way to debug is to fire off the script in a terminal with fake variables. Like this:
Code: Select all
./path/to/script 'Inception (2011)' '' 'movies' '' '-100' '1710099138' 'alt.binaries.boneless' '' '' '' ''

The output tell you what is wrong or how the variables are changed.

I did a complete rewrite of my own script. (I like the size thing, I lower priority if filesize is greater than 9GB, and give high priority to sickbeard and couchpotato-nzb's.

Code: Select all
#!/bin/sh
#
# Sabnzbd Queue Script by Mar2zz v0.2

### USAGE ###
#
# Below are examples for changing inputparameters before the nzb enters the queue.
# By manipulating variables with bash you can have almost complete automated control over your queue.
# Those variables need to be echoed in specific order. So, don't change the order below this.
#
###


### INPUT VARIABLES ###
# input (this is what sabnzbd feeds to this script and can be changed before the nzb hits the queue)
# All parameters (except 1) can be empty, meaning a default value.
# 1 : Name of the NZB (no path, no ".nzb")
# 2 : PP (0, 1, 2 or 3)
# 3 : Category
# 4 : Script (no path)
# 5 : Priority (-100, -1, 0 or 1 meaning Default, Low, Normal, High)
# 6 : Size of the download (in bytes)
# 7 : Group list (separated by spaces)
# 8 : Show name
# 9 : Season (1..99)
# 10 : Episode (1..99)
# 11: Episode name



##################
### ACCEPTANCE ###
##################

### Refuse or Accept script in queue
### 0=Refuse, 1=Accept
echo 1


##########################
### NZB NAME HANDLING ####
##########################

### Name of the NZB (no path, no ".nzb")
### Clean up unwanted words in nzbnames
case $3 in
   movies)
      # cleanup _ and everything after (year)
      nzb=$(echo $1 | sed -e '
               s/_/ /
               s/).*/)/
            ' )
      ;;
   *)
      # cleanup unwanted words
      nzb=(echo $1 | sed -e '
               s/.par2//
               s/.rar//
               s/.001//
               s/_/ /
            ' )
      ;;
esac


#####################
#### XBMC NOTIFY ####
#####################

### If you want to be notified on snatched downloads in XBMC, please specify your xbmc credentials:
### Note: curl needs to be installed for this (sudo apt-get install curl)
xbmc_notify () {
xbmc_host=xbmc # hostname or ipaddress
xbmc_port=8080
xbmc_user=xbmc # if user is not set delete '$xbmc_user:$pass@' in url below
xbmc_pass=xbmc # if password is not set delete ':$pass@' in url below

if which curl > /dev/null
   then
   snatched=$(echo $nzb | sed -e 's/ /%20/g')
   curl -s "http://$xbmc_user:$xbmc_pass@$xbmc_host:$xbmc_port/xbmcCmds/xbmcHttp/?command=ExecBuiltIn(Notification(Snatched,$snatched))" | xargs echo > /dev/null
fi
}
xbmc_notify # comment this line if you do not want to use xbmc-notify.


#####################
#### PP HANDLING ####
#####################

# PP (0, 1, 2 or 3)
echo $2


###########################
#### CATEGORY HANDLING ####
###########################

### Set category if no category is specified
if [ -n $3 ]
   then
   case $7 in
      *movie*)
         echo movies
         ;;
      *book*)
         echo ebooks
         ;;
      *teevee*)
         echo tv
         ;;
      *)
         echo $3
         ;;
   esac
else
   echo $3
fi



#########################
#### SCRIPT HANDLING ####
#########################

### Use this to set a script (No Path, just basename, e.g. sabToSickbeard.py)
echo $4


##########################
### PRIORITY HANDLING ####
##########################

### Priority (-100 -2, -1, 0 or 1, meaning Default, Paused, Low, Normal, High )

### Set priority X if it's a file from category X
case $3 in
   tv)   # <- change this to your Sickbeard category
      priority="1"
      ;;
   couchpotato)
      priority="1"
      ;;
   *)
      priority="$5"
      ;;
esac

### Set a Size treshold for priority in bytes
### To calculate see http://easycalculation.com/bandwidth-calculator.php (9663676416 bytes = 9 GB)
maxsize="9663676416" # <- change the number for a maximum filesize
if [ "$6" -gt "$maxsize" ]
   then
   priority="-1"
fi

echo $priority


########################
#### GROUP HANDLING ####
########################

### Group to be used (in case your provider doesn't carry all groups and there are multiple groups in the NZB)
echo $7

User avatar
Mar2zz
Jr. Member
Jr. Member
 
Posts: 85
Joined: February 4th, 2011, 9:30 am

Re: [BASH] Pre-processing script (or queue-filter scripts)

Postby Mar2zz » September 24th, 2011, 2:04 pm

Prequeue prowl (notify iOS when a nzb is snatched):
Code: Select all
######################
#### PROWL NOTIFY ####
######################

### If you want to be notified on snatched downloads with Prowl, please specify your prowl credentials.

nzb=$1

### Priority options
# Very Low   -2
# Moderate   -1
# Normal    0
# High       1
# Emergency    2

### EDIT VARIABLES
prowl_api='xxxxxxxxxxxxxxxxx'      # apikey goes here
prowl_prio='0'                  # priority goes here, see list for options
prowl_title='Added%20to%20queue'   # jobtitle goes here, replace invalid html-characters with %20 (which shows as a space)

prowl_notify () {
http_nzbname=$(echo $nzb | sed "s/ /%20/g")
if ! wget -q --delete-after "https://prowl.weks.net/publicapi/add?apikey=$prowl_api&priority=$prowl_prio&application=Sabnzbd&event=$prowl_title&description=$http_nzbname") | xargs echo > /dev/null
   then
   echo "Error in Prowl-notification; $http_jobname @ https://prowl.weks.net/publicapi/add?apikey=$prowl_api failed!" > /dev/null
fi
}
prowl_notify   # comment this line if you do not want to use prowl-notify.
User avatar
Mar2zz
Jr. Member
Jr. Member
 
Posts: 85
Joined: February 4th, 2011, 9:30 am

Re: [BASH] Pre-processing script (or queue-filter scripts)

Postby hrtrulz » October 14th, 2011, 6:55 am

avalonai2 wrote:Is there a script like this for windows? This looks like a fantastic feature.


I spent some time working out how to do this in windows. Below is the code I came up with.
You need to name the file with '.cmd' and save into your scripts folder.

Code: Select all
@echo off
set priority=-100
if %3==tv set priority=-2

echo 1
echo %1
echo %2
echo.
echo.
echo %priority%
hrtrulz
Newbie
Newbie
 
Posts: 2
Joined: July 25th, 2011, 10:03 am

Next

Return to Post-Processing Scripts