[BASH] How to create your own postprocessingscript

Come up with a useful post-processing script? Share it here!

[BASH] How to create your own postprocessingscript

Postby Mar2zz » February 9th, 2011, 9:11 am

Bash is a flexible language and not that hard to write. Just open a blank textfile and write a shebang in the first line: #!/bin/bash (or #!/usr/bin/env bash) and commands on the next.
When you want to explain something in the script, or stop a command from executing, put a # in front of it, it disables all text after that line that line specifically.


# Commands
Basically it's just commands you would enter normally in the terminal and write them down in the script.
Sab serves some things to a script so you can use them to execute the commands with, listed here: http://wiki.sabnzbd.org/user-scripts . In Bash you can use these by naming them $1 to $7. Copying a downloaded folder ($1) is easy that way, open a textfile, shebang at top, enter cp $1 /new/location. Save it make it executable, and there you have it, your first postprocessingscript.

# Variables
As above Sabnzbd executes a script and gives it 7 variables ($1 to $7). You can create your own variables too, or rename the variables Sabnzbd passes to the script. Variables are made by word=something (= is used to define a variable) and then callable by using a $ sign for it, like $word.
Some examples:
name=mars ($name) >> output your name in terminal or log >usage: echo "Hi my name is $name"
location=/some/folder/on/my/nas ($location) >> copy a file to that folder on nas > usage: cp $1/*.avi $location
cleannzb=$3 ($cleannzb) >> output that name in terminal or log > usage: echo "$name copied $1 to $location, job $cleannzb is done"

Variables can be a command to, use the command inside $(). actioncopy=$(cp *.avi /home/user/Videos) $actioncopy will execute it.
Using variables makes a script flexible and editable, change a variable and everywhere it's used it will be changed too. Usually they are written before all functions.

# Functions
Using code from someone else is also easy. Just use functions. You can call them whatever you like and write 'm like this:
Code: Select all
nameoffunction () {
commands you want to execute
}

Make sure you put the functionname (without the '() {') also at the bottom of the script execute the functions. This makes it easy to run just a part of a script (comment functions to disable them) and easy for others to edit the script to specific needs or borrow functions from other scripts.

Now you want to combine two scripts that cleans up your downloadfolder and then copy's it to a specific directory and then tells (echo) you you are a great guy and then start a script someone else wrote. Search this forum for other scripts that do that, for example [url=http://"http://forums.sabnzbd.org/index.php?topic=5031.0"]this[/url] and [url=http://"http://forums.sabnzbd.org/index.php?topic=4748.0"]this[/url] one. Embed them in functions and the new script will be:
Code: Select all
!/bin/bash


#### clean up directory so execute cleanupscript
topic1 () {
GARBAGE=".nfo .srr .sfv .nzb .jpg" #### Add or remove extensions here

for junk in $GARBAGE
do
find $1 -name *$junk -type f -exec rm -f {} \;
done
}

#### the above function does the same as the more readable:
removefiles () {
rm -f $1/*.nfo
rm -f $1/*.srr
rm -f $1/*.sfv
rm -f $1/*.nzb
rm -f $1/*.jpg
}

#### copy to someplace if movies and another if tvshows copied from forum and embedded in function
topic2 () {
if [ "$5" = "movies" ]; then
  mv -fv "$1" "/Volumes/External HD/Media/Movies"
elif [ "$5" = "tv" ]; then
  mv -fv "$1" "/Volumes/External HD/Media/TV"
fi
}

#### Yes I wrote my own bashscript
tellme () {
echo "I love Linux and Sabnzbd!"
}


#### execute all the above function, except the first
#topic1       #because this one is commented it won't execute.
removefiles      # you can test one function of your script by commenting all the other functions and then run the script.
topic2
tellme



# More difficult functions
Bash has many ways to check for conditions. Those are called statements. For example if-statements are most used in script.
If statements condition like "if this then do x else do y fi". A simple example:
Code: Select all
if $name=mars
   then
   echo "hi $name" #this can be a command too, or multiple commands like this
        rm -f /home/mars/*.rar
        echo "Oh noes, I accidentally 98mb of rar-files, is that bad?
else
   echo  "hi i don't know your name"
fi


As you see if, else and fi are on the left and the things in between a tab (or 4 spaces) to the right. This is for readability, and most of the time not really necessary. Google will help if you need more difficult functions to do something, also see the sources below.


# Running multiple scripts from 1 script
This is also easy. You can do 2 things. First the easiest:
1. Blank textfile, shebang @ top and the following lines in the script:
./path/toscript/one/scriptname1
./path/toscript/two/scriptname2 &&
./and/so/on.
2. Copy all text in all scripts, put the code from script one in function1, script two in function 2 etc. (watch out with variables here by the way).

If you put && after an executescriptcommand the next command will only be started if the previous one was succesful (see example scriptname2).


# Great sources for bashwriting are:
http://mywiki.wooledge.org/EnglishFrontPage
http://www.faqs.org/docs/Linux-HOWTO/Ba ... HOWTO.html
and google of course.


The basic thing is:
If you repeatedly do the same stuff on your linux-pc, over and over again, then write a bashscript for it. Throw all commands you use in a file, shebang at top and functions at bottom and chmod +x it. It will save you a lot of time (after you invest some first) and you will learn a lot about bash and linux. You can make Sabnzbd do fun stuff and make nice logs.
for example a script with:
echo "Hi $name, I did this job called $3 for you, as you feeded me $2, I moved it to $1 for you, hope you will enjoy this $5."
(You can log this into a real file with > /home/user/filename.txt at the end).

Have fun scripting!  ;)

Forgot one thing:
It's always a good idea to use #/bin/bash -x as a shebang when writing scripts from scratch. This will enter debugmode and the output in terminal will be much more verbose, so it will be easier to find mistakes.
Last edited by Mar2zz on February 9th, 2011, 10:16 am, edited 1 time in total.
User avatar
Mar2zz
Jr. Member
Jr. Member
 
Posts: 85
Joined: February 4th, 2011, 8:30 am

Re: [BASH] How to create your own postprocessingscript

Postby inpheaux » February 9th, 2011, 12:45 pm

I stuck your thread, looks like a great starting place!
User avatar
inpheaux
Administrator
Administrator
 
Posts: 729
Joined: January 16th, 2008, 9:14 pm

Re: [BASH] How to create your own postprocessingscript

Postby john3voltas » February 9th, 2011, 6:59 pm

Damn! Nice job!!
SABnzbd 0.6.0Alpha11 on Fedora 14-64bit laptop.
Usenet-News, TeraNews, newszilla6.xs4all.nl and reader.ipv6.xsnews.nl.
IPv6 connections powered by Hurricane Electric.
Can pull 30Mbit nntp on a 30Mbit FTTH link.
User avatar
john3voltas
Release Testers
Release Testers
 
Posts: 143
Joined: January 17th, 2008, 5:35 pm
Location: Lisbon/Portugal

Re: [BASH] How to create your own postprocessingscript

Postby Orbi » February 23rd, 2011, 4:30 am

Very informative for programming amateurs. Thanks for this!
Orbi
Newbie
Newbie
 
Posts: 16
Joined: February 26th, 2010, 3:34 am

Re: [BASH] How to create your own postprocessingscript

Postby TheStretchedElf » March 22nd, 2011, 1:03 pm

Does this guide cover Windows?
TheStretchedElf
Newbie
Newbie
 
Posts: 10
Joined: March 14th, 2011, 3:40 pm

Re: [BASH] How to create your own postprocessingscript

Postby Chanch » July 25th, 2011, 9:24 pm

I am trying to write a script that will move certain files from ($1), the destination folder, to another folder specified relative to $1.
This is what I have tried:
Code: Select all
find "$1" \( -name "*.avi" -or -name "*.mkv" -or -name "*.mp4" -or -name "*.mov" -or -name "*.mpeg" -or -name "*.mpg" -or -name "*.wmv" -or -name "*.m4v" \) -exec mv -fv '{}' "$1"/../proc \; 
rm -r "$1"

my problem is with the
Code: Select all
"$1"/../proc
, I'm trying to go up one level from $1's path and to the folder "proc"
For example if $1= Volumes/Movie/temp, I'm trying to describe Volumes/Movie/proc
I can't explicitly name the folder because, it will change if I change the destination folder in Sab's configuration

Any comments and suggestions are appreciated
Chanch
Newbie
Newbie
 
Posts: 5
Joined: July 25th, 2011, 8:22 pm

Re: [BASH] How to create your own postprocessingscript

Postby sander » July 26th, 2011, 1:37 am

(Disclaimer: I hate shell programming when it comes to quotes, double quotes, wildcards and escaping them)

Does this work in the script:

ls $1/../proc

(So, without double-quotes).

If so, things to consider:
- no double-quotes in the find / mv
- or, first cd to $1/../proc, then the find & mv
User avatar
sander
Release Testers
Release Testers
 
Posts: 5875
Joined: January 22nd, 2008, 2:22 pm

Re: [BASH] How to create your own postprocessingscript

Postby Chanch » July 26th, 2011, 10:48 am

Thanks for your reply
Code: Select all
ls $1/../proc

this didn't work, I ended up doing this :
Code: Select all
final=$(echo "$1" | sed 's-temp.*-proc-')
and
Code: Select all
find "$1" \( -name "*.avi" -or -name "*.mkv" -or -name "*.mp4" -or -name "*.mov" -or -name "*.mpeg" -or -name "*.mpg" -or -name "*.wmv" -or -name "*.m4v" \) -exec mv -fv '{}' "$final" \;


it works but, I'm sure there must be a much simpler way
Chanch
Newbie
Newbie
 
Posts: 5
Joined: July 25th, 2011, 8:22 pm

Re: [BASH] How to create your own postprocessingscript

Postby starsys » July 27th, 2011, 2:05 pm

Hello and thanks for your work.

I need a simple script to index my file on a synology NAS after a successfull download.
I found this :
viewtopic.php?f=9&t=6913&p=52390#p52390
and
viewtopic.php?f=2&t=2559&p=52389#p52389

Can anyone tell me if it works ? What should be name of the script file (with which extension ?)

Thanks a lot in advance.
starsys
Newbie
Newbie
 
Posts: 33
Joined: July 26th, 2011, 4:17 am

Re: [BASH] How to create your own postprocessingscript

Postby Mar2zz » August 9th, 2011, 2:23 pm

Chanch wrote:Thanks for your reply
Code: Select all
ls $1/../proc

this didn't work, I ended up doing this :
Code: Select all
final=$(echo "$1" | sed 's-temp.*-proc-')
and
Code: Select all
find "$1" \( -name "*.avi" -or -name "*.mkv" -or -name "*.mp4" -or -name "*.mov" -or -name "*.mpeg" -or -name "*.mpg" -or -name "*.wmv" -or -name "*.m4v" \) -exec mv -fv '{}' "$final" \;


it works but, I'm sure there must be a much simpler way


The only way I can think of is doing it easier is a "for" loop.

Code: Select all
final=$(echo "$1" | sed 's-temp.*-proc-')
movies="avi,mkv,mp4,mov,mpeg,mpg,wmv,m4v"
for video in $movies
do
find "$1" -name "*.$video" -exec mv -fv '{}' "$final" \;
done
User avatar
Mar2zz
Jr. Member
Jr. Member
 
Posts: 85
Joined: February 4th, 2011, 8:30 am

Re: [BASH] How to create your own postprocessingscript

Postby Mikie » November 6th, 2011, 9:30 am

Quick question. Do scripts run before or after the file is moved to a folder by category? I know I could just use the final directory of the job and eliminate the need to know this but for interest sake.

Thanks
Mikie
Newbie
Newbie
 
Posts: 10
Joined: October 21st, 2011, 10:46 am

Re: [BASH] How to create your own postprocessingscript

Postby Mar2zz » November 24th, 2011, 1:27 am

I guess you allready know, but sabnzbd saves to category-folder first, then it will run your script.
User avatar
Mar2zz
Jr. Member
Jr. Member
 
Posts: 85
Joined: February 4th, 2011, 8:30 am

Re: [BASH] How to create your own postprocessingscript

Postby tnowakow » December 7th, 2011, 11:28 am

Hi guys, this is a great thread so thank you! I'm trying to write a simple script where it will unpack an iso file for me. Here is my code;

#!/bin/sh
extract-xiso -xsd $1 $1/*.iso

Problem is that the error I got is:
open error: of No such file or directory

extract-xiso is in my $PATH and running it works, but I think this has something to do with it not going into the correct path where the iso was extracted to? Is there a command line I can force sabnzbd to run my script so I can test it to see what path the variables are using? (specifically $1) Thanks!
tnowakow
Newbie
Newbie
 
Posts: 4
Joined: December 7th, 2011, 11:23 am

Re: [BASH] How to create your own postprocessingscript

Postby Mar2zz » December 7th, 2011, 1:33 pm

On unix it's always best practice to quote paths, or else your shell will read until the first space or special character, cuts of the variable right there and takes that as input.
Code: Select all
extract-xiso -xsd "$1" "$1/*.iso"
User avatar
Mar2zz
Jr. Member
Jr. Member
 
Posts: 85
Joined: February 4th, 2011, 8:30 am

Re: [BASH] How to create your own postprocessingscript

Postby tnowakow » December 7th, 2011, 6:23 pm

That was it, thank you! I ended up having to do this:

extract-xiso -xsd "$1" "$1/"*.iso

Now my next attempt will be to write a 'smarter' script where it will check the number of iso's, extract the first, if successful then extract the second, if successful delete both iso's. I think this will be a lot harder :) Thanks for the help though with the quotes!
tnowakow
Newbie
Newbie
 
Posts: 4
Joined: December 7th, 2011, 11:23 am

Next

Return to Post-Processing Scripts