One post processing job holds up all others

Get help with all aspects of SABnzbd
Forum rules
Help us help you:
  • Are you using the latest stable version of SABnzbd? Downloads page.
  • Tell us what system you run SABnzbd on.
  • Adhere to the forum rules.
  • Do you experience problems during downloading?
    Check your connection in Status and Interface settings window.
    Use Test Server in Config > Servers.
    We will probably ask you to do a test using only basic settings.
  • Do you experience problems during repair or unpacking?
    Enable +Debug logging in the Status and Interface settings window and share the relevant parts of the log here using [ code ] sections.
Post Reply
jrtwynam
Newbie
Newbie
Posts: 6
Joined: December 1st, 2018, 12:38 pm

One post processing job holds up all others

Post by jrtwynam »

Hi,

I recently wrote a post processing script that would clean up unnecessary files as well as reencode the downloaded video file using handbrake cli. The script works nicely, but of course reencoding takes a bunch of time. The problem I'm having now is that if I download something in a category that I have set to reencode, and then download something that doesn't get reencoded, the second download won't post process until the first one is done. The second one should take a matter of maybe 5 seconds to fully post process, while the first one could take an hour. What I want is for a reencode job to happily work away at reencoding, but also have sabnzbd continue to post process anything else that it finishes downloading while that reencode is happening.

I've set the priority of post processing to low, and I understand that handbrake can take a lot of CPU usage. But I would think that other post processing could be done simultaneously, albeit possibly a bit slower, rather than stopping altogether.

Any ideas?

Thanks.
User avatar
OneCD
Hero Member
Hero Member
Posts: 557
Joined: March 4th, 2017, 3:47 pm

Re: One post processing job holds up all others

Post by OneCD »

Which OS?
Stuff I like: Apache bash cron DD-WRT Debian DNSMasq Entware FireFox GitHub ImageMagick Kate KDE LibreELEC Netrunner NFS NVIDIA OpenVPN Orvibo-S20 pfSense Python Raspberry-Pi RAID SABnzbd Transmission Usenet VirtualBox Watcher3 XFCE
jrtwynam
Newbie
Newbie
Posts: 6
Joined: December 1st, 2018, 12:38 pm

Re: One post processing job holds up all others

Post by jrtwynam »

I'm running it on windows 7.
User avatar
OneCD
Hero Member
Hero Member
Posts: 557
Joined: March 4th, 2017, 3:47 pm

Re: One post processing job holds up all others

Post by OneCD »

Ah, sorry - cant help with Windows. :(
Stuff I like: Apache bash cron DD-WRT Debian DNSMasq Entware FireFox GitHub ImageMagick Kate KDE LibreELEC Netrunner NFS NVIDIA OpenVPN Orvibo-S20 pfSense Python Raspberry-Pi RAID SABnzbd Transmission Usenet VirtualBox Watcher3 XFCE
jrtwynam
Newbie
Newbie
Posts: 6
Joined: December 1st, 2018, 12:38 pm

Re: One post processing job holds up all others

Post by jrtwynam »

I should add that there's actually two scripts in play here. The first is my ProcessTV.bat file, which does the cleanup of unwanted files - this is the script that SABnzbd calls for post processing. Then, there's my Convert.bat file, which is called by ProcessTV.bat using this:

Code: Select all

call "%ConversionScript%" "%JobFullPath%"
Where ConversionScript is the full path to the script. From what I've read, using "start" instead of "call" might fix this particular problem because it'll start this script asynchronously, but it'd likely cause another problem. I don't want the job to be finished in SAB until the conversion is actually finished, because Sonarr and Radarr would then see the finished job and try to move the file while it's still converting.
jrtwynam
Newbie
Newbie
Posts: 6
Joined: December 1st, 2018, 12:38 pm

Re: One post processing job holds up all others

Post by jrtwynam »

OneCD wrote: May 11th, 2019, 5:39 pm Ah, sorry - cant help with Windows. :(
D'oh! I'm useless with other stuff like Linux. At one point, I was using a Synology DS516j as my download station and had it running SickRage, SAB, and CouchPotato. One of the reasons I switched back to Windows was that it was always a challenge figuring out how to do something on that, that I would know how to do quite easily with Windows. Not saying that Windows is better or anything, it's just what I'm used to and have much more experience with. Another reason I switched was that the Synology was not anywhere near powerful enough for all the tasks I was throwing at it.
jrtwynam
Newbie
Newbie
Posts: 6
Joined: December 1st, 2018, 12:38 pm

Re: One post processing job holds up all others

Post by jrtwynam »

And to add more complexity to this...

Say I download two files that I have set to re-encode. I'd want SAB to start processing the first file once it's done downloading, but NOT start processing the second file until the first one is done. I figure I don't need my system to be converting two files at once, so in this case I'd like them to be queued. However, if I were to then download a third file that I don't have set to re-encode, I'd want SAB to process this one immediately while file 1 is being re-encoded and file 2 is in queue.

If there were maybe a "post process priority" value on the categories, this might help. So if a download completes with a category with a higher post process priority than one that's currently re-encoding, have SAB pause all system processes related to the re-encode (in particular, Handbrake), process the newly downloaded file, and then re-start all processes for the re-encode.

Maybe I'm asking too much. :)
User avatar
safihre
Administrator
Administrator
Posts: 5338
Joined: April 30th, 2015, 7:35 am
Contact:

Re: One post processing job holds up all others

Post by safihre »

Unfortuanly these things are not part of the program. On Linux you could start a new process and return right away. And there's tools like nice and ionice. But for windows.. I'm not sure!
If you like our support, check our special newsserver deal or donate at: https://sabnzbd.org/donate
User avatar
OneCD
Hero Member
Hero Member
Posts: 557
Joined: March 4th, 2017, 3:47 pm

Re: One post processing job holds up all others

Post by OneCD »

safihre wrote: May 12th, 2019, 6:48 am On Linux you could start a new process and return right away. And there's tools like nice and ionice. But for windows.. I'm not sure!
+1
Stuff I like: Apache bash cron DD-WRT Debian DNSMasq Entware FireFox GitHub ImageMagick Kate KDE LibreELEC Netrunner NFS NVIDIA OpenVPN Orvibo-S20 pfSense Python Raspberry-Pi RAID SABnzbd Transmission Usenet VirtualBox Watcher3 XFCE
Post Reply