Intentionally misnamed rar sets unusable with Sabnzbd?

Get help with all aspects of SABnzbd
Forum rules
Help us help you:
  • Are you using the latest stable version of SABnzbd? Downloads page.
  • Tell us what system you run SABnzbd on.
  • Adhere to the forum rules.
  • Do you experience problems during downloading?
    Check your connection in Status and Interface settings window.
    Use Test Server in Config > Servers.
    We will probably ask you to do a test using only basic settings.
  • Do you experience problems during repair or unpacking?
    Enable +Debug logging in the Status and Interface settings window and share the relevant parts of the log here using [ code ] sections.
dm8000s
Newbie
Newbie
Posts: 4
Joined: January 16th, 2013, 9:27 pm

Intentionally misnamed rar sets unusable with Sabnzbd?

Post by dm8000s »

Hello,

I was conducting a little test with intentionally misnamed rar files. Quickpar will auto-recognize them and will rename/repair whenever needed. Not so with the par2.exe included with Sabnzbd.

Here's what quickpar does. Very much wanted:

Image


Included with Sabnzbd are commandline versions of the par2 tool that seem to date back to the original 2003-07-15 (!!!!!) version from Peter Brian Clements (only treated with Intel Threading Building Blocks):

Code: Select all

[newsunpack:1211] PAR2 output was
par2cmdline version 0.4, Copyright (C) 2003 Peter Brian Clements.
par2cmdline comes with ABSOLUTELY NO WARRANTY.
This is free software, and you are welcome to redistribute it and/or modify
it under the terms of the GNU General Public License as published by the
Free Software Foundation; either version 2 of the License, or (at your
option) any later version. See COPYING for details.
Loading "test.par2".
Loading: 2.3%
Loading: 17.6%
Loading: 19.9%
Loading: 35.3%
Loading: 37.6%
Loading: 53.0%
Loading: 55.3%
Loading: 70.6%
Loading: 72.9%
Loading: 88.3%
Loading: 90.6%
Loading: 95.7%
Loading: 98.7%
Loaded 14 new packets
There are 6 recoverable files and 0 other files.
The block size used was 384000 bytes.
There are a total of 216 data blocks.
The total size of the data files is 82591272 bytes.
Verifying source files:
Target: "test.part1.rar" - missing.
Target: "test.part2.rar" - missing.
Target: "test.part3.rar" - missing.
Target: "test.part4.rar" - missing.
Target: "test.part5.rar" - missing.
Target: "test.part6.rar" - missing.
Scanning extra files:
Repair is required.
6 file(s) are missing.
You have 0 out of 216 data blocks available.
Repair is not possible.
You need 216 more recovery blocks to be able to repair.
As can be seen from the debug output above: There is not even an attempt to check if the rar archives match anything related to the par2 blocks. I've tried all options in the 'switches' section. No succes.

Also: I can not find any commandline switch that may force something like quickpar does. Not in the original, not in the tbb rebuild.

Have I missed some options? Should I use another par commandline tool?

Is the following not related to this problem (I read it like it is):

What's new in SABnzbd 0.7.6 Beta 1:
November 10th, 2012

Features:
· Properly handle par2-sets that were renamed after creation by the poster


I use SABnzb 0.7.9.


?

User avatar
shypike
Administrator
Administrator
Posts: 21043
Joined: January 18th, 2008, 12:49 pm

Re: Intentionally misnamed rar sets unusable with Sabnzbd?

Post by shypike »

We're working on a solution to be included in the next release.

dm8000s
Newbie
Newbie
Posts: 4
Joined: January 16th, 2013, 9:27 pm

Re: Intentionally misnamed rar sets unusable with Sabnzbd?

Post by dm8000s »

shypike wrote:We're working on a solution to be included in the next release.
Thanks for the answer. Sounds good! :)

iDragon
Newbie
Newbie
Posts: 4
Joined: September 24th, 2012, 1:12 pm

Re: Intentionally misnamed rar sets unusable with Sabnzbd?

Post by iDragon »

A very quick and dirty fix is to run par2 repair with * at the end of the command line.

Tomeor
Newbie
Newbie
Posts: 2
Joined: January 20th, 2013, 3:44 am

Re: Intentionally misnamed rar sets unusable with Sabnzbd?

Post by Tomeor »

Hello,

This is a quick fix to solve this problem of file renaming in sabnzbd. The objective is to make sabnzbd call par2 cmd with a "*" at the end of the command line.

So you have to edit the file newsunpack.py (under ubuntu it is in /usr/share/sabnzbdplus/sabnzbd ) and modify lines 871 & 873 (on 0.6.8) in PAR_Verify function to add the ", nzo.downpath+'/*'" part at the end of the brackets :

Code: Select all

...
if (is_new_partype(nzo, setname) and not classic) or not PAR2C_COMMAND:
        if cfg.par_option():
            command = [str(PAR2_COMMAND), cmd, str(cfg.par_option().strip()), parfile, nzo.downpath+'/*'] # <-- HERE
        else:
            command = [str(PAR2_COMMAND), cmd, parfile, nzo.downpath+'/*'] # <-- AND HERE
        classic = not PAR2C_COMMAND
    else:
...
I tried it on a few movies and it works fine.

PS : I made this patch on 0.6.8 but i guess it should work on later release also.
Regards
Tom

User avatar
shypike
Administrator
Administrator
Posts: 21043
Joined: January 18th, 2008, 12:49 pm

Re: Intentionally misnamed rar sets unusable with Sabnzbd?

Post by shypike »

It may work most of the time, but it's not the way we're going to implement it.
First of all, it will cause multiple verification runs after a repair.
Second: it will become even worse when you have multi-set NZBs.
So again: it will work, but it may have side-effects.

ptr727
Newbie
Newbie
Posts: 16
Joined: January 22nd, 2013, 10:57 pm

Re: Intentionally misnamed rar sets unusable with Sabnzbd?

Post by ptr727 »

@shypike, can you give any indication of when this will be released?

User avatar
shypike
Administrator
Administrator
Posts: 21043
Joined: January 18th, 2008, 12:49 pm

Re: Intentionally misnamed rar sets unusable with Sabnzbd?

Post by shypike »

Halfway next week.

dm8000s
Newbie
Newbie
Posts: 4
Joined: January 16th, 2013, 9:27 pm

Re: Intentionally misnamed rar sets unusable with Sabnzbd?

Post by dm8000s »

Thanks for the update in version 0.7.10!

Works great now. But only with archives that do *not* have a password.
If I add an archive with a password it correctly gets renamed after the download,but sabnzbd wants to use the password on the archive's as they were named *before* sabnzbd correctly renamed them!
But since those archives are renamed now, sabnzbsd quits with an error saying "can't find insertname.rar".

Maybe that is something that can be fixed in the next update? :)

User avatar
shypike
Administrator
Administrator
Posts: 21043
Joined: January 18th, 2008, 12:49 pm

Re: Intentionally misnamed rar sets unusable with Sabnzbd?

Post by shypike »

dm8000s wrote: Maybe that is something that can be fixed in the next update? :)
Not likely, I have no examples for that, unless you email an NZB and a matching password to [email protected]
One work-around is to use a password file, like described here: http://wiki.sabnzbd.org/password-protected-rars

dm8000s
Newbie
Newbie
Posts: 4
Joined: January 16th, 2013, 9:27 pm

Re: Intentionally misnamed rar sets unusable with Sabnzbd?

Post by dm8000s »

shypike wrote:
dm8000s wrote: Maybe that is something that can be fixed in the next update? :)
Not likely, I have no examples for that, unless you email an NZB and a matching password to [email protected]
One work-around is to use a password file, like described here: http://wiki.sabnzbd.org/password-protected-rars
I was ready to email a nzb but decided to doublecheck some things first. The error was completely on my side (I had thrown away a nzb I used to test, and tried to recreate it through binsearch - and mixed up some rar's that didn't belong to the original rar set. So the password -of course- didn't work there).

Processing renamed files with a password works as intended with 0.7.10. (and I indeed use the nzb naming the wiki suggests).

Thanks!

User avatar
E71
Newbie
Newbie
Posts: 42
Joined: January 13th, 2009, 10:08 pm

Re: Intentionally misnamed rar sets unusable with Sabnzbd?

Post by E71 »

I don't mean to hijack this thread but I'm wondering if my problem is related.

Certain jobs, namely the ones with obfuscated filenames get stuck in verification stage, holding up the queue and according to 'top' par2cmdline takes up a lot of CPU.

I'm on 0.7.10 now (on CentOS) but the issue persists.

Do I need to create a new thread?

User avatar
shypike
Administrator
Administrator
Posts: 21043
Joined: January 18th, 2008, 12:49 pm

Re: Intentionally misnamed rar sets unusable with Sabnzbd?

Post by shypike »

That problem has been located.
Due to a programming error, for NZBs containing mutliple rar/par sets
each set will process ALL available files instead of the just the ones belonging to the current set.
This takes a long time on slow systems.
Solution is near.

User avatar
E71
Newbie
Newbie
Posts: 42
Joined: January 13th, 2009, 10:08 pm

Re: Intentionally misnamed rar sets unusable with Sabnzbd?

Post by E71 »

Thanks. That's great news.

scavenger
Newbie
Newbie
Posts: 10
Joined: February 17th, 2012, 4:01 am

Re: Intentionally misnamed rar sets unusable with Sabnzbd?

Post by scavenger »

most of the time, misnamed rar files are like "filename.[0-9]*"
using this trick here is the command i use to guess the files to pass to par2 command line, including multisets. this is a command line tool for a script or a function under the shell, but it's ready to be translated as python i guess :

Code: Select all

par2Repair ()
{
filesets=$(ls *.par2 | cut -d\. -f1 | uniq)
echo "Now reparing these fileset : ${filesets}"

for fileset in ${filesets};
do
	echo "Now reparing fileset : ${fileset}"
	par2 r ${fileset}.par2 ${fileset}.r* ${fileset}.[0-9]*
done
}


Post Reply