Intentionally misnamed rar sets unusable with Sabnzbd?

[dm8000s]

Hello,

I was conducting a little test with intentionally misnamed rar files. Quickpar will auto-recognize them and will rename/repair whenever needed. Not so with the par2.exe included with Sabnzbd.

Here's what quickpar does. Very much wanted:




Included with Sabnzbd are commandline versions of the par2 tool that seem to date back to the original 2003-07-15 (!!!!!) version from Peter Brian Clements (only treated with Intel Threading Building Blocks):

Code: Select all

[newsunpack:1211] PAR2 output was
par2cmdline version 0.4, Copyright (C) 2003 Peter Brian Clements.
par2cmdline comes with ABSOLUTELY NO WARRANTY.
This is free software, and you are welcome to redistribute it and/or modify
it under the terms of the GNU General Public License as published by the
Free Software Foundation; either version 2 of the License, or (at your
option) any later version. See COPYING for details.
Loading "test.par2".
Loading: 2.3%
Loading: 17.6%
Loading: 19.9%
Loading: 35.3%
Loading: 37.6%
Loading: 53.0%
Loading: 55.3%
Loading: 70.6%
Loading: 72.9%
Loading: 88.3%
Loading: 90.6%
Loading: 95.7%
Loading: 98.7%
Loaded 14 new packets
There are 6 recoverable files and 0 other files.
The block size used was 384000 bytes.
There are a total of 216 data blocks.
The total size of the data files is 82591272 bytes.
Verifying source files:
Target: "test.part1.rar" - missing.
Target: "test.part2.rar" - missing.
Target: "test.part3.rar" - missing.
Target: "test.part4.rar" - missing.
Target: "test.part5.rar" - missing.
Target: "test.part6.rar" - missing.
Scanning extra files:
Repair is required.
6 file(s) are missing.
You have 0 out of 216 data blocks available.
Repair is not possible.
You need 216 more recovery blocks to be able to repair.

As can be seen from the debug output above: There is not even an attempt to check if the rar archives match anything related to the par2 blocks. I've tried all options in the 'switches' section. No succes.

Also: I can not find any commandline switch that may force something like quickpar does. Not in the original, not in the tbb rebuild.

Have I missed some options? Should I use another par commandline tool?

Is the following not related to this problem (I read it like it is):

What's new in SABnzbd 0.7.6 Beta 1:
November 10th, 2012

Features:
· Properly handle par2-sets that were renamed after creation by the poster


I use SABnzb 0.7.9.


?

[shypike]

We're working on a solution to be included in the next release.

[dm8000s]

We're working on a solution to be included in the next release.

Thanks for the answer. Sounds good!

[iDragon]

A very quick and dirty fix is to run par2 repair with * at the end of the command line.

[Tomeor]

Hello,

This is a quick fix to solve this problem of file renaming in sabnzbd. The objective is to make sabnzbd call par2 cmd with a "*" at the end of the command line.

So you have to edit the file newsunpack.py (under ubuntu it is in /usr/share/sabnzbdplus/sabnzbd ) and modify lines 871 & 873 (on 0.6. in PAR_Verify function to add the ", nzo.downpath+'/*'" part at the end of the brackets :

Code: Select all

...
if (is_new_partype(nzo, setname) and not classic) or not PAR2C_COMMAND:
        if cfg.par_option():
            command = [str(PAR2_COMMAND), cmd, str(cfg.par_option().strip()), parfile, nzo.downpath+'/*'] # <-- HERE
        else:
            command = [str(PAR2_COMMAND), cmd, parfile, nzo.downpath+'/*'] # <-- AND HERE
        classic = not PAR2C_COMMAND
    else:
...

I tried it on a few movies and it works fine.

PS : I made this patch on 0.6.8 but i guess it should work on later release also.
Regards
Tom

[shypike]

It may work most of the time, but it's not the way we're going to implement it.
First of all, it will cause multiple verification runs after a repair.
Second: it will become even worse when you have multi-set NZBs.
So again: it will work, but it may have side-effects.

[ptr727]

@shypike, can you give any indication of when this will be released?

[shypike]

Halfway next week.

[dm8000s]

Thanks for the update in version 0.7.10!

Works great now. But only with archives that do *not* have a password.
If I add an archive with a password it correctly gets renamed after the download,but sabnzbd wants to use the password on the archive's as they were named *before* sabnzbd correctly renamed them!
But since those archives are renamed now, sabnzbsd quits with an error saying "can't find insertname.rar".

Maybe that is something that can be fixed in the next update?

[shypike]

Maybe that is something that can be fixed in the next update?

Not likely, I have no examples for that, unless you email an NZB and a matching password to [email protected]
One work-around is to use a password file, like described here: http://wiki.sabnzbd.org/password-protected-rars

[dm8000s]

Maybe that is something that can be fixed in the next update?

Not likely, I have no examples for that, unless you email an NZB and a matching password to [email protected]
One work-around is to use a password file, like described here: http://wiki.sabnzbd.org/password-protected-rars

I was ready to email a nzb but decided to doublecheck some things first. The error was completely on my side (I had thrown away a nzb I used to test, and tried to recreate it through binsearch - and mixed up some rar's that didn't belong to the original rar set. So the password -of course- didn't work there).

Processing renamed files with a password works as intended with 0.7.10. (and I indeed use the nzb naming the wiki suggests).

Thanks!

[E71]

I don't mean to hijack this thread but I'm wondering if my problem is related.

Certain jobs, namely the ones with obfuscated filenames get stuck in verification stage, holding up the queue and according to 'top' par2cmdline takes up a lot of CPU.

I'm on 0.7.10 now (on CentOS) but the issue persists.

Do I need to create a new thread?

[shypike]

That problem has been located.
Due to a programming error, for NZBs containing mutliple rar/par sets
each set will process ALL available files instead of the just the ones belonging to the current set.
This takes a long time on slow systems.
Solution is near.

[E71]

Thanks. That's great news.

[scavenger]

most of the time, misnamed rar files are like "filename.[0-9]*"
using this trick here is the command i use to guess the files to pass to par2 command line, including multisets. this is a command line tool for a script or a function under the shell, but it's ready to be translated as python i guess :

Code: Select all

par2Repair ()
{
filesets=$(ls *.par2 | cut -d\. -f1 | uniq)
echo "Now reparing these fileset : ${filesets}"

for fileset in ${filesets};
do
	echo "Now reparing fileset : ${fileset}"
	par2 r ${fileset}.par2 ${fileset}.r* ${fileset}.[0-9]*
done
}