Page 2 of 3
Re: Repair failed, not enough repair blocks nearly everytime
Posted: January 9th, 2016, 11:23 am
by shypike
I'm taking the easy way out, by just enabling extra parameters for non-Windows.
The user needs to enter -N by him/herself in Config->Switches->Extra par2 parameters.
Maybe later we can handle this automatically.
The whole handling of classic/non-classis/tbb is too convoluted right now.
I'm doing a minimal touch here. Enabling what should have worked any way.
Re: Repair failed, not enough repair blocks nearly everytime
Posted: January 9th, 2016, 2:32 pm
by sander
FYI: The updated git version is working for me; the download is OK thanks to the "-N".
Code: Select all
2016-01-09 20:18:49,386::DEBUG::[newsunpack:1199] Starting par2: ['/usr/local/bin/par2', 'r', '-N', '/home/sander/Downloads/incomplete/buck bunny - bad because of missing segments/Big.Buck.Bunny.2008.
2016-01-09 20:19:11,113::DEBUG::[newsunpack:1199] Starting par2: ['/usr/local/bin/par2', 'r', '-N', '/home/sander/Downloads/incomplete/buck bunny - bad because of missing segments/Big.Buck.Bunny.2008.
Re: Repair failed, not enough repair blocks nearly everytime
Posted: January 9th, 2016, 2:47 pm
by safihre
I understand correctly that this happens with the Par2 we recommend? The par2cmdline? But not the multi core version?
Is this check of supporting the - N not worthwhile since many people will use this version?
Side question: does this extra parameter make it slower?
Re: Repair failed, not enough repair blocks nearly everytime
Posted: January 9th, 2016, 3:15 pm
by sander
With 99% certainty:
Yes.
Yes. On Ubuntu par2 is in the package 'par2'. The source is indeed called par2cmdline:
https://github.com/Parchive/par2cmdline ... ndline.cpp (See the -N option)
Multi core as in tbb? It seems to be at version 0.4 (I justed a question in that thread of jcfp), so it won't have the -N option
I think it is very worthwile seeing the difference it makes, but I also think releasing 0.8.0 is more important. For the time being people could gain experience with a manual "-N".
Slower or not: I don't think so, but I will do a test
EDIT:
Testresults on the same download, which is mildly damaged (so that both plain and -N will work):
Code: Select all
time par2 r *par2
real 1m8.053s
user 0m54.600s
sys 0m1.312s
Code: Select all
time par2 r -N *par2
real 0m29.815s
user 0m13.392s
sys 0m1.356s
So ... yes, -N has a time impact ... it's two times faster!
(But I'm still not sure if there any downsides. Not in time, but for example a plain par2repair works, but a par2repair -N does not?)
Re: Repair failed, not enough repair blocks nearly everytime
Posted: January 9th, 2016, 3:19 pm
by ALbino
I'm confused on the Windows support. Has the Windows version always used -N and now you're just adding it to *nix? Or will we need to wait for the next beta for -N support in Windows as well?
Re: Repair failed, not enough repair blocks nearly everytime
Posted: January 9th, 2016, 3:35 pm
by sander
ALbino wrote:I'm confused on the Windows support. Has the Windows version always used -N and now you're just adding it to *nix? Or will we need to wait for the next beta for -N support in Windows as well?
I don't think so. But you can check your SAB-on-Windows logging: find the log line "Starting par2" and see if there is "-N" like I posted.
And get the NZB I created and posted, and download it with your SAB-on-Windows. If it fails you have no -N option. If you have the par2-tbb version, you can specify the -N option in SABnzbd, and test again.
Re: Repair failed, not enough repair blocks nearly everytime
Posted: January 9th, 2016, 3:58 pm
by ALbino
FWIW, using a default 0.7.20 install on Windows 7 with no extra PAR2 parameters it downloaded, repaired, and extracted just fine, without running "-N". In case it's helpful, here's a pastebin from the Debug log:
http://pastebin.com/ypSpUmn8
Re: Repair failed, not enough repair blocks nearly everytime
Posted: January 9th, 2016, 4:45 pm
by sander
@ALbino:
Is that the par2.exe provided by SABnzbd-on-Windows. or is it your own par2.exe with TBB?
Code: Select all
2016-01-09 12:48:49,835::DEBUG::[newsunpack:987] Starting par2: ['C:\\Program Files (x86)\\SABnzbd\\win\\par2\\x64\\par2.exe', 'r', 'E:\\Downloads\\incomplete\
Concurrent processing utilises Intel Thread Building Blocks 2.0,
Re: Repair failed, not enough repair blocks nearly everytime
Posted: January 9th, 2016, 5:09 pm
by sander
I added some detection and logging (but no par2 option setting) to SABnzbd.py:
par2 does know -N
Code: Select all
2016-01-09 23:04:01,961::INFO::[SABnzbd:501] par2 binary... found (/usr/local/bin/par2)
2016-01-09 23:04:02,048::INFO::[SABnzbd:503] par2 binary does support option -N
2016-01-09 23:04:02,049::INFO::[SABnzbd:510] par2-classic binary... found (/usr/local/bin/par2)
2016-01-09 23:04:02,062::INFO::[SABnzbd:512] par2-classic binary does support option -N
par2 does NOT know -N:
Code: Select all
2016-01-09 22:53:33,191::INFO::[SABnzbd:501] par2 binary... found (/usr/bin/par2)
2016-01-09 22:53:33,204::INFO::[SABnzbd:505] par2 binary does not support option -N
2016-01-09 22:53:33,204::INFO::[SABnzbd:510] par2-classic binary... found (/usr/bin/par2)
2016-01-09 22:53:33,214::INFO::[SABnzbd:514] par2-classic binary does not support option -N
Code is on github in a separate branch:
Code: Select all
git clone https://github.com/sanderjo/sabnzbd.git
cd sabnzbd/
git branch -a
git checkout par2optionN
grep hasoption SABnzbd.py
./SABnzbd.py
Re: Repair failed, not enough repair blocks nearly everytime
Posted: January 9th, 2016, 5:18 pm
by ALbino
sander wrote:@ALbino:
Is that the par2.exe provided by SABnzbd-on-Windows. or is it your own par2.exe with TBB?
Code: Select all
2016-01-09 12:48:49,835::DEBUG::[newsunpack:987] Starting par2: ['C:\\Program Files (x86)\\SABnzbd\\win\\par2\\x64\\par2.exe', 'r', 'E:\\Downloads\\incomplete\
Concurrent processing utilises Intel Thread Building Blocks 2.0,
That's the par2.exe that comes by default with the SABnzbd 0.7.20 Windows installer. So far as I can tell it doesn't even support the -N switch:
32bit version:
http://pastebin.com/TJgXq0eP
64bit version:
http://pastebin.com/tf84VbCS
Re: Repair failed, not enough repair blocks nearly everytime
Posted: January 11th, 2016, 4:08 am
by SigHunter
thanks so much for all your investigating and testing,
I really appreciate it!
so I'll wait till the next beta hits Ubuntu 15.10 repositories and then fire on my custom par2 parameter -N
Re: Repair failed, not enough repair blocks nearly everytime
Posted: January 11th, 2016, 3:01 pm
by sander
@SigHunter and others:
I created a SABnzbd that automagically uses "-N" if the par2 knows it. I would appreaciate it if you could test it:
Stop SABnzbd.
Then:
Code: Select all
git clone https://github.com/sanderjo/sabnzbd.git
cd sabnzbd
git branch -a
git checkout par2optionN
grep -irn hasoption *
./SABnzbd.py
The "grep" should show a few lines (4 to be exact)
Do a download and check if it works.
Some logging:
Code: Select all
2016-01-11 18:45:19,505::INFO::[SABnzbd:491] par2 binary... found (/usr/local/bin/par2)
2016-01-11 18:45:19,561::INFO::[SABnzbd:493] par2 binary does support option -N
2016-01-11 18:45:19,566::INFO::[SABnzbd:500] par2-classic binary... found (/usr/local/bin/par2)
2016-01-11 18:45:19,622::INFO::[SABnzbd:502] par2-classic binary does support option -N
and (debug) logging:
Code: Select all
2016-01-11 20:17:20,597::DEBUG::[newsunpack:1191] SJ: command is ['/usr/local/bin/par2', 'r', u'/home/pi/Downloads/incomplete/Big.Buck.Bunny%20---%20missing%20segemnts/Big.Buck.Bunny.2008.1080p.x264-60fps.par2']
2016-01-11 20:17:20,711::DEBUG::[newsunpack:1193] SJ: Yes, par2 has option -N
2016-01-11 20:17:20,719::DEBUG::[newsunpack:1196] SJ: command is ['/usr/local/bin/par2', 'r', '-N', u'/home/pi/Downloads/incomplete/Big.Buck.Bunny%20---%20missing%20segemnts/Big.Buck.Bunny.2008.1080p.x264-60fps.par2']
2016-01-11 20:17:20,746::DEBUG::[newsunpack:1215] Starting par2: ['/usr/local/bin/par2', 'r', '-N', '/home/pi/Downloads/incomplete/Big.Buck.Bunny%20---%20missing%20segemnts/Big.Buck.Bunny.2008.1080p.x264-60fps.par2',
Re: Repair failed, not enough repair blocks nearly everytime
Posted: January 11th, 2016, 3:25 pm
by shypike
For 0.8.0 we're sticking with the current solution (set the extra parameter manually).
The par2 situation needs to be thoroughly researched for multiple platforms.
Re: Repair failed, not enough repair blocks nearly everytime
Posted: January 12th, 2016, 9:19 am
by SigHunter
@sander,
I will test it this evening
Re: Repair failed, not enough repair blocks nearly everytime
Posted: January 12th, 2016, 5:46 pm
by SigHunter
seems as if it works! my problematic file repaired without a problem, thanks sander!
Code: Select all
2016-01-12 23:41:49,416::INFO::[newsunpack:1288] Verified in 6 seconds, repair is required
2016-01-12 23:41:53,372::INFO::[newsunpack:1420] Repaired in 3 seconds
2016-01-12 23:41:53,372::INFO::[newsunpack:1069] Par verify finished ok on x.par2
2016-01-12 23:41:53,373::INFO::[newsunpack:1099] Deleting x
2016-01-12 23:41:53,394::INFO::[newsunpack:1099] Deleting x
2016-01-12 23:41:53,420::INFO::[newsunpack:1109] Deleting xpar2
2016-01-12 23:41:53,420::INFO::[newsunpack:1138] Deleting xvol02+02.par2
2016-01-12 23:41:53,421::INFO::[newsunpack:1138] Deleting xvol01+01.par2
2016-01-12 23:41:53,421::INFO::[newsunpack:1138] Deleting xvol00+01.par2
2016-01-12 23:41:53,421::INFO::[postproc:626] Par2 check finished on x
2016-01-12 23:41:53,422::INFO::[misc:793] Creating directories: /mnt/storage/home/sighunter/Downloads/complete/x
2016-01-12 23:41:53,423::INFO::[postproc:352] Running unpack_magic on x
2016-01-12 23:41:53,423::INFO::[newsunpack:229] Unrar starting on /mnt/storage/home/sighunter/Downloads/incomplete/x
2016-01-12 23:42:01,675::INFO::[newsunpack:763] Unpacked 2 files/folders in 8 seconds