Repair failed, not enough repair blocks nearly everytime

Get help with all aspects of SABnzbd
Forum rules
Help us help you:
  • Are you using the latest stable version of SABnzbd? Downloads page.
  • Tell us what system you run SABnzbd on.
  • Adhere to the forum rules.
  • Do you experience problems during downloading?
    Check your connection in Status and Interface settings window.
    Use Test Server in Config > Servers.
    We will probably ask you to do a test using only basic settings.
  • Do you experience problems during repair or unpacking?
    Enable +Debug logging in the Status and Interface settings window and share the relevant parts of the log here using [ code ] sections.
User avatar
shypike
Administrator
Administrator
Posts: 19774
Joined: January 18th, 2008, 12:49 pm

Re: Repair failed, not enough repair blocks nearly everytime

Post by shypike »

I'm taking the easy way out, by just enabling extra parameters for non-Windows.
The user needs to enter -N by him/herself in Config->Switches->Extra par2 parameters.
Maybe later we can handle this automatically.

The whole handling of classic/non-classis/tbb is too convoluted right now.
I'm doing a minimal touch here. Enabling what should have worked any way.
User avatar
sander
Release Testers
Release Testers
Posts: 8811
Joined: January 22nd, 2008, 2:22 pm

Re: Repair failed, not enough repair blocks nearly everytime

Post by sander »

FYI: The updated git version is working for me; the download is OK thanks to the "-N".

Code: Select all

2016-01-09 20:18:49,386::DEBUG::[newsunpack:1199] Starting par2: ['/usr/local/bin/par2', 'r', '-N', '/home/sander/Downloads/incomplete/buck bunny - bad because of missing segments/Big.Buck.Bunny.2008.
2016-01-09 20:19:11,113::DEBUG::[newsunpack:1199] Starting par2: ['/usr/local/bin/par2', 'r', '-N', '/home/sander/Downloads/incomplete/buck bunny - bad because of missing segments/Big.Buck.Bunny.2008.
User avatar
safihre
Administrator
Administrator
Posts: 5338
Joined: April 30th, 2015, 7:35 am
Contact:

Re: Repair failed, not enough repair blocks nearly everytime

Post by safihre »

I understand correctly that this happens with the Par2 we recommend? The par2cmdline? But not the multi core version?

Is this check of supporting the - N not worthwhile since many people will use this version?
Side question: does this extra parameter make it slower?
If you like our support, check our special newsserver deal or donate at: https://sabnzbd.org/donate
User avatar
sander
Release Testers
Release Testers
Posts: 8811
Joined: January 22nd, 2008, 2:22 pm

Re: Repair failed, not enough repair blocks nearly everytime

Post by sander »

With 99% certainty:

Yes.
Yes. On Ubuntu par2 is in the package 'par2'. The source is indeed called par2cmdline: https://github.com/Parchive/par2cmdline ... ndline.cpp (See the -N option)
Multi core as in tbb? It seems to be at version 0.4 (I justed a question in that thread of jcfp), so it won't have the -N option

I think it is very worthwile seeing the difference it makes, but I also think releasing 0.8.0 is more important. For the time being people could gain experience with a manual "-N".
Slower or not: I don't think so, but I will do a test

EDIT:

Testresults on the same download, which is mildly damaged (so that both plain and -N will work):

Code: Select all

time par2 r *par2
real	1m8.053s
user	0m54.600s
sys	0m1.312s

Code: Select all

time par2 r -N *par2
real	0m29.815s
user	0m13.392s
sys	0m1.356s
So ... yes, -N has a time impact ... it's two times faster! :o :o

(But I'm still not sure if there any downsides. Not in time, but for example a plain par2repair works, but a par2repair -N does not?)
ALbino
Full Member
Full Member
Posts: 214
Joined: October 23rd, 2014, 12:28 am

Re: Repair failed, not enough repair blocks nearly everytime

Post by ALbino »

I'm confused on the Windows support. Has the Windows version always used -N and now you're just adding it to *nix? Or will we need to wait for the next beta for -N support in Windows as well?
User avatar
sander
Release Testers
Release Testers
Posts: 8811
Joined: January 22nd, 2008, 2:22 pm

Re: Repair failed, not enough repair blocks nearly everytime

Post by sander »

ALbino wrote:I'm confused on the Windows support. Has the Windows version always used -N and now you're just adding it to *nix? Or will we need to wait for the next beta for -N support in Windows as well?
I don't think so. But you can check your SAB-on-Windows logging: find the log line "Starting par2" and see if there is "-N" like I posted.

And get the NZB I created and posted, and download it with your SAB-on-Windows. If it fails you have no -N option. If you have the par2-tbb version, you can specify the -N option in SABnzbd, and test again.
ALbino
Full Member
Full Member
Posts: 214
Joined: October 23rd, 2014, 12:28 am

Re: Repair failed, not enough repair blocks nearly everytime

Post by ALbino »

FWIW, using a default 0.7.20 install on Windows 7 with no extra PAR2 parameters it downloaded, repaired, and extracted just fine, without running "-N". In case it's helpful, here's a pastebin from the Debug log:

http://pastebin.com/ypSpUmn8
User avatar
sander
Release Testers
Release Testers
Posts: 8811
Joined: January 22nd, 2008, 2:22 pm

Re: Repair failed, not enough repair blocks nearly everytime

Post by sander »

@ALbino:

Is that the par2.exe provided by SABnzbd-on-Windows. or is it your own par2.exe with TBB?

Code: Select all

2016-01-09 12:48:49,835::DEBUG::[newsunpack:987] Starting par2: ['C:\\Program Files (x86)\\SABnzbd\\win\\par2\\x64\\par2.exe', 'r', 'E:\\Downloads\\incomplete\
Concurrent processing utilises Intel Thread Building Blocks 2.0,
User avatar
sander
Release Testers
Release Testers
Posts: 8811
Joined: January 22nd, 2008, 2:22 pm

Re: Repair failed, not enough repair blocks nearly everytime

Post by sander »

I added some detection and logging (but no par2 option setting) to SABnzbd.py:

par2 does know -N

Code: Select all

2016-01-09 23:04:01,961::INFO::[SABnzbd:501] par2 binary... found (/usr/local/bin/par2)
2016-01-09 23:04:02,048::INFO::[SABnzbd:503] par2 binary does support option -N
2016-01-09 23:04:02,049::INFO::[SABnzbd:510] par2-classic binary... found (/usr/local/bin/par2)
2016-01-09 23:04:02,062::INFO::[SABnzbd:512] par2-classic binary does support option -N

par2 does NOT know -N:

Code: Select all

2016-01-09 22:53:33,191::INFO::[SABnzbd:501] par2 binary... found (/usr/bin/par2)
2016-01-09 22:53:33,204::INFO::[SABnzbd:505] par2 binary does not support option -N
2016-01-09 22:53:33,204::INFO::[SABnzbd:510] par2-classic binary... found (/usr/bin/par2)
2016-01-09 22:53:33,214::INFO::[SABnzbd:514] par2-classic binary does not support option -N
Code is on github in a separate branch:

Code: Select all

git clone https://github.com/sanderjo/sabnzbd.git
cd sabnzbd/
git branch -a
git checkout par2optionN
grep hasoption SABnzbd.py 
./SABnzbd.py 
ALbino
Full Member
Full Member
Posts: 214
Joined: October 23rd, 2014, 12:28 am

Re: Repair failed, not enough repair blocks nearly everytime

Post by ALbino »

sander wrote:@ALbino:

Is that the par2.exe provided by SABnzbd-on-Windows. or is it your own par2.exe with TBB?

Code: Select all

2016-01-09 12:48:49,835::DEBUG::[newsunpack:987] Starting par2: ['C:\\Program Files (x86)\\SABnzbd\\win\\par2\\x64\\par2.exe', 'r', 'E:\\Downloads\\incomplete\
Concurrent processing utilises Intel Thread Building Blocks 2.0,
That's the par2.exe that comes by default with the SABnzbd 0.7.20 Windows installer. So far as I can tell it doesn't even support the -N switch:

32bit version: http://pastebin.com/TJgXq0eP
64bit version: http://pastebin.com/tf84VbCS
SigHunter
Newbie
Newbie
Posts: 5
Joined: January 8th, 2016, 4:45 am

Re: Repair failed, not enough repair blocks nearly everytime

Post by SigHunter »

thanks so much for all your investigating and testing,
I really appreciate it!

so I'll wait till the next beta hits Ubuntu 15.10 repositories and then fire on my custom par2 parameter -N
User avatar
sander
Release Testers
Release Testers
Posts: 8811
Joined: January 22nd, 2008, 2:22 pm

Re: Repair failed, not enough repair blocks nearly everytime

Post by sander »

@SigHunter and others:

I created a SABnzbd that automagically uses "-N" if the par2 knows it. I would appreaciate it if you could test it:

Stop SABnzbd.
Then:

Code: Select all

git clone https://github.com/sanderjo/sabnzbd.git
cd sabnzbd
git branch  -a
git checkout par2optionN

grep -irn hasoption *

./SABnzbd.py
The "grep" should show a few lines (4 to be exact)

Do a download and check if it works.
Some logging:

Code: Select all

2016-01-11 18:45:19,505::INFO::[SABnzbd:491] par2 binary... found (/usr/local/bin/par2)
2016-01-11 18:45:19,561::INFO::[SABnzbd:493] par2 binary does support option -N
2016-01-11 18:45:19,566::INFO::[SABnzbd:500] par2-classic binary... found (/usr/local/bin/par2)
2016-01-11 18:45:19,622::INFO::[SABnzbd:502] par2-classic binary does support option -N
and (debug) logging:

Code: Select all

2016-01-11 20:17:20,597::DEBUG::[newsunpack:1191] SJ: command is ['/usr/local/bin/par2', 'r', u'/home/pi/Downloads/incomplete/Big.Buck.Bunny%20---%20missing%20segemnts/Big.Buck.Bunny.2008.1080p.x264-60fps.par2']
2016-01-11 20:17:20,711::DEBUG::[newsunpack:1193] SJ: Yes, par2 has option -N
2016-01-11 20:17:20,719::DEBUG::[newsunpack:1196] SJ: command is ['/usr/local/bin/par2', 'r', '-N', u'/home/pi/Downloads/incomplete/Big.Buck.Bunny%20---%20missing%20segemnts/Big.Buck.Bunny.2008.1080p.x264-60fps.par2']
2016-01-11 20:17:20,746::DEBUG::[newsunpack:1215] Starting par2: ['/usr/local/bin/par2', 'r', '-N', '/home/pi/Downloads/incomplete/Big.Buck.Bunny%20---%20missing%20segemnts/Big.Buck.Bunny.2008.1080p.x264-60fps.par2',
User avatar
shypike
Administrator
Administrator
Posts: 19774
Joined: January 18th, 2008, 12:49 pm

Re: Repair failed, not enough repair blocks nearly everytime

Post by shypike »

For 0.8.0 we're sticking with the current solution (set the extra parameter manually).
The par2 situation needs to be thoroughly researched for multiple platforms.
SigHunter
Newbie
Newbie
Posts: 5
Joined: January 8th, 2016, 4:45 am

Re: Repair failed, not enough repair blocks nearly everytime

Post by SigHunter »

@sander,
I will test it this evening
SigHunter
Newbie
Newbie
Posts: 5
Joined: January 8th, 2016, 4:45 am

Re: Repair failed, not enough repair blocks nearly everytime

Post by SigHunter »

seems as if it works! my problematic file repaired without a problem, thanks sander!

Code: Select all

2016-01-12 23:41:49,416::INFO::[newsunpack:1288] Verified in 6 seconds, repair is required
2016-01-12 23:41:53,372::INFO::[newsunpack:1420] Repaired in 3 seconds
2016-01-12 23:41:53,372::INFO::[newsunpack:1069] Par verify finished ok on x.par2
2016-01-12 23:41:53,373::INFO::[newsunpack:1099] Deleting x
2016-01-12 23:41:53,394::INFO::[newsunpack:1099] Deleting x
2016-01-12 23:41:53,420::INFO::[newsunpack:1109] Deleting xpar2
2016-01-12 23:41:53,420::INFO::[newsunpack:1138] Deleting xvol02+02.par2
2016-01-12 23:41:53,421::INFO::[newsunpack:1138] Deleting xvol01+01.par2
2016-01-12 23:41:53,421::INFO::[newsunpack:1138] Deleting xvol00+01.par2
2016-01-12 23:41:53,421::INFO::[postproc:626] Par2 check finished on x
2016-01-12 23:41:53,422::INFO::[misc:793] Creating directories: /mnt/storage/home/sighunter/Downloads/complete/x
2016-01-12 23:41:53,423::INFO::[postproc:352] Running unpack_magic on x
2016-01-12 23:41:53,423::INFO::[newsunpack:229] Unrar starting on /mnt/storage/home/sighunter/Downloads/incomplete/x
2016-01-12 23:42:01,675::INFO::[newsunpack:763] Unpacked 2 files/folders in 8 seconds
Post Reply