Page 3 of 4
Re: Howto: multicore par2 on Debian, Ubuntu & derivatives
Posted: January 10th, 2016, 3:19 am
by an3k
Oh, it finally got fixed? I talked about the issue with shypike and offered my help more than six month ago but see below:
Summarizing: the 64bit and Multicore options make only sense for Windows and OSX where we can distribute the binaries for par2 and unrar.
All other platforms are too much out of our control.
Maybe it would make more sense if the user could enter an alternative name (for non-Win non-OSX).
Re: Howto: multicore par2 on Debian, Ubuntu & derivatives
Posted: January 10th, 2016, 5:16 am
by safihre
@shypyke, is it really not worth to do a check after the module-detection if this -t and/or -N parameters are supported and then update par2-options config accordingly?
It seems this is part of the main par2 modules we support and definitely bumps performance without much effort..
We don't have the luxury like NZBGet to compile par2 ourselves, this seems a good start
Re: Howto: multicore par2 on Debian, Ubuntu & derivatives
Posted: January 10th, 2016, 10:21 am
by jcfp
The chuchusoft releases are forked from the old (2004 or so) par2 0.4, primarily adding tbb support; so yes, that's 0.4. Any version of par2 greater than 0.4 is from (or based on) the github-fork-turned-official hosted at
https://github.com/Parchive/par2cmdline/ Their 0.6.8 release changed the way the program tries to find misplaced blocks, 0.6.13 added the -S and -N options. The latter re-enables the old file scanning behaviour found in pre-0.6.8, including any and all versions that identify themselves as 0.4.
The ones you have to be careful with are 0.6.8-0.6.12 (inclusive), they suffer from a bug that assumes the offset of a misplaced block is always positive (see
this bug report) and don't have any way to adjust or override the new bahaviour.
Re: Howto: multicore par2 on Debian, Ubuntu & derivatives
Posted: March 15th, 2016, 7:57 pm
by TioSolid
Im having trouble building the multicore par2 in the ARM architeture too. The thing is that the problem seems to be related to the par2 package itself (verificationhashtable.h).
Here is my output (from the last command in the tutorial in the first post of this thread):
Code: Select all
In file included from par2cmdline.h:615:0,
from par1repairersourcefile.cpp:20:
verificationhashtable.h:429:49: warning: suggest parentheses around '&&' within '||' [-Wparentheses]
checksummer.ShortBlock() && checksummer.BlockLength() != nextentry->GetDataBlock()->GetLength()
^
g++ -DHAVE_CONFIG_H -I. -D_FORTIFY_SOURCE=2 -Wall -I./../tbb43_20141023oss/include -std=c++11 -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -c -o par2creator.o par2creator.cpp
In file included from par2cmdline.h:615:0,
from par2creator.cpp:32:
verificationhashtable.h: In member function 'bool VerificationHashEntry::operator<(const VerificationHashEntry&) const':
verificationhashtable.h:69:40: warning: suggest parentheses around '&&' within '||' [-Wparentheses]
return crc < r.crc || crc == r.crc && hash < r.hash;
^
verificationhashtable.h: In member function 'bool VerificationHashEntry::operator>(const VerificationHashEntry&) const':
verificationhashtable.h:73:40: warning: suggest parentheses around '&&' within '||' [-Wparentheses]
return crc > r.crc || crc == r.crc && hash > r.hash;
^
verificationhashtable.h: In static member function 'static const VerificationHashEntry* VerificationHashEntry::Search(const VerificationHashEntry*, const MD5Hash&)':
verificationhashtable.h:186:47: warning: suggest parentheses around '&&' within '||' [-Wparentheses]
if (entry->crc < crc || entry->crc == crc && entry->hash < hash)
^
verificationhashtable.h:190:52: warning: suggest parentheses around '&&' within '||' [-Wparentheses]
else if (entry->crc > crc || entry->crc == crc && entry->hash > hash)
^
verificationhashtable.h: In member function 'const VerificationHashEntry* VerificationHashTable::FindMatch(const VerificationHashEntry*, const Par2RepairerSourceFile*, const FileCheckSummer&, bool&) const':
verificationhashtable.h:405:54: warning: suggest parentheses around '&&' within '||' [-Wparentheses]
checksummer.ShortBlock() && checksummer.BlockLength() != currententry->GetDataBlock()->GetLength()
^
verificationhashtable.h:412:56: warning: suggest parentheses around '&&' within '||' [-Wparentheses]
checksummer.ShortBlock() && checksummer.BlockLength() != currententry->GetDataBlock()->GetLength()
^
In file included from par2cmdline.h:615:0,
from par2creator.cpp:32:
verificationhashtable.h:429:49: warning: suggest parentheses around '&&' within '||' [-Wparentheses]
checksummer.ShortBlock() && checksummer.BlockLength() != nextentry->GetDataBlock()->GetLength()
^
/tmp/cco7GNtB.s: Assembler messages:
/tmp/cco7GNtB.s:4993: Error: thumb conditional instruction should be in IT block -- `cmpeq r1,r3'
/tmp/cco7GNtB.s:4994: Error: thumb conditional instruction should be in IT block -- `strexdeq lr,r2,r3,[fp]'
/tmp/cco7GNtB.s:5098: Error: thumb conditional instruction should be in IT block -- `cmpeq r3,r5'
/tmp/cco7GNtB.s:5099: Error: thumb conditional instruction should be in IT block -- `strexdeq r1,r4,r5,[fp]'
/tmp/cco7GNtB.s:5165: Error: thumb conditional instruction should be in IT block -- `strexeq r2,r10,[r3]'
/tmp/cco7GNtB.s:26751: Error: thumb conditional instruction should be in IT block -- `cmpeq r7,r3'
/tmp/cco7GNtB.s:26752: Error: thumb conditional instruction should be in IT block -- `strexdeq lr,r0,r1,[r5]'
make[2]: *** [par2creator.o] Error 1
make[2]: Leaving directory `/root/par2tbb/debpkg-par2tbb/par2cmdline-tbb-0.4+20150503'
make[1]: *** [all] Error 2
make[1]: Leaving directory `/root/par2tbb/debpkg-par2tbb/par2cmdline-tbb-0.4+20150503'
dh_auto_build: make -j1 returned exit code 2
make: *** [build] Error 2
dpkg-buildpackage: error: debian/rules build gave error exit status 2
Is there any way to solve this?
Edit: Sorry, here is my specs:
- Ubuntu 14.04.4 LTS
- Odroid X2 (1.7GHz Quad core ARM Cortex-A9 MPCore)
Re: Howto: multicore par2 on Debian, Ubuntu & derivatives
Posted: March 16th, 2016, 1:41 am
by jcfp
Looks like a compiler and/or assembler issue. The internets suggest passing
-Wa,-mimplicit-it=thumb to the build process, which you could do by editing the debian/rules file just after the uscan step in the tutorial. Insert something like this above the line that starts with a percent sign:
Code: Select all
CXXFLAGS:=$(shell dpkg-buildflags --get CXXFLAGS) -Wa,-mimplicit-it=thumb
export CXXFLAGS
And with that, I'm out of suggestions....
Re: Howto: multicore par2 on Debian, Ubuntu & derivatives
Posted: March 16th, 2016, 11:00 am
by TioSolid
Using this command gives me a "shell - command not found" error
I tried issuing
dpkg-buildpackage -S -us -uc -d -Wa -mimplicit-it=thumb
after the uscan command, but it gives me a "dpkg-buildflags: unknown option `-Wa'" error too
Re: Howto: multicore par2 on Debian, Ubuntu & derivatives
Posted: July 23rd, 2016, 3:15 am
by sandersonjobs
Hi all. Did anyone ever figure this out? I have a Raspberry Pi 3 ARMv8 Quad core and Sabnzb is really sucking in single core mode. I have been using this app a long time, but from what I hear, nzbget has this functionality built in already. Anyone have a work-around, or should I just make the switch?
Re: Howto: multicore par2 on Debian, Ubuntu & derivatives
Posted: August 19th, 2016, 9:35 pm
by kcallis
jcfp wrote:
The chuchusoft releases are forked from the old (2004 or so) par2 0.4, primarily adding tbb support; so yes, that's 0.4. Any version of par2 greater than 0.4 is from (or based on) the github-fork-turned-official hosted at
https://github.com/Parchive/par2cmdline/ Their 0.6.8 release changed the way the program tries to find misplaced blocks, 0.6.13 added the -S and -N options. The latter re-enables the old file scanning behaviour found in pre-0.6.8, including any and all versions that identify themselves as 0.4.
The ones you have to be careful with are 0.6.8-0.6.12 (inclusive), they suffer from a bug that assumes the offset of a misplaced block is always positive (see
this bug report) and don't have any way to adjust or override the new bahaviour.
So how does one get the update version on the Ubuntu box. I thought i could just use git and the new URL, but when I do the uscan I get the same issue:
Code: Select all
kcc@cortana:/tmp/tmp.SCY2wr3ZeE/debpkg-par2tbb$ uscan --force-downloaduscan warning: In watchfile debian/watch, reading webpage
https://github.com/jcfp/par2tbb-chuchusoft-sources/releases failed: 500 Can't connect to 127.0.0.1:8888
Re: Howto: multicore par2 on Debian, Ubuntu & derivatives
Posted: August 22nd, 2016, 10:49 am
by sandersonjobs
Sorry...I s witched to nzbget and have not had a single issue. I'm now just going to recommend the switch since nzbget makes it's own par2 that works with multiple architectures.
Re: Howto: multicore par2 on Debian, Ubuntu & derivatives
Posted: November 25th, 2016, 4:44 am
by safihre
EDITED: par2cmdline-mt doesn't work well with SABnzbd, because often it complains about files being 'out of basepath'.
Re: Howto: multicore par2 on Debian, Ubuntu & derivatives
Posted: January 17th, 2017, 4:04 pm
by piSAB
Hi Jcfp
Unfortunately this procedure no longer seems to work on a Raspberry PI 2 (Jessie build).
I posted what's listed below on
http://www.htpcguides.com/install-par2- ... ntu-linux/
but was prompted to contact you here:
Unfortunately this procedure no longer seems to work on a Raspberry PI 2 (Jessie build)
Even after switching the installed dependencies to use libopencv-dev instead of libtbb-dev:
sudo apt-get install devscripts build-essential git debhelper dh-autoreconf libwww-perl libopencv-dev -y
The Install PAR2 Multicore on ARMv7 procedure fails with a fatal error at the make command:
sudo make -j2
par2cmdline.h:495:39: fatal error: tbb/task_scheduler_init.h: No such file or directory
#include "tbb/task_scheduler_init.h"
^
In file included from par2cmdline.cpp:26:0:
par2cmdline.h:495:39: fatal error: tbb/task_scheduler_init.h: No such file or directory
#include "tbb/task_scheduler_init.h"
^
compilation terminated.
compilation terminated.
Makefile:445: recipe for target 'buffer.o' failed
make[1]: *** [buffer.o] Error 1
make[1]: *** Waiting for unfinished jobs....
Makefile:445: recipe for target 'par2cmdline.o' failed
make[1]: *** [par2cmdline.o] Error 1
make[1]: Leaving directory '/home/pi/par2tbb/debpkg-par2tbb/par2cmdline-tbb-0.4+20150503'
Makefile:327: recipe for target 'all' failed
make: *** [all] Error 2
Any help here would be appreciated. Thanks in advance.
Mike Mod
It could be a temporarily buggy script Simon, you can try asking the developer of the script on the sabnzbd forum
Re: Howto: multicore par2 on Debian, Ubuntu & derivatives
Posted: January 18th, 2017, 3:34 am
by jcfp
piSAB wrote:libopencv-dev
An image processing library? Why?? You
must install libtbb and its development headers for this version of par2. As it says right next to the instructions:
supported architectures and operating systems are probably limited to whatever has a recent version of libtbb available. Everything else leads to this:
piSAB wrote:fatal error: tbb/task_scheduler_init.h: No such file or directory
Please stick to the build instructions in the topic start: no installing libopencv as if it were a replacement for tbb, no manual configure runs, no 'sudo make' with parellel build options set. It's hard enough already to guess what's going on all these different platforms and hardware architectures.
Re: Howto: multicore par2 on Debian, Ubuntu & derivatives
Posted: January 18th, 2017, 3:36 am
by safihre
I also tried many ways to get tbb on my RP3, but never succeeded.
Re: Howto: multicore par2 on Debian, Ubuntu & derivatives
Posted: January 18th, 2017, 3:44 am
by jcfp
Maybe the hardware isn't compatible, maybe the os provides only a subset of debian packages (remember users having to build their own unrar?). I dunno.
Re: Howto: multicore par2 on Debian, Ubuntu & derivatives
Posted: January 18th, 2017, 3:53 am
by piSAB
Thanks for your replies. I was simply following the guide on:
http://www.htpcguides.com/install-par2- ... ntu-linux/
I made the replacement for libtbb-dev as suggested in the above mentioned guide as others had reported success in doing so.
libtbb-dev is not not available on the PI2 so as you suggest this is now most probably a no go for Raspberry PI.