Howto: multicore par2 on Debian, Ubuntu & derivatives

[jd01]

Running Stretch 9.5

I seem to have multicore par2 running in a much simpler way, did I do something wrong?

I installed par2-tbb by simply putting the referenced deb lines within into sources.list from:
https : //launchpad . net/~jcfp/+archive/ubuntu/sab-addons

i.e., these lines:
deb http:// ppa .launchpad. net/jcfp/sab-addons/ubuntu xenial main
deb-src http:// ppa. launchpad. net/jcfp/sab-addons/ubuntu xenial main

and then
# apt-get install par2-tbb

SAB reports "Multicore par2" with a checkmark, and a test download/repair/unpack worked just fine.

[sander]

Yes, correct: jcfp has added a few nice things to sab-addons: sabyenc, par2cmdline and par2cmdline-mt. So it's now easer than in 2015 (=date of FiPo). See https://launchpad.net/~jcfp/+archive/ubuntu/sab-addons

But what's more: the whole process is much easier than last year:

Code: Select all

apt-get install software-properties-common
add-apt-repository ppa:jcfp/ppa
apt-get install sabnzbdplus

Done. Including sabyenc (for high speed yenc decoding):

Code: Select all

# python -c "import sabyenc; print sabyenc.__version__ " 
3.3.1

Pure magic by @jcfp

[jd01]

But what's more: the whole process is much easier than last year:

Code: Select all

apt-get install software-properties-common
add-apt-repository ppa:jcfp/ppa
apt-get install sabnzbdplus

Done. Including sabyenc (for high speed yenc decoding):

Code: Select all

# python -c "import sabyenc; print sabyenc.__version__ " 
3.3.1

Pure magic by @jcfp

Too bad it doesn't work for Debian so easily. Nice though for those on Ubuntu.

[pinn]

Hi,

This is quite an old topic - is multicore par2 enabled by default now on Debian. If not, are there updated instructions?

Ta

[safihre]

Maybe you can use the par2mt version that we supply through the PPA?

[pinn]

Maybe you can use the par2mt version that we supply through the PPA?

I'm not very au fait how to do this. Does this all seem right?

1
osmc@htpc:~$ sudo add-apt-repository ppa:jcfp/sab-addons
sudo: add-apt-repository: command not found
osmc@htpc:~$ sudo apt install devscripts build-essential software-properties-com mon
Reading package lists... Done
Building dependency tree
Reading state information... Done
build-essential is already the newest version (12.3).
The following additional packages will be installed:
distro-info-data gir1.2-packagekitglib-1.0 iso-codes libfile-homedir-perl
libfile-which-perl libpackagekit-glib2-18 lsb-release python3-dbus
python3-gi python3-pycurl python3-software-properties
Suggested packages:
adequate autopkgtest bls-standalone bsd-mailx | mailx check-all-the-things
cvs-buildpackage devscripts-el diffoscope disorderfs dose-extra duck
faketime gnuplot how-can-i-help libauthen-sasl-perl
libfile-desktopentry-perl libnet-smtps-perl libterm-size-perl
libtimedate-perl libyaml-syck-perl mozilla-devscripts mutt piuparts ratt
reprotest svn-buildpackage w3m isoquery lsb python-dbus-doc python3-dbus-dbg
libcurl4-gnutls-dev python-pycurl-doc python3-pycurl-dbg
Recommended packages:
at dctrl-tools dput | dupload fakeroot file libdistro-info-perl
libencode-locale-perl libgit-wrapper-perl liblist-compare-perl liburi-perl
libwww-perl licensecheck lintian man-db patchutils python3-debian
python3-magic strace wdiff debian-keyring equivs liblwp-protocol-https-perl
libsoap-lite-perl packagekit unattended-upgrades
The following NEW packages will be installed:
devscripts distro-info-data gir1.2-packagekitglib-1.0 iso-codes
libfile-homedir-perl libfile-which-perl libpackagekit-glib2-18 lsb-release
python3-dbus python3-gi python3-pycurl python3-software-properties
software-properties-common
0 upgraded, 13 newly installed, 0 to remove and 153 not upgraded.
Need to get 4,387 kB of archives.
After this operation, 22.6 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://ftp.debian.org/debian stretch/main armhf libfile-which-perl all 1.2 1-1 [14.3 kB]
Get:2 http://ftp.debian.org/debian stretch/main armhf libfile-homedir-perl all 1 .00-1 [48.9 kB]
Get:3 http://ftp.debian.org/debian stretch/main armhf devscripts armhf 2.17.6+de b9u2 [938 kB]
Get:4 http://ftp.debian.org/debian stretch/main armhf distro-info-data all 0.36 [5,810 B]
Get:5 http://ftp.debian.org/debian stretch/main armhf libpackagekit-glib2-18 arm hf 1.1.5-2+deb9u1 [99.8 kB]
Get:6 http://ftp.debian.org/debian stretch/main armhf gir1.2-packagekitglib-1.0 armhf 1.1.5-2+deb9u1 [34.7 kB]
Get:7 http://ftp.debian.org/debian stretch/main armhf iso-codes all 3.75-1 [2,38 9 kB]
Get:8 http://ftp.debian.org/debian stretch/main armhf lsb-release all 9.20161125 [27.1 kB]
Get:9 http://ftp.debian.org/debian stretch/main armhf python3-dbus armhf 1.2.4-1 +b1 [178 kB]
Get:10 http://ftp.debian.org/debian stretch/main armhf python3-gi armhf 3.22.0-2 [460 kB]
Get:11 http://ftp.debian.org/debian stretch/main armhf python3-pycurl armhf 7.43 .0-2 [58.6 kB]
Get:12 http://ftp.debian.org/debian stretch/main armhf python3-software-properti es all 0.96.20.2-1 [49.5 kB]
Get:13 http://ftp.debian.org/debian stretch/main armhf software-properties-commo n all 0.96.20.2-1 [83.6 kB]
Fetched 4,387 kB in 0s (5,923 kB/s)
Selecting previously unselected package libfile-which-perl.
(Reading database ... 37082 files and directories currently installed.)
Preparing to unpack .../00-libfile-which-perl_1.21-1_all.deb ...
Unpacking libfile-which-perl (1.21-1) ...
Selecting previously unselected package libfile-homedir-perl.
Preparing to unpack .../01-libfile-homedir-perl_1.00-1_all.deb ...
Unpacking libfile-homedir-perl (1.00-1) ...
Selecting previously unselected package devscripts.
Preparing to unpack .../02-devscripts_2.17.6+deb9u2_armhf.deb ...
Unpacking devscripts (2.17.6+deb9u2) ...
Selecting previously unselected package distro-info-data.
Preparing to unpack .../03-distro-info-data_0.36_all.deb ...
Unpacking distro-info-data (0.36) ...
Selecting previously unselected package libpackagekit-glib2-18:armhf.
Preparing to unpack .../04-libpackagekit-glib2-18_1.1.5-2+deb9u1_armhf.deb ...
Unpacking libpackagekit-glib2-18:armhf (1.1.5-2+deb9u1) ...
Selecting previously unselected package gir1.2-packagekitglib-1.0.
Preparing to unpack .../05-gir1.2-packagekitglib-1.0_1.1.5-2+deb9u1_armhf.deb .. .
Unpacking gir1.2-packagekitglib-1.0 (1.1.5-2+deb9u1) ...
Selecting previously unselected package iso-codes.
Preparing to unpack .../06-iso-codes_3.75-1_all.deb ...
Unpacking iso-codes (3.75-1) ...
Selecting previously unselected package lsb-release.
Preparing to unpack .../07-lsb-release_9.20161125_all.deb ...
Unpacking lsb-release (9.20161125) ...
Selecting previously unselected package python3-dbus.
Preparing to unpack .../08-python3-dbus_1.2.4-1+b1_armhf.deb ...
Unpacking python3-dbus (1.2.4-1+b1) ...
Selecting previously unselected package python3-gi.
Preparing to unpack .../09-python3-gi_3.22.0-2_armhf.deb ...
Unpacking python3-gi (3.22.0-2) ...
Selecting previously unselected package python3-pycurl.
Preparing to unpack .../10-python3-pycurl_7.43.0-2_armhf.deb ...
Unpacking python3-pycurl (7.43.0-2) ...
Selecting previously unselected package python3-software-properties.
Preparing to unpack .../11-python3-software-properties_0.96.20.2-1_all.deb ...
Unpacking python3-software-properties (0.96.20.2-1) ...
Selecting previously unselected package software-properties-common.
Preparing to unpack .../12-software-properties-common_0.96.20.2-1_all.deb ...
Unpacking software-properties-common (0.96.20.2-1) ...
Setting up python3-pycurl (7.43.0-2) ...
Setting up python3-dbus (1.2.4-1+b1) ...
Setting up libfile-which-perl (1.21-1) ...
Setting up iso-codes (3.75-1) ...
Setting up libfile-homedir-perl (1.00-1) ...
Setting up distro-info-data (0.36) ...
Setting up devscripts (2.17.6+deb9u2) ...
Setting up libpackagekit-glib2-18:armhf (1.1.5-2+deb9u1) ...
Setting up python3-gi (3.22.0-2) ...
Setting up lsb-release (9.20161125) ...
Processing triggers for libc-bin (2.24-11+deb9u4) ...
Setting up gir1.2-packagekitglib-1.0 (1.1.5-2+deb9u1) ...
Setting up python3-software-properties (0.96.20.2-1) ...
Processing triggers for dbus (1.10.28-0+deb9u1) ...
Setting up software-properties-common (0.96.20.2-1) ...
Processing triggers for dbus (1.10.28-0+deb9u1) ...
osmc@htpc:~$ sudo add-apt-repository --enable-source ppa:jcfp/sab-addons
Assorted programs and modules for use with SABnzbd+
More info: https://launchpad.net/~jcfp/+archive/ubuntu/sab-addons
Press [ENTER] to continue or ctrl-c to cancel adding it

gpg: keybox '/tmp/tmpe72_axtk/pubring.gpg' created
gpg: /tmp/tmpe72_axtk/trustdb.gpg: trustdb created
gpg: key F13930B14BB9F05F: public key "Launchpad PPA for jcfp" imported
gpg: Total number processed: 1
gpg: imported: 1
gpg: no valid OpenPGP data found.
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python3.5/threading.py", line 914, in _bootstrap_inner
self.run()
File "/usr/lib/python3.5/threading.py", line 862, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib/python3/dist-packages/softwareproperties/SoftwareProperties.py" , line 688, in addkey_func
func(**kwargs)
File "/usr/lib/python3/dist-packages/softwareproperties/ppa.py", line 386, in add_key
return apsk.add_ppa_signing_key()
File "/usr/lib/python3/dist-packages/softwareproperties/ppa.py", line 273, in add_ppa_signing_key
cleanup(tmp_keyring_dir)
File "/usr/lib/python3/dist-packages/softwareproperties/ppa.py", line 234, in cleanup
shutil.rmtree(tmp_keyring_dir)
File "/usr/lib/python3.5/shutil.py", line 480, in rmtree
_rmtree_safe_fd(fd, path, onerror)
File "/usr/lib/python3.5/shutil.py", line 438, in _rmtree_safe_fd
onerror(os.unlink, fullname, sys.exc_info())
File "/usr/lib/python3.5/shutil.py", line 436, in _rmtree_safe_fd
os.unlink(name, dir_fd=topfd)
FileNotFoundError: [Errno 2] No such file or directory: 'S.gpg-agent.ssh'

osmc@htpc:~$ sudo apt-get update
Get:1 http://security.debian.org stretch/updates InRelease [94.3 kB]
Hit:2 http://ppa.launchpad.net/jcfp/ppa/ubuntu xenial InRelease
Ign:3 http://ftp.debian.org/debian stretch InRelease
Get:5 http://ftp.debian.org/debian stretch-updates InRelease [91.0 kB]
Hit:6 http://ppa.launchpad.net/jcfp/sab-addons/ubuntu xenial InRelease
Get:7 http://ppa.launchpad.net/jcfp/sab-addons/ubuntu focal InRelease [15.4 kB]
Hit:8 http://ftp.debian.org/debian stretch Release
Hit:4 http://ftp.fau.de/osmc/osmc/apt stretch InRelease
Hit:9 https://apt.sonarr.tv/debian stretch InRelease
Hit:10 https://download.mono-project.com/repo/debian stable-stretch InRelease
Get:11 http://security.debian.org stretch/updates/main armhf Packages [487 kB]
Get:12 http://security.debian.org stretch/updates/main arm64 Packages [491 kB]
Get:13 http://security.debian.org stretch/updates/main Translation-en [224 kB]
Get:14 http://ppa.launchpad.net/jcfp/sab-addons/ubuntu focal/main Sources [772 B]
Get:15 http://ppa.launchpad.net/jcfp/sab-addons/ubuntu focal/main armhf Packages [556 B]
Get:16 http://ppa.launchpad.net/jcfp/sab-addons/ubuntu focal/main arm64 Packages [560 B]
Get:17 http://ppa.launchpad.net/jcfp/sab-addons/ubuntu focal/main Translation-en [532 B]
Fetched 1,405 kB in 4s (339 kB/s)
Reading package lists... Done
N: Skipping acquire of configured file 'main/binary-arm64/Packages' as repository 'https://apt.sonarr.tv/debian stretch InRelease' doesn't support architecture 'arm64'
osmc@htpc:~$ sudo apt-get install par2-tbb
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
libtbb2
The following packages will be REMOVED:
par2
The following NEW packages will be installed:
libtbb2 par2-tbb
0 upgraded, 2 newly installed, 1 to remove and 153 not upgraded.
Need to get 213 kB of archives.
After this operation, 232 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://ftp.debian.org/debian stretch/main armhf libtbb2 armhf 4.3~20150611-2 [106 kB]
Get:2 http://ppa.launchpad.net/jcfp/sab-addons/ubuntu focal/main armhf par2-tbb armhf 0.4+20150503-0ubuntu1~jcfp2~focal [106 kB]
Fetched 213 kB in 0s (540 kB/s)
(Reading database ... 38613 files and directories currently installed.)
Removing par2 (0.8.0-0ubuntu1~jcfp1~16.04) ...
Selecting previously unselected package libtbb2:armhf.
(Reading database ... 38599 files and directories currently installed.)
Preparing to unpack .../libtbb2_4.3~20150611-2_armhf.deb ...
Unpacking libtbb2:armhf (4.3~20150611-2) ...
Selecting previously unselected package par2-tbb.
Preparing to unpack .../par2-tbb_0.4+20150503-0ubuntu1~jcfp2~focal_armhf.deb ...
Unpacking par2-tbb (0.4+20150503-0ubuntu1~jcfp2~focal) ...
Setting up libtbb2:armhf (4.3~20150611-2) ...
Processing triggers for libc-bin (2.24-11+deb9u4) ...
Setting up par2-tbb (0.4+20150503-0ubuntu1~jcfp2~focal) ...
osmc@htpc:~$ ^C
osmc@htpc:~$
t-repository --enable-source ppa:jcfp/sab-addons

[jcfp]

Par2-mt seized to exist as a seperate project years ago, its patches have been merged into the main par2 project. That means standard par2 is multicore using openmp, which is slightly slower than tbb but widely supported. The other option is par2-tbb, which has been abandoned by its upstream developer, hasn't seen fixes and updates in a long time, and only works on platforms that have libtbb. On the plus side, it is faster.

[pinn]

Par2-mt seized to exist as a seperate project years ago, its patches have been merged into the main par2 project. That means standard par2 is multicore using openmp, which is slightly slower than tbb but widely supported. The other option is par2-tbb, which has been abandoned by its upstream developer, hasn't seen fixes and updates in a long time, and only works on platforms that have libtbb. On the plus side, it is faster.

Thanks for the info. So what comes bundled by default with Sab?

[jcfp]

On Linux: nothing. Packages simply declare a dependency on par2, and installing sabnzbd thus pulls in a stand-alone package for that. On Debian or Ubuntu that would be 'par2' from the standard repos, but par2-tbb (only available from the ppa) also fits the bill as far as apt/dpkg are concerned because it claims to provide the same functionality (excerpt from 'apt-cache show par2-tbb'):

Code: Select all

Package: par2-tbb
[...]
Conflicts: par2, par2-mt
Provides: par2