threaded 7-zip

Questions and bug reports for Beta releases should be posted here.
Forum rules
Help us help you:
  • Tell us what system you run SABnzbd on.
  • Adhere to the forum rules.
  • Do you experience problems during downloading?
    Check your connection in Status and Interface settings window.
    Use Test Server in Config > Servers.
    We will probably ask you to do a test using only basic settings.
  • Do you experience problems during repair or unpacking?
    Enable +Debug logging in the Status and Interface settings window and share the relevant parts of the log here using [ code ] sections.
Post Reply
bpwats
Newbie
Newbie
Posts: 28
Joined: May 16th, 2008, 10:13 am

threaded 7-zip

Post by bpwats »

Hi all,

On the Wiki i read unrar/unzip will be replaced with 7-zip for the 0.5 release.
I use a 2008 4-core Mac Pro with 4x1GB RAM on Mac OS X 10.5 and run the latest beta SabNZBd from source.

Instead of using Par264 from the MacPar Deluxe package, I installed Vincent Tan's version, which uses Intel Threading Building Blocks to create a 64-bit UB.
http://www.chuchusoft.com/par2_tbb/index.html
It shows dramatic improvements up to 50% on repairing a DVD5-sized Wii game with 232 repair blocks over the plain old par2 version. Being 64-bit helps only a little to seed up processing in this case.

Being encouraged by this dramatic result, i started looking for a multithreaded unpack util.
Not being a developer, I still managed to make 7z from source.

Get the P7ZIP source files from http://downloads.sourceforge.net/p7zip/ ... g_mirror=0 and unpack.
In my case, copy makefile.macosx over makefile.machine
To make a parallel build :

Code: Select all

make -j4 -f makefile.parallel_jobs

Code: Select all

make 7z
Contrary to 7za (available in .DMG format http://homepage.mac.com/krmathis/applic ... 204.57.dmg), 7z and it's plugins will also be able to unrar.
The binaries are created in the /bin subfolder.

7z shows no verbose output, but takes only just over 2 mins to unrar a Wii game (90 50MB parts, DVD-5)!
Rar 3.71 (64 bit) takes 3 minutes!

For SabNZBd, it makes sense to create a symbolic link from unrar to /usr/bin/7z I suppose, rather than over writing.
Of course, 7z can also be used in packages or compiled for a variety of other machines.

Hope this helps.

bpwats
User avatar
shypike
Administrator
Administrator
Posts: 19774
Joined: January 18th, 2008, 12:49 pm

Re: threaded 7-zip

Post by shypike »

You mix packages. par2 and unrar/7zip are not related.

You cannot just replace unrar by 7zip.
SABnzbd carefully analyses the (console) output from unrar to check progress
and results. Also unrar and 7zip need different paramters.
You cannot just drop in 7ZIP as a replacement.
7zip interfacing needs to be built into SABnzbd, which is quite some work.
bpwats
Newbie
Newbie
Posts: 28
Joined: May 16th, 2008, 10:13 am

Re: threaded 7-zip

Post by bpwats »

Thanks for pointing out unrar cannot simply be replaced with 7z.
(I will not try now  ;) and wait patiently for the next release).
I know par2 and rar are two different things, but they both benefit from multi-threaded processing.

Thanks for your great work on SabNZBd so far!
bpwats
Post Reply