multi format unpackaging: .zip, .rar, .iso, .7z, .gzip, ...

Want something added? Ask for it here.
chest069
Newbie
Newbie
Posts: 5
Joined: February 24th, 2008, 6:05 pm

Re: multi format unpackaging: .zip, .rar, .iso, .7z, .gzip, ...

Post by chest069 »

Would love to see this ability added too. Great idea.
User avatar
neilt0
Full Member
Full Member
Posts: 120
Joined: January 22nd, 2008, 4:16 am

Re: multi format unpackaging: .zip, .rar, .iso, .7z, .gzip, ...

Post by neilt0 »

Can you retain the option of using unzip and unrar on non X86 Linux machines (e.g. PPC and ARM9 NAS boxes) in case there's no easy way to get a binary of the p7zip program?
real_per
Newbie
Newbie
Posts: 6
Joined: May 18th, 2008, 4:14 pm

Re: multi format unpackaging: .zip, .rar, .iso, .7z, .gzip, ...

Post by real_per »

Is this feature planned in any way ?
Or can someone point me to a post processing script, that would do the trick ?
elfurbe
Newbie
Newbie
Posts: 7
Joined: January 23rd, 2008, 2:22 am

Re: multi format unpackaging: .zip, .rar, .iso, .7z, .gzip, ...

Post by elfurbe »

I suggested this a while ago when we were still on the SourceForge forums and was told that 7z had some issues with some features of the RAR format so it wasn't tenable as a full replacement.  I have, so far, never come across a RAR set that 7z can't extract, but I'm not calling anyone a liar.
User avatar
shypike
Administrator
Administrator
Posts: 19774
Joined: January 18th, 2008, 12:49 pm

Re: multi format unpackaging: .zip, .rar, .iso, .7z, .gzip, ...

Post by shypike »

Probably in 0.5.0, definitely not in 0.4.0
Haskell Noob
Newbie
Newbie
Posts: 9
Joined: December 11th, 2008, 6:44 pm

Re: multi format unpackaging: .zip, .rar, .iso, .7z, .gzip, ...

Post by Haskell Noob »

I know this is an old post, but the basic premise of supporting 7zip at some point hasn't changed, or has it?
User avatar
shypike
Administrator
Administrator
Posts: 19774
Joined: January 18th, 2008, 12:49 pm

Re: multi format unpackaging: .zip, .rar, .iso, .7z, .gzip, ...

Post by shypike »

It's  just that it's an awful lot of work with only limited benefits.
trog
Newbie
Newbie
Posts: 5
Joined: February 6th, 2011, 11:02 pm

Re: multi format unpackaging: .zip, .rar, .iso, .7z, .gzip, ...

Post by trog »

shypike wrote: It's  just that it's an awful lot of work with only limited benefits.
Just chiming in late on this one - I've been meaning to ask about this for a while, because I think the benefits would be a bigger than merely opening up a new range of formats for unzipping.

I upload large volumes of binaries to Usenet (primarily gaming related, but right now I'm uploading the new Debian v6.0.0 release) and I always find it vexing to have to use RAR because it's still technically payware and a proprietary system.

If it was possible, I'd much rather shift to an open format like 7z for my archiving needs - it would pretty much be a good drop-in replacement for me uploading.

I know I could switch over to 7zip now and just let people extract their archives manually, but I'm a very happy sabnzbd user and would love to see 7z supported natively in the hope that it would encourage more submissions to Usenet to be compressed with 7z as opposed to RAR.

I am sure it's a lot of work to make the switch, but just wanted to chime in and say that it would certainly be greatly appreciated by me!
User avatar
sander
Release Testers
Release Testers
Posts: 8811
Joined: January 22nd, 2008, 2:22 pm

Re: multi format unpackaging: .zip, .rar, .iso, .7z, .gzip, ...

Post by sander »

shypike wrote: It's  just that it's an awful lot of work with only limited benefits.
Can it be put in a custom post-process script by a user her/himself? (Maybe a stupid question; I don't know anything about the scripts ...)
Please don't send me unrequested PM's; the forum is the best way to communicate.
If someone helps you, please reply to that help.
f you like our support, check our special newsserver deal or donate at: https://sabnzbd.org/donate
User avatar
shypike
Administrator
Administrator
Posts: 19774
Joined: January 18th, 2008, 12:49 pm

Re: multi format unpackaging: .zip, .rar, .iso, .7z, .gzip, ...

Post by shypike »

Yes, that's possible, but not very efficient.
The 7z files will have been copied to "complete" before the script runs.

For the programmers among you.
What is needed for 7zip is an equivalent of the function rar_extract_core()
in the module sabnzbd/newsunpack.py
This function starts unrar with the right parameters and analyzes the
console output generated by it.
It's a fairly isolated piece of code with little interaction with other parts of SABnzbd.
User avatar
sander
Release Testers
Release Testers
Posts: 8811
Joined: January 22nd, 2008, 2:22 pm

Re: multi format unpackaging: .zip, .rar, .iso, .7z, .gzip, ...

Post by sander »

shypike, some questions on post-processing:

I've created a script cd-ing to $1 and then doing a 7z-unzip in that directory and some file writing in that directory. I've filled out the post-processing-script directory in sabnzbd/ However, I don't see any results in that $1 directory.

So:
- should sab 060 run the post process? Can you see it being called in the stdout output below?
- has the post-process-script the same input parameters as the pre-process? So like http://wiki.sabnzbd.org/user-scripts ?

Code: Select all

2011-02-21 11:45:24,248::INFO::[downloader:607] Thread [email protected]:119: forcing disconnect
2011-02-21 11:45:24,373::INFO::[postproc:230] Starting PostProcessing on 0394857.7z => Repair:True, Unpack:True, Delete:True, Script:None
2011-02-21 11:45:24,375::INFO::[postproc:461] Par2 check starting on 0394857.7z
2011-02-21 11:45:24,383::INFO::[postproc:476] Running repair on set 0394857.7z
2011-02-21 11:45:24,401::INFO::[postproc:83] Saving postproc queue
2011-02-21 11:45:24,403::INFO::[__init__:823] Saving data for postproc1.sab in /home/sander/.sabnzbd/admin/postproc1.sab
2011-02-21 11:45:24,416::INFO::[newsunpack:731] Quick-check for 0394857.7z is OK, skipping repair
2011-02-21 11:45:24,418::INFO::[newsunpack:788] Deleting /home/sander/Downloads/incomplete/0394857.7z/0394857.7z.par2
2011-02-21 11:45:24,422::INFO::[newsunpack:802] Deleting /home/sander/Downloads/incomplete/0394857.7z/0394857.7z.vol00+1.par2
2011-02-21 11:45:24,424::INFO::[postproc:491] Par2 check finished on 0394857.7z
2011-02-21 11:45:24,426::INFO::[misc:976] Creating directories: /home/sander/Downloads/complete/0394857.7z
2011-02-21 11:45:24,454::INFO::[postproc:279] Running unpack_magic on 0394857.7z
2011-02-21 11:45:24,458::INFO::[newsunpack:178] Filejoin starting on /home/sander/Downloads/incomplete/0394857.7z
2011-02-21 11:45:25,710::INFO::[newsunpack:182] Filejoin finished on /home/sander/Downloads/incomplete/0394857.7z
2011-02-21 11:45:25,713::INFO::[postproc:281] unpack_magic finished on 0394857.7z
2011-02-21 11:45:26,018::INFO::[postproc:434] Cleaning up 0394857.7z (keep_basic=False)
2011-02-21 11:45:26,028::INFO::[postproc:83] Saving postproc queue
2011-02-21 11:45:26,029::INFO::[__init__:823] Saving data for postproc1.sab in /home/sander/.sabnzbd/admin/postproc1.sab
2011-02-21 11:45:26,031::INFO::[downloader:225] Post-processing finished, resuming download


Please don't send me unrequested PM's; the forum is the best way to communicate.
If someone helps you, please reply to that help.
f you like our support, check our special newsserver deal or donate at: https://sabnzbd.org/donate
User avatar
shypike
Administrator
Administrator
Posts: 19774
Joined: January 18th, 2008, 12:49 pm

Re: multi format unpackaging: .zip, .rar, .iso, .7z, .gzip, ...

Post by shypike »

The pre-process script is not related to post-processing.
It has it own rules: http://wiki.sabnzbd.org/user-pre-queue-script

There is no general post-processing script.
You either set one per job or you associate one with a category.
Check whether the queue entry for the NZB has a script set.

Your logging says: Repair:True, Unpack:True, Delete:True, Script:None
User avatar
sander
Release Testers
Release Testers
Posts: 8811
Joined: January 22nd, 2008, 2:22 pm

Re: multi format unpackaging: .zip, .rar, .iso, .7z, .gzip, ...

Post by sander »

Aha: the post-process-script is *not* run on all downloads. You have to select a/the scrip manually per download (or category) on the right hand side of the download (most right dropdown).

Clear. Thanks!

After doing that, my 7z-unzip script works on the first test downloads.

To do:
- handle spaces in the directory name (my current sh-script cannot handle that)
- handle password protected 7z files (meaning: skip them)
Please don't send me unrequested PM's; the forum is the best way to communicate.
If someone helps you, please reply to that help.
f you like our support, check our special newsserver deal or donate at: https://sabnzbd.org/donate
User avatar
sander
Release Testers
Release Testers
Posts: 8811
Joined: January 22nd, 2008, 2:22 pm

Re: multi format unpackaging: .zip, .rar, .iso, .7z, .gzip, ...

Post by sander »

screenshot attached ...
Please don't send me unrequested PM's; the forum is the best way to communicate.
If someone helps you, please reply to that help.
f you like our support, check our special newsserver deal or donate at: https://sabnzbd.org/donate
User avatar
sander
Release Testers
Release Testers
Posts: 8811
Joined: January 22nd, 2008, 2:22 pm

Re: multi format unpackaging: .zip, .rar, .iso, .7z, .gzip, ...

Post by sander »

OK, done: I've implemented 7z-unzipping as a post-process script and it works on all the 7z-downloads I've tested.

Developed and tested on Ubuntu. It will need some tweaking for Windows; I can assist


@shypike: some catch-all post-proces-script would be a nice feature. The script could for the existence of *.7z and do it's thing.

Final remark: I see little posts with 7z, so let's see if anybody uses my script. If it is used a lot, the 7z-unzipping could get into SAB's mainstream
Please don't send me unrequested PM's; the forum is the best way to communicate.
If someone helps you, please reply to that help.
f you like our support, check our special newsserver deal or donate at: https://sabnzbd.org/donate
Post Reply