Obfuscated files without PARs
Forum rules
Help us help you:
Help us help you:
- Are you using the latest stable version of SABnzbd? Downloads page.
- Tell us what system you run SABnzbd on.
- Adhere to the forum rules.
- Do you experience problems during downloading?
Check your connection in Status and Interface settings window.
Use Test Server in Config > Servers.
We will probably ask you to do a test using only basic settings. - Do you experience problems during repair or unpacking?
Enable +Debug logging in the Status and Interface settings window and share the relevant parts of the log here using [ code ] sections.
-
- Newbie
- Posts: 3
- Joined: July 23rd, 2019, 8:44 am
Obfuscated files without PARs
Is there any work being done to implement de-obfuscation for files without pars? If you are interested in adding this to your app, please contact me, I do this using powershell, it is pretty simple.
Re: Obfuscated files without PARs
Can you post the powershell script, and a link to such a NZB?
-
- Newbie
- Posts: 3
- Joined: July 23rd, 2019, 8:44 am
Re: Obfuscated files without PARs
I am sure you can find one with obfuscated files, but here is the script:
$FileList = Get-ChildItem -Path X:\Downloads\Folder
foreach ($File in $FileList)
{
$test = C:\WinRAR\rar l $file.FullName
$option = [System.StringSplitOptions]::RemoveEmptyEntries
$newvar = ($test[11].split(' ',$option)).foreach{$_}
$number = $newvar[2].PadLeft(4,'0')
$newname = $file.DirectoryName + '\' + 'recovered.' + 'part' + $number + '.rar'
ren $file.FullName $newname
}
$FileList = Get-ChildItem -Path X:\Downloads\Folder
foreach ($File in $FileList)
{
$test = C:\WinRAR\rar l $file.FullName
$option = [System.StringSplitOptions]::RemoveEmptyEntries
$newvar = ($test[11].split(' ',$option)).foreach{$_}
$number = $newvar[2].PadLeft(4,'0')
$newname = $file.DirectoryName + '\' + 'recovered.' + 'part' + $number + '.rar'
ren $file.FullName $newname
}
-
- Newbie
- Posts: 3
- Joined: July 23rd, 2019, 8:44 am
Re: Obfuscated files without PARs
one caveat, run the files through a renamer first or do it via powershell... the files will need an RAR extension for this script to function
Re: Obfuscated files without PARs
There is a way to indeed extract this from the header of the rar-files.
Maybe you have some coding skills and can help us out with this? I have some basic documentation!
Maybe you have some coding skills and can help us out with this? I have some basic documentation!
If you like our support, check our special newsserver deal or donate at: https://sabnzbd.org/donate
Re: Obfuscated files without PARs
Some questions:
- can you put / name the rar files in random order?
- what if there is more than 1 file? The "rar l ..." will list more than 1 file, and will pick ... which name? See example below. Does your script work then (I can only read & understand vaguely the powershell commands)?
- can you put / name the rar files in random order?
- what if there is more than 1 file? The "rar l ..." will list more than 1 file, and will pick ... which name? See example below. Does your script work then (I can only read & understand vaguely the powershell commands)?
Code: Select all
$ rar l 69a04687-a31d-42c0-80d0-06ceeb074336
RAR 5.50 Copyright (c) 1993-2017 Alexander Roshal 11 Aug 2017
Trial version Type 'rar -?' for help
Archive: 69a04687-a31d-42c0-80d0-06ceeb074336
Details: RAR 5, volume 1
Attributes Size Date Time Name
----------- --------- ---------- ----- ----
-rw-r--r-- 1048576 2019-07-26 15:33 Anothername.bin
-rw-r--r-- 104857600 2019-07-26 15:32 Thisissomename.bin
----------- --------- ---------- ----- ----
105906176 2
Re: Obfuscated files without PARs
Ah: the order / index is in the "Details: RAR 5, volume 2" ?
Let's try:
Ah, nice
Code: Select all
sander@witte:~/post_this/my_obfuscation/bla$ rar l 3b21332d-f6f7-4d67-9b84-1b63e37d7e11
RAR 5.50 Copyright (c) 1993-2017 Alexander Roshal 11 Aug 2017
Trial version Type 'rar -?' for help
Archive: 3b21332d-f6f7-4d67-9b84-1b63e37d7e11
Details: RAR 5, volume 5
Attributes Size Date Time Name
----------- --------- ---------- ----- ----
-rw-r--r-- 104857600 2019-07-26 15:32 Thisissomename.bin
----------- --------- ---------- ----- ----
0 0
sander@witte:~/post_this/my_obfuscation/bla$ rar l f1530e06-cd88-47fe-b500-a6cb36636003
RAR 5.50 Copyright (c) 1993-2017 Alexander Roshal 11 Aug 2017
Trial version Type 'rar -?' for help
Archive: f1530e06-cd88-47fe-b500-a6cb36636003
Details: RAR 5, volume 2
Attributes Size Date Time Name
----------- --------- ---------- ----- ----
-rw-r--r-- 104857600 2019-07-26 15:32 Thisissomename.bin
----------- --------- ---------- ----- ----
0 0
Code: Select all
$ for f in *; do echo "Processing $f file.." ; rar l $f | grep Details ; done
Processing 3b21332d-f6f7-4d67-9b84-1b63e37d7e11 file..
Details: RAR 5, volume 5
Processing 448e77e7-f685-42a2-81a6-1581c3d37979 file..
Details: RAR 5, volume 3
Processing 69a04687-a31d-42c0-80d0-06ceeb074336 file..
Details: RAR 5, volume 1
Processing 9bf8f3c6-672a-41e6-ae86-073afc9c3cb0 file..
Details: RAR 5, volume 4
Processing f1530e06-cd88-47fe-b500-a6cb36636003 file..
Details: RAR 5, volume 2
Re: Obfuscated files without PARs
OK, bingo:
Start: fully obfuscated:
Rename with correct order number (but random file name):
Nice result so far:
Let unrar work on it:
Check:
Done!
To do: checking the file is really a rar file before renaming.
Start: fully obfuscated:
Code: Select all
for f in *.rar; do echo "Processing $f file.."; mv $f `uuidgen` ; done
Code: Select all
sander@witte:~/post_this/my_obfuscation/bla$ ls -al
total 103436
drwxr-xr-x 2 sander sander 4096 jul 26 15:48 .
drwxr-xr-x 3 sander sander 4096 jul 26 15:40 ..
-rw-r--r-- 1 sander sander 13632578 jul 26 15:48 3b21332d-f6f7-4d67-9b84-1b63e37d7e11
-rw-r--r-- 1 sander sander 23068672 jul 26 15:48 448e77e7-f685-42a2-81a6-1581c3d37979
-rw-r--r-- 1 sander sander 23068672 jul 26 15:48 69a04687-a31d-42c0-80d0-06ceeb074336
-rw-r--r-- 1 sander sander 23068672 jul 26 15:48 9bf8f3c6-672a-41e6-ae86-073afc9c3cb0
-rw-r--r-- 1 sander sander 23068672 jul 26 15:48 f1530e06-cd88-47fe-b500-a6cb36636003
Rename with correct order number (but random file name):
Code: Select all
$ for f in *; do echo "Processing $f file.." ; ordernumber=`rar l $f | grep Details | awk '{ print $NF }'` ; echo $ordernumber ; mv $f blabla.part$ordernumber.rar ; done
Processing 3b21332d-f6f7-4d67-9b84-1b63e37d7e11 file..
5
Processing 448e77e7-f685-42a2-81a6-1581c3d37979 file..
3
Processing 69a04687-a31d-42c0-80d0-06ceeb074336 file..
1
Processing 9bf8f3c6-672a-41e6-ae86-073afc9c3cb0 file..
4
Processing f1530e06-cd88-47fe-b500-a6cb36636003 file..
2
Nice result so far:
Code: Select all
sander@witte:~/post_this/my_obfuscation/bla$ ll
total 103436
drwxr-xr-x 2 sander sander 4096 jul 26 15:53 ./
drwxr-xr-x 3 sander sander 4096 jul 26 15:40 ../
-rw-r--r-- 1 sander sander 23068672 jul 26 15:48 blabla.part1.rar
-rw-r--r-- 1 sander sander 23068672 jul 26 15:48 blabla.part2.rar
-rw-r--r-- 1 sander sander 23068672 jul 26 15:48 blabla.part3.rar
-rw-r--r-- 1 sander sander 23068672 jul 26 15:48 blabla.part4.rar
-rw-r--r-- 1 sander sander 13632578 jul 26 15:48 blabla.part5.rar
Code: Select all
sander@witte:~/post_this/my_obfuscation/bla$ unrar x *part1.rar
UNRAR 5.71 freeware Copyright (c) 1993-2019 Alexander Roshal
Extracting from blabla.part1.rar
Extracting Anothername.bin OK
Extracting Thisissomename.bin 21%
Extracting from blabla.part2.rar
... Thisissomename.bin 43%
Extracting from blabla.part3.rar
... Thisissomename.bin 65%
Extracting from blabla.part4.rar
... Thisissomename.bin 87%
Extracting from blabla.part5.rar
... Thisissomename.bin OK
All OK
Code: Select all
sander@witte:~/post_this/my_obfuscation/bla$ ll
total 206860
drwxr-xr-x 2 sander sander 4096 jul 26 15:54 ./
drwxr-xr-x 3 sander sander 4096 jul 26 15:40 ../
-rw-r--r-- 1 sander sander 1048576 jul 26 15:33 Anothername.bin
-rw-r--r-- 1 sander sander 23068672 jul 26 15:48 blabla.part1.rar
-rw-r--r-- 1 sander sander 23068672 jul 26 15:48 blabla.part2.rar
-rw-r--r-- 1 sander sander 23068672 jul 26 15:48 blabla.part3.rar
-rw-r--r-- 1 sander sander 23068672 jul 26 15:48 blabla.part4.rar
-rw-r--r-- 1 sander sander 13632578 jul 26 15:48 blabla.part5.rar
-rw-r--r-- 1 sander sander 104857600 jul 26 15:32 Thisissomename.bin
sander@witte:~/post_this/my_obfuscation/bla$
Done!
To do: checking the file is really a rar file before renaming.
Re: Obfuscated files without PARs
Python script is working
Shypike ... now what? Where/how to put this into SABnzbd?
Edit
reference NZB:
https://raw.githubusercontent.com/sande ... 9c3cb0.nzb
Current SAB does some strange renaming :
Code: Select all
$ python deobfuscate.py
deobfuscate.py
Skipping (as not a rar file): deobfuscate.py
3b21332d-f6f7-4d67-9b84-1b63e37d7e11
Renaming 3b21332d-f6f7-4d67-9b84-1b63e37d7e11 to blablabla.part5.rar
notarar-blablabla
Skipping (as not a rar file): notarar-blablabla
448e77e7-f685-42a2-81a6-1581c3d37979
Renaming 448e77e7-f685-42a2-81a6-1581c3d37979 to blablabla.part3.rar
f1530e06-cd88-47fe-b500-a6cb36636003
Renaming f1530e06-cd88-47fe-b500-a6cb36636003 to blablabla.part2.rar
9bf8f3c6-672a-41e6-ae86-073afc9c3cb0
Renaming 9bf8f3c6-672a-41e6-ae86-073afc9c3cb0 to blablabla.part4.rar
69a04687-a31d-42c0-80d0-06ceeb074336
Renaming 69a04687-a31d-42c0-80d0-06ceeb074336 to blablabla.part1.rar
Edit
reference NZB:
https://raw.githubusercontent.com/sande ... 9c3cb0.nzb
Current SAB does some strange renaming :
Code: Select all
2019-07-26 17:08:42,163::INFO::[postproc:376] Finished unpack_magic on blabla123
2019-07-26 17:08:42,164::DEBUG::[filesystem:625] Moving (overwrite: 0) /home/sander/Downloads/incomplete/blabla123/reftestnzb obfuscated rar files no par 073afc9c3cb0 [2+5] - "448e77e7-f685-42a2-81a6-1581c3d37979" yEnc (1+33) 23068672 => /home/sander/Downloads/complete/_UNPACK_blabla123/reftestnzb obfuscated rar files no par 073afc9c3cb0 [2+5] - "448e77e7-f685-42a2-81a6-1581c3d37979" yEnc (1+33) 23068672
2019-07-26 17:08:42,165::DEBUG::[filesystem:718] Renaming "/home/sander/Downloads/incomplete/blabla123/reftestnzb obfuscated rar files no par 073afc9c3cb0 [2+5] - "448e77e7-f685-42a2-81a6-1581c3d37979" yEnc (1+33) 23068672" to "/home/sander/Downloads/complete/_UNPACK_blabla123/reftestnzb obfuscated rar files no par 073afc9c3cb0 [2+5] - "448e77e7-f685-42a2-81a6-1581c3d37979" yEnc (1+33) 23068672"
2019-07-26 17:08:42,166::DEBUG::[filesystem:625] Moving (overwrite: 0) /home/sander/Downloads/incomplete/blabla123/reftestnzb obfuscated rar files no par 073afc9c3cb0 [1+5] - "3b21332d-f6f7-4d67-9b84-1b63e37d7e11" yEnc (1+20) 13632578 => /home/sander/Downloads/complete/_UNPACK_blabla123/reftestnzb obfuscated rar files no par 073afc9c3cb0 [1+5] - "3b21332d-f6f7-4d67-9b84-1b63e37d7e11" yEnc (1+20) 13632578
2019-07-26 17:08:42,167::DEBUG::[filesystem:718] Renaming "/home/sander/Downloads/incomplete/blabla123/reftestnzb obfuscated rar files no par 073afc9c3cb0 [1+5] - "3b21332d-f6f7-4d67-9b84-1b63e37d7e11" yEnc (1+20) 13632578" to "/home/sander/Downloads/complete/_UNPACK_blabla123/reftestnzb obfuscated rar files no par 073afc9c3cb0 [1+5] - "3b21332d-f6f7-4d67-9b84-1b63e37d7e11" yEnc (1+20) 13632578"
Re: Obfuscated files without PARs
While we could do it, we should do this with the header (like Unrar does). Needs a seperate version for rar3 and rar5.
If you like our support, check our special newsserver deal or donate at: https://sabnzbd.org/donate
-
- Newbie
- Posts: 43
- Joined: September 15th, 2019, 6:00 pm
Re: Obfuscated files without PARs
Has this been implemented? Currently on the latest beta and I have one where SAB downloads the files just fine, uses the par2 to repair them but then I'm left with the obfuscated filenames. Those are all rars but how do I figure out which one is which in what order? I read over at NZBGet that they implemented something to read the rar file header and rename files according to this but I cant test that in Get since Get refuses to download this nzb (for health reasons) but SAB downloads it just fine.
Re: Obfuscated files without PARs
SABnzbd has now built-in deobfuscation of obfuscated rar files, and more deobfuscation.
Did you turn on Debofuscate Final Filenames? See http://127.0.0.1:8080/sabnzbd/config/sw ... _filenames
If it doesn't work, share your NZB via pastebin.com, or [email protected]
PS: there is one thing that SAB (nor any other too) cannot do: a whole season, fulle obfuscated, and then determine which file is which ... Impossible ...
Did you turn on Debofuscate Final Filenames? See http://127.0.0.1:8080/sabnzbd/config/sw ... _filenames
If it doesn't work, share your NZB via pastebin.com, or [email protected]
PS: there is one thing that SAB (nor any other too) cannot do: a whole season, fulle obfuscated, and then determine which file is which ... Impossible ...
-
- Newbie
- Posts: 43
- Joined: September 15th, 2019, 6:00 pm
Re: Obfuscated files without PARs
[email protected] does not work, I sent it to [email protected] instead. I turned on the above mentioned switch and SAB did rename the obfuscated files now but I think still not properly as far as I can tell. It's a single episode btw.
Re: Obfuscated files without PARs
Thanks for the NZB.
SAB downloaded it, and indeed the result is obfuscated files, which are rar files.
SAB can handle obfuscated rar files. I don't know why SAB didn't do that on these files. First guess: double obfuscated, and SAB only does one run / one layer of deobfuscation.
I'm currently working on another thing for SAB, so no time yet to investigate this.
One question to you: is it just this NZB, or do have more of these problematics posts?
SAB downloaded it, and indeed the result is obfuscated files, which are rar files.
SAB can handle obfuscated rar files. I don't know why SAB didn't do that on these files. First guess: double obfuscated, and SAB only does one run / one layer of deobfuscation.
I'm currently working on another thing for SAB, so no time yet to investigate this.
One question to you: is it just this NZB, or do have more of these problematics posts?
-
- Newbie
- Posts: 43
- Joined: September 15th, 2019, 6:00 pm
Re: Obfuscated files without PARs
it is just this nzb for now. don't worry. I'll get it reupped with proper renaming.