help with dup detection and unwanted extensions.

[sander]

Solaris? Wow!

Things to do:
Check your rar supports "lb":

Code: Select all

$ rar | grep List
  l[t[a],b]     List archive contents [technical[all], bare]

If so, continue: find the file sabnzbd/assembler.py, and then

Short: replace it with this file http://pastebin.com/EhLL9f57

Or long & detailed: in assembler.py find the line:

Code: Select all

def rar_contains_unwanted_file(filepath):

AROUND that function, change the code to this:

Code: Select all

def rarfilecontents(myrarfile):
	cmd = sabnzbd.newsunpack.RAR_COMMAND + ' lb ' + myrarfile
	result = os.popen(cmd).readlines()
	return map(str.rstrip, result )	# return the list of files, with \r and \n etc removed


def rar_contains_unwanted_file(filepath):
    # checks for unwanted extensions in the rar file 'filepath'
    # ... unwanted extensions are defined in global variable cfg.unwanted_extensions()
    # returns False if no unwanted extensions are found in the rar file
    # returns name of file if unwanted extension is found in the rar file
    unwanted = None
    if cfg.unwanted_extensions() and is_rarfile(filepath):
        logging.debug('SJ: rar file to check: %s',filepath)
        # logging.debug('unwanted extensions are: %s', cfg.unwanted_extensions())
        try:
            '''
            zf = RarFile(filepath, all_names=True)
            logging.debug('SJ: files in rar file: %s', zf.namelist())
            for somefile in zf.namelist():
            '''
            for somefile in rarfilecontents(filepath):
                logging.debug('file in rar file: %s', somefile)
                if os.path.splitext(somefile)[1].replace('.', '').lower() in cfg.unwanted_extensions():
                    logging.debug('Unwanted file %s', somefile)
                    unwanted = somefile
                    zf.close()
        except:
            logging.debug('RAR file %s cannot be inspected.', filepath)
    return unwanted

The diff looks like this:

Code: Select all

> def rarfilecontents(myrarfile):
> 	cmd = 'rar lb ' + myrarfile	# sabnzbd.newsunpack.RAR_COMMAND
> 	cmd = sabnzbd.newsunpack.RAR_COMMAND + ' lb ' + myrarfile
> 	result = os.popen(cmd).readlines()
> 	print "SJ:", result
> 	return map(str.rstrip, result )
> 	'''
> 	for thisline in result:
> 		print thisline.rstrip()
> 	'''
> 
337c348
<         # logging.debug('rar file to check: %s',filepath)
---
>         logging.debug('SJ: rar file to check: %s',filepath)
339a351
>             '''
341c353
<             # logging.debug('files in rar file: %s', zf.namelist())
---
>             logging.debug('SJ: files in rar file: %s', zf.namelist())
342a355,356
>             '''
>             for somefile in rarfilecontents(filepath):

HTH

[diamaunt]

awesome, thanks.

[sander]

awesome, thanks.

Does that mean you tested it and it works for you?

[diamaunt]

awesome, thanks.

Does that mean you tested it and it works for you?

well, no, It meant I trusted your excellence and installed the patch.

I did test it now, and just got an email:

Hi,

SABnzbd has failed to download "The Blacklist S04E01"

Finished at 2016-10-20 13:53:53
Downloaded 102 MB

Results of the job:

Stage Failure
Aborted, unwanted extension detected



Sorry!

I'd say it should say "You're welcome!" instead of "Sorry!"

I also noticed several things paused for duplicates, though I can't tell if that's "duplicate nzb" or "duplicate series/season/episode" dupe... but the series/season/episode hasn't been working for me, hopefully, that's started working.

[sander]

"Aborted, unwanted extension detected", so the detection now works on your RAR5 formatted post? If so: good.

(I know nothing about duplicate detection, so I can't help you there. Different subject. That's the danger of two subjects in one thread)