Page 1 of 1

py3-branch - Check instance type before using startswith

Posted: May 28th, 2019, 1:49 pm
by Puzzled
Apparently in Python 3 the line in line.startswith('X-') can be a byte type, which requires different parameters. The article that caused it contains no metadata , only what seems like base64 encoded content. The header or subject has no encoding info.

2019-05-28 15:12:59,702::INFO::[downloader:302] Post-processing finished, resuming download
Exception in thread Thread-6:
Traceback (most recent call last):
File "C:\git\sabnzbd\sabnzbd\decoder.py", line 115, in run
data = self.decode(article, lines, raw_data)
File "C:\git\sabnzbd\sabnzbd\decoder.py", line 212, in decode
decoded_data, output_filename, crc, crc_expected, crc_correct = sabyenc3.decode_usenet_chunks(raw_data, article.bytes)
ValueError: Could not get filename

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users\t\AppData\Local\Programs\Python\Python37\lib\threading.py", line 917, in _bootstrap_inner
self.run()
File "C:\git\sabnzbd\sabnzbd\decoder.py", line 160, in run
if not line.startswith('X-') and match_str(lline, ('dmca', 'removed', 'cancel', 'blocked')):
TypeError: startswith first arg must be bytes or a tuple of bytes, not str

Re: py3-branch - Check instance type before using startswith

Posted: May 28th, 2019, 3:45 pm
by sander
I don't understand the first errror. Oh, wait, is it in sabyenc? That would be bad.

the startswith() seems the same as here: https://stackoverflow.com/a/19827633

Re: py3-branch - Check instance type before using startswith

Posted: May 28th, 2019, 4:53 pm
by Puzzled
I assume that sabyenc doesn't do base64 and doesn't find the filename. The exception is from the decoder: https://github.com/sabnzbd/sabyenc/blob ... nc3.c#L614

Then there is a new exception in the ValueError exception handling because of the byte problem.

I'm just downloading all the content in random groups for testing and so far I've had 5 files which have failed for this. It has never happened when I have downloaded specific things.

Re: py3-branch - Check instance type before using startswith

Posted: May 29th, 2019, 4:10 am
by safihre
The sabyenc error is correct, it's just a malformed article or one that we don't support. But then it shouldn't fail later on so we need to do some tricks there. I thought I had wrapped that, but seems we need a ubtou call on it.