Fatal error in Downloader - invalid literal for int() with base 10

[sander]

garbage in the answer in the precheck: thousands of \x00 (limited to 80 characters in logging below)

Code: Select all

2024-03-11 20:17:12,608::DEBUG::[newswrapper:97] SJ 200: bytearray(b'223 0 <JxFgMzBwLqVoRcPuJzHoSgFy-1658672651406@nyuu>\r\[email protected] (no posting)\r\n\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')
2024-03-11 20:17:12,608::DEBUG::[newswrapper:97] SJ 200: bytearray(b'223 0 <JxFgMzBwLqVoRcPuJzHoSgFy-1658672651406@nyuu>\r\[email protected] (no posting)\r\n\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')
2024-03-11 20:17:12,608::DEBUG::[newswrapper:97] SJ 200: bytearray(b'223 0 <JxFgMzBwLqVoRcPuJzHoSgFy-1658672651406@nyuu>\r\[email protected] (no posting)\r\n\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')
2024-03-11 20:17:12,608::DEBUG::[downloader:742] Article <JxFgMzBwLqVoRcPuJzHoSgFy-1658672651406@nyuu> is present

Code: Select all

2024-03-11 20:17:12,608::DEBUG::[newswrapper:97] SJ 200: bytearray(b'223 0 <TsJbMqVtYcIaGqEvShTyEhWf-1658672651793@nyuu>\r\[email protected] (no posting)\r\n\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')
2024-03-11 20:17:12,609::DEBUG::[newswrapper:97] SJ 200: bytearray(b'223 0 <TsJbMqVtYcIaGqEvShTyEhWf-1658672651793@nyuu>\r\[email protected] (no posting)\r\n\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')
2024-03-11 20:17:12,609::DEBUG::[newswrapper:97] SJ 200: bytearray(b'223 0 <TsJbMqVtYcIaGqEvShTyEhWf-1658672651793@nyuu>\r\[email protected] (no posting)\r\n\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')
2024-03-11 20:17:12,609::DEBUG::[downloader:742] Article <TsJbMqVtYcIaGqEvShTyEhWf-1658672651793@nyuu> is present

[sander]

And the problematic nzb:

Code: Select all

2024-03-11 20:24:29,726::DEBUG::[newswrapper:97] SJ 200: bytearray(b'R>\r\n0 <iAdIWztxRMwPakAtm8Fh@0d673917d82003195SYgm3DUmevgaPMqTvAD\n\r\n (no posting)\r\n\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')
2024-03-11 20:24:29,727::INFO::[notifier:157] Sending notification: Error - Fatal error in Downloader (type=error, job_cat=None)
2024-03-11 20:24:29,726::ERROR::[downloader:697] Fatal error in Downloader
Traceback (most recent call last):
  File "/home/sander/git/premiumize-sab/sabnzbd/downloader.py", line 693, in process_nw_worker
    self.process_nw(read_fds[nw_queue.get()])
  File "/home/sander/git/premiumize-sab/sabnzbd/downloader.py", line 732, in process_nw
    if nw.status_code != 222 and not done:
       ^^^^^^^^^^^^^^
  File "/home/sander/git/premiumize-sab/sabnzbd/newswrapper.py", line 99, in status_code
    return int(self.data[:3])
           ^^^^^^^^^^^^^^^^^^
ValueError: invalid literal for int() with base 10: b'R>\r'
2024-03-11 20:24:29,727::INFO::[downloader:394] Pausing
2024-03-11 20:24:29,727::INFO::[notifier:157] Sending notification: SABnzbd - Paused (type=pause_resume, job_cat=None)
2024-03-11 20:24:29,727::INFO::[downloader:414] Forcing disconnect

That answer

Code: Select all

bytearray(b'R>\r\n0 <iAdIWztxRMwPakAtm8Fh@0d673917d82003195SYgm3DUmevgaPMqTvAD\n\r\n (no posting)\r\n\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00') 

is total garbage.

See my following post.

[sander]

My conclusion:

usenet.premiumize.me produces total and random garbage. Non-standard. Not something SAB should try to handle.

My advice: complain at them, or dump them and choose a reliable news provider

[sander]

This happens only at the beginning of a job, right? I can drop a support ticket and let them know their downloads are 400-ing in the beginning.

Is it possible that this behaviour is interfering with the checking script?

Yes, at the beginning. It looks like the server is not yet ready / is spinning up.

And yes, you can report it to them.

[danieljai]

But this only happens when it's pre-checking, right? I'll turn of checking for now, and report to them.

I'm using them because they bundle with a bunch of other services that I use. Thank you Sander!

[sander]

But this only happens when it's pre-checking, right? I'll turn of checking for now, and report to them.

I'm using them because they bundle with a bunch of other services that I use. Thank you Sander!

No: see viewtopic.php?p=131530#p131530

"And without "Check before download", no Traceback, but again the weird response immediatly at the beginning of the download" ... with the 400 codes at the beginning.

You should be able to reproduce:
turn OFF the "check before download"
stop sabnzbd. Start a download ... do you get the 400 errors too?

[danieljai]

I'll try tomorrow night. I'm away from my home network atm.

I didn't notice them because it didn't freeze sabnzbd. Or did it in your case?

The only error that resulted to a froze and require restart was the 222 status code.

[danieljai]

garbage in the answer in the precheck: thousands of \x00 (limited to 80 characters in logging below)

Code: Select all

2024-03-11 20:17:12,608::DEBUG::[newswrapper:97] SJ 200: bytearray(b'223 0 <JxFgMzBwLqVoRcPuJzHoSgFy-1658672651406@nyuu>\r\[email protected] (no posting)\r\n\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')
2024-03-11 20:17:12,608::DEBUG::[newswrapper:97] SJ 200: bytearray(b'223 0 <JxFgMzBwLqVoRcPuJzHoSgFy-1658672651406@nyuu>\r\[email protected] (no posting)\r\n\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')
2024-03-11 20:17:12,608::DEBUG::[newswrapper:97] SJ 200: bytearray(b'223 0 <JxFgMzBwLqVoRcPuJzHoSgFy-1658672651406@nyuu>\r\[email protected] (no posting)\r\n\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')
2024-03-11 20:17:12,608::DEBUG::[downloader:742] Article <JxFgMzBwLqVoRcPuJzHoSgFy-1658672651406@nyuu> is present

...

Actually I was able to remote back home. Quickly tried a 100M and 1G file, and downloaded the logs. I searched through the entire log, I didn't get any \x00 at all.

[safihre]

Thanks for sharing your login!
I was able to reproduce and identify the bug in SABnzbd.
The summary is here: https://github.com/sabnzbd/sabnzbd/issues/2821

I will try to fix this in the next few days.

[danieljai]

Thank you for your hard work! Glad you guys were able to identify the bug.

Looks like the response was prematurely parsed. I'll turn off the checker now, but I'm not sure whether it happens outside of checking as well.

Also, I'm not sure whether the \x00 reported by sander is related/isolated cause I didn't see that error on my end.

[sander]

The \x00 was debug output. And/but Safihre made clear 1on1 that is NOT caused by the newsserver (but a SAB internal method)

[danieljai]

Awesome. Thank you both for your help!

[safihre]

We fixed it.
You could use the latest Docker image I think to get these code changes.

[danieljai]

Thanks safihre! Cheers!