Missing articles & repair files on 1st priority server, 2nd not used automatically

[mysabqtaa20]

Hello,

my setup:
- SABnzbd 2.3.9 win64 Standalone
- I have two providers: an unlimited with low retention and a limited with high retention time, which I added to sabnzbd. Unlimited account with priority 1, limited account with priority 2.

When I now try to download something, which isn't complete on my first provider, the download fails in the end (missing articles / malformed / not enough repair blocks). Now I have to disable my 1st priority server and I can try two things:
(1) retry the download with the original nzb-file as "additional nzb" on retry-dialog - sometimes that works.
(2) delete the old download and retry the whole thing from my 2nd provider - works.

As far as I understood the "priority" settings, sabnzbd would automatically try the 2nd priority server, when it hits missing articles etc., but it doesn't...?

Is my understanding completely off, did I forget to change another option or is this how it is supposed to be?

Thanks in advance.

[safihre]

That depends, on the missing download, what does it show in the details?
X articles missing
or
X articles corrupt?

[mysabqtaa20]

Hello,

thanks for getting back to me!
I only deleted the "Servers: xxx" line at code-logs.

First test:
(1) (both provider active) After adding the nzb the first time:
- During the download, the status bar already shows "x MB missing articles"
-->

Code: Select all

Failed with message: Age: 48d # 23 articles were malformed # Aborted, cannot be completed

(2) (both provider active) After pressing retry without adding nzb again or changing anything:
- switches to Download-Area again, starts at ~50%, shows increasing number of x MB missing articles and failes with message:
-->

Code: Select all

Age: 48d # 22 articles were malformed
Repair
[6r7c2e7c2y6j29fp7p7m] Verified in 2 seconds, all files correct
[6r7c2e7c2y6j29fp7p7m] Repair failed, not enough repair blocks (117 short)

(3) turning provider off/on and adding nzb as additional doesn't help. -> Failed overall.

Second test: (differs at (2))
(1) (both provider active) After adding the nzb the first time:
- During the download, the status bar already shows "x MB missing articles"
-->

Code: Select all

Failed with message: Age: 48d # 23 articles were malformed # Aborted, cannot be completed

(2)(turn 1st priority server off, only 2nd priority server still active)
- after pressing "retry" and adding original nzb as additional nzb: it switches to Download-Area again, starts at ~50%, but doesn't show no any missing article anymore, but fails with message:
-->

Code: Select all

Age: 48d
Repair
[6r7c2e7c2y6j29fp7p7m] Repair failed, not enough repair blocks (51 short)

Third test and only 2nd priority server active - Download finishes without any problem.

-----
Do you see a pattern?
Unfortunately, I've only this one nzb-file right now to test, but I had "missing article" messages as well some time ago. I didn't save the nzb - my bad.

Btw: Isn't there an option "download malformed articles again from different server" oder something like this?

Thanks in advance!

[safihre]

When articles are malformed, we don't try to fetch them from a different server because usually these articles just failed to upload correctly.
Seems that's not the case here.. We don't have an option to retry malformed ones on a different server, but maybe I should..

[mysabqtaa20]

We don't have an option to retry malformed ones on a different server, but maybe I should..

That would be great! Thank you!

Just two more questions:
(1) Why does it say "x MB missing articles" during the download? Aren't these "missing MB" the reason for the malformation?
(2) If articles would be really missing, sabnzbd should try to download them from another or lower priority server, right? If that happens, I would revive this thread.

[safihre]

Indeed, there is some crossing of terms during and after the downloading dat can cause some confusion, sorry about that!
The missing articles are "missing" from the final files, but in this specific NZB they are not "missing" from the server but corrupt.
Indeed if the server returned missing-status-codes, we would try the second servers.

[mysabqtaa20]

Thanks for clarifying!
Perhaps the following wording would be better: "x MB missing chunks, Y corrupted parts" suited.

I would really appreciate it, if SABnzbd could download these missing chunks (or if that's not possible the whole corrupted part) from the 2nd server in the future! Thanks for opening a request @ Github: #1466 !

[safihre]

After this post I did some digging when we removed it and how other clients handle it.
I decided to restore the old behavior that articles with CRC errors will be retried.

[mysabqtaa20]

Hello,

first of all: thanks for re-adding the feature at all.
Using the old version, I got these "MB missing articles" messages again, so I switched to the newer version:

I tried the 3.0B2 and I got the following result with both server active:

Code: Select all

[FCdxxxxxxx] Verified in 14 seconds, all files correct
[FCdxxxxxxx] Repair failed, not enough repair blocks (728 short)

Yes, during the download, no "MB missing articles" was shown and server 2 was used to download some parts/articles. So it is definitely better than before.
When I turn off server 1, just using server 2, no problem at all. Should it work like this?

And I run into a bug, which isn't in the issue list on github:
When I try to "retry" the failed download with only server 2 active and I add the nzb file again on retry-dialog, I get the following error message and trace:

Code: Select all

Traceback (most recent call last):
  File "lib\site-packages\cherrypy\_cprequest.py", line 638, in respond
  File "lib\site-packages\cherrypy\_cprequest.py", line 697, in _do_respond
  File "lib\site-packages\cherrypy\lib\encoding.py", line 219, in __call__
  File "lib\site-packages\cherrypy\_cpdispatch.py", line 54, in __call__
  File "sabnzbd\interface.py", line 118, in internal_wrap
  File "sabnzbd\interface.py", line 432, in retry_pp
  File "sabnzbd\api.py", line 1435, in retry_job
  File "sabnzbd\nzbqueue.py", line 180, in repair_job
  File "sabnzbd\__init__.py", line 678, in add_nzbfile
TypeError: a bytes-like object is required, not 'NoneType'

[safihre]

Which providers do you use? And could you send me the NZB at [email protected]?
Then I can try to reproduce.

[mysabqtaa20]

First server is "Speedium" and second server is Newshosting. Yes, as one of the pull request answers guessed right, Speedium is running in beta mode right now, but this gives SABnzbd also a chance for rare errors. But 2nd: if these errors are too irregular and not worth the time or effort, it's fine by me.

I sent you a link via PM, hope this works as well.

[safihre]

Thanks! Going to investigate!

[sander]

FWIW: I have a Speedium too, and I've now made it my primary newsserver account on my download machine. So far no tracebacks, although very slow downloads on old posts (backfilling?): 1 MB/s, instead of 10 MB/s

@safihre if you want, I can share the Speedium account with you.

[mysabqtaa20]

So far no tracebacks, although very slow downloads on old posts (backfilling?): 1 MB/s, instead of 10 MB/s

No slowdown on my end with 20 connections (maximum by provider), but my max speed is only 3MB/s. Either it's full speed or errors for me.

Btw: During beta phase anyone can register and use Speedium for free. No need for account sharing .

Thanks for investigating!

[safihre]

Strange, I made the same setup as you and it completed fine.
But then I also didn't have CRC errors.
Are you also on newsreader1.speedium.nl ?