Missing articles & repair files on 1st priority server, 2nd not used automatically

Get help with all aspects of SABnzbd
Forum rules
Help us help you:
  • Are you using the latest stable version of SABnzbd? Downloads page.
  • Tell us what system you run SABnzbd on.
  • Adhere to the forum rules.
  • Do you experience problems during downloading?
    Check your connection in Status and Interface settings window.
    Use Test Server in Config > Servers.
    We will probably ask you to do a test using only basic settings.
  • Do you experience problems during repair or unpacking?
    Enable +Debug logging in the Status and Interface settings window and share the relevant parts of the log here using [ code ] sections.
mysabqtaa20
Newbie
Newbie
Posts: 8
Joined: May 26th, 2020, 6:31 am

Missing articles & repair files on 1st priority server, 2nd not used automatically

Post by mysabqtaa20 »

Hello,

my setup:
- SABnzbd 2.3.9 win64 Standalone
- I have two providers: an unlimited with low retention and a limited with high retention time, which I added to sabnzbd. Unlimited account with priority 1, limited account with priority 2.

When I now try to download something, which isn't complete on my first provider, the download fails in the end (missing articles / malformed / not enough repair blocks). Now I have to disable my 1st priority server and I can try two things:
(1) retry the download with the original nzb-file as "additional nzb" on retry-dialog - sometimes that works.
(2) delete the old download and retry the whole thing from my 2nd provider - works.

As far as I understood the "priority" settings, sabnzbd would automatically try the 2nd priority server, when it hits missing articles etc., but it doesn't...?

Is my understanding completely off, did I forget to change another option or is this how it is supposed to be?

Thanks in advance.

User avatar
safihre
Administrator
Administrator
Posts: 3658
Joined: April 30th, 2015, 7:35 am
Location: Switzerland
Contact:

Re: Missing articles & repair files on 1st priority server, 2nd not used automatically

Post by safihre »

That depends, on the missing download, what does it show in the details?
X articles missing
or
X articles corrupt?

mysabqtaa20
Newbie
Newbie
Posts: 8
Joined: May 26th, 2020, 6:31 am

Re: Missing articles & repair files on 1st priority server, 2nd not used automatically

Post by mysabqtaa20 »

Hello,

thanks for getting back to me!
I only deleted the "Servers: xxx" line at code-logs.

First test:
(1) (both provider active) After adding the nzb the first time:
- During the download, the status bar already shows "x MB missing articles"
-->

Code: Select all

Failed with message: Age: 48d # 23 articles were malformed # Aborted, cannot be completed
(2) (both provider active) After pressing retry without adding nzb again or changing anything:
- switches to Download-Area again, starts at ~50%, shows increasing number of x MB missing articles and failes with message:
-->

Code: Select all

Age: 48d # 22 articles were malformed
Repair
[6r7c2e7c2y6j29fp7p7m] Verified in 2 seconds, all files correct
[6r7c2e7c2y6j29fp7p7m] Repair failed, not enough repair blocks (117 short)
(3) turning provider off/on and adding nzb as additional doesn't help. -> Failed overall.

Second test: (differs at (2))
(1) (both provider active) After adding the nzb the first time:
- During the download, the status bar already shows "x MB missing articles"
-->

Code: Select all

Failed with message: Age: 48d # 23 articles were malformed # Aborted, cannot be completed
(2)(turn 1st priority server off, only 2nd priority server still active)
- after pressing "retry" and adding original nzb as additional nzb: it switches to Download-Area again, starts at ~50%, but doesn't show no any missing article anymore, but fails with message:
-->

Code: Select all

Age: 48d
Repair
[6r7c2e7c2y6j29fp7p7m] Repair failed, not enough repair blocks (51 short)
Third test and only 2nd priority server active - Download finishes without any problem.

-----
Do you see a pattern?
Unfortunately, I've only this one nzb-file right now to test, but I had "missing article" messages as well some time ago. I didn't save the nzb - my bad.

Btw: Isn't there an option "download malformed articles again from different server" oder something like this?

Thanks in advance!

User avatar
safihre
Administrator
Administrator
Posts: 3658
Joined: April 30th, 2015, 7:35 am
Location: Switzerland
Contact:

Re: Missing articles & repair files on 1st priority server, 2nd not used automatically

Post by safihre »

When articles are malformed, we don't try to fetch them from a different server because usually these articles just failed to upload correctly.
Seems that's not the case here.. We don't have an option to retry malformed ones on a different server, but maybe I should..

mysabqtaa20
Newbie
Newbie
Posts: 8
Joined: May 26th, 2020, 6:31 am

Re: Missing articles & repair files on 1st priority server, 2nd not used automatically

Post by mysabqtaa20 »

safihre wrote:
May 26th, 2020, 10:05 am
We don't have an option to retry malformed ones on a different server, but maybe I should..
That would be great! Thank you!

Just two more questions:
(1) Why does it say "x MB missing articles" during the download? Aren't these "missing MB" the reason for the malformation?
(2) If articles would be really missing, sabnzbd should try to download them from another or lower priority server, right? If that happens, I would revive this thread.

User avatar
safihre
Administrator
Administrator
Posts: 3658
Joined: April 30th, 2015, 7:35 am
Location: Switzerland
Contact:

Re: Missing articles & repair files on 1st priority server, 2nd not used automatically

Post by safihre »

Indeed, there is some crossing of terms during and after the downloading dat can cause some confusion, sorry about that!
The missing articles are "missing" from the final files, but in this specific NZB they are not "missing" from the server but corrupt.
Indeed if the server returned missing-status-codes, we would try the second servers.

mysabqtaa20
Newbie
Newbie
Posts: 8
Joined: May 26th, 2020, 6:31 am

Re: Missing articles & repair files on 1st priority server, 2nd not used automatically

Post by mysabqtaa20 »

Thanks for clarifying!
Perhaps the following wording would be better: "x MB missing chunks, Y corrupted parts" suited.

I would really appreciate it, if SABnzbd could download these missing chunks (or if that's not possible the whole corrupted part) from the 2nd server in the future! Thanks for opening a request @ Github: #1466 !

User avatar
safihre
Administrator
Administrator
Posts: 3658
Joined: April 30th, 2015, 7:35 am
Location: Switzerland
Contact:

Re: Missing articles & repair files on 1st priority server, 2nd not used automatically

Post by safihre »

After this post I did some digging when we removed it and how other clients handle it.
I decided to restore the old behavior that articles with CRC errors will be retried.

mysabqtaa20
Newbie
Newbie
Posts: 8
Joined: May 26th, 2020, 6:31 am

Re: Missing articles & repair files on 1st priority server, 2nd not used automatically

Post by mysabqtaa20 »

Hello,

first of all: thanks for re-adding the feature at all.
Using the old version, I got these "MB missing articles" messages again, so I switched to the newer version:

I tried the 3.0B2 and I got the following result with both server active:

Code: Select all

[FCdxxxxxxx] Verified in 14 seconds, all files correct
[FCdxxxxxxx] Repair failed, not enough repair blocks (728 short)
Yes, during the download, no "MB missing articles" was shown and server 2 was used to download some parts/articles. So it is definitely better than before.
When I turn off server 1, just using server 2, no problem at all. Should it work like this?

And I run into a bug, which isn't in the issue list on github:
When I try to "retry" the failed download with only server 2 active and I add the nzb file again on retry-dialog, I get the following error message and trace:

Code: Select all

Traceback (most recent call last):
  File "lib\site-packages\cherrypy\_cprequest.py", line 638, in respond
  File "lib\site-packages\cherrypy\_cprequest.py", line 697, in _do_respond
  File "lib\site-packages\cherrypy\lib\encoding.py", line 219, in __call__
  File "lib\site-packages\cherrypy\_cpdispatch.py", line 54, in __call__
  File "sabnzbd\interface.py", line 118, in internal_wrap
  File "sabnzbd\interface.py", line 432, in retry_pp
  File "sabnzbd\api.py", line 1435, in retry_job
  File "sabnzbd\nzbqueue.py", line 180, in repair_job
  File "sabnzbd\__init__.py", line 678, in add_nzbfile
TypeError: a bytes-like object is required, not 'NoneType'

User avatar
safihre
Administrator
Administrator
Posts: 3658
Joined: April 30th, 2015, 7:35 am
Location: Switzerland
Contact:

Re: Missing articles & repair files on 1st priority server, 2nd not used automatically

Post by safihre »

Which providers do you use? And could you send me the NZB at [email protected]?
Then I can try to reproduce.

mysabqtaa20
Newbie
Newbie
Posts: 8
Joined: May 26th, 2020, 6:31 am

Re: Missing articles & repair files on 1st priority server, 2nd not used automatically

Post by mysabqtaa20 »

First server is "Speedium" and second server is Newshosting. Yes, as one of the pull request answers guessed right, Speedium is running in beta mode right now, but this gives SABnzbd also a chance for rare errors. But 2nd: if these errors are too irregular and not worth the time or effort, it's fine by me.

I sent you a link via PM, hope this works as well.
Last edited by mysabqtaa20 on June 5th, 2020, 5:29 am, edited 1 time in total.

User avatar
safihre
Administrator
Administrator
Posts: 3658
Joined: April 30th, 2015, 7:35 am
Location: Switzerland
Contact:

Re: Missing articles & repair files on 1st priority server, 2nd not used automatically

Post by safihre »

Thanks! Going to investigate!

User avatar
sander
Release Testers
Release Testers
Posts: 6985
Joined: January 22nd, 2008, 2:22 pm

Re: Missing articles & repair files on 1st priority server, 2nd not used automatically

Post by sander »

FWIW: I have a Speedium too, and I've now made it my primary newsserver account on my download machine. So far no tracebacks, although very slow downloads on old posts (backfilling?): 1 MB/s, instead of 10 MB/s

@safihre if you want, I can share the Speedium account with you.

mysabqtaa20
Newbie
Newbie
Posts: 8
Joined: May 26th, 2020, 6:31 am

Re: Missing articles & repair files on 1st priority server, 2nd not used automatically

Post by mysabqtaa20 »

sander wrote:
June 5th, 2020, 2:21 am
So far no tracebacks, although very slow downloads on old posts (backfilling?): 1 MB/s, instead of 10 MB/s
No slowdown on my end with 20 connections (maximum by provider), but my max speed is only 3MB/s. Either it's full speed or errors for me.

Btw: During beta phase anyone can register and use Speedium for free. No need for account sharing :).

Thanks for investigating!

User avatar
safihre
Administrator
Administrator
Posts: 3658
Joined: April 30th, 2015, 7:35 am
Location: Switzerland
Contact:

Re: Missing articles & repair files on 1st priority server, 2nd not used automatically

Post by safihre »

Strange, I made the same setup as you and it completed fine.
But then I also didn't have CRC errors.
Are you also on newsreader1.speedium.nl ?

Post Reply