downloads slow significantly when downloading across multiple usenet servers

[quixotic120]

hello,

my config works fine 90% of the time and I am able to fully saturate my gigabit internet connection (and then some), getting downloads of about 110MB/s and no issues with direct unpacking. However, when files are missing on my main server and backup servers are used to grab parts my connection speed drops to 5-10MB/s and I cannot figure out why.

this shows up in the log a lot, dozens or hundred of times:

([email protected] // [email protected])
2024-03-11 18:12:57,170::INFO::[nzbparser:437] Duplicate part 152, but different ID-s

things i've tried:
making each backup server primary and running test downloads, can saturate connection to ~100MB/s with all servers
limiting connections, no change
lowering timeout, no change, does not seem to be any timeouts looking at the logs
restarting container, no change
force disconnect, no change other than the progress goes down a bit

running sab 4.2.3 [e7e06de] in the linuxserver docker image on unraid

[safihre]

The log lines are not related to the download, but to the importing the NZB's.

Missing data will always be slow, since we first try the main server which has to tell us it's missing before we go to the next server. So it can't be full speed, since 2 requests have to be made that are sequential.

[quixotic120]

The log lines are not related to the download, but to the importing the NZB's.

Missing data will always be slow, since we first try the main server which has to tell us it's missing before we go to the next server. So it can't be full speed, since 2 requests have to be made that are sequential.

that makes sense but the download stays slow and eventually completes, should it not eventually recover in speed or is this expected behavior? the download that prompted the thread was a 55GB download with 149MB missing articles. What I am experiencing is that the download speed drops and stays at 5-10MBs for the entire 55GB download and does not recover until it moves on to another NZB. is this normal behavior then?

[safihre]

If it's onky available on the secondary server, then it might indeed be a bit slower. Although from 100 to 5 is quite a drop.

How many connections do you use for all your servers?

[quixotic120]

If it's onky available on the secondary server, then it might indeed be a bit slower. Although from 100 to 5 is quite a drop.

How many connections do you use for all your servers?

25-30 depending on server

what I am seeing in the connections tab on status is that the overwhelming majority of the job will be on my primary server and then 1-5 articles will be consistently grabbed on backup servers throughout the download. This is happening consistently with any download that has missing articles so it is reproducible and wasn't just the one NZB.

However, trying the same NZB with a different NZB downloader on my laptop but same server config downloads full speed throughout. Later today i'll try a fresh install of SAB onto my laptop (separate from the NAS this issue is occurring on) to see if it reproduces, I had just already had an old setup ready that could quickly test.

[safihre]

Every missing article should directly jump to the next server, or at least with minimal delay.
Strange.

[safihre]

What providers do you use?
Could you send an NZB to me at [email protected]?
I couldn't find a job that's complete on 1 provider but missing on the other.

[quixotic120]

What providers do you use?
Could you send an NZB to me at [email protected]?
I couldn't find a job that's complete on 1 provider but missing on the other.

servers set up as
usenetexpress
frugalusenet (omicron)
usenetexpress EU server
frugalusenet EU
blocknews, which is rarely touched

i don't believe it was complete on one but complete enough between the two, if that made sense? it still had to repair after finishing. Can send the nzb still if you'd like

[safihre]

Could you send the nzb to [email protected]?
Which servers were used for the NZB?

[quixotic120]

Could you send the nzb to [email protected]?
Which servers were used for the NZB?

sent the nzb

tried redownloading it now, downloads from usenetexpress primarily with some articles downloading from the EU usenetexpress server and the frugalusenet us server. However, when I attempted just now it downloaded at full speed. I deleted the download and restarted it several times and even tried a few other things that had caused the same issue. Perhaps this was an issue with one of the providers? If so sorry to waste your time! Very confused

edit: never mind, i must've just been confused about what NZBs it was occurring on, sorry. I tried downloading some more and found another where the problem occurs. I sent that NZB to you just now, sorry for two emails. they'll be coming from separate addresses as I use an email mask for outgoing email with stuff like this, hope thats not an issue, let me know if you need me to resend from something not hidden like that.

and can confirm - most articles are downloading via usenetexpress, occasionally some are downloading via frugalusenet and eu usenetexpress server. this seems to be occurring for each file within the set for the download - 25/25 connections open to usenet express at all times but 1-7 pop up for frugalnews and eu usenetexpress server. However, the download has stayed between 5-8MB/s for the duration whereas NZBs before it were fine and running ~100MB/s

also doubt its relevant but to correct frugalusenet switched backbones about 2 weeks ago and is no longer omicron, is now netnews. I don't see how that would be related but thought i would include it for completeness' sake and coincidentally is about as long as the problem has been occurring

[safihre]

I only got 1 email, with the Clockwork NZB.
Is that the right one?

[quixotic120]

I only got 1 email, with the Clockwork NZB.
Is that the right one?

no, thats incorrect, I resent from my gmail, sorry