0.7.x more missing data blocks than nzbget

Questions and bug reports for Beta releases should be posted here.
Forum rules
Help us help you:
  • Tell us what system you run SABnzbd on.
  • Adhere to the forum rules.
  • Do you experience problems during downloading?
    Check your connection in Status and Interface settings window.
    Use Test Server in Config > Servers.
    We will probably ask you to do a test using only basic settings.
  • Do you experience problems during repair or unpacking?
    Enable +Debug logging in the Status and Interface settings window and share the relevant parts of the log here using [ code ] sections.
Post Reply
thepainless
Newbie
Newbie
Posts: 3
Joined: January 3rd, 2015, 8:53 am

0.7.x more missing data blocks than nzbget

Post by thepainless »

Code: Select all

Version: 	0.7.x
Uptime: 	17h
Config File: 	/home/xxx/.sabnzbd/sabnzbd.ini
Used cache: 	Cached 0 articles (0 B)
Parameters: 	SABnzbd.py --config-file /home/xxx/.sabnzbd/sabnzbd.ini --logging 1 --daemon --pid /var/run/sabnzbd
Python Version: 	2.7.9 (default, Dec 25 2014, 21:45:52) [GCC 4.9.2]
i'm trying to download this nzb

the download fails with sabnzbd:

Code: Select all

Repair is required.
8 file(s) exist but are damaged.
24 file(s) are ok.
You have 1829 out of 2175 data blocks available.
You have 217 recovery blocks available.
Repair is not possible.
You need 129 more recovery blocks to be able to repair.
When I run sabnbd from terminal these errors appear:

Code: Select all

...
2015-01-03 15:08:35,082::INFO::[decoder:121] CRC Error in [email protected] (6BEFF437 -> 616D3873)
2015-01-03 15:08:35,085::INFO::[decoder:230] <Article: [email protected], bytes=793743, partnum=48, art_id=None> => missing from all servers, discarding
...
when I use the same nzb in nzbget or grabit (with wine) repair is possible:

Code: Select all

Repair is required.
8 file(s) exist but are damaged.
24 file(s) are ok.
You have 2165 out of 2175 data blocks available.
You have 216 recovery blocks available.
Repair is possible.
You have an excess of 206 recovery blocks.
10 recovery blocks will be used to repair.
missing articles cause more harm in sabnzbd than in nzbget or grabit?
Top
User avatar
sander
Release Testers
Release Testers
Posts: 8830
Joined: January 22nd, 2008, 2:22 pm

Re: 0.7.x more missing data blocks than nzbget

Post by sander »

Could it be time-based? Easy to check: run both downloads again.
Top
thepainless
Newbie
Newbie
Posts: 3
Joined: January 3rd, 2015, 8:53 am

Re: 0.7.x more missing data blocks than nzbget

Post by thepainless »

It's not time based. I tried several times today and always the same result.
I heard some other people with sabnzbd who can succesfully download with this nzb, so it's something with my install/version.
There are 8 damaged and 24 ok files on the server. When I download with sabnzbd I only get 1829 out of 2175 data blocks available, with nzbget 2165 out of 2175 data blocks available.

sabnzbd output

nzbget log:

Code: Select all

warning	Sun Jan 04 2015 15:04:45	File 5QIhnjfY4pPpLQTPWpo3MChnvJqxC7fpYp7H08qlynXkWHTL7clIohWgJSoSBxQaix.part25.rar has 1 bad block(s) of total 69 block(s)
warning	Sun Jan 04 2015 15:04:45	File 5QIhnjfY4pPpLQTPWpo3MChnvJqxC7fpYp7H08qlynXkWHTL7clIohWgJSoSBxQaix.part23.rar has 1 bad block(s) of total 69 block(s)
warning	Sun Jan 04 2015 15:04:45	File 5QIhnjfY4pPpLQTPWpo3MChnvJqxC7fpYp7H08qlynXkWHTL7clIohWgJSoSBxQaix.part19.rar has 1 bad block(s) of total 69 block(s)
warning	Sun Jan 04 2015 15:04:45	File 5QIhnjfY4pPpLQTPWpo3MChnvJqxC7fpYp7H08qlynXkWHTL7clIohWgJSoSBxQaix.part14.rar has 2 bad block(s) of total 69 block(s)
warning	Sun Jan 04 2015 15:04:45	File 5QIhnjfY4pPpLQTPWpo3MChnvJqxC7fpYp7H08qlynXkWHTL7clIohWgJSoSBxQaix.part12.rar has 1 bad block(s) of total 69 block(s)
warning	Sun Jan 04 2015 15:04:45	File 5QIhnjfY4pPpLQTPWpo3MChnvJqxC7fpYp7H08qlynXkWHTL7clIohWgJSoSBxQaix.part09.rar has 1 bad block(s) of total 69 block(s)
warning	Sun Jan 04 2015 15:04:45	File 5QIhnjfY4pPpLQTPWpo3MChnvJqxC7fpYp7H08qlynXkWHTL7clIohWgJSoSBxQaix.part07.rar has 2 bad block(s) of total 69 block(s)
warning	Sun Jan 04 2015 15:04:44	File 5QIhnjfY4pPpLQTPWpo3MChnvJqxC7fpYp7H08qlynXkWHTL7clIohWgJSoSBxQaix.part01.rar has 1 bad block(s) of total 69 block(s)
warning	Sun Jan 04 2015 15:01:09	1 of 69 article downloads failed for "5QIhnjfY4pPpLQTPWpo3MChnvJqxC7fpYp7H08qlynXkWHTL7clIohWgJSoSBxQaix/5QIhnjfY4pPpLQTPWpo3MChnvJqxC7fpYp7H08qlynXkWHTL7clIohWgJSoSBxQaix.part25.rar"
warning	Sun Jan 04 2015 14:59:59	1 of 69 article downloads failed for "5QIhnjfY4pPpLQTPWpo3MChnvJqxC7fpYp7H08qlynXkWHTL7clIohWgJSoSBxQaix/5QIhnjfY4pPpLQTPWpo3MChnvJqxC7fpYp7H08qlynXkWHTL7clIohWgJSoSBxQaix.part23.rar"
warning	Sun Jan 04 2015 14:57:39	1 of 69 article downloads failed for "5QIhnjfY4pPpLQTPWpo3MChnvJqxC7fpYp7H08qlynXkWHTL7clIohWgJSoSBxQaix/5QIhnjfY4pPpLQTPWpo3MChnvJqxC7fpYp7H08qlynXkWHTL7clIohWgJSoSBxQaix.part19.rar"
warning	Sun Jan 04 2015 14:54:53	2 of 69 article downloads failed for "5QIhnjfY4pPpLQTPWpo3MChnvJqxC7fpYp7H08qlynXkWHTL7clIohWgJSoSBxQaix/5QIhnjfY4pPpLQTPWpo3MChnvJqxC7fpYp7H08qlynXkWHTL7clIohWgJSoSBxQaix.part14.rar"
warning	Sun Jan 04 2015 14:53:45	1 of 69 article downloads failed for "5QIhnjfY4pPpLQTPWpo3MChnvJqxC7fpYp7H08qlynXkWHTL7clIohWgJSoSBxQaix/5QIhnjfY4pPpLQTPWpo3MChnvJqxC7fpYp7H08qlynXkWHTL7clIohWgJSoSBxQaix.part12.rar"
warning	Sun Jan 04 2015 14:52:05	1 of 69 article downloads failed for "5QIhnjfY4pPpLQTPWpo3MChnvJqxC7fpYp7H08qlynXkWHTL7clIohWgJSoSBxQaix/5QIhnjfY4pPpLQTPWpo3MChnvJqxC7fpYp7H08qlynXkWHTL7clIohWgJSoSBxQaix.part09.rar"
warning	Sun Jan 04 2015 14:50:58	2 of 69 article downloads failed for "5QIhnjfY4pPpLQTPWpo3MChnvJqxC7fpYp7H08qlynXkWHTL7clIohWgJSoSBxQaix/5QIhnjfY4pPpLQTPWpo3MChnvJqxC7fpYp7H08qlynXkWHTL7clIohWgJSoSBxQaix.part07.rar"
warning	Sun Jan 04 2015 14:47:37	1 of 69 article downloads failed for "5QIhnjfY4pPpLQTPWpo3MChnvJqxC7fpYp7H08qlynXkWHTL7clIohWgJSoSBxQaix/5QIhnjfY4pPpLQTPWpo3MChnvJqxC7fpYp7H08qlynXkWHTL7clIohWgJSoSBxQaix.part01.rar"


For now I made a workaround for when this happens. This post-processing script will send the nzb to nzbget when verification and unpacking fails in sabnzbd. Yes it will download everything again, luckily I have an unlimited data plan :P

Code: Select all

#!/bin/sh
# $7: Status of post processing. 0 = OK, 1=failed verification, 2=failed unpack, 3=1+2, -1=failed download
# $2: The original name of the NZB file

sabnzbdnzbs="/media/Seagate/downloads/nzb"
nzbgetnzbs="/media/Seagate/downloads/nzbget/nzb"

if [ $7 -eq 3 ]; then
	zcat $sabnzbdnzbs/"$2".gz >$nzbgetnzbs/"$2"
fi
Top
User avatar
shypike
Administrator
Administrator
Posts: 19774
Joined: January 18th, 2008, 12:49 pm

Re: 0.7.x more missing data blocks than nzbget

Post by shypike »

That would be a) worrisome and b) potentially easy to fix.

Can you email one or more sample NZB files?
(Please include the URL of this message and which Usenet provider(s) you are using.)
Top
User avatar
shypike
Administrator
Administrator
Posts: 19774
Joined: January 18th, 2008, 12:49 pm

Re: 0.7.x more missing data blocks than nzbget

Post by shypike »

Got your email and looked at the logging again.
Do you have multiple servers?
Is the option Config->Switches->"Fail on yenc crc Errors" on?
If so, could you try again with the option off?

The option should make sure that when a primary server delivers a malformed article,
it will be retried on a backup server.
There's a catch: when the backup server doesn't have the article,
I suspect that the original one from the primary servers isn't used.
That would be a weak design.
OTOH there may be just a plain bug here.
SABnzbd should just use the malformed article when there is no backup server at all, but maybe it doesn't.
I'll dig into this. Can you answer the above questions?
Top
thepainless
Newbie
Newbie
Posts: 3
Joined: January 3rd, 2015, 8:53 am

Re: 0.7.x more missing data blocks than nzbget

Post by thepainless »

when I enable the backup block servers. part of the log:

Code: Select all

2015-01-02 18:36:15,311::INFO::[downloader:381] [email protected]:119: Initiating connection
2015-01-02 18:36:15,314::INFO::[downloader:381] [email protected]:119: Initiating connection
2015-01-02 18:36:15,316::INFO::[downloader:381] [email protected]:119: Initiating connection
2015-01-02 18:36:15,318::INFO::[downloader:381] [email protected]:119: Initiating connection
2015-01-02 18:36:15,321::INFO::[downloader:381] [email protected]:119: Initiating connection
2015-01-02 18:36:15,323::INFO::[downloader:381] [email protected]:119: Initiating connection
2015-01-02 18:36:15,326::INFO::[downloader:381] [email protected]:119: Initiating connection
2015-01-02 18:36:15,328::INFO::[downloader:381] [email protected]:119: Initiating connection
2015-01-02 18:36:16,569::INFO::[downloader:577] Connecting [email protected]:119 finished
2015-01-02 18:36:16,576::INFO::[downloader:577] Connecting [email protected]:119 finished
2015-01-02 18:36:16,656::INFO::[downloader:577] Connecting [email protected]:119 finished
2015-01-02 18:36:16,662::INFO::[downloader:577] Connecting [email protected]:119 finished
2015-01-02 18:36:16,668::INFO::[downloader:577] Connecting [email protected]:119 finished
2015-01-02 18:36:16,672::INFO::[downloader:577] Connecting [email protected]:119 finished
2015-01-02 18:36:16,794::INFO::[downloader:577] Connecting [email protected]:119 finished
2015-01-02 18:36:16,805::INFO::[downloader:577] Connecting [email protected]:119 finished
2015-01-02 18:36:56,406::INFO::[decoder:121] CRC Error in [email protected] (6BEFF437 -> 616D3873)
2015-01-02 18:36:56,414::INFO::[downloader:381] [email protected]:443: Initiating connection
2015-01-02 18:36:57,197::INFO::[downloader:577] Connecting [email protected]:443 finished
2015-01-02 18:37:01,717::INFO::[decoder:121] CRC Error in [email protected] (6BEFF437 -> 616D3873)
2015-01-02 18:37:01,725::INFO::[downloader:381] [email protected]:119: Initiating connection
2015-01-02 18:37:01,975::INFO::[downloader:577] Connecting [email protected]:119 finished
2015-01-02 18:37:04,655::INFO::[decoder:121] CRC Error in [email protected] (6BEFF437 -> 616D3873)
2015-01-02 18:37:04,668::INFO::[downloader:381] [email protected]:443: Initiating connection
2015-01-02 18:37:07,465::INFO::[__init__:919] Saving data for totals9.sab in /home/dweezil/.sabnzbd/admin/totals9.sab
2015-01-02 18:37:10,652::INFO::[downloader:577] Connecting [email protected]:443 finished
2015-01-02 18:37:14,224::INFO::[decoder:121] CRC Error in [email protected] (6BEFF437 -> 616D3873)
2015-01-02 18:37:14,227::INFO::[decoder:230] <Article: [email protected], bytes=793743, partnum=48, art_id=None> => missing from all servers, discarding
same result when I turn off "Fail on yenc crc Errors". I'll email a debug log of the download.
Top
User avatar
shypike
Administrator
Administrator
Posts: 19774
Joined: January 18th, 2008, 12:49 pm

Re: 0.7.x more missing data blocks than nzbget

Post by shypike »

Yes, there's a problem in SABnzbd.
Advice: disable the "CRC check" option.
Top
User avatar
shypike
Administrator
Administrator
Posts: 19774
Joined: January 18th, 2008, 12:49 pm

Re: 0.7.x more missing data blocks than nzbget

Post by shypike »

It isn't so bad as it looked.
The only difference between enabling and disabling the CRC check
is that when enabled: 1) the other servers are tried and 2) more missing articles are logged.
However, the bad data is still always written into the file.
When the data is actually bad, you'll see par2 starting to repair the files.
So conclusion: when all servers have a CRC error in the same article(s), you will end up with exactly the same downloaded files
independent of the setting of the CRC option.
The only difference is that the "missing" count will be different.
The CRC option is useful in the (less likely) scenario where one of the servers has the correct article.
Top
yannsab
Newbie
Newbie
Posts: 2
Joined: March 9th, 2015, 4:04 am

Re: 0.7.x more missing data blocks than nzbget

Post by yannsab »

Hi, I had exactly the same issue with SABnzbd 0.7.20
I upgraded to 0.7.20 and suddenly most of the download went corrupted and impossible to repair.
I used nzbget ... no pb .... tried again sabnzbd 0.7.20 same pb. So it is not time based.
I have only one server.
I revert back to 0.7.19 and the issue disappeared, could you please fix it for the next release ?
Thanks
Top
User avatar
shypike
Administrator
Administrator
Posts: 19774
Joined: January 18th, 2008, 12:49 pm

Re: 0.7.x more missing data blocks than nzbget

Post by shypike »

Can you email the NZB that you used to [email protected]
Please also add the URL of this message and the name of the Usenet service that you use.
Top
User avatar
shypike
Administrator
Administrator
Posts: 19774
Joined: January 18th, 2008, 12:49 pm

Re: 0.7.x more missing data blocks than nzbget

Post by shypike »

There are no differences between 0.7.19 and 0.7.20 that can explain your observations.
On which operating system do you use SABnzbd?
Top
yannsab
Newbie
Newbie
Posts: 2
Joined: March 9th, 2015, 4:04 am

Re: 0.7.x more missing data blocks than nzbget

Post by yannsab »

I am using it with windows server 2012 R2 and I am using Ngroups.NET
A colleague of mine at work noticed exactly the same issue and he is using a MAC and Ngroups.NET:
- if the post is complete : no pb at all
- but if the post needs repair SABnzbd 0.7.20 fails 100% where NZBget succeeds
I still notice it a little bit with 0.7.19 but much less frequently
Not at home this week, but will try to send you a NZB with this behavior
Thanks !
Top
User avatar
shypike
Administrator
Administrator
Posts: 19774
Joined: January 18th, 2008, 12:49 pm

Re: 0.7.x more missing data blocks than nzbget

Post by shypike »

An example NZB would be very welcome.
Hopefully it's not the provider that's a factor in this.
Top
Post Reply

Return to “Beta Releases”