About CRC Errors

Get help with all aspects of SABnzbd
Forum rules
Help us help you:
  • Are you using the latest stable version of SABnzbd? Downloads page.
  • Tell us what system you run SABnzbd on.
  • Adhere to the forum rules.
  • Do you experience problems during downloading?
    Check your connection in Status and Interface settings window.
    Use Test Server in Config > Servers.
    We will probably ask you to do a test using only basic settings.
  • Do you experience problems during repair or unpacking?
    Enable +Debug logging in the Status and Interface settings window and share the relevant parts of the log here using [ code ] sections.
Nemesh
Newbie
Newbie
Posts: 11
Joined: April 23rd, 2009, 4:13 am

Re: About CRC Errors

Post by Nemesh » June 23rd, 2010, 3:56 am

A bad .NZB will also do the trick.
At this moment a .NZB generated by Binsearch will give therse errors, while the same .NZB generated bij Yabse works fine -> no CRC errors.
Last edited by Nemesh on June 23rd, 2010, 4:07 am, edited 1 time in total.

colson22
Newbie
Newbie
Posts: 2
Joined: June 9th, 2010, 7:34 pm

Re: About CRC Errors

Post by colson22 » June 23rd, 2010, 12:13 pm

I just wanted to post an update to this issue.

It appears that the issue was VirtualBox and EXT4.  Under high load using EXT4 in VirtualBox the files being written to the volume would get corrupted.  After installing a fresh copy of Ubuntu Server 10.04 on EXT3, the problem has disappeared.  I have also setup a new Physical server with Ubuntu Server 10.04 on EXT4 and have had NO CRC errors while using SabNzbd.  It must be a combination of Virtualbox and EXT4.

Thanks for the great software and your help,

Frank

psycik
Jr. Member
Jr. Member
Posts: 65
Joined: September 28th, 2009, 2:33 pm

Re: About CRC Errors

Post by psycik » July 5th, 2010, 3:57 pm

While i have no issue with what's causing it, understand it will happen ever so often, what I'm finding annoying is when getting CRC errors, the par files are deleted. 

I have quickpar on my system as well, fo sorting this out, but when I see one of these CRC error files, I then have to go and redownload all of the par files.

How can I keep the par files from being deleted until AFTER a successful unpack?

User avatar
shypike
Administrator
Administrator
Posts: 21005
Joined: January 18th, 2008, 12:49 pm

Re: About CRC Errors

Post by shypike » July 6th, 2010, 2:17 am

psycik wrote: How can I keep the par files from being deleted until AFTER a successful unpack?
See: http://wiki.sabnzbd.org/configure-switches-v2 item "Enable Par cleanup".

psycik
Jr. Member
Jr. Member
Posts: 65
Joined: September 28th, 2009, 2:33 pm

Re: About CRC Errors

Post by psycik » July 6th, 2010, 3:29 am

shypike wrote:
psycik wrote: How can I keep the par files from being deleted until AFTER a successful unpack?
See: http://wiki.sabnzbd.org/configure-switches-v2 item "Enable Par cleanup".
yeah this is the weirdness.  That is turned on.  So would have expected one of these jobs - with a crc error to KEEP the par files.  Turning it off turning it off for everything, getting annoying really quick.

Does the unrar not return an error code that tells the program its been successful or not? BTW on .5.2 on windows xp (just upgraded to .5.3 today)

User avatar
shypike
Administrator
Administrator
Posts: 21005
Joined: January 18th, 2008, 12:49 pm

Re: About CRC Errors

Post by shypike » July 6th, 2010, 10:06 am

psycik wrote: Does the unrar not return an error code that tells the program its been successful or not? BTW on .5.2 on windows xp (just upgraded to .5.3 today)
The par2 files are deleted before unrar runs.
If par2 says that the files verify properly, there's no use in keeping those files.
The only way out is to use a user script that removes the par2 files.
The script will receive a status code for the success of par2 and unrar.
See: http://wiki.sabnzbd.org/user-scripts

tpas
Newbie
Newbie
Posts: 7
Joined: September 29th, 2010, 2:41 pm

Re: About CRC Errors

Post by tpas » September 29th, 2010, 2:56 pm

Well, after days of research I can add a new cause of crc errors! For some reason either sabnzbd or python can't deal properly with raid 5 arrays on Linux. I tried at least 10 different downloads and all failed because of crc errors. Then I downloaded the same  again on a Windows system, wihout errors.
After that I tried changing all kinds of settings on my Linux sytem, all without result.
Finally I completely reinstalled Ubuntu (10.4), again wihout result.
As a last try I installed an extra single harddisk in my Linux system and let sabnzbd dowload on that disk: it worked!!!
The same downloads again on my raid array: crc erors again....
Extra info: The raidarray consists of 3 WD 1.5TB disks and is built in mdadm.

User avatar
shypike
Administrator
Administrator
Posts: 21005
Joined: January 18th, 2008, 12:49 pm

Re: About CRC Errors

Post by shypike » September 29th, 2010, 4:48 pm

tpas wrote: For some reason either sabnzbd or python can't deal properly with raid 5 arrays on Linux.
RAID systems are handled 100% by the operating system.
It's not like applications need to do something special to support them.
I'd sooner doubt the OS and/or the disk array.
par2 and unrar are quite capable of giving the disk system a good exercise.
BTW: the influence of SABnzbd is low.
par2 is responsible for verifying the downloaded files.
If it says they're OK, SABnzbd just launches unrar and lets it do its job.

tpas
Newbie
Newbie
Posts: 7
Joined: September 29th, 2010, 2:41 pm

Re: About CRC Errors

Post by tpas » September 30th, 2010, 6:50 am

shypike wrote:
RAID systems are handled 100% by the operating system.
It's not like applications need to do something special to support them.
I'd sooner doubt the OS and/or the disk array.
Yeah, that was off course my first impression too.  I did very many extensive disk tests with the raid array, and it passed them all.
shypike wrote: par2 and unrar are quite capable of giving the disk system a good exercise.
BTW: the influence of SABnzbd is low.
par2 is responsible for verifying the downloaded files.
If it says they're OK, SABnzbd just launches unrar and lets it do its job.
That's something I tested also. I downloaded a file on the raid array and let sabnzbd do the whole job, e.g. par2 and rar. par2 was necessary and failed, so rar didn't start.
Then I downloaded on the single (not raid) disk and copied all the rar and par2 files to the raidarray and did the postprocessing "by hand". Then par2 wasn't necessary and rar unpacked flawlessly. So I still suspect the download-part messing things up, and not the par2 or rar part.

User avatar
shypike
Administrator
Administrator
Posts: 21005
Joined: January 18th, 2008, 12:49 pm

Re: About CRC Errors

Post by shypike » September 30th, 2010, 9:13 am

tpas wrote: So I still suspect the download-part messing things up, and not the par2 or rar part.
SABnzbd and the underlying Python don't know whether they operate on a simple disk or on a disk array.
So when things don't work on a disk array but do work on a single disk, how can this
be blamed on SABnzbd and/or Python?
I wouldn't know where to begin looking for such an issue.

tpas
Newbie
Newbie
Posts: 7
Joined: September 29th, 2010, 2:41 pm

Re: About CRC Errors

Post by tpas » September 30th, 2010, 10:29 am

shypike wrote:
I wouldn't know where to begin looking for such an issue.
I hear you.... I will try and check my logfile tonight, to see if I can find any indication. Maybe this thread helps?

http://forums.sabnzbd.org/index.php?topic=4150.0

User avatar
shypike
Administrator
Administrator
Posts: 21005
Joined: January 18th, 2008, 12:49 pm

Re: About CRC Errors

Post by shypike » September 30th, 2010, 11:01 am

tpas wrote: Maybe this thread helps?
http://forums.sabnzbd.org/index.php?topic=4150.0
It's about the same complaint.
For me it's still an OS problem, or maybe an error in the Python run-time library.
The assumption of SABnzbd is that the OS and Python are OK.
Have you looked for similar complaints in the Ubuntu forums?
Python is supported by Canonical and is an essential tool for Ubuntu.
Of course, I cannot prove that the error is not in SABnzbd,
but like I said before applications have no need to something special for raid.
Apart from that, we simply don't have the hardware to test it.
We already have several VMWare images to do some basic compatibility tests.

About the only thing I can imagine is that the size of the disk confuses the
Python run-time. It could be that there is some magic number (2T ?) above which
things go wrong.
We've fixed SABnzbd in the past because it used an older method to request
the disk size, which had problems with anything above 1T (I forgot what is was exactly).
Last edited by shypike on September 30th, 2010, 11:06 am, edited 1 time in total.

tpas
Newbie
Newbie
Posts: 7
Joined: September 29th, 2010, 2:41 pm

Re: About CRC Errors

Post by tpas » October 1st, 2010, 1:41 am

OK. Thanks for all your replies.
On the seaparte disk it works now, but I don't like the idea of possibly having an unstable raid-array (nerver noticed it though......)
I will try to change python-versions tonight, to see if THAT brings anything. Which version of python do you recommend?

User avatar
shypike
Administrator
Administrator
Posts: 21005
Joined: January 18th, 2008, 12:49 pm

Re: About CRC Errors

Post by shypike » October 1st, 2010, 3:02 am

tpas wrote: I will try to change python-versions tonight, to see if THAT brings anything. Which version of python do you recommend?
Most testing is done on Python 2.6.x where X is the latest supported by Ubuntu and ActiveState (for Windows).
As an extra test you could try to run a large par2 repair followed by an urar,
from the command line, without SABnzbd.
par2 and unrar are notorious disk bashers.
Also you can try to download stuff without post-processing and par and unar
like described above outside of SABnzbd.
That should give you some insight in the reliability of the disk array and might give
you clues about where the problem is located.

BTW: do you use a multi-code version of par2?

tpas
Newbie
Newbie
Posts: 7
Joined: September 29th, 2010, 2:41 pm

Re: About CRC Errors

Post by tpas » October 1st, 2010, 9:24 am

shypike wrote:
Most testing is done on Python 2.6.x where X is the latest supported by Ubuntu and ActiveState (for Windows).
That's the one installed in my distro....
shypike wrote: As an extra test you could try to run a large par2 repair followed by an urar,
from the command line, without SABnzbd.
par2 and unrar are notorious disk bashers.
Also you can try to download stuff without post-processing and par and unar
like described above outside of SABnzbd.
That should give you some insight in the reliability of the disk array and might give
you clues about where the problem is located.
Did that... When the download has been succesfull (without crc failures) the commandline par2 and unrar work perfectly, when there were failures par2 sometimes is able to correct, sometimes not. The disk seems to work OK.
shypike wrote:
BTW: do you use a multi-code version of par2?
I'm not sure what you mean...

Locked