downloads never finish, stay in queue, files disappear 0.6.5.

[bluenote]

Hey guys

I recently set up a PVR for my dad, and for the most part its been working well.
What hasnt been working well, is, after upgrading from 0.5.x to 0.6.2? and now to 0.6.5 ,
the queue acts very strangely.  SOME items are process.  But others
stay in the queue, appear fully downloaded, but when I look at the drive the actual files are nowhere to be seen.
The log reflects this, complaining it can't find all the pieces of these various files.

Please help!

I've tried pastebin and uploading here and neither have big enough limits for a 3meg log.
any suggestions would be much appreciated!

I am attaching a picture of the queue since it demonstrates whats happening.
But cannot because upload folder is full.
Here's the pic hosted elsewhere:

[shypike]

Are these all new NZB's or were they in the queue before the upgrade?
Check Config->Switches item "Only Get Articles for Top of Queue".
It should be off.
Also, check whether you have multiple Usenet servers defined,
one of which isn't responding anymore.

Have you tried to restart SABnzbd?

[bluenote]

The queue was populated when I upgraded, but it doesn't seem to matter if I delete an item and re-add.
I do have the 'only get articles for top of queue' so that it isn't blocked by an item, so that's already off.
(I think the default is on though)
I do have multiple usenet servers, and I use the free ipv6 server which does go unresponsive a lot set up as one of two primaries.
I also have a paid fill server setup as backup.

This morning when I logged in, only 3 downloads had been moved to error status, and the whole rest of the screen
was filled with movies that were done (full bar of completed) but 'stuck'. 

I have restarted and not noticed a difference, I also had tried a queue repair yesterday, no luck.
I'll try restarting again though and see what happens.

I can't get on the box right this second but I think the commonality is that the underlying files have somehow been removed.
I dont know whether this is a post process gone wrong, or what though.

thanks for any more suggestions

[bluenote]

OK, I just tried a restart and all the 'stuck' items went to the history in error status (I guess because none of them had any files in their download directories)
I'm going to try re-adding them but every time it seems like the behaviour is the same. (I have deleted and readded, and re-tried several times now)
So Im still trying to figure out why all the files are missing.  For these items nothing ever ends up in the destination directory, no FAILED dir or anything like that.


thanks

Here's an example of the log when sab finds the files missing:

Code: Select all

2011-06-18 03:27:35,653::INFO::[__init__:786] C:\downloads\sabnzbd\Milk 2008 BRRip XvidHD 720p NPW.1\__ADMIN__\SABnzbd_nzf_ryrx_r missing

2011-06-18 03:27:35,653::ERROR::[nzbstuff:1003] Error importing <NzbFile: filename=None, type=None>

2011-06-18 03:27:35,654::INFO::[__init__:786] C:\downloads\sabnzbd\Milk 2008 BRRip XvidHD 720p NPW.1\__ADMIN__\SABnzbd_nzf_tsnp3a missing

2011-06-18 03:27:35,654::ERROR::[nzbstuff:1003] Error importing <NzbFile: filename=None, type=None>

2011-06-18 03:27:35,654::INFO::[__init__:786] C:\downloads\sabnzbd\Milk 2008 BRRip XvidHD 720p NPW.1\__ADMIN__\SABnzbd_nzf_6c7m9h missing

2011-06-18 03:27:35,654::ERROR::[nzbstuff:1003] Error importing <NzbFile: filename=None, type=None>

2011-06-18 03:27:35,654::INFO::[__init__:786] C:\downloads\sabnzbd\Milk 2008 BRRip XvidHD 720p NPW.1\__ADMIN__\SABnzbd_nzf_q2x3pc missing

2011-06-18 03:27:35,654::ERROR::[nzbstuff:1003] Error importing <NzbFile: filename=None, type=None>

2011-06-18 03:27:35,654::INFO::[__init__:786] C:\downloads\sabnzbd\Milk 2008 BRRip XvidHD 720p NPW.1\__ADMIN__\SABnzbd_nzf_rmwiau missing

2011-06-18 03:27:35,654::ERROR::[nzbstuff:1003] Error importing <NzbFile: filename=None, type=None>

2011-06-18 03:27:35,654::INFO::[__init__:786] C:\downloads\sabnzbd\Milk 2008 BRRip XvidHD 720p NPW.1\__ADMIN__\SABnzbd_nzf_xz8ui7 missing

2011-06-18 03:27:35,654::ERROR::[nzbstuff:1003] Error importing <NzbFile: filename=None, type=None>

2011-06-18 03:27:35,654::INFO::[__init__:786] C:\downloads\sabnzbd\Milk 2008 BRRip XvidHD 720p NPW.1\__ADMIN__\SABnzbd_nzf_achxqz missing

2011-06-18 03:27:35,654::ERROR::[nzbstuff:1003] Error importing <NzbFile: filename=None, type=None>

2011-06-18 03:27:35,655::INFO::[__init__:786] C:\downloads\sabnzbd\Milk 2008 BRRip XvidHD 720p NPW.1\__ADMIN__\SABnzbd_nzf_zodoh_ missing

2011-06-18 03:27:35,655::ERROR::[nzbstuff:1003] Error importing <NzbFile: filename=None, type=None>

2011-06-18 03:27:35,655::INFO::[__init__:786] C:\downloads\sabnzbd\Milk 2008 BRRip XvidHD 720p NPW.1\__ADMIN__\SABnzbd_nzf_mn8slv missing

2011-06-18 03:27:35,655::ERROR::[nzbstuff:1003] Error importing <NzbFile: filename=None, type=None>

2011-06-18 03:27:35,655::INFO::[__init__:786] C:\downloads\sabnzbd\Milk 2008 BRRip XvidHD 720p NPW.1\__ADMIN__\SABnzbd_nzf_uinnjf missing

2011-06-18 03:27:35,655::ERROR::[nzbstuff:1003] Error importing <NzbFile: filename=None, type=None>

2011-06-18 03:27:35,657::INFO::[__init__:786] C:\downloads\sabnzbd\Milk 2008 BRRip XvidHD 720p NPW.1\__ADMIN__\SABnzbd_nzf_y22rnn missing

2011-06-18 03:27:35,657::ERROR::[nzbstuff:1003] Error importing <NzbFile: filename=None, type=None>

2011-06-18 03:27:35,657::INFO::[__init__:786] C:\downloads\sabnzbd\Milk 2008 BRRip XvidHD 720p NPW.1\__ADMIN__\SABnzbd_nzf_ddjnlo missing

2011-06-18 03:27:35,657::ERROR::[nzbstuff:1003] Error importing <NzbFile: filename=None, type=None>

2011-06-18 03:27:35,657::INFO::[__init__:786] C:\downloads\sabnzbd\Million Dollar Baby 2004 720p BDRip AC3 x264 MacGuffin\__ADMIN__\SABnzbd_nzf_qmx4gk missing

2011-06-18 03:27:35,657::ERROR::[nzbstuff:1003] Error importing <NzbFile: filename=None, type=None>

2011-06-18 03:27:35,657::INFO::[__init__:786] C:\downloads\sabnzbd\Million Dollar Baby 2004 720p BDRip AC3 x264 MacGuffin\__ADMIN__\SABnzbd_nzf_wcjja9 missing

2011-06-18 03:27:35,657::ERROR::[nzbstuff:1003] Error importing <NzbFile: filename=None, type=None>

2011-06-18 03:27:35,657::INFO::[__init__:786] C:\downloads\sabnzbd\Million Dollar Baby 2004 720p BDRip AC3 x264 MacGuffin\__ADMIN__\SABnzbd_nzf_hymha6 missing

2011-06-18 03:27:35,657::ERROR::[nzbstuff:1003] Error importing <NzbFile: filename=None, type=None>

2011-06-18 03:27:35,657::INFO::[__init__:786] C:\downloads\sabnzbd\Million Dollar Baby 2004 720p BDRip AC3 x264 MacGuffin\__ADMIN__\SABnzbd_nzf_iz0eds missing

2011-06-18 03:27:35,658::ERROR::[nzbstuff:1003] Error importing <NzbFile: filename=None, type=None>

2011-06-18 03:27:35,658::INFO::[__init__:786] C:\downloads\sabnzbd\Million Dollar Baby 2004 720p BDRip AC3 x264 MacGuffin\__ADMIN__\SABnzbd_nzf_c7l7zt missing

2011-06-18 03:27:35,658::ERROR::[nzbstuff:1003] Error importing <NzbFile: filename=None, type=None>

2011-06-18 03:27:35,658::INFO::[__init__:786] C:\downloads\sabnzbd\Million Dollar Baby 2004 720p BDRip AC3 x264 MacGuffin\__ADMIN__\SABnzbd_nzf_3kqmg1 missing

2011-06-18 03:27:35,658::ERROR::[nzbstuff:1003] Error importing <NzbFile: filename=None, type=None>

2011-06-18 03:27:35,658::INFO::[__init__:786] C:\downloads\sabnzbd\Million Dollar Baby 2004 720p BDRip AC3 x264 MacGuffin\__ADMIN__\SABnzbd_nzf_6c4rko missing

2011-06-18 03:27:35,660::ERROR::[nzbstuff:1003] Error importing <NzbFile: filename=None, type=None>

[shypike]

It looks like SABnzbd cannot (or does not) save the job administration.
Can you see whether folders like C:\downloads\sabnzbd\Milk 2008 BRRip XvidHD 720p NPW.1
are being created?
If even those aren't there, then it's either "disk full" or a problem with access rights.

A numbers of items in the queue is actually almost done.
Didn't the download folders have any content? Are these folders present?

[bluenote]

The funny thing is, *some* downloads have gone through.
I can't seem to find the commonality though.
After I posted I deleted all directories inside the download folder and tried to start some again.
But what I remember being inside is only C:\downloads\sabnzbd\(movie name)\__ADMIN__ but nothing else.

Right now I am watching one download though, and files are populating (I see the .r00 .rar file etc).
So as best I can tell, at some point all these files are deleted.
I think this means permissions are fine?

[shypike]

If .R?? files are created, then access rights are OK.

If you pause the queue and add an NZB, the following should be created:
C:\downloads\sabnzbd\(movie name)
      empty
C:\downloads\sabnzbd\(movie name)\__ADMIN__
      SABnzbd_attrib
      SABnzbd_nzf_xxxxxx  A bunch of these
      SABnzbd_nzo_yyyyyy  One file
      (movie name).nzb.gz

[bluenote]

I just put a new download in, and it downloaded and finished correctly.

I will try re-adding the rest that were in my queue and see what happens.
I think two factors are : many downloads in the queue,
plus, one very unreliable main server which can be unreachable for hours or half a day at a time.
Still, downloaded files should not disappear so I guess there might be a bug somewhere in the process.
I'll post back as soon as I have more info,
thanks

[Charlos]

I have to be what pretty much is the same issue.
My SAB shows the movie as being completed, 0 Time Left.. Full Progress Bar.  But it stay in the top in the queue section instead of moving down to the history section.
Its not in the downloads folder or the destination folder for post processing completion on the NAS...
its like it doesn't know what to do with it once its finished the download so it just locks up.
It also only seems to happen with some files but seems to be happening more often than not lately.
I can't seem to figure out why some do and others don't either.

[shypike]

I have to be what pretty much is the same issue.

Does the problem occur for each NZB you enter?
Does a restart of SABnzbd help?
Please set the logging level to "Debug", and after a few minutes of stalling,
look in the log file. Particularly the end of it.

[bluenote]

Hey shypike

I managed to catch an example of this (and surprisingly there's some that have gone through, so it does seem to be intermittent).



All 4 have

  -> __ADMIN__
       --> (nzb gz file)
       --> SABnzbd_attrib
       --> SABnzbd_nzo_tskqn

and nothing else, no sign of download info.
If I restart at this point I believe they are moved to failed status, but before I do that I'll try switching to debug.

After a few minutes set to debug, nothing new appears in the log.

Code: Select all

2011-06-21 22:01:35,253::INFO::[newsunpack:200] Unrar finished on C:\downloads\sabnzbd\Deadliest.Catch.S07E11.Birds.Bones.and.Blood.720p.HDTV.x264-MOMENTUM

2011-06-21 22:01:35,253::INFO::[postproc:296] unpack_magic finished on Deadliest.Catch.S07E11.Birds.Bones.and.Blood.720p.HDTV.x264-MOMENTUM

2011-06-21 22:01:36,737::INFO::[newsunpack:132] Running external script C:\program files (x86)\sickbeard\autoprocesstv\sabToSickBeard.exe(D:\videos\new\TV\Deadliest.Catch.S07E11.Birds.Bones.and.Blood.720p.HDTV.x264-MOMENTUM, Deadliest.Catch.S07E11.Birds.Bones.and.Blood.720p.HDTV.x264-MOMENTUM.nzb, Deadliest.Catch.S07E11.Birds.Bones.and.Blood.720p.HDTV.x264-MOMENTUM, , tv, alt.binaries.teevee, 0)

2011-06-21 22:01:41,301::INFO::[downloader:610] Thread [email protected]:119: server closed connection

2011-06-21 22:01:42,348::INFO::[downloader:610] Thread [email protected]:119: server closed connection

2011-06-21 22:01:43,426::INFO::[downloader:610] Thread [email protected]:119: server closed connection

2011-06-21 22:01:43,697::INFO::[postproc:448] Cleaning up Deadliest.Catch.S07E11.Birds.Bones.and.Blood.720p.HDTV.x264-MOMENTUM (keep_basic=False)

2011-06-21 22:01:43,707::INFO::[postproc:83] Saving postproc queue

2011-06-21 22:01:43,707::INFO::[__init__:828] Saving data for postproc1.sab in C:\Users\jrn\AppData\Local\sabnzbd\admin\postproc1.sab

2011-06-21 22:01:43,709::INFO::[downloader:229] Post-processing finished, resuming download

2011-06-21 22:01:44,654::INFO::[downloader:610] Thread [email protected]:119: server closed connection

2011-06-21 22:01:45,114::INFO::[downloader:610] Thread [email protected]:119: server closed connection

2011-06-21 22:01:47,329::INFO::[downloader:610] Thread [email protected]:119: server closed connection

2011-06-21 22:01:47,414::INFO::[downloader:610] Thread [email protected]:119: server closed connection

2011-06-21 22:01:52,019::INFO::[downloader:610] Thread [email protected]:119: server closed connection

2011-06-21 22:01:53,858::INFO::[downloader:610] Thread [email protected]:119: server closed connection

2011-06-21 22:03:42,102::INFO::[downloader:610] Thread [email protected]:119: server closed connection

2011-06-21 22:03:44,012::INFO::[downloader:610] Thread [email protected]:119: server closed connection

2011-06-21 22:03:44,233::INFO::[downloader:610] Thread [email protected]:119: server closed connection

2011-06-21 22:03:45,391::INFO::[downloader:610] Thread [email protected]:119: server closed connection

2011-06-21 22:03:47,875::INFO::[downloader:610] Thread [email protected]:119: server closed connection

2011-06-22 02:00:00,000::INFO::[downloader:194] Resuming

Yes, after restarting they are all moved to error/failed status.


thanks for any suggestions!

[shypike]

Very likely a combination of out-of-retention and a silent server.
Please email one of the problematic NZB to bugs at sabnzbd.org
Which Usenet provider do you use?

[bluenote]

Nothing accounts for all the files being missing after it shows it has downloaded the whole movie though.
I haven't caught it in the act, but I believe that it does in fact download all that data but then it goes away.
I am using local ISP news (very reliable, short retention), + free ipv6 (unreliable connection), +paid backup fill server
I use the exact same setup at home (well, as exact as I can make it) and it all works perfectly, this never happens.
I grabbed a GZ file out of the empty download (_ADMIN_) folder, I believe this is the nzb for the example above.

thanks for your help

whups, can't attach .gz files, I will only email it.

EDIT: ipv6 is xsnews, and my fill server is blocknews

[bluenote]

One more note: I used the "retry" button, and watched the download directory, and I can now see the rar's appearing there.

[shypike]

Could be a temporary server problem.

However, this is one very bad post that subsequently isn't handled very well by SABnzbd.
This is a so-called par2-joinable set, meaning the poster cut up the RAR file
into segments AFTER generating the par2 files.
In addition to that he used a non-standard way of numbering the segments so that
both par2 and SABnzbd miss the fact that it needs special treatment.

After download completes, SABnzbd will claim that the post cannot be repaired.
You'll have to repair it manually by running the par2 program.
Open a command prompt, then type:
cd c:\path\to\job-folder
"c:\program files\sabnzbd\win\par2\par2.exe" r name-a-par2.par2 *

This will reconstruct one large RAR file.
After that you unrar the result:
"c:\program files\sabnzbd\win\unrar\unrar.exe" x name-of-rar.rar

We are working on a solution that will fix this issue (and some others).