Reading "incomplete" NZB files fails

[FlyveHest]

I am an extensive user of the NZB blackhole feature, saving NZBs locally, and then moving them over a (slow) WAN link to my SABnzbd server.

But, if SABnzbd tries to read the files, while they are being transmitted, they fail, and SABnzbd does not try to read them again. (Renaming the files when they are done being moved, makes them submit correctly)

Would it be possible to add a more robust file-lock check before trying to parse the files? (A possible fix, on the clientside of this, would be to move them using a .nzbmoving extension, and then renaming them when done, but I think SABnzbd should check if files are locked instead)

[shypike]

SABnzbd already checks to see if the attributes of the file are stable for 1 second.
Checking for file locking would be nice, but this is very much platform dependent
and difficult to implement reliably over all platforms.

In fact, the error lies with the writing process, it should lock the file during writing.

An alternative would be to use the API functions to send the NZB through the web-interface.

[FlyveHest]

An alternative would be to use the API functions to send the NZB through the web-interface.

It would, but, I "cache" NZBs locally, and they are then transferred to the SABnzbd server when it is running.

I just changed my local routine to rename them before moving them, and renaming then back when they are all moved, this should solve the problem.

[FlyveHest]

I just fell over this Python implementation of file locking / checking that might be useful for this problem

http://aspn.activestate.com/ASPN/Cookbo ... cipe/65203

[shypike]

I'd rather use poor-man's locking, i.e. renaming the file before using it.
I'm note sure it will work correctly on Posix.
Will look into it.

I'm no fan of implementing over-complex solutions for this particular problem.
If people/programs have problems creating proper black-hole compatible files,
they should use the other ways to feed the NZB files.

[FlyveHest]

I'd rather use poor-man's locking, i.e. renaming the file before using it.

I've done this, and it works, but maybe it should be mentioned in a FAQ / the docs, so people are aware of the potential problems.

I'm no fan of implementing over-complex solutions for this particular problem.
If people/programs have problems creating proper black-hole compatible files, they should use the other ways to feed the NZB files.

For me, this is by far the easiest way (as I queue up NZBs on one machine, and download on another, thats not online 24/7), and the "program" that have problems creating proper black-hole files in this instance (at least, for me), is Windows itself, and thats pretty hard to fix (I have no idea how Windows uses locks when files are being copied, but moving a 2mb file over a slow WAN link creates the problem each and every time, and I'm still pretty sure its because SAB grabs hold of the file before Windows are done moving it)

[shypike]

An alternative could be to retry the file later if it turns out to be incomplete.
You can see that an XML file is complete or not (it should contain a field).

Now SABnzbd puts a failed file into an internal ignore-list and never looks at it again (until a restart).

It should only be done in combination with a relatively long interval. Reading in XML
is relatively expensive.