Fix for invalidly formed NZB files (patch included)

[ev98]

I've recently had an issue with certain NZB files from a forum I frequent. Depending on the size of the NZB, the spooler software will deliver either a RAR or a raw NZB file. It's a pretty neat utility, save that I have to extract the RAR files before SABnzbd+ will accept the files into the blackhole. (The new GZ and ZIP support is awesome by the way!)

The main issue is that the spooler adds a custom DOCTYPE tag that contains square brackets. According to the built-in XML parser, this is an invalid token and the NZB file just disappears with an exception in the logfile. I'd thought about writing a separate utility to handle this case, but figured others might be using even more sketchily-written NZB spooler software and that the main program should at least have an option to handle things.

I've attached a patch that does a regex replace for the DOCTYPE tag in the NZB file. It's not exactly the most efficient way of handling things, but it seems to fix my specific issue. I've only tested it with a stub launcher - the entire SABnzbd+ app doesn't seem to want to compile under PyDev on Win32, which is probably my stupid fault for running Win32 anyways. Also, please don't shoot me if this code is crap, as I've only briefly looked at Python before; I'm more of a PHP/MySQL guy.

Other devs: Please let me know what you think - this is probably something that should be set with a global preference, but I didn't want to go changing code all over the place in the event that this was a "you suck and shouldn't waste your time" issue.

[shypike]

First quick reaction, sounds like somebody doesn't know how to write an XML file.
Go to the source, there's no excuse for delivering a NZB file that's non-conforming.
Newzbin spec is clean and simple XML, there's no reason for people not to follow it.

I may be wrong (and it's a Python/SABnzbd problem), so it would help if you sent an example NZB file to [email protected].

As for the RAR support in the black hole. I haven't seen it before, but it's something we can pick up later.
(As Python does not have built-in RAR support, it's more work than supporting ZIP and GZ).

[ev98]

Your quick reaction is correct: there's no issue that I can see with SABnzbd+'s handling of XML files. It just so happens to choke on the following tag, which is the second line in the NZB file:



SABnzbd+'s parser fails at the "[" character and trashes the NZB as a result, which I assume is because it's not within legit XML conformance. Other apps don't seem to have issues, which leads me to believe that they handle this situation by completely ignoring the DOCTYPE tag.

I would go to the source and try to push for change, but I don't really have the ear of the people who run it and I expect they'd tell me to go pound salt over what's essentially two characters in an XML file. It's just such an incredibly convenient resource, even with things not working 100%.

A utility like this would really suit a preprocessing script, but I figured I'd spend a few minutes in Python either way and try and get familiar with the code.

[shypike]

The only thing your source has to do is use this line:



I've tried a few XML utilities and they all balk at this incorrect line.
I see no reason to support sloppy developers and standards-breakers.

Besides, ignoring the tag means skipping the standard Python XML library and create our own parser.
Your solution may work, but where does it end?

Which source do you get your NZB's from?

[ev98]

I'd rather not post a link to the site here in case it breaks forum rules. Is there an email address or other place I can toss a link to?

[inpheaux]

If you're talking about nzbsrus, they're the same state of grey-but-ok-area as Newzbin, so you don't have to dance around it. Also, if you're talking about nzbsrus, we already know they're good at spewing out broken nzb's. PM me details of specifically which nzbs are malformed and I'll look into talking to their admins.

[ev98]

It's a different site entirely. PM'd you with the details.