0.7.0beta3 issue: connection drops and "missing" articles

[shypike]

I'm just as amazed by this as you are.
Just checking, you're now using the original 0.7.0b3 code,
with just this one thing disabled?
I mean, if this code is a problem why then only for your provider?
Weird, especially since the code is only useful for very rare
posts that violate Usenet standards.
Last question, which Python version are you using?

[Silverlock]

Just checking, you're now using the original 0.7.0b3 code,
with just this one thing disabled?

Yes. It's a clean install with those few lines deleted.

I mean, if this code is a problem why then only for your provider?

I know; it's weird. I've been trying various things in python this afternoon to see if I could come up with something that looked like it might be a problem. I thought it might be something to do with a trailing lf but I haven't seen anything in that regard. The reason I looked at that was the message 'server closed connection', which seemed to come up with a zero byte block read. Nothing on that front has shown up though.

What's really nagging at me is that the if shouldn't even be executed in virtually all cases. But the fact that removing it makes the problem go away would seem to indicate the opposite.

Last question, which Python version are you using?

This is what python reports when I start it up from the command line: Python 2.7.1+ (r271:86832, Apr 11 2011, 18:05:24). The same string is reported in the sabnzbd.log file, so no multiple installed versions by the looks of it.

[shypike]

I'll see if can come up with some scenarios that go wrong.
Your suggestion that a zero byte result leads to the "server disconnected" message
is correct, sharp observation.
The easiest solution is to remove the feature because it's for very rare posts.
Python 2.7 should be OK.
I'll keep you posted. Thanks for your test work so far.

[Silverlock]

I think I figured it out. Shouldn't the test in the if be against self.data, not just data? Or am I missing a bigger picture (since my python skills are only slightly better than 'read it like it's C#')?

Code: Select all

# See if incorrect newline-only was used
         # Do this as a special case to prevent using extra memory
         # for normal articles
         #                                      vvvvv
         if len(new_lines) == 1 and '\r' not in self.data:
             new_lines = self.data.split('\n')

For what it's worth, I changed a clean 0.7 install to use self.data, and it works.

[shypike]

Hangs head in deep shame

That's indeed the problem.
Yes, I did test this thing, but probably modified the code afterwards...
It probably was this first (when tested):

Code: Select all

        if len(new_lines) == 1:
            new_lines = self.data.split('\n')


And I later optimized it to:

Code: Select all

        if len(new_lines) == 1 and '\r' not in data:
            new_lines = self.data.split('\n')


And, hey, there's no need to test this simple change, is there

It's still a riddle why it goes wrong with your provider, but it doesn't matter that much.
When the code triggers, it runs into an exception, which is caught but will result in a "server disconnected" state.

Thank you very much for your support!

[EDIT]
I suspect that your provider sometimes sends single line article chunks,
because a single line chunk will trigger the crash.
It's a very good thing that you found this, because it's very likely that this can
happen with other providers too.

[Silverlock]

And, hey, there's no need to test this simple change, is there

Oh, if I only had a nickel for every time I've said that over the years (and another nickel for every time it's turned around and bit me)!

I suspect that your provider sometimes sends single line article chunks, because a single line chunk will trigger the crash.

That makes perfect sense. The 'and' is a short circuit operator, so unless it was a single line chunk the program would never get to the 'in' test.

Thank you very much for your support!

You're more than welcome. I'm glad the mystery is solved.