[SOLVED] EVERYTHING = "WAIT xx seconds"

Get help with all aspects of SABnzbd
Forum rules
Help us help you:
  • Are you using the latest stable version of SABnzbd? Downloads page.
  • Tell us what system you run SABnzbd on.
  • Adhere to the forum rules.
  • Do you experience problems during downloading?
    Check your connection in Status and Interface settings window.
    Use Test Server in Config > Servers.
    We will probably ask you to do a test using only basic settings.
  • Do you experience problems during repair or unpacking?
    Enable +Debug logging in the Status and Interface settings window and share the relevant parts of the log here using [ code ] sections.
johnny121b
Newbie
Newbie
Posts: 7
Joined: November 9th, 2012, 10:28 pm

[SOLVED] EVERYTHING = "WAIT xx seconds"

Post by johnny121b »

Version: 0.7.20
OS: Linux
Install-type: Python source / PhaZe plugin
Skin (if applicable): Plush Gold
Firewall Software: None
Are you using IPV6? No
Is the issue reproducible? Ohhhyeah! I'm reproducing the daylights out of it.

I can manually add an NZB file and it will download correctly, but if the item is pushed to Sabnzbd, say via Dog or PFMonkey, it simply stalls in the queue, with "WAIT xx sec. trying to fetch NZB from https://......

I've seen mention on here of similar situations caused by Python problems, but I'll be honest- I don't understand how to implement the workarounds they're referencing. IF that's my only solution, I need a 'For Dummies' explanation.

The plugin reports using Python 2.7.10

Help?

SOLUTION:

Go into UnRAID and stop the Sabnzbd plugin
Edit /usr/local/Sabnzbd/sabnzbd.py
I used MidnightCommander "MC" because that's all I had access to (that I knew how to use)
I'm no programmer. I'm simply leaving this here to potentially save someone the work of re-finding the workaround. (i.e. I won't be able to help with questions/problems)
I (can) tell you this- it does matter WHERE you insert the lines of code. I placed it between the two lines that read:

import re

try:


Add the following lines. (inbetween the two statements quoted above worked for me)

Code: Select all

opt_out_of_certificate_verification = True
if opt_out_of_certificate_verification:
        try:
           import ssl
           ssl._create_default_https_context = ssl._create_unverified_context
        except:
           pass
After adding these lines, SAVE the code and start the plugin.

I can also tell you this- This change does not survive a reboot. If you restart your server, you'll have to re-do the change. I rarely restart my server, but just to make life less complicated, after making the edits, I used MC to SAVE a copy of the edited file to my cache drive- so I can simply copy the edited version back into memory IF I have to reboot. (Which goodness knows I will- since there's now a penalty for doing so)
Last edited by johnny121b on July 20th, 2015, 6:28 am, edited 2 times in total.
User avatar
shypike
Administrator
Administrator
Posts: 19774
Joined: January 18th, 2008, 12:49 pm

Re: EVERYTHING = "WAIT xx seconds" or "Trying to fetch NZB"

Post by shypike »

Possibly your indexers use invalid SSL certificates.
Since Python 2.7.10, certificates are validated by default.
You can disable this check by setting Config->Special->enable_https_verification off.
This is only possible when using release 0.8.0Alpha3
johnny121b
Newbie
Newbie
Posts: 7
Joined: November 9th, 2012, 10:28 pm

Re: EVERYTHING = "WAIT xx seconds" or "Trying to fetch NZB"

Post by johnny121b »

Thanks for the response. I patched it up using suggestions from another post here, modifying the .py code. It won't survive reboots, but it'll keep me going.

Any firm estimates on when 0.8.0Alpha3 will be ready for mainstream use?
User avatar
shypike
Administrator
Administrator
Posts: 19774
Joined: January 18th, 2008, 12:49 pm

Re: EVERYTHING = "WAIT xx seconds" or "Trying to fetch NZB"

Post by shypike »

Alpha4 this week probably.
Good Beta's within a month or so.
There won't be any more 0.7.x releases.
velkkor
Newbie
Newbie
Posts: 7
Joined: July 27th, 2015, 5:35 pm

Re: [SOLVED] EVERYTHING = "WAIT xx seconds"

Post by velkkor »

How would I go about putting the 0.8.0Alpha 3 on a QNAP nas? I'm having this issue as well. Apparently my Python updated to 2.7.3 on Friday and since then I can't fetch NZBs from any of my indexers. I tried using QPython 2.7.9 and have the same issue.

It's odd (to me) that it's doing this. Both https://api.nzbgeek.info and https://nzbs.in are showing up as valid SSL certs at https://www.digicert.com/help/.
User avatar
sander
Release Testers
Release Testers
Posts: 8811
Joined: January 22nd, 2008, 2:22 pm

Re: [SOLVED] EVERYTHING = "WAIT xx seconds"

Post by sander »

@velkkor, as this is your first post, I want to make sure you follow up, so please say something back.
velkkor
Newbie
Newbie
Posts: 7
Joined: July 27th, 2015, 5:35 pm

Re: [SOLVED] EVERYTHING = "WAIT xx seconds"

Post by velkkor »

No problem, I'm still following. Thanks.
User avatar
sander
Release Testers
Release Testers
Posts: 8811
Joined: January 22nd, 2008, 2:22 pm

Re: [SOLVED] EVERYTHING = "WAIT xx seconds"

Post by sander »

Good.

What is the exect message you get? Because the SSL indeeds seems to be OK, and I can't reproduce the problem with SAB 0.7.20 & python 2.7.9:

Code: Select all

2015-07-28 14:41:57,439::DEBUG::[rss:332] Running feedparser on https://api.nzbgeek.info/rss?num=50&r=needvigsubscription
2015-07-28 14:41:57,990::DEBUG::[rss:334] Done parsing https://api.nzbgeek.info/rss?num=50&r=needvigsubscription
2015-07-28 14:41:57,991::INFO::[rss:357] RSS Feed https://api.nzbgeek.info/rss?num=50&r=needvigsubscription was empty
And run ssl-checker.py:

Code: Select all

#!/usr/bin/env python
import urllib2
import sys
import platform
print "Python version:", platform.python_version()

url = sys.argv[1]
try:
   response = urllib2.urlopen(url, timeout=3)
   print url, "OK!"
except Exception as e: 
   print url, "Not good:", e
with

Code: Select all

$ python ssl-checker.py https://api.nzbgeek.info/
Python version: 2.7.9
https://api.nzbgeek.info/ OK!
Post output here:
velkkor
Newbie
Newbie
Posts: 7
Joined: July 27th, 2015, 5:35 pm

Re: [SOLVED] EVERYTHING = "WAIT xx seconds"

Post by velkkor »

I'll try running the scripts you mentioned and let you know what I see.

Basically I'm just getting the "WAIT XX sec / Trying to fetch NZB from https://nzbs.in/getnzb/blablahblah.nzb" or "WAIT XX sec / Trying to fetch NZB from https://api.nzbgeek.info/api?t=get&id=y ... ahblahblah.

**edit** I guess I can't run what you posted. I was thinking I just needed to start entering in all the commands (keep in mind my linux knowledge is incredibly limited) and that doesn't appear to be what I should have been doing:

Code: Select all

[~] # /usr/bin/env python
Python 2.7.9 (default, Mar 24 2015, 00:42:24)
[GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import urllib2
>>> import sys
>>> import platform
>>> print "Python Version:", platform.python_version ()
Python Version: 2.7.9
>>> url = sys.argv[1]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IndexError: list index out of range
>>> try:
... response = urllib2.urlopen(url, timeout=3)
  File "<stdin>", line 2
    response = urllib2.urlopen(url, timeout=3)
           ^
IndentationError: expected an indented block
I then thought about the IndentationError and tried again, using a tab:

Code: Select all

[~] # /usr/bin/env python
Python 2.7.9 (default, Mar 24 2015, 00:42:24)
[GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import urllib2
>>> import sys
>>> import platform
>>> print "Python version:", platform.python_version()
Python version: 2.7.9
>>> url = sys.argv[1]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IndexError: list index out of range
>>> try:
...     response = urllib2.urlopen(url, timeout=3)
...     print url, "OK!"
...
  File "<stdin>", line 4

    ^
SyntaxError: invalid syntax
>>> except Exception as e:
  File "<stdin>", line 1
    except Exception as e:
         ^
SyntaxError: invalid syntax
>>> except Exception as e:
  File "<stdin>", line 1
    except Exception as e:
         ^
SyntaxError: invalid syntax
>>>
Last edited by velkkor on July 28th, 2015, 9:47 am, edited 1 time in total.
User avatar
sander
Release Testers
Release Testers
Posts: 8811
Joined: January 22nd, 2008, 2:22 pm

Re: [SOLVED] EVERYTHING = "WAIT xx seconds"

Post by sander »

No, save that text in a file ssl-checker.py, and then run it like this:

Code: Select all

$ python ssl-checker.py 'https://api.nzbgeek.info/api?t=get&id=yaddayaddayadda&apikey=blahblahblah'
Python version: 2.7.9
https://api.nzbgeek.info/api?t=get&id=yaddayaddayadda&apikey=blahblahblah OK!

Code: Select all

$ python ssl-checker.py https://nzbs.in/
Python version: 2.7.9
https://nzbs.in/ OK!
... which is good.


Example of SSL-certificate problem:

Code: Select all

$ python ssl-checker.py  https://nzbindex.nl/
Python version: 2.7.9
https://nzbindex.nl/ Not good: hostname 'nzbindex.nl' doesn't match either of 'www.nzbindex.com', 'nzbindex.com'

EDIT:

If you want the interactive mode, do this:

Code: Select all

import urllib2
url='https://nzbs.in/'
urllib2.urlopen(url, timeout=3).read()[:40]

Result:

Code: Select all

$ python
Python 2.7.9 (default, Apr  2 2015, 15:33:21) 
[GCC 4.9.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import urllib2
>>> url='https://nzbs.in/'
>>> urllib2.urlopen(url, timeout=3).read()[:40]
'<!doctype html>\r\n <head>\r\n\t<meta http-eq'
... which is good.

Example of site with incorrect SSL-certificate:

Code: Select all

$ python
Python 2.7.9 (default, Apr  2 2015, 15:33:21) 
[GCC 4.9.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import urllib2
>>> url='https://nzbindex.nl/'
>>> urllib2.urlopen(url, timeout=3).read()[:40]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/urllib2.py", line 154, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python2.7/urllib2.py", line 431, in open
    response = self._open(req, data)
  File "/usr/lib/python2.7/urllib2.py", line 449, in _open
    '_open', req)
  File "/usr/lib/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 1240, in https_open
    context=self._context)
  File "/usr/lib/python2.7/urllib2.py", line 1194, in do_open
    h.request(req.get_method(), req.get_selector(), req.data, headers)
  File "/usr/lib/python2.7/httplib.py", line 1048, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib/python2.7/httplib.py", line 1088, in _send_request
    self.endheaders(body)
  File "/usr/lib/python2.7/httplib.py", line 1044, in endheaders
    self._send_output(message_body)
  File "/usr/lib/python2.7/httplib.py", line 888, in _send_output
    self.send(msg)
  File "/usr/lib/python2.7/httplib.py", line 850, in send
    self.connect()
  File "/usr/lib/python2.7/httplib.py", line 1269, in connect
    server_hostname=server_hostname)
  File "/usr/lib/python2.7/ssl.py", line 352, in wrap_socket
    _context=self)
  File "/usr/lib/python2.7/ssl.py", line 579, in __init__
    self.do_handshake()
  File "/usr/lib/python2.7/ssl.py", line 816, in do_handshake
    match_hostname(self.getpeercert(), self.server_hostname)
  File "/usr/lib/python2.7/ssl.py", line 271, in match_hostname
    % (hostname, ', '.join(map(repr, dnsnames))))
ssl.CertificateError: hostname 'nzbindex.nl' doesn't match either of 'www.nzbindex.com', 'nzbindex.com'
>>> 

HTH
velkkor
Newbie
Newbie
Posts: 7
Joined: July 27th, 2015, 5:35 pm

Re: [SOLVED] EVERYTHING = "WAIT xx seconds"

Post by velkkor »

Code: Select all

# python ssl-checker.py 'https://api.nzbgeek.info/api?t=get&id=cdcbe94b416d54c13e993c6ab4220f85&apikey=yaddayadda'
Python version: 2.7.9
https://api.nzbgeek.info/api?t=get&id=cdcbe94b416d54c13e993c6ab4220f85&apikey=yaddayadda Not good: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)>
User avatar
sander
Release Testers
Release Testers
Posts: 8811
Joined: January 22nd, 2008, 2:22 pm

Re: [SOLVED] EVERYTHING = "WAIT xx seconds"

Post by sander »

velkkor wrote:

Code: Select all

# python ssl-checker.py 'https://api.nzbgeek.info/api?t=get&id=cdcbe94b416d54c13e993c6ab4220f85&apikey=yaddayadda'
Python version: 2.7.9
https://api.nzbgeek.info/api?t=get&id=cdcbe94b416d54c13e993c6ab4220f85&apikey=yaddayadda Not good: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)>
(I've put your code in code brackets to make it more readable)

Wow ... fascinating. The problem must be locally on your device; your device stores the trused (root?) certificates, but does something weird. I'm not sure how to solve that.

Other solutions:
Get 0.8.0 alpha on your system. I don't know how to do that on a QNAP NAS.
... patch sabnzbd with the code posted. Let's go that path. But first a test:

Can you run this in the interactive shell of your python 2.7.9:

Code: Select all

import ssl
import urllib2
ssl._create_default_https_context = ssl._create_unverified_context
url='https://nzbindex.nl/'
urllib2.urlopen(url, timeout=3).read()[:40]
Example output:

Code: Select all

$ python
Python 2.7.9 (default, Apr  2 2015, 15:33:21) 
[GCC 4.9.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl
>>> import urllib2
>>> ssl._create_default_https_context = ssl._create_unverified_context
>>> url='https://nzbindex.nl/'
>>> urllib2.urlopen(url, timeout=3).read()[:40]
'<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML '
>>> 
(See? No more error message ;) )
velkkor
Newbie
Newbie
Posts: 7
Joined: July 27th, 2015, 5:35 pm

Re: [SOLVED] EVERYTHING = "WAIT xx seconds"

Post by velkkor »

Code: Select all

# python
Python 2.7.9 (default, Mar 24 2015, 00:42:24)
[GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl
>>> import urllib2
>>> ssl._create_defautl_https_context = ssl._create_unverified_context
>>> url='https://nzbindex.nl/'
>>> urllib2.urlopen(url, timeout=3) .read() [:40]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/share/CACHEDEV1_DATA/.qpkg/Python/lib/python2.7/urllib2.py", line 154, in urlopen
    return opener.open(url, data, timeout)
  File "/share/CACHEDEV1_DATA/.qpkg/Python/lib/python2.7/urllib2.py", line 431, in open
    response = self._open(req, data)
  File "/share/CACHEDEV1_DATA/.qpkg/Python/lib/python2.7/urllib2.py", line 449, in _open
    '_open', req)
  File "/share/CACHEDEV1_DATA/.qpkg/Python/lib/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/share/CACHEDEV1_DATA/.qpkg/Python/lib/python2.7/urllib2.py", line 1240, in https_open
    context=self._context)
  File "/share/CACHEDEV1_DATA/.qpkg/Python/lib/python2.7/urllib2.py", line 1197, in do_open
    raise URLError(err)
urllib2.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)>
One thing I'm noticing here, however:

In part of the troubleshooting (thinking it was a Python issue per some posts on nzbgeek.info) I saw the version of Python installed on my QNAP device was 2.7.3. They had suggested running 2.7.9. Perusing the QNAP forums I found a .qpkg for QPython2, which was 2.7.9. I installed that package and added it to my PATH.

Now, running python and checking the version output is showing 2.7.9. However, the information above is running Python scripts from .qpkg/Python, which is the 2.7.3 installation. QPython is installed in /.qpkg/QPython2. So I'm wondering why the version echo says 2.7.9 (the QPython version) but the scripts are running from the 2.7.3 installation.

I of course could be totally off base with this, particularly since I know nothing of Python and it would seem more like my default certificate store may no longer be present for some reason, but I wanted to point it out.
User avatar
sander
Release Testers
Release Testers
Posts: 8811
Joined: January 22nd, 2008, 2:22 pm

Re: [SOLVED] EVERYTHING = "WAIT xx seconds"

Post by sander »

Ouch.

Let's get back to the basic: what happens when you use

Code: Select all

url='https://www.google.nl/'
as url in the interactive script? Is https working at all?
velkkor
Newbie
Newbie
Posts: 7
Joined: July 27th, 2015, 5:35 pm

Re: [SOLVED] EVERYTHING = "WAIT xx seconds"

Post by velkkor »

Ok, I apologize. I had at least one space in the script that shouldn't have been there.

This is the actual output of the nzbindex.nl script in the interactive Python prompt:

Code: Select all

Python 2.7.9 (default, Mar 24 2015, 00:42:24)
[GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl
>>> import urllib2
>>> ssl._create_default_https_context = ssl._create_unverified_context
>>> url='https://nzbindex.nl/'
>>> urllib2.urlopen(url, timeout=3).read()[:40]
'<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML '
I tried it against https://api.nzbgeek.info/ and get this:

Code: Select all

Python 2.7.9 (default, Mar 24 2015, 00:42:24)
[GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl
>>> import urllib2
>>> ssl._create_default_https_context = ssl._create_unverified_context
>>> url='https://api.nzbgeek.info/'
>>> urllib2.urlopen(url, timeout=3).read()[:40]
'\n<!DOCTYPE html PUBLIC "-//W3C//DTD XHTM'
I tried plugging in the full URL that sabnzbd is waiting on (with my API key and everything) and the response was:

Code: Select all

'<?xml version="1.0" encoding="UTF-8"?>\n<'
Post Reply