MovieGrabber - Completely Automated Movie Downloader

[binhex]

Tried to start it with : python MovieGrabber.py

But see the following error ;

File "MovieGrabber.py", line 80, in <module>
import cherrypy
ImportError: No module named cherrypy

hmm ok i will need to try and spin up a copy of ubuntu in a vm and take a look at this as its def a incompatibility with some of the code and linux i will get back to you on monday.

[splnut]

Tried to install it on Unraid and got this.

root@Tower:/mnt/disk/sdc1/Downloader1# python MovieGrabber.py
Traceback (most recent call last):
File "MovieGrabber.py", line 79, in <module>
import cherrypy
ImportError: No module named cherrypy

I am currently running Sickbeard, Sabnzbd, CouchPotato, Headphones and your auto_movie_downloader script without any issues.

[binhex]

Tried to install it on Unraid and got this.

root@Tower:/mnt/disk/sdc1/Downloader1# python MovieGrabber.py
Traceback (most recent call last):
File "MovieGrabber.py", line 79, in <module>
import cherrypy
ImportError: No module named cherrypy

I am currently running Sickbeard, Sabnzbd, CouchPotato, Headphones and your auto_movie_downloader script without any issues.

Ok thanks I will get on this asap, have a good weekend :-)

[war59312]

Should post a screenshot on the first page.

Also the download link is no good for Chrome users.

Perhaps use http://www.datafilehost.com/ instead please ?

rascalli and splnut you both simply need to install cherrypy via:

Code: Select all

sudo apt-get install python-cherrypy3

Ok here is what I got. Running on Ubuntu 11.04 Server:

Code: Select all

rob@RobsUbuntuServer:~/Desktop/MovieGrabber$ python MovieGrabber.py
[15/Aug/2011:23:03:15] ENGINE Listening for SIGHUP.
[15/Aug/2011:23:03:15] ENGINE Listening for SIGTERM.
[15/Aug/2011:23:03:15] ENGINE Listening for SIGUSR1.
[15/Aug/2011:23:03:15] ENGINE Bus STARTING
CherryPy Checker:
'/home/rob/Desktop/MovieGrabber/interfaces\\classic\\templates\\static\\javascript' (root + dir) is not an existing filesystem path.
section: [/javascript]
root: '/home/rob/Desktop/MovieGrabber'
dir: 'interfaces\\classic\\templates\\static\\javascript'

CherryPy Checker:
'/home/rob/Desktop/MovieGrabber/interfaces\\classic\\templates\\static\\stylesheets' (root + dir) is not an existing filesystem path.
section: [/stylesheets]
root: '/home/rob/Desktop/MovieGrabber'
dir: 'interfaces\\classic\\templates\\static\\stylesheets'

[15/Aug/2011:23:03:15] ENGINE Started monitor thread 'Monitor'.
[15/Aug/2011:23:03:15] ENGINE Started monitor thread '_TimeoutMonitor'.
[15/Aug/2011:23:03:20] ENGINE Error in 'start' listener <bound method Server.start of <cherrypy._cpserver.Server object at 0x1942e10>>
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.7/cherrypy/process/wspbus.py", line 147, in publish
    output.append(listener(*args, **kwargs))
  File "/usr/lib/pymodules/python2.7/cherrypy/_cpserver.py", line 90, in start
    ServerAdapter.start(self)
  File "/usr/lib/pymodules/python2.7/cherrypy/process/servers.py", line 53, in start
    wait_for_free_port(*self.bind_addr)
  File "/usr/lib/pymodules/python2.7/cherrypy/process/servers.py", line 251, in wait_for_free_port
    raise IOError("Port %r not free on %r" % (port, host))
IOError: Port 80 not free on '127.0.0.1'

[15/Aug/2011:23:03:20] ENGINE Shutting down due to error in start listener:
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.7/cherrypy/process/wspbus.py", line 184, in start
    self.publish('start')
  File "/usr/lib/pymodules/python2.7/cherrypy/process/wspbus.py", line 147, in publish
    output.append(listener(*args, **kwargs))
  File "/usr/lib/pymodules/python2.7/cherrypy/_cpserver.py", line 90, in start
    ServerAdapter.start(self)
  File "/usr/lib/pymodules/python2.7/cherrypy/process/servers.py", line 53, in start
    wait_for_free_port(*self.bind_addr)
  File "/usr/lib/pymodules/python2.7/cherrypy/process/servers.py", line 251, in wait_for_free_port
    raise IOError("Port %r not free on %r" % (port, host))
IOError: Port 80 not free on '127.0.0.1'

[15/Aug/2011:23:03:20] ENGINE Bus STOPPING
[15/Aug/2011:23:03:20] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('127.0.0.1', 80)) already shut down
[15/Aug/2011:23:03:20] ENGINE Stopped thread 'Monitor'.
[15/Aug/2011:23:03:20] ENGINE Stopped thread '_TimeoutMonitor'.
[15/Aug/2011:23:03:20] ENGINE Bus STOPPED
[15/Aug/2011:23:03:20] ENGINE Bus EXITING
[15/Aug/2011:23:03:20] ENGINE Bus EXITED
Traceback (most recent call last):
  File "MovieGrabber.py", line 3960, in <module>
    ErrorLogging("Webui Function")
  File "MovieGrabber.py", line 3956, in <module>
    start_webgui()
  File "MovieGrabber.py", line 3842, in start_webgui
    cherrypy.quickstart(HomeRoot(), config = webconfig_ini)
  File "/usr/lib/pymodules/python2.7/cherrypy/__init__.py", line 248, in quickstart
    engine.start()
  File "/usr/lib/pymodules/python2.7/cherrypy/process/wspbus.py", line 184, in start
    self.publish('start')
  File "/usr/lib/pymodules/python2.7/cherrypy/process/wspbus.py", line 147, in publish
    output.append(listener(*args, **kwargs))
  File "/usr/lib/pymodules/python2.7/cherrypy/_cpserver.py", line 90, in start
    ServerAdapter.start(self)
  File "/usr/lib/pymodules/python2.7/cherrypy/process/servers.py", line 53, in start
    wait_for_free_port(*self.bind_addr)
  File "/usr/lib/pymodules/python2.7/cherrypy/process/servers.py", line 251, in wait_for_free_port
    raise IOError("Port %r not free on %r" % (port, host))
IOError: Port 80 not free on '127.0.0.1'

Of course you should NOT use port 80. A lot of use are running web servers.

Please use a different port by default and please allow the user to specify via the command line what port to use. I tried changing line 250 to a different port, but has no effect it seems.

And looks like you used the wrong slashes a few times and too many as well. Thus you get bad paths and files go all over the place. Whoops!!

So I stopped nginx and tried again. I then get:

Code: Select all

rob@RobsUbuntuServer:~/Desktop/MovieGrabber$ python MovieGrabber.py
[15/Aug/2011:23:15:26] ENGINE Listening for SIGHUP.
[15/Aug/2011:23:15:26] ENGINE Listening for SIGTERM.
[15/Aug/2011:23:15:26] ENGINE Listening for SIGUSR1.
[15/Aug/2011:23:15:26] ENGINE Bus STARTING
CherryPy Checker:
'/home/rob/Desktop/MovieGrabber/interfaces\\classic\\templates\\static\\javascript' (root + dir) is not an existing filesystem path.
section: [/javascript]
root: '/home/rob/Desktop/MovieGrabber'
dir: 'interfaces\\classic\\templates\\static\\javascript'

CherryPy Checker:
'/home/rob/Desktop/MovieGrabber/interfaces\\classic\\templates\\static\\stylesheets' (root + dir) is not an existing filesystem path.
section: [/stylesheets]
root: '/home/rob/Desktop/MovieGrabber'
dir: 'interfaces\\classic\\templates\\static\\stylesheets'

[15/Aug/2011:23:15:26] ENGINE Started monitor thread 'Monitor'.
[15/Aug/2011:23:15:26] ENGINE Started monitor thread '_TimeoutMonitor'.
[15/Aug/2011:23:15:26] ENGINE Error in HTTP server: shutting down
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.7/cherrypy/process/servers.py", line 73, in _start_http_thread
    self.httpserver.start()
  File "/usr/lib/pymodules/python2.7/cherrypy/wsgiserver/__init__.py", line 1596, in start
    raise socket.error, msg
error: [Errno 13] Permission denied

[15/Aug/2011:23:15:26] ENGINE Bus STOPPING
[15/Aug/2011:23:15:26] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('127.0.0.1', 80)) already shut down
[15/Aug/2011:23:15:26] ENGINE Stopped thread 'Monitor'.
[15/Aug/2011:23:15:26] ENGINE Stopped thread '_TimeoutMonitor'.
[15/Aug/2011:23:15:26] ENGINE Bus STOPPED
[15/Aug/2011:23:15:26] ENGINE Bus EXITING
[15/Aug/2011:23:15:26] ENGINE Bus EXITED
Exception in thread HTTPServer Thread-3:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 552, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 505, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/usr/lib/pymodules/python2.7/cherrypy/process/servers.py", line 73, in _start_http_thread
    self.httpserver.start()
  File "/usr/lib/pymodules/python2.7/cherrypy/wsgiserver/__init__.py", line 1596, in start
    raise socket.error, msg
error: [Errno 13] Permission denied

[15/Aug/2011:23:15:26] ENGINE Error in 'start' listener <bound method Server.start of <cherrypy._cpserver.Server object at 0x2291e10>>
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.7/cherrypy/process/wspbus.py", line 147, in publish
    output.append(listener(*args, **kwargs))
  File "/usr/lib/pymodules/python2.7/cherrypy/_cpserver.py", line 90, in start
    ServerAdapter.start(self)
  File "/usr/lib/pymodules/python2.7/cherrypy/process/servers.py", line 60, in start
    self.wait()
  File "/usr/lib/pymodules/python2.7/cherrypy/process/servers.py", line 95, in wait
    raise self.interrupt
error: [Errno 13] Permission denied

[15/Aug/2011:23:15:26] ENGINE Shutting down due to error in start listener:
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.7/cherrypy/process/wspbus.py", line 184, in start
    self.publish('start')
  File "/usr/lib/pymodules/python2.7/cherrypy/process/wspbus.py", line 147, in publish
    output.append(listener(*args, **kwargs))
  File "/usr/lib/pymodules/python2.7/cherrypy/_cpserver.py", line 90, in start
    ServerAdapter.start(self)
  File "/usr/lib/pymodules/python2.7/cherrypy/process/servers.py", line 60, in start
    self.wait()
  File "/usr/lib/pymodules/python2.7/cherrypy/process/servers.py", line 95, in wait
    raise self.interrupt
error: [Errno 13] Permission denied

[15/Aug/2011:23:15:26] ENGINE Bus STOPPING
[15/Aug/2011:23:15:26] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('127.0.0.1', 80)) already shut down
[15/Aug/2011:23:15:26] ENGINE No thread running for Monitor.
[15/Aug/2011:23:15:26] ENGINE No thread running for _TimeoutMonitor.
[15/Aug/2011:23:15:26] ENGINE Bus STOPPED
[15/Aug/2011:23:15:26] ENGINE Bus EXITING
[15/Aug/2011:23:15:26] ENGINE Bus EXITED
Traceback (most recent call last):
  File "MovieGrabber.py", line 3960, in <module>
    ErrorLogging("Webui Function")
  File "MovieGrabber.py", line 3956, in <module>
    start_webgui()
  File "MovieGrabber.py", line 3842, in start_webgui
    cherrypy.quickstart(HomeRoot(), config = webconfig_ini)
  File "/usr/lib/pymodules/python2.7/cherrypy/__init__.py", line 248, in quickstart
    engine.start()
  File "/usr/lib/pymodules/python2.7/cherrypy/process/wspbus.py", line 184, in start
    self.publish('start')
  File "/usr/lib/pymodules/python2.7/cherrypy/process/wspbus.py", line 147, in publish
    output.append(listener(*args, **kwargs))
  File "/usr/lib/pymodules/python2.7/cherrypy/_cpserver.py", line 90, in start
    ServerAdapter.start(self)
  File "/usr/lib/pymodules/python2.7/cherrypy/process/servers.py", line 60, in start
    self.wait()
  File "/usr/lib/pymodules/python2.7/cherrypy/process/servers.py", line 95, in wait
    raise self.interrupt
socket.error: [Errno 13] Permission denied
rob@RobsUbuntuServer:~/Desktop/MovieGrabber$

Too scared to run with sudo at the moment. >:D

[rascalli]

@war59312 , I do not want to install anything .. so that's why binhex tried to put it all in 1 package


Off-Topic : looks like chrome needs to get a bit better , as a lot of things do not work with it (not related to this app)

[binhex]

ok firstly thanks for your post war59312, feedback is always good , replies as follows:-

Should post a screenshot on the first page.

yep will do :-)

Also the download link is no good for Chrome users.

for the time being i will provide a direct link to the file, i will be moving over to github at some point, but i got biger fish to fry at the mo.

rascalli and splnut you both simply need to install cherrypy via:

as rascalli posted, you shouldnt need to download any additional libraries as i have now included them, the reason why rascalli was having issues with cherrypy was due to the way the code constructs the pathname, looks like it wasnt compatible with linux.

Of course you should NOT use port 80. A lot of use are running web servers.

i take your point, just thought it would be easiest to start with port 80, but i understand people are running multiple webapps nowadays, i shall set the default port to port 9191 which hopefully shouldnt clash with any other known webapps.

root: '/home/rob/Desktop/MovieGrabber'
dir: 'interfaces\\classic\\templates\\static\\javascript'
'

im afraid i didnt have a linux install to test my code against and thus wasnt aware of the backslash issue causing problems, ive now resolved the issue by normalisiing the path and then using os.path.join instead which should result in linux and windows compatible paths.

fyi you can change the ip and port used by editing the webconfig.ini file which should be created once you run the moviegrabber in the config folder, saying this it may not of generated it due to the path errors you have been having.

ok short story is im on it, i know what the issue is and i have a copy of ubuntu up and running so i will do some further testing today to make sure my fix works, hang tight for version 2.0.2

thanks splnut, rascalli and war59312 for your help.

binhex.

[binhex]

new version 2.0.2 just uploaded, ive done some testing on ubuntu as well as windows and it looks a lot better, please let me know if you find any bugs.

[indomiti]

Very good, this is awesome!

I have a ReadyNAS Ultra (Linux with Python 2.6.6) and i downloaded the source for 2.0.2
Extracted and run "python 1MovieGrabber.py" i dont know what the file 2MovieGrabber.py is? Is it another version?
First it failed to start because 127.0.0.1:80 was busy so i edited file configs/webconfig.ini to my needs, restarted it and it starts successfully.

Configured everything with web interface (NICE!) and click the "Run Now" button.
Could not see if anything happened from the web interface so i checked the logs and found this in logs/moviegrabber_cherrypy.log:

Code: Select all

16/08/2011 14:32:35 Error In IMDB Movie Variables Function
Traceback (most recent call last):
  File "1MovieGrabber.py", line 1022, in IMDBVars
    GlobalVars.imdb_movie_rating = decimal.Decimal(imdb_movie_rating).quantize(decimal.Decimal('.1'))
  File "/usr/lib/python2.6/decimal.py", line 649, in __new__
    "First convert the float to a string")
TypeError: Cannot convert float to Decimal.  First convert the float to a string
16/08/2011 14:32:35 Error In NZBMatrix Search Page Function
Traceback (most recent call last):
  File "1MovieGrabber.py", line 2299, in RunDownloader
    IMDBVars()
  File "1MovieGrabber.py", line 1229, in IMDBVars
    ErrorLogging("IMDB Movie Variables Function")
  File "1MovieGrabber.py", line 1022, in IMDBVars
    GlobalVars.imdb_movie_rating = decimal.Decimal(imdb_movie_rating).quantize(decimal.Decimal('.1'))
  File "/usr/lib/python2.6/decimal.py", line 649, in __new__
    "First convert the float to a string")
TypeError: Cannot convert float to Decimal.  First convert the float to a string

I also have a request for a change, could you change/enhance the scheduler so it would be able to run at specific times of the day? Maybe at 02:00 and at 14:00 for example As it is now i think it might start downloading, repairing, extracting etc etc on bad times when i dont want my ReadyNAS to be at 100% CPU for some hours!

Thanks

[binhex]

Very good, this is awesome!

I have a ReadyNAS Ultra (Linux with Python 2.6.6) and i downloaded the source for 2.0.2
Extracted and run "python 1MovieGrabber.py" i dont know what the file 2MovieGrabber.py is? Is it another version?
First it failed to start because 127.0.0.1:80 was busy so i edited file configs/webconfig.ini to my needs, restarted it and it starts successfully.

Configured everything with web interface (NICE!) and click the "Run Now" button.
Could not see if anything happened from the web interface so i checked the logs and found this in logs/moviegrabber_cherrypy.log:

Code: Select all

16/08/2011 14:32:35 Error In IMDB Movie Variables Function
Traceback (most recent call last):
  File "1MovieGrabber.py", line 1022, in IMDBVars
    GlobalVars.imdb_movie_rating = decimal.Decimal(imdb_movie_rating).quantize(decimal.Decimal('.1'))
  File "/usr/lib/python2.6/decimal.py", line 649, in __new__
    "First convert the float to a string")
TypeError: Cannot convert float to Decimal.  First convert the float to a string
16/08/2011 14:32:35 Error In NZBMatrix Search Page Function
Traceback (most recent call last):
  File "1MovieGrabber.py", line 2299, in RunDownloader
    IMDBVars()
  File "1MovieGrabber.py", line 1229, in IMDBVars
    ErrorLogging("IMDB Movie Variables Function")
  File "1MovieGrabber.py", line 1022, in IMDBVars
    GlobalVars.imdb_movie_rating = decimal.Decimal(imdb_movie_rating).quantize(decimal.Decimal('.1'))
  File "/usr/lib/python2.6/decimal.py", line 649, in __new__
    "First convert the float to a string")
TypeError: Cannot convert float to Decimal.  First convert the float to a string

I also have a request for a change, could you change/enhance the scheduler so it would be able to run at specific times of the day? Maybe at 02:00 and at 14:00 for example As it is now i think it might start downloading, repairing, extracting etc etc on bad times when i dont want my ReadyNAS to be at 100% CPU for some hours!

Thanks

hi indomiti, looks like i forgot to remove the old version i was working on, please ignore 1MovieGrabber.py and run version 2MovieGrabber.py, ive altered the source zip now so its correct and uploaded, this also includes the default port of 9191. let me know if your still getting the same bug.

as far as your request goes i will add it to the list :-), btw you will find MovieGrabber very light on resources so i dont think you will have issues with cpu usage.if you worried about the actual download/unpacking causing a cpu spike then i would recommend setting the schedulder already built into sabnzbd+ as this is what i use and it works very well, you can set it to limit download speeds between any period so you could simply use this to throttle speeds for the the times you dont want your readynas to be hammered.

edit - indomiti i took a look at your traceback and it looks like python 2.6 cannot handle the conversion from float to dec (version im using is 2.7.1) so ive now altered the code, uploaded as 2.0.3 give it a whirl and let me know if your still seeing the issue, thanks for the traceback very useful!

[indomiti]

Edit: Going to try 2.0.3

[binhex]

check my previous post, this issue is solved in 2.0.3, just uploaded! :-)

[indomiti]

Alright! Grabbed 2.0.3 and replaced with my config files.
Its getting further now, but something else is wrong

The file logs/moviegrabber_warning.log is created and contains this:

Code: Select all

16/08/2011 16:02:38 Warning IMDB Poster Download Failed For Movie The Tie That Binds
16/08/2011 16:02:38 Error In NZBMatrix Search Page Function
Traceback (most recent call last):
  File "MovieGrabber.py", line 2329, in RunDownloader
    DLResult()
  File "MovieGrabber.py", line 1789, in DLResult
    SqliteHistoryInsert()
  File "MovieGrabber.py", line 1739, in SqliteHistoryInsert
    GlobalVars.sqlite_cursor_downloader.execute("INSERT into history ('imdbposter','imdblink','imdbplot','imdbname','imdbyear','imdbrating','postdate','postsize','postnfo','postdetails','postname','dlstatus','dldate') values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",(GlobalVars.poster_image_file, eval("GlobalVars." + GlobalVars.index_site  + "_imdb_link_full"), (GlobalVars.imdb_movie_description), (GlobalVars.imdb_movie_title), ' (' + str(GlobalVars.imdb_movie_year) + ') ', str(GlobalVars.imdb_movie_rating) + '/10', eval("GlobalVars." + GlobalVars.index_site  + "_post_date"), eval("GlobalVars." + GlobalVars.index_site  + "_post_size_view"), eval("GlobalVars." + GlobalVars.index_site  + "_post_nfo"), eval("GlobalVars." + GlobalVars.index_site  + "_post_details"), eval("GlobalVars." + GlobalVars.index_site  + "_post_title"), GlobalVars.nzb_download_status, time.strftime("%d" + "/" + "%m" + "/" + "%Y" + " " + "%H" + ":" + "%M" + ":" + "%S" + "\n")))
AttributeError: class GlobalVars has no attribute 'poster_image_file'

[binhex]

indomiti, bug squashed!, grab 2.0.4

[indomiti]

Nice work binhex! You are fast!
Lowered my imdb rating a bit so it found something worth downloading And it works!

I have a problem though and it fails to download every poster so running through the script takes a long time.. moviegrabber_warning.log looks like this:

Code: Select all

16/08/2011 16:57:21 Warning IMDB Poster Download Failed For Movie Lethal Weapon 4
16/08/2011 16:57:53 Warning IMDB Poster Download Failed For Movie Bronson
16/08/2011 16:58:25 Warning IMDB Poster Download Failed For Movie Benny & Joon

I changed the setting:

Code: Select all

[switches]
enable_posters = no

But still tries to download them, it just changed the way it displays posters in history view..

[binhex]

Nice work binhex! You are fast!
Lowered my imdb rating a bit so it found something worth downloading And it works!

I have a problem though and it fails to download every poster so running through the script takes a long time.. moviegrabber_warning.log looks like this:

Code: Select all

16/08/2011 16:57:21 Warning IMDB Poster Download Failed For Movie Lethal Weapon 4
16/08/2011 16:57:53 Warning IMDB Poster Download Failed For Movie Bronson
16/08/2011 16:58:25 Warning IMDB Poster Download Failed For Movie Benny & Joon

I changed the setting:

Code: Select all

[switches]
enable_posters = no

But still tries to download them, it just changed the way it displays posters in history view..

thats really odd, it should be downloading the posters no problems, its grabbing these directly from imdb, ive tested this on both windows and linux and it works, hmmm.

ok one thing you can try is to upgrade to python 2.7.x as your still using 2.6.x at the mo, perhaps there is change in a module which is causing the problem.

fyi changing enable_posters to no will not disable poster downloads, it wil just disable them from being visible in history view/queued view.

let me know if the upgrade fixes the issue.

edit - i have just uploaded a debug copy, can you place this file in the root folder (same location as MovieGrabber.py) and then run this, im expecting it to error out but it should at least give us a traceback to work with, can you post it.

link:- http://www.mediafire.com/?nsxfy4akdy966d7

cheers.

binhex.