MovieGrabber - Completely Automated Movie Downloader

[baje]

ok 2.1.5 released just in time for the weekend :-), see first post in this thread for changelog.

upgrading?
1. shutdown moviegrabber
2. unzip to same folder overwriting all files/folders
3. start moviegrabber

please note if you are using nzbs.org as your index site you will need to re-enter in your id and hash values in configuration/website in the webui, these values have now been split to stop confusion, you do NOT need to prefix/append these values with "&" or "=" values as previously, you can find your id and hash values from the following url http://www.nzbs.org/user.php?action=profile

any bugs please post the traceback and details on how to reproduce the error.

binhex.

Thx for this update, moviegrabber now works like a charm.

Maybe you can add an encrypted form for "Password".

And one other thing. It would be great if you can add a multiple search criteria. Something like ";" how can separe two search criteria and after that launch two separate request for nzbmatrix.

[daichiasuka]

First time installing MovieGrabber. After I run the python file and configure the settings when I click Run Now it produces a SQLite traceback.

Using 2.1.5 on Linux.

Code: Select all

Process downloader_process:
Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "MovieGrabber.py", line 2914, in NZBMatrixIndex
    ErrorLogging("NZBMatrix RSS Function")
  File "MovieGrabber.py", line 2791, in NZBMatrixIndex
    GlobalVars.sqlite_cursor_downloader.execute("SELECT postname FROM history WHERE postname = ?", [GlobalVars.post_title])
ProgrammingError: SQLite objects created in a thread can only be used in that same thread.The object was created in thread id 139790057535232 and this is thread id 139789567256320


[RKM]

This plugin sounds like an excellent concept. And while reading through the posts it seems everyone has good luck with it.

So I must be doing something very wrong, because I cannot get the search-results (under "View History") to match the results of an identical search in NZBmatrix or NZBs.org.

Take NZBs.org for example: I set MG to "all formats", search criteria "xvid" (no quotes), minimum size 1 maximimum size 10000. I leave everything else blank.

I then search NZBs.org, using the exact same criteria, and the lists ahve nothing in common. Same issue with NZBmatrix.

I've double-checked and re-entered my API keys several times, I've tried different search terms, etc.

Is there a way to turn on verbose logging so that I can see what gets passed to the NZB index sites? I suppose I could use something like Wireshark if not -- but it just seems like it must be something obvious or I'd have been able to find a post on it.

Thank you in advance!
RKM

[binhex]

hi daichiasuka, ive spotted the bug, this will be fixed in 2.1.6, wierdly im not seeing the bug causing any issues when running 2.1.5, strange, but at least im pretty sure what the cause is.

[binhex]

hi RKM, i think you maybe misunderstanding how moviegrabber works, it doesnt do searching as you would do if you were a user, but instead it uses the rss feed from the index site youve choosen (or api in the case of newznab sites) to "filter" by your criteria, in your case only xvid movies. i guess you could call it "filter criteria" but its not as easily understood by users who prefer the more friendly phrase of "search criteria". i have had previous requests to include backlog searching (which is what your on about i think?) but it would cause a large amount of traffic for the index sites as your not searching for a movie title but a loose term of "720p" or "xvid" which will return thousands of results. its something i can potential revisit now that queued movies are linked instead of downloaded.

i hope this does help, any other questions let me know.

binhex.

[RKM]

Hi Binhex,
First, thank you for taking the time to reply.

Second, I apologize for not doing a better job communicating. I was using ther term "search" or "search results" to indicate the output of the RSS feed. And yes, I do understand that you consume standardized RSS rather than dealing with the complexities of a changing markup.

I will try to be more specific in my explanation. First, imagine the search criteria I mentioned above. Now I attempt to duplicate that search on NZBs.org (I also tested NZBmatrix, but I'll start with NZB's for simplicity).

Here is the RSS request I would concatenate, expecting it to match the "view history" under MovieGrabber:
http://www.nzbs.org/rss.php?type=2&q=xvid&i=<API key removed for privacy>

I first clear the history on MG. Then I submit the request on MG and pull down the RSS on NZBs.org at the same time. Then I was 5+ minutes and check the history on MG. Somehow MG has movies that are not included in NZBs.org, and NZBs.org has movies that are not included on MG.

Now, if it were a matter of those items being excluded for IMDB reasons, etc -- I'd understand. However, this is the logging area of the tool I'm using, not the final list of what's been imported.

Is there any way to get the RSS URI's tha you're building, logged? This way I could better understand if they are working correctly.

Here's one example of where this feature would come in handy for other users .. you have a tooltip that advises people to use "+" or "-" in front of search terms. You example is "+720 -hdtv". The problem is that unless a user is familiar or can test the RSS you create, they may not realize that (a) this syntax does not work on all of the NZB index sites (for example, off the top of my head, "+" in NZBmatrix will eliminate searching the term as a wildcard, and "-" is not supported).

Does this make better sense? I guess in short, my question is a two parter: (1) can you help me determine why my results don't appear to be matching, and/or (2) is it possible to get the RSS request logged, so that I could determine the cause without troubling you :-).

Thank you in advance!
RKM

[LegendsOfDarkness]

Hi

Would really like the to work, but I'm getting this error:

Code: Select all

28/02/2012 23:05:25 Error In NZBMatrix RSS Function
Traceback (most recent call last):
  File "MovieGrabber.py", line 2791, in NZBMatrixIndex
    GlobalVars.sqlite_cursor_downloader.execute("SELECT postname FROM history WHERE postname = ?", [GlobalVars.post_title])
ProgrammingError: SQLite objects created in a thread can only be used in that same thread.The object was created in thread id 140068018411232 and this is thread id 1177647440

Config.ini looks like this:

Code: Select all

[folders]
movies_downloaded_dir = /home/user/moviegrabber/1
movies_to_replace_dir = /home/user/moviegrabber/2
watch_dir = /home/user/nzb-watch
nzb_dir = /home/user/SABnzbdDownloads/Ongoing
completed_dir = /home/user/SABnzbdDownloads/Finished
metadata_dir = /home/user/moviegrabber/3
cherrypylog_dir = /home/user/Desktop/moviegrabber/logs
warninglog_dir = /home/user/Desktop/moviegrabber/logs
errorlog_dir = /home/user/Desktop/moviegrabber/logs

[switches]
enable_downloaded = no
enable_replace = no
enable_favorites = yes
enable_preferred = yes
enable_queuing = yes
enable_metadata = no
enable_release_group = no
enable_email_notify = no
enable_append_year = yes
enable_posters = yes

[imdb]
good_genre = action,adventure,animation,biography,comedy,crime,documentary,drama,family,fantasy,film-Noir,game-show,history,horror,music,musical,mystery,news,reality-tv,romance,sci-fi,short,sport,talk-show,thriller,war,western
good_rating = 4.0
good_votes = 0
good_date = 1900
preferred_genre = action,adventure,animation,biography,comedy,crime,documentary,drama,family,fantasy,film-Noir,game-show,history,horror,music,musical,mystery,news,reality-tv,romance,sci-fi,short,sport,talk-show,thriller,war,western
preferred_rating = 0.0
queue_genre = action,adventure,animation,biography,comedy,crime,documentary,drama,family,fantasy,film-Noir,game-show,history,horror,music,musical,mystery,news,reality-tv,romance,sci-fi,short,sport,talk-show,thriller,war,western
queue_date = 1900
bad_genre =
bad_title =
fav_dir =
fav_writer =
fav_actor =
fav_char =
fav_title =

[email_settings]
email_server =
email_server_port = 587
email_server_ssl =
email_username =
email_password =
email_from =
email_to =

[website_settings]
index_site = nzbmatrix
nzbmatrix_username = my username
nzbmatrix_key = my matrix api key
nzbmatrix_cat = hd/x264
nzbmatrix_lang = any
nzbmatrix_search = +1080 -ts
nzbmatrix_minsize = 0
nzbmatrix_maxsize = 0
nzbmatrix_badreport =
nzbmatrix_release_group =
nzbs_id =
nzbs_hash =
nzbs_cat = all formats
nzbs_search =
nzbs_minsize = 0
nzbs_maxsize = 0
nzbs_badreport =
nzbs_release_group =
newznab_hostname =
newznab_portnumber =
newznab_key =
newznab_cat = all formats
newznab_search =
newznab_minsize = 0
newznab_maxsize = 0
newznab_min_completion = 1
newznab_badreport =
newznab_release_group =

[misc]
color_scheme = black
schedule_hour = 0
schedule_minute = 30
max_items_shown = 100
history_sort_order = date_desc
queued_sort_order = date_desc
moviegrabber_username =
moviegrabber_password =
moviegrabber_ssl = no
launch_browser = yes
last_run = 28/02/2012 23:05:25
version = 2.1.5



Please help!

[funnyname]

Hello,

I'm a Windows user, where can I find full logging for MovieGrabber?
In my LOG folder I only find the moviegrabber_cherrypy.log & moviegrabber_warning.log, none of these give useful information.

I think I have a working config, but nothing is happening after I click the Run now button. I've tried nzbmatrix and nzb.su as search providers.
Many thanks for the reply, keep up the good work.

Cheers,
FunnyName

[binhex]

hi rkm, firstly sorry for the delay in responding, very busy at work at the moment!.

ok in answer to your question:-

ok i can give you the rss url with the variables in in, then its just a case of your replacing the variables you have entered into the config and voila you will have the url that moviegrabber is using:-
rss_feed = "https://nzbs.org/rss.php?q=" + nzbs_search + nzbs_cat + "&i=" + nzbs_id + "&h=" + nzbs_hash + "&dl=1&num=100"

there is some regex replacement going on here to make the config entries more user friendly, here is the real url you need, just replace your id and hash values

so an example would be https://nzbs.org/rss.php?q=+xvid&type=2 ... =1&num=100

if you then copy this url into your web browser you should then see an rss feed appear, if you then open another tab in your web browser go to the main page select "movies" as your category and enter in a search criteria of xvid the two will match (just done it to confirm :-)

if they dont then please let me know.

binhex.

[binhex]

hi legendsofdarkness, this bug has already been reported by daichiasuka, this will be fixed in 2.1.6, i dont believe the bug is present in 2.1.4 so if you need to get going then i would advise using this version until 2.1.6 is out (hopefully by the end of this week).

binhex.

[binhex]

hi funnyname, the most common cause of this is a incorrect search criteria, can you please post what you have entered in the configiguration/website/search critera

at the moment i dont have advanced logging, but i will def be including this in future versions.

binhex.

[funnyname]

Hey Binhex,

Thanks for the quick reply, sorry to bother with these trivial questions ..
Here is my config.ini:

Code: Select all

[folders]
movies_downloaded_dir = D:\Downloads\complete\Movies\
movies_to_replace_dir =
watch_dir = D:\Downloads\watched\
nzb_dir = D:\Downloads\nzb\
completed_dir = D:\Downloads\complete\Movies\
metadata_dir =
cherrypylog_dir = D:\Downloads\logs\
warninglog_dir = D:\Downloads\logs\
errorlog_dir = D:\Downloads\logs\

[switches]
enable_downloaded = yes
enable_replace = no
enable_favorites = yes
enable_preferred = yes
enable_queuing = yes
enable_metadata = no
enable_release_group = no
enable_email_notify = no
enable_append_year = no
enable_posters = yes

[imdb]
good_genre = action,adventure,animation,crime,documentary,fantasy,film-Noir,history,mystery,news,reality-tv,sci-fi,short,sport,thriller,war,western
good_rating = 8.0
good_votes = 100
good_date = 1960
preferred_genre = action,adventure,animation,crime,documentary,fantasy,film-Noir,history,mystery,news,reality-tv,sci-fi,short,sport,thriller,war,western
preferred_rating = 7.0
queue_genre = action,adventure,animation,crime,documentary,fantasy,film-Noir,history,mystery,news,reality-tv,sci-fi,short,sport,thriller,war,western
queue_date = 1960
bad_genre = biography,comedy,drama,family,game-show,horror,music,musical,romance,talk-show
bad_title =
fav_dir = stephen spielberg,Darabont,Ford Coppola,Quentin Tarantino,Christopher Nolan,David Fincher,Martin Scorsese,Bryan Singer,Luc Besson,James Cameron,Ridley Scott,Roman Polanski,Darren Aronofsky,Fernando Meirelles
fav_writer =
fav_actor = Harrison Ford,Densel washington,John Travolta,Samuel L,Jackson,Tim Roth,Bruce Willis,clint eastwood,Christian Bale,Heath Ledger,Jack Nicholson,Keanu Reeves,Stephen Baldwin,Benicio Del Toro,Kevin Spacey,Edward Norton,Brad Pitt,Tom Hanks,Steve Buscemi,Natalie Portman,Charlize Theron,Jessica Biel
fav_char =
fav_title =

[email_settings]
email_server =
email_server_port = 587
email_server_ssl =
email_username =
email_password =
email_from =
email_to =

[website_settings]
index_site = nzbmatrix
nzbmatrix_username = %looser%
nzbmatrix_key = %key%
nzbmatrix_cat = all formats
nzbmatrix_lang = english
nzbmatrix_search = +divx,+xvid,+720p,-cam,-telesync,-ts,-german
nzbmatrix_minsize = 650
nzbmatrix_maxsize = 8000
nzbmatrix_badreport =
nzbmatrix_release_group =
nzbs_id =
nzbs_hash =
nzbs_cat = all formats
nzbs_search =
nzbs_minsize = 0
nzbs_maxsize = 0
nzbs_badreport =
nzbs_release_group =
newznab_hostname =
newznab_portnumber =
newznab_key =
newznab_cat = all formats
newznab_search =
newznab_minsize = 0
newznab_maxsize = 0
newznab_min_completion = 1
newznab_badreport =
newznab_release_group =

[misc]
color_scheme = darkblue
schedule_hour = 1
schedule_minute = 0
max_items_shown = 100
history_sort_order = date_desc
queued_sort_order = date_desc
moviegrabber_username =
moviegrabber_password =
moviegrabber_ssl = no
launch_browser = yes
last_run = 28/02/2012 18:09:21
version = 2.1.5


The NZBMATRIX info is adjusted of course, for obvious reasons ..

Cheers,
FunnyName

[RKM]

Hi Binhex,
Thank you so much for such hard work on developing the tool, then doing such a great job supporting it.

I did test your template querystring, and I received exactly what I expected from NZBs.org.

However, the results in the "View History" did not match.

I don't know if it would be possible, but if you could add a tiny bit more logging, or even if you allowed purging the database (not just the current history, but all history forcing a full re-download of all shows) .. this would help me get to the bottom of this issue.

Here's what I mean .. when I run the template querystring you gave me, which matches the criteria I have entered in your tool, I get the standard 100 most recent movies containing XVID in the title, as you'd expect. The most recent movies being: GWDT, Paranormal Activity, Neverland, Art of Getting By, Recoil, ... with Muppets as the 7th movie.

Yet when I run it in the app, I get only the Muppets. Now, this could be because there is old history in the database, and those first 6 movies were previously downloaded. However, because I cannot truly purge all data, or because I cannot get more detailed logs, there is no way to tell.

But I can definitely tell you that something is not matching up in my testing. I'm sure it's unique to my situation somehow, because it seems you've done a great job with this plug-in, I just can't imagine what it would be!

Thank you in advance!
RKM

[binhex]

hi rkm, i think the reason your not seeing the same result in the history log is because there are some checks done before it gets to processing the movie, one of the checks it to make sure there is a imdb link for the movie, i think you will probably find that the movies your seeing as missing dont have an imdb link, without a link they cannot be processed as moviegrabber relies on imdb for rating/votes/director name etc etc.

ive just done a quick search for neverland xvid for category "movies" and can confirm there is no "imdb link" for this movie, thus its getting skipped.

i hope this explains it.

binhex.

[RKM]

Hi Binhex,
Yes, that's extremely helpful. It makes a lot of sense.

The only problem all of the other movies that were ahead of the Muppets (mentioned in my post) *did* indeed have IMDB metadata. Even the version of Neverland that was listed last night, which you mentioned, has the IMDB metadata(XML field: "report:imdbid").

If you could enable logging, simply to the directory, no fancy HTML interface required, it would be huge.

Thank you,
RKM