MovieGrabber - Completely Automated Movie Downloader

[neo_x]

Hi Binhex

glad to see you are still keeping the development going ( without any beers )

recently started trusting Moviegrabber again (not sure why i ran away - bad providers i guess )

in any event, i am having strange scenario's , and i was hoping you can confirm if this is bug or functionality:

i have both Usenet and torrent configured ( why not.. ), and have thus setup both to queue everything, after which i will decide what to release
on the 2nd i grabbed a USENET copy of a certain famous movie about a guy with a unbreakable hammer - and all is good, it is loaded by the client, and the NZB file is stored in the backup folder - problem being my client is working on a backlog of three weeks, and thus wont get to downloading and even processing said hammer movie.

further USENET content is picked up over the following few days , and skipped with message "Result: Archive" - SYSYEM NZB/Torrent is in archive folder - Great.

but - once in a while, a torrent version is released - but regretfully, it isn't skipped ( obviously because the torrent isn't there), and thus lands up back in the que.

luckily my memory is good enough to remember that there is only one famous hammer movie, but i was wondering if you cant figure out an easier way to fix this scenario? ( either a database of downloaded content or a check by torrents in Usenet folder or Usenet in Torrent folder?)

thank you for your continued effort

really owe you plenty by now

Neo_x

[binhex]

yep getting really thirsty now neo_x i see the issue you posted, so basically because there are two separate archive folders, one for torrents and one for usenet, that your getting a potential re-queuing from either one, ok thats fairly simple to fix, i will just get the usenet/torrent index process to check both torrent and nzb archive folders before queuing, if either folder have a matching file (based on imdb title) then the movie wont be queued, sound good?.

oh btw i would be cautious about having said "long haired god" movie queued for a long period of time, dmca is a kicker nowadays!, i personally move anything with year release from 2013 onward up to the top of the queue for quick processing, i advise you do the same.

[neo_x]

Hi Binhex

any chance on adding the change. I know the beers and family and work is keeping you hectically busy as usual

Thx

neo_x

[binhex]

hi neo, will do soon, got internet outage at my house and currently still fighting with BT about it, once im back online i will do my best to get b8 out asap.

[herbdavisy2k]

hey binhex,

I'm trying to run moviegrabber on freenas but I'm getting the following error:

Code: Select all

Traceback (most recent call last):
  File "/moviegrabber/lib/site-packages/cherrypy/_cprequest.py", line 656, in respond
    response.body = self.handler()
  File "/moviegrabber/lib/site-packages/cherrypy/lib/encoding.py", line 188, in __call__
    self.body = self.oldhandler(*args, **kwargs)
  File "/moviegrabber/lib/site-packages/cherrypy/_cpdispatch.py", line 34, in __call__
    return self.callable(*self.args, **self.kwargs)
  File "MovieGrabber.py", line 6165, in run_now
    SearchIndexThread().checks()
  File "MovieGrabber.py", line 6369, in checks
    self.run()
  File "MovieGrabber.py", line 6411, in run
    search_index_function = getattr(SearchIndex(self.download_type,self.index_site_item), self.search_index_function)
  File "MovieGrabber.py", line 1181, in __init__
    self.movies_downloaded_cache = list(itertools.chain.from_iterable(os.walk(root_path) for root_path in movies_downloaded_dir_list))
  File "/usr/local/lib/python2.7/os.py", line 284, in walk
    if isdir(join(top, name)):
  File "/usr/local/lib/python2.7/posixpath.py", line 78, in join
    path +=  b
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 34: ordinal not in range(128)

Here's my config file:

Code: Select all

[folders]
movies_downloaded_dir = /mnt/media/Movies/,/mnt/media/New Downloads/New Movies/
movies_replace_dir = 
usenet_watch_dir = /mnt/media/Downloads/Usenet/Import/
usenet_archive_dir = /mnt/media/Downloads/Usenet/NZB
usenet_completed_dir = /mnt/media/Downloads/Usenet/Done/Movies
torrent_watch_dir = 
torrent_archive_dir = 
torrent_completed_dir = 
cherrypylog_dir = /moviegrabber/logs
moviegrabberlog_dir = /moviegrabber/logs

[switches]
enable_downloaded = no
enable_replace = no
enable_favorites = no
enable_preferred = no
enable_queuing = no
enable_group_filter = no
enable_email_notify = no
enable_append_year = yes
enable_posters = yes
enable_post_processing = no
enable_xbmc = no

[imdb]
good_genre = action,adventure,animation,biography,comedy,crime,documentary,drama,family,fantasy,film-Noir,game-show,history,horror,music,musical,mystery,news,reality-tv,romance,sci-fi,short,sport,talk-show,thriller,war,western
good_rating = 6.5
good_votes = 1000
good_date = 2000
preferred_genre = 
preferred_rating = 0.0
queue_genre = 
queue_date = 2000
bad_title = 
fav_dir = 
fav_writer = 
fav_actor = 
fav_char = 
fav_title = 

[email_settings]
email_server = 
email_server_port = 587
email_server_ssl = 
email_username = 
email_password = 
email_from = 
email_to = 

[xbmc]
xbmc_host = 
xbmc_port = 80
xbmc_username = xbmc
xbmc_password = 
xbmc_notification = no
xbmc_library_update = no

[usenet]
index_site = nzbs_org_1
nzbs_org_1_hostname = https://nzbs.org
nzbs_org_1_portnumber = 443
nzbs_org_1_path = 
nzbs_org_1_key = 12345
nzbs_org_1_cat = all formats
nzbs_org_1_search_and = 
nzbs_org_1_search_or = 
nzbs_org_1_search_not = 
nzbs_org_1_minsize = 0
nzbs_org_1_maxsize = 0
nzbs_org_1_bad_report = 
nzbs_org_1_spotweb_support = no
nzbs_org_1_enabled = yes

[torrent]
index_site = 

[post_processing]
post_rule = 

[webconfig]
address = 0.0.0.0
port = 9191
username = 
password = 
enable_ssl = no

[misc]
post_schedule_hour = 1
post_schedule_minute = 0
post_rename_files = no
post_cert_system = us
index_preferred_group = 
index_bad_group = 
index_posts_to_process = 50
color_scheme = darkblue
theme = classic
index_schedule_hour = 1
index_schedule_minute = 0
max_items_shown = all
history_sort_order = desc,postdatesort
queued_sort_order = desc,postdatesort
movie_title_separator = <>
launch_browser = yes
log_level = WARNING
last_run = 
check_version = daily
last_version_check = 06/04/2014 09:03:56
remote_version = 2.2.0.b8
remote_download = http://sourceforge.net/projects/moviegrabber/files/beta/moviegrabber-src-2.2.0.b8.zip/download
local_version = 2.2.0.b8

Any suggestions?

[binhex]

hi, this is the crutial bit from the traceback:-

UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 34: ordinal not in range(128)

basically its saying it cannot decode the string as its not in ascii, this is caused by running the process without first setting the locale, im assuming your running moviegrabber using some sort of plugin on your nas?, if so you would need to specify the correct locale there, a fairly normal locale would be utf-8, this will support both ascii chars and a subset of non ascii.

[herbdavisy2k]

Thank you sir! I now have MovieGrabber working on FreeNAS

hi, this is the crutial bit from the traceback:-

UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 34: ordinal not in range(128)

basically its saying it cannot decode the string as its not in ascii, this is caused by running the process without first setting the locale, im assuming your running moviegrabber using some sort of plugin on your nas?, if so you would need to specify the correct locale there, a fairly normal locale would be utf-8, this will support both ascii chars and a subset of non ascii.

[J03 8LACK]

Hey binhex,

Are there any logs for post processing. I'm Trying to figure out why none of my certificate rules are not working?

Thanks
J03

[binhex]

hi J03, if you switch logging to info it should then log as much detail as possible, i will take a look at this for you as well, to be honest i need to re-visit post processing again and make sure its all working as expected, so it could well be a bug.

[neo_x]

Is anybody else maybe using Torrents as an option?

My last 8 releases downloaded via Moviegrabber was corrupt (eg the actual torrent file is malformed). Comparing the file with a file obtained vie the physical download button on KAT, shows definite differences (eg a few more bytes in the KAT file versus the MG file)

any ideas as to why?

Will keep checking on other releases to see if it is maybe an issue on KAT'side.

[neo_x]

Is anybody else maybe using Torrents as an option?

My last 8 releases downloaded via Moviegrabber was corrupt (eg the actual torrent file is malformed). Comparing the file with a file obtained vie the physical download button on KAT, shows definite differences (eg a few more bytes in the KAT file versus the MG file)

any ideas as to why?

Will keep checking on other releases to see if it is maybe an issue on KAT'side.

HI Binhex

Related to the above, some further troubleshooting is needed - not sure what is the cause though.
Basically a torrent is found, queued , but upon releasing it a corrupt file is placed in the torrent_in folder (eg when trying to open via utorrent it says "Unable to load torrent, torrent is not valid bencoding).
Opening the file, and comparing with a valid torrent, the first few lines already shows a difference ( normal torrent has standard ascii text "d8:announce32:" followed by a list of Udp trackers), where as the file released from the que doesn't have

I enabled info level logging, with some lines below :

Code: Select all

2014-04-27 10:36:42,637 INFO download_thread MovieGrabber download_read :: Read of nzb/torrent file successful for url http://torcache.net/torrent/28BCFF857964D0435567BA31509F61ECD6CC4C3B.torrent?title=[kickass.to]rio.2.2014.720p.hdts.x264.mp3.rarbg
2014-04-27 10:36:42,637 INFO download_thread MovieGrabber download_write :: Watched folder exists C:\Downloads\Torrent_in
2014-04-27 10:36:42,823 INFO download_thread MovieGrabber download_write :: Write of nzb/torrent file successful for file  C:\Downloads\Torrent_in\Rio 2 (2014).torrent
2014-04-27 10:36:43,197 INFO download_thread MovieGrabber download_write :: Deleted queued item from database
2014-04-27 10:36:43,197 INFO download_thread MovieGrabber download_read :: Download of nzb/torrent file successful for movie Rio 2 (2014)
2014-04-27 10:36:45,835 INFO download_thread MovieGrabber download_read :: Read of nzb/torrent file successful for url http://torcache.net/torrent/2E54F3110A066C7CB9E80CF8AF1D990FC1EA8978.torrent?title=[kickass.to]that.awkward.moment.2014.720p.brrip.xvid.ac3.rarbg
2014-04-27 10:36:45,835 INFO download_thread MovieGrabber download_write :: Watched folder exists C:\Downloads\Torrent_in
2014-04-27 10:36:46,006 INFO download_thread MovieGrabber download_write :: Write of nzb/torrent file successful for file  C:\Downloads\Torrent_in\That Awkward Moment (2014).torrent
2014-04-27 10:36:46,302 INFO download_thread MovieGrabber download_write :: Deleted queued item from database
2014-04-27 10:36:46,302 INFO download_thread MovieGrabber download_read :: Download of nzb/torrent file successful for movie That Awkward Moment (2014)
2014-04-27 10:36:47,301 INFO download_thread MovieGrabber download_read :: Read of nzb/torrent file successful for url 

Opening the url presented in the log, downloads a valid torrent file immediately, which thus leaves the question - where is the other file appearing from.

I did try restarting the Moviegrabber process via the restart button on the webgui, but so far no luck.

Any other ideas to get to the bottom of this?

thx

Neo_x

Ps
the combined NZB scanning (eg queue and release) is working fine though.

[binhex]

Hi neo_x I will take a look at this, one quick question for you, is this affecting all torrents or just certain ones, if certain torrents can you pm me a link.

Cheers,
Binhex

[binhex]

ok quick update on the torrent download issue, spotted the problem, its basically that kat is now gzipping torrents as well as gzipping their rss feed, im now detecting this and uncompressing as required, so this will be included in next beta which i will push out fairly shortly.

[neo_x]

ok quick update on the torrent download issue, spotted the problem, its basically that kat is now gzipping torrents as well as gzipping their rss feed, im now detecting this and uncompressing as required, so this will be included in next beta which i will push out fairly shortly.

Hi Binhex

Thx for the update on this. Appreciate ur dedication on this!

Getting close to 3 years of Moviegrabber now -keep it up!

[binhex]

no probs, if you get an arch linux vm running on unraid and want to get moviegrabber running then download the following file and run pacman -U <package name> working well for me

http://sourceforge.net/projects/moviegr ... z/download