Media Rover

[rollingeyeball]

Lockzi, I sort of proposed an idea about fetching old eps to Kierse a while back but I actually forget it
For older eps I am catching up on pro-actively I use nzbindex.nl or nzbclub.com. Both have good searches and great retention.

Wally, your idea sounds interesting.
I don't see any flaws. It sounds really good actually.

Lockzi, why do you think it's better to have Media Rover use a database?
I think keeping track of what's been downloaded totally nulls one of the amazing things about MR (imho) and that is that it doesn't actually keep track of your stuff. Just makes decisions on snapshots when it's run. That way you CAN download stuff from anywhere using any method and MR doesn't care.
I also have a hunch, though for no particular reason since i have essentially no python experience -- that implementing a database would be more work than something essentially simple like wally's suggestion.

Good food for thought in this thread

[wally007]

Hey everyone,

A few people have been asking and I've finally done it.  Media Rover now supports nzbmatrix.com!  Download the latest release from the development branch at http://mediarover.tv/#installation

Kierse

Is this version working for anybody ? I've tried on two machines and on both i get same error. Older version works fine on both machines but this latest release wont even launch. here's my terminal log

Code: Select all

PCserver:~ vladik1$ python /Users/vladik1/Downloads/mediarover/mediarover.py 
Traceback (most recent call last):
  File "/Users/vladik1/Downloads/mediarover/mediarover.py", line 19, in <module>
    mediarover.main()
  File "/Users/vladik1/Downloads/mediarover/mediarover/__init__.py", line 75, in main
    config = read_config(config_dir)
  File "/Users/vladik1/Downloads/mediarover/mediarover/config.py", line 565, in read_config
    results = config.validate(_get_validator(), preserve_errors=True)
  File "/Users/vladik1/Downloads/mediarover/mediarover/utils/configobj.py", line 2299, in validate
    check = self.validate(validator, preserve_errors=preserve_errors, copy=copy, section=section[entry])
  File "/Users/vladik1/Downloads/mediarover/mediarover/utils/configobj.py", line 2299, in validate
    check = self.validate(validator, preserve_errors=preserve_errors, copy=copy, section=section[entry])
  File "/Users/vladik1/Downloads/mediarover/mediarover/utils/configobj.py", line 2299, in validate
    check = self.validate(validator, preserve_errors=preserve_errors, copy=copy, section=section[entry])
  File "/Users/vladik1/Downloads/mediarover/mediarover/utils/configobj.py", line 2245, in validate
    missing=missing
  File "/Users/vladik1/Downloads/mediarover/mediarover/utils/validate.py", line 606, in check
    return self._check_value(value, fun_name, fun_args, fun_kwargs)
  File "/Users/vladik1/Downloads/mediarover/mediarover/utils/validate.py", line 638, in _check_value
    return fun(value, *fun_args, **fun_kwargs)
  File "/Users/vladik1/Downloads/mediarover/mediarover/config.py", line 664, in check_url
    if not re.match("^\w+://", url):
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/re.py", line 137, in match
TypeError: expected string or buffer

[rollingeyeball]

I've still not upgraded, sorry =/

[wally007]

I've still not upgraded, sorry =/

...And what version are you using ? 0.2 or 0.3 ? I was using 0.2 with sabnzbd.4.12 which worked great so i had no reason to upgrade to 0.3 but when this new version of 0.3 came out with nzbmatrix support i really wanted to try it. Seems like some Python error but at least one user got it working so it might be OSX 10.6.2 specific.

[kierse]

Hey wally007,

I was just thinking the other day at work how i'd go about solving SD->HD problem and i had an idea. I think MediaRover could have 2 types of RSS readers each with different privileges. Lets say lower priority feeds ( Number 1 ) would be SD and higher priority feeds would be HD ( Number 2 ) ( All nzb sites offer ability to filter HD and SD files so it wouldnt be problem to make sure SD stuff (No.1) doesnt get into HD (No.2) and vice versa )

So if No.1 feeds find nzb i'm looking for it'd set permissions READ-WRITE to resulting video file. Once this is done No.1 feeds stop looking for this episode. Feeds No.2 will still consider this episode as "not downloaded" since it has READ-WRITE permissions and it is considering only "READ ONLY" episodes as downloaded. So if No.2 finds same episode it'll have permissions to delete previously downloaded file by feeds No.1 and lock the new video file to READ-ONLY status. At this point both No.1 and No.2 consider this episode as downloaded.

I hope this makes sense. If i could program my way out of the box i'd try something like this but i unfortunately lack the knowledge. I'm not sure if something like this is possible so correct me if i'm wrong or if there's better way doing it without using crazy database tables which IMO would be overkill.

Interesting idea, I hadn't considered using the access permissions on a file as a means of indicating episode quality.  This would be fairly trivial to implement as well.  The only problem I have with this approach though is that it essentially hijacks a very important aspect of a files metadata (access permission) and prevents it from being used for its designed purpose.  While you (and many other MR users) may not have a problem with the approach but it could very well cause problems for some users.

Here's what I had envisioned to solve this problem:

• MR will support a range of qualities, say SD, Digital, and HD
• for each RSS feed, the user would indicate one of the supported quality levels
• the user would also indicate a preferred quality level in the config file
• MR would schedule downloads as usual, giving preference to episodes that  meet the user defined quality level
• MR would also attempt to replace lower quality episodes on disk with higher quality offerings whenever they appear in a higher quality tagged feed
• MR would record each successful download (along with its quality level) in an internally maintained data store (likely an sqlite database)

Note:

• MR would still defer to the file system when checking for the presence of episodes
• MR functionality would NOT be dependent on a populated data store.  Meaning that if the data store were to be deleted MR would continue to function.  The only functionality that would be disrupted would be episode quality upgrades

Thoughts / comments / concerns?  Any particular reason why you are opposed to MR maintaining a database (or other internally maintained record) of episode quality? 

Kierse

[kierse]

Is this version working for anybody ? I've tried on two machines and on both i get same error. Older version works fine on both machines but this latest release wont even launch. here's my terminal log

Code: Select all

PCserver:~ vladik1$ python /Users/vladik1/Downloads/mediarover/mediarover.py 
  File "/Users/vladik1/Downloads/mediarover/mediarover/config.py", line 664, in check_url
    if not re.match("^\w+://", url):
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/re.py", line 137, in match
TypeError: expected string or buffer

Hmm, it seems to be dying while processing your config file, specifically when processing the queue root field or one of the source url fields.  Do you have any weird characters in a url?  Did you regenerate your config files (things have changed a bit since 0.2.0)?

Kierse

[kierse]

...And what version are you using ? 0.2 or 0.3 ? I was using 0.2 with sabnzbd.4.12 which worked great so i had no reason to upgrade to 0.3 but when this new version of 0.3 came out with nzbmatrix support i really wanted to try it. Seems like some Python error but at least one user got it working so it might be OSX 10.6.2 specific.

You're going to want to upgrade to SABnzbd 0.5.0 as well (if you haven't already).  I haven't specifically tested any of the recent changes (since 0.3.0) on the old SABnzbd 0.4.xx branch but I doubt things will work properly.  The SAB team updated the API in 0.5.x and I upgrade MR to specifically support some of those additions. 

Rollingeyeball is correct, you won't regret the upgrade

Kierse

[wally007]

Hey wally007,

I was just thinking the other day at work how i'd go about solving SD->HD problem and i had an idea. I think MediaRover could have 2 types of RSS readers each with different privileges. Lets say lower priority feeds ( Number 1 ) would be SD and higher priority feeds would be HD ( Number 2 ) ( All nzb sites offer ability to filter HD and SD files so it wouldnt be problem to make sure SD stuff (No.1) doesnt get into HD (No.2) and vice versa )

So if No.1 feeds find nzb i'm looking for it'd set permissions READ-WRITE to resulting video file. Once this is done No.1 feeds stop looking for this episode. Feeds No.2 will still consider this episode as "not downloaded" since it has READ-WRITE permissions and it is considering only "READ ONLY" episodes as downloaded. So if No.2 finds same episode it'll have permissions to delete previously downloaded file by feeds No.1 and lock the new video file to READ-ONLY status. At this point both No.1 and No.2 consider this episode as downloaded.

I hope this makes sense. If i could program my way out of the box i'd try something like this but i unfortunately lack the knowledge. I'm not sure if something like this is possible so correct me if i'm wrong or if there's better way doing it without using crazy database tables which IMO would be overkill.

Interesting idea, I hadn't considered using the access permissions on a file as a means of indicating episode quality.  This would be fairly trivial to implement as well.  The only problem I have with this approach though is that it essentially hijacks a very important aspect of a files metadata (access permission) and prevents it from being used for its designed purpose.  While you (and many other MR users) may not have a problem with the approach but it could very well cause problems for some users.

Here's what I had envisioned to solve this problem:

• MR will support a range of qualities, say SD, Digital, and HD
• for each RSS feed, the user would indicate one of the supported quality levels
• the user would also indicate a preferred quality level in the config file
• MR would schedule downloads as usual, giving preference to episodes that  meet the user defined quality level
• MR would also attempt to replace lower quality episodes on disk with higher quality offerings whenever they appear in a higher quality tagged feed
• MR would record each successful download (along with its quality level) in an internally maintained data store (likely an sqlite database)

Note:

• MR would still defer to the file system when checking for the presence of episodes
• MR functionality would NOT be dependent on a populated data store.  Meaning that if the data store were to be deleted MR would continue to function.  The only functionality that would be disrupted would be episode quality upgrades

Thoughts / comments / concerns?  Any particular reason why you are opposed to MR maintaining a database (or other internally maintained record) of episode quality?  

Kierse

Obviously your solution is a lot more elegant but i don't know how long it would take to develop. I have absolutely nothing against database except (false?) sense of complexity for developer, you! :-)

Since my knowledge ends with xhtml/css and little bit of PHP i'm not the most qualified person to guess, but your solution seems like major undertaking. Do you have estimate how long it can take to develop your idea vs my idea ?

[wally007]

...And what version are you using ? 0.2 or 0.3 ? I was using 0.2 with sabnzbd.4.12 which worked great so i had no reason to upgrade to 0.3 but when this new version of 0.3 came out with nzbmatrix support i really wanted to try it. Seems like some Python error but at least one user got it working so it might be OSX 10.6.2 specific.

You're going to want to upgrade to SABnzbd 0.5.0 as well (if you haven't already).  I haven't specifically tested any of the recent changes (since 0.3.0) on the old SABnzbd 0.4.xx branch but I doubt things will work properly.  The SAB team updated the API in 0.5.x and I upgrade MR to specifically support some of those additions.  

Rollingeyeball is correct, you won't regret the upgrade

Kierse

Using sab 0.5.Beta6
( i was actually using sab .5.Beta but switched to .4.12 awhile back when MediaRover wasnt compatible with sabnzbd 0.5 )
Regenerated new config files.
But used copy/paste my rss feeds from 0.2 version. Gonna give it a try again. Thanks for response.  

EDIT -> Got it working. Seems like your implementation of nzbmatrix doesn't like multiple subcategories selected when generating RSS feed at nzbmatrix.com web site.

http://rss.nzbmatrix.com/rss.php?page=d ... &english=1
doesnt work

while

http://rss.nzbmatrix.com/rss.php?page=d ... &english=1
works.

So it's the comas that divide subcategories that break it. Thanks for steering me in right direction i now set up my rss feed like this to avoid comas yet still include all TV subcategories

url = http://rss.nzbmatrix.com/rss.php?page=d ... &english=1

[kierse]

http://rss.nzbmatrix.com/rss.php?page=d ... &english=1
works.

So it's the comas that divide subcategories that break it. Thanks for steering me in right direction i now set up my rss feed like this to avoid comas yet still include all TV subcategories

Hmm, yes the commas are definitely what's causing the problem.  I use a third party library to parse and validate the config file.  One of the features of this library is that it turns a comma separated string into a list of values.  My guess is that its taking your url with commas and returning a list.  I'll have a look and see if there is an easy workaround

Kierse

[daniel_owen_uk]

Hmm I spotted on a few reports that actually have "Passworded" in the title which with a bit of work on my RSS feeds, I could filter out.

A couple of days ago MR scheduled an nzb for download that had (Passworded) appended to the nzb title.  This pissed me off enough to go explore the newzbin search options and try to  figure out a way to filter them out.  For anyone who is interested, you can filter out passworded nzb's by putting '- "(Passworded)"' or 'NOT "(Passworded)"' in a search.  Obviously this won't filter out passworded nzbs that haven't yet been tagged (as rollingeyeball pointed out) but it will eliminate the ones that have.

Kierse

OLD post I know.

But just saw some news on newzbins site and you can now elect to exclude passworded posts

[jkristoph]

Fantastic Script!!  So far I have 3 feeds setup and its working great from what I can tell.  There is one small problem. I am using your new release for sabnzbd 0.5.0 beta6 and I keep getting an error user HISTORY in the sab webinterface when it attempts to call on the rover sorting script

"Script Exit{1} NameError: global name 'exit' is not defined" 

The funny thing is that the sorting script is working like it should and all my episodes are being sorted as configured.

Thanks again for your time and effort.

[jkristoph]

One more thing..."It monitors your filesystem for watched TV series, scours various newsgroup indexing sources for missing episodes"

How does it find missing episodes and not just grab newly released ones that?

I assume that one would have to build a custom report for this in newzbin for "All Tv Shows" and then feed the generated RSS link to Rover...is this correct if one wants to download all old episodes of a show?

[wally007]

Fantastic Script!!  So far I have 3 feeds setup and its working great from what I can tell.  There is one small problem. I am using your new release for sabnzbd 0.5.0 beta6 and I keep getting an error user HISTORY in the sab webinterface when it attempts to call on the rover sorting script

"Script Exit{1} NameError: global name 'exit' is not defined" 

The funny thing is that the sorting script is working like it should and all my episodes are being sorted as configured.

Thanks again for your time and effort.

I get that in sab's history log too. I dont think its anything important since its downloading / sorting as it should.

[rollingeyeball]

How does it find missing episodes and not just grab newly released ones that?

I assume that one would have to build a custom report for this in newzbin for "All Tv Shows" and then feed the generated RSS link to Rover...is this correct if one wants to download all old episodes of a show?

Spot on. Also in the event someone reposts an older season, MR will grab missing episodes then too.
Personally I'd use per-show custom newzbin feeds.