Limiting file size of nzb or queued file?

[clockwrkz]

A few people have been misnaming 40GB+ files at tvnzb.com which are finding their way into my RSS feed (expected file size would be 300-500mb).

Is it possible to either limit the size of the .nzb files or (better yet) the actual file to prevent exceedingly large files from being added? Seems like a good denial of service attack method if you want to prevent people from using RSS feeds to download content via RSS.

If neither of the above is possible today, can I have SABnzb add the files to queue but not download them until I manually approve them?

Using 0.5.0 with the Plush interface.

Thanks in advance

[shypike]

Alas no.
You might give them low priority based on the filter.

[clockwrkz]

Thanks for the quick response.

I haven't been using SABnzb for long but is there a plug-in type system where I could write my own filters for RSS feeds? I know there is a FILTER space which can accept regex commands (I believe) but I'm assuming this doesn't have access to the file size variable based on your response.

Otherwise, the only other option I can think of would be creating a RSS proxy which could handle the file size filter.

[shypike]

There's no plugin system.
The filters work only on the RSS titles.

What will come sooner is a pre-queue user script.

[RKM]

@shypike

Will you please consider adding a filter-by-size option as mentioned by the original poster?

I will personally pay a bounty on this.  I'm sure others will too.

This has recently become a very important concern in NZB downloading tools, here's the reason:

There is a new trend of newsgroup spam, where the person uploads a HUGE amount of content under the name of an normally automated .NZB under RSS.

For example, the one that occurred yesterday was a "Chuck" episode that slammed everyone.  It was supposed to be a 350mb episode, and instead we all got hit with a 24 GB download.  Later, "House" downloaded -- all before I realized that SABnzbd+ had downloaded 24 GB instead of the intended 350.  That's a total of 48 GB wasted on NZB spam.

I'm already close to my limit with Comcast (250GB/month) and I can't risk another hit like this, so I have to disable RSS feeds entirely until we can integrate a "filter" for "file size".

Thank you in advance,
RKM

[shypike]

I'll see what I can do.
Have you also complained on tvnzb's forum?
It's best to tackle this sort of nonsense as close as possible to the source.

[fusen]

I'm also one of the tvnzb users that got hit by this;

I solved it by creating a yahoo pipe that filters the tvnzb rss feed based on the reported filesize;

http://pipes.yahoo.com/pipes/pipe.info? ... 66775e05e5

[shypike]

Good idea!
We should be promoting Yahoo Pipes as they are much more powerful
than what we ever can  accomplish in SABnzbd's RSS support.

BTW: are these pipes always public?
I can see this one.

[fusen]

I made it public for cases like this where I can show others it

you can make private pipes though if you want

[RKM]

@fusen-
Thanks a million for the post.  I use Yahoo Pipes myself, so this was an easy addition.  I had spent about 30 minutes looking all over for a method in Yahoo Pipes, but thought it was impossible until I noticed your demo.

For you/others I have some related tips for Pipes, if you're interested.

First, if you'd like to get a perfect match for your shows, the easiest way is to use TVNZB's ID's.

Under their feed: www.tvnzb.com/tvnzb_new.rss

Simply select to match the column: item.show_id

To get a list of Shows and their ID's, simply "View Source" on the home page of TVNZB and view the list shown in the drop-down box.  Alternatively, I've posted a list of shows here: http://rapidshare.com/files/364593515/T ... D.txt.html

The easiest way to do this is to use "OR" logic.  To do this, you'll use a "pipe" character between each show ID. 

For example, here is the full Yahoo rule:
item.show_id ... matches regex ... 299|583|167

That's a snippet from my show list, the first three in alphabetical order (299=30 Rock, 583=Better Off Ted, 167=Bones)

That first section of rules would be the "include" section.  The second section would be the "exclude".  For that, we create a rule that "Blocks" "Any" of the following.

If you want to remove season rips (past shows, etc), then remove anything with these names: season (or) dvdrip

If you want to remove high def content (which I realize Fusen, the poster above, does not), then exclude these:
x264 (or) 720p (or) 1080p (or) 1080i

Other exclusion rules you might play with include ".AU." to remove Australian versions of shows that are shown here, for example.  The only reason I mention this, is because you have to be careful with certain filters.  If I used "AU" alone, then many shows would get caught, so it's important to list it with the "dots" around it, which is how the releases will be named, and therefore, only "AU" will be isolated, rather than an episode with a title containing AU, like "Author" (etc).

Finally, apply Fusen's exclusion, to remove spam:
item.enclosure.length (is greater than) 300,000

In this case, you'll successfully download anything up to roughly 1.1 GB.  Since 300,000 is not a perfect match to the size (but is very close, roughly 100k to 350 MB), this should cover any TV episode.  If you need this number to be higher/lower, modify knowing that most SD shows fit within 100,000 (both one hour and half-hour) or HD shows about double that -- and work around that.

Hope this helps some others with Yahoo Pipes.  IMHO, it's the perfect compliment to SABnzbd+!

[bdavies]

You guys rock!!! What a great idea, thanks very much, this is some great collaboration work and people should continue to share ideas.

Thanks again.

[bdavies]

Heheh any ideas on how to now use the categories

Would it need a post processing script since the feed is being brought in externally?

[shypike]

A feed is a feed for SABnzbd.
Just enter the Yahoo feed URL in Config->RSS like any other feed.
You can still setup some filters to distinguish categories.
You may want to let the last filter match on "*", to capture anything non-categorized.

[bdavies]

hahah yeah i figured out my stupidity after posting that.  Thanks for the quick response.

[RKM]

Now we just need to figure out some method of accomplishing the holy grail of newsgroup auto-downloading:

Sniffing out which NZB's occasionally refer to content that is zipped with password requirements.

In all NZB's containing password protected files, there is usually an HTML or TXT file (could be any extension really, but will be less than 50k) that notes something about the password.  If we could see-out, and avoid downloading files with attribute (much the same way that SABnzbd+ avoids "sample" files), we'd be golden!