Manipulating Powerboost

[indolering]

I use VLC RAR-Loader/VLC to watch videos from multipart RAR files before they are decompressed.  I have Comcast who has something called Powerboost which maxes the line speed out for the first 10 megs of a download (Cox, Time Warner, and some Canadian cable Telco offers the same thing).  It would be awesome if SabNZBd could stagger downloads so it is always catching the 10 meg window, that would essentially rip Comcast off by using their top tier service for half the price!

You guys rock!

-Zac

[dasfiend]

disclaimer: I'm not 100% sure since I've never had a news client that would do it, however.....

I believe that if sabnzbd were to keep a certain number of connections active, let's say 2 for arguments sake, and then open say a 3rd and a 4th for a certain amount of data, then close those connections and open new ones right away this would do as "pair of dimes" desires.

Alternatively you could simply rotate all threads after X amount of data, so you have 8 connections that do the following:

1) connect, download a set amount of data
2) disconnect
3) reconnect and repeat the process

not sure if that would annoy the usenet hosts to have that many logins and logouts either......

//edit: Powerboost works such that the first X megabytes are not throttled, and thereafter they are. I don't have any good technical information on how this is achieved, and the comcast site for it is utterly useless: http://www.comcast.net/powerboost/

it should be noted that my comcast service is 16 Mbps full time and then Powerboost above that.  When I was using altbinz and getting realtime speed updates my speed would peak at about 3500 kB/s before the throttling would kick in and send me to 2000.

Hope that helps,

[Snatch]

I would like to second this feature request.

If we could have an option to cycle the connections after X Bytes/Seconds, it would really helps us *take advantage* of the powerboost feature! 

Comcast also has a FAQ about SpeedBoost here: http://www.comcast.com/Customers/FAQ/FaqCategory.ashx?CatId=377&CMP=ILC-fcomcastnetpbhelp

I'm sure the usenet providers may not like us reconnecting so often, but maybe it doesn't make any difference to them...

What's the likelihood of such a feature/option being added? I'm fairly new to sabnzbd, and I'm very impressed thus far!

Thanks for everything!

[shypike]

We're not very motivated to include specialty stuff which is
a) of limited use to other users
b) potentially a disadvantage for other users
c) almost impossible to test for us

Cannot you get a better ISP? 

I'll discuss it with the team.

[Snatch]

First off, thanks for the quick response! In all reality, the feature that we would be looking for is an option to drop active server threads after a static byte limit that the user could set. Sort of like your speed limit option, for example.

Theoretically, a) I assume that there is a significant amount of users with an ISP that offers this boosting feature, b) you could make this optional for users to select as to not become a disadvantage to them, and c) all you'd have to test is verifying that individual server threads terminate at the set byte limits (as configured by the user). It shouldn't be hard to prove with Wireshark or some other network sniffer. I'd be happy to help test if you'd like, I'm just not good at coding.

Please don't get me wrong, I consider my ISP to be pretty decent. There really is no other broadband options that offer speeds at the 6mbps+ right now in my area at least. Honestly, I'm very happy with the speeds they provide. However, as they offer this feature that boosts their speeds, it is natural to question if we can use this to our advantage.

Thanks again!

[switch]

Are you sure the speeds will go up by simply reconnecting to the server? What happens when you press "Force Disconnect" in the Connections page after SABnzbd has been downloading for a while?

[Snatch]

I can verify and assure that they start about 1500KBps and drop to 750KBps. It does this with every download I do. This is consistent regardless if it is usenet, HTTP, FTP, etc.

Specifically, I configure sabnzbd to disconnect when checking and unpacking the NZB. When it reconnects it jumps to 1500KBps for a short time and drops back down to 750KBps like clockwork.

EDIT: I tested the force disconnect and the throughout does jump up to the 1500KBps and then ramp back down to 750KBps

[Karyudo]

Canadian cable provider Shaw uses this same kind of SpeedBoost. I don't know if it's time- or size-based, but I get about a 20-30 second boost at 20Mb/sec before dropping down to a steady and reliable 10Mb/sec. Stopping and re-starting a download in NZB-O-Matic+ (NOMP) definitely "reset" the SpeedBoost; I don't see why it'd be different with SAB. I think this would be a cool feature.

[shypike]

Filed as ticket #78.

[inpheaux]

Hate to be the voice of dissent here, but I really don't think it's a good idea to start building features that specifically side-step ISP limitations. It's one thing to include something like SSL, which similarly eludes packet filtering of NNTP traffic, but has legitimate use. It's a completely different thing to add a feature which is nothing but a checkbox labeled "FUCK COMCAST".

[Snatch]

I don't consider it as F'ing Comcast. To the contrary, I would consider it a utilizing a feature that my ISP offers for a service that I pay well for.

To call it F'ing Comast it analogous to say that if I were to use the 2GB a month they give us on Giganews as also F'ing them. Or if I used the Antivirus client software they offered for our regular monthly fees, is that F'ing Comcast? No, it is simply using the features that the ISP provides with your stated monthly service agreement. Hell, to expand on that theory, simply browsing the web would be considered as F'ing them as well!

Furthermore to reiterate the feature we're talking about, it is simply to reconnect each thread of the server connection after a stated byte limit. That has nothing to do with the ISP. It's really just refreshing server threads at regular intervals. Granted the side effect of this feature would be increased throughput, but the ISP has control over that throughput not the users.

Also to expand, I expect the feature would be listed as a checkbox with a text field where the user can state their preferred byte limit before cycling the connections. Letting the user control the threshold is far more useful for users. Not all ISP's offer that boost at the same byte limits. As noted by other users, there are numerous ISP's that use this feature and I assume they have varying limits they use with the technology.

Cheers!

[inpheaux]

I don't consider it as F'ing Comcast. To the contrary, I would consider it a utilizing a feature that my ISP offers for a service that I pay well for.

Except the intended use of the feature is to only boost the first 5 Megs. Gaming Powerboost to be active all the time isn't use of the feature, it's abuse.

Also to expand, I expect the feature would be listed as a checkbox with a text field where the user can state their preferred byte limit before cycling the connections. Letting the user control the threshold is far more useful for users.

What would be the legitimate use of this feature? That's my concern, adding features which have no legitimate use and could thereby piss large telecommunications conglomerates off.

One workaround I was thinking was to build this as an arbitrary cap-size scheduling feature. It would have two inputs, a size in bytes and length of time. So if you wanted to cap your usenet access to 10Gb over 7 days, you'd enter that and when it reaches the capacity it stops until you hit the time limit. But if you wanted to abuse Powerboost or similar services, you would determine how long it takes to download the byte limit at the boosted speed, and then enter that byte limit with a slightly higher time limit. For instance, for Comcast Powerboost you'd enter 10Mb and however many seconds it takes to download 10Mb at the uncapped speed, plus one second.

Then you'd have a legitimate use and if you wanted to use it in an unsupported manner that helps you game Powerboost, so be it.

[Snatch]

This insinuation of abuse and the theory of "sticking it to the ISP" is only as valid as the reasoning behind allowing a user to change their NNTP server port setting! Seriously, why allow users to change this port setting? Maybe to bypass certain ISP limitations, per chance? Wouldn't this also follow your abuse philosophy?

I seriously don't intend on having a religious debate on whether or not using a feature that I'm paying for is considered abuse. IMHO, if you don't feel it's appropriate for your usage, then you don't have to turn it on. However, I firmly believe in using the services that I'm paying for.

[inpheaux]

This insinuation of abuse and the theory of "sticking it to the ISP" is only as valid as the reasoning behind allowing a user to change their NNTP server port setting! Seriously, why allow users to change this port setting? Maybe to bypass certain ISP limitations, per chance? Wouldn't this also follow your abuse philosophy?

Setting the port has valid usage. NNTP - just like most protocols - isn't required to run on port 119, it's just suggested to run on 119. To ensure compatibility with any NNTP server we have to allow you to change the port. Similar to SSL, it just has a side-effect of dodging poor attempts by ISP's to filter NNTP traffic (assuming they base it off the port, not the content).

I may have come up with a compromise: use the API.

Here's some pseudocode for a potential third-party application:

Code: Select all

while there are bytes queued...
   stat the queue
   if [x] bytes have been downloaded since last cycle
      force disconnect
      force reconnect
      start new cycle
   endif
endwhile

We presently don't have api hooks for force disconnect / force reconnect, just pause/unpause. To my understanding, pausing doesn't do enough, you have to actually disconnect for it to work, so we'd have to write those. Those hooks, however, would have legitimate use.

[Snatch]

First off, I can see your point of view and I honestly appreciate your offering for a solution that may fit this feature request. It is sincerely appreciated and I hope that it can be incorporated effectively.

However, I would like to offer some correct to a couple points that have been made:

Setting the port has valid usage. NNTP - just like most protocols - isn't required to run on port 119, it's just suggested to run on 119. To ensure compatibility with any NNTP server we have to allow you to change the port. Similar to SSL, it just has a side-effect of dodging poor attempts by ISP's to filter NNTP traffic (assuming they base it off the port, not the content).
...

You are correct in the fact that destination port control does have a "valid" use. In so much as does the feature that is being requested in this thread.

However, RFC 977 (Section 2.1) for NNTP specifically specifies TCP port 119 as the "assigned" destination port. Please reference: http://www.ietf.org/rfc/rfc977.txt:

When used via Internet TCP, the contact port assigned for this service is 119.

Given RFC 977, the only "legitimate" reason a Usenet server would reference a non-standard port would be to bypass usage restrictions set forth by some down-steam ISP (from the perspective of the Usenet provider). In my opionion, your reference of "valid" use in this case would still fall under the same theory of abuse and misuse in this case.

As noted, some ISP's will not permit NNTP via it's assigned port or throttle said traffic. Given an ISP that blocks this port on their system, the user would not be able to access their Usenet provider. Hence, their only option is to change ports or tunnel the traffic via SSL or some other mechanism. Having sabnzbd support alternate port assignments, and SSL for that matter, alleviates these shortfalls. I believe this defines your reference to it's "valid" use. However, it still does fall into the theory of abuse and misuse of the ISP's services.

All I'm requesting is that we stop the inference of considering this feature to be for the purposes of abuse or intentionally bypassing acceptable usage policies. Additionally, I hope we can accomplish a solution that will make us all happy. 

Cheers!