Limiting the bandwidth for large downloads on a slow internet connection
On our super-fast 446 kbit/s internet connection, which seems above all to be configured wrong by our ISP, downloading a file with full speed results in ping times around 5000 ms. This is of course unacceptable, as every single web page takes several minutes to load then. The problem can be fixed by manually limiting the download rate to 20 KB/s or so when you plan to download a large file, but this is not always easily possible, for example when watching a YouTube video. An additional problem is that two clients that both limit their download rate to 20 KB/s can hang up the internet again when they download simultaneously. Furthermore it would be great if normal surfing, small downloads, would not be limited in bandwidth for keeping the speed at the possible maximum.
The best solution I could imagine was a transparent proxy that limited the download rate of large downloads automatically, letting small downloads pass through at full speed. Those large downloads should share a specified bandwidth, so multiple downloads would not make the internet slower than a single one.
So I wrote a simple TCP proxy that does exactly this. It listens on a port for incoming TCP connections. For each connection, it opens another TCP connection to a specified port on a specified server (so it is best used in combination with a HTTP proxy or whatever procotol you want to use it for) and redirects the traffic between these two connections. As soon as a configured download traffic limit is reached, the transfer is switched to “low-priority mode”, where it has to share a specified bandwidth with all other downloads running on low priority.
On my local server there already is a HTTP proxy running as cache. If I just used my bandwidth-limiter TCP proxy now and connected that to my HTTP proxy, the bandwidth limit would also be applied for transfers that are already cached by the HTTP proxy and thus make the cache useless. So I have to start an additional transparent HTTP proxy server that does not do anything except for redirecting the incoming connections from my bandwidth limiter to the internet. Sounds a bit like too much work and too many redirects, but it is the easiest solution I have found, as Squid does not seem to support the kind of bandwidth limit I need. So my configuration looks like this: My HTTP proxy cache (in my case Squid) is configured to use another proxy, which I set to localhost and the port my bandwidth limiter listens on. The bandwidth limiter connects to another HTTP proxy running on localhost, which is configured to do nothing but forward the request to the Web.
My bandwidth limiter (called “bwproxy”) is written in Java, you can download it from https://github.com/cdauth/bwproxy. It is available under the terms of GPL-3, feel free to send modifications or bug reports to me.
Filed under bugs