Download Cache Server

[RubenPortier]

I have no idea how much work it would be and if it's even possible on the hardware supported by Asuswrt-merlin, but I think this would be a really welcome addition to the already largely featured firmware.

My idea is a download cache server, which will cache everything you download, which should result in less bandwidth usage on the ISP-side. There are some software apps that are available on various linux-distributions (e. g. Squid, Polipo etc.), but most of them seem to not be able to do the task as wanted. You don't want to cache every webpage you visit or e-mail you download, but only from specific applications, Steam, Origin, UPlay and GOG-Galaxy for example.

A lot of people probably have multiple persons that one or more of these services to download and install there games, but having to re-download the whole game again every time consumes a lot of bandwidth. I think a good download cache server would help a lot with this, as long as you have the available disk space attached to the router of course. I know I could install this kind of service myself, but having it included in the webui would make for a better experience overall and would be very user friendly.

This was just an idea I came up with and I'd like to hear your thoughts on this. Would it be too much work, is it not needed or is this a must have?

 

[System Error Message]

If you can install squid proxy on the router you can use it as a cache. I am already doing this with my ubiquiti edgerouter pro so it should be possible with asus depending on if there is a version for the ARM used and the linux kernel and any extra libraries needed.

 

[ColinTaylor]

I gave up using squid a long time ago. Nowadays, so many web pages contain dynamically created (and large) content that it just wasn't effective any more.

Even back in the day when there was more static content you needed a lot of free RAM to get any kind of decent hit rate. Something ASUS routers don't have.

 

[RubenPortier]

@System Error Message: I've no experience in using Squid, but I think it's shouldn't be that hard to install it on an Asus router. The main reason I created this topic is that I wanted to point out that having Squid or any other kind of caching feature included in the webui would make it a whole lot easier for people who are not really into IT tech stuff.

@ColinTaylor: Thank you for the response. I'm aware that there is a lot of dynamic content nowadays, but there are still a lot of services out there where a caching server would be useful. Think of huge downloads in Steam for example. The problem is however, they use a lot of CDN's to download from and that will make the cache server think it's a different location and therefor different content. There are however ways to accomplish that the cache server knows about all (or at least a lot) of the different CDN's used.

I know it's something hard to discuss, because we are getting faster networking speeds by our ISP's (in most countries/regions) and the download limits that our ISP's are setting is mostly enough for downloading two or three games a month and do other average network usage. However, downloading from a local cached server on a 1 Gbits network could really make our lives a lot easier. I've had many times where I want to play a game I have on Steam for instance, but it's not installed. However, I did install and reinstall is a couple of times, but had to remove it because I didn't play it for a while and needed the local disk space. When download from a cache server, it would just be seconds to just a little couple of minutes to download this game (it would need to update and download soms small files from Steam servers, but most of it would come from the cache server).

 

[RMerlin]

I have no plan to implement this.

 

[sfx2000]

I have no idea how much work it would be and if it's even possible on the hardware supported by Asuswrt-merlin, but I think this would be a really welcome addition to the already largely featured firmware.

My idea is a download cache server, which will cache everything you download, which should result in less bandwidth usage on the ISP-side. There are some software apps that are available on various linux-distributions (e. g. Squid, Polipo etc.), but most of them seem to not be able to do the task as wanted. You don't want to cache every webpage you visit or e-mail you download, but only from specific applications, Steam, Origin, UPlay and GOG-Galaxy for example.

Interesting idea - not sure if this is something, e.g. one more service, to run on a Router/AP - but having this on a NAS would be a good thing - windows updates, iTunes, etc... (some of these can be quite large), whether it's for the home, or a small office (10 PC's getting Windows updates can be quite a bit of data)...

 

[L&LD]

Every time I've tested a cache server, it would perform flawlessly during testing and fall on it's face in actual use. It was disabled more time than it was enabled.

That was a few years ago, I don't see what would have changed today.

 

[Hawk]

My thought on this it's more trouble then worth. With ever increasing Internet speed I see little or no benefit, but in some cases it may benefit.

 

[lancethepants]

I've always wanted to implement a lancache for game downloads for lanparties. I'd setup dedicated hardware for this though. Having 20+ people download multi-gig games all at once sucks, even on fast connections, and takes away from game time.
https://www.google.com/webhp?sourceid=chrome-instant&ion=1&espv=2&ie=UTF-8#q=nginx steam cache
https://wiki.openwrt.org/doc/howto/cache.steam

 

[Nullity]

Encrypted traffic (which the internet is migrating towards) cannot be cached.

 

[RubenPortier]

Encrypted traffic (which the internet is migrating towards) cannot be cached.

You have a point there!

It was just a poll to see if people would be interested in such a feature. But as noted, RMerlin has no plan to implement this kind of feature for now and we can still install the cache server ourselves on the router. It could also be installed on a NAS-server of course, but it's always a lot of work setting up the proxy on various clients.

 

[System Error Message]

you wont get gig traffic from cache from an asus router as even NAS is slow.

 

[RubenPortier]

you wont get gig traffic from cache from an asus router as even NAS is slow.

Depends on what NAS you have. I have a Intel Q6600 2.4 Ghz with 6 GB DDR2 Memory and 2x 500 GB HDD in RAID0, I can can assure you I can max. my 1 Gbit/s line with writing and reading to and from that NAS with ease!

 

[System Error Message]

Depends on what NAS you have. I have a Intel Q6600 2.4 Ghz with 6 GB DDR2 Memory and 2x 500 GB HDD in RAID0, I can can assure you I can max. my 1 Gbit/s line with writing and reading to and from that NAS with ease!

What i mean is if you were using the ARM based ASUS router as a cache.