TP-Link ER605 + ASUS GT-AC5300 - DUAL WAN

[Smokey613]

2x EAP610 V2

I was thinking about the EAP650

 

[Tech9]

More money for something not guaranteed to work.

 

[Smokey613]

More money for something not guaranteed to work.

True and my network really would not benefit from them…. even if they worked.

TBH, the eero Pro do all I really need, if they would allow an option of when a firmware update was applied I would look no further.

 

[Tech9]

No way to find and block the update servers?

 

[Smokey613]

No way to find and block the update servers?

There have been several that have tried but unfortunately, the eero system relies on cloud control and blocking the update also prevents the eero units from working.

 

[ForkWNY]

Here's a router comparison, Fios 1Gb symmetrical ISP - I have a speedtest script run off a Raspberry Pi midday and midnight every day. You can see the performance difference across the 3 routers, it says a lot about reliability...it's not fair to compare a $60 ER605 to devices that are 10x as powerful and cost 4-5x as much, but it gives you a good idea of the kind of performance to expect when you get up into the 1Gb range for upstream/downstream.

This graph is charted over a 30-day span, the last few days on the OPNsense device show promising trends as far as performance. Some of those 0/0 dips are caused by the speedtest server being down or something else like glitching, I mostly ignore those unless I have no connection that day. The overall trends are what matter. Some of the TP-Link box results are from when I was messing with configuration and making some tweaks...the dip in the OPNsense graph was a result of some tweaking/testing where I accidentally throttled my network to 100Mbps. Been fairly smooth sailing for the past few days w/no reboots on the OPNsense device.

Upstream was rather dicey on the ASUS, while it could achieve 1Gbps upstream, it was spotty. The TP-Link ER605 simply could not achieve above 1Gbps throughput against most speedtest servers I tested. The OPNsense device has no throughput issues upstream or downstream and appears to be reliable at this point.

Blue line: downstream
Red line: upstream
Green line: ping times (which vary for reasons well outside the control of my network hardware)

 

[Smokey613]

Interesting information. Yea, this little ER605 is impressive for the money. I have even dropped back to running only 1 eero Pro. I noticed that the indicator light on the eero Pro I was using for my "primary" unit was a lot dimmer than the other 3 units I have. I swapped it for one of my spare units and now, that single eero Pro easily covers all my wireless devices.

I am not sure but maybe there was a hardware issue with that eero unit or something. All I know is the eero unit I replaced it with handles everything on it's own.

 

[ForkWNY]

Nah the hardware was fine...it was capable of 1Gbps upstream, and I could get it to hit 900Mbps+ upstream using iPerf and linking two Gig laptops together with the ER605 in the middle. The ER605 is a great solution for most...I would not recommend it to anyone with a 1gb symmetrical connection. It seems ok over the WAN for speeds 300-400Mbps or less upstream, and it does perfectly fine up to 1Gbps downstream. Definitely a solid choice at its $60 price point.

 

[Samir]

A little late to the party, but used enterprise routers can easily handle multi-wan and in speeds well beyond what you have. Our older watchguard m200 which you can find on ebay for like $50 handles 500/500 and 500/50 and 300/20 connections without the cpu even going above 0% and they have options galore. The best part is that these were still current models as of just a few months ago, so a newer model would easily handle pretty much any number of wan connections with ease.

And I've been doing multi-wan since 2004 on the cisco rv016 with 3x cable modems back then to aggregate upload bandwidth. The enterprise implementations of multi-wan are seamless as can be and are just set and forget and work fantastic compared to smb or consumer or even diy options.

 

[ForkWNY]

So far my OPNsense box has been doing a great job of load balancing/failover across two 1Gb WANs. For the most part it has been seamless, nearly set-and-forget.

The one minor issue I did notice was that sometimes Netflix would freak out when loading new shows, or if it was going from one episode to another in a mini-series...I noticed if OPNsense dumped Netflix out over another WAN between episodes, it would sometimes just sit there (with the circling arrow) and I'd have to exit out of the show or movie and go back in to re-load it. This problem also only impacted a Roku device, which is connected via Wifi and not a cable. The fix was simple...create a firewall rule, on the LAN interface, above the LAN-to-any rule, to force the Roku out the primary gateway by MAC address. This is fairly common for not-so-smart streaming media devices, from what I've read in various forums, when load balancing between two WANs, regardless of the device being used for load balancing. Otherwise OPNsense has handled load balancing with ease, and since I can apply weight to each WAN, I can route the majority of traffic over the faster fiber WAN, which is a 1Gb symmetrical connection.

I also set up LACP LAG aggregation between the OPNsense router and my gig managed switch, which definitely comes in handy when simultaneous 1Gb downloads are occurring over both WANs as that would saturate any of the 1Gb ports on the switch, and limit the downloads to 500mbps if LAG wasn't in use. I've run speedtests over both WANs simultaneously, 1Gbps downstream and upstream, and the OPNsense router handles the 2x1Gbps concurrent downloads with ease, CPU doesn't break a sweat.

 

[Samir]

Definitely a nice opnsense setup that's doing the job. Surprising that it uses the second wan when the first is saturated if you've got failover set. A lot of multi-wans won't touch the second wan in failover mode unless the first is down.

 

[ForkWNY]

Definitely a nice opnsense setup that's doing the job. Surprising that it uses the second wan when the first is saturated if you've got failover set. A lot of multi-wans won't touch the second wan in failover mode unless the first is down.

If you set both WANs to the same Tier in the failover gateway group, the two links will be load balanced. If there's packet loss on either of the WAN interfaces, OPNsense will consider the link down and route all traffic out on the WAN that is up/online, unless you have firewall rules that state otherwise. When the WAN link comes back online, it'll resume routing/load balancing traffic over that link. If you want to favor one WAN interface over another, you can apply weight values to the WAN where you want most of the traffic routed. Sticky connections should be enabled when load balancing is used, otherwise some websites and streaming apps won't work correctly if the external IP changes during the request/receive process.

https://docs.opnsense.org/manual/how-tos/multiwan.html - you can see that failover/load balancing can be combined. Might as well utilize both links if they are the same speed (though the upstream on my cable WAN is 40Mbs vs 1000Mbps on my fiber WAN).

You can see how the weighting works between the two WAN interfaces...I have a 2:1 weighting on Fios so most traffic goes over that link. It's not quite 2:1 because I force a number of devices out over Fios to benefit from the 1Gb symmetrical link, especially streaming devices where Netflix sometimes has some issues if the IP changes because of load balancing.

The speedtest testing I was referring to I purposely set up to run over each WAN to test LACP LAGG between the router and the switch. I don't believe OPNsense will automatically use the other link if one is saturated. I intentionally set up the speedtest to ensure I could run them concurrently over each WAN link (via Firewall rules) to verify that LACP LAG was doing its job.

The screenshot above (from Omada) shows that link aggregation is working as expected over switch ports 17 & 18 on my TP-L SG2428P managed switch, with the traffic being nearly 50/50 over those two switch ports which are 1Gbps ports that are aggregated with the OPNsense router. The LAG config is overkill and it probably only adds marginal performance gains on my network...most of the time I don't come close to saturating any of the 1Gbps ports on the switch. It's more beneficial for NAS where saturation is more likely if numerous clients are hitting the NAS concurrently.

 

[Samir]

The load balancing makes sense now as it's basically 'load-balancing with failover' which is the same thing I've seen in enterprise routers. The older smb ones would literally only do failover or load-balancing, not both concurrently. Weighting and forced routing is nice too since a lot of sites that use ssl will break without it. The LAG is pretty neat. I think your set up is the first time I've seen someone set up something like this successfully.

 

[ForkWNY]

Most get hung up with load balancing in OPNsense/pfSense here: Firewall > Settings > Advanced > Multi-WAN

Shared forwarding is enabled by default, sticky connections is not enabled by default. There's a lot of confusion as to why this is the case (it's probably because most just want failover, and not load balancing), but if sticky connections is enabled and shared forwarding is left checked/enabled, load balancing will not work. The key is to turn off Shared Forwarding when both WANs are set to the same Tier for load balancing. Ran into this issue myself and had to dig through OPNsense's forums to find this info.

Now that ChatGPT is out there, it literally walked me through a working OPNsense multi-wan setup with VLAN configuration in a matter of 20 minutes. Amazing what AI can do. It knew with precise granularity how to set up my network config.

 

[Samir]

Yep, these type of quirks usually exist on any platform that's dealing with multi-wan, but what I was impressed was how you were able to get the LAG to work with speed tests to be able to do 2Gbps in tests.

 

[ForkWNY]

Not the best screenshot, but shows 2 concurrent speedtests running against the same speedtest server on 2 separate WAN links. The 2 shades of red represent the concurrent downstream WAN tests over 2 ISP's, where the shade of blue on the bottom represents the combined LAN output (~1.60Gbps...the tag was just before the crest of ~1.8Gbps) going out the LAN interface during the tests. That combined LAN throughput wouldn't be achievable without LAG between the router/switch.

It's difficult to time them just right...one test was run from a script using the speedtest CLI on a Pi, the other was run from a Win 11 laptop from a web browser (both devices are directly cabled to the 1Gbps managed switch), and the timing on the web browser is always variable with regards to how quickly the throughput ramps up to 1Gbps from the laptop. I wanted to have them plateau at 1Gbps each at the same time, but only got a brief overlap. The last blue shaded graph at the top was the 1Gbps upstream ramp-up which overlapped the 1Gbps downstream feed from the laptop.

 

[Sandoak]

I am trying to deploy a TPLink er605 in front of my network, just as described in the post, in front of an Asus ax-89x router.

I configured the er605 to handle DHCP for a network that is different than the rest of the network being handled by the Asus router and plugged the er605 lan port in to the wan port of the ax89x.

The ax89x sees the internet connection and is able to use the internet connection. The er605 sees both of my incoming wan connections. I have turned on load balancing. My wan connections are a cable 1000 down / 35 up and a cellular ~100 down/20-30 up.

Previously when I tried to use the dual wan functionality of the ASUS I could see the combined speeds of both wans when doing speedtests. The reason I am looking for a new solution is because the Asus router kept dropping the internet connection and would require a restart.

The er605 seems to send all internet traffic over the cable connection in the wan port and is not really using the second connection. I have tried a variety of speedtest scenarios and I can’t achieve any speeds faster than my cable connection, even when running speedtests from multiple devices/wired vs wireless.

Any help would be greatly appreciated. I have a bit of network activity happening behind my Asus router. I did not want to transfer the full DHCP/routing functionality to the er605 because I wanted to keep the 10g processing of the Asus router for the network.

Thank you

 

[Smokey613]

To do proper load balancing on the ER605, you need to go into each WAN connection and set your bandwidth speeds so the ER605 will know how to handle the traffic. Also, do not create an entry in the WAN failover part of the configuration. I am not at home right now but I can take some screenshots to explain it better later today.

Check my signature. I can do a speed test and get around 390/125 results.

The one caveat is you may need to enable the AP routing features on the Load Balancing page for secure services like online banking. This feature will maintain the secure connection made to the online secure sight depending on which WAN link it originally connected with.

I am more impressed with this little $60 router all the time.

 

[Sandoak]

To do proper load balancing on the ER605, you need to go into each WAN connection and set your bandwidth speeds so the ER605 will know how to handle the traffic. Also, do not create an entry in the WAN failover part of the configuration. I am not at home right now but I can take some screenshots to explain it better later today.

Check my signature. I can do a speed test and get around 390/125 results.

The one caveat is you may need to enable the AP routing features on the Load Balancing page for secure services like online banking. This feature will maintain the secure connection made to the online secure sight depending on which WAN link it originally connected with.

I am more impressed with this little $60 router all the time.

Thank you so much!

Using the actual speeds in the WAN settings and shutting off the failover setting has allowed me to get the combined speeds up. What I find odd is that the speeds slow down when I click the box to turn on bandwidth based balance routing. That seems odd.

Now I need to set up ddclient and I will be home free. Much better than shopping for a new 10g dual wan router!!

 

[Smokey613]

I found the using the bandwidth based balancing was not useful.

BTW, I found little help on all of this. I just did a lot of trial and error to get to where I am today with the ER605.