ASUS Dual WAN Failover, Then Failback, Leaves Secondary LAN in "Hot Standby"

[AndrewL733]

Hello all,

I have an RT-AC68U with 386.12.4 firmware. I am successfully running a Dual WAN with the Secondary WAN set up for Failover + Failback. (I'm actually kind of surprised it works so reliably, given the many references I'v seen in this forum to a user-script to take the place of the native ASUS Failover code. See: https://www.snbforums.com/threads/dual-wan-failover-v2-release.83674/ )

Anyway, I do have one serious problem with the native ASUS Failover/Failback feature. When my router "fails back" to the Primary WAN, it leaves the Secondary WAN in "Hot Standby" and both WAN paths are actually still accessible. Something I don't want. I'll explain this all in detail below, but this is a problem for me because I have a "VPN Appliance" (home-made little box, running Ubuntu + Wireguard + GRETAP to create a layer-2 virtual ethernet switch) that continues sending traffic over the Secondary WAN even once the Primary WAN is back up. Given that my Secondary WAN is a 4G/LTE device, this has caused me to run out of mobile data on my SIM card.

I have come up with a workaround to solve my problem (so that I don't keep running out of data). I am using a "wan-event" user script to detect when WAN ports get connected and disconnected and to send me an email notification. And then I'm also calling another script that checks to see if both WAN 0 and WAN 1 are up at the same time, and if "yes" I reboot the router to get back to a state where my Secondary WAN is in "Cold Standby". It works. But it would be better to fix the failover/failback feature. I have sent this information to ASUS but I'm kind of skeptical they'll ever get back to me. Maybe someone here knows what's the problem and how to fix it? So, here's the long story (as sent to ASUS support).

(PART 1. PART 2 CONTINUES IN REPLY)

Topology / Configuration

• ASUS RT-AC68U is configured in Dual WAN mode
• WAN port is connected to ASKEY HGU RTF3505VW Fiber Router. Router is configured in “Bridge” Mode, so ASUS Router gets the public IPV4 address
• LAN port 2 is designated for secondary WAN and is connected to a TP-LINK TL-MR100 4G/LTE Router.

After first booting up router, the Secondary WAN is in “Cold Standby”. This is Good!

Primary WAN (ppp0) is “UP” and has a public IP address

ppp0      Link encapoint-to-Point Protocol
          inet addr:88.1.XXX.XXX  P-t-P:192.168.144.1  Mask:255.255.255.255
          UP POINTOPOINT RUNNING MULTICAST  MTU:1492  Metric:1
          RX packets:9347 errors:0 dropped:0 overruns:0 frame:0
          TX packets:10132 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:3
          RX bytes:4843312 (4.6 MiB)  TX bytes:2841385 (2.7 MiB)

Secondary WAN (vlan3) is “DOWN” and does not have an IP Address

vlan3     Link encap:Ethernet  HWaddr 385:47:1F:5A:12
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:4756 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:1236450 (1.1 MiB)  TX bytes:0 (0.0 B)

10: vlan3@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
    link/ether 38:d5:47:1f:5a:12 brd ff:ff:ff:ff:ff:ff

Routing Table Shows Only Primary WAN (ppp0) Listed:

asusadmin@RT-AC68U-5B30:/tmp/home/root# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
8.8.4.4         192.168.144.1   255.255.255.255 UGH   1      0        0 ppp0
192.168.144.1   0.0.0.0         255.255.255.255 UH    0      0        0 ppp0
8.8.8.8         192.168.144.1   255.255.255.255 UGH   1      0        0 ppp0
10.16.0.0       192.168.15.1    255.255.255.0   UG    1      0        0 br0
192.168.101.0   0.0.0.0         255.255.255.0   U     0      0        0 br1
192.168.102.0   0.0.0.0         255.255.255.0   U     0      0        0 br2
192.168.15.0    0.0.0.0         255.255.255.0   U     0      0        0 br0
169.254.0.0     0.0.0.0         255.255.0.0     U     0      0        0 vlan2
127.0.0.0       0.0.0.0         255.0.0.0       U     0      0        0 lo
0.0.0.0         192.168.144.1   0.0.0.0         UG    0      0        0 ppp0

I simulate a failure of the primary WAN by pulling the fiber cable out of the ASKEY ROUTER. I don't touch the Ethernet connection between ASKEY and ASUS routers.
After Failover, the Secondary WAN shows "Connected" and the Primary WAN is in “Cold Standby”. Good! I have Internet access on my LAN going through the 4G/LTE router.

Indeed, the Primary WAN (ppp0) is DOWN

asusadmin@RT-AC68U-5B30:/tmp/home/root# ifconfig ppp0
ifconfig: ppp0: error fetching interface information: Device not found

The Secondary WAN (vlan3) is UP and has a “LAN ADDRESS” issued from the TP-LINK 4G/LTE ROUTER. (by the way, I don't care about double NAT here).

vlan3     Link encap:Ethernet  HWaddr 385:47:1F:5A:12
          inet addr:192.168.5.100  Bcast:192.168.5.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:16251 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1578 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:4436327 (4.2 MiB)  TX bytes:443561 (433.1 KiB)

Routing Table Shows VLAN3 in use and does not show any entries for ppp0

asusadmin@RT-AC68U-5B30:/jffs/downloads# route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
192.168.5.1     *               255.255.255.255 UH    0      0        0 vlan3
8.8.4.4         192.168.5.1     255.255.255.255 UGH   1      0        0 vlan3
8.8.8.8         192.168.5.1     255.255.255.255 UGH   1      0        0 vlan3
10.16.0.0       192.168.15.1    255.255.255.0   UG    1      0        0 br0
192.168.101.0   *               255.255.255.0   U     0      0        0 br1
192.168.5.0     *               255.255.255.0   U     0      0        0 vlan3
192.168.102.0   *               255.255.255.0   U     0      0        0 br2
192.168.15.0    *               255.255.255.0   U     0      0        0 br0
169.254.0.0     *               255.255.0.0     U     0      0        0 vlan2
127.0.0.0       *               255.0.0.0       U     0      0        0 lo
default         192.168.5.1     0.0.0.0         UG    0      0        0 vlan3

At this point, my VPN Appliance is sending data out to the Internet via the vlan3 interface on the ASUS router. Still all good.
Now I plug the Fiber cable back into my ASKEY ROUTER

 

[AndrewL733]

PART 2

NOW I PLUG FIBER CABLE BACK INTO ASKEY ROUTER

After Failback, Primary WAN connected and Secondary WAN in “Hot Standby”. BAD!

Primary WAN (ppp0) is UP and has a public IP address again:

ppp0     Link encapoint-to-Point Protocol
          inet addr:88.1.192.55  P-t-P:192.168.144.1  Mask:255.255.255.255
          UP POINTOPOINT RUNNING MULTICAST  MTU:1492  Metric:1
          RX packets:9728 errors:0 dropped:0 overruns:0 frame:0
          TX packets:12919 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:3
          RX bytes:3111396 (2.9 MiB)  TX bytes:11329948 (10.8 MiB)

Secondary WAN (vlan3) is ALSO UP and has a TP-LINK LAN IP address again:

vlan3     Link encap:Ethernet  HWaddr 385:47:1F:5B:31
          inet addr:192.168.5.100  Bcast:192.168.5.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:67892 errors:0 dropped:0 overruns:0 frame:0
          TX packets:57967 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:44386470 (42.3 MiB)  TX bytes:14823597 (14.1 MiB)

Routing Table shows both Primary and Secondary WANs are accessible. There are entries for both networks.

asusadmin@RT-AC68U-5B30:/tmp/home/root# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
192.168.5.1     0.0.0.0         255.255.255.255 UH    0      0        0 vlan3
8.8.4.4         192.168.144.1   255.255.255.255 UGH   1      0        0 ppp0
8.8.4.4         192.168.5.1     255.255.255.255 UGH   1      0        0 vlan3
192.168.144.1   0.0.0.0         255.255.255.255 UH    0      0        0 ppp0
8.8.8.8         192.168.144.1   255.255.255.255 UGH   1      0        0 ppp0
8.8.8.8         192.168.5.1     255.255.255.255 UGH   1      0        0 vlan3
10.16.0.0       192.168.15.1    255.255.255.0   UG    1      0        0 br0
192.168.101.0   0.0.0.0         255.255.255.0   U     0      0        0 br1
192.168.5.0     0.0.0.0         255.255.255.0   U     0      0        0 vlan3
192.168.102.0   0.0.0.0         255.255.255.0   U     0      0        0 br2
192.168.15.0    0.0.0.0         255.255.255.0   U     0      0        0 br0
169.254.0.0     0.0.0.0         255.255.0.0     U     0      0        0 vlan2
127.0.0.0       0.0.0.0         255.0.0.0       U     0      0        0 lo
0.0.0.0         192.168.144.1   0.0.0.0         UG    0      0        0 ppp0

At this point, my VPN Appliance should be using the Primary WAN but in fact it is still sending and receiving data through the Secondary WAN, using up all my mobile data.

To avoid this problem, I believe the ASUS Router should have put the Secondary WAN back into "Cold Standby". I tried to simulate this by deleting the vlan3 references from the routing table but I'm not convinced that was a good idea, and I'm worried that doing this could make a future failover unreliable. Perhaps another approach would be to simply bring the vlan3 interface down with "ifconfig vlan3 down"?

In any event, as I wrote at the top, I just worked around the problem by creating a "wan-event" script, detecting whether both interfaces were "up", and if "yes" rebooting the router. When the router comes back up it has the Secondary WAN in "Cold Standby" and all is good again.

But really, in my opinion, ASUS should fix this. Most ASUS users aren't going to move over to asuswrt-merlin, and even if they do most won't write user scripts. There is no reason to have the Secondary LAN in "Hot Standby" in a Failover / Failback configuration.

Perhaps the asuswrt-merlin community can contribute a fix to ASUS if ASUS doesn't fix it themselves?

 

[Kingp1n]

Unfortunately, Dual-WaN for stock firmware does not work properly. You can provide feedback to them thru the stock GUI firmware. Yes, please see here for a fix:

Thread 'Dual WAN Failover ***v2 Release***' https://www.snbforums.com/threads/dual-wan-failover-v2-release.83674/

 

[AndrewL733]

Hello @Kingp1n,

Can you tell me what exactly isn’t working with the ASUS version of Failover. I’m finding it working perfectly except for this one issue where after Failback I have the Seconday WAN in “Hot Standby” instead of “Cold Standby”. Is this something the script you linked to fixes?

Andrew

 

[Kingp1n]

Hello @Kingp1n,

Can you tell me what exactly isn’t working with the ASUS version of Failover. I’m finding it working perfectly except for this one issue where after Failback I have the Seconday WAN in “Hot Standby” instead of “Cold Standby”. Is this something the script you linked to fixes?

Andrew

They're quite of few issues with the Asus version scattered thru these forums. The script addresses all these issues and makes Dual-WaN work as intended.

 

[tournakos]

Hello @Kingp1n,

Can you tell me what exactly isn’t working with the ASUS version of Failover. I’m finding it working perfectly except for this one issue where after Failback I have the Seconday WAN in “Hot Standby” instead of “Cold Standby”. Is this something the script you linked to fixes?

Andrew

I had the same issue and I found that enabling both an ip ping and dns checks in auto network detection resolved it and now it leaves my secondary wan as cold standby when reverting back to primary. Unfortunately it is still not 100% reliable as there is a small(er) chance that it will not revert back to primary when back online and simply leave it as hot standby.

 

[Kingp1n]

I had the same issue and I found that enabling both an ip ping and dns checks in auto network detection resolved it and now it leaves my secondary wan as cold standby when reverting back to primary. Unfortunately it is still not 100% reliable as there is a small(er) chance that it will not revert back to primary when back online and simply leave it as hot standby.

Try the Dual-WAN-failover 3rd-party script:

WANFailover - WAN Failover v2.1.3 Release

WAN Failover is designed to replace the factory ASUS WAN Failover functionality, this script will monitor the WAN Interfaces using a Target IP Address and pinging these targets to determine when a failure occurs. When a failure is detected in Failover Mode, the script will switch to the...

www.snbforums.com

 

[tournakos]

Try the Dual-WAN-failover party script:

WANFailover - WAN Failover v2.1.3 Release

WAN Failover is designed to replace the factory ASUS WAN Failover functionality, this script will monitor the WAN Interfaces using a Target IP Address and pinging these targets to determine when a failure occurs. When a failure is detected in Failover Mode, the script will switch to the...

www.snbforums.com

Yes I know this has been around for a while but I am all for stock firmware and simple solutions.

 

[Kingp1n]

Yes I know this has been around for a while but I am all for stock firmware and simple solutions.

Good luck with stock!

 

[AndrewL733]

Yes I know this has been around for a while but I am all for stock firmware and simple solutions.

Totally agree. Also, the WAN Failover script is 6792 lines of code, which is just totally scary for me.

I am using 3.0.0.6 stock firmware on both AX and BE routers and the WAN failover basically works okay. It has been 100 % reliable to failover. It's a bit of a lottery about failing back automatically. I often find the Primary WAN comes back but goes into Hot Standby, and that it's necessary to go into the WAN configuration page and click on the Apply button and then it fails back properly. (I wish I knew a command line equivalent so that I could do that in a script).

I figured out how to get Entware to run automatically on the stock firmware router (In a cron job from another Linux server in my house, I run a script over SSH on the router every 5 minutes and check if the Entware partition is mounted on the router, and if not run another script to mount it) and created a very simple cron job on the router that notifies me when the Router fails over and then I manually connect to it and force it to fail back to the Primary WAN when it comes back (if it doesn't do that automatically).

 

[Kingp1n]

(I wish I knew a command line equivalent so that I could do that in a script).

I figured out how to get Entware to run automatically on the stock firmware router (In a cron job from another Linux server in my house, I run a script over SSH on the router every 5 minutes and check if the Entware partition is mounted on the router, and if not run another script to mount it) and created a very simple cron job on the router that notifies me when the Router fails over and then I manually connect to it and force it to fail back to the Primary WAN when it comes back (if it doesn't do that automatically).

If you're dng all this extra work with stock ... Might as well run Asuswrt-Merlin firmware

There's a script that already works! Best of luck though!

 

[AndrewL733]

If you're dng all this extra work with stock ... Might as well run Asuswrt-Merlin firmware

There's a script that already works! Best of luck though!

Unfortunately, at least for the RT-AX86U Pro router, Merlin is still on a 3.0.0.4 base, and as far as I can tell, ASUS only fixed the WAN failover issues in 3.0.0.6.

 

[tournakos]

Totally agree. Also, the WAN Failover script is 6792 lines of code, which is just totally scary for me.

I am using 3.0.0.6 stock firmware on both AX and BE routers and the WAN failover basically works okay. It has been 100 % reliable to failover. It's a bit of a lottery about failing back automatically. I often find the Primary WAN comes back but goes into Hot Standby, and that it's necessary to go into the WAN configuration page and click on the Apply button and then it fails back properly. (I wish I knew a command line equivalent so that I could do that in a script).

I figured out how to get Entware to run automatically on the stock firmware router (In a cron job from another Linux server in my house, I run a script over SSH on the router every 5 minutes and check if the Entware partition is mounted on the router, and if not run another script to mount it) and created a very simple cron job on the router that notifies me when the Router fails over and then I manually connect to it and force it to fail back to the Primary WAN when it comes back (if it doesn't do that automatically).

Yes thats exactly right, if it fails to revert back to primary the apply button there does the trick. What I havent tried is whether rebooting the router would also make it switch back to primary, have you tried that? Because in that case, you could have daily scheduled reboots and thus ensure that eventually, it will switch back to primary.. not ideal but it would cover a situation where a failover has happened but the failback has not and you are not around or have remote access to the router.