ASUS ZenWifi XT8 and XD5 issue with 2.4GHz clients

[peterdze]

Hello everyone,

I have MikroTik router (with 5G modem) and ASUS ZenWifi mesh network in AP mode with two XT8 units. Main unit is connected to MikroTik via ethernet cable. The second unit is connected to the main one via 5 GHz-2 backhaul. Recently I bought two additional ASUS XD5 routers and added them as two more mesh nodes. Soon after that started to notice frequent connectivity drops for some devices in the home network, such as Shelly smart switch, Xiaomi desktop lamp. If I ping one of these devices, I'm getting 10 to 80% of lost packets. Normally responses come in 5-40ms, but from time to time there are no responses at all. Sometimes those gaps appear every 20 seconds or so and then device becomes reachable again. I tried to do hard factory reset for all ASUS routers (using WPS button), turn off smart connect and roaming assistant, even downgraded XT8 from 3.0.0.4.388_24609 to 3.0.0.4.388_23285. Finally, renamed SSID so that no wireless clients are connected and reconfigured only one device, Shelly smart switch, to connect to new SSID. Even after all these steps connectivity to the only wireless client, Shelly, would be very unstable.

Then I SSH'ed to ASUS access points and ran the following command to monitor bridge MAC address: watch -n 1 "brctl showmacs br0 | grep '<SHELLY_MAC_ADDRESS>'". Turns out Shelly client was jumping between XD5 2.4GHz interface port and XT8 5GHz interface port. This is very odd, since Shelly doesn't have 5GHz radio at all. Almost like some sort of network loop was present, or routing got periodically corrupted.
Connection was: Shelly --(2.4GHz)--> XD5 --(5GHz backhaul)--> XT8(node2) --(5GHz-2 backhaul)--> XT8(main node) --(ethernet)--> MikroTik --> 5G internet

When Shelly was reachable MAC tables on both XD5 and XT8 (node2) were correct: XD5 showed Shelly's MAC address mapped to the port corresponding to 2.4GHz interface (wl0.1) and XT8 showed it mapped to backhaul connection to XD5 (wds1.1.1).

When Shelly was not reachable, it miraculously appeared as if being connected to XT8 (node2) 5GHz (which is impossible): XD5 showed it in the backhaul connection to XT8 (wl1) and XT8 showed it in 5GHz port (wl1.1). Sometimes the jumping occurred only in XT8 MAC table but not in XD5. In these cases Shelly was still pingable if I ran ping while connected to the same XD5, but not from XT8. I also tried swapping units so that Shelly connects to XD5 which connects to another XD5 and only then to XT8. This time around jumping was between XD5 and the first XT8 on the route. Noticed similar jumping in other 2.4GHz clients connected to XD5, such as Xiaomi desktop lamp.

I ran out of options to try and wanted to ask for a help to fix the issue. Is this a known issue? Is XD5 compatible with XT8 mesh network at all? Any help would be greatly appreciated.

Thank you.

 

[peterdze]

Just to update, I have installed GNUton's Asus Merlin on both XT8 units and the issue is still there. Starting to suspect that there is a race condition between wds1.1.1 and wl1.1 interfaces of XT8 connected to XD5 unit (both are for 5GHz-1, the first is a backhaul to XD5 and the second is for regular 5GHz clients). Since XD5 doesn't support a separate 5GHz band for backhaul connection, it is using the same 5GHz-1 as regular clients. Probably XT8 "sees" packets coming from Shelly via XD5 on both interfaces and randomly updates MAC table based on where a packet was noticed first. Another suspicious log entry in XT8 is: ZenWiFi_XT8-5740-2E9C0DC-R kernel: br0: received packet on wl1.1 with own address as source address. Might be an indication of the same issue, but not sure yet.

Going to try installing tcpdump on XT8 to check. In the current situation "mix and match" of XT8 (v1) and XD5 doesn't seem very usable.

 

[peterdze]

Finally identified the problem with XT8/XD5 combo and Shelly smart plug. Shelly is periodically sending multicast UDP packets to 224.0.1.187 port 5683. This is not handled well by ASUS routers as it appears on XT8 5GHz interface and updates MAC table for Shelly in XT8. The subsequent ping requests are sent to 5GHz interface on XT8 and obviously are left without response until MAC tables are updated again. Found some pages describing it as Shelly's CoAP protocol communication. Sadly I wasn't able to fix it by changing Wireless settings in Professional tab (such as Enable IGMP Snooping), so I resorted to bringing an old Zyxel router and using it in repeater mode specifically for Shelly. The ping is more stable now. Hopefully at some point ASUS will fix the firmware, but until then sticking with a separate router.

Here is an example of pinging from my PC (on the left side) and tcpdump output on XT8. I highlighted the lines when Shelly becomes unreachable after sending UDP packets.

 

[ArtyMarty]

That is interesting.
I had a similar problem when I put an Asus RP-AX56 in the mix with my two XT8 units.
I didn't do as advanced checking as you did sniffing packets, but all sorts of issues with various smart home devices showing randomly on routers or bands physically impossible.

I did have all the XT8 and RP-AX56 on wired backhall mode.

Remove RP-AX56, problems go away, reconnect, they are back.
I ended up replacing the RP-AX56 with 2 more XT8 units, didn't have that issue.

HOWEVER, now I need a wifi point at the end of my driveway to extend coverage for some smart home devices.
I am installing a MoCA adapter for the backhall and was going to put a Asus XD5 there.

So I'm guessing there is no way to extend the mesh without putting another XT8 there? (not really practical due to size, heat issues in a weather proof box)

What do you recommend?