AiMesh with 7 RT-AC68U - a single node is constantly crashing/rebooting

[muddymind]

Hi!

I have 7 RT-AC68U in a AiMesh config with 5 nodes connected via ethernet and 1 node connected via WiFi:

I had a bit of trouble setting these up. Initially I tried flashing merlin firmware v386.3_2 on every router (both main router and nodes) but it always failed at 1% when adding a new node. Unfortunately the logs didn't have any information on why it failed.
I flashed the Asus v3.0.0.4.386_43129 into the main router but it still failed at 1%. I flashed v3.0.0.4.386_43129 into every node router and I finally managed to add them all.
(every flash was followed by settings reset with the WPS button press on boot)

Everything ok so far! It is unfortunate I can't use merlin's FW since it has some useful features but for know it is ok for my immediate needs.

The main problem started when I started to notice that "-1 Pilates" Node was dropping from time to time (it's the only one with wifi backhaul).

I tried dirty flashing merlin's v386.3_2 but still crashed (rebooted) after a while.
I tried dirty flashing the old Asus v3.0.0.4.386.41634 but still showed the same behavior.

Since the router is mounted to the ceiling and is upside down I thought it might be an issue of temperature but I put a 120mm fan stucked to its back to keep it cool and it still is crashing.

The way I can quickly force the crash (reboot) is to fetch a huge file from a NAS and it starts downloading it at around 110Mbps and after 2-10min it drops the speed to 10Mbps.
If I reconnect the client it gets back up to 110Mbps but if I don't renew the client's connection, and it stays at 10Mbps, it will crash (reboot) the router after 1min.

On the main router I don't have any relevant logs about this happening.
I looked at the node's /tmp/syslog.log and it doesn't seem to have any relevant info on why it rebooted:

Sep 6 17:06:06 syslog: wlceventd_proc_event(536): wl0.1: ReAssoc 00:04:4B:2D:60:FF, status: Successful (0), rssi:0
Sep 6 17:08:23 syslog: wlceventd_proc_event(526): wl0.1: Auth 74:15:75:74:66:6D, status: Successful (0), rssi:0
Sep 6 17:08:23 syslog: wlceventd_proc_event(536): wl0.1: ReAssoc 74:15:75:74:66:6D, status: Successful (0), rssi:0
Sep 6 17:13:46 roamast: [EXAP]Deauth old sta in 0 1: 00:04:4B:2D:60:FF
Sep 6 17:13:46 roamast: wl0.1: disconnect weak signal strength station [00:04:4b:2d:60:ff]
Sep 6 17:13:46 roamast: wl0.1: remove client [00:04:4b:2d:60:ff] from monitor list
Sep 6 17:13:46 syslog: wlceventd_proc_event(490): wl0.1: Deauth_ind 00:04:4B:2D:60:FF, status: 0, reason: Deauthenticated because sending station is leaving (or has left) IBSS or ESS (3), rssi:0
Sep 6 17:16:40 roamast: [EXAP]Deauth old sta in 0 1: 74:15:75:74:66:6D
Sep 6 17:16:40 roamast: wl0.1: disconnect weak signal strength station [74:15:75:74:66:6d]
Sep 6 17:16:40 roamast: wl0.1: remove client [74:15:75:74:66:6d] from monitor list
Sep 6 17:16:41 syslog: wlceventd_proc_event(490): wl0.1: Deauth_ind 74:15:75:74:66:6D, status: 0, reason: Deauthenticated because sending station is leaving (or has left) IBSS or ESS (3), rssi:0
Sep 6 17:16:43 syslog: wlceventd_proc_event(526): wl0.1: Auth 74:15:75:74:66:6D, status: Successful (0), rssi:0
Sep 6 17:16:43 syslog: wlceventd_proc_event(536): wl0.1: ReAssoc 74:15:75:74:66:6D, status: Successful (0), rssi:0
Sep 6 17:16:59 roamast: [EXAP]Deauth old sta in 0 1: 74:15:75:74:66:6D
Sep 6 17:16:59 roamast: wl0.1: disconnect weak signal strength station [74:15:75:74:66:6d]
Sep 6 17:16:59 roamast: wl0.1: remove client [74:15:75:74:66:6d] from monitor list
Sep 6 17:17:00 syslog: wlceventd_proc_event(490): wl0.1: Deauth_ind 74:15:75:74:66:6D, status: 0, reason: Deauthenticated because sending station is leaving (or has left) IBSS or ESS (3), rssi:0
Sep 6 17:34:27 syslog: wlceventd_proc_event(526): wl1.1: Auth D6:2D:0B:9C:15:C3, status: Successful (0), rssi:0
Sep 6 17:34:27 syslog: wlceventd_proc_event(555): wl1.1: Assoc D6:2D:0B:9C:15:C3, status: Successful (0), rssi:0
Sep 6 17:34:36 roamast: [EXAP]Deauth old sta in 1 1: D6:2D:0B:9C:15:C3
Sep 6 17:34:36 roamast: wl1.1: disconnect weak signal strength station [d6:2d:0b:9c:15:c3]
Sep 6 17:34:36 roamast: wl1.1: remove client [d6:2d:0b:9c:15:c3] from monitor list
Sep 6 17:34:36 syslog: wlceventd_proc_event(490): wl1.1: Deauth_ind D6:2D:0B:9C:15:C3, status: 0, reason: Deauthenticated because sending station is leaving (or has left) IBSS or ESS (3), rssi:0
Sep 6 17:37:40 syslog: wlceventd_proc_event(526): wl0.1: Auth 74:15:75:74:66:6D, status: Successful (0), rssi:0
Sep 6 17:37:40 syslog: wlceventd_proc_event(536): wl0.1: ReAssoc 74:15:75:74:66:6D, status: Successful (0), rssi:0
Sep 6 17:39:04 roamast: [EXAP]Deauth old sta in 0 1: 74:15:75:74:66:6D
Sep 6 17:39:04 roamast: wl0.1: disconnect weak signal strength station [74:15:75:74:66:6d]
Sep 6 17:39:04 roamast: wl0.1: remove client [74:15:75:74:66:6d] from monitor list
Sep 6 17:39:05 syslog: wlceventd_proc_event(490): wl0.1: Deauth_ind 74:15:75:74:66:6D, status: 0, reason: Deauthenticated because sending station is leaving (or has left) IBSS or ESS (3), rssi:0
Sep 6 1May 5 06:05:02 kernel: klogd started: BusyBox v1.25.1 (2021-01-05 17:01:55 CST)
May 5 06:05:02 kernel: Linux version 2.6.36.4brcmarm (root@asus) (gcc version 4.5.3 (Buildroot 2012.02) ) #1 SMP PREEMPT Tue Jan 5 17:08:14 CST 2021
May 5 06:05:02 kernel: CPU: ARMv7 Processor [413fc090] revision 0 (ARMv7), cr=10c53c7f
May 5 06:05:02 kernel: CPU: VIPT nonaliasing data cache, VIPT nonaliasing instruction cache
May 5 06:05:02 kernel: Machine: Northstar Prototype
May 5 06:05:02 kernel: Ignoring unrecognised tag 0x00000000
May 5 06:05:02 kernel: Memory policy: ECC disabled, Data cache writealloc
May 5 06:05:02 kernel: Built 1 zonelists in Zone order, mobility grouping on. Total pages: 60416
May 5 06:05:02 kernel: Kernel command line: root=/dev/mtdblock2 console=ttyS0,115200 init=/sbin/preinit earlyprintk debug
May 5 06:05:02 kernel: Memory: 255496k/255496k available, 6648k reserved, 0K highmem

Any ideas why this keeps happening?

 

[L&LD]

Try different Control Channels to fix the issue you're having.

But, being curious, how many SqFt is the area you're covering with these routers?\

I believe you have too much Wi-Fi unless you're providing wireless internet access to a small/medium apartment building.

 

[muddymind]

I found a solution to my issue. I have a bunch of different revisions of the RT-AC68U from the 1st ones available to the latest V3 revision.

I had a revision A1 in the trouble spot as the wireless backhaul node. I switched it with a newer V3 from another node and now everything works as intended

This proves there's some issue with the FW in the initial versions of this model.

Try different Control Channels to fix the issue you're having.

Can I do that with AiMesh? I think it forces every node to the same control channel unless I switch it to AP mode. Unfortunately I really really need a guest network so my only option for now is to use them in AiMesh mode.

But, being curious, how many SqFt is the area you're covering with these routers?

I believe you have too much Wi-Fi unless you're providing wireless internet access to a small/medium apartment building.

I don't know how many sq meters are but it's a lot! It's a very large 3 floors villa and also covers a bit of outdoor area.

I have 3 nodes per floor except for 1 that only has the single wireless backhaul.

The floor and walls are made in such a way that very little 5GHz signal can pass trough them and each floor has many small divisions so I require a lot of devices to provide proper coverage. Even using this many nodes I still have a single dark spot in the house which I'll cover with an older rt-n66U in AP mode just to provide wireless connectivity to a single tv-box and a smartphone.

 

[L&LD]

Different Control Channels (period). Not different Control Channels per AiMesh node.

 

[muddymind]

Different Control Channels (period). Not different Control Channels per AiMesh node.

Ah! Lol

This is in the country side with no other houses nearby that could generate noise so I doubt it would make too much of a difference (unless there's some other device generating interference in the 5GHz channel.... Unlikely but always possible).

 

[L&LD]

Whether it's non-Wi-Fi related interference or not, I find that routers have their own 'best' Control Channel that they work optimally on.

Glad you found a workable solution!

 

[RangerZ]

I tried flashing merlin firmware v386.3_2 on every router (both main router and nodes) but it always failed at 1% when adding a new node.

There are some issues with the RT-AC68U being able to form an AIMesh config as you describe. See this post. This began with 386.2_6.

LSS, it appears that you should be able to configure AIMesh by connecting the Routers LAN to the Nodes WAN via Ethernet and then Adding a node. This is NOT the process suggested by ASUS. There also is a post by Merlin in which he basically suggests there is no benefit to running Merlin on the nodes. Unclear if this will work as I suggest in an environment as complex as yours.