AX-92U mesh: single devices unable to communicate at seemingly-random intervals

[Thw0rted]

I've been running a 3xAX-92U mesh for a couple of months now -- all on (official) firmware 3.0.0.4.386_40451-g30f1b6c. The configuration is mostly stock, in Dual Band Smart Connect mode, channels manually specified on both bands, 1@20MHz and 36@20/40/80MHz. One node is on wired backhaul, the other is on the 5-2 band. Connections are solid for all my devices, *almost* all the time.

Every once in a while, one of my devices simply can't get on the network -- no configuration changes, either on the device or the router(s). The weird part is, the device thinks that the connection is successful -- "Connected, No Internet" in Windows -- but no traffic goes through. I haven't busted out wireshark yet, but I can't pull a DHCP address, and if I assign a static IP, I can't ping any other devices on the network. The other weird part is that I've had very similar symptoms on 3 Windows laptops and two Rokus, but never all at once, just one or two at a time, and sometimes they all work fine.

I don't think there's anything relevant in the system logs. I posted here a while back when the problem started, after which it sort of cleared up on its own, but in the past week or so it's been happening more often. I'd really like to figure out how to keep this from happening in the future.

 

[Thw0rted]

There's a new firmware version as of today. I don't know if it's more likely to help or to make things worse. If I don't get any suggestions here soon, I might give it a shot, since it's been getting really bad lately, knocking devices offline at random.

 

[screamjojo]

Hi,
I've the same problem with the same router.
Anybody can help us ?
Thanks

 

[Thw0rted]

I updated the firmware to the new release last night. I don't want to jinx it but so far, no more random dropouts. I used the upgrade function on the web UI firmware page and guess it did all 3 devices in the right order. I haven't tried a reset yet (so, "dirty" upgrade) but will try that if things get bad again.

ETA: I also disabled universal beam forming since my last post, so that might have helped.

 

[screamjojo]

@Thw0rted

Can you please give me your settings ? I've always the same log error kernel and disconnected client (same on your post http://www.snbforums.com/threads/rt-ax92u-issue.66110/post-633795) :

Dec 31 18:57:11 kernel: wl0: random key value: E71CA6A70CB28EFB313901A254B2AB4517C54046BE75089052D35EEBD2E0B907
Dec 31 18:57:11 wlceventd: wlceventd_proc_event(507): eth5: Disassoc 98:E8:FA:06:XXX, status: 0, reason: Disassociated because sending station is leaving (or has left) BSS (8), rssi:0
Dec 31 18:57:11 wlceventd: wlceventd_proc_event(507): eth5: Disassoc 98:E8:FA:XXX, status: 0, reason: Disassociated because sending station is leaving (or has left) BSS (8), rssi:0
Dec 31 18:57:17 wlceventd: wlceventd_proc_event(526): eth6: Auth 44:74:6C:XXX, status: Successful (0), rssi:0
Dec 31 18:57:17 wlceventd: wlceventd_proc_event(555): eth6: Assoc 44:74:XXX, status: Successful (0), rssi:0
Dec 31 18:57:45 kernel: wl0: random key value: 2EA6952D2985014604DF0270B8CCDBBFE85E3C35C3BA0DA2211629581CB37E71
Dec 31 18:57:45 wlceventd: wlceventd_proc_event(490): eth5: Deauth_ind FC:F8:AE:XX, status: 0, reason: Unspecified reason (1), rssi:0
Dec 31 18:58:35 kernel: wl0: random key value: 601517111E9CF1115FBE0156659648B2FFE947C260A20133C87485B28C7133B2
Dec 31 18:58:35 wlceventd: wlceventd_proc_event(490): eth5: Deauth_ind 34XX, status: 0, reason: Unspecified reason (1), rssi:0
Dec 31 18:58:42 wlceventd: wlceventd_proc_event(526): eth6: Auth 48:26:2XX, status: Successful (0), rssi:0
Dec 31 18:58:42 wlceventd: wlceventd_proc_event(555): eth6: Assoc 48:26:2XXX, status: Successful (0), rssi:0
Dec 31 18:58:43 kernel: wl1: random key value: 527B5F5463F305626ABB03EE437F43EEFEF911FD4C4506211758CF985DAD328F
Dec 31 18:58:43 wlceventd: wlceventd_proc_event(507): eth6: Disassoc 48:26:2C:6EXXX, status: 0, reason: Disassociated because sending station is leaving (or has left) BSS (8), rssi:0
Dec 31 18:58:43 wlceventd: wlceventd_proc_event(507): eth6: Disassoc 48:2XXXX, status: 0, reason: Disassociated because sending station is leaving (or has left) BSS (8), rssi:0
Dec 31 18:58:57 wlceventd: wlceventd_proc_event(526): eth6: Auth 34:13:E8:BXXX, status: Successful (0), rssi:0
Dec 31 18:58:57 wlceventd: wlceventd_proc_event(555): eth6: Assoc 34:13:E8:BXXX, status: Successful (0), rssi:0
Dec 31 19:00:51 disk_monitor: Got SIGALRM...
Dec 31 19:01:39 wlceventd: wlceventd_proc_event(526): eth5: Auth 48:26:2C:XX, status: Successful (0), rssi:0
Dec 31 19:01:39 wlceventd: wlceventd_proc_event(555): eth5: Assoc 48:26:2C:6E:1XXX, status: Successful (0), rssi:0
Dec 31 19:01:42 wlceventd: wlceventd_proc_event(526): eth6: Auth 48:XX, status: Successful (0), rssi:0
Dec 31 19:01:42 wlceventd: wlceventd_proc_event(536): eth6: ReAssoc 48:26:2XXXX, status: Successful (0), rssi:0
Dec 31 19:01:42 kernel: wl0: random key value: 225AC82B6A6A548F25B70082E5E6FA9BAC055C3E3286062494E6E39065903FAA
Dec 31 19:01:42 wlceventd: wlceventd_proc_event(507): eth5: Disassoc 48:26XXXX, status: 0, reason: Disassociated because sending station is leaving (or has left) BSS (8), rssi:0
Dec 31 19:01:42 wlceventd: wlceventd_proc_event(507): eth5: Disassoc 48:26:2C:6XXX, status: 0, reason: Disassociated because sending station is leaving (or has left) BSS (8), rssi:0
Dec 31 19:01:43 kernel: wl1: random key value: 1E8C8A2102ED80725F8702FE50A98124DFA9C3CBFA5F074ECE1F3A17BDF786DF
Dec 31 19:01:43 wlceventd: wlceventd_proc_event(507): eth6: Disassoc 48:26:XXX, status: 0, reason: Disassociated because sending station is leaving (or has left) BSS (8), rssi:0
Dec 31 19:01:43 wlceventd: wlceventd_proc_event(507): eth6: Disassoc 48:26XXXXX, status: 0, reason: Disassociated because sending station is leaving (or has left) BSS (8), rssi:0

Thanks in advance

 

[Axion42]

Yes, this same exact thing has been happening to me since latest firmware update. Even with the 3.0.0.4.386_41535 that was released yesterday I've had 2 drop outs on my phone today. It says I'm connected to WiFi but says no Internet. Think I'll just go back to Version 3.0.0.4.384.9177, I had zero problems for months now unless I can figure out whats happening on this new version.

 

[screamjojo]

Do you use QOS ? When I deactivate it looks like I have no more disassoc ...

 

[Axion42]

QoS is not enabled for me.

 

[screamjojo]

QoS is not enabled for me.

In the end no change for me ...
With last firmware 3.0.0.4.386_41535 and hard reset, problem always here

 

[L&LD]

Fully Reset Router and Network

Try the above...

 

[Thw0rted]

@L&LD, do I read your instructions correctly that you want people to do a WPS-button reset, then a web-UI "restore/initialize" reset, then re-flash the firmware manually, *twice*? ("repeat steps 11, 12, and 13") That's going to take hours, especially in a mesh configuration where you'd have to apply those steps to each node individually. It definitely shouldn't be necessary to re-flash the firmware -- because the distribution contains an integrity check, and the device should refuse to apply it if it's corrupted -- much less to re-flash it twice. Especially in a mesh configuration, people should be using the one-click automatic upgrade process. It's there for a reason. (By all means, do a restore/initialize afterwards, to start from a clean slate.)

The trouble with telling people to "completely wipe everything, then test as you go" is that oddball problems like these can take several days to show up. So if I spend half my Saturday doing a full mega-wipe, then wait "only" two days to change some settings, then devices start dropping off... was it the setting I changed, or is there a hardware problem? Or actually, is there a bug in this firmware that only shows up in region X, when you live within Y miles of an airport, that happens after *four* days but I can't do anything about?

At any rate, I *did* a full router/network reset when the previous firmware dropped, though not by choice -- I had to re-flash every device manually and reset it a number of times just to be able to join my nodes to the network. The thing is, everything was fine for weeks, then I saw this "wifi but no internet" problem on one or two devices, then it stopped for a while, then it came back worse, and in all that time I never changed a setting.

It would be much more helpful to give people the tools they need to diagnose the actual cause of the problem. What devices are getting knocked off? Do they have a chipset in common, are they all on the same band? SSH in and enable XYZ log level, then look for Z; other people see similar behavior when they (have implicit beamforming enabled) (use a DFS channel) (connect a noisy USB device) so try fixing those, etc etc.

 

[Thw0rted]

@screamjojo, I've seen a number of posts like this one that suggest that the pattern of "Disassociated because sending station is leaving", then "Auth", then "Assoc", is totally fine and normal, and not indicative of a problem. They're saying that Asus left some debug-level logging code enabled at higher log levels accidentally and haven't gotten around to fixing it. I noticed that the MACs in those statements do not correspond to the MACs of the devices that are being blocked.

For my settings, basic wireless is as described in the OP, AP mode. No USB devices, WPS and Mac Filter diabled on all bands, no device bindings or roaming block list. Professional looks like

and

I just noticed that I failed to disable roaming assistant and universal beamforming on 5GHz; if I start dropping devices again I'll change that, but I don't want to touch anything since it's all working at the moment. Oh, also, I haven't touched the default Smart Connect Rules.

 

[L&LD]

@Thw0rted, does it seem unreal to suggest those things? Yes.

Does it help some people determine if the hardware or the configuration was the issue? Yes.

It won't take hours to do. I know. I deliver routers to customers doing these steps (as needed).

I am not negating anything you state of how 'things should' work. These suggestions are for when that doesn't seem to apply to certain (specific) routers. Or, when the firmware updates, features used, and options/scripts enabled have brought the router to its knees.

I have taken many routers that a customer has called and asked for a new one and they are working great years later by doing this method.

If you upgrade (with a single click) and the network, works. No worries!

When it doesn't, it is easier to nuke away old assumptions and other gremlins and start at a good/known state.

Then, troubleshooting can begin. Before that? You're chasing your tail or those not-so-mythical wild geese.

Wishing you a Happy New Year!