Instabilities and hangups with AIMesh

[MarkusI]

Hi,

I know that AC86U and AC68U are now EOL, but I wanted to report the issue anyway, just in case it might also affect other devices.

Friends of mine have an AIMesh setup, which I maintain.

The setup is:
AC86U (main)
3 AC86U nodes (2 of them connected via cable to each other, connection to main is Wifi only)
2 AC68U nodes (usually) connected via Wifi to one of the AC86U nodes

For the last 1-2 years, I had the latest Merlin-Firmware installed on all devices and did regular updates, when available.

Disclaimer: I am not sure whether the issues I faced were due to the mesh setup, but I suppose if not, people would have reported issues in masses.

As far as I remember, the AC68Us always ran stably.
The AC86Us on the other hand were very unstable in that setup.

The (lan-connected) main router's UI regularly stopped working (endless loading until timeout), sometimes even minutes to hours after a reboot.
It was still possible to SSH into it and remotely restart the router, so I initially assumed that the UI issue was due to some issue with my VPN until I got confirmation that the UI did not work within the LAN either.

The AC86U nodes regularly dropped out of the mesh and stayed unconnected, so that manually switching off and power-on was needed to revive them.
Obviously, due to them being disconnected to the mesh, SSHing and remotely rebooting was not possible.

As a result, the whole mesh more or less broke down completely every few days (at least once a week, but usually twice a week).
The issue was not that critical until two months ago, when we extended the mesh with the AC68Us, of which one is used to connect some IP cameras for observing the sheep's barn. Right now is lambing season and my friends need to check multiple times per night if everything is ok... which is not possible, if the AC86U to which the AC68U is connected went down...

I tried to work around this issue by scheduling a reboot every night, but it did not solve the issue.
I observed that the scheduled reboot did not execute reliably, supposedly (not sure here, but assuming) in cases, where the main router was somehow in a messed-up state.
If a scheduled reboot did not work, it did not on all routers - I assume rebooting the nodes might be triggered by the main router.
It also looked like rebooting did not just skip one day and continued the next... when rebooting stopped working, a cold-start of the device(s) was needed to make it functional again.

Long story short, that setup with AsusWRT Merlin ran absolutely unstable for months and months, but I did not relate that to AsusWRT Merlin in the first place.

Last week, I reverted all routers to the latest stock firmware and also disabled the scheduled reboot to increase the chances of failure but until now, everything runs rock-solid.

It is fair to say that it might need some more days to be really certain, but usually, the mesh network would not have survived 3-5 days without complete failure.

I wanted to report this here, since my observations might help with stabilizing the software, even if only on newer devices.

Best
Markus

 

[uwish]

I never could get Merlin working with AIMESH. I know people have and do run it, but for some reason it tends to degrade into something unreliable. No idea, I ran Merlin for years then tried mesh, never got it working. Even after I tried stock on one, merlin in various types of router swapping and blending but never got it to work.

 

[OzarkEdge]

Holy moly!

OE

 

[bennor]

Last week, I reverted all routers to the latest stock firmware ....

If you read or search around for the AiMesh discussions you'll see that many, if not most, people here recommend using stock Asus firmware on the AiMesh nodes. As always, when having weird or unstable issues with Asus routers, go back to basics and perform a hard factory reset on the router and nodes. Then manually configure the main router without importing a saved configuration file, without adding any USB hard drives, and without adding any Asus-Merlin addon screipts. Then try attaching the AiMesh nodes one at a time, testing each node after it's been successfully linked to the main router. Do not add any add-on scripts until everything is verified as working.

 

[Tech9]

The setup is:
AC86U (main)
3 AC86U nodes (2 of them connected via cable to each other, connection to main is Wifi only)
2 AC68U nodes (usually) connected via Wifi to one of the AC86U nodes

I would be surprised if this thing worked well...

 

[MarkusI]

If you read or search around for the AiMesh discussions you'll see that many, if not most, people here recommend using stock Asus firmware on the AiMesh nodes. As always, when having weird or unstable issues with Asus routers, go back to basics and perform a hard factory reset on the router and nodes. Then manually configure the main router without importing a saved configuration file, without adding any USB hard drives, and without adding any Asus-Merlin addon screipts. Then try attaching the AiMesh nodes one at a time, testing each node after it's been successfully linked to the main router. Do not add any add-on scripts until everything is verified as working.

I posted some AiMesh questions some time back and did not get too many answers, which is why - I have to admit - I thought that not many people would use AiMesh at all or with Merlin...

And sure, I did exactly that - all nodes were in factory settings and I did the same with the main router.
No scripts, etc., only very basic settings (Wifi, LAN, DNS, etc., no ipv6) + OpenVPN.
Merlin just gave me some more options setting the thing up in a convenient way.

I assumed it would be a bad idea to have mixed-firmware AiMesh setups, but as you described, it seems that this would have been the way to go.
Still, the main router was unstable, too.
I don't know if the main router also suffered from non-stock-AIMesh-nodes, which would be weird, imho.

I used and loved Merlin since ages in my own (non-AiMesh) settings without any issue, which is why I defaulted for that firmware in the AiMesh setup, too.

But looking back, this might not have been a good idea.
Even adding nodes to the AiMesh only worked stably if I temporarily connected them via cable to the main router.

Anyway, now that I reverted everything back to stock, everything runs smoothly.
Also, I was able to setup the OpenVPN client by manually modifying the .ovpn-file before importing to meet my needs instead of using the Merlin-UI, which provides more options than stock.

Thus, I will stick with stock in that setting now, since I was able to work around the missing options which Merlin provides but stock don't.

Cheers!

 

[MarkusI]

I would be surprised if this thing worked well...

You mean with Merlin or in general?
With stock, this currently looks to run very smoothly.
Sure, with every hop, we use quite some bandwidth, but we don't need high throughput in the remote locations of the farm.
We just need network / internet, e.g. to run the cashier, check the security camera, connect the google nest bell or stream music in the workshop...

Even in the remote locations, we get at least 15-20MBits, depending on the Wifi quality between the nodes.
In one location we even use a directional radio antenna connected to one of the connectors of an AC68U to bridge a longer distance to the next node.
In that scenario (AC68U with directional antenna -> AC86U node connected via CAT6 to -> AC86U connected via Wifi to -> main node) we still get 50-80 MBits internet throughput...

That is plenty for our purposes - as long as the routers run and connect stably.
At least since the weekend, this seems to be the case.

We now plan to connect one or two of the AC86U nodes which are located in a separate building with CAT6 directly to the main router to increase bandwidth.
Old farm buildings here in good old Germany have thick brick walls

 

[naitkris]

For AiMesh, try keeping the ASUS Router on latest Asuswrt-Merlin and the ASUS Node(s) on the latest official stock firmware. I posted about the same at https://www.snbforums.com/threads/a...e-version-3-0-0-4-388_24621.89343/post-927315 and since doing this everything has been very stable the past half year or so.