Need help diagnosing AC86U spontaneous reboot

[captaink]

My AC86U reboots itself every 3 weeks or so. I am trying to figure out why. The symptoms are:
--public internet stops being accessible,
--LAN is accessible,
--memory is about 95% (returns to 70% after reboot),
--swap is 10MB of 255MB (0 of 255MB after reboot),
--router reboot was triggered this morning when I rebooted my modem instead of router.

The AC86U is running 384.19. I have 2 AC68s running as mesh nodes. Those nodes are currently running 386.1_beta1. I realize the nodes should be running identical firmware as the AC86U. I attempted to fix this problem with beta software and had to back out on the AC86U due to issues. This problem pre-dates the nodes running different firmware from main router.

I am remote logging and cannot find anything obvious in the logs that would suggest the cause for this. It looks like a memory leak to me.

 

[bbunge]

My AC86U reboots itself every 3 weeks or so. I am trying to figure out why. The symptoms are:
--public internet stops being accessible,
--LAN is accessible,
--memory is about 95% (returns to 70% after reboot),
--swap is 10MB of 255MB (0 of 255MB after reboot),
--router reboot was triggered this morning when I rebooted my modem instead of router.

The AC86U is running 384.19. I have 2 AC68s running as mesh nodes. Those nodes are currently running 386.1_beta1. I realize the nodes should be running identical firmware as the AC86U. I attempted to fix this problem with beta software and had to back out on the AC86U due to issues. This problem pre-dates the nodes running different firmware from main router.

I am remote logging and cannot find anything obvious in the logs that would suggest the cause for this. It looks like a memory leak to me.

When I ran Diversion the RAM usage went up quickly and caused concern. I dumped Diversion and set up Pi-Hole. FlexQOS is the only add on I use now. I also use a swap partition and not a swap file.

 

[Jeffrey Young]

Anything in the logs that stand out or give a hint?

 

[mikeyd810]

Interesting, I have an ac86u that has been doing this recently as well. Hung up twice in 5 days, had to reboot it both times to get the network stable again. Mine is acting as an AImesh node with an ax86u as the main router.

Didn't notice the hangs until I stated testing out the 386 beta releases, watching to see if it occurs now that i'm running beta5

Wondering what logs would be key to check and their locations? I'm familiar with Linux/Unix but not sure where the relevant logs are stored on the routers.

 

[Wisiwyg]

My AC86U reboots itself every 3 weeks or so. I am trying to figure out why. The symptoms are:
--public internet stops being accessible,
--LAN is accessible,
--memory is about 95% (returns to 70% after reboot),
--swap is 10MB of 255MB (0 of 255MB after reboot),
--router reboot was triggered this morning when I rebooted my modem instead of router.

The AC86U is running 384.19. I have 2 AC68s running as mesh nodes. Those nodes are currently running 386.1_beta1. I realize the nodes should be running identical firmware as the AC86U. I attempted to fix this problem with beta software and had to back out on the AC86U due to issues. This problem pre-dates the nodes running different firmware from main router.

I am remote logging and cannot find anything obvious in the logs that would suggest the cause for this. It looks like a memory leak to me.

Recommend you update to the B5 release, out just yesterday. Seems to be running better across all devices, including AC86U. That update has latest fixes for Mesh code from Official code.

 

[Jeffrey Young]

Interesting, I have an ac86u that has been doing this recently as well. Hung up twice in 5 days, had to reboot it both times to get the network stable again. Mine is acting as an AImesh node with an ax86u as the main router.

Didn't notice the hangs until I stated testing out the 386 beta releases, watching to see if it occurs now that i'm running beta5

Wondering what logs would be key to check and their locations? I'm familiar with Linux/Unix but not sure where the relevant logs are stored on the routers.

From the main page on the GUI, there should be a "system log" tab down the left hand pane. Look through the log up to the moment that the router rebooted and see if there is any abnormal messages. You should see a sudden date/time into the past which shows when the router come back to life.

 

[captaink]

Just an update on my issue. I flashed to 386.1 on both mesh nodes and main router and will have to monitor for a few weeks to determine whether the "memory leak" issue is resolved. Interestingly, the mesh node flashes went smoothly however the main router went into a reboot loop after the flash. I had to reset to factory and manually reconfigure the router from scratch to get 386.1 running. Restoring router settings from a settings backup (after factory reset) also triggered the reboot loop. Only the jffs partition could be restored from backup.

 

[L&LD]

You're asking for more issues in the future if you insist on restoring old backup config files (JFFS, or otherwise).

There is no known memory leak issue that manifests itself after 3 weeks.

What drive do you have attached that has the 255MB swap (file or partition) on it?

Does your rebooting stop when you don't plug in that drive?

 

[captaink]

Rebooting loop did not stop after removing the USB thumb drive (named entware and it is old). I just ran a scan and and the drive did not cleanly unmount. I have never run a health scan on this drive. Am thinking I need to switch to a more modern drive.

entware was not cleanly unmounted, check forced.

496 inodes used (0.40%, out of 122880)
9 non-contiguous files (1.8%)
4 non-contiguous directories (0.8%)
# of inodes with ind/dind/tind blocks: 53/14/1
295273 blocks used (60.27%, out of 489948)
0 bad blocks
0 large files

366 regular files
62 directories
0 character device files
0 block device files
0 fifos
0 links
59 symbolic links (59 fast symbolic links)
0 sockets
------------
487 files

 

[L&LD]

Update/Reset Mini Guide

Reset Mini Guide + Control Channel Setup Details

Control Channel Setup 2021

Fully Reset Router and Network

amtm Step-by-Step https://www.snbforums.com/threads/amtm-step-by-step-install-guide-l-ld.56237/#post-483421
Note, no need to 'install' amtm in the link above if using RMerlin 384.15_0 or later).

 

[captaink]

Update/Reset Mini Guide

Reset Mini Guide + Control Channel Setup Details

Control Channel Setup 2021

Fully Reset Router and Network

amtm Step-by-Step https://www.snbforums.com/threads/amtm-step-by-step-install-guide-l-ld.56237/#post-483421
Note, no need to 'install' amtm in the link above if using RMerlin 384.15_0 or later).

Oh, I am very late to the AMTM party. That was a sweet experience. Now have my USB drive formatted and config'd with swap the AMTM way. Entware installed along with Diversion. Have a feeling my "memory leak" issue is going to be resolved now. Thanks for that info!!

 

[captaink]

Just want to bring closure to my issue. Memory behavior on my router is dramatically different after re-config from scratch (plus AMTM usb setup) vs restore from settings backup. Memory is being released. Swap is being used in small amounts. This looks solved to me.

 

[Jeffrey Young]

Yup, sure is amazing what a clean install does!! (goes for any OS really). Glad it all worked out.