amtm amtm errors: grep memory exhausted and others

[mxoz]

Hi,

I recently got RT-AX58U, installed Asuswrt-Merlin, then installed Diversion, uiDivStats, Skynet. All the time following various instructions from this forum (mainly posts from L&LD).
Everything was working fine for a couple of days. Today I run amtm and got "grep: memory exhausted" errors as in screenshot1. If I then run, for example, upgrade command it shows different errors as in screenshot2 (though those hosts are in practice reachable). If I run, for example, "dc" command it again prints me another errors as in screenshot3. Any ideas what might be the problem?

 

[L&LD]

Does this issue continue if you reboot the router (via the GUI)?

 

[mxoz]

Does this issue continue if you reboot the router (via the GUI)?

Yes, it does. (if it matters, both GUI and top show enough free RAM).

 

[mxoz]

Simply running grep in terminal with any option also produces segfault:

# grep --version
Segmentation fault

or another reproducer:

# grep "THEME" /opt/share/diversion/.conf/diversion.conf
grep: memory exhausted

 

[L&LD]

Did the USB drive die on you?

 

[mxoz]

Did the USB drive die on you?

No.

 

[L&LD]

How do you know?

 

[mxoz]

How do you know?

GUI USB Health Scanner doesn't complain.
The disk is mounted.
I can do read/write on it.
New logs (for example from Skynet) land on it successfully.

 

[L&LD]

What brand/model and capacity is it?

What file format are you using? Are you using a swap file or a swap partition?

Have you 'restored' any scripts/files/settings from a backup since you created it?

 

[mxoz]

What brand/model and capacity is it?

What file format are you using? Are you using a swap file or a swap partition?

Have you 'restored' any scripts/files/settings from a backup since you created it?

Verbatim Store 'n' Go, 16GB.

ext4, swap file (I followed amtm and Diversion guidelines to create it).

Didn't restore anything so far.

 

[mxoz]

I assume it is not amtm's issue per se as even a simple grep commands fail. I am curious what devs working on apps on Asuswrt-Merlin can say about this behaviour. @RMerlin , @thelonelycoder , @Jack Yaz have you seen such weird behaviour of grep? is there something in OS which may cause it? USB? Any troubleshooting advice?

 

[Jack Yaz]

I assume it is not amtm's issue per se as even a simple grep commands fail. I am curious what devs working on apps on Asuswrt-Merlin can say about this behaviour. @RMerlin , @thelonelycoder , @Jack Yaz have you seen such weird behaviour of grep? is there something in OS which may cause it? USB? Any troubleshooting advice?

interesting is the screenshot of amtm saying it's adding the missing swap entry. please post output of:

free -m

 

[mxoz]

interesting is the screenshot of amtm saying it's adding the missing swap entry. please post output of:

free -m

# free -m
             total       used       free     shared    buffers     cached
Mem:        512260     479904      32356       2336      55092      49776
-/+ buffers/cache:     375036     137224
Swap:      2097148        704    2096444

This is after 24h of uptime.
Just after reboot it usually shows 160MB free.

 

[mxoz]

I also noticed that Diversion stats have not been updated since yesterday and "ps" shows tens and tens of "{uiDivStats} /bin/sh /jffs/scripts/uiDivStats querylog" commands and tens and tens of "sleep 1" commands. (not sure what is the cause and what is the effect here).

 

[Jack Yaz]

I also noticed that Diversion stats have not been updated since yesterday and "ps" shows tens and tens of "{uiDivStats} /bin/sh /jffs/scripts/uiDivStats querylog" commands and tens and tens of "sleep 1" commands. (not sure what is the cause and what is the effect here).

can you share the output of

ps | grep -i uiDivStats

please?

 

[mxoz]

can you share the output of

ps | grep -i uiDivStats

please?

I can't, because grep gives segfault.

I didn't find how to temporarily disable uiDivStats, so I had to uninstall it for now (as CPU usage was too high and growing as well as uiDivStats processes). Now CPU usage is back to normal. Anyway, I assume it is all results of some fault in the system, which also causes "grep" to fail..

 

[ColinTaylor]

It's trying to use Entware's grep. My guess (based on nothing at all) is that some of the Entware stuff isn't compatible with the RT-AX58U.

 

[Jack Yaz]

It's trying to use Entware's grep. My guess (based on nothing at all) is that some of the Entware stuff isn't compatible with the RT-AX58U.

that was my thought too

ps | /bin/grep -i uiDivStats

FWIW I'm building in a stale querylog process killer into the next version of uiDivStats

 

[mxoz]

It's trying to use Entware's grep. My guess (based on nothing at all) is that some of the Entware stuff isn't compatible with the RT-AX58U.

Are there two different "grep"s? I found one in /opt/bin and another one in /tmp/mnt/usb/entware/bin, but they have at least the same size.
Curious what could be so special in AX58U wrt compatibility. There are other users of AX58U and I haven't seen such problems reported by them (at least in this forum).
Not sure how to troubleshoot it further.. If there would be gdb at least..