RT68U Kernel memory exaustion(?)

[Gnammi]

Hello, hope someone here can figure it out.
I've a strange problem with my rt-ac68u and I haven't any idea on how to fix it/or what is the cause of it.
Simply put, from time to time router becomes irresponsive to the point of not being able to route WIFi and/or wired network. Only fix is a manual restart.
Time between "events" seems random, network is not under terribly heavy load, only 1 pc on WiFi not generating much traffic and trasmission client on a small nas serving just one torrent.

Via SSH I was able to catch the dmesg just before one of those freezes and it looked scary.

...
Free swap = 0kB
Total swap = 0kB
65536 pages of RAM
24014 free pages
1600 reserved pages
6071 slab pages
4179 pages shared
0 pages swap cached
[<c0044000>] (unwind_backtrace+0x0/0xf8) from [<c009e188>] (__alloc_pages_nodemask+0x61c/0x698)
[<c009e188>] (__alloc_pages_nodemask+0x61c/0x698) from [<c0371f44>] (__slab_alloc+0x794/0x814)
[<c0371f44>] (__slab_alloc+0x794/0x814) from [<c0372178>] (__kmalloc_track_caller+0xf8/0x108)
[<c0372178>] (__kmalloc_track_caller+0xf8/0x108) from [<c0374128>] (__alloc_skb+0x54/0x104)
[<c0374128>] (__alloc_skb+0x54/0x104) from [<c01d7ce8>] (osl_alloc_skb+0x18/0x30)
[<c01d7ce8>] (osl_alloc_skb+0x18/0x30) from [<c0383190>] (osl_pktget+0x4c/0x154)
[<c0383190>] (osl_pktget+0x4c/0x154) from [<c038270c>] (_dma_rxfill+0x174/0x3d4)
[<c038270c>] (_dma_rxfill+0x174/0x3d4) from [<bf016ac4>] (chiprx+0xf0/0x140 [et])
unwind: Index not found bf016ac4
Mem-info:
DMA per-cpu:
CPU 0: hi: 42, btch: 7 usd: 13
CPU 1: hi: 42, btch: 7 usd: 38
Normal per-cpu:
CPU 0: hi: 42, btch: 7 usd: 17
CPU 1: hi: 42, btch: 7 usd: 15
active_anon:1156 inactive_anon:1202 isolated_anon:0
active_file:432 inactive_file:566 isolated_file:0
unevictable:0 dirty:0 writeback:0 unstable:0
free:23780 slab_reclaimable:104 slab_unreclaimable:33257
mapped:764 shmem:1084 pagetables:118 bounce:0
DMA free:4144kB min:11020kB low:13772kB high:16528kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:130048kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:122744kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 109 109
Normal free:90976kB min:9456kB low:11820kB high:14184kB active_anon:4624kB inactive_anon:4808kB active_file:1728kB inactive_file:2264kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:111616kB mlocked:0kB dirty:0kB writeback:0kB mapped:3056kB shmem:4336kB slab_reclaimable:416kB slab_unreclaimable:10284kB kernel_stack:456kB pagetables:472kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0
DMA: 4*4kB 0*8kB 0*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 4144kB
Normal: 18*4kB 203*8kB 136*16kB 130*32kB 80*64kB 40*128kB 8*256kB 2*512kB 0*1024kB 2*2048kB 16*4096kB = 90976kB
2082 total pagecache pages
0 pages in swap cache
Swap cache stats: add 0, delete 0, find 0/0
Free swap = 0kB
Total swap = 0kB
65536 pages of RAM
24013 free pages
1600 reserved pages
6071 slab pages
4180 pages shared
0 pages swap cached
osl_ctfpool_add: skb alloc of len 2256 failed
osl_ctfpool_add: skb alloc of len 2256 failed
...

This one is just a small part of it but messages are similar(mostly change just the stack trace of the crash).
From what I can tell, kernel fails to allocate memory at some point but I'm unable to get any hint on what is the reason for the failure.

The router is currently running asuswrt-merlin, but the problem was appearing also with stock firmware, there is nothing installed on it (optware/entware/...), pretty standard "set ip/set password/.../profit?" configuration, I've tried the last two version of asus firmware and the last two of merlin, and erased flash between installation.

Do anyone has the same problem?
This is the second rt-ac68u I get, the first was not powering on :v, I really would prefer to not return it too.

Thanks

 

[System Error Message]

try disabling hardware acceleration.. Try downloading latest merlin firmware, run the md5 hash to make sure that it properly downloaded before flashing it. If you use windows you can try md5sums.

 

[Gnammi]

Would be terribly disappointing if the problem is in hw acceleration.
Anyway I'm going to reflash again to see if the problem is still there.
Wish me luck :v

 

[System Error Message]

make sure to do an md5 check of the firmware file to make sure it properly downloaded before flashing. It involves using a program to calculate the md5 hash of the file which you than compare with the md5 hash given on the download page. Verifying this is very very important. Flashing a corrupted firmware can give all sorts of problems or even brick the device.

 

[joegreat]

Hi,

Are you running Transmission (torrent sharing) or other software on your router?
I have a similar memory related issue (documented here) on my AC68U router, which I am currently trying to mail down via optimized settings for buffers and cache:
In my case Transmission is using (indirect) to much memory and not freeing it up fast enugh. Actually it's the memory handling by the Linux kernel, which is reacting odd on the heavy disk load from Transmission.

I created a init-start user script which sets the parameters to flush the memory requently, which allows (other) process to claim memory when needed.

Maybe you want to give it a try?

With kind regards
Joe

init-start user script:

#!/bin/sh
/usr/bin/logger -t START_$(basename $0) "started [$@]"
SCRLOG=/tmp/$(basename $0).log
touch $SCRLOG
NOW=$(date +"%Y-%m-%d %H:%M:%S")
echo $NOW "START_$(basename $0) started [$@]" >> $SCRLOG
# exec 3>&1 4>&2 >$SCRLOG 2>&1
#
echo 4194304 > /proc/sys/net/core/rmem_max
echo 1048576 > /proc/sys/net/core/wmem_max
echo 32768 > /proc/sys/vm/min_free_kbytes
echo 5 > /proc/sys/vm/dirty_background_ratio
echo 10 > /proc/sys/vm/dirty_ratio
echo 500 /proc/sys/vm/vfs_cache_pressure
echo 20 > /proc/sys/vm/swappiness
#
NOW=$(date +"%Y-%m-%d %H:%M:%S")
if [ "$?" -ne 0 ]
then
echo $NOW "Error in init-start execution! Script: $0" >> $SCRLOG
else
echo $NOW "Init-start execution OK. Script: $0" >> $SCRLOG
fi
/usr/bin/logger -t STOP_$(basename $0) "return code $?"
# exec 1>&3 2>&4
exit $?