AX86U Merlin and Cake QoS

[anotherengineer]

So I was/am having issues with DL/UL pings all over the place, as in up to 600ms. Connection is 250/30. Setting cake qos manual to 245-DL basically fixed the DL. UL manual settings from 30 to 20 in 1Mbps increments, no change.

WAN packet overhead set to internet with VLAN.

Just wondering if there is a bug?

 

[dave14305]

Is that a wired or wireless test you’re doing? What’s the ISP connection? Using “Ethernet with VLAN” may not be the most appropriate.

Keep reducing the upload bandwidth until you get the bloat under control. Start with 5 and work your way up. If it’s not under control at 5, something outside the router is pretty wrong.

 

[anotherengineer]

Is that a wired or wireless test you’re doing? What’s the ISP connection? Using “Ethernet with VLAN” may not be the most appropriate.

Keep reducing the upload bandwidth until you get the bloat under control. Start with 5 and work your way up. If it’s not under control at 5, something outside the router is pretty wrong.

Bell Fibreop FTTH - comes into a Nokia G-240G-A ONT then into the Asus AX86U. VLAN 35 for bell fibreop (northerntel)..............but maybe it's not ideal, I just took the best guess from the Wan packet overhead drop-down menu. Wired cat6 from router to PC.

Thanks I will try from 5 and go up from there.

 

[anotherengineer]

Is that a wired or wireless test you’re doing? What’s the ISP connection? Using “Ethernet with VLAN” may not be the most appropriate.

Keep reducing the upload bandwidth until you get the bloat under control. Start with 5 and work your way up. If it’s not under control at 5, something outside the router is pretty wrong.

Tried 5, 10 etc. same thing. Odd thing is the speed tests are still hitting 30Mbps+ even with qos set to 5 or 10 and the loaded ping is all over the place for upload.

 

[Tech9]

Odd thing is the speed tests are still hitting 30Mbps+ even with qos set to 5 or 10

IPv6 enabled? Disable and test again.

 

[dave14305]

Odd thing is the speed tests are still hitting 30Mbps+ even with qos set to 5 or 10 and the loaded ping is all over the place for upload.

That’s not normal. Can you post the output of:

tc -s qdisc
fc status

Any unusual WAN setup?

 

[anotherengineer]

IPv6 enabled? Disable and test again.

Disabled in router long ago and confirmed with test at Gibson Research. But I will double check this weekend. Thanks

 

[anotherengineer]

That’s not normal. Can you post the output of:

tc -s qdisc
fc status

Any unusual WAN setup?

I can. I’m guessing I have to log into ssh terminal or whatever?? Which I don’t know how to do though but I will figure it out. Thanks.

 

[DJones]

I can. I’m guessing I have to log into ssh terminal or whatever?? Which I don’t know how to do though but I will figure it out. Thanks.

Yes in ssh before you type in amtm.

Might as well provide this information also.

Do these line by line in ssh:

tf qdisc
fc status
cat /tmp/qos
sh -x /tmp/qos stop
sh -x /tmp/qos start

Also checking if runner/archer is enabled and running might be something to consider.

 

[anotherengineer]

As Requested

ASUSWRT-Merlin RT-AX86U 3004.388.8_2 Wed Jul 31 23:48:50 UTC 2024

administrator@RT-AX86U-5B08:/tmp/home/root# tc -s qdisc
qdisc pfifo_fast 0: dev eth0 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
Sent 6185299307 bytes 26892236 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc pfifo_fast 0: dev eth1 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
Sent 284793429 bytes 782370 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc pfifo_fast 0: dev eth2 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
Sent 126377241 bytes 537271 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc pfifo_fast 0: dev eth3 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc pfifo_fast 0: dev eth4 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
Sent 11104797885 bytes 8291705 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc pfifo_fast 0: dev eth5 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc pfifo_fast 0: dev spu_us_dummy root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc pfifo_fast 0: dev spu_ds_dummy root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc pfifo_fast 0: dev eth6 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
Sent 37514284425 bytes 30518497 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc pfifo_fast 0: dev eth7 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
Sent 56688037464 bytes 42946959 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc pfifo 8019: dev vlan35 root refcnt 2 limit 1p
Sent 2560144410 bytes 4822066 pkt (dropped 5, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc ingress ffff: dev vlan35 parent ffff:fff1 ----------------
Sent 10667228818 bytes 9293687 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc cake 801a: dev ifb4vlan35 root refcnt 2 bandwidth 250880Kbit besteffort dual-dsthost nat wash ingress no-ack-filter split-gso rtt 100ms noatm overhead 42 mpu 84
Sent 10773939859 bytes 9278213 pkt (dropped 15474, overlimits 13066373 requeues 0)
backlog 0b 0p requeues 0
memory used: 4469248b of 12250Kb
capacity estimate: 250880Kbit
min/max network layer size: 42 / 1500
min/max overhead-adjusted size: 84 / 1542
average network hdr offset: 14

Tin 0
thresh 250880Kbit
target 5ms
interval 100ms
pk_delay 1.21ms
av_delay 612us
sp_delay 19us
backlog 0b
pkts 9293687
bytes 10797340436
way_inds 281768
way_miss 95756
way_cols 0
drops 15474
marks 4
ack_drop 0
sp_flows 2
bk_flows 1
un_flows 0
max_len 1514
quantum 1514

administrator@RT-AX86U-5B08:/tmp/home/root# fc status
Flow Timer Interval = 10000 millisecs
Pkt-HW Activate Deferral rate = 1
Pkt-HW Idle Deactivate = 0
Pkt-SW Activate Deferral count = 0
Flow Low Pkt Rate = 10
Acceleration Mode: <L2 & L3>
MCast Learning <Disabled>
MCast Acceleration IPv4<Enabled> IPv6<Enabled>
IPv6 Learning <Enabled>
GRE Learning <Enabled>
4o6 Fragmentation <Enabled>
TCP Ack Prioritization <Enabled>
HW Acceleration <Disabled>
Notify Processing Mode <Hybrid>
OVS Flow Learning <Disabled>
Flow Learning Disabled : Max<16383>, Active<0>, Cummulative [ 1 - 1 ]

administrator@RT-AX86U-5B08:/tmp/home/root# cat /tmp/qos
#!/bin/sh
source /etc/cake-qos.conf

case "$1" in
start)
# Upload
tc qdisc add dev $ULIF root cake $ULPRIOQUEUE $ULBW $OVERHEAD $FRAMING $ULOPTIONS 2>/dev/null

# Download
ip link add name $MIF type ifb 2>/dev/null
tc qdisc add dev $DLIF handle ffff: ingress 2>/dev/null
tc qdisc add dev $MIF root cake $DLPRIOQUEUE $DLBW $OVERHEAD $FRAMING $DLOPTIONS 2>/dev/null
ip link set $MIF up 2>/dev/null
tc filter add dev $DLIF parent ffff: prio 10 matchall action mirred egress redirect dev $MIF 2>/dev/null

;;
stop)
tc qdisc del dev $ULIF root 2>/dev/null
tc qdisc del dev $DLIF ingress 2>/dev/null
tc qdisc del dev $MIF root 2>/dev/null
ip link set $MIF down 2>/dev/null
ip link del dev $MIF 2>/dev/null
;;
*)
esac

administrator@RT-AX86U-5B08:/tmp/home/root# sh -x /tmp/qos start
+ source /etc/cake-qos.conf
+ ULIF=vlan35
+ DLIF=vlan35
+ MIF=ifb4vlan35
+ ULBW=bandwidth 28672kbit
+ DLBW=bandwidth 250880kbit
+ OVERHEAD=overhead 42 mpu 84
+ FRAMING=
+ ULPRIOQUEUE=diffserv3
+ DLPRIOQUEUE=besteffort
+ ULOPTIONS=nat dual-srchost
+ DLOPTIONS=nat wash dual-dsthost ingress
+ tc qdisc add dev vlan35 root cake diffserv3 bandwidth 28672kbit overhead 42 mpu 84 nat dual-srchost
+ ip link add name ifb4vlan35 type ifb
+ tc qdisc add dev vlan35 handle ffff: ingress
+ tc qdisc add dev ifb4vlan35 root cake besteffort bandwidth 250880kbit overhead 42 mpu 84 nat wash dual-dsthost ingress
+ ip link set ifb4vlan35 up
+ tc filter add dev vlan35 parent ffff: prio 10 matchall action mirred egress redirect dev ifb4vlan35
administrator@RT-AX86U-5B08:/tmp/home/root# sh -x /tmp/qos start
+ source /etc/cake-qos.conf
+ ULIF=vlan35
+ DLIF=vlan35
+ MIF=ifb4vlan35
+ ULBW=bandwidth 28672kbit
+ DLBW=bandwidth 250880kbit
+ OVERHEAD=overhead 42 mpu 84
+ FRAMING=
+ ULPRIOQUEUE=diffserv3
+ DLPRIOQUEUE=besteffort
+ ULOPTIONS=nat dual-srchost
+ DLOPTIONS=nat wash dual-dsthost ingress
+ tc qdisc add dev vlan35 root cake diffserv3 bandwidth 28672kbit overhead 42 mpu 84 nat dual-srchost
+ ip link add name ifb4vlan35 type ifb
+ tc qdisc add dev vlan35 handle ffff: ingress
+ tc qdisc add dev ifb4vlan35 root cake besteffort bandwidth 250880kbit overhead 42 mpu 84 nat wash dual-dsthost ingress
+ ip link set ifb4vlan35 up
+ tc filter add dev vlan35 parent ffff: prio 10 matchall action mirred egress redirect dev ifb4vlan35

administrator@RT-AX86U-5B08:/tmp/home/root#

 

[dave14305]

qdisc pfifo 8019: dev vlan35 root refcnt 2 limit 1p
Sent 2560144410 bytes 4822066 pkt (dropped 5, overlimits 0 requeues 0)
backlog 0b 0p requeues 0

There’s no Cake on upload. Try to run this and post errors:

tc qdisc add dev vlan35 root cake diffserv3 bandwidth 28672kbit overhead 42 mpu 84 nat dual-srchost

 

[anotherengineer]

There’s no Cake on upload. Try to run this and post errors:

tc qdisc add dev vlan35 root cake diffserv3 bandwidth 28672kbit overhead 42 mpu 84 nat dual-srchost

ASUSWRT-Merlin RT-AX86U 3004.388.8_2 Wed Jul 31 23:48:50 UTC 2024

administrator@RT-AX86U-5B08:/tmp/home/root# tc qdisc add dev vlan35 root cake diffserv3 bandwidth 28672kbit overhead 42 mpu 84 nat dual-srchost

RTNETLINK answers: File exists

administrator@RT-AX86U-5B08:/tmp/home/root#

 

[dave14305]

RTNETLINK answers: File exists

Try

tc qdisc replace dev vlan35 root cake diffserv3 bandwidth 28672kbit overhead 42 mpu 84 nat dual-srchost

 

[anotherengineer]

Try

tc qdisc replace dev vlan35 root cake diffserv3 bandwidth 28672kbit overhead 42 mpu 84 nat dual-srchost

ASUSWRT-Merlin RT-AX86U 3004.388.8_2 Wed Jul 31 23:48:50 UTC 2024

administrator@RT-AX86U-5B08:/tmp/home/root# tc qdisc replace dev vlan35 root cake diffserv3 bandwidth 28672kbit overhead 42 mpu 84 nat dual-srchost

administrator@RT-AX86U-5B08:/tmp/home/root#

Nothing came back?
edit 2
just wow

Thank you very much Sir!!

 

[dave14305]

ASUSWRT-Merlin RT-AX86U 3004.388.8_2 Wed Jul 31 23:48:50 UTC 2024

administrator@RT-AX86U-5B08:/tmp/home/root# tc qdisc replace dev vlan35 root cake diffserv3 bandwidth 28672kbit overhead 42 mpu 84 nat dual-srchost

administrator@RT-AX86U-5B08:/tmp/home/root#

nothing came back?

Good. Now run a speedtest then run tc -s qdisc again.

 

[anotherengineer]

Try

tc qdisc replace dev vlan35 root cake diffserv3 bandwidth 28672kbit overhead 42 mpu 84 nat dual-srchost

I guess the only question I have now is can I manually edit the speeds in the router GUI and will it pick it up? thanks

 

[anotherengineer]

Good. Now run a speedtest then run tc -s qdisc again.

as requested

administrator@RT-AX86U-5B08:/tmp/home/root# tc -s qdisc

qdisc pfifo_fast 0: dev eth0 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
Sent 6441565206 bytes 27776372 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc pfifo_fast 0: dev eth1 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
Sent 288470256 bytes 806212 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc pfifo_fast 0: dev eth2 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
Sent 130309443 bytes 559089 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc pfifo_fast 0: dev eth3 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc pfifo_fast 0: dev eth4 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
Sent 12531754846 bytes 9325007 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc pfifo_fast 0: dev eth5 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc pfifo_fast 0: dev spu_us_dummy root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc pfifo_fast 0: dev spu_ds_dummy root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc pfifo_fast 0: dev eth6 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
Sent 38149785883 bytes 31124673 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc pfifo_fast 0: dev eth7 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
Sent 56812307228 bytes 43072029 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc cake 801b: dev vlan35 root refcnt 2 bandwidth 28672Kbit diffserv3 dual-srchost nat nowash no-ack-filter split-gso rtt 100ms noatm overhead 42 mpu 84
Sent 218885332 bytes 728707 pkt (dropped 296, overlimits 429925 requeues 0) (is that ok?)
backlog 0b 0p requeues 0
memory used: 329472b of 4Mb
capacity estimate: 28672Kbit
min/max network layer size: 28 / 1500
min/max overhead-adjusted size: 84 / 1542
average network hdr offset: 14

Bulk Best Effort Voice
thresh 1792Kbit 28672Kbit 7168Kbit
target 10.2ms 5ms 5ms
interval 105ms 100ms 100ms
pk_delay 0us 506us 278us
av_delay 0us 15us 13us
sp_delay 0us 1us 2us
backlog 0b 0b 0b
pkts 0 728815 188
bytes 0 219290012 39022
way_inds 0 3283 0
way_miss 0 2828 6
way_cols 0 0 0
drops 0 296 0
marks 0 0 0
ack_drop 0 0 0
sp_flows 0 1 2
bk_flows 0 1 0
un_flows 0 0 0
max_len 0 1514 393
quantum 300 875 300

qdisc ingress ffff: dev vlan35 parent ffff:fff1 ----------------
Sent 12822100126 bytes 10971365 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc cake 801a: dev ifb4vlan35 root refcnt 2 bandwidth 250880Kbit besteffort dual-dsthost nat wash ingress no-ack-filter split-gso rtt 100ms noatm overhead 42 mpu 84
Sent 12941908701 bytes 10949028 pkt (dropped 22337, overlimits 15720159 requeues 0)
backlog 0b 0p requeues 0
memory used: 6088064b of 12250Kb
capacity estimate: 250880Kbit
min/max network layer size: 42 / 1500
min/max overhead-adjusted size: 84 / 1542
average network hdr offset: 14

Tin 0
thresh 250880Kbit
target 5ms
interval 100ms
pk_delay 7us
av_delay 2us
sp_delay 1us
backlog 0b
pkts 10971365
bytes 12975699236
way_inds 322972
way_miss 108808
way_cols 0
drops 22337
marks 4
ack_drop 0
sp_flows 2
bk_flows 1
un_flows 0
max_len 1514
quantum 1514

administrator@RT-AX86U-5B08:/tmp/home/root#

 

[dave14305]

To make this correction persist, you need to create a custom script to modify the QoS commands. If you just copy/paste all these commands at once, it should set it up for you in one fell swoop.

cat >/jffs/scripts/qos-start <<-EOF
#!/bin/sh
sed -i 's/tc qdisc add dev /tc qdisc replace dev /g' /tmp/qos
EOF
chmod 755 /jffs/scripts/qos-start
nvram set jffs2_scripts=1
nvram commit
service restart_qos
tc qdisc
cat /tmp/qos

If you ever want to revert this, just run:

rm /jffs/scripts/qos-start
service restart_qos

 

[anotherengineer]

To make this correction persist, you need to create a custom script to modify the QoS commands. If you just copy/paste all these commands at once, it should set it up for you in one fell swoop.

cat >/jffs/scripts/qos-start <<-EOF
#!/bin/sh
sed -i 's/tc qdisc add dev /tc qdisc replace dev /g' /tmp/qos
EOF
chmod 755 /jffs/scripts/qos-start
nvram set jffs2_scripts=1
nvram commit
service restart_qos
tc qdisc
cat /tmp/qos

If you ever want to revert this, just run:

rm /jffs/scripts/qos-start
service restart_qos

Interesting, so that a merlin bug or cake bug or both?

Edit - when you say persist, does that mean on a reboot the cake upload will revert to being off/disabled?

 

[dave14305]

Interesting, so that a merlin bug or cake bug or both?

I assume it’s something peculiar about Asus setting up the vlan35 interface, but I’m not familiar with that part. Would it ever work after a fresh reboot before?