FlexQoS FlexQoS issues with 388.4 HND5.04 models

[dave14305]

Regardless just letting you know.

Yes, I usually hear about iptables rules not working with newer routers. tc filters (AppDB rules) hadn’t been a problem so far. If plain Adaptive QoS works for you, just stick with that, or CAKE.

 

[Yiannis]

Is there any known incompatibility between 3004.388.4 and FlexQoS? I was installing the beta versions off 388.4 and at beta 3 the router became very unstable with reboots every half an hour maybe. Went back to 388.2_2 and everything returned back to normal. Then 388.4 came out, tried with dirty upgrade and again the unstable issue appeared. This is when I decided to do a WPS factory reset and reconfigure from scratch 388.4. All addons I am using are FlexQoS and YazDHCP. Again, I had multiple reboots of the rooter every few minutes. Things returned back to normal when I just uninstalled FlexQoS. That’s why I am asking. My next move will be to install it again and monitor the situation.

 

[harry]

Is there any known incompatibility between 3004.388.4 and FlexQoS? I was installing the beta versions off 388.4 and at beta 3 the router became very unstable with reboots every half an hour maybe. Went back to 388.2_2 and everything returned back to normal. Then 388.4 came out, tried with dirty upgrade and again the unstable issue appeared. This is when I decided to do a WPS factory reset and reconfigure from scratch 388.4. All addons I am using are FlexQoS and YazDHCP. Again, I had multiple reboots of the rooter every few minutes. Things returned back to normal when I just uninstalled FlexQoS. That’s why I am asking. My next move will be to install it again and monitor the situation.

did you reformatted the USB/ssd drive ? if not try doing that.

 

[Yiannis]

did you reformatted the USB/ssd drive ? if not try doing that.

I did not. I have attached a 2.5 SSD 120GB with a SATA to USB enclosure and haven't experienced any issues so far. If the issues arise after I install again FlexQoS, I will reformat the drive.

 

[maghuro]

You're not alone.
Flexqos gives me reboots on latest releases, I talked about this on releases threads.
I did everything, including format router and USB and started from scratch. As soon as I installed flexqos, reboots started making router unusable.

 

[harry]

You're not alone.
Flexqos gives me reboots on latest releases, I talked about this on releases threads.
I did everything, including format router and USB and started from scratch. As soon as I installed flexqos, reboots started making router unusable.

I agree. I did had to fiddle around a bit too. but after FD command and reinstalling in same order as 1,2,3 div, sky, flex it's sort of sorted. still have some latency on upload but I do get A score which is good enough. up until beta 3 it was A+ same settings. so it could be some flaws in new firmware too.

 

[SomeWhereOverTheRainBow]

I agree. I did had to fiddle around a bit too. but after FD command and reinstalling in same order as 1,2,3 div, sky, flex it's sort of sorted. still have some latency on upload but I do get A score which is good enough. up until beta 3 it was A+ same settings. so it could be some flaws in new firmware too.

I concur, there seems to be some issues that were introduced based on the latest changes from ASUS to the QoS. I have noticed some nuances when adding maximum rules to FlexQoS using the iptables options. I maxed out with 25 rules. Upon extreme network bombardment (kids playing on their gaming platforms, while I was streaming movies), the router decided to do a random reboot. I detected the router reboot once all my devices lost wireless signal. Once I logged into the router, I observed the harddrive mounting and all scripts firing off as if the router just came out of a reboot. Since I went back to default rules on FlexQoS, the issue went away. Cannot get above an A on bufferbloat tests (no A+, with or without FlexQoS installed bufferbloat never goes above an A). This leads me to suspect, there may be something bugged out in the QoS not necessarily directly introduced from FlexQoS. However, I believe those using FlexQoS are more susceptible to discovering the bug. If I understood more of the dynamics of the QoS internals, I would try to debug further, but I am out of my league in that arena.

 

[grateful70]

Is there any known incompatibility between 3004.388.4 and FlexQoS? I was installing the beta versions off 388.4 and at beta 3 the router became very unstable with reboots every half an hour maybe. Went back to 388.2_2 and everything returned back to normal. Then 388.4 came out, tried with dirty upgrade and again the unstable issue appeared. This is when I decided to do a WPS factory reset and reconfigure from scratch 388.4. All addons I am using are FlexQoS and YazDHCP. Again, I had multiple reboots of the rooter every few minutes. Things returned back to normal when I just uninstalled FlexQoS. That’s why I am asking. My next move will be to install it again and monitor the situation.

that's exactly what I listed above in my post. I narrowed all of my issues down to flexqos APPDB.
This was the response "If plain Adaptive QoS works for you, just stick with that, or CAKE."
I like flexqos but if they aren't going to support it, I guess I'm throwing in the towel.
Having said that the June release of Asus 3.0.0.6 addresses a number of CVEs and Merlin isn't going to look into 3006 for some time so right now I'm back to the latest stock firmware.

 

[Yiannis]

Having said that the June release of Asus 3.0.0.6 addresses a number of CVEs and Merlin isn't going to look into 3006 for some time so right now I'm back to the latest stock firmware.

A bit out of scope, but to my understanding latest Merlin's firmware 388.4 uses Asus code base GPL 388_23588. ASUS did patch a number of CVEs in June's update. For my router (RT-AX86U Pro) that was 3.0.0.4_388.23565. Hence, Merlin's 23588 seems to already includes what Asus stock 23565 has. That's my understanding.

 

[grateful70]

A bit out of scope, but to my understanding latest Merlin's firmware 388.4 uses Asus code base GPL 388_23588. ASUS did patch a number of CVEs in June's update. For my router (RT-AX86U Pro) that was 3.0.0.4_388.23565. Hence, Merlin's 23588 seems to already includes what Asus stock 23565 has. That's my understanding.

thanks. I went back to 388.4 with skynet. I tried flexqos again on a lark. nope. problems with it again. uninstalled and stable running just skynet

 

[Kingp1n]

I concur, there seems to be some issues that were introduced based on the latest changes from ASUS to the QoS. I have noticed some nuances when adding maximum rules to FlexQoS using the iptables options. I maxed out with 25 rules. Upon extreme network bombardment (kids playing on their gaming platforms, while I was streaming movies), the router decided to do a random reboot. I detected the router reboot once all my devices lost wireless signal. Once I logged into the router, I observed the harddrive mounting and all scripts firing off as if the router just came out of a reboot. Since I went back to default rules on FlexQoS, the issue went away. Cannot get above an A on bufferbloat tests (no A+, with or without FlexQoS installed bufferbloat never goes above an A). This leads me to suspect, there may be something bugged out in the QoS not necessarily directly introduced from FlexQoS. However, I believe those using FlexQoS are more susceptible to discovering the bug. If I understood more of the dynamics of the QoS internals, I would try to debug further, but I am out of my league in that arena.

Yes definitely seems to be a problem with the newer AX models. I do remember having a lot of disconnect issues (with A. QOS enabled) while trying out the AXE16000 and AX11000 Pro version a few months ago when it was 1st supported by asus-wrt merlin. I never tried it with stock firmware. I ended up returning them not figuring out what the issues was at the time and I ended going back to my old AX11000. Hopefully this ssue can be resolved.

 

[grateful70]

I concur, there seems to be some issues that were introduced based on the latest changes from ASUS to the QoS. I have noticed some nuances when adding maximum rules to FlexQoS using the iptables options. I maxed out with 25 rules. Upon extreme network bombardment (kids playing on their gaming platforms, while I was streaming movies), the router decided to do a random reboot. I detected the router reboot once all my devices lost wireless signal. Once I logged into the router, I observed the harddrive mounting and all scripts firing off as if the router just came out of a reboot. Since I went back to default rules on FlexQoS, the issue went away. Cannot get above an A on bufferbloat tests (no A+, with or without FlexQoS installed bufferbloat never goes above an A). This leads me to suspect, there may be something bugged out in the QoS not necessarily directly introduced from FlexQoS. However, I believe those using FlexQoS are more susceptible to discovering the bug. If I understood more of the dynamics of the QoS internals, I would try to debug further, but I am out of my league in that arena.

I wouldn't put too much stock in Bufferbloat. There are too many factors involved and I've run tests back to back before where a C magically turns to an A with no router config change and vice versa. We simply can't control the whole path nor control their internet feeds.
Having said that, router reboots are horrifying.

 

[grateful70]

It isn't to say bufferbloat is the main concern. Instead the main concern is "did asus break anything in any of their recent endeavors to improve our QoS experience?".That is what I suspect, but cannot verify or substantiate.

Understood. I try to refrain from using qos at all unless I have saturation. There's always added latency when the router has to tag every flow. I'm certainly not defending Asus if they have botched something.

 

[SomeWhereOverTheRainBow]

Understood. I try to refrain from using qos at all unless I have saturation. There's always added latency when the router has to tag every flow. I'm certainly not defending Asus if they have botched something.

In my network, I have no choice but to use QoS. First factor is my download, upload connections are symmetric. Upload can be horrible without it. Second factor is the amount of clients in the pecking order and saturation to the network.

 

[grateful70]

In my network, I have no choice but to use QoS. First factor is my download, upload connections are symmetric. Upload can be horrible witho

Ya, different strokes. I have about 30 devices that can take or leave QoS. Gaming, Dropbox, Netflix, Prime, Disney, VOIP, Work from home, torrenting, MS Teams. Even with everything full bore QoS adds little benefit for me. Teams, gaming and Voip phones all work fine. Everything else can use it's cache. I was big on QoS back in the days of early hi-speed (5 meg) but a lot of people don't realize that the ISPs tag traffic along the path which takes care of most of it. (and our markings don't persist through the ISP anyway as it's local outbound only )

 

[dave14305]

The first step should be to change QoS Mode in FlexQoS from fq_codel to ASUS and see if the problems remain. I have no way to troubleshoot this myself.

 

[Yiannis]

The first step should be to change QoS Mode in FlexQoS from fq_codel to ASUS and see if the problems remain. I have no way to troubleshoot this myself.

Thanks Dave! FlexQoS is one of my favourites Add Ons. I'll try it.

I can see the discussion goes on what Asus might have done with their code, but for me the problems started on 388.4 beta 3. Was any codebase change between 388.4 beta 2 to 388.4 beta 3?

Also, I have another RT-AX86U Pro that I am remotely managing which is still in 388.4 beta 2 with no reboot problems. When I saw the problems with my main one in beta 3, I just skipped the upgrade.

 

[Yiannis]

Just installed FlexQoS and got my first reboot in two days and half days. Here are some logs

Aug 26 07:26:00 FlexQoS: Updated flexqos.asp
Aug 26 07:26:05 rc_service: service 17598:notify_rc restart_qos;restart_firewall
Aug 26 07:26:05 custom_script: Running /jffs/scripts/service-event (args: restart qos)
Aug 26 07:26:08 BWDPI: fun bitmap = 83
Aug 26 07:26:08 A.QoS: qos_count=0, qos_check=0
Aug 26 07:26:11 custom_script: Running /jffs/scripts/service-event-end (args: restart qos)
Aug 26 07:26:11 custom_script: Running /jffs/scripts/service-event (args: restart firewall)
Aug 26 07:26:12 custom_script: Running /jffs/scripts/firewall-start (args: eth0)
Aug 26 07:26:12 custom_script: Running /jffs/scripts/service-event-end (args: restart firewall)
Aug 26 07:26:12 FlexQoS: /jffs/addons/flexqos/flexqos.sh (pid=19695) called in unattended mode with 1 args: -start
Aug 26 07:26:12 FlexQoS: Applying iptables static rules
Aug 26 07:26:12 FlexQoS: Applying iptables custom rules
Aug 26 07:26:13 FlexQoS: Flushing conntrack table
Aug 26 07:26:13 FlexQoS: Applying AppDB rules and TC rates
May  5 08:05:09 syslogd started: BusyBox v1.25.1
May  5 08:05:09 syslogd: using server 192.168.177.100:514
May  5 08:05:09 kernel: klogd started: BusyBox v1.25.1 (2023-08-21 15:34:18 EDT)
May  5 08:05:09 kernel: Booting Linux on physical CPU 0x0000000000 [0x420f1000]
May  5 08:05:09 kernel: Linux version 4.19.183 (merlin@ubuntu-dev) (gcc version 9.2.0 (Buildroot 2019.11.1)) #1 SMP PREEMPT Mon Aug 21 15:34:47 EDT 2023
May  5 08:05:09 kernel: Machine model: RTAX86U_PRO
May  5 08:05:09 kernel: earlycon: pl11 at MMIO 0x00000000ff812000 (options '115200n8')
May  5 08:05:09 kernel: bootconsole [pl11] enabled
May  5 08:05:09 kernel: Reserved memory: created CMA memory pool at 0x0000000035000000, size 176 MiB

I also got another one, probably when tried to save configuration of fq_codel >> Asus , but no flexqos entries on the logs on the second reboot. I will be monitoring the situation and report back.

 

[SomeWhereOverTheRainBow]

@dave14305 This is on RT-AX88U-PRO running the latest release using only the "ASUS" option.

Spoiler: FlexQoS Debug

FlexQoS v1.3.3 released 2023-08-09

Debug date      : 2023-08-26 02:01:46-0400
Router Model    : RT-AX88U_Pro
Firmware Ver    : 388.4_0
DPI/Sig Ver     : 2.0.3 / 2.368
WAN iface       : eth0
tc WAN iface    : eth0
IPv6            : dhcp6
Undf Prio       : 2
Down Band       : 1105920
Up Band         : 36864
*****************
Net Control     : 1:10
Work-From-Home  : 1:13
Gaming          : 1:16
Others          : 1:11
Web Surfing     : 1:15
Streaming       : 1:12
File Transfers  : 1:17
Learn-From-Home : 1:14
*****************
Downrates       :   55296,  110592,  331776,  221184,   55296,  110592,  165888,   55296
Downceils       : 1105920, 1105920, 1105920, 1105920, 1105920, 1105920, 1105920, 1105920
Uprates         :    1843,   11059,    3686,    7372,    1843,    3686,    5529,    1843
Upceils         :   36864,   36864,   36864,   36864,   36864,   36864,   36864,   36864
*****************
iptables settings: <>>udp>>500,4500>>3<>>udp>16384:16415>>>3<>>udp>>19302:19309>>3<>>udp>>3478:3481>000000>3<>>udp>>8801:8810>000000>3<>>tcp>>119,563>>5<>>tcp>>80,443>08****>5
-A FlexQoS_down -p udp -m multiport --sports 500,4500 -j MARK --set-mark 0x8006ffff/0xc03fffff
-A FlexQoS_up -p udp -m multiport --dports 500,4500 -j MARK --set-mark 0x4006ffff/0xc03fffff
-A FlexQoS_down -p udp -m multiport --sports 500,4500 -j MARK --set-mark 0x8006ffff/0xc03fffff
-A FlexQoS_up -p udp -m multiport --dports 500,4500 -j MARK --set-mark 0x4006ffff/0xc03fffff
-A FlexQoS_down -p udp -m multiport --dports 16384:16415 -j MARK --set-mark 0x8006ffff/0xc03fffff
-A FlexQoS_up -p udp -m multiport --sports 16384:16415 -j MARK --set-mark 0x4006ffff/0xc03fffff
-A FlexQoS_down -p udp -m multiport --dports 16384:16415 -j MARK --set-mark 0x8006ffff/0xc03fffff
-A FlexQoS_up -p udp -m multiport --sports 16384:16415 -j MARK --set-mark 0x4006ffff/0xc03fffff
-A FlexQoS_down -p udp -m multiport --sports 19302:19309 -j MARK --set-mark 0x8006ffff/0xc03fffff
-A FlexQoS_up -p udp -m multiport --dports 19302:19309 -j MARK --set-mark 0x4006ffff/0xc03fffff
-A FlexQoS_down -p udp -m multiport --sports 19302:19309 -j MARK --set-mark 0x8006ffff/0xc03fffff
-A FlexQoS_up -p udp -m multiport --dports 19302:19309 -j MARK --set-mark 0x4006ffff/0xc03fffff
-A FlexQoS_down -p udp -m multiport --sports 3478:3481 -m mark --mark 0x80000000/0xc03fffff -j MARK --set-mark 0x8006ffff/0xc03fffff
-A FlexQoS_up -p udp -m multiport --dports 3478:3481 -m mark --mark 0x40000000/0xc03fffff -j MARK --set-mark 0x4006ffff/0xc03fffff
-A FlexQoS_down -p udp -m multiport --sports 3478:3481 -m mark --mark 0x80000000/0xc03fffff -j MARK --set-mark 0x8006ffff/0xc03fffff
-A FlexQoS_up -p udp -m multiport --dports 3478:3481 -m mark --mark 0x40000000/0xc03fffff -j MARK --set-mark 0x4006ffff/0xc03fffff
-A FlexQoS_down -p udp -m multiport --sports 8801:8810 -m mark --mark 0x80000000/0xc03fffff -j MARK --set-mark 0x8006ffff/0xc03fffff
-A FlexQoS_up -p udp -m multiport --dports 8801:8810 -m mark --mark 0x40000000/0xc03fffff -j MARK --set-mark 0x4006ffff/0xc03fffff
-A FlexQoS_down -p udp -m multiport --sports 8801:8810 -m mark --mark 0x80000000/0xc03fffff -j MARK --set-mark 0x8006ffff/0xc03fffff
-A FlexQoS_up -p udp -m multiport --dports 8801:8810 -m mark --mark 0x40000000/0xc03fffff -j MARK --set-mark 0x4006ffff/0xc03fffff
-A FlexQoS_down -p tcp -m multiport --sports 119,563 -j MARK --set-mark 0x8003ffff/0xc03fffff
-A FlexQoS_up -p tcp -m multiport --dports 119,563 -j MARK --set-mark 0x4003ffff/0xc03fffff
-A FlexQoS_down -p tcp -m multiport --sports 119,563 -j MARK --set-mark 0x8003ffff/0xc03fffff
-A FlexQoS_up -p tcp -m multiport --dports 119,563 -j MARK --set-mark 0x4003ffff/0xc03fffff
-A FlexQoS_down -p tcp -m multiport --sports 80,443 -m mark --mark 0x80080000/0xc03f0000 -j MARK --set-mark 0x8003ffff/0xc03fffff
-A FlexQoS_up -p tcp -m multiport --dports 80,443 -m mark --mark 0x40080000/0xc03f0000 -j MARK --set-mark 0x4003ffff/0xc03fffff
-A FlexQoS_down -p tcp -m multiport --sports 80,443 -m mark --mark 0x80080000/0xc03f0000 -j MARK --set-mark 0x8003ffff/0xc03fffff
-A FlexQoS_up -p tcp -m multiport --dports 80,443 -m mark --mark 0x40080000/0xc03f0000 -j MARK --set-mark 0x4003ffff/0xc03fffff
*****************
appdb rules: Defaults
filter change dev br0 prio 2 protocol all handle 828::800 u32 flowid 1:11
filter change dev eth0 prio 2 protocol all handle 828::800 u32 flowid 1:11
filter add dev br0 protocol all prio 2 u32 match mark 0x8000006B 0xc03fffff flowid 1:11
filter add dev eth0 protocol all prio 2 u32 match mark 0x4000006B 0xc03fffff flowid 1:11
filter add dev br0 protocol all prio 15 u32 match mark 0x800D0007 0xc03fffff flowid 1:17
filter add dev eth0 protocol all prio 15 u32 match mark 0x400D0007 0xc03fffff flowid 1:17
filter add dev br0 protocol all prio 15 u32 match mark 0x800D0086 0xc03fffff flowid 1:17
filter add dev eth0 protocol all prio 15 u32 match mark 0x400D0086 0xc03fffff flowid 1:17
filter add dev br0 protocol all prio 15 u32 match mark 0x800D00A0 0xc03fffff flowid 1:17
filter add dev eth0 protocol all prio 15 u32 match mark 0x400D00A0 0xc03fffff flowid 1:17
filter add dev br0 protocol all prio 20 u32 match mark 0x8012003F 0xc03fffff flowid 1:15
filter add dev eth0 protocol all prio 20 u32 match mark 0x4012003F 0xc03fffff flowid 1:15
filter change dev br0 prio 22 protocol all handle 802::800 u32 flowid 1:15
filter change dev eth0 prio 22 protocol all handle 802::800 u32 flowid 1:15
filter change dev br0 prio 23 protocol all handle 804::800 u32 flowid 1:15
filter change dev eth0 prio 23 protocol all handle 804::800 u32 flowid 1:15
class change dev br0 parent 1:1 classid 1:10 htb overhead 42 linklayer ethernet prio 0 rate 55296Kbit ceil 1105920Kbit burst 138240b cburst 1382400b quantum 691200
class change dev eth0 parent 1:1 classid 1:10 htb overhead 42 linklayer ethernet prio 0 rate 1843Kbit ceil 36864Kbit burst 4608b cburst 44800b quantum 23037
class change dev br0 parent 1:1 classid 1:11 htb overhead 42 linklayer ethernet prio 1 rate 110592Kbit ceil 1105920Kbit burst 138240b cburst 1382400b quantum 1382400
class change dev eth0 parent 1:1 classid 1:11 htb overhead 42 linklayer ethernet prio 1 rate 11059Kbit ceil 36864Kbit burst 4608b cburst 44800b quantum 138237
class change dev br0 parent 1:1 classid 1:12 htb overhead 42 linklayer ethernet prio 2 rate 331776Kbit ceil 1105920Kbit burst 138240b cburst 1382400b quantum 4147200
class change dev eth0 parent 1:1 classid 1:12 htb overhead 42 linklayer ethernet prio 2 rate 3686Kbit ceil 36864Kbit burst 4608b cburst 44800b quantum 46075
class change dev br0 parent 1:1 classid 1:13 htb overhead 42 linklayer ethernet prio 3 rate 221184Kbit ceil 1105920Kbit burst 138240b cburst 1382400b quantum 2764800
class change dev eth0 parent 1:1 classid 1:13 htb overhead 42 linklayer ethernet prio 3 rate 7372Kbit ceil 36864Kbit burst 4608b cburst 44800b quantum 92150
class change dev br0 parent 1:1 classid 1:14 htb overhead 42 linklayer ethernet prio 4 rate 55296Kbit ceil 1105920Kbit burst 138240b cburst 1382400b quantum 691200
class change dev eth0 parent 1:1 classid 1:14 htb overhead 42 linklayer ethernet prio 4 rate 1843Kbit ceil 36864Kbit burst 4608b cburst 44800b quantum 23037
class change dev br0 parent 1:1 classid 1:15 htb overhead 42 linklayer ethernet prio 5 rate 110592Kbit ceil 1105920Kbit burst 138240b cburst 1382400b quantum 1382400
class change dev eth0 parent 1:1 classid 1:15 htb overhead 42 linklayer ethernet prio 5 rate 3686Kbit ceil 36864Kbit burst 4608b cburst 44800b quantum 46075
class change dev br0 parent 1:1 classid 1:16 htb overhead 42 linklayer ethernet prio 6 rate 165888Kbit ceil 1105920Kbit burst 138240b cburst 1382400b quantum 2073600
class change dev eth0 parent 1:1 classid 1:16 htb overhead 42 linklayer ethernet prio 6 rate 5529Kbit ceil 36864Kbit burst 4608b cburst 44800b quantum 69112
class change dev br0 parent 1:1 classid 1:17 htb overhead 42 linklayer ethernet prio 7 rate 55296Kbit ceil 1105920Kbit burst 138240b cburst 1382400b quantum 691200
class change dev eth0 parent 1:1 classid 1:17 htb overhead 42 linklayer ethernet prio 7 rate 1843Kbit ceil 36864Kbit burst 4608b cburst 44800b quantum 23037

 

[Yiannis]

After leaving the script active for 2 hours, having the ASUS setting (not fq_codel) , I continued getting multiple reboots.