AC86U on 386.2 cpu load with adaptive qos

[goncalossilva]

That's probably the best you can get with the almost 8-year-old RT-AC68U model.

You may want to try doing a full reset, but I doubt it will help with such limited hardware resources though.

Fully Reset Router and Network

Best Practice Update/Setup Router/AiMesh Node(s) 2021

I can confirm 500/100 worked wonderfully with Adaptive QoS and FlexQoS (and previously, FreshJR) until roughly ~2/3 months ago (last time I checked). So the HW should really not be a problem in this case. I'd love to avoid a full reset, but if it must, I'll do it.

What’s the difference between these two statements? Is FlexQoS enabled in the first case? Is Adaptive QoS still enabled in the second case?

Sorry for the confusion! And yes, your interpretation is correct. Adaptive QoS is enabled in both cases, FlexQoS is only enabled in the first case. Basically, disabling FlexQoS makes everything work as intended. Edited the previous post, it should be much clearer now.

In sum, all else equal:
- No QoS: 500/120
- Adaptive QoS + FlexQoS: 180/110
- Adaptive QoS only: 450/110

With these speeds...doubt you even need QoS.

I'll be the judge of this. But to provide a bit of context, we have a mix of heavy and latency-sensitive usage and QoS makes a huge difference.

 

[dave14305]

Sorry for the confusion! And yes, your interpretation is correct. Adaptive QoS is enabled in both cases, FlexQoS is only enabled in the first case. Basically, disabling FlexQoS makes everything work as intended. Edited the previous post, it should be much clearer now.

In sum, all else equal:
- No QoS: 500/120
- Adaptive QoS + FlexQoS: 180/110
- Adaptive QoS only: 450/110

When you have a chance to enable FlexQoS again, please run flexqos debug and post the output. I’m guessing there’s something sub-optimal with bursting at your speeds. How did you decide on WAN overhead of 4?

 

[goncalossilva]

When you have a chance to enable FlexQoS again, please run flexqos debug and post the output. I’m guessing there’s something sub-optimal with bursting at your speeds. How did you decide on WAN overhead of 4?

Thanks for the help! Here goes:

Spoiler: FlexQoS Debug

FlexQoS v1.2.5 released 2021-06-13

Debug date    : 2021-06-25 18:14:11+0100
Router Model  : RT-AC68U
Firmware Ver  : 386.2_6
DPI/Sig Ver   : 2.0.1 / 2.234
WAN iface     : eth0
tc WAN iface  : eth0
IPv6          : dhcp6
Undf Prio     : 2
Down Band     : 486400
Up Band       : 112640
**************
Net Control   : 1:10
Work-From-Home: 1:12
Gaming        : 1:11
Others        : 1:13
Web Surfing   : 1:14
Streaming     : 1:15
File Transfers: 1:17
Learn-From-Home: 1:16
**************
Downrates     :   24320,   72960,   97280,   48640,   48640,  145920,   24320,   24320
Downceils     :  486400,  486400,  486400,  486400,  486400,  486400,  486400,  486400
Uprates       :    5632,   16896,   22528,   33792,   11264,   11264,    5632,    5632
Upceils       :  112640,  112640,  112640,  112640,  112640,  112640,  112640,  112640
**************
iptables settings: <>>udp>>500,4500>>3<>>udp>16384:16415>>>3<>>tcp>>119,563>>5<>>tcp>>80,443>08****>5<>>udp>>19302:19309>>3<>>udp>>3478:3481>000000>3<>>udp>>8801:8810>000000>3<192.168.1.10>>both>32768:60999>>>5
-o br0 -p udp -m multiport --sports 500,4500 -j MARK --set-mark 0x8006ffff/0xc03fffff
-o eth0 -p udp -m multiport --dports 500,4500 -j MARK --set-mark 0x4006ffff/0xc03fffff
-o br0 -p udp -m multiport --sports 500,4500 -j MARK --set-mark 0x8006ffff/0xc03fffff
-o eth0 -p udp -m multiport --dports 500,4500 -j MARK --set-mark 0x4006ffff/0xc03fffff
-o br0 -p udp -m multiport --dports 16384:16415 -j MARK --set-mark 0x8006ffff/0xc03fffff
-o eth0 -p udp -m multiport --sports 16384:16415 -j MARK --set-mark 0x4006ffff/0xc03fffff
-o br0 -p udp -m multiport --dports 16384:16415 -j MARK --set-mark 0x8006ffff/0xc03fffff
-o eth0 -p udp -m multiport --sports 16384:16415 -j MARK --set-mark 0x4006ffff/0xc03fffff
-o br0 -p tcp -m multiport --sports 119,563 -j MARK --set-mark 0x8003ffff/0xc03fffff
-o eth0 -p tcp -m multiport --dports 119,563 -j MARK --set-mark 0x4003ffff/0xc03fffff
-o br0 -p tcp -m multiport --sports 119,563 -j MARK --set-mark 0x8003ffff/0xc03fffff
-o eth0 -p tcp -m multiport --dports 119,563 -j MARK --set-mark 0x4003ffff/0xc03fffff
-o br0 -p tcp -m multiport --sports 80,443 -m mark --mark 0x80080000/0xc03f0000 -j MARK --set-mark 0x8003ffff/0xc03fffff
-o eth0 -p tcp -m multiport --dports 80,443 -m mark --mark 0x40080000/0xc03f0000 -j MARK --set-mark 0x4003ffff/0xc03fffff
-o br0 -p tcp -m multiport --sports 80,443 -m mark --mark 0x80080000/0xc03f0000 -j MARK --set-mark 0x8003ffff/0xc03fffff
-o eth0 -p tcp -m multiport --dports 80,443 -m mark --mark 0x40080000/0xc03f0000 -j MARK --set-mark 0x4003ffff/0xc03fffff
-o br0 -p udp -m multiport --sports 19302:19309 -j MARK --set-mark 0x8006ffff/0xc03fffff
-o eth0 -p udp -m multiport --dports 19302:19309 -j MARK --set-mark 0x4006ffff/0xc03fffff
-o br0 -p udp -m multiport --sports 19302:19309 -j MARK --set-mark 0x8006ffff/0xc03fffff
-o eth0 -p udp -m multiport --dports 19302:19309 -j MARK --set-mark 0x4006ffff/0xc03fffff
-o br0 -p udp -m multiport --sports 3478:3481 -m mark --mark 0x80000000/0xc03fffff -j MARK --set-mark 0x8006ffff/0xc03fffff
-o eth0 -p udp -m multiport --dports 3478:3481 -m mark --mark 0x40000000/0xc03fffff -j MARK --set-mark 0x4006ffff/0xc03fffff
-o br0 -p udp -m multiport --sports 3478:3481 -m mark --mark 0x80000000/0xc03fffff -j MARK --set-mark 0x8006ffff/0xc03fffff
-o eth0 -p udp -m multiport --dports 3478:3481 -m mark --mark 0x40000000/0xc03fffff -j MARK --set-mark 0x4006ffff/0xc03fffff
-o br0 -p udp -m multiport --sports 8801:8810 -m mark --mark 0x80000000/0xc03fffff -j MARK --set-mark 0x8006ffff/0xc03fffff
-o eth0 -p udp -m multiport --dports 8801:8810 -m mark --mark 0x40000000/0xc03fffff -j MARK --set-mark 0x4006ffff/0xc03fffff
-o br0 -p udp -m multiport --sports 8801:8810 -m mark --mark 0x80000000/0xc03fffff -j MARK --set-mark 0x8006ffff/0xc03fffff
-o eth0 -p udp -m multiport --dports 8801:8810 -m mark --mark 0x40000000/0xc03fffff -j MARK --set-mark 0x4006ffff/0xc03fffff
-o br0 -d 192.168.1.10 -p tcp -m multiport --dports 32768:60999 -j MARK --set-mark 0x8003ffff/0xc03fffff
-o eth0 -s 192.168.1.10 -p tcp -m multiport --sports 32768:60999 -j MARK --set-mark 0x4003ffff/0xc03fffff
-o br0 -d 192.168.1.10 -p udp -m multiport --dports 32768:60999 -j MARK --set-mark 0x8003ffff/0xc03fffff
-o eth0 -s 192.168.1.10 -p udp -m multiport --sports 32768:60999 -j MARK --set-mark 0x4003ffff/0xc03fffff
**************

(split in 2, please see next reply as well)

 

[goncalossilva]

Spoiler: FlexQoS Debug

**************
appdb rules: <000000>6<00006B>6<0D0007>5<0D0086>5<0D00A0>5<12003F>4<13****>4<14****>4<1A****>5
filter change dev br0 prio 2 protocol all handle 828::800 u32 flowid 1:13
filter change dev eth0 prio 2 protocol all handle 828::800 u32 flowid 1:13
filter add dev br0 protocol all prio 2 u32 match mark 0x8000006B 0xc03fffff flowid 1:13
filter add dev eth0 protocol all prio 2 u32 match mark 0x4000006B 0xc03fffff flowid 1:13
filter add dev br0 protocol all prio 15 u32 match mark 0x800D0007 0xc03fffff flowid 1:17
filter add dev eth0 protocol all prio 15 u32 match mark 0x400D0007 0xc03fffff flowid 1:17
filter add dev br0 protocol all prio 15 u32 match mark 0x800D0086 0xc03fffff flowid 1:17
filter add dev eth0 protocol all prio 15 u32 match mark 0x400D0086 0xc03fffff flowid 1:17
filter add dev br0 protocol all prio 15 u32 match mark 0x800D00A0 0xc03fffff flowid 1:17
filter add dev eth0 protocol all prio 15 u32 match mark 0x400D00A0 0xc03fffff flowid 1:17
filter add dev br0 protocol all prio 20 u32 match mark 0x8012003F 0xc03fffff flowid 1:14
filter add dev eth0 protocol all prio 20 u32 match mark 0x4012003F 0xc03fffff flowid 1:14
filter change dev br0 prio 22 protocol all handle 802::800 u32 flowid 1:14
filter change dev eth0 prio 22 protocol all handle 802::800 u32 flowid 1:14
filter change dev br0 prio 23 protocol all handle 804::800 u32 flowid 1:14
filter change dev eth0 prio 23 protocol all handle 804::800 u32 flowid 1:14
filter change dev br0 prio 2 protocol all handle 828::802 u32 flowid 1:17
filter change dev eth0 prio 2 protocol all handle 828::802 u32 flowid 1:17
class change dev br0 parent 1:1 classid 1:10 htb overhead 4 linklayer ethernet prio 0 rate 24320Kbit ceil 486400Kbit burst 60800b cburst 608000b quantum 304000
class change dev eth0 parent 1:1 classid 1:10 htb overhead 4 linklayer ethernet prio 0 rate 5632Kbit ceil 112640Kbit burst 14080b cburst 140800b quantum 70400
class change dev br0 parent 1:1 classid 1:11 htb overhead 4 linklayer ethernet prio 1 rate 72960Kbit ceil 486400Kbit burst 60800b cburst 608000b quantum 912000
class change dev eth0 parent 1:1 classid 1:11 htb overhead 4 linklayer ethernet prio 1 rate 16896Kbit ceil 112640Kbit burst 14080b cburst 140800b quantum 211200
class change dev br0 parent 1:1 classid 1:12 htb overhead 4 linklayer ethernet prio 2 rate 97280Kbit ceil 486400Kbit burst 60800b cburst 608000b quantum 1216000
class change dev eth0 parent 1:1 classid 1:12 htb overhead 4 linklayer ethernet prio 2 rate 22528Kbit ceil 112640Kbit burst 14080b cburst 140800b quantum 281600
class change dev br0 parent 1:1 classid 1:13 htb overhead 4 linklayer ethernet prio 3 rate 48640Kbit ceil 486400Kbit burst 60800b cburst 608000b quantum 608000
class change dev eth0 parent 1:1 classid 1:13 htb overhead 4 linklayer ethernet prio 3 rate 33792Kbit ceil 112640Kbit burst 14080b cburst 140800b quantum 422400
class change dev br0 parent 1:1 classid 1:14 htb overhead 4 linklayer ethernet prio 4 rate 48640Kbit ceil 486400Kbit burst 60800b cburst 608000b quantum 608000
class change dev eth0 parent 1:1 classid 1:14 htb overhead 4 linklayer ethernet prio 4 rate 11264Kbit ceil 112640Kbit burst 14080b cburst 140800b quantum 140800
class change dev br0 parent 1:1 classid 1:15 htb overhead 4 linklayer ethernet prio 5 rate 145920Kbit ceil 486400Kbit burst 60800b cburst 608000b quantum 1824000
class change dev eth0 parent 1:1 classid 1:15 htb overhead 4 linklayer ethernet prio 5 rate 11264Kbit ceil 112640Kbit burst 14080b cburst 140800b quantum 140800
class change dev br0 parent 1:1 classid 1:16 htb overhead 4 linklayer ethernet prio 6 rate 24320Kbit ceil 486400Kbit burst 60800b cburst 608000b quantum 304000
class change dev eth0 parent 1:1 classid 1:16 htb overhead 4 linklayer ethernet prio 6 rate 5632Kbit ceil 112640Kbit burst 14080b cburst 140800b quantum 70400
class change dev br0 parent 1:1 classid 1:17 htb overhead 4 linklayer ethernet prio 7 rate 24320Kbit ceil 486400Kbit burst 60800b cburst 608000b quantum 304000
class change dev eth0 parent 1:1 classid 1:17 htb overhead 4 linklayer ethernet prio 7 rate 5632Kbit ceil 112640Kbit burst 14080b cburst 140800b quantum 70400
qdisc replace dev br0 parent 1:2 handle 102: fq_codel limit 1000 noecn
qdisc replace dev eth0 parent 1:2 handle 102: fq_codel limit 1000 noecn
qdisc replace dev br0 parent 1:10 handle 110: fq_codel  limit 1000
qdisc replace dev eth0 parent 1:10 handle 110: fq_codel  limit 1000  noecn
qdisc replace dev br0 parent 1:11 handle 111: fq_codel  limit 1000
qdisc replace dev eth0 parent 1:11 handle 111: fq_codel  limit 1000  noecn
qdisc replace dev br0 parent 1:12 handle 112: fq_codel  limit 1000
qdisc replace dev eth0 parent 1:12 handle 112: fq_codel  limit 1000  noecn
qdisc replace dev br0 parent 1:13 handle 113: fq_codel  limit 1000
qdisc replace dev eth0 parent 1:13 handle 113: fq_codel  limit 1000  noecn
qdisc replace dev br0 parent 1:14 handle 114: fq_codel  limit 1000
qdisc replace dev eth0 parent 1:14 handle 114: fq_codel  limit 1000  noecn
qdisc replace dev br0 parent 1:15 handle 115: fq_codel  limit 1000
qdisc replace dev eth0 parent 1:15 handle 115: fq_codel  limit 1000  noecn
qdisc replace dev br0 parent 1:16 handle 116: fq_codel  limit 1000
qdisc replace dev eth0 parent 1:16 handle 116: fq_codel  limit 1000  noecn
qdisc replace dev br0 parent 1:17 handle 117: fq_codel  limit 1000
qdisc replace dev eth0 parent 1:17 handle 117: fq_codel  limit 1000  noecn

As for WAN overhead, this is admitedly a bit of an unknown for me. I used 4 because there was an old discussion here mentioning that would be the correct value for a regular switch (i.e., ethernet/vlan). I have FTTH, but I use my ISP's router in bridge mode, so I assumed that was a fair assumption. I forgot to mention it before, but I did try an overhead of 0, but there was no change. Should I try something else?

 

[dave14305]

I can’t think of a reason yet why there would be such a performance difference between stock Adaptive QoS and FlexQoS. It’s like HW acceleration is disabled.

Perhaps the output of these commands will show something misconfigured. Please run them once with FlexQoS enabled and once with it disabled.

tc -d qdisc
tc -d class show dev br0 parent 1:
tc -d class show dev eth0 parent 1:

 

[Michael Schäfer]

I'm having the same problem on my RT-AC68U on Merlin 386.2_6 and FlexQoS v1.2.5.

My connection is 500/100. Some scenarious:
- No QoS, speedtest shows 510/120
- Adaptive QoS + FlexQoS, speedtest shows 180/110 w/ CPU pegged at 100%
- Adaptive QoS only (i.e., FlexQoS disabled), speedtest shows 450/110 w/ CPU at 75%

To note, Hardware acceleration is "Enabled" in Tools > Sysinfo, and NAT Acceleration is "Auto" + "CTF (Cut Through Forwarding) is enabled." in LAN > Switch Control. No traffic analysis or "ai features" are enabled.

I've changed a few settings, like:

• Clearing the list of rules, no change
• Using ASUS instead of fq_codel, speedtest drops further to ~165/110
• Disabling NAT Acceleration entirely, no change

So it seems that disabling FlexQoS is the only way to get things working as intended... except I want to use FlexQoS. Any advice?

Did you use the same configuration with an earlier firmware successfully? I'll probably try a firmware reset when I've got some time on my hands, it is the only idea so far that may change something.

 

[goncalossilva]

I can’t think of a reason yet why there would be such a performance difference between stock Adaptive QoS and FlexQoS. It’s like HW acceleration is disabled.

Perhaps the output of these commands will show something misconfigured. Please run them once with FlexQoS enabled and once with it disabled.

tc -d qdisc
tc -d class show dev br0 parent 1:
tc -d class show dev eth0 parent 1:

Sure!

• With Adaptive QoS on and FlexQoS enabled: https://gist.github.com/goncalossilva/19cebae77c2cfc479b426d45960a1458#file-tc-on-enabled-txt
• With Adaptive QoS on and FlexQoS disabled: https://gist.github.com/goncalossilva/19cebae77c2cfc479b426d45960a1458#file-tc-on-disabled-txt
• With Adaptive QoS off: https://gist.github.com/goncalossilva/19cebae77c2cfc479b426d45960a1458#file-tc-off-txt

Did you use the same configuration with an earlier firmware successfully? I'll probably try a firmware reset when I've got some time on my hands, it is the only idea so far that may change something.

Indeed... Although I haven't been monitoring for the past few months due to a slow contract renegotiation. Last time I checked was probably around November/October 2020. I am unsure when it broke (I kept updating).

 

[joesixpack]

I'm having the same problem on my RT-AC68U on Merlin 386.2_6 and FlexQoS v1.2.5.

Same. Something's rotten in FlexQoS. Have you tried earlier FlexQoS versions?

 

[dave14305]

Sure!

• With Adaptive QoS on and FlexQoS enabled: https://gist.github.com/goncalossilva/19cebae77c2cfc479b426d45960a1458#file-tc-on-enabled-txt
• With Adaptive QoS on and FlexQoS disabled: https://gist.github.com/goncalossilva/19cebae77c2cfc479b426d45960a1458#file-tc-on-disabled-txt
• With Adaptive QoS off: https://gist.github.com/goncalossilva/19cebae77c2cfc479b426d45960a1458#file-tc-off-txt

Indeed... Although I haven't been monitoring for the past few months due to a slow contract renegotiation. Last time I checked was probably around November/October 2020. I am unsure when it broke (I kept updating).

I setup my old AC68U and installed 386.2_6 and FlexQoS 1.2.5 and ran some speedtests. It had been a long time since I used the AC68U, and I forgot how CPU-bound it can be. Where were you running your speedtests? On a LAN client (PC, iPad, etc.) or using the router's Internet Speed page? Running the speedtest from the router will impact the results because some of the CPU capacity must be given to the speedtest program instead of routing the speedtest traffic.

When I was using the router's Internet Speed page to run the tests, my results were:

• Adaptive QoS off (Trend Micro consent withdrawn, router rebooted, WAN interface is vlan2): 230 Mbps down
• Adaptive QoS on (FlexQoS disabled): 191 Mbps down
• Adaptive QoS on (FlexQoS enabled fq_codel): 176 Mbps down

When I was using an iPad connected via 5Ghz using the Speedtest app to run the tests, my results were:

• Adaptive QoS off (Trend Micro consent withdrawn, router rebooted, WAN interface is vlan2): 384 Mbps down
• Adaptive QoS on (FlexQoS disabled): 346 Mbps down
• Adaptive QoS on (FlexQoS enabled fq_codel): 343 Mbps down

Scenario	Router Internet Speed page	Speedtest App on iPad (5Ghz WiFi)
No QoS (Trend Micro disabled)	230 Mbps	384 Mbps
Adaptive QoS w/o FlexQoS	191 Mbps	346 Mbps
Adaptive QoS w/ FlexQoS (fq_codel)	176 Mbps	343 Mbps

In the QoS scenarios, the download bandwidth was configured for 440 Mbps, which the AC68U didn't seem to be able to achieve in my test scenario, but it is setup behind my AC86U main router, so not a perfectly isolated test. I only ran most tests once because it's getting late and I'm tired.

When running top over SSH, I run top -d 1 to refresh the display every second, and I hit the number 1 key within top to display the CPU stats for each individual CPU. When using the router speedtest, you can observe all or almost all of CPU 0 being consumed during the test.

What I take away from these tests is that there is no notable difference using FlexQoS or just Adaptive QoS when a LAN client runs the speedtest.

So back to my question of where were you running the tests you performed?

Something's rotten in FlexQoS.

Maybe. Maybe not. Hard to tell from your detailed feedback.

 

[slidermike]

Not to derail the thread but man, Dave thats some support. Dusting off an old router just to assist a couple of folks in the forums. Now that is some service.
Kudos to you sir.

 

[dave14305]

Not to derail the thread but man, Dave thats some support. Dusting off an old router just to assist a couple of folks in the forums. Now that is some service.
Kudos to you sir.

I asked myself WWJD? (What Would Jack Do?)

 

[goncalossilva]

I setup my old AC68U and installed 386.2_6 and FlexQoS 1.2.5 and ran some speedtests. It had been a long time since I used the AC68U, and I forgot how CPU-bound it can be. Where were you running your speedtests? On a LAN client (PC, iPad, etc.) or using the router's Internet Speed page? Running the speedtest from the router will impact the results because some of the CPU capacity must be given to the speedtest program instead of routing the speedtest traffic.

When I was using the router's Internet Speed page to run the tests, my results were:

• Adaptive QoS off (Trend Micro consent withdrawn, router rebooted, WAN interface is vlan2): 230 Mbps down
• Adaptive QoS on (FlexQoS disabled): 191 Mbps down
• Adaptive QoS on (FlexQoS enabled fq_codel): 176 Mbps down

When I was using an iPad connected via 5Ghz using the Speedtest app to run the tests, my results were:

• Adaptive QoS off (Trend Micro consent withdrawn, router rebooted, WAN interface is vlan2): 384 Mbps down
• Adaptive QoS on (FlexQoS disabled): 346 Mbps down
• Adaptive QoS on (FlexQoS enabled fq_codel): 343 Mbps down

Scenario	Router Internet Speed page	Speedtest App on iPad (5Ghz WiFi)
No QoS (Trend Micro disabled)	230 Mbps	384 Mbps
Adaptive QoS w/o FlexQoS	191 Mbps	346 Mbps
Adaptive QoS w/ FlexQoS (fq_codel)	176 Mbps	343 Mbps

In the QoS scenarios, the download bandwidth was configured for 440 Mbps, which the AC68U didn't seem to be able to achieve in my test scenario, but it is setup behind my AC86U main router, so not a perfectly isolated test. I only ran most tests once because it's getting late and I'm tired.

When running top over SSH, I run top -d 1 to refresh the display every second, and I hit the number 1 key within top to display the CPU stats for each individual CPU. When using the router speedtest, you can observe all or almost all of CPU 0 being consumed during the test.

What I take away from these tests is that there is no notable difference using FlexQoS or just Adaptive QoS when a LAN client runs the speedtest.

So back to my question of where were you running the tests you performed?

Maybe. Maybe not. Hard to tell from your detailed feedback.

Thank you so much for going through all of this effort. My RT-AC68U is overclocked to 1200/800, so it typically has more headroom than stock. As for how I run the tests, it's typically using a wired computer on the network, using http://dslreports.com/speedtest (though speedtest.net, fast.com, et al, are consistent).

I've investigated a little more in the last couple of hours, going through earlier FlexQoS versions and different configurations, and I think I have narrowed down the problem to an iptables rule I had for a specific IP in the network, but with a wide port range (32768:60999). Removing that rule in the FlexQoS UI unlocks full speed again. Curiously, I can't reproduce this problem with FreshJR, but with it, I used to set the rule manually like so:

iptables -D POSTROUTING -t mangle -o br0 -d 192.168.1.10 --dport 32768:60999 -j MARK --set-mark ${Downloads_mark_down} &> /dev/null
iptables -A POSTROUTING -t mangle -o br0 -d 192.168.1.10 --dport 32768:60999 -j MARK --set-mark ${Downloads_mark_down}

(...)

iptables -D POSTROUTING -t mangle -o $wan -s 192.168.1.10 --sport 32768:60999 -j MARK --set-mark ${Downloads_mark_up} &> /dev/null
iptables -A POSTROUTING -t mangle -o $wan -s 192.168.1.10 --sport 32768:60999 -j MARK --set-mark ${Downloads_mark_up}

But it's been a long time since we were writing these by hand, and I assume FlexQoS is doing more work behind the scenes. That rule is not that important in my setup anymore, so I've removed it which ultimately addresses the problem. I'm back to 450/110 with QoS set to limit bandwidth (or 510/120 without, which is the maximum my connection will do). I'd recommend that others facing similar issues check their custom rules... There might be an isolated cause like in my case. Thanks again, Dave!

 

[dave14305]

I've investigated a little more in the last couple of hours, going through earlier FlexQoS versions and different configurations, and I think I have narrowed down the problem to an iptables rule I had for a specific IP in the network, but with a wide port range (32768:60999). Removing that rule in the FlexQoS UI unlocks full speed again. Curiously, I can't reproduce this problem with FreshJR, but with it, I used to set the rule manually like so:

iptables -D POSTROUTING -t mangle -o br0 -d 192.168.1.10 --dport 32768:60999 -j MARK --set-mark ${Downloads_mark_down} &> /dev/null
iptables -A POSTROUTING -t mangle -o br0 -d 192.168.1.10 --dport 32768:60999 -j MARK --set-mark ${Downloads_mark_down}

(...)

iptables -D POSTROUTING -t mangle -o $wan -s 192.168.1.10 --sport 32768:60999 -j MARK --set-mark ${Downloads_mark_up} &> /dev/null
iptables -A POSTROUTING -t mangle -o $wan -s 192.168.1.10 --sport 32768:60999 -j MARK --set-mark ${Downloads_mark_up}

But it's been a long time since we were writing these by hand, and I assume FlexQoS is doing more work behind the scenes. That rule is not that important in my setup anymore, so I've removed it which ultimately addresses the problem. I'm back to 450/110 with QoS set to limit bandwidth (or 510/120 without, which is the maximum my connection will do). I'd recommend that others facing similar issues check their custom rules... There might be an isolated cause like in my case. Thanks again, Dave!

It looks like in v1.1.0 I switched all ports to use the multiport iptables syntax. That was in December. Interesting find though, so may be worth further testing.