FlexQoS FlexQoS 1.0 - Flexible QoS Enhancement Script for Adaptive QoS

[dave14305]

You hit the nail … This is the reason, thanks.

My first thought is to always blame @Jack Yaz.

 

[Jack Yaz]

My first thought is to always blame @Jack Yaz.

does restarting qos drop connections because of a flushed conntrack that got implemented a while back? ;-)

 

[dave14305]

does restarting qos drop connections because of a flushed conntrack that got implemented a while back? ;-)

Let’s stick to the evidence that paints me in the best light, please.

I did force conntrack flushing in v1.0.6 because I hadn’t heard of anyone wanting it disabled anymore. I could bring back the option if it avoids this scenario.

 

[Jack Yaz]

Let’s stick to the evidence that paints me in the best light, please.

I did force conntrack flushing in v1.0.6 because I hadn’t heard of anyone wanting it disabled anymore. I could bring back the option if it avoids this scenario.

haha, i kid. i might add an option to run the autobw stuff every X tests for the last X results. at the moment it's every run and uses the last 10. that would allow users of autobw to not get interrupted every 30/60mins.

 

[NGI]

haha, i kid. i might add an option to run the autobw stuff every X tests for the last X results. at the moment it's every run and uses the last 10. that would allow users of autobw to not get interrupted every 30/60mins.

Another option perhaps could be (if possible/easy to implement) only to update the values of Up/Down bandwidth in NVRAM as a result of AutoBW in speedtest only if they are outside some tolerance level +/- X% compared to the existing values?

 

[smithy]

If I want to prioritise google stadia streaming, do i need to add a custom rule in the iptables or can I just put "Web surfing" which it seems to be catergorised as looking at live traffic, higher up the order? Gaming, web surfing, streaming etc . I am Running 1.0.6.

 

[dave14305]

haha, i kid. i might add an option to run the autobw stuff every X tests for the last X results. at the moment it's every run and uses the last 10. that would allow users of autobw to not get interrupted every 30/60mins.

In the meantime, I'm pushing a hotfix to reinstate the noflushct CLI option if it helps spdMerlin users. By default, flushing is enabled, so to disable it, run flexqos noflushct

 

[marko]

It says that FlexQOS is tested with adaptive and manual qos. But now, since the latest merlin release (386.1 alpha) it seems that adaptive qos and traditional qos are kind of switched. There is no option to choose fq_codel or packet overhead in adaptive qos, but in traditional qos. Does this mean that we should also switch to traditional qos or to stay on adaptive if we want to get full benefit of flexqos?

 

[NGI]

In the meantime, I'm pushing a hotfix to reinstate the noflushct CLI option if it helps spdMerlin users. By default, flushing is enabled, so to disable it, run flexqos noflushct

Great stuff, I can confirm that after flexqos noflushct my audio streams do not drop after updating the QoS values via spdMerlin AutoBW.

 

[dave14305]

It says that FlexQOS is tested with adaptive and manual qos. But now, since the latest merlin release (386.1 alpha) it seems that adaptive qos and traditional qos are kind of switched. There is no option to choose fq_codel or packet overhead in adaptive qos, but in traditional qos. Does this mean that we should also switch to traditional qos or to stay on adaptive if we want to get full benefit of flexqos?

fq_codel is not available with Adaptive QoS in the 386 releases due to ASUS/Trend Micro changes, but FlexQoS should continue to work fine with Adaptive QoS (FlexQoS is not compatible in any way with Traditional QoS). Adaptive QoS on 386 will only use sfq (which I've been using anyway on 384.19 for a few weeks now).

 

[marko]

fq_codel is not available with Adaptive QoS in the 386 releases due to ASUS/Trend Micro changes, but FlexQoS should continue to work fine with Adaptive QoS (FlexQoS is not compatible in any way with Traditional QoS). Adaptive QoS on 386 will only use sfq (which I've been using anyway on 384.19 for a few weeks now).

Yeah, but differences (theoretical at least) are evident between sfq and fq_codel specially for gamers. And as you said, you are not gaming and therefore packets transferring does not play much for you, i guess

 

[NGI]

Hi @dave14305, I think something broke after the last Hotfix. I can't see anymore any data in pie charts associated with Other Class despite having traffic using Other Class (see attached). It happens with flushed conntrack disabled or enabled.

Here is a debug:

Spoiler: Debug

FlexQoS v1.0.6 released 2020-11-23

Debug date: 2020-11-23 15:44:00+0000
Router Model: RT-AC86U
Firmware Ver: 384.19_0
WAN iface: ppp0
tc WAN iface: eth0
Undf Prio: 2
Down Band: 26620
Up Band : 5297
***********
Net Control: 1:10
Work-From-Home: 1:13
Gaming: 1:15
Others: 1:11
Web Surfing: 1:12
Streaming: 1:14
File Downloads: 1:16
Game Downloads: 1:17
***********
Downrates: 1331, 7986, 2662, 3993, 6655, 2395, 1331, 266
Downceils: 26620, 26620, 26620, 26620, 26620, 26620, 26620, 26620
Downbursts: 3200, 9600, 3200, 4800, 8000, 3200, 3200, 3200
DownCbursts: 32000, 32000, 32000, 32000, 32000, 32000, 32000, 32000
DownQuantums: 16637, 99825, 33275, 49912, 83187, 29937, 16637, 3325
***********
Uprates: 264, 1589, 529, 794, 1324, 476, 264, 52
Upceils: 5297, 5297, 5297, 5297, 5297, 5297, 5297, 5297
Upbursts: 3200, 3200, 3200, 3200, 3200, 3200, 3200, 3200
UpCbursts: 6400, 6400, 6400, 6400, 6400, 6400, 6400, 6400
UpQuantums: 3300, 19862, 6612, 9925, 16550, 5950, 3300, 1514
***********
iptables settings: <>>udp>>500,4500>>3<>>udp>16384:16415>>>3<>>tcp>>119,563>>5<192.168.1.66>>tcp>>>030005>6<>>udp>>3478:3481>000000>3<>>udp>>8801:8810>000000>3<192.168.1.66>>tcp>>>040050>6<192.168.1.66>>tcp>>>1400C2>6
-o br0 -p udp -m multiport --sports 500,4500 -j MARK --set-mark 0x8006ffff/0x3fffff
-o ppp0 -p udp -m multiport --dports 500,4500 -j MARK --set-mark 0x4006ffff/0x3fffff
-o br0 -p udp --dport 16384:16415 -j MARK --set-mark 0x8006ffff/0x3fffff
-o ppp0 -p udp --sport 16384:16415 -j MARK --set-mark 0x4006ffff/0x3fffff
-o br0 -p tcp -m multiport --sports 119,563 -j MARK --set-mark 0x8003ffff/0x3fffff
-o ppp0 -p tcp -m multiport --dports 119,563 -j MARK --set-mark 0x4003ffff/0x3fffff
-o br0 -d 192.168.1.66 -p tcp -m mark --mark 0x80030005/0xc03fffff -j MARK --set-mark 0x800affff/0x3fffff
-o ppp0 -s 192.168.1.66 -p tcp -m mark --mark 0x40030005/0xc03fffff -j MARK --set-mark 0x400affff/0x3fffff
-o br0 -p udp --sport 3478:3481 -m mark --mark 0x80000000/0xc03fffff -j MARK --set-mark 0x8006ffff/0x3fffff
-o ppp0 -p udp --dport 3478:3481 -m mark --mark 0x40000000/0xc03fffff -j MARK --set-mark 0x4006ffff/0x3fffff
-o br0 -p udp --sport 8801:8810 -m mark --mark 0x80000000/0xc03fffff -j MARK --set-mark 0x8006ffff/0x3fffff
-o ppp0 -p udp --dport 8801:8810 -m mark --mark 0x40000000/0xc03fffff -j MARK --set-mark 0x4006ffff/0x3fffff
-o br0 -d 192.168.1.66 -p tcp -m mark --mark 0x80040050/0xc03fffff -j MARK --set-mark 0x800affff/0x3fffff
-o ppp0 -s 192.168.1.66 -p tcp -m mark --mark 0x40040050/0xc03fffff -j MARK --set-mark 0x400affff/0x3fffff
-o br0 -d 192.168.1.66 -p tcp -m mark --mark 0x801400C2/0xc03fffff -j MARK --set-mark 0x800affff/0x3fffff
-o ppp0 -s 192.168.1.66 -p tcp -m mark --mark 0x401400C2/0xc03fffff -j MARK --set-mark 0x400affff/0x3fffff
***********
appdb rules: <0D0007>5<0D0086>5<0D00A0>5<12003F>4<00006B>5<0400F8>5<04000A>5<0D0017>2<000029>5<180003>5<0B0041>7<170005>5<000000>7<13****>4<14****>4<1A****>5
filter add dev br0 protocol all prio 15 u32 match mark 0x800D0007 0xc03fffff flowid 1:16
filter add dev eth0 protocol all prio 15 u32 match mark 0x400D0007 0xc03fffff flowid 1:16
filter add dev br0 protocol all prio 15 u32 match mark 0x800D0086 0xc03fffff flowid 1:16
filter add dev eth0 protocol all prio 15 u32 match mark 0x400D0086 0xc03fffff flowid 1:16
filter add dev br0 protocol all prio 15 u32 match mark 0x800D00A0 0xc03fffff flowid 1:16
filter add dev eth0 protocol all prio 15 u32 match mark 0x400D00A0 0xc03fffff flowid 1:16
filter add dev br0 protocol all prio 20 u32 match mark 0x8012003F 0xc03fffff flowid 1:12
filter add dev eth0 protocol all prio 20 u32 match mark 0x4012003F 0xc03fffff flowid 1:12
filter add dev br0 protocol all prio 2 u32 match mark 0x8000006B 0xc03fffff flowid 1:16
filter add dev eth0 protocol all prio 2 u32 match mark 0x4000006B 0xc03fffff flowid 1:16
filter add dev br0 protocol all prio 6 u32 match mark 0x800400F8 0xc03fffff flowid 1:16
filter add dev eth0 protocol all prio 6 u32 match mark 0x400400F8 0xc03fffff flowid 1:16
filter add dev br0 protocol all prio 6 u32 match mark 0x8004000A 0xc03fffff flowid 1:16
filter add dev eth0 protocol all prio 6 u32 match mark 0x4004000A 0xc03fffff flowid 1:16
filter add dev br0 protocol all prio 15 u32 match mark 0x800D0017 0xc03fffff flowid 1:14
filter add dev eth0 protocol all prio 15 u32 match mark 0x400D0017 0xc03fffff flowid 1:14
filter add dev br0 protocol all prio 2 u32 match mark 0x80000029 0xc03fffff flowid 1:16
filter add dev eth0 protocol all prio 2 u32 match mark 0x40000029 0xc03fffff flowid 1:16
filter add dev br0 protocol all prio 26 u32 match mark 0x80180003 0xc03fffff flowid 1:16
filter add dev eth0 protocol all prio 26 u32 match mark 0x40180003 0xc03fffff flowid 1:16
filter add dev br0 protocol all prio 13 u32 match mark 0x800B0041 0xc03fffff flowid 1:17
filter add dev eth0 protocol all prio 13 u32 match mark 0x400B0041 0xc03fffff flowid 1:17
filter add dev br0 protocol all prio 25 u32 match mark 0x80170005 0xc03fffff flowid 1:16
filter add dev eth0 protocol all prio 25 u32 match mark 0x40170005 0xc03fffff flowid 1:16
filter change dev br0 prio 2 protocol all handle 828::802 u32 flowid 1:17
filter change dev eth0 prio 2 protocol all handle 828::802 u32 flowid 1:17
filter change dev br0 prio 22 protocol all handle 802::800 u32 flowid 1:12
filter change dev eth0 prio 22 protocol all handle 802::800 u32 flowid 1:12
filter change dev br0 prio 23 protocol all handle 804::800 u32 flowid 1:12
filter change dev eth0 prio 23 protocol all handle 804::800 u32 flowid 1:12
filter change dev br0 prio 2 protocol all handle 828::803 u32 flowid 1:16
filter change dev eth0 prio 2 protocol all handle 828::803 u32 flowid 1:16

 

[dave14305]

Yeah, but differences (theoretical at least) are evident between sfq and fq_codel specially for gamers. And as you said, you are not gaming and therefore packets transferring does not play much for you, i guess

In Adaptive QoS, both the fq_codel and sfq qdiscs end up only scheduling packets from a single category (e.g. Gaming) and for a single device (e.g. PS4). There is no mixing of device traffic in Adaptive QoS qdiscs, which is why I don't think it matters too much which one you use. If your PS4 is competing with itself for traffic within the Gaming category, then maybe more fairness is needed, but it's not available anymore in 386.

 

[dave14305]

Hi @dave14305, I think something broke after the last Hotfix. I can't see anymore any data in pie charts associated with Other Class despite having traffic using Other Class (see attached). It happens with flushed conntrack disabled or enabled.

Here is a debug:

The only thing that changed in the hotfix is the ability to enable/disable the flushing. You also need to restart qos for the flushing setting to take effect on existing connections. Check if your iptables rules for Others traffic is being hit.

iptables -t mangle -nvL POSTROUTING

 

[andresmorago]

hello
when filtering connections by a specific device (lets say 10.0.0.2), is it normal to see on the list other devices which their ip address also contains 10.0.0.2 (ie. 10.0.0.22) ?
Im trying to filter by DiskStation (which has 10.0.0.2) and my sonos devices will also appear (their ip address segment is 10.0.0.20 - 23)

thanks

 

[NGI]

The only thing that changed in the hotfix is the ability to enable/disable the flushing. You also need to restart qos for the flushing setting to take effect on existing connections. Check if your iptables rules for Others traffic is being hit.

iptables -t mangle -nvL POSTROUTING

Ahh that was it, I didn't restart the active connection. Now it works after I restarted the audio stream. Many thanks for that Dave.

 

[dave14305]

hello
when filtering connections by a specific device (lets say 10.0.0.2), is it normal to see on the list other devices which their ip address also contains 10.0.0.2 (ie. 10.0.0.22) ?
Im trying to filter by DiskStation (which has 10.0.0.2) and my sonos devices will also appear (their ip address segment is 10.0.0.20 - 23)

thanks

Yes, it's a known deficiency at the moment. I need to implement regular expression matching on that field so that if it's picked from the menu, I can add an EOL $ character to avoid the extra matches.

 

[Kingp1n]

In Adaptive QoS, both the fq_codel and sfq qdiscs end up only scheduling packets from a single category (e.g. Gaming) and for a single device (e.g. PS4). There is no mixing of device traffic in Adaptive QoS qdiscs, which is why I don't think it matters too much which one you use. If your PS4 is competing with itself for traffic within the Gaming category, then maybe more fairness is needed, but it's not available anymore in 386.

Any gamers here testing 386 alpha with sfq? I read in one of ddwrt forums that sfq is older technology but not recommended for wifi due to latency. Fq-codel was better with managing traffic and latency compared to sfq. @RMerlin are plans in the future to re-implement fq-codel or this no longer will be included in future firmwares. i wonder if sfq has gotten updates thru the years. Although aimesh 2.0 looks really nice, from a gamers point of view, it seems we're also getting a downgrade inside QoS from fq-codel to sfq only. Again I've never used sfq due to always running the rmerlin fw. Hope it's not as bad as people say in other forums when it comes bufferbloat. Thanks for all you do!

 

[marko]

In Adaptive QoS, both the fq_codel and sfq qdiscs end up only scheduling packets from a single category (e.g. Gaming) and for a single device (e.g. PS4). There is no mixing of device traffic in Adaptive QoS qdiscs, which is why I don't think it matters too much which one you use. If your PS4 is competing with itself for traffic within the Gaming category, then maybe more fairness is needed, but it's not available anymore in 386.

Alright, that is regarding queue discipline.

But there is also one setting "WAN packet overhead" that definitely affects gaming experience. And that is forbidden now for us.

One note, I do not know if i have already read somewhere, is now 0 default?

 

[dave14305]

Any gamers here testing 386 alpha with sfq? I read in one of ddwrt forums that sfq is older technology but not recommended for wifi due to latency. Fq-codel was better with managing traffic and latency compared to sfq. @RMerlin are plans in the future to re-implement fq-codel or this no longer will be included in future firmwares. i wonder if sfq has gotten updates thru the years. Although aimesh 2.0 looks really nice, from a gamers point of view, it seems we're also getting a downgrade inside QoS from fq-codel to sfq only. Again I've never used sfq due to always running the rmerlin fw. Hope it's not as bad as people say in other forums when it comes bufferbloat. Thanks for all you do!

Use caution when comparing QoS implementations in different firmware. I don't think any other firmware has a QoS structure as complicated and segmented as ASUS' Adaptive QoS. Much like ASUS' Traditional QoS, most other firmware implementations probably rely on a priority-based class hierarchy (Highest, High, Med, Low, Lowest, etc.) and then a single qdisc underneath each priority class managing flows from all the devices on the network. The qdisc is very important in that scenario due to competing traffic.

In Adaptive QoS, the structure is 3-level (visible by running tc -g class show dev br0):

1. Category hierarchy (e.g. Net Control, Gaming, Work-From-Home, Gaming, etc.) based on the appdb mark.
2. Device hierarchy (one class per device on the network) based on additional device identifier bits in the full mark.
3. leaf qdisc per category/device combination (sfq, fq_codel, codel).

There is no contention between devices when packets reach the leaf qdisc, since other devices each have their own distinct qdisc separate from every other device. It's the social distancing equivalent for QoS.