FlexQoS FlexQoS 1.0 - Flexible QoS Enhancement Script for Adaptive QoS

[RMerlin]

Just the unsensible '1>' would be confusing to look at.

Use something more visual, like maybe:

"Cat:1"

Then at sort time, replace "Cat:1" with "1>". Just need to ensure that this processing is as simple as possible, so not to slow things down too badly.

 

[JGrana]

Most of my rules are focused on Work from home traffic. I had to re-add the Zoom rule when I saw some Zoom traffic still being classified as Untracked. The CF Warp rule is to de-emphasize my kids' traffic when they hide their activities behind the 1.1.1.1 app (Gaming is 6th priority in my list above File Downloads and Game Transferring).

View attachment 26901

Since I'm also experimenting with my Raspberry Pi again with Pi-Hole, I am also enjoying the benefits of my new inverted filter. Pi-Hole generates a lot of connection noise in the connection list and I can put a !DNS in the filter to hide them all.

I agree that the new inverted filter is great. One of my Pi's is a part of ntp.pool.org. Needless to say, I am swamped with tiny UDP Port 123 messages.

As a side, I have switched yet again from cake-qos to FlexQos. In looking at your TV Streaming rule you use !04****. Isn't that the mark for Streaming?

I wanted to setup similar rules to give Streaming priority for my Apple TV and NVidia Shield.

 

[dave14305]

As a side, I have switched yet again from cake-qos to FlexQos. In looking at your TV Streaming rule you use !04****. Isn't that the mark for Streaming?

Yes, so any traffic from my TV that is NOT 04**** will get marked as Streaming. I did this deliberately to test the inverted Mark feature, and to avoid mangling packets from my TV that are already classified as Streaming.

 

[dave14305]

Use something more visual, like maybe:

"Cat:1"

Then at sort time, replace "Cat:1" with "1>". Just need to ensure that this processing is as simple as possible, so not to slow things down too badly.

Done, but I chose to use "Class:" since a class can contain multiple QoS categories.

And I can still invert it by prefixing the text with !

Enough fun for one day. Time to test and release by Sunday. Thanks for the input everyone!

 

[JGrana]

Yes, so any traffic from my TV that is NOT 04**** will get marked as Streaming. I did this deliberately to test the inverted Mark feature, and to avoid mangling packets from my TV that are already classified as Streaming.

Thanks, now it makes sense! Very nice (and good luck with testing and release ;-)

 

[New2This]

I just might have to fire back up the RPI 4 myself...try this out

LMAO...How things get delivered in my area...........

https://imgur.com/FrXnpt6

 

[Treadler]

Done, but I chose to use "Class:" since a class can contain multiple QoS categories.

And I can still invert it by prefixing the text with !

View attachment 26924View attachment 26925

Enough fun for one day. Time to test and release by Sunday. Thanks for the input everyone!

Iptables rules doesn’t allow input of an IPv6 address?

 

[dave14305]

Iptables rules doesn’t allow input of an IPv6 address?

Correct, not at this time. Rules based only on ports or marks will also be applied to IPv6 traffic, but chasing IPv6 IPs is a problem due to the common scenario of devices changing IPv6 addresses frequently as a privacy measure.

I also stopped using IPv6 myself when I realized IPv6 upload traffic was not being processed by QoS in the latest firmware. So kind of a stalemate for now.

 

[Treadler]

Correct, not at this time. Rules based only on ports or marks will also be applied to IPv6 traffic, but chasing IPv6 IPs is a problem due to the common scenario of devices changing IPv6 addresses frequently as a privacy measure.

I also stopped using IPv6 myself when I realized IPv6 upload traffic was not being processed by QoS in the latest firmware. So kind of a stalemate for now.

Wow, so many variables - herding cats comes to mind!

 

[Kanji-San]

Hi Dave,
according to your release notes: Add DoT port 853 to router traffic exclusion hard-coded rule.
However I see my 853 (mark 1400C5) traffic being redirected to Web Surfing by the AppDB redirection rule:

I thought "hard-coded" referred to this not being applied?

 

[dave14305]

Hi Dave,
according to your release notes: Add DoT port 853 to router traffic exclusion hard-coded rule.
However I see my 853 (mark 1400C5) traffic being redirected to Web Surfing by the AppDB redirection rule:
View attachment 26941

I thought "hard-coded" referred to this not being applied?

Traffic originating from the router itself doesn't show up in the Tracked Connections list. If you are running Stubby in the firmware, that hardcoded iptables rule will prevent outgoing DNS, NTP and DoT traffic from being marked as Downloads, but you never see the impact on the FlexQoS page.

But if you're running DoT on a LAN device or a Pi-Hole, it will be passing through the router and get tracked and classified correctly. But since all 14**** traffic is being directed to Web Surfing, it's necessary to add an AppDB rule for 1400c5 to move DoT back into Net Control (where it was originally).

I don't understand why HTTPS/SSL traffic hasn't been moved out of Net Control by Trend Micro yet. Seems like such an obvious flaw. Truthfully, if they did that, I probably wouldn't even run this script on my own router. I'd just use the stock A.QoS. Alas, we aren't there yet.

 

[Kanji-San]

Traffic originating from the router itself doesn't show up in the Tracked Connections list. If you are running Stubby in the firmware, that hardcoded iptables rule will prevent outgoing DNS, NTP and DoT traffic from being marked as Downloads, but you never see the impact on the FlexQoS page.

But if you're running DoT on a LAN device or a Pi-Hole, it will be passing through the router and get tracked and classified correctly. But since all 14**** traffic is being directed to Web Surfing, it's necessary to add an AppDB rule for 1400c5 to move DoT back into Net Control (where it was originally).
View attachment 26948
I don't understand why HTTPS/SSL traffic hasn't been moved out of Net Control by Trend Micro yet. Seems like such an obvious flaw. Truthfully, if they did that, I probably wouldn't even run this script on my own router. I'd just use the stock A.QoS. Alas, we aren't there yet.

Thank you for the explanation. Now this makes sense

 

[dave14305]

Version 1.0.4 (18-Oct-2020)

NEW:

• Added WebUI-based Check for Update feature.
• Added dropdown menu for Well-Known rules (Default rules plus Skype/Teams and Zoom so far). Gaming rule defaults to Local IP of the router UI session.
• Allow iptables rules to use an inverted match for Mark (e.g. !1400C5, !04**** to match traffic not matching that Mark).
• Allow inverted filters for Tracked Connections using exclamation point (!). !443, !YouTube, !9.9.9.9, etc.
• Added Class dropdown menu on Application filter for Tracked Connections. (e.g. Net Control = Class:0). Based on user-selected priority order. Filter can also be inverted with ! after menu choice is made (e.g. !Class:0).

CHANGED:

• Converted Local IP filter menu to free-text with dropdown based on Network Map data
• Brought back FreshJR_QOS feature to indicate AppDB redirection labels with a tilde ~. Only applies to specific marks (i.e. not wildcard marks nor Untracked mark). You can use tilde ~ in the Application name filter field to find AppDB redirected connections.
• Redirected startup tc errors to file instead of dumping in syslog.log

FIXED:

• Now we filter Tracked Connections by Application Name AFTER rules logic applied (in case of label change). For best page performance, filter on any other field first to limit the number of connections processed by the rules logic.
• Ignore Rule Name when detecting duplicate iptables rules (only evaluate the IP/port/proto/Mark fields)
• Cleanup temp files created during FlexQoS start

KNOWN ISSUES:

• Local IP filter will now do partial matches, so if you select a device with IP 192.168.1.10, it will also include 192.168.1.100, .102, etc. Hope to fix soon with regular expressions.
• Alignment of filter dropdowns is not optimal on iOS/iPadOS Safari

Update with flexqos update or via amtm check for update. Next time, you can use the WebUI.

 

[chris.at]

Is there a way to either auto-update or get a hint in the gui if a new version is available?

 

[dave14305]

Is there a way to either auto-update or get a hint in the gui if a new version is available?

I'm not in a rush to initiate an update check without user interaction. Maybe someday, once I'm sure the update code is reliable. Auto-update is not in my plans, since I don't want to automatically break someone's router when I release a buggy update. At least those of you who search it out can share in the blame.

 

[chris.at]

At least for folks like me who wondered why it's been so silent about FlexQoS shortly, not realizing that there is a new section for AddOns in the forum and the thread has moved, some kind of "Look, here it is" would be a nice feature. Someday, sometime, fully understand your thoughts.

 

[Treadler]

At least for folks like me who wondered why it's been so silent about FlexQoS shortly, not realizing that there is a new section for AddOns in the forum and the thread has moved, some kind of "Look, here it is" would be a nice feature. Someday, sometime, fully understand your thoughts.

[Announcement] New AddOns subforum

Howdy folks, As you might have noticed, there is now a new dedicated Asuswrt-Merlin AddOns sub-forum. All discussions about addons such as Diversion, Skynet, etc... as well as anything specific to Entware should now be posted in that forum. If you start a new thread there, make sure to use...

www.snbforums.com

 

[Sinner]

@Vexira i also still used/unsing the 90% rule from freshjr.

Rule:
DownCeil="$(expr ${DownCeil} \* 90 / 100)"
UpCeil="$(expr ${UpCeil} \* 90 / 100)"

So its not good to use this anymore ?

it never was.. the 90 or 95% recomendation was only ever a recomended starting point for adjustments and in no way was meant to be considered the "optimal" setting.. it might be for 1 out of 100 people but chances are thats none of us here. Your connection speed and type and several other factors all play a role in the actual optimal number.

 

[chris.at]

Had a wan failover right now (from eth0 to usb0) and got some errors in log, don't know if it helps, but here they are.

Spoiler: Error Log

Oct 19 12:32:37 FlexQoS: /jffs/addons/flexqos/flexqos.sh (pid=13422) called with 2 args: -start usb0
Oct 19 12:32:37 FlexQoS: [*] Killing Delayed Process (pid=12289)
Oct 19 12:32:37 custom_script: Running /jffs/scripts/firewall-start (args: usb0)
Oct 19 12:32:37 custom_script: Running /jffs/scripts/service-event-end (args: restart firewall)
Oct 19 12:32:37 FlexQoS: [*] 12289 admin     1572 S    sh /jffs/addons/flexqos/flexqos.sh -start usb0
Oct 19 12:32:38 custom_script: Running /jffs/scripts/service-event-end (args: start vpnserver1)
Oct 19 12:32:40 dhcp_client: bound 192.168.0.148/255.255.255.0 via 192.168.0.1 for 86400 seconds.
Oct 19 12:32:40 FlexQoS: Applying iptables static rules
Oct 19 12:32:40 FlexQoS: Applying iptables custom rules
Oct 19 12:32:41 FlexQoS: Flushing conntrack table
Oct 19 12:32:41 FlexQoS: TC Modification Delayed Start
Oct 19 12:32:43 FlexQoS: /jffs/addons/flexqos/flexqos.sh (pid=14432) called with 2 args: -start usb0
Oct 19 12:32:43 FlexQoS: [*] Killing Delayed Process (pid=13422)
Oct 19 12:32:43 FlexQoS: [*] 13422 admin     1572 S    sh /jffs/addons/flexqos/flexqos.sh -start usb0
Oct 19 12:32:44 FlexQoS: Applying iptables static rules
Oct 19 12:32:44 FlexQoS: Applying iptables custom rules
Oct 19 12:32:45 FlexQoS: Flushing conntrack table
Oct 19 12:32:45 FlexQoS: TC Modification Delayed Start
Oct 19 12:33:16 FlexQoS: TC Modification delayed for 30 seconds
Oct 19 12:33:18 FlexQoS: Applying AppDB rules and TC rates
Oct 19 12:33:18 FlexQoS: ERROR! Check /tmp/flexqos_tcrules.log


admin@router:/tmp/home/root# cat /tmp/flexqos_tcrules.log
RTNETLINK answers: No such file or directory
Command failed /tmp/flexqos_tcrules:35
RTNETLINK answers: No such file or directory
Command failed /tmp/flexqos_tcrules:37
RTNETLINK answers: No such file or directory
Command failed /tmp/flexqos_tcrules:38
RTNETLINK answers: No such file or directory
Command failed /tmp/flexqos_tcrules:39
RTNETLINK answers: No such file or directory
Command failed /tmp/flexqos_tcrules:40
RTNETLINK answers: No such file or directory
Command failed /tmp/flexqos_tcrules:41
RTNETLINK answers: No such file or directory
Command failed /tmp/flexqos_tcrules:42
RTNETLINK answers: No such file or directory
Command failed /tmp/flexqos_tcrules:43
RTNETLINK answers: No such file or directory
Command failed /tmp/flexqos_tcrules:44
RTNETLINK answers: No such file or directory
Command failed /tmp/flexqos_tcrules:45
RTNETLINK answers: No such file or directory
Command failed /tmp/flexqos_tcrules:46
RTNETLINK answers: No such file or directory
Command failed /tmp/flexqos_tcrules:47
RTNETLINK answers: No such file or directory
Command failed /tmp/flexqos_tcrules:48

greetings,
chris

 

[dave14305]

Had a wan failover right now (from eth0 to usb0) and got some errors in log, don't know if it helps, but here they are.

Interesting, since I have no way to test dual WAN.

Is it still failed to usb0? Can you run:

tc qdisc ls | grep root
cat /tmp/bwdpi/dev_wan