FlexQoS FlexQoS 1.2.5 - Flexible QoS Enhancement Script for Adaptive QoS

[dave14305]

FlexQoS Version 1.2.5 - Released 13-Jun-2021

NEW:

• Added Shellcheck integration with Github Actions (stolen from @Jack Yaz again)

CHANGED:

• Removed support for Hotfix updates. Updates will only be detected by a version number change from now on.
• Updated code generation of iptables rules for handling multiple protocol options (BOTH = -p all).
• Create predictable qdisc handles for fq_codel to make it easier to relate fq_codel back to parent htb class.
• Many code style changes to adapt to the Shellcheck integration.
• Moved README screenshots from imgur.com to Github repo.

Previous versions discussed here.

Update via the WebUI, amtm, or the command line with flexqos update

Requirements:

• ASUSWRT-Merlin 384.15 or higher
• SSH, JFFS scripts enabled
• Adaptive QoS enabled

Installation:
The simplest method for installation is to use amtm, option i, option 3.

For manual installation, paste the following command into your SSH terminal session:

/usr/sbin/curl "https://raw.githubusercontent.com/dave14305/FlexQoS/master/flexqos.sh" -o /jffs/addons/flexqos/flexqos.sh --create-dirs && chmod +x /jffs/addons/flexqos/flexqos.sh && sh /jffs/addons/flexqos/flexqos.sh -install

Source: https://github.com/dave14305/FlexQoS

Updates:

flexqos -update
# or
sh /jffs/addons/flexqos/flexqos.sh -update

You can also update via the "Check for Update" button in the WebUI.

User documentation available on Github:

FlexQoS/README.md at master · dave14305/FlexQoS

FlexQoS - Flexible QoS Enhancement Script for Adaptive QoS on ASUSWRT-Merlin - dave14305/FlexQoS

github.com

 

[dave14305]

This is a minor release to clear out the backlog of some coding changes made a while ago, languishing in the develop branch. Nothing exciting, but since the old thread reached the 6 month maximum lifetime, it seemed a good time to push this as a release.

 

[ColDen]

FlexQoS Version 1.2.5 - Released 13-Jun-2021

NEW:

• Added Shellcheck integration with Github Actions (stolen from @JackYaz again)

CHANGED:

• Removed support for Hotfix updates. Updates will only be detected by a version number change from now on.
• Updated code generation of iptables rules for handling multiple protocol options (BOTH = -p all).
• Create predictable qdisc handles for fq_codel to make it easier to relate fq_codel back to parent htb class.
• Many code style changes to adapt to the Shellcheck integration.
• Moved README screenshots from imgur.com to Github repo.

Previous versions discussed here.

Update via the WebUI, amtm, or the command line with flexqos update

Requirements:

• ASUSWRT-Merlin 384.15 or higher
• SSH, JFFS scripts enabled
• Adaptive QoS enabled

Installation:
The simplest method for installation is to use amtm, option i, option 3.

For manual installation, paste the following command into your SSH terminal session:

/usr/sbin/curl "https://raw.githubusercontent.com/dave14305/FlexQoS/master/flexqos.sh" -o /jffs/addons/flexqos/flexqos.sh --create-dirs && chmod +x /jffs/addons/flexqos/flexqos.sh && sh /jffs/addons/flexqos/flexqos.sh -install

Source: https://github.com/dave14305/FlexQoS

Updates:

flexqos -update
# or
sh /jffs/addons/flexqos/flexqos.sh -update

You can also update via the "Check for Update" button in the WebUI.

User documentation available on Github:

FlexQoS/README.md at master · dave14305/FlexQoS

FlexQoS - Flexible QoS Enhancement Script for Adaptive QoS on ASUSWRT-Merlin - dave14305/FlexQoS

github.com

Donations
Donations are graciously accepted via Paypal.

No issues to report following the update. Thanks dave14305!

 

[geobernd]

Updated using the GUI (first time trying that instead of command line). All is running well. Thanks dave14305

 

[BreakingDad]

Thank you. As you know, quite partial to an update.

 

[Mario64]

On an AC-3100 running rmerlin 486.2 should I be using fq _codel or ASUS?

 

[dave14305]

On an AC-3100 running rmerlin 486.2 should I be using fq _codel or ASUS?

fq_codel is much better than the sfq used by Asus stock Adaptive QoS.

 

[CriticJay]

Just wanted to say that, with my particular combination of Internet service + router hardware, this AddOn remains extremely useful and I'm very grateful that Dave is continuing to maintain it.

My internet: DOCSIS 3.1 Cable Internet, with Gigabit download (1024mbps) but only 30mbps upload. As such, QoS is a must!

My router: AC86U. Although lots of people have switched over to CAKE, my understanding is that the AC86U can only handle ~350mbps download on CAKE, due to lack of HND hardware acceleration.

But thanks to AsusMerlin FW + FlexQoS addon + me configuring/tweaking everything "just right" I'm able to have hardware-accelerated 950mbps downloads with only ~60-70% CPU usage on the AC86U, and -- most importantly -- NO discernable buffer-bloat.

I can easily maintain a 1080p Zoom video call without A/V sync issues or lag (on my laptop), while a wired Ethernet device is downloading at ~940-950mbps. (I'm sure that my TC4400 modem is helping a bit, since it does not use the problematic Puma chipset)

Thanks Dave for keeping this add-on alive!

 

[ttgapers]

Added Shellcheck integration with Github Actions (stolen from @Jack Yaz again)

Me likes Thanks @Jack Yaz for adding that early on for Cake as well!

 

[el pescador]

Dave...is it easy to add a delay on upload or download packets like in Netem? I noticed some of the jargon is the same used in the code...tc etc.
I ask because some games companies , maybe all , use Round Trip Time when calculating lag comp based on latency.
Technically this is wrong as both UDP upload and download use different routes....and can sometimes have different latencies both ways.
This may not be large but can cause the feeing of "out of sync " in some games.
Say the download route is quicker then then there will be a small delay on every packet.
If the outgoing route is quicker then you will see the play slightly late.

A possible solution is to be able to tweak either upload or download to get it in sync.

 

[dave14305]

Dave...is it easy to add a delay on upload or download packets like in Netem? I noticed some of the jargon is the same used in the code...tc etc.
I ask because some games companies , maybe all , use Round Trip Time when calculating lag comp based on latency.
Technically this is wrong as both UDP upload and download use different routes....and can sometimes have different latencies both ways.
This may not be large but can cause the feeing of "out of sync " in some games.
Say the download route is quicker then then there will be a small delay on every packet.
If the outgoing route is quicker then you will see the play slightly late.

A possible solution is to be able to tweak either upload or download to get it in sync.

In theory, you could replace the fq_codel qdisc on the Gaming class with netem if you know what parameters to pass it. Assuming Gaming is top QoS priority (class 1:11), you could experiment by manually running these commands to add 10ms delay on upload (eth0), download (br0) or both. I have no idea if the syntax is correct, but it’s a good experiment for you gamers.

tc qdisc replace dev eth0 parent 1:11 handle 111: netem delay 10ms 7ms distribution normal
tc qdisc replace dev br0 parent 1:11 handle 111: netem delay 10ms 7ms distribution normal

You can keep re-running the tc qdisc replace command every time you want to adjust the delay (10ms) and jitter (7ms).
Reference:

routerperf/SimpleHFSCgamerscript.sh at master · dlakelan/routerperf

OpenWRT router performance data collection and analysis script - dlakelan/routerperf

github.com

EDIT: Step 1: check if netem is built in the kernel. Nope. Not possible now.

 

[Vexira]

hi dave had a question i noticed google dns over tls was set as untracked, oddly also wondering how to get the adguard dns over quic detected as net control or quic
using the rules to have them identified correctly
ad guard dns over quic
quic://dns-unfiltered.adguard.com:784

8.8.8.8 and 8.8.4.4 TCP port 853 is what i saw in the log.

 

[dave14305]

hi dave had a question i noticed google dns over tls was set as untracked, oddly also wondering how to get the adguard dns over quic detected as net control or quic
using the rules to have them identified correctly
ad guard dns over quic
quic://dns-unfiltered.adguard.com:784

8.8.8.8 and 8.8.4.4 TCP port 853 is what i saw in the log.

The information you need should be in your Tracked Connections list when those DNS connections are active.

 

[xlr]

I'm using FlexQOS for some time now, and since day 1 I was trying to figure out the best/easiest way to locate active connection + IP or hostname.

Step one - I use web history in order to get hostnames. That often doesn't help so I look at tracked connections - filter by device and pick category which is generating traffic in graph above - then I try to figure out which IP(s) are active connection in order to create new iptable rule.
I'm playing around with classifing important traffic on my network. Some streaming activities are classified as web surfing or other, rarely untracked.


Next issue is probably some bug in Asus code, but I would like to confirm that. From time to time by looking on the QOS graph, I see my stream classified traffic becomes web surfing (wrong class and colour). When I check tracked connections, I see my custom iptable rule IS applied to active IPs and connection has correct class - BUT graph is not following the data.
Do I need to worry about it? Who do I trust - tracked connection info or QOS graph? It worries me because I keep streaming and web surfing far away from each other, and I would preffer not to compromise streaming priority.

Edit:
I've added screenshots. It can be seen how streaming class suddenly change to web surfing. I did nothing, just watching the graph. Somehow, my IPTV connection is no longer listed under tracked connections.
On second screenshot there is active IPTV connection, but it is shown as web surfing in graph.

Edit2:
Hmm, if new separated topic would be better, just say it and I'll gather as much data and open it separately from main topic.

 

[dave14305]

Step one - I use web history in order to get hostnames. That often doesn't help so I look at tracked connections - filter by device and pick category which is generating traffic in graph above - then I try to figure out which IP(s) are active connection in order to create new iptable rule.
I'm playing around with classifing important traffic on my network. Some streaming activities are classified as web surfing or other, rarely untracked.

Bandwidth Monitor with App Analysis enabled can usually tell you which device is generating traffic and which app category it belongs to.

Next issue is probably some bug in Asus code, but I would like to confirm that. From time to time by looking on the QOS graph, I see my stream classified traffic becomes web surfing (wrong class and colour). When I check tracked connections, I see my custom iptable rule IS applied to active IPs and connection has correct class - BUT graph is not following the data.
Do I need to worry about it? Who do I trust - tracked connection info or QOS graph? It worries me because I keep streaming and web surfing far away from each other, and I would preffer not to compromise streaming priority.

The graphs tell you where the traffic actually gets classified. Tracked connections list where it should be classified, but not where it actually was classified.

Do you already have a rule setup for your Shield TV device? If not, the classification is limited by whether Trend Micro signatures can accurately detect the traffic as streaming. In your second screenshot, how do you know the traffic isn’t happening on the HTTP connection?

 

[xlr]

Bandwidth Monitor with App Analysis enabled can usually tell you which device is generating traffic and which app category it belongs to.

I'm doing tests alone in network and with very few devices. Shield is test device and according to Bandwidth monitor, traffic comes from Shield - HTTP protocol. In QoS tab it is web surfing.
I was hoping there is a way to pinpoint exact IP or hostname of active connection - for example, it would be great if tracked connections could be sorted by activity. Let's say, on every refresh, latest active connection goes to the first place and so on. Or some sort of activity indicator near each connection.

When I filter out just web surfing connections for Shield device there is a lot of data. Not to mention untracked connections.

The graphs tell you where the traffic actually gets classified. Tracked connections list where it should be classified, but not where it actually was classified.

So that means something (Trend Micro?) is overriding my custom classification for IPTV traffic? That's exactly what I would like to avoid :\

Do you already have a rule setup for your Shield TV device? If not, the classification is limited by whether Trend Micro signatures can accurately detect the traffic as streaming. In your second screenshot, how do you know the traffic isn’t happening on the HTTP connection?

I've made few basic rules for IPTV (I know remote IPs so that was easy) and few other services which are wrongly classified. I don't have any appDB rules.

I'm sure this is IPTV traffic on graph, no doubt on that. In my case IPTV traffic is going through port 80 and that's a reason why Asus marks it as web surfing. As you can see on second screenshot in my last post, custom iptable rule succesfully marked this IPTV connection over port 80 as streaming class (yellow) meaning iptable is hopefully fine.

Marking ALL traffic from Shield as streaming is not an option since I sometimes run torrent client on it and preffer such traffic as low priority.
Thank you!

 

[TonyK132]

Dave - How can I reset the rules for the Tracked Connection? I may have screwed it up and want to get back to your defaults so I can start over.

 

[dave14305]

I was hoping there is a way to pinpoint exact IP or hostname of active connection - for example, it would be great if tracked connections could be sorted by activity. Let's say, on every refresh, latest active connection goes to the first place and so on. Or some sort of activity indicator near each connection.

I am not aware of any way to track per-connection bandwidth in realtime.

So that means something (Trend Micro?) is overriding my custom classification for IPTV traffic? That's exactly what I would like to avoid :\

I don’t know that anything is overriding your custom classification. If your IPTV is using IPs outside of your 3 rules, you need a more general rule.

 

[dave14305]

Dave - How can I reset the rules for the Tracked Connection? I may have screwed it up and want to get back to your defaults so I can start over.

There are reset links in each rule section of the GUI.

 

[TonyK132]

Sorry I could not find that.

Is there someplace I can see all the rules that are in effect, either in the GUI or in the CLI?

Also, I just noticed that not all the Class colors are the same between the FlexQoS and Classification tabs. Is that intentional?