FlexQoS FlexQoS 1.0 - Flexible QoS Enhancement Script for Adaptive QoS

[dave14305]

Some new features on the develop branch for brave testers:

• Web-based Check for Update / Update
• Respect Custom rule names when filtering Tracked Connections by Application field
• Append tilde (~) to AppDB override labels in Tracked Connections (an old FreshJR feature lost and now found)
• Add support for inverted Mark in iptables rules using "!" (e.g. "!0400C1", "!04****" means NOT matching Amazon Video or NOT matching the Streaming category)
• Make Application label sorting case insensitive to handle situation where iCloud and iTunes were sorted after uppercase A-Z.

"So, brave knights, if you do doubt your courage or your strength, come no further, for bugs await you all with nasty, big, pointy teeth." -- Dave the Enchanter (in a Monty Python mood tonight)

 

[dave14305]

So far so good only using sfq with Adaptive QoS instead of fq_codel. No ill effects and no perceived performance issues.

 

[RMerlin]

Wade summarized my thinking very succinctly. There is probably little difference in outcomes between sfq and fq_codel with Adaptive QoS. I could also be completely wrong or misguided since I spend most of my time studying HTB and not fq_codel or sfq. I just switched my own router to sfc to see if I notice any differences or not.

It's probably situational. Back when I initially implemented fq_codel, I was getting a very clear bufferbloat/latency improvement when switching to fq_codel.

Here is the ping plot I took at the time, showing the latency difference on pinging www.google.com while running a speed test from Speedtest.net at the same time.

 

[Sinner]

I agree its situational.. If within a category lets say "others" you have 2 transfers going, one something large like video trying to hog it and another like voip then the fq_codel will improve the voip because it favours the small packets 1st. If its just those two transfer again but in different categories i dont think it does much of anything.

if eveything is in the correct categories i find sfq does the job and is even lower cpu as well. For higher speed connection this may be key to squeezing that extra out of it.

 

[dave14305]

Another item to test in the develop branch: Using ! as an inverted filter for Tracked connections. In this screenshot, I'm only including connections that are NOT port 443 AND are NOT Apple. I really want to implement regex, but I think the negation is more intuitive than learning regular expressions for most users.

Use the Check Update button if you're already on develop. Otherwise, you can switch with flexqos develop at the CLI.

 

[dave14305]

Bug fix for WebUI update handler pushed to the develop branch. Thanks @Cam!

There's also a change there to transition the Local IP filter dropdown to a text field plus dropdown (like DHCP page, DNS Filter page, etc. using stock Client Dropdown list). It's not finished, but I'm curious to hear how it works for others right now. It's still an exact match field, meaning it won't do partial matches yet, so you need to type in a complete IP to get a match, or select an IP from the dropdown.

 

[S.claus]

@dave14305
My WebUI update doesn't work simply runs that little circle next to the v1.0.4dev .I can still ssh into the router and run flexqos -update which completes in 2 seconds.

Thank you for this amazing script.

 

[dave14305]

@dave14305
My WebUI update doesn't work simply runs that little circle next to the v1.0.4dev .I can still ssh into the router and run flexqos -update which completes in 2 seconds.

Thank you for this amazing script.

Can you post the contents of /jffs/scripts/service-event-end? I’m wondering if my last bugfix missed cleaning up the bad entry.

EDIT: in fact, I’m guessing the service-event-end script has a syntax error as a result of the previous bug, so it’s probably not running for any FlexQoS function.

Please run

sed -i '\~# FlexQoS Addition~d' /jffs/scripts/service-event-end
flexqos forceupdate

And let me know if it works again. I am aware I’m missing a notification when no update is available, but the spinning should stop.

EDIT 2: I've pushed an update to the develop branch that will hopefully fix this (update via CLI). Apologies to any other scripts relying on service-event-end that may have temporarily broken due to my bug (I did warn you about bugs with nasty pointy teeth...).

 

[S.claus]

Can you post the contents of /jffs/scripts/service-event-end? I’m wondering if my last bugfix missed cleaning up the bad entry.

EDIT: in fact, I’m guessing the service-event-end script has a syntax error as a result of the previous bug, so it’s probably not running for any FlexQoS function.

Please run

sed -i '\~# FlexQoS Addition~d' /jffs/scripts/service-event-end
flexqos forceupdate

And let me know if it works again. I am aware I’m missing a notification when no update is available, but the spinning should stop.

EDIT 2: I've pushed an update to the develop branch that will hopefully fix this (update via CLI). Apologies to any other scripts relying on service-event-end that may have temporarily broken due to my bug (I did warn you about bugs with nasty pointy teeth...).

I enjoy testing things so I don't mind a few bugs every now and then. After the update, it seems to work

Oct 13 15:40:29 hour_monitor: daemon is starting
Oct 13 15:40:36 rc_service: httpd 1224:notify_rc start_flexqosupdatecheck
Oct 13 15:40:36 custom_script: Running /jffs/scripts/service-event-end (args: start flexqosupdatecheck)
Oct 13 15:40:36 FlexQoS: Checking for updates...
Oct 13 15:40:36 FlexQoS: No updates

 

[dave14305]

I enjoy testing things so I don't mind a few bugs every now and then. After the update, it seems to work

Oct 13 15:40:29 hour_monitor: daemon is starting
Oct 13 15:40:36 rc_service: httpd 1224:notify_rc start_flexqosupdatecheck
Oct 13 15:40:36 custom_script: Running /jffs/scripts/service-event-end (args: start flexqosupdatecheck)
Oct 13 15:40:36 FlexQoS: Checking for updates...
Oct 13 15:40:36 FlexQoS: No updates

Great! I just pushed another small fix to the AppDB labels if you want to test the update button again.

 

[S.claus]

Great! I just pushed another small fix to the AppDB labels if you want to test the update button again.

This is amazing no more SSH required.

Oct 13 15:58:24 rc_service: httpd 1224:notify_rc start_flexqosupdatecheck
Oct 13 15:58:24 custom_script: Running /jffs/scripts/service-event-end (args: start flexqosupdatecheck)
Oct 13 15:58:24 FlexQoS: Checking for updates...
Oct 13 15:58:25 FlexQoS: Hotfix available!
Oct 13 15:58:29 rc_service: httpd 1224:notify_rc start_flexqosupdateforce
Oct 13 15:58:29 custom_script: Running /jffs/scripts/service-event-end (args: start flexqosupdateforce)
Oct 13 15:58:29 FlexQoS: Updated flexqos.asp
Oct 13 15:58:29 rc_service: service 8822:notify_rc restart_qos;restart_firewall
Oct 13 15:58:30 kernel: Cpuidle Host Clock divider is disabled
Oct 13 15:58:32 kernel: The For ALL DEVICES flag of Prof 1 has been set to ENABLE
Oct 13 15:58:32 BWDPI: fun bitmap = 4ff
Oct 13 15:58:32 A.QoS: qos_count=0, qos_check=0
Oct 13 15:58:35 custom_script: Running /jffs/scripts/service-event-end (args: restart qos)
Oct 13 15:58:35 nat: apply nat rules (/tmp/nat_rules_ppp0_eth0)
Oct 13 15:58:35 custom_script: Running /jffs/scripts/firewall-start (args: ppp0)
Oct 13 15:58:35 custom_script: Running /jffs/scripts/service-event-end (args: restart firewall)
Oct 13 15:58:36 FlexQoS: /jffs/addons/flexqos/flexqos.sh (pid=9718) called with 2 args: -start ppp0
Oct 13 15:58:36 FlexQoS: Applying iptables static rules
Oct 13 15:58:36 FlexQoS: Applying iptables custom rules
Oct 13 15:58:36 FlexQoS: Flushing conntrack table
Oct 13 15:58:36 FlexQoS: No TC modifications necessary

 

[dave14305]

This is amazing no more SSH required.

Oct 13 15:58:24 rc_service: httpd 1224:notify_rc start_flexqosupdatecheck
Oct 13 15:58:24 custom_script: Running /jffs/scripts/service-event-end (args: start flexqosupdatecheck)
Oct 13 15:58:24 FlexQoS: Checking for updates...
Oct 13 15:58:25 FlexQoS: Hotfix available!
Oct 13 15:58:29 rc_service: httpd 1224:notify_rc start_flexqosupdateforce
Oct 13 15:58:29 custom_script: Running /jffs/scripts/service-event-end (args: start flexqosupdateforce)
Oct 13 15:58:29 FlexQoS: Updated flexqos.asp
Oct 13 15:58:29 rc_service: service 8822:notify_rc restart_qos;restart_firewall
Oct 13 15:58:30 kernel: Cpuidle Host Clock divider is disabled
Oct 13 15:58:32 kernel: The For ALL DEVICES flag of Prof 1 has been set to ENABLE
Oct 13 15:58:32 BWDPI: fun bitmap = 4ff
Oct 13 15:58:32 A.QoS: qos_count=0, qos_check=0
Oct 13 15:58:35 custom_script: Running /jffs/scripts/service-event-end (args: restart qos)
Oct 13 15:58:35 nat: apply nat rules (/tmp/nat_rules_ppp0_eth0)
Oct 13 15:58:35 custom_script: Running /jffs/scripts/firewall-start (args: ppp0)
Oct 13 15:58:35 custom_script: Running /jffs/scripts/service-event-end (args: restart firewall)
Oct 13 15:58:36 FlexQoS: /jffs/addons/flexqos/flexqos.sh (pid=9718) called with 2 args: -start ppp0
Oct 13 15:58:36 FlexQoS: Applying iptables static rules
Oct 13 15:58:36 FlexQoS: Applying iptables custom rules
Oct 13 15:58:36 FlexQoS: Flushing conntrack table
Oct 13 15:58:36 FlexQoS: No TC modifications necessary

Just to verify, did the flexqos -check make TC modifications at 16:03? They should have been required at 15:58 but timing must have been too soon. It kind of bothers me, but that's why the delayed 5 minute check is there.

 

[S.claus]

Just to verify, did the flexqos -check make TC modifications at 16:03? They should have been required at 15:58 but timing must have been too soon. It kind of bothers me, but that's why the delayed 5 minute check is there.

It just says no TC modifications necessary. I also noticed that it says there's a hotfix available even after updating 2 minutes ago

 

[dave14305]

It just says no TC modifications necessary. I also noticed that it says there's a hotfix available even after updating 2 minutes ago

Yes, I just noticed the same thing. Had a leftover "developer" hack to prevent the download of the script file from Github while I was developing. Just sent another hotfix. Everyone using develop branch will need to SSH one more time to run a forced update from the CLI menu.

 

[guardog]

hi there, I realise that Discord (VOIP) traffic isn't being classified properly as Work-From-Home. I have narrowed down the connection and noticed that it is classified as Untracked hence it is at a pretty low priority.

MARK for the connection is all 0s but I have found this piece of information online

https://www.reddit.com/r/discordapp/comments/58xkko

Does DSCP / 802.1p play a role in this algorithm?

EDIT: This occurs with "Enable Quality of Service High Packet Priority" setting both enabled and disabled. AC68U on 384.19, FlexQoS 1.0.3

 

[Sinner]

hi there, I realise that Discord (VOIP) traffic isn't being classified properly as Work-From-Home. I have narrowed down the connection and noticed that it is classified as Untracked hence it is at a pretty low priority.

MARK for the connection is all 0s but I have found this piece of information online

https://www.reddit.com/r/discordapp/comments/58xkko

Does DSCP / 802.1p play a role in this algorithm?

most pc games and voice apps etc are untracked.

 

[Vexira]

@dave14305 sorry to bother you just wanted to as if it's possible for a future GUI option to set reserved bandwidth for QoS similar to the overhead box where you can enter a custom value if that's not too much trouble to ask, just thinking it might be helpful to everyone and people like me who on resetting the router forget to make required changes to settings.

 

[dave14305]

just wanted to as if it's possible for a future GUI option to set reserved bandwidth for QoS similar to the overhead box where you can enter a custom value...

I'm not sure I understand what you're asking for. Min/max reserved bandwidth is already there, so please elaborate or draw a picture. If this is related to the old 95% hack, I'm not going there because it creates conflicts with the underlying ceilings of the device-level child htb classes (the top level classes 1:10 - 1:17 get artificially lowered outside the scope of bwdpi, but the child classes 10:2 - 17:256 remain with the original higher ceiling which may result in unexpected bottlenecks at high load).[/USER][/QUOTE]

 

[Vexira]

I'm not sure I understand what you're asking for. Min/max reserved bandwidth is already there, so please elaborate or draw a picture. If this is related to the old 95% hack, I'm not going there because it creates conflicts with the underlying ceilings of the device-level child htb classes (the top level classes 1:10 - 1:17 get artificially lowered outside the scope of bwdpi, but the child classes 10:2 - 17:256 remain with the original higher ceiling which may result in unexpected bottlenecks at high load).[/USER]

[/QUOTE]
I didn't know the old 95 breaks QoS well in that case I'll go reset my script didn't realise everything had changed that much to the point it breaks things.

Come to think of it I thought originally the QoS system by default set bandwidth to 95% form the Asus set ups guides if I remember correctly my old Netgear did something like that, so hence my confusion.

 

[dave14305]

I didn't know the old 95 breaks QoS well in that case I'll go reset my script didn't realise everything had changed that much to the point it breaks things.

It creates a conflict when the child has a higher ceiling than the parent. It's not broken but it wasn't an ideal situation if you max out your bandwidth often/ever.