FlexQoS FlexQoS 1.0 - Flexible QoS Enhancement Script for Adaptive QoS

[dave14305]

@dave14305
Here's log related to ppp0 or eth0

Spoiler: Sys Log (ppp0/eth0)

Jul 29 05:00:20 custom_script: Running /jffs/scripts/firewall-start (args: ppp0)
Jul 29 05:00:21 FlexQoS: /jffs/addons/flexqos/flexqos.sh (pid=3330) called with 2 args: -start ppp0
Jul 29 05:00:22 FlexQoS: [*] Killing Delayed Process (pid=1215)
Jul 29 05:00:23 FlexQoS: [*] 1215 thunder 1560 S sh /jffs/addons/flexqos/flexqos.sh -start ppp0
Jul 29 05:00:27 FlexQoS: Applying iptables static rules
Jul 29 05:00:28 FlexQoS: Applying iptables custom rules
Jul 29 05:00:33 FlexQoS: Applying AppDB static rules
Jul 29 05:00:33 FlexQoS: Cannot find device "ppp0,eth0"
Jul 29 05:00:34 FlexQoS: Applying AppDB custom rules and TC rates
Jul 29 05:00:34 FlexQoS: Cannot find device "ppp0,eth0"
Jul 29 05:00:34 FlexQoS: Cannot find device "ppp0,eth0"
Jul 29 05:00:34 FlexQoS: Cannot find device "ppp0,eth0"
Jul 29 05:00:34 FlexQoS: Cannot find device "ppp0,eth0"
Jul 29 05:00:34 FlexQoS: Cannot find device "ppp0,eth0"
Jul 29 05:00:34 FlexQoS: Cannot find device "ppp0,eth0"
Jul 29 05:01:18 Skynet: [*] WebUI Integration Requires Logging To Be Enabled
Jul 29 05:01:23 Skynet: [#] 323603 IPs (+0) -- 1569 Ranges Banned (+0) || 1 Inbound -- 0 Outbound Connections Blocked! [start] [115s]
Jul 29 00:01:46 transmission-daemon[2264]: Port Forwarding Starting (port-forwarding.c:107)
Jul 29 05:05:00 FlexQoS: /jffs/addons/flexqos/flexqos.sh (pid=6620) called with 1 args: -check
Jul 29 05:05:01 FlexQoS: No TC modifications necessary

Dirty flashed over 0.9.5

So file /tmp/bwdpi/dev_wan contains 2 entries? What do you get running tc qdisc ls | grep root

 

[dave14305]

I pushed a hotfix to hopefully avoid the issue immi803 ran into.

 

[immi803]

I pushed a hotfix to hopefully avoid the issue immi803 ran into.

Gonna try fix

By the way, your required result is

ASUSWRT-Merlin RT-AC68U 384.18_0 Sun Jun 28 17:57:07 UTC 2020
thunder@RT-AC68U-4370:/tmp/home/root# tc qdisc ls | grep root
qdisc htb 1: dev eth0 root refcnt 2 r2q 10 default 0 direct_packets_stat 416
qdisc pfifo_fast 0: dev eth1 root refcnt 2 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: dev eth2 root refcnt 2 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc htb 1: dev br0 root refcnt 2 r2q 10 default 0 direct_packets_stat 5492
qdisc pfifo_fast 0: dev ppp0 root refcnt 2 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
thunder@RT-AC68U-4370:/tmp/home/root#

 

[brummygit]

Version 0.9.6

• NEW: Added experimental option to flush conntrack table after applying iptables rules (enable with flexqos flushct or disable with flexqos noflushct)
• NEW: Added reset option to Tracked Connections filters
• CHANGED: Don't allow iptables rules that only contain a Mark (these should be AppDB rules)
• CHANGED: Converted Download/Upload rates in pie chart table from Kibibytes to Kilobits
• CHANGED: Improve detection of WAN interface used for tc commands
• CHANGED: Changed the way we detect if FlexQoS rules have been applied yet
• FIXED: Coloring of Untracked traffic if Untracked AppDB rule deleted
• FIXED: AppDB new rule classes were overridden by the default class for the app due to logic bug

sh /jffs/addons/flexqos/flexqos.sh update

Thank you - just updated and the conntrack flush seems to be working fine.

I have a couple of queries if you can help

1. I'm using dual WAN failover and always assumed that in failover mode I have no FlexQoS. What would you expect to see?
2. I recall (and read the script) that you round the QoS bandwidth values down to the nearest Mb. My upload is around 8.5Mbps and I find setting to 8Mbps doesn't give quite enough headroom, but 7Mbps is taking another 12.5% off my precious available bandwidth. Is it a big change to allow a more granular change?

Thanks as always for your hard work - it is much appreciated.

 

[chris.at]

I'm also using dual wan failover and if it fails over to usb lte stick flexqos gets reinitialized with wan port usb instead of eth, so it's still working, even if the up/down values don't match anymore. But since it's failover and normally only lasts some minutes that's not a big deal. Sure it would be nice to allow different bandwidth values for eth and usb wan if it's not too much work to implement it but at least for me that's quite unimportant.

 

[dave14305]

I recall (and read the script) that you round the QoS bandwidth values down to the nearest Mb. My upload is around 8.5Mbps and I find setting to 8Mbps doesn't give quite enough headroom, but 7Mbps is taking another 12.5% off my precious available bandwidth. Is it a big change to allow a more granular change?

The bandwidth is stored in kbps in nvram, so you have to see how your value looks by running nvram get qos_obw

If you‘re storing fractional kilobits, I don’t think it’s going to make a difference if that gets rounded (says someone with 25 Mb up!).

 

[dave14305]

I'm using dual WAN failover and always assumed that in failover mode I have no FlexQoS. What would you expect to see?

I'm also using dual wan failover and if it fails over to usb lte stick flexqos gets reinitialized with wan port usb instead of eth, so it's still working, even if the up/down values don't match anymore. But since it's failover and normally only lasts some minutes that's not a big deal. Sure it would be nice to allow different bandwidth values for eth and usb wan if it's not too much work to implement it but at least for me that's quite unimportant.

My expectation would be that the firewall rules would work ok, but what happens to the tc commands in a failover? I’m fighting with how to detect eth0 or eth4 as the tc WAN interface, but does it look different in a failover to usb? Run these after a failover:

tc qdisc ls | grep root
cat /tmp/bwdpi/dev_wan

 

[geobernd]

Version 0.9.6

• NEW: Added experimental option to flush conntrack table after applying iptables rules (enable with flexqos flushct or disable with flexqos noflushct)

sh /jffs/addons/flexqos/flexqos.sh update

Applied the update yesterday, enabled flushct and did a couple of 'random' restarts to see if it causes any issues - all my IOT devices are still running fine.

 

[chris.at]

My expectation would be that the firewall rules would work ok, but what happens to the tc commands in a failover? I’m fighting with how to detect eth0 or eth4 as the tc WAN interface, but does it look different in a failover to usb? Run these after a failover:

tc qdisc ls | grep root
cat /tmp/bwdpi/dev_wan

Ok, here we go:

DSL active:

admin@router:/tmp/home/root# tc qdisc ls | grep root
qdisc htb 1: dev eth0 root refcnt 2 r2q 10 default 0 direct_packets_stat 37490
qdisc pfifo_fast 0: dev eth1 root refcnt 2 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: dev eth2 root refcnt 2 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc htb 1: dev br0 root refcnt 2 r2q 10 default 0 direct_packets_stat 3
qdisc pfifo_fast 0: dev wl0.1 root refcnt 2 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: dev usb0 root refcnt 2 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: dev tun21 root refcnt 2 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1

Failover to usb:

Jul 29 15:48:49 FlexQoS: /jffs/addons/flexqos/flexqos.sh (pid=10998) called with 2 args: -start usb0

That's why I thought it would be reinitialized correctly after a failover.

USB active:

admin@router:/tmp/home/root# tc qdisc ls | grep root
qdisc pfifo_fast 0: dev eth0 root refcnt 2 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: dev eth1 root refcnt 2 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: dev eth2 root refcnt 2 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: dev wl0.1 root refcnt 2 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: dev usb0 root refcnt 2 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1

No idea what it means but hopefully it helps your investigations
But even if initializiation would be fine there still is the problem with wrong bandwidth values (they would need to be much lower with usb active in my case).

kr, chris

 

[dave14305]

Failover to usb:

Jul 29 15:48:49 FlexQoS: /jffs/addons/flexqos/flexqos.sh (pid=10998) called with 2 args: -start usb0

That's why I thought it would be reinitialized correctly after a failover.

USB active:

admin@router:/tmp/home/root# tc qdisc ls | grep root
qdisc pfifo_fast 0: dev eth0 root refcnt 2 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: dev eth1 root refcnt 2 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: dev eth2 root refcnt 2 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: dev wl0.1 root refcnt 2 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: dev usb0 root refcnt 2 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1

No idea what it means but hopefully it helps your investigations
But even if initializiation would be fine there still is the problem with wrong bandwidth values (they would need to be much lower with usb active in my case).

kr, chris

This tells me Asus QoS didn't even try to apply after a failover, since all the qdisc are default pfifo_fast. So maybe QoS doesn't work in failover mode? Another test would be to increase debugging and do a failover again.

echo 2 > /proc/bw_qos_debug

Do a failover, copy syslog between start of failover and end of any DBG messages in syslog related to classes, filters, qdisc.

Then reset debug level with:

echo 1 > /proc/bw_qos_debug

 

[chris.at]

Your wish is my command

One comment to my first test: I disabled eth0 (ifconfig eth0 down) to simulate a dsl outage so maybe that's why in my first test the qdisc htb entries were missing while usb was active. Sorry if this was a mistake, I didn't know it better. For this test now I rebooted the dsl modem and did no other tricks on the router itself and this time there were no differences in qdisc output between eth0 and usb.

So here is the output you requested attached as a file (removed).

 

[dave14305]

An interesting wrinkle in my own QoS experience is that my company has recently started deploying Zscaler Proxy Agent to all workstations, so many of my outbound connections now go look like they're heading to a Zscaler IP instead of the real remote IP. It makes any remote IP based rules useless from this machine. But it still seems to be categorizing some things correctly, except for HTTP Proxy Server.

 

[brummygit]

I'm also using dual wan failover and if it fails over to usb lte stick flexqos gets reinitialized with wan port usb instead of eth, so it's still working, even if the up/down values don't match anymore. But since it's failover and normally only lasts some minutes that's not a big deal. Sure it would be nice to allow different bandwidth values for eth and usb wan if it's not too much work to implement it but at least for me that's quite unimportant.

Thanks, I'm using my Ethernet 1 for connection to a 4G router, but I had a failover today and the upload was not capped to the set bandwidth. My 4G failover had significantly higher upload so I wasn't too worried and in some ways it might be better not to have QoS capping at the wrong values

The bandwidth is stored in kbps in nvram, so you have to see how your value looks by running nvram get qos_obw

If you‘re storing fractional kilobits, I don’t think it’s going to make a difference if that gets rounded (says someone with 25 Mb up!).

I must have misunderstood originally as I see that setting 7.5Mbps as my upload does work - I'd thought you were rounding to 7Mbps but that's not what I see. Thanks.

Applied the update yesterday, enabled flushct and did a couple of 'random' restarts to see if it causes any issues - all my IOT devices are still running fine.

Same for me so far - looking good

 

[dave14305]

I must have misunderstood originally as I see that setting 7.5Mbps as my upload does work - I'd thought you were rounding to 7Mbps but that's not what I see. Thanks.

I am also much smarter now than I was before. I said before that the script will round down what you put in the GUI fields. I think that was before I paid close enough attention to the firmware behavior that converts those GUI values from Mb to kb before saving.

 

[Morris]

Hi Dave,

I decided to play around with the IP tables as you had suggested to address Youtube in Chrome Browser. I was able to get all traffic classified correctly. I'm fine tuning the IP ranges involved and there are a surprising number of them. I've been looking up the owner of the ranges and most are Google yet are few are not and I'm investigating this anomaly. I'm working on another project so it will take a few days. I'll keep you posted. The changes you have made seem to work well. I have noticed that when I make changes to the IP Table, existing streams are not effected by the change. Even with reset enabled. I simply close the Youtube tab and open it and then I get the desired results.

More to follow,

Morris

 

[Morris]

I am also much smarter now than I was before. I said before that the script will round down what you put in the GUI fields. I think that was before I paid close enough attention to the firmware behavior that converts those GUI values from Mb to kb before saving.

Glad you converted to using bits. I was getting getting tied of doing math to understand what was going on.

 

[dave14305]

Your wish is my command

One comment to my first test: I disabled eth0 (ifconfig eth0 down) to simulate a dsl outage so maybe that's why in my first test the qdisc htb entries were missing while usb was active. Sorry if this was a mistake, I didn't know it better. For this test now I rebooted the dsl modem and did no other tricks on the router itself and this time there were no differences in qdisc output between eth0 and usb.

So here is the output you requested attached as a file.

If you create a /jffs/scripts/wan-event script and observe in syslog the interface numbers and action parameters when called during a failover, you could use that script to change your QoS nvram bandwidth settings accordingly. Just an idea for you...

 

[chris.at]

I already have that script in use to send the wan failover events to me so that's a brilliant idea! As you took a look at the logs, is qos and qdisc set up correctly when failover occurs and should work with secondary wan? If so, it's enough to change qos_obw and qos_ibw variable (any idea what's their empty ...1 twin used for in nvram btw?), sounds nice and easy.

Edit: not sure if it works for both directions (let's wait for your analysis results) as my lte is rubbish (nearly no upload speed) but at least download got limited to the nvram value set.

 

[dave14305]

I already have that script in use to send the wan failover events to me so that's a brilliant idea! As you took a look at the logs, is qos and qdisc set up correctly when failover occurs and should work with secondary wan? If so, it's enough to change qos_obw and qos_ibw variable (any idea what's their empty ...1 twin used for in nvram btw?), sounds nice and easy.

Edit: not sure if it works for both directions (let's wait for your analysis results) as my lte is rubbish (nearly no upload speed) but at least download got limited to the nvram value set.

I don’t think QoS restarted after your failover, so maybe add a service "restart_qos;restart_firewall" to your wan-event script. The iptables firewall rules will get wiped by the QoS restart, so firewall must also be restarted.

The 1 nvram vars seem to be related to multiwan support, but I don’t know exactly what that’s about. It’s different from dual-wan support, and my router doesn’t support it.

 

[chris.at]

Interesting result, as there are flexqos lines in the log wouldn't this suggest qos got restarted? Maybe I should do it again with syslog level lowered to collect more detailled information... Will try it in a moment.

If it's relevant, I'm using a RT-AC68U with following values in wan-event script:

DSL (eth0):
nvram set qos_ibw=81920
nvram set qos_obw=15360

LTE (usb0):
nvram set qos_ibw=8192
nvram set qos_obw=1024

Edit: qos and firewall both get restarted in a failover event

Edit2: qos def. gets restarted because the "debug file" is missing shortly after each failover and then reinstalled with value 1. I immediately set it to 2 so this time the log includes really everything