[Release] FreshJR Adaptive QOS (Improvements / Custom Rules / and Inner workings)

[jpclarke]

Hi,

Would a qos restart via script update the values once a scripted speedtest updates the up and down rates?

I have implemented the qos restart after the rates are updated but any help on how to check the figures have actually been updated would be appreciated.

 

[FreshJR]

Hi,

Would a qos restart via script update the values once a scripted speedtest updates the up and down rates?

I have implemented the qos restart after the rates are updated but any help on how to check the figures have actually been updated would be appreciated.

Your method is definitely a waste of resources ... just reissue the 14 tc class change commands.

To check current rates you can do

tc class show dev br0 | grep "parent 1:1"
tc class show dev eth0 | grep "parent 1:1"

I told you can reference the syntax from the old version of the script.

        tc class change dev br0 parent 1:1 classid 1:10 htb prio 0 rate ${DownRate0}Kbit ceil ${DownCeil}Kbit burst ${DownBurst0} cburst ${DownCburst0}
        tc class change dev br0 parent 1:1 classid 1:11 htb prio 1 rate ${DownRate1}Kbit ceil ${DownCeil}Kbit burst ${DownBurst1} cburst ${DownCburst1}
        tc class change dev br0 parent 1:1 classid 1:12 htb prio 2 rate ${DownRate2}Kbit ceil ${DownCeil}Kbit burst ${DownBurst2} cburst ${DownCburst2}
        tc class change dev br0 parent 1:1 classid 1:13 htb prio 3 rate ${DownRate3}Kbit ceil ${DownCeil}Kbit burst ${DownBurst3} cburst ${DownCburst3}
        tc class change dev br0 parent 1:1 classid 1:14 htb prio 4 rate ${DownRate4}Kbit ceil ${DownCeil}Kbit burst ${DownBurst4} cburst ${DownCburst4}
        tc class change dev br0 parent 1:1 classid 1:15 htb prio 5 rate ${DownRate5}Kbit ceil ${DownCeil}Kbit burst ${DownBurst5} cburst ${DownCburst5}
        tc class change dev br0 parent 1:1 classid 1:16 htb prio 6 rate ${DownRate6}Kbit ceil ${DownCeil}Kbit burst ${DownBurst6} cburst ${DownCburst6}
        tc class change dev br0 parent 1:1 classid 1:17 htb prio 7 rate ${DownRate7}Kbit ceil ${DownCeil}Kbit burst ${DownBurst7} cburst ${DownCburst7}
 

        tc class change dev eth0 parent 1:1 classid 1:10 htb prio 0 rate ${UpRate0}Kbit ceil ${UpCeil}Kbit burst ${UpBurst0} cburst ${UpCburst0}
        tc class change dev eth0 parent 1:1 classid 1:11 htb prio 1 rate ${UpRate1}Kbit ceil ${UpCeil}Kbit burst ${UpBurst1} cburst ${UpCburst1}
        tc class change dev eth0 parent 1:1 classid 1:12 htb prio 2 rate ${UpRate2}Kbit ceil ${UpCeil}Kbit burst ${UpBurst2} cburst ${UpCburst2}
        tc class change dev eth0 parent 1:1 classid 1:13 htb prio 3 rate ${UpRate3}Kbit ceil ${UpCeil}Kbit burst ${UpBurst3} cburst ${UpCburst3}
        tc class change dev eth0 parent 1:1 classid 1:14 htb prio 4 rate ${UpRate4}Kbit ceil ${UpCeil}Kbit burst ${UpBurst4} cburst ${UpCburst4}
        tc class change dev eth0 parent 1:1 classid 1:15 htb prio 5 rate ${UpRate5}Kbit ceil ${UpCeil}Kbit burst ${UpBurst5} cburst ${UpCburst5}
        tc class change dev eth0 parent 1:1 classid 1:16 htb prio 6 rate ${UpRate6}Kbit ceil ${UpCeil}Kbit burst ${UpBurst6} cburst ${UpCburst6}
        tc class change dev eth0 parent 1:1 classid 1:17 htb prio 7 rate ${UpRate7}Kbit ceil ${UpCeil}Kbit burst ${UpBurst7} cburst ${UpCburst7}

So as to the undefined variables per call

Rate can actually be whatever, as FreshJR_QOS_fakeTC will over write it to calulcated values. So just plug in a placeholder like 2048kbit.

Do keep in mind after you update your nvram qos_ibw & qos_obw varaibles from your script, you have to force FreshJR_QOS update recalculate the class rates stored in NVRAM so it doesn't use the old values.

You can force it to do so by

/jffs/scripts/FreshJR_QOS -init

You already have Ceil from your speed test.

So really the only thing you have left undefined is the burst/cburst defined values.
You can hardcode the existing values that ASUS somehow calculates (I have not found an exact correlation). Additionally, as a side note, I have honestly thought about increasing burst/cburst values for gaming and VOIP categories on my personal setup.

If you leave out of the TC class change command, the TC program also chooses some default value to substitute in aswell.


Your current method of restarting QOS seems like too much of an overkill. Starting QOS runs like 182 calls to TC and takes like 3 minutes!

 

[Gingernut]

@Gingernut

Here are my results after disabling FreshJR_QOS

Fresh BOOT (QOS NOT DONE SETTING ITSELF UP)

Feb 22 21:53:43 qos: Applying codel patch
Feb 22 21:53:44 kernel: Init chrdev /dev/detector with major 190
Feb 22 21:53:44 kernel: tdts: tcp_conn_max = 8000
Feb 22 21:53:44 kernel: tdts: tcp_conn_timeout = 300 sec
Feb 22 21:53:59 kernel: SHN Release Version: 2.0.1 3529123_patch
Feb 22 21:53:59 kernel: UDB Core Version: 0.2.14 r3529123
Feb 22 21:53:59 kernel: Init chrdev /dev/idpfw with major 191
Feb 22 21:54:00 kernel: IDPfw: IDPfw is ready
Feb 22 21:54:00 kernel: sizeof forward pkt param = 192
Feb 22 21:54:05 kernel: ERR[qos_start:3356] qos_ops is not registered!
Feb 22 21:54:05 kernel: ioctl_iqos_op_switch(1) fail!
Feb 22 21:54:05 kernel: ERR[qos_start:3356] qos_ops is not registered!
Feb 22 21:54:05 kernel: ioctl_iqos_op_switch(1) fail!
Feb 22 21:54:05 rc_service: udhcpc 392:notify_rc start_firewall
Feb 22 21:54:06 miniupnpd[535]: shutting down MiniUPnPd

Toggle QOS (OFF -> ON) -- messages present but not repeatable upon every toggle

Feb 22 21:56:14 rc_service: httpd 282:notify_rc restart_qos;restart_firewall
Feb 22 21:56:35 kernel: ERR[parse_qos_conf:932] Can't set new QoS conf while QoS is started!
Feb 22 21:56:35 kernel: ERR[ioctl_iqos_op_config:3592] parse qos_conf error!!
Feb 22 21:56:35 kernel: ioctl_iqos_op_config() fail!
Feb 22 21:56:35 kernel: ERR[qos_start:3344] QoS is already started!
Feb 22 21:56:35 kernel: ioctl_iqos_op_switch(1) fail!

Toggle QOS (ON -> OFF) -- no messages

Feb 22 21:58:28 rc_service: httpd 282:notify_rc restart_qos;restart_firewall
Feb 22 21:58:49 miniupnpd[1630]: shutting down MiniUPnPd
Feb 22 21:58:49 nat: apply nat rules (/tmp/nat_rules_eth0_eth0)
Feb 22 21:58:49 custom_script: Running /jffs/scripts/firewall-start (args: eth0)
Feb 22 21:58:49 miniupnpd[14019]: HTTP listening on port 57778
Feb 22 21:58:49 miniupnpd[14019]: Listening for NAT-PMP/PCP traffic on port 5351

I still have more debugging ideas, but I am not entirely convinced these messages are coming from my end, since the messages are present without the script running and even a reboot after the removal of the script.

Thanks for the continued investigation and time you put into this project.

The problem was that if I didn't reboot, the toogle on/off - off/on didn't work as the restart_qos rc service was ignored/canceled for some reason.

I'm not blaiming your script in any way, it's just without it this doesn't happen.

The qos service does seem very flakey on the new 384 branch and even more so on the new hnd platform.

 

[jpclarke]

Hi,

I added a new function to simply update the nvram rates and then apply the tc commands as FreshJR recommended.

I am also pulling through the burst and cburst rates applied by ASUS by reading these in, is there a better way to calculate these burst and cburst rates to achieve the best performance for the rates available?

Currently there doesn’t appear to be a simple correlation between rate, category and burst/cburst as two categories with the same rate get different burst rates applied by ASUS.

Thanks and great work on this script!

 

[FreshJR]

Hi,

I added a new function to simply update the nvram rates and then apply the tc commands as FreshJR recommended.

I am also pulling through the burst and cburst rates applied by ASUS by reading these in, is there a better way to calculate these burst and cburst rates to achieve the best performance for the rates available?

Currently there doesn’t appear to be a simple correlation between rate, category and burst/cburst as two categories with the same rate get different burst rates applied by ASUS.

Thanks and great work on this script!

I think from my quick investigation I saw VOIP gets a slightly higher burst, so that could be the discrepancy.

I found somewhat of a correlation, but it doesn't hold true 1:1 for all rates I input, but it does get close.
I am wondering if its a rounding truncation within ASUS's code that prevents a 1:1 match.

So yup, no idea on the correlation.

 

[FreshJR]

One thing I have noticed is that the script fails when the router is rebooted

@jpclarke,

I have finally been able to recreate this issue. It was a race condition between RMerlin and FreshJR mounting the fakeTC script upon bootup. Usually RMerlin's would mount first and then I would overwrite it, but I have been able to have the events happen in reverse order in the logs shown below.

Feb 26 23:23:24 custom_script: Running /jffs/scripts/firewall-start (args: eth0)
Feb 26 23:23:27 adaptive QOS: FreshJR FakeTC Started
Feb 26 23:23:27 qos: Applying codel patch

<< RMerlin's fake TC overwrote FreshJR fake TC due to race condition on boot>>

Feb 26 23:24:18 rc_service: udhcpc 435:notify_rc start_firewall
Feb 26 23:24:21 custom_script: Running /jffs/scripts/firewall-start (args: eth0)
Feb 26 23:24:22 adaptive QOS: FreshJR FakeTC Started

<< FreshJR fake TC overwrote RMerlin's fake TC at point in time shortly after, but some QOS commands already been missed at this point >>

Feb 26 23:24:23 adaptive QOS: Parsing QOS config ->  (No Change Required)
Feb 26 23:24:25 adaptive QOS: Changing 1:15 class rate
Feb 26 23:24:28 adaptive QOS: Changing 1:16 class rate
Feb 26 23:24:30 adaptive QOS: Applying Custom Upload Rules
Feb 26 23:24:32 adaptive QOS: Changing 1:17 class rate
Feb 26 23:24:33 adaptive QOS: Applying Custom Download Rules

I have a fix ready and working for this, but will test it for another day or so.

The current approach is to prevent RMerlin fakeTC from being mounted in the first place to avoid this race condition. If that proves to introduce errors, another solution will be to enable QOS twice.

 

[FreshJR]

Here is BETA6, which fixes a race condition where sometimes the script would not start properly after a system boot/reboot.
ASUS's kernel errors are still present. No more debugging will be done as they happen without the script running, so they are not coming from me, and if they are!! then I am out of ideas as I went over every section of code multiple times. I spent a considerable amount of time checking everything.

It was freshly tested on the v384.4 beta1 released just today.

This will most likely be the final version, unless something terrible pops up in the upcomming weeks.

Enjoy!

Prerequisites:

You will need putty and pscp, as well as the script. Also you need to be on fw 382+

Spoiler

http://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html

You do not have to install these executables, they are portable. Move them, and the script, into a new folder called Putty on your C:\ drive.

It should look like this:

• C:\Putty\putty.exe
• C:\Putty\pscp.exe
• C:\Putty\FreshJR_QOS_beta6.txt
• C:\Putty\FreshJR_QOS_fakeTC_beta6.txt

Install/Update Guide:
1) turn OFF qos
2) In command prompt

cd C:\putty\
pscp -scp C:\Putty\FreshJR_QOS_beta6.txt admin@192.168.1.1:/jffs/scripts/FreshJR_QOS
pscp -scp C:\Putty\FreshJR_QOS_fakeTC_beta6.txt admin@192.168.1.1:/jffs/scripts/FreshJR_QOS_fakeTC

3) In putty

dos2unix /jffs/scripts/FreshJR_QOS
dos2unix /jffs/scripts/FreshJR_QOS_fakeTC
chmod 0755 /jffs/scripts/FreshJR_QOS
chmod 0755 /jffs/scripts/FreshJR_QOS_fakeTC
sed -i '/FreshJR_QOS/d' /jffs/scripts/firewall-start
/jffs/scripts/FreshJR_QOS install

4) turn ON qos


<< OUTDATED VERSION, NEWER RELEASE AVAILABLE >>

 

[Quoc Huynh]

Thank you very much for your wonderful work, FreshJR

 

[Gingernut]

Updated to latest beta from RMerlin & your beta6 release.

After a minute or two QOS is working fine & without errors present in log.

Thank you.

 

[Sebastien Bougie]

Hello,

I just install the new script and i'm getting

Feb 28 07:31:25 kernel: ioctl_iqos_op_config() fail!
Feb 28 07:31:25 kernel: ERR[qos_start:3344] QoS is already started!
Feb 28 07:31:25 kernel: ioctl_iqos_op_switch(1) fail!
Feb 28 07:31:28 kernel: ERR[parse_qos_conf:932] Can't set new QoS conf while QoS is started!
Feb 28 07:31:28 kernel: ERR[ioctl_iqos_op_config:3592] parse qos_conf error!!
Feb 28 07:31:28 kernel: ioctl_iqos_op_config() fail!
Feb 28 07:31:28 kernel: ERR[qos_start:3344] QoS is already started!
Feb 28 07:31:28 kernel: ioctl_iqos_op_switch(1) fail!
Feb 28 07:31:32 kernel: ERR[parse_qos_conf:932] Can't set new QoS conf while QoS is started!
Feb 28 07:31:32 kernel: ERR[ioctl_iqos_op_config:3592] parse qos_conf error!!
Feb 28 07:31:32 kernel: ioctl_iqos_op_config() fail!
Feb 28 07:31:32 kernel: ERR[qos_start:3344] QoS is already started!
Feb 28 07:31:32 kernel: ioctl_iqos_op_switch(1) fail!


Waited 20 mins, reboot same problem, disable QOS of course the message stop. after a couple of minutes i got
Feb 28 07:37:48 adaptive QOS: FreshJR fakeTC Running (No Change Required)
Feb 28 07:37:48 adaptive QOS: Parsing QOS config -> (No Change Required)

But when i turn back on the QOS the message is back

any ideas?

 

[Sinner]

Hello,

I just install the new script and i'm getting

Feb 28 07:31:25 kernel: ioctl_iqos_op_config() fail!
Feb 28 07:31:25 kernel: ERR[qos_start:3344] QoS is already started!
Feb 28 07:31:25 kernel: ioctl_iqos_op_switch(1) fail!
Feb 28 07:31:28 kernel: ERR[parse_qos_conf:932] Can't set new QoS conf while QoS is started!
Feb 28 07:31:28 kernel: ERR[ioctl_iqos_op_config:3592] parse qos_conf error!!
Feb 28 07:31:28 kernel: ioctl_iqos_op_config() fail!
Feb 28 07:31:28 kernel: ERR[qos_start:3344] QoS is already started!
Feb 28 07:31:28 kernel: ioctl_iqos_op_switch(1) fail!
Feb 28 07:31:32 kernel: ERR[parse_qos_conf:932] Can't set new QoS conf while QoS is started!
Feb 28 07:31:32 kernel: ERR[ioctl_iqos_op_config:3592] parse qos_conf error!!
Feb 28 07:31:32 kernel: ioctl_iqos_op_config() fail!
Feb 28 07:31:32 kernel: ERR[qos_start:3344] QoS is already started!
Feb 28 07:31:32 kernel: ioctl_iqos_op_switch(1) fail!


Waited 20 mins, reboot same problem, disable QOS of course the message stop. after a couple of minutes i got
Feb 28 07:37:48 adaptive QOS: FreshJR fakeTC Running (No Change Required)
Feb 28 07:37:48 adaptive QOS: Parsing QOS config -> (No Change Required)

But when i turn back on the QOS the message is back

any ideas?

I get the exact same thing on my 3100 running 343 stable. Ive been trying to figure it out and based on what ive read i think thats rmerlin faketc and freshJR faketc battling it out at the same time. Ive been told you can ingore those errors and from what i can tell the script is actually working perfectly. If you look at the qos stats page and no data on default its working.

 

[maxbraketorque]

Correct. The automatic setting does not work. Do not use it.

For working QOS you have to manually define your limits in the web interface.

It is 80-90% of actual speeds on both upload and download. It varies between ISPs. Experiment with values while checking your buffer bloat.

If your lazy set it to 80% and call it a day.

For me I got away with 93% on download but fell all the way to 80% on upload.

...

I have Comcast 250/12 service, but my bandwidth varies throughout the day. A speedtest to Comcast's servers will get me 265/14 at night, but by midday, it can drop to <200/12. If I set the QOS bandwidth limit to 180/11, then anytime my actual bandwidth is above the limit set in QOS, will QOS limit my bandwidth to the set values and cause me to give up available bandwidth? Sorry if this question has already been asked. I don't have time to read through all 45 pages.

 

[FreshJR]

@Gingernut, @Sinner, @Sebastien Bougie, & @everyone else complaining about the errors

I have bolded a statement that was present in the post releasing beta6.

@maxbraketorque

Yes it WILL limit your internet speeds to the rates you input into to the manual bandwidth setting.

You have 2 options.

(1) Set your rates to 250/12 (I would actually recommend 235/10.5, as you are supposed to go slightly under your max). This will provide traffic prioritization, BUT bufferbloat will exist during the periods your supplied bandwidth is lower than these values.

(2) Set it to 180/11, and never receive buffer bloat.

I find option2 wasteful, as you lose significant bandwidth exhibiting poor network performance when your speeds available are higher than the limit you input since QOS will be dropping many packets unnecessarily.

Option1 seems like the significantly better solution, but it's your call. Experiment and see.

Either way at least you will have prioritization. It seems like your node is conjected, so if you go with less bandwidth, it just means one of your neighbors will just get more.

 

[Sinner]

anyone know of a command I could use to determine what ports are being directed where in putty. like for instance I'm curious if 5223(apple push notifications) and 5228(google play and similar messaging service) are being directed to VOIP or somewhere else already possibly by the FW or this script? Its difficult to generate events for these to determine if theyre working.

 

[maxbraketorque]

...

@maxbraketorque

Yes it WILL limit your internet speeds to the rates you input into to the manual bandwidth setting.

You have 2 options.

(1) Set your rates to 250/12 (I would actually recommend 235/10.5, as you are supposed to go slightly under your max). This will provide traffic prioritization, BUT bufferbloat will exist during the periods your supplied bandwidth is lower than these values.

(2) Set it to 180/11, and never receive buffer bloat.

I find option2 wasteful, as you lose significant bandwidth exhibiting poor network performance when your speeds available are higher than the limit you input since QOS will be dropping many packets unnecessarily.

Option1 seems like the significantly better solution, but it's your call. Experiment and see.

Either way at least you will have prioritization. It seems like your node is conjected, so if you go with less bandwidth, it just means one of your neighbors will just get more.

Thanks much. I'll see how it goes with the higher settings.

 

[FreshJR]

anyone know of a command I could use to determine what ports are being directed where in putty. like for instance I'm curious if 5223(apple push notifications) and 5228(google play and similar messaging service) are being directed to VOIP or somewhere else already possibly by the FW or this script? Its difficult to generate events for these to determine if theyre working.

I found the following to work:

You can turn on app analysis and check what name the bwdpi engine is identifying that devices traffic.

Then you cross reference that identified name with the the named entry in /tmp/bwdpi/bwdpi.app.db

then the first digit beside this named entry should give you what QOS catagory ASUS is sticking that identified traffic into. You can cross reference this digit with RMerlins post #3 in this thread OR you can change the hex matches into decimal and see what flowid that hex digit is pumping traffic into.

----

Another thing you can investigate is the output of

cat /proc/net/nf_conntrack | grep 192.168.100.1


As that will show you the packet marks, I have no idea at what point in the chain these marks have been made as they do not match up into 0x8000 and 0x4000 expected format that TC is filtering on.

Feel free to experiment more and return your findings.

--

Or just for fun, checking the app.db file shows they identify a NICE chunk of traffic using deep packet inspection.

--

The simplest way is to produce traffic and see where it is being directed too via QOS Statistics. If the traffic does not use much bandwidth, I wouldn't worry too much about it.

--

For your original question, it seems iMessages is an entry and it should be assigned a traffic type 0 mark. Traffic type 0 matches the prio 3/ pref 3 / 3rd rule.

filter parent 1: protocol all pref 3 u32 fh 805::800 order 2048 key ht 805 bkt 0 flowid 1:11
  mark 0x80000000 0xc03f0000

I doubt they are doing filter by port usage.

--

After some experimenting I found out that you can create rules to modify the default destination of the specific applications identified within app analysis. Lets go over two examples

From /tmp/bwdpi/bwdpi.app.db
4,132,0,Netflix
13,7,0,Speedtest.net

netflix
general category: 4dec = 04 hex
specific identification: 132dec = 0084 hex

speedtest
general category: 13dec = 0d hex
specific identification: 7dec = 0007 hex

so

realtc filter add dev br0 protocol all prio 2 u32 match mark 0x80040084 0xc03fffff flowid ${Default}     # will stick netflix into defaults

realtc filter add dev br0 protocol all prio 2 u32 match mark 0x800d0007 0xc03fffff flowid ${Default}     # will stick speedtest.net into defaults

I don't like too many rules, but if some app is causing havoc **cough speedtest.net cough**, you can stick it into defaults/filedownloads instead of web browsing.

The general form goes like this

8# 0d 0007 = (Download, Web Browsing, Speedtest)

where

8 = traffic direction [ download (8) or upload (4) ]
# = unknown so far
0d = traffic type (see post 3 or bwdpi.app.db)
0007 = specific application/traffic within category (see bwdpi.app.db )

Note: numbers in TC are in hex. Numbers in post#3 or "bwdpi.app.db" are in dec. Conversion is necessary.

--

With this I might look into removing custom rules 22/23. Instead I might a specific case to detect the following

19,94,0,HTTP Protocol over TLS SSL
20,185,0,SSL/TLS

into the regular custom rules. I will be directing this into Web Surfing instead of Net Control.

 

[Sinner]

wow ty for this... I also found these commands in another thread I think may be of some use. I'm making a list of useful stuff at the moment and trying to make sense of it all
tc class show dev br0 | grep "parent 1:1"
tc class show dev eth0 | grep "parent 1:1"
tc -s -d qdisc

 

[FreshJR]

perhaps whats most interesting are the new tc rules for marks

0x801c -> 0x802b  (which is 28-43 decimal)

as there are no corresponding app/traffic names starting with 28-43 yet in the app analysis database.

These new tc rules all point into Web Surfing. I wonder what they are planning with this.

 

[FreshJR]

RMerlin figured out what is causing the errors. There will be an update that should REDUCE the chance of errors.

 

[1219chicho]

RMerlin figured out what is causing the errors. There will be an update that should REDUCE the chance of errors.

That's some great news