[Release] FreshJR Adaptive QOS (Improvements / Custom Rules / and Inner workings)

[Kingp1n]

@FreshJR I'm trying to use the alternate script and I'm running into a minor problem. Within the script, I see the following under Iptable Up rules:

echo "Applying - Iptable Up Rules (ppp0)"

when I changed the rule here to reflect "eth0" as such:

echo "Applying - Iptable Up Rules (eth0)"

When I run the script, I keep getting the following message, when changing it to eth0:

Jul 25 18:45:02 adaptive QOS: Applying - Iptable Down Rules
Jul 25 18:45:02 adaptive QOS: Applying - Iptable Up Rules (eth0)
Jul 25 18:45:03 adaptive QOS: TC Modification Delayed Start (5min)
Jul 25 18:50:03 adaptive QOS: Applying TC Down Rules
Jul 25 18:50:03 adaptive QOS: /jffs/scripts/FreshJR_QOS: line 701: filter: not found

However when I changed it back to ppp0:

Jul 25 19:14:01 adaptive QOS: Applying - Iptable Down Rules
Jul 25 19:14:01 adaptive QOS: Applying - Iptable Up Rules (ppp0)
Jul 25 19:14:01 adaptive QOS: TC Modification Delayed Start (5min)
Jul 25 19:19:02 adaptive QOS: No modifications necessary

I don't think this is normal and maybe i'm not supposed to make a change here.

 

[FreshJR]

@FreshJR I'm trying to use the alternate script and I'm running into a minor problem. Within the script, I see the following under Iptable Up rules:

echo "Applying - Iptable Up Rules (ppp0)"

when I changed the rule here to reflect "eth0" as such:

echo "Applying - Iptable Up Rules (eth0)"

Why are you making this change? RMerlin automatically detects this variable, the override was only for users on STOCK firmware.

Anyway, the override should of only been made at the following location

#wan="ppp0"             ## WAN interface over-ride for upload traffic -- Variable ONLY needs to be defined for users on STOCK firmware
                        # RMerlin Firmware AUTOMATICALLY detects correct interface --> so variable should be left COMMENTED/DISABLED
                        # Stock Firmware configured to assume ppp0 --> variable should be manually set if connection type differs ( eth0, ppp0, vlanXXX, etc)

--

Also your error is completely unrelated to the iptables wan interface.

You have some malformed rule in the "tc_down_rules". It didn't show up in the 2nd run since you would have to restart QOS for the tc_down_rules to apply again.

 

[Kingp1n]

Im using it stock asus FW

 

[FreshJR]

Im using it stock asus FW

First get me the output of /jffs/scripts/FreshJR_QOS -debug

Second

-Turn off QOS
-Reupload the script with no modifications
-Turn on QOS
-Run /jffs/scripts/FreshJR_QOS -start
-Check for errors in the log

--

Something is wrong in "tc_down_rules" and I do not know if you made any modifications in that section.

 

[Kingp1n]

FreshJR

here is copy of the script,
https://pastebin.com/Ppfvri1a

you're right I totally screwed up something since the debug command is not giving me anything

jffs/scripts/FreshJR_QOS: line 701: set_all_varabiles: not found
adaptive QOS: Undf Prio: 2
adaptive QOS: Undf FlowID: 1:12
adaptive QOS: Classes Present:
adaptive QOS: Down Band:
adaptive QOS: Up Band :
adaptive QOS: ***********
adaptive QOS: Net =
adaptive QOS: VOIP =
adaptive QOS: Gaming =
adaptive QOS: Others =
adaptive QOS: Web =
adaptive QOS: Streaming =
adaptive QOS: Downloads =
adaptive QOS: Defaults =
adaptive QOS: ***********
adaptive QOS: Downrates -- , , , , , , ,
adaptive QOS: Downbursts -- , , , , , , ,
adaptive QOS: DownCbursts -- , , , , , , ,
adaptive QOS: ***********
adaptive QOS: Uprates -- , , , , , , ,
adaptive QOS: Upbursts -- , , , , , , ,
adaptive QOS: UpCbursts -- , , , , , , ,
admin@RT-AC68U:/tmp/home/root#

 

[FreshJR]

FreshJR

jffs/scripts/FreshJR_QOS: line 701: set_all_varabiles: not found

You fixed the spelling of the "set_all_varabiles" function.
(I did notice it was misspelled recently, but it was misspelled in every instance it was used, so it worked.)

You cannot just fix one instance of the misspelling, as programming is strict on matching variable/function names. Which is why what you thought was a typo "correction" made it not work.

Revert it back, but you might as well revert the entire script if you were making other changes you dont remember.

 

[jpclarke]

Hi,

I just noticed when I restart the qos service via script the iptables configuration disappears, does anyone else see this happening?

The command I am using to check is:
iptables --table mangle --list

To overcome the issue I need to reapply the iptables commands after the qos service is restarted.

Any thoughts as to why the iptables rules are being lost?

 

[FreshJR]

Hi,

I just noticed when I restart the qos service via script the iptables configuration disappears, does anyone else see this happening?

The command I am using to check is:
iptables --table mangle --list

To overcome the issue I need to reapply the iptables commands after the qos service is restarted.

Any thoughts as to why the iptables rules are being lost?

This is standard ASUS QOS behavior.

You need to both restart QOS and firewall in the following order (if doing it from a script).

service "restart_qos;restart_firewall"

If you observe ASUS behavior during initialization, it does the same.

 

[kvic]

I guess my question then becomes what are the underlying qdiscs adaptive qos, traditional qos, and freshJR's stuff uses?

Few years ago when I still used Asus routers, I looked at adaptive qos though not as in-depth as FreshJR.

It's vanilla linux qdisc. Can't recall if it's htb.. then paired with a "proprietary" TrendMicro module. Quite similar to how most ppl do with htb+fq_codel I believe but the fq_codel replaced by TrendMicro's.

"traditional qos" is vanilla linux qdisc but I think later Rmerlin back ported fq_codel from Tomato.

Recently I played with cake ported to Edgerouter by Lochnair. Very pleased with its performance. Was searching for a post to respond to here and found this one. Great stuff.

 

[Kingp1n]

Need some assistance, when Im currently using AC68R router with latest Merlim FW. My speeds when adaptive QOS is off are between 170/179 mbps. I used comcast and pay for the have 150/5mpbs speeds.

When i turn on adaptive QOS my speeds drop to 90-100mpbs.

Is this normal? If so, Sld i use the 80-95% bandwidth limits speeds from the average 175mbps speeds or from when I turn on QOS speeds of 100mpbs?

 

[FreshJR]

When i turn on adaptive QOS my speeds drop to 90-100mpbs.

Is this normal?

Put in very high limits and see what your router can push.

You might be limited to 100mbps down since you limited upload too much. (You need upload speed during a download). But 5mbps should of been sufficient.

If you can’t hit your speeds then don’t use QOS.

 

[Kingp1n]

Put in very high limits and see what your router can push.

You might be limited to 100mbps down since you limited upload too much. (You need upload speed during a download). But 5mbps should of been sufficient.

If you can’t hit your speeds then don’t use QOS.

Thanks FrshJR, ill play with both on/off but i do notice when its on my bufferbloat ratings are A/B rating not so great when its off.

 

[Sinner]

Need some assistance, when Im currently using AC68R router with latest Merlim FW. My speeds when adaptive QOS is off are between 170/179 mbps. I used comcast and pay for the have 150/5mpbs speeds.

When i turn on adaptive QOS my speeds drop to 90-100mpbs.

Is this normal? If so, Sld i use the 80-95% bandwidth limits speeds from the average 175mbps speeds or from when I turn on QOS speeds of 100mpbs?

You could also try using just sfq instead of fq_codel as i believe its a little less cpu intensive but doesnt perform quite as well for qos in most cases.

 

[Kingp1n]

Guys, does the log entry below mean my script is not working:

Jul 29 14:52:17 adaptive QOS: FreshJR FakeTC Installed
Jul 29 14:52:17 adaptive QOS: Parsing User QOS Rates -> (Updating)
Jul 29 14:52:31 rc_service: httpd 263:notify_rc restart_qos;restart_firewall
Jul 29 14:52:31 adaptive QOS: Clearing qos environment
Jul 29 14:52:46 kernel: ERR[parse_qos_conf:932] Can't set new QoS conf while QoS is started!
Jul 29 14:52:46 kernel: ERR[ioctl_iqos_op_config:3592] parse qos_conf error!!
Jul 29 14:52:46 kernel: ioctl_iqos_op_config() fail!
Jul 29 14:52:46 kernel: ERR[qos_start:3344] QoS is already started!
Jul 29 14:52:46 kernel: ioctl_iqos_op_switch(1) fail!
Jul 29 14:52:47 adaptive QOS: Clearing qos environment
Jul 29 14:52:47 adaptive QOS: Clearing qos environment
Jul 29 14:52:49 adaptive QOS: Changing 1:10 class rate
Jul 29 14:52:50 adaptive QOS: Changing 1:11 class rate
Jul 29 14:52:51 adaptive QOS: Changing 1:12 class rate
Jul 29 14:52:53 adaptive QOS: Changing 1:13 class rate
Jul 29 14:52:53 kernel: HTB: quantum of class 10013 is big. Consider r2q change.
Jul 29 14:52:55 adaptive QOS: Changing 1:14 class rate
Jul 29 14:52:58 adaptive QOS: Changing 1:15 class rate
Jul 29 14:52:59 adaptive QOS: Changing 1:16 class rate
Jul 29 14:53:00 adaptive QOS: Applying ------ Up Rules
Jul 29 14:53:00 adaptive QOS: Changing 1:17 class rate
Jul 29 14:53:01 adaptive QOS: Applying ---- Down Rules
Jul 29 14:53:17 adaptive QOS: *- Clearing Iptables -*
Jul 29 14:53:19 miniupnpd[30337]: shutting down MiniUPnPd
Jul 29 14:53:19 nat: apply nat rules (/tmp/nat_rules_eth0_eth0)
Jul 29 14:53:20 custom_script: Found firewall-start, but custom script execution is disabled!
Jul 29 14:53:20 miniupnpd[8156]: HTTP listening on port 48100
Jul 29 14:53:20 miniupnpd[8156]: Listening for NAT-PMP/PCP traffic on port 5351
Jul 29 14:53:21 miniupnpd[8156]: upnp_event_recv: recv(): Connection reset by peer
Jul 29 14:53:21 miniupnpd[8156]: upnpevents_processfds: 0x2b198, remove subscriber uuid:a411a9c1-feee-44dd-93df-41fa8c5a6e00 after an ERROR cb: http://192.168.1.97:2869/upnp/eventing/jpevsjzexg

 

[FreshJR]

It’s 80% working.

Jul 29 14:53:20 custom_script: Found firewall-start, but custom script execution is disabled!

Reread the prerequisites in the install instructions to gain the other 20%

 

[skeal]

Jul 29 14:53:20 custom_script: Found firewall-start, but custom script execution is disabled!

This means you need to: under administration tab make sure to allow the use of custom scripts.

 

[skeal]

Guys, does the log entry below mean my script is not working:

Jul 29 14:52:17 adaptive QOS: FreshJR FakeTC Installed
Jul 29 14:52:17 adaptive QOS: Parsing User QOS Rates -> (Updating)
Jul 29 14:52:31 rc_service: httpd 263:notify_rc restart_qos;restart_firewall
Jul 29 14:52:31 adaptive QOS: Clearing qos environment
Jul 29 14:52:46 kernel: ERR[parse_qos_conf:932] Can't set new QoS conf while QoS is started!
Jul 29 14:52:46 kernel: ERR[ioctl_iqos_op_config:3592] parse qos_conf error!!
Jul 29 14:52:46 kernel: ioctl_iqos_op_config() fail!
Jul 29 14:52:46 kernel: ERR[qos_start:3344] QoS is already started!
Jul 29 14:52:46 kernel: ioctl_iqos_op_switch(1) fail!
Jul 29 14:52:47 adaptive QOS: Clearing qos environment
Jul 29 14:52:47 adaptive QOS: Clearing qos environment
Jul 29 14:52:49 adaptive QOS: Changing 1:10 class rate
Jul 29 14:52:50 adaptive QOS: Changing 1:11 class rate
Jul 29 14:52:51 adaptive QOS: Changing 1:12 class rate
Jul 29 14:52:53 adaptive QOS: Changing 1:13 class rate
Jul 29 14:52:53 kernel: HTB: quantum of class 10013 is big. Consider r2q change.
Jul 29 14:52:55 adaptive QOS: Changing 1:14 class rate
Jul 29 14:52:58 adaptive QOS: Changing 1:15 class rate
Jul 29 14:52:59 adaptive QOS: Changing 1:16 class rate
Jul 29 14:53:00 adaptive QOS: Applying ------ Up Rules
Jul 29 14:53:00 adaptive QOS: Changing 1:17 class rate
Jul 29 14:53:01 adaptive QOS: Applying ---- Down Rules
Jul 29 14:53:17 adaptive QOS: *- Clearing Iptables -*
Jul 29 14:53:19 miniupnpd[30337]: shutting down MiniUPnPd
Jul 29 14:53:19 nat: apply nat rules (/tmp/nat_rules_eth0_eth0)
Jul 29 14:53:20 custom_script: Found firewall-start, but custom script execution is disabled!
Jul 29 14:53:20 miniupnpd[8156]: HTTP listening on port 48100
Jul 29 14:53:20 miniupnpd[8156]: Listening for NAT-PMP/PCP traffic on port 5351
Jul 29 14:53:21 miniupnpd[8156]: upnp_event_recv: recv(): Connection reset by peer
Jul 29 14:53:21 miniupnpd[8156]: upnpevents_processfds: 0x2b198, remove subscriber uuid:a411a9c1-feee-44dd-93df-41fa8c5a6e00 after an ERROR cb: http://192.168.1.97:2869/upnp/eventing/jpevsjzexg

It looks like this:
Enable JFFS custom scripts and configs Yes No

 

[Kingp1n]

Thanks Skeal, i upgraded to the latest FW, and I forgot to enabled this back, thanks again

 

[FreshJR]

If so, Sld i use the 80-95% bandwidth limits speeds from the average 175mbps speeds or from when I turn on QOS speeds of 100mpbs?

If you are not able to reach your ISP's sustainable bandwidth of 175mbps, you should still input 85-95% of the 175mbps value, so like ~160mbps.

You would NOT do 85-95% of the 100mbps cpu limited rate.

Side note:

Is hardware acceleration enabled??

Tools --> HW acceleration --> Enabled (CTF only)
​

If not, restart your router, and see if HW acceleration enables itself.
(If you temporarily checked a some setting that disables HW acceleration and then even if unchecked it later, HW acceleration will still remain disabled until a restart)

**restart means power off and power on. It does not mean factory reset.

Thanks FrshJR, ill play with both on/off but i do notice when its on my bufferbloat ratings are A/B rating not so great when its off.

I am not sure how good your bufferbloat will be if your CPU is maxing out at 100% with QoS enabled, but if you see an improvement, that's your call.

What is your quality grade when CPU limited?

Are you using 100mbps ethernet cable. The number is very close that limit.

 

[dtaht]

Thanks for all your contributions!!! Big fan.

I watched a couple YouTube videos where gave you explanations and even the one with the bucket demonstration . Very informative!

I fully agree where you can have a single htb, to perform limiting, and have a single fq-codel disc per user. This leads to solid performance across the board, without the need for deep packet analysis.

ASUS is approached it from a different angle.

They have a HTB to limit traffic across categories and then they have another HTB within each category to separate traffic per user.

This essentially gives each user 8 queuing disciplines each instead of 1.

It works well for the most part but the cascaded HTB's have a poor configuration, in my opinion, so users within each category cannot be allotted bandwidth in different proportions between each other, effectively.

ASUS is hellbent on making NOTHING adjustable, so this script is simply a hacky workaround to give us partial flexibility in the initial HTB configuration and additional rules for the DPI categorization. Regrettably it is not a full standalone solution.



The underlying configuration is cemented as to how Asus has configured Adaptive QOS (this scripts is simply surface modifications on top of it). Even the underlying schedulers are restricted from configuration when using AdaptiveQOS.

To actually implement CoDel, RMerlin had to rename the TC executable into RealTC, have Asus send their configuration what they thought was TC, and then modify those commands on the fly by forwarding them to the actual executable named RealTC.

We do have full flexibility for Traditional QOS but we then would have to forego the use of the DPI engine.

So in the end we do not have much freedom on this side of the pond.

For your question

Out of the box, Asus only offers the outdated SFQ
With RMerlin's mods, we have the choice between SFQ, Codel, fq-Codel

You already heard about our kernel limitations.

--

Once again, Thank You for your contributions!

Thank you for trying to make the Net better on this side!

For the record the entertaining videos using water bottles as the core analogy were mine at the university of modena and Stephen Hemmingers at some linux con or another. Stephen's is both shorter and funnier:

https://www.bufferbloat.net/projects/cerowrt/wiki/Bloat-videos/

We try to convey intuition about how packets really interact.

I note that we are old farts and rely on a mailing list for most of our discussions on bufferbloat.net. Stuff from there seems rarely googlable but the lists are archived and joinable at at https://lists.bufferbloat.net if you want to join "bloat", "cake", "make-wifi-fast", or "cerowrt-devel;". There is also a #bufferbloat irc channel.

Thx for the info about adaptive qos and what you struggle with here.. I am glad to hear that the current broadcom devkit is 4.1 based, at least.