[Release] FreshJR Adaptive QOS (Improvements / Custom Rules / and Inner workings)

[bbunge]

Sorry if this is covered somewhere.

This appeared in my log after a QOS restart:
Aug 19 08:15:10 admin: Adaptive QOS: Modification Script Started
Aug 19 08:15:41 admin: Adaptive QOS: Changing container for Unidentified Traffic & Applying Custom Rules
Aug 19 08:15:41 admin: Adaptive QOS: Changing minimum alloted bandwidth per QOS category to user defined percentages
Aug 19 08:15:41 kernel: HTB: quantum of class 10010 is small. Consider r2q change.
Aug 19 08:15:41 kernel: HTB: quantum of class 10015 is small. Consider r2q change.
Aug 19 08:15:41 kernel: HTB: quantum of class 10017 is small. Consider r2q change.

Potential problem? Anything I need to change?
RT-AC66U B1 Merlin 380.68 Beta1

 

[Vexira]

The quantum is just a warning. Your internet speed is so fast that the calculated quantum is so large that it freaks out and gives you a false warning.

Correct on your rule functionality.

For r2q explanation read this post from 2 pages back. In short, its not an error!

https://www.snbforums.com/threads/r...-and-inner-workings.36836/page-12#post-341122

But rather your internet faster than it expects.

Try following this analogy:

Think of quantum as the size of food bites you are eating. Think of your internet speed as to how fast you can eat. To not be scooping little food pebbles one at a time super fast with your spoon, you simply can change quantum and eat bigger chunks of food.

TC automatically calculates quantum based on rate. The resulting quantom was food expected to be measured in the size from pebbles to rocks. Now a days, some peoples internet is so fast that the calculated value amounts to them eating boundlers with a backhoe.

Just a warning letting you know that you are eating boulders.

What this means is that the granularity of qos bandwidth allocation is course (boulder size). In reality, the bandwidth rates are fast that the coarseness of the food size is not an issue.
It is not a problem for you to start eating pebbles again and just shovel your spoon at a significantly faster rate. This would reintroduce more granularity.

This r2q variable to compute quantum based on rate is defined within the TC application. You would have to recompile your own version if you want to change the auto calculated quantum values.

Merlin has a TC interposer to modify commands sent to ASUS's TC and forward them to his version of TC. This would allow you to change quantum's without recompiling TC. If you modify his interposer functionality you can manually define quantum for each class instead of using the calculated values based on r2q.

The interposer is located in /usr/sbin/faketc

I am not going to be changing variables at this level. Feel free to experiment

I was under the assumption to fix the quantum error all that was needed was to adjust the r2q, to a value equal to the mtu, but under the hard coded quantum limit, feel free to correct me if I'm wrong.

 

[Vexira]

Sorry if this is covered somewhere.

This appeared in my log after a QOS restart:
Aug 19 08:15:10 admin: Adaptive QOS: Modification Script Started
Aug 19 08:15:41 admin: Adaptive QOS: Changing container for Unidentified Traffic & Applying Custom Rules
Aug 19 08:15:41 admin: Adaptive QOS: Changing minimum alloted bandwidth per QOS category to user defined percentages
Aug 19 08:15:41 kernel: HTB: quantum of class 10010 is small. Consider r2q change.
Aug 19 08:15:41 kernel: HTB: quantum of class 10015 is small. Consider r2q change.
Aug 19 08:15:41 kernel: HTB: quantum of class 10017 is small. Consider r2q change.

Potential problem? Anything I need to change?
RT-AC66U B1 Merlin 380.68 Beta1

its nothing to technically worry about, read wicked shrapnel's post about quantum its more an inconvenice than a problem, but there is a way to "fix it", which involves modding the r2q value, otherwise don't worry about it it wont harm any thing. I quoted his post if you go back a bit.
but this is what he posted:

Short answer, here is how you fix it. They fixed it in Redhat. https://www.redhat.com/archives/libvir-list/2016-February/msg00125.html <- way above my pay grade.

Long answer below

What quantum it is.

"It might be good time to touch concept of quantums now. In fact when more classes want to borrow bandwidth they are each given some number of bytes before serving other competing class. This number is called quantum. You should see that if several classes are competing for parent's bandwidth then they get it in proportion of their quantums. It is important to know that for precise operation quantums need to be as small as possible and larger than MTU.
Normaly you don't need to specify quantums manualy as HTB chooses precomputed values. It computes classe's quantum (when you add or change it) as its rate divided by r2q global parameter. Its default value is 10 and because typical MTU is 1500 the default is good for rates from 15 kBps (120 kbit). For smaller minimal rates specify r2q 1 when creating qdisc - it is good from 12 kbit which should be enough. If you will need you can specify quantum manualy when adding or changing the class. You can avoid warnings in log if precomputed value would be bad. When you specify quantum on command line the r2q is ignored for that class."

Source: http://luxik.cdi.cz/~devik/qos/htb/manual/userg.htm

Basically I think the issue is this algorithm was written at a time when bandwidth speeds in the hundreds of megabytes or gigabit wasn't feasible and letting r2q calculate the quantum value freaks it out when that number is so high.

If rate is less than 80 the quantum is small warning is provided.
If rate is greater than 16000 quantum is big warning is provided

Quantum is calculated as rate (in bytes) / r2q.
r2q is by default 10

For example, my download rate is set at 300Mb you take that (in bytes) divided by the default r2q of 10 and that equals 3,750,000 bytes for the quantum. The hard coded limit of quantum is 60,000 so it throws a warning.

So in order to prevent these warnings for my 300Mb connection, my r2q needs to be 20,000. That will keep the quantum under 16,000 where it cries but I think it would need to be even a bigger number to get my quantum closer to my MTU of 1500. Changing the r2q value would be the easiest option I think since there are a lot of classes and that would suck to change all of them manually.

There are a lot of classes, you can see them by running the following command
Code:
tc class show dev eth0
The class in the log " HTB: quantum of class 10011 is big. Consider r2q change." will be shown in the command output truncated like "class htb 1:11" they substitute a : in place of the zeros.

 

[FreshJR]

I was under the assumption to fix the quantum error all that was needed was to adjust the r2q, to a value equal to the mtu, but under the hard coded quantum limit, feel free to correct me if I'm wrong.

No you do NOT set r2q = mtu.

Your minimum rate(bytes)/ r2q >= mtu

So our r2q of 10 is good for rates of with 15kB/s as a minimum since MTU is 1500.

If you are able to change r2q I wouldn't. Asus has some subclasses set to 128kbps.
15kB/s = 120 kbps. You cannot even set r2q to 11 without breaking those classes.

Getting back on topic. It is not a ERROR. Just a warning telling you to consider sorting packets with a finer screen.

If you want a finer screen use the interposer and change the default calculated Quantum's manually upon class creation. I have made some, different, modifications to my script using he interposer, so you can do it aswell. I already have enough questions on the easy process. This more advanced stuff is on you, I am not going to be maintaining and debugging individual issues at that level.


Actually with jumbo packets being available I don't know how I feel about asus choosing 128kbps for some subclasses. The correct minimum rate available, since jumbo packets have 6500mtu, should really be 520kbps+

@Vexira no the interposer is not how fqcodel works. The interposer is a workaround to take the fixed commands asus sends to TC, capture them, change them on the fly, then submit it to the realTC. It's a workaround since the asus commands were not modifable.

Even though we can now input any schedular into TC instead of the preprogrammed one, that doesn't mean we can have use any scheduler we want now. The queuing and scheduling algorithms relay on kernel level controls. We are missing some controls that some other algorithms use.

Those devs cannot port their algorithms without support. Why even ask.

I'll give you an analogy. Think of it as pipe flow. We have elbows, pressure regulators, and filters as materials in the kernel. The plumbing blueprints are the algorithms. Traffic control is the labor to set it up and keep it working. We can only use blueprints based on the materials we have. Newer kernel introduce timers and drain valves. We cannot use blueprints with timers and drain valves.

Fqcodel is a blueprint. We did not have all the materials nessasry to build it. So instead someone created an fq_codel lite blueprint with some missing features to alleviate the materials.

Your new schedulers may also have materials in the blueprints that we don't have.

The devs can't perform magic when those materials are integral to their plumbing system.

Those are impossible requests. You have to learn up on the materials before asking.

 

[Jack Yaz]

its nothing to technically worry about, read wicked shrapnel's post about quantum its more an inconvenice than a problem, but there is a way to "fix it", which involves modding the r2q value, otherwise don't worry about it it wont harm any thing. I quoted his post if you go back a bit.
but this is what he posted:

Please use quote/spoiler tags, walls of repeated text are not helpful

 

[maxsteel]

Sorry in advance for my silly question. How can I know the script works fine? I followed the installation instructions as explaind but I got no confirmation message in Putty.

 

[Vexira]

Please use quote/spoiler tags, walls of repeated text are not helpful

sry my bad ill keep that in mind for next time.

 

[bbunge]

Sorry in advance for my silly question. How can I know the script works fine? I followed the installation instructions as explaind but I got no confirmation message in Putty.

See your system log for the confirmation message.

Sent from my P01M using Tapatalk

 

[FreshJR]

Sorry in advance for my silly question. How can I know the script works fine? I followed the installation instructions as explaind but I got no confirmation message in Putty.

The "Defaults" still subclass in QOS statistics should also stop processing traffic entirely. It will have a little blip of traffic while the script is initiating then NON after.

But yea script produces system logs.

 

[maxsteel]

See your system log for the confirmation message.

Sent from my P01M using Tapatalk

Thanks, what should I look for exactly?

 

[maxsteel]

Defaults still subclass in QOS statistics should also stop processing traffic. It will have a little blip of traffic while the script is initiating then NON after.

But yea script produces system logs.

942.00 Bytes shown next to Defaults in Upload section and Zero in download. Good sign?

 

[john9527]

See your system log for the confirmation message.

@FreshJR
Just an FYI....you can add the '-s' switch to the logger command if you want to send the messages to both the console (stderr) and the syslog.

 

[ThiagoCururu]

As a few users have pointed out, some https traffic bypasses QOS entirely. Add these two rules, and retest.

${tc} filter add dev br0 protocol all prio 22 u32 match mark 0x80130000 0x803f0000 flowid ${Web}
${tc} filter add dev eth0 protocol all prio 22 u32 match mark 0x40130000 0x403f0000 flowid ${Web}

When some traffic bypasses QOS, it ignores QOS limits and can saturate the connection. This means our whole concept of throttling by different traffic types is moot since the bypassed traffic is able to clog up our prioritized traffic, leading QOS having no effect.

Hope this made sense. We need to catch all traffic.

I'm not exactly sure if I put those lines in the correct spot. I put it like this:

Re enabled QoS and restarted device. It didn't work.

Also, as I did some testing today, there's few things that I noticed:
- Latency spikes only occur while YouTube or Netflix is streaming, no matter if peak hour or not. It only happens when that connection comes from a Samsung TV via an access point. I tested with a Chromecast with a direct connection to AC68U and it worked fine.
- I have NAT Acceleration enabled. On Sysinfo tab it's showing "HW acceleration Enabled (CTF only)"
- Router's web interface gets extremely slow during that latency spike periods, as it was struggling to process something.
- Every AiProtection is off
- What information would be helpful? I'm running out of ideas...
EDIT: I'm using Adaptive QoS, if I didn't mention it before.

Thanks everyone for helping!

 

[Vexira]

No you do NOT set r2q = mtu.

Your minimum rate(bytes)/ r2q >= mtu

So our r2q of 10 is good for rates of with 15kB/s as a minimum since MTU is 1500.

If you are able to change r2q I wouldn't. Asus has some subclasses set to 128kbps.
15kB/s = 120 kbps. You cannot even set r2q to 11 without breaking those classes.

Getting back on topic. It is not a ERROR. Just a warning telling you to consider sorting packets with a finer screen.

If you want a finer screen use the interposer and change the default calculated Quantum's manually upon class creation. I have made some, different, modifications to my script using he interposer, so you can do it aswell. I already have enough questions on the easy process. This more advanced stuff is on you, I am not going to be maintaining and debugging individual issues at that level.


Actually with jumbo packets being available I don't know how I feel about asus choosing 128kbps for some subclasses. The correct minimum rate available, since jumbo packets have 6500mtu, should really be 520kbps+

@Vexira no the interposer is not how fqcodel works. The interposer is a workaround to take the fixed commands asus sends to TC, capture them, change them on the fly, then submit it to the realTC. It's a workaround since the asus commands were not modifable.

Even though we can now input any schedular into TC instead of the preprogrammed one, that doesn't mean we can have use any scheduler we want now. The queuing and scheduling algorithms relay on kernel level controls. We are missing some controls that some other algorithms use.

Those devs cannot port their algorithms without support. Why even ask.

I'll give you an analogy. Think of it as pipe flow. We have elbows, pressure regulators, and filters as materials in the kernel. The plumbing blueprints are the algorithms. Traffic control is the labor to set it up and keep it working. We can only use blueprints based on the materials we have. Newer kernel introduce timers and drain valves. We cannot use blueprints with timers and drain valves.

Fqcodel is a blueprint. We did not have all the materials nessasry to build it. So instead someone created an fq_codel lite blueprint with some missing features to alleviate the materials.

Your new schedulers may also have materials in the blueprints that we don't have.

The devs can't perform magic when those materials are integral to their plumbing system.

Those are impossible requests. You have to learn up on the materials before asking.

look i understand the kernel is restricted in what can be done, and what schedulers can be used, i know how FQ_Codel works, and the lack of certain components that exist in newer kernel versions, i was entertain the idea of it being possible to add, since if you read what i sent to you about cake, the devs are actually trying to back port it them selves to older kernels for more compatablity. If i could contact them id ask if its possible, if they could back port it to the kernel versions our routers run, since they got tit working with version 3 kernel.
i do understand where your coming from though.
sry i wrote it wrong, its r2q has to be closer to mtu, but your saying to modify the r2q it breaks sub classes, so may i inquire as to what issues occur eg, bandwidth allocations i broken completely, etc.
Also are you saying i have to create a new class entirely for each bit of traffic, and modify the [Flow ID] htb [rate], for each individual class.
just been reading this
linux-image-2.6.39-2-amd64: HFSC puts lots of "WARNING: at .../sch_hfsc.c:1427 hfsc_dequeue+0x155/0x28b [sch_hfsc]()" in dmesg
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=631945

Traffic Control in linux
http://linux-ip.net/articles/Traffic-Control-HOWTO/classful-qdiscs.html
https://www.linux.com/blog/tc-show-manipulate-traffic-control-settings
http://lartc.org/manpages/tc.txt
http://lartc.org/howto/lartc.qdisc.classful.html
http://luxik.cdi.cz/~devik/qos/htb/manual/userg.htm

 

[Vexira]

I'm not exactly sure if I put those lines in the correct spot. I put it like this:

Re enabled QoS and restarted device. It didn't work.

Also, as I did some testing today, there's few things that I noticed:
- Latency spikes only occur while YouTube or Netflix is streaming, no matter if peak hour or not. It only happens when that connection comes from a Samsung TV via an access point. I tested with a Chromecast with a direct connection to AC68U and it worked fine.
- I have NAT Acceleration enabled. On Sysinfo tab it's showing "HW acceleration Enabled (CTF only)"
- Router's web interface gets extremely slow during that latency spike periods, as it was struggling to process something.
- Every AiProtection is off
- What information would be helpful? I'm running out of ideas...
EDIT: I'm using Adaptive QoS, if I didn't mention it before.

Thanks everyone for helping!

check the cpu usage see if its spiking when you get a ping spike due to traffic.

 

[FreshJR]

@maxsteel yes that means it's working.

@ThiagoCururu typically for custom rules, that would be the wrong spot. For that specific rule I pasted, that spot you choose will work since it is being inserted into rule 22 position instead of rule 1.


What typically happens is that the script deletes rule 1. Creates multiple custom rules in rule 1 position. Recreates original rule 1 at the rule 2 position.

If that command was a rule1 position instead of rule22 position, it would get deleted in subsequent lines.

In general

TC filter add = create rule
TC filter del = delete rule
br0 = download
eth0 = upload
prio x = rule position

If you read these keywords you should see what is going on.

Back on topic, send me a qos statistic screenshot when you are getting bufferbloat. It maybe that all devices connected via your access point bypass QOS. Can you describe how to access point is connected to the router.

slow UI behavior is weird. Qos doesn't really bog down my system. Try just the TV itself doing something bandwidth intensive and a picture of the qos statistics page to see if a bypass is happening.

@Vexira nice i went through the same resources.

We can have a 1-1 chat in some instant mesage Clint.

As for an immediate answer. Rate/r2q = quantum.

Quantum always has to be be bigger that MTU (this means the largest possible packet size).

This is because qos is dividing traffic into quantum size chunks. This quantum chunk has to at least fit the entire packet. You cannot split a packet into two separate shipping boxes or it will break the packet.

If you check tc class show results, you will see subclasses with 128kbit rates. That 128kbit subclass needs a r2q of 10 or lower or else the default calculated quantum will be incorrect and destroy all packets. For the record that 128kbit rate is terrible and I don't know what asus was thinking.

I modified those sub class rates (not quantum) for my own personal QOS tuning but I am not ready to share that code yet due to how I implemented it.

Using the same technique, instead of changing r2q instead you can change easily change quantums of the classes when are generated instead of changing the r2q. A simple sed replacement would suffice in the /usr/sbin/faketc to modify quantums but I would invest a few more lines and mathematically base them on rates.

Honestly I feel the hardcoded warning is the the issue instead of the quantum values.

 

[Vexira]

@maxsteel yes that means it's working.

@ThiagoCururu typically for custom rules, that would be the wrong spot. For that specific rule I pasted, that spot you choose will work since it is being inserted into rule 22 position instead of rule 1.


What typically happens is that the script deletes rule 1. Creates multiple custom rules in rule 1 position. Recreates original rule 1 at the rule 2 position.

If that command was a rule1 position instead of rule22 position, it would get deleted in subsequent lines.

In general

TC filter add = create rule
TC filter del = delete rule
br0 = download
eth0 = upload
prio x = rule position

If you read these keywords you should see what is going on.

Back on topic, send me a qos statistic screenshot when you are getting bufferbloat. It maybe that all devices connected via your access point bypass QOS. Can you describe how to access point is connected to the router.

slow UI behavior is weird. Qos doesn't really bog down my system. Try just the TV itself doing something bandwidth intensive and a picture of the qos statistics page to see if a bypass is happening.

@Vexira nice i went through the same resources.

We can have a 1-1 chat in some instant mesage Clint.

As for an immediate answer. Rate/r2q = quantum.

Quantum always has to be be bigger that MTU (this means the largest possible packet size).

This is because traffic is dividing into quantum size chunks. This Quantum chunk has to fit at least the entire packet. You cannot split a packet into two containors or it will break the packet.

If you check tc class show, you will see subclasses with 128kbit rates. That 128kbit subclass needs a r2q of 10 or lower.

I modified those sub rates for my own personal tuning but I am not ready to share that code yet due to how I implemented it.

Using the same technique, instead of changing r2q you can change the quantums of the classes as they are generated instead of changing r2q.

we should I have xbox steam and skype, and you told me I should read , also that's cool ive been devoting my time to , reading and debugging things so I don't have to ask some one else I learn also, I sadly never took up programming after high school, makes me regret that a tad.

 

[Jack Yaz]

Am I right in thinking if I want to force port 563 traffic to Downloads (NZB) I would use:
Code:
${tc} filter add dev br0 protocol all prio 1 u32 match ip dport 563 0xffff flowid ${Downloads}

Since 563 is the server port, should I change dport to sport, since its traffic coming down from server port 563 to LAN?

 

[maxsteel]

@maxsteel yes that means it's working.

Thanks, Glad it is working. However, the script seems not dealing with my NZB issue.

All NZB traffic shows as "VoIP and Instance Messages". Any idea how to fix this?

Also, please tell me if I'm setting up the bandwidth values correctly based on percentage reduction.

ISP Modem Values:

QoS Bandwidth:

 

[Jack Yaz]

Thanks, Glad it is working. However, the script seems not dealing with my NZB issue.

All NZB traffic shows as "VoIP and Instance Messages". Any idea how to fix this? Thanks

See my posts, I think its a matter of custom rule for the server port (563 in my case), and directing it to the Downloads container.