[Release] FreshJR Adaptive QOS (Improvements / Custom Rules / and Inner workings)

[Kingp1n]

When doing DSLReports speedtest thru a mobile phone or on wifi, sld the bufferbloat reflect an "A" rating as well or only when a laptop is hard wired? Im using latest script as well. Just curious...

 

[Kingp1n]

Yes that was slightly changed.

It used to be

If unidentifed —> goto gaming

Now it is

If unidentifed && not from ports 80,443
—> goto gaming
else if unidentifed && from ports 80,443
—> goto others

This is more for users who put pcs as gaming devices to help cut down on non-gaming unidentified traffic from going into gaming.

Old rules were probably better for consoles. Doesn’t matter as both should work fine.
Everything is user modifyable if someone needs to revert back manually.

I don’t game much.

You mentioned old rules were better for consoles but with new script its better for PCs, if i dont have a PC, you recommend copy/pasting old gaming rules to new script?

 

[FreshJR]

I have to add 192.168.2.12 in the new rules 80 and 443? (the same IP that i use in the Gaming rules for my consoles?)

Yes. I even have a picture up as part of the first three posts ...

You mentioned old rules were better for consoles but with new script its better for PCs, if i dont have a PC, you recommend copy/pasting old gaming rules to new script?

All rules relating to ports or IP's have to be implemented via iptables now.

If your old rules were of the tc variant, then no, do not use them.

Just erase from [-p through 443,80] and you will be left with one gaming rule in each download and upload section behaving like the old setup.

When doing DSLReports speedtest thru a mobile phone or on wifi, sld the bufferbloat reflect an "A" rating as well or only when a laptop is hard wired? Im using latest script as well. Just curious...

It should be an A on every device.

Wasn't QOS for all wireless devices broken on your router. I think you already know the answer as to what is going on.

 

[Sinner]

Yes that was slightly changed.

It used to be

If unidentifed —> goto gaming

Now it is

If unidentifed && not from ports 80,443
—> goto gaming
else if unidentifed && from ports 80,443
—> goto others

This is more for users who put pcs as gaming devices to help cut down on non-gaming unidentified traffic from going into gaming.

Old rules were probably better for consoles. Doesn’t matter as both should work fine.
Everything is user modifyable if someone needs to revert back manually.

I don’t game much.

Im gonna have to take a peek at this new one.. im running the previous compatiable customized but the new unitdentifed and not on port 80/443 etc rules is interesting. Might slide that into what ive got now =)

 

[HowIFix]

Yes. I even have a picture up as part of the first three posts.

I already know about Gaming rules of unidentified traffic.

01. what I want to say what this is for: (lines #115 to 119 and #148 to 152)

• -p tcp -m multiport ! --sports 443,80
• -p udp -m multiport ! --sports 443,80

02. Is there a rule that sends ports 80 and 443 to the Default category (lines #109 to 113 and #142 to 146) and another that brings them back the ports 80 and 443 to the Gaming category (lines #115 to 119 and #148 to 152).

03. Or they have nothing to do and I'm wrong? (I'm just referring to traffic on ports 80 and 443)

 

[FadgewackeR]

It should be an A on every device.

Wasn't QOS for all wireless devices broken on your router. I think you already know the answer as to what is going on.

This is interesting, as I get A+ A+ A+ on my wired connections, yet the bloat wildly varies (C to F) on my wireless clients. As the 86U and Upload traffic is currently broken, I have to assume (with your conf that all connections should provide the same bloat figures) that this is why.

Thankfully ASUS have beta firmware out with a fix for the Wifi Upload QOS bug. Hopefully that fix will be released for a Merlin update.

 

[FreshJR]

I already know about Gaming rules of unidentified traffic.

01. what I want to say what this is for:

02. Is there a rule that sends ports 80 and 443 to the Default category and another that brings them back the ports 80 and 443 to the Gaming category

03. Or they have nothing to do and I'm wrong?

1) Lines 115 - 119 simply do the following

If unidentifed && not from ports 80,443
—> goto gaming
**for the devices specified in the rule**

2) No we are not moving from Gaming -> Defaults -> Gaming.
That would default the purpose and be a waste of resources.

The big picture is the following:

(a) If Gaming && from ports 80, 443 --> Go to Defaults
(b) If Unidentifed && not from ports 80,443 --> Go to Gaming
(c) If Unidentified && from ports 80,443 --> Do nothing --> TC will take this unidentified traffic to Others​

3) The rules do not overlap. See answer to #2

---

Gaming is rarely on ports 80, 443.

The idea is to remove port 80, 443 traffic from gaming section.
This portion of "gaming" traffic is most likely a download, but can also be lobby data, authentication, etc.
This is achieved via rule (a)

The next idea is to take all unidentified traffic for user specified devices and assume it is gaming.
As before, we still don't want unidentified traffic on ports 80, 443 to be put in the gaming section, so an exclusion has to be defined.
This is achieved via rule (b).

The excluded traffic from rule (b) will go into "Others".
This is achieved via rule (c).
Keep in mind that rule (c) is not a real rule. It is the lack of a rule present / action taken which lets TC filtering take care of it downstream.

---

Consoles usually wont have ANY unidentified port 80 / 443 traffic, but just in case you have one of those rare games that is unidentified and actually communicates on port 80 / 443, it then becomes be a better assumption to stick ALL of a consoles unidentified traffic into gaming, including port 80/443, as they don't do different types of traffic. They are generally dedicated devices. This was the previous script setup.

---

Just use the rules as is !! ..... I don't typically enable evil rules.
Read the port description in this post, as you have to understand how port rules operate and have partial understanding of typical contents on each port before trying to implement or modify them.

 

[Sinner]

1) Lines 115 - 119 simply do the following

If unidentifed && not from ports 80,443
—> goto gaming

2) No that is not what is happening. That would default the purpose of the two rules.
The big picture is the following:

If Gaming && from ports 80, 443 --> Go to Defaults
If Unidentified && from ports 80,443 --> Do nothing --> Go to Others (TC will take all unidentified traffic to Others)
If Unidentifed && not from ports 80,443 --> Go to Gaming

3) The rules do not overlap. See answer to #2

im curious about this.. im I to assume the gaming systems download are on port 80/443 and your directing them to defaults? not a bad idea..jus wanna confirm

 

[FreshJR]

im curious about this.. im I to assume the gaming systems download are on port 80/443 and your directing them to defaults? not a bad idea..jus wanna confirm

no that is not what I am doing. Most of internet traffic is done between ports 80 / 443. This means if you specified a PC as your gaming device and directed its port 80/443 traffic into a specific category the end results would be horrible as most of it's traffic would be adversely routed.

Read the post above for what is actually happening.
It's similar.

 

[HowIFix]

Thank you for explaining to me.


Just out of curiosity what happens if I use the VPN rule on an IP that is not using VPN.
- QOS + VPN Client fix via custom rules.

Example:
IP range in the script 192.168.1.0/24, but in some IP in OpenVPN does not use the VPN.

Config in OpenVPN like this by @pusb87: (Post link)

 

[FreshJR]

Just out of curiosity what happens if I use the VPN rule on an IP that is not using VPN.

Nothing really, just will waste extra cpu power.

Normally the rule takes an upload mark and flips it into download.

If you were to apply it to a non-vpn device, it will take a download mark and flip it into download and this action would leave you with the same mark.

The traffic will be go to its correct category but you used / wasted CPU power to "flip" it.

I haven't tested how the VPN rule affects LAN 2 LAN transfers.

 

[HowIFix]

Thank you!

 

[h3lmut]

While investigating rules not working on a forum members configuration, I found some very interesting results.

In this case, it was found that any rules related to IPs or Ports on the AC86U were not working even though it had a typical “automatically dhcp assigned // eth0” connection. (MARK only rules did work as expected)

It might be worth your time to create a temporary TC (traffic control) rule to push all traffic for a specific local device, by IP, into a lightly used catagory, like VOIP. You should then see if the temporary rule, filtering by the local device’s LAN IP, is correctly placing traffic into the VOIP category as expected during a speedtest. This is test procedure is recommended no matter your router model.

If the temporary rule above is not working as expected, I would recommended switching to the alternative version of the script linked in the third post as that implementation was found to work correctly on all setups.

I didn’t dig deeper as to what setting could potentially be causing this conflict.

I believe the users AC86U was cascaded behind another router, so it might be possible that his packets had VLAN TAGGING enabled and that shifted the packet a few bytes invalidating the packet hard coded offset locations that TC uses when filtering by IP address or port. As I said, I didn’t look deeper into it, since a working solution was readily available.

Other users have reported no issues with the AC86U. So this is just an issue to keep in mind if debugging or just to check for peace of mind.

I have a new modem which you cannot configure. But I have to set VLAN 7 for internet. With my old modem I could set it on modem side. With your modified PPPOE script QOS worked like a charm.

Now I have to set VLAN Tag 7 on my AC56U. QOS is working bad. I have the same results at dslreports like with your normal script with my old modem. Its like it misses something. Ping DL/UL is +40-60ms.
With your pppoe script and my old modem I had DL +20-30ms and UL +2-5ms

 

[ctyt]

Dumb question. I have the script installed and I use an Obi202 VoIP adapter with Google Voice. It seems to be highlighted red in the Bandwidth Monitor without me doing anything. Does this mean it's automatically prioritized without me making any custom rules?

 

[shelleyevans]

Hi!
(POST HAS BEEN EDITED TO REFLECT MY SOLUTION)
I finally had a chance to install the script to my new aiMesh set-up (I wanted to let it run for a week just to make sure aiMesh was stable, which it seems to be!). I pasted the single-line command as instructed, in terminal, and got the following response-- "-sh: curl: not found"

After looking all over the internet, it appears that the "curl" solution may not work. Mac comes with curl installed, but ssh is not enabled for that curl. From what I read, it looks like installing curl is far more complicated than manually installing the script.

I think I succeeded in manually installing the new script on stock firmware, using a modified version of the Power Users instructions in the first post. On my Mac computer neither the "curl" nor the "dos2unix" commands work (I keep getting "not found" error messages with those two commands), so here's what I did:

1. turn ON qos
2. CREATE DIRECTORY ON ROUTER:

In TERMINAL type:
SSH admin@192.168.1.1(You will be asked for a password; THIS GETS YOU INTO THE ROUTER)
Then type: mkdir /jffs/scripts
Then type: exit (THIS GETS YOU OUT OF THE ROUTER)​

2. COPY SCRIPT TO DIRECTORY YOU JUST MADE:

In TERMINAL type:
scp /WHATEVER YOUR PATH IS TO THE SCRIPT FILE/FreshJR_QOS admin@192.168.1.1:/jffs/scripts/FreshJR_QOS

(NOTE ALL OF THE ABOVE GOES ON ONE LINE, with SPACE between “QOS" and "admin@” The first half identifies where the script is on the computer, and the second half moves that script onto the router. You can get the correct path location by typing “scp” in terminal, then a space, and then just drag and drop the file into the terminal window. I was asked to enter the router password and then got a message that the file had successfully copied.)​

3. EXECUTE THE SCRIPT

In terminal, type:
SSH admin@192.168.1.1(You will be asked for a password; THIS GETS YOU INTO THE ROUTER)
Then type:
sh /jffs/scripts/FreshJR_QOS -install
(THIS INSTALLS THE SCRIPT; YOU WILL GET A MESSAGE ASKING YOU TO CONFIRM THAT YOU ARE USING STOCK FIRMWARE)​

4. REBOOT THE ROUTER​

Please excuse this lengthy post, but I thought it was worth saying how I got it to work. I hope somebody will let me know if I did something wrong!

 

[FreshJR]

I pasted the single-line command as instructed, in terminal, and got the following response-- "-sh: curl: not found"

I thought stock firmware had curl. Guess not. Anyone else able to confirm? I am not going to downgrade to default firmware just to check.

@shelleyevans

Just to confirm, you have tried executing the curl command within the ssh session?

The curl command should have been executed locally on the router inside the ssh session, with zero reliance of having curl on your Mac .

 

[TicoMan]

I thought stock firmware had curl. Guess not. Anyone else able to confirm? I am not going to downgrade to default firmware just to check.

I can confirm the commands work with ASUS stock firmware.

Only concern I have is, why does enabling (with and without your script) QOS cause a higher Packet Loss Rate? Cause when I run Netalyzr with QOS enabled it will almost always have a higher drop rate of around 8%

 

[skeal]

I can confirm the commands work with ASUS stock firmware.



View attachment 14042

Only concern I have is, why does enabling (with and without your script) QOS cause a higher Packet Loss Rate? Cause when I run Netalyzr with QOS enabled it will almost always have a higher drop rate of around 8%

Looks to me like you don't have things set up right. Quality should be traded off for Bufferbloat IMHO...

 

[HowIFix]

This message appears in the log after adding the VPN rules:

Aug 12 22:47:22 adaptive QOS: Applying - Iptable Down Rules
Aug 12 22:47:23 adaptive QOS: Applying - Iptable Up   Rules (eth0)
Aug 12 22:47:23 adaptive QOS: iptables v1.4.15: unknown option "--set-mark"
Aug 12 22:47:23 adaptive QOS: Try `iptables -h' or 'iptables --help' for more information.
Aug 12 22:47:23 adaptive QOS: TC Modification Delayed Start (5min)

Protocol and Port in the OpenVPN file:

proto tcp
remote 199.115.112.86 443

Download rules:

Upload rules:

 

[FreshJR]

@HowIFix
Wrong variable used in upload rules. Use
${Downloads_mark_up}

@TicoMan
Packet drop occurs after queues fill up.
Queues fill up if the tcp connection is too slow to negotiate down to your max speed.
This can happen if your bandwidth limit is set too low from your actual atainable speeds.
Overall, the big picture is that your connection has to negotiate underneath set limits at a rate faster than the packet queue fills up to get no packet loss.



Fq-codel is amazing at queue management.

1) It drops invalid timed out packets in the middle of the queue which opens more space within the queue for fresh packets. This effectively makes the queue larger

2) Since the toxic packets are ignored faster, tcp will renegotiate the connection rate to set limits faster which is great since we are at a race to prevent to queue from filling up in order to get no drop.

Sfq not so much.

In contrast to the above, sfq keeps timed out packets, and drops the new packets since they can’t fit into the queue. Not only will the entire queue turn toxic while tcp renegotiates down, new packets are dropped since there’s is no room for them.

This is especially severe with sfq when your bandwidth limits are too low for your connection since negotiating will take even longer!

This would be seen as a poor quality grade but Dslreports reported your pocket drop/quality as an A+ so I don’t see the issue you are mentioning.

An improvement would be to switch to fq-codel, but unfortunately, only sfq is present on stock firmware. In the end, it is what it is, since that is how QOS works under the hood.

If you are curious as to the differences check out the sfq vs fq-codec animated graphic in the first post.

You can see faster nogotiation rate to max speed && higher quality with fq-codel compared to sfq with the same exact settings.

Cake is even better from reports on the net, but we don’t have that at our disposal.

l It seems to be highlighted red in the Bandwidth Monitor without me doing anything. Does this mean it's automatically prioritized without me making any custom rules?

Custom rules are more to get traffic into the right category while prioritization is the device priority within the category.

It is nice that it was set to highest automatically (are you sure you didn’t assign this manually) but I still am not sure if the default rules would place the traffic correctly into the VoIP category without testing it.

Anyway as a failsafe, there are default rules in the script that route most VoIP traffic into VoIP, even if it is redundant.