[Release] FreshJR Adaptive QOS (Improvements / Custom Rules / and Inner workings)

[chris.at]

Looks very impressive!

So, if someone would input for example "local=192.168.1.2,remote=<emtpy>,proto=tcp,local-port=80,remote-port=<empty>,mark=<empty>,cat=netcontrol" this would create both inbound and outbound rules to categorize traffic coming from and to this local ip/port into netcontrol category? (that's how I set it up here for example using two rules inside the script for up and down categorization. of course there is a port forwarding set up to this ip/port from the internet)

 

[Therion87]

Here's another progress report for everyone impatient.
Everything is coming along nicely and working as expected! Do not know how busy I will be this weekend but I am trying to finish everything up soon.

Don't worry, that bulky table (and the other two yet to be added) will be hidden on page load by default.

You have to click the orange"FreshJR Mod Customize" button before the bulky "QoS Modification" sub menu is shown. You can click the orange text again to hide.

--

I do not know if I should create 6 extra (non-modifiable) entries to represent the scripts built-in hard-coded rules.

On one hand I want to keep everything small, compact, and clean.

But on the other hand, I do not want to make the built in rules ambiguous.
(I am leaning on adding them in).

Will the new installer include an interactive way to enable the "gaming rules" for devices as well as the, custom rules shown?

 

[Tekneek]

@FreshJR Will this new GUI also handle IPv6 rules or will those have to be handled manually? If so, will that cause disruption to anything else presented in the UI? I realize IPv6 is not as important, I don't mind manually handling them at all. Just curious after seeing that screenshot.

 

[L&LD]

Here's another progress report for everyone impatient.
Everything is coming along nicely and working as expected! Do not know how busy I will be this weekend but I am trying to finish everything up soon.

Don't worry, that bulky table (and the other two yet to be added) will be hidden on page load by default.

You have to click the orange"FreshJR Mod Customize" button before the bulky "QoS Modification" sub menu is shown. You can click the orange text again to hide.

--

I do not know if I should create 6 extra (non-modifiable) entries to represent the scripts built-in hard-coded rules.

On one hand I want to keep everything small, compact, and clean.

But on the other hand, I do not want to make the built in rules ambiguous.
(I am leaning on adding them in).

Please do create the 6 extra entries. More information is always, always preferable to less!

Small, compact and clean? Of course. But not at the cost of usefulness.

FreshJR, I wish I could send you the help you need this weekend so you could be busy with this instead.

 

[jsbeddow]

I do not know if I should create 6 extra (non-modifiable) entries to represent the scripts built-in hard-coded rules.

On one hand I want to keep everything small, compact, and clean.

But on the other hand, I do not want to make the built in rules ambiguous.
(I am leaning on adding them in).

Yes, another vote for "more info", even in regards to non-modifiable defaults/values.

 

[FreshJR]

this would create both inbound and outbound rules to categorize traffic coming from and to this local ip/port into netcontrol category?

Yes

@FreshJR Will this new GUI also handle IPv6 rules or will those have to be handled manually? If so, will that cause disruption to anything else presented in the UI?

I have ignored ipv6 functionality with this release for a few reasons.

1) tracked connections doesn’t show the full ipv6 address due to an asus bug. The last two digits in tracked connections are actually truncated so reflecting local Ip address rules in tracked connection would not work. With this you cannot know which local devices would be affected by your custom rule since that most significant portion of the local device identifier is chopped off. Remote connections would work with some assumptions made but still be slightly inaccurate.

2) minor reason: didn’t want to start messing with comparing 128bit numbers (that’s what an ipv6 address is) but this is easily overcome.

3) not yet familiar with ipv6 syntax. I don’t use it but I sometimes see the middle part omitted with successive ::: colons. I have to look it up before I start parsing it.

My question to you. Do you only need ipv4 port rules to to reimplemented into iptables6 so the port rules take effect on ipv6 connections? Or do you actually need support for ipv6 rules that will actually filtering on specified ipv6 ranges?

Will the new installer include an interactive way to enable the "gaming rules" for devices as well as the, custom rules shown?

Will be present in both UI & interactive menu.

 

[RMerlin]

3) not yet familiar with ipv6 syntax. I don’t use it but I sometimes see the middle part omitted with successive ::: colons. I have to look it up before I start parsing it.

This will fully expand a shortened version:

https://github.com/RMerl/asuswrt-me...elease/src/router/www/js/table/table.js#L1453

I added that function so I could properly sort by IPv6 addresses (that function will also fudge IPv4 addresses so they can be sorted by the same sort function, but you could strip that part off).

 

[Tekneek]

My question to you. Do you only need ipv4 port rules to to reimplemented into iptables6 so the port rules take effect on ipv6 connections? Or do you actually need support for ipv6 rules that will actually filtering on specified ipv6 ranges?

I do both, although the second is more important than the first. I don't want to derail your work.

If push comes to shove, I can stick with what I have. Being able to track connections is gravy for me, it is more important that I have the desired functionality under the hood.

 

[FreshJR]

I do both, although the second is more important than the first. I don't want to derail your work.

If push comes to shove, I can stick with what I have. Being able to track connections is gravy for me, it is more important that I have the desired functionality under the hood.

Under the hood, the script is remaining the same.
So yes, any manual modifications (like added ipv6 rules) will STILL work exactly the same like in previous versions.
Your added rules just wouldn't be reflected under the "tracked connections" table.

The issue with adding ipv6 functionality remains the same in where local/remote addresses cannot be visually parsed for matches in the table since the ipv6 addresses are incomplete.

The Asus data bug is throwing a wrench at fully implementing WebUI IPv6 functionality!!!

--

I can allow the creation/modification of ipv6 rules from the WebUI (not reflected in the table) but this would have to wait until next release due to **sigh** MORE limitations on how data is currently stored and passed back into the script.

Spoiler: technical note

The WebUI actually doesn't have access to read/write files on the router itself.
I am limited to passing data back and forth with nvram variables .
The issue with nvram varaibles is that they are limited to 255 characters each.
Now you may think the solution simple, just create more nvram varables and call it a day.

Well...

1) I am limited to using **PRE-EXISTING** nvram varaibles since as ANOTHER limitation, the WebUI is limited to accessing/modifying predefined variables instead of ones that I create on my own.
2) There is limited nvram space on the router.​

To counter this limitation (1) I have hijacked two pre-existing nvram variables that are currently unused.
These two variables gave me space for 4 iptable rules, 4 appdb rules, a gaming device range , and to store rates/ceilings per category.

The nvram varaibles I took were originally used in the feedback page http://192.168.1.1/Advanced_Feedback.asp
This page is disabled in RMerlin firmware (and also stock firmware I believe), so I don't think stealing these nvram variables for personal use will be an issue.

Now back to the main point with the ipv6 WebUI rule functionality delay.

Ipv6 addresses are very long and there is not much room to store 4 complete ipv6 rules entries in nvram without resorting to even MORE trickery.
As another workaround, I could encode the rules individual parameters as characters and then decode these characters back into original data when parsed.

This is possible, but it is another change that would take require more time consuming testing (since the first 31 elements of the ASCII table may begin to act weird when saved/read). As such it will have to wait for next release.

(kind of a miracle everything works with the recursive workarounds at play)

 

[Kontakinte78]

Hi there !

Im new in this forum .
At first, i would like to say i find the job you are doing great and thank you a lot.
I have a few questions.If you please can help me. I play battlefield 5 a lot. My wife is chatting a lot and she is watching videos on youtube. So, i took a look at the script FreshJR qos an i would like to modificate it, since the battlefield 5 falls into others and messenger and youtube videos from my wife too.I would like too set the gaming rule for battlefield 5 in the script.I removed the ## in the download tcp line and change the cidr.My ip set on my computer is 192.168.1.10 , i play battlefield on pc.My subnetmask is 255.255.255.0 . So i set the cidr as 192.168.1.0/24 , i hope thats correct.But now battlefield 5 still falls into others category. What i am doing wrong? Can you help me please ? Thanks.

 

[Kontakinte78]

Hi there !

Im new in this forum .
At first, i would like to say i find the job you are doing great and thank you a lot.
I have a few questions.If you please can help me. I play battlefield 5 a lot. My wife is chatting a lot and she is watching videos on youtube. So, i took a look at the script FreshJR qos an i would like to modificate it, since the battlefield 5 falls into others and messenger and youtube videos from my wife too.I would like too set the gaming rule for battlefield 5 in the script.I removed the ## in the download tcp line and change the cidr.My ip set on my computer is 192.168.1.10 , i play battlefield on pc.My subnetmask is 255.255.255.0 . So i set the cidr as 192.168.1.0/24 , i hope thats correct.But now battlefield 5 still falls into others category. What i am doing wrong? Can you help me please ? Thanks.

Btw, my router is an asus ac68u with latest merlin and the freshjr qos script.
When i start the game, with origin , the classification shows me it is recognized in the gaming section, but as soon i join a server then battlefield falls into the others section.And so it was before i modified the script, so through my changes didnt change something.

 

[QuikSilver]

Hi there !

Im new in this forum .
At first, i would like to say i find the job you are doing great and thank you a lot.
I have a few questions.If you please can help me. I play battlefield 5 a lot. My wife is chatting a lot and she is watching videos on youtube. So, i took a look at the script FreshJR qos an i would like to modificate it, since the battlefield 5 falls into others and messenger and youtube videos from my wife too.I would like too set the gaming rule for battlefield 5 in the script.I removed the ## in the download tcp line and change the cidr.My ip set on my computer is 192.168.1.10 , i play battlefield on pc.My subnetmask is 255.255.255.0 . So i set the cidr as 192.168.1.0/24 , i hope thats correct.But now battlefield 5 still falls into others category. What i am doing wrong? Can you help me please ? Thanks.

Might help to post the portions you added to the script. I also believe you have to turn off and ON the QOS option for the changes to work.

 

[L&LD]

Hi there !

My wife is chatting a lot and she is watching videos on youtube.

Welcome! Just be prepared that FreshJR won't be able to fix a chatty wife as easily as he has fixed Asus' QOS shortcomings!

 

[Kontakinte78]

Might help to post the portions you added to the script. I also believe you have to turn off and ON the QOS option for the changes to work.

Yes, i restarted qos.I turned it off and then i turned it on again.Thanks for the reply.


iptables -D POSTROUTING -t mangle -o br0 -d 192.168.1.0/24 -m mark --mark 0x80000000/0x8000ffff -p tcp -m multiport ! --sports 443,80 -j MARK --set-mark ${Gaming_mark_down} &> /dev/null #Gaming (3/3) - Routes Unidentified Traffic into "Gaming", instead of "Others", for LAN clients specified
iptables -A POSTROUTING -t mangle -o br0 -d 192.168.1.0/24 -m mark --mark 0x80000000/0x8000ffff -p tcp -m multiport ! --sports 443,80 -j MARK --set-mark ${Gaming_mark_down}

Only this i changed.

 

[Kontakinte78]

Welcome! Just be prepared that FreshJR won't be able to fix a chatty wife as easily as he has fixed Asus' QOS shortcomings!

Hahaha ...you are 100% wright man...if i cant, then it will be very difficult for him to change that..

 

[FreshJR]

@Kontakinte78 Welcome to the forum, and also wrong ip

It’s supposed to be the IP of your gaming device (/32 CIDR for single device) and NOT the IP range of your entire network

192.168.1.10/32

(If that doesn’t work, copy paste rule and add it as a 2nd entry but changing the protocol to udp)

(If that still doesn’t work, grab the battlefield connection from tracked connections table so I can take a look at it)

Also how are you verifying that the game data is going into others? The tracked connections table doesn’t work with my script yet. You have to look at the pie charts above the table.

 

[Kontakinte78]

@Kontakinte78 Welcome to the forum, and also wrong ip

It’s supposed to be the IP of your gaming device (/32 CIDR for single device) and NOT the IP range of your entire network

192.168.1.10/32

(If that doesn’t work, copy paste rule and add it as a 2nd entry but changing the protocol to udp)

(If that still doesn’t work, grab the battlefield connection from tracked connections table so I can take a look at it)

Also how are you verifying that the game data is going into others? The tracked connections table doesn’t work with my script yet. You have to look at the pie charts above the table.

So i am verifying this over the classification tab. There are shown the packets, packet rate.When i start origin, it shows me about 25 packets that are going into the gaming section, in the upload.As soon as i join a server, then it changes to the others section and it shows 66 packet rate on both up and down.

OK, i will try what you written. Thanks a lot.

 

[Kontakinte78]

Thanks a lot freshJr.That helped. It now shows like this in the screenshot i have uploaded.And now the battlefield goes into the gaming section.66 packet rate on both up and down.

 

[Kontakinte78]

View attachment 16264 View attachment 16265 Thanks a lot freshJr.That helped. It now shows like this in the screenshot i have uploaded.And now the battlefield goes into the gaming section.66 packet rate on both up and down.

But now i still have questions. I had to change the up rules like in the screenshot, so that gaming up is going into gaming.If i leave it disabled, commented, then my up goes into others.Is it normal, do i need my up in the gaming section, or should i leave it disabled ?And now, after i set the rule correct, my wifes chat and youtube all goes into others, so how do i set the priority?Should i leave it as recommended or should i set the priority with gaming on first place and others at last place?

 

[Kontakinte78]

But now i still have questions. I had to change the up rules like in the screenshot, so that gaming up is going into gaming.If i leave it disabled, commented, then my up goes into others.Is it normal, do i need my up in the gaming section, or should i leave it disabled ?And now, after i set the rule correct, my wifes chat and youtube all goes into others, so how do i set the priority?Should i leave it as recommended or should i set the priority with gaming on first place and others at last place?

And i have a other problem.I have a internet connection 400 Mb cable .Download from isp is 400 and 25 upstream . I set manually the download and the upload rate at 360 and 23 .When adaptive qos is off, then i get in a speed test by oakla my full download, its about 400.When i turn qos on, then i get about 150 download.The upstream is not affected.It always shows something like 25.Do you maybe know what causes this problem?