[Release] FreshJR Adaptive QOS (Improvements / Custom Rules / and Inner workings)

[FreshJR]

because some game updates or downloads if i recall fall into gaming category

Recent updates of the script (like 1-2 months ago) try to redirect game "update" patches into downloads instead of gaming traffic

This is what the:

"if gaming && from port 80 --> downloads"

&

"if gaming && from port 443 --> downloads"

rules are intended to do!

--

Along with you, I too didn't agree with bulk game updates receiving "ping-sensitive" priority.

 

[Atomic7431]

Does this effect time scheduling?

I have today tested out settings times when the kids can access the internt via their PS4s.
So come 16:00 tonight they should be able to use the internet, but they cant? The PS4 test for connection fails with time scheduling on, but is fine with it off/

I even waited until 16:20 to see if there was a delay in it working but still no go.

Pic below to show what i have tested.

and then time scheduling on in the client list for each PS4.

Im using the Compatible version of the QoS

 

[FreshJR]

Does this effect time scheduling?

I have today tested out settings times when the kids can access the internt via their PS4s.
So come 16:00 tonight they should be able to use the internet, but they cant? The PS4 test for connection fails with time scheduling on, but is fine with it off/

I even waited until 16:20 to see if there was a delay in it working but still no go.

Pic below to show what i have tested.

and then time scheduling on in the client list for each PS4.

Im using the Compatible version of the QoS

It shouldn't, but I haven't tested in this area. I'm 99% your issue is unrelated tho.

I will look into it.

Edit:

Just confirmed the blacklist and whitelist are both working as intended. Feel free to pm me the output of

iptables -L

And I will take a look at your setup as actually setup in the firewall rules instead of WebUI.

 

[skeal]

@FreshJR Finnaly got this figured out and tested.

Excellent!!

 

[Kingp1n]

@FreshJR Finnaly got this figured out and tested.

Excellent!!

Skeal...Are you using the comp or fast version? I can never get my bufferbloat to an A rating, just Bs and Cs..are u making any special adjustments to the scripts?

 

[FreshJR]

Skeal...Are you using the comp or fast version? I can never get my bufferbloat to an A rating, just Bs and Cs..are u making any special adjustments to the scripts?

There is ZERO performance difference between the two.
I don't care what anyone else claims!

If you are getting poor bufferbloat, reduce defined speeds
If you are getting poor quality, increase defined speeds.

With these effects in mind, zero in on the ideal speed to get highest overall grade as quality/bufferbloat have an inverse relationship.

Note, your actual ISP network performance has large variance depending on it's load. Testing off peak hours would be recommended so that variance is eliminated from your results.

 

[Kingp1n]

@FreshJR yes i'm getting poor bufferbloat. My quality reflects A and also the overall score also shows an A rating. When you say reduce the defined speeds, do you mean adjust within the script or under the QOS settings under upload/download bandwidth ?

 

[skeal]

Skeal...Are you using the comp or fast version? I can never get my bufferbloat to an A rating, just Bs and Cs..are u making any special adjustments to the scripts?

The only adjustments to the script is just using the bandwidth management settings and the game enabling settings. Like @FreshJR said read up on what the basic description is of buffer bloat and set the bandwidth and max up and down bandwidth with that in mind. I started at 95% but found that my upload needed to be set at 90% and then because I stream my bandwidth favours that.

 

[skeal]

Number one first thing is if you use dslreports then sign up it really helps and allows for further settings to be used.
Edit: Always test with nothing going on. Network quiet.

 

[FreshJR]

When you say reduce the defined speeds, do you mean adjust within the script or under the QOS settings under upload/download bandwidth ?

In WebUI aka

under the QOS settings under upload/download bandwidth

 

[Sinner]

Regarding this what ive done is base mine on ONPEAK rates. For me our rates can drop as much as 40% onpeak. Only very intermittent and briefly. As such ive found my best setting for overall performance in real life usage was about 10% over the onpeak minimum so at about 70% of my max off peak. Keeps everything running smoothly 99% of the time with maybe the ocasional super brief minor lag while not reducing total bandwith more than nessissary. Works for me anyway

There is ZERO performance difference between the two.
I don't care what anyone else claims!

If you are getting poor bufferbloat, reduce defined speeds
If you are getting poor quality, increase defined speeds.

With these effects in mind, zero in on the ideal speed to get highest overall grade as quality/bufferbloat have an inverse relationship.

Note, your actual ISP network performance has large variance depending on it's load. Testing off peak hours would be recommended so that variance is eliminated from your results.

 

[skeal]

@FreshJR or anyone why would a video stream for a news channel be categorised as "web surfing" or "network control packets". Is there a way to direct this to video, and if so is it done by ip? The problem only arises when using one device. Its an android media box.

 

[FreshJR]

@FreshJR or anyone why would a video stream for a news channel be categorised as "web surfing" or "network control packets". Is there a way to direct this to video, and if so is it done by ip? The problem only arises when using one device. Its an android media box.

If its an android media box, you can simply create a rule so that all traffic on that box is "Video Streaming".
With that you wouldn't have to micro manage and look for other identification deficiencies.

You can use these templates ( keep in mind that these templates have always been included inside the script itself )

Download
#    ${tc} filter add dev br0 protocol all prio $1 u32 match ip dst 192.168.1.100/32 flowid ${VOIP}                #Template Rule 2 (All incoming traffic w/ LAN destination ip 192.168.1.100                  goes to "VOIP" Traffic Container)                             
Upload
#    iptables -D POSTROUTING -t mangle -o eth0 -s 192.168.1.100/32 -j MARK --set-mark ${VOIP_mark} &> /dev/null                        #Template Rule 2 WORKING ALTERNATIVE (Line1/2)
#    iptables -A POSTROUTING -t mangle -o eth0 -s 192.168.1.100/32 -j MARK --set-mark ${VOIP_mark}                                     #Template Rule 2 WORKING ALTERNATIVE (Line2/2)

Everyone should feel free open the script with Notepad++ and take a look inside the script there as there is additional documentation not present in the first posts of this thread that can be beneficial depending on user need).

As for mis-categorization, it's simply deficiency in the trend micro rules responsible for deep packet analysis. This is the same reason I have custom rules to identify my VoIP traffic.

What I am really interested in how that traffic is getting into "Net Control". Can you turn on "App Analysis" and tell me the app name that the traffic is being identified as?

---

Also a new update for gaming rules is coming. Currently the gaming rules direct 80/443 traffic into downloads.

Users have reported that actual download traffic is drowning out port 80/443 gaming traffic.

Going forward, the default configuration will be to place 80/443 gaming traffic "Defaults" category. (This is the non WebUI adjustable category that is locked in the bottom place.)
This will allow gaming port 80/443 traffic to have it's own guaranteed traffic allotment making it so file downloads, torrent, etc will not drown it out anymore.
(This is necessary since log-in + matchmaking also operates on these ports in addition to the bulky game updates, This traffic was being heavily drown out by downloads leading to a poor experience logging in/matchmaking )

I will also move the priority of "Defaults" up one position. This means that if you last traffic category in WebUI is "Downloads", then the actual QOS traffic priority will be "Defaults" and then "Downloads".

I am a little busy right now, but the update will come soon.

 

[skeal]

If its an android media box, you can simply create a rule so that all traffic on that box is "Video Streaming".
With that you wouldn't have to micro manage and look for other identification deficiencies.

You can use these templates ( keep in mind that these templates have always been included inside the script itself )

Download
#    ${tc} filter add dev br0 protocol all prio $1 u32 match ip dst 192.168.1.100/32 flowid ${VOIP}                #Template Rule 2 (All incoming traffic w/ LAN destination ip 192.168.1.100                  goes to "VOIP" Traffic Container)                             
Upload
#    iptables -D POSTROUTING -t mangle -o eth0 -s 192.168.1.100/32 -j MARK --set-mark ${VOIP_mark} &> /dev/null                        #Template Rule 2 WORKING ALTERNATIVE (Line1/2)
#    iptables -A POSTROUTING -t mangle -o eth0 -s 192.168.1.100/32 -j MARK --set-mark ${VOIP_mark}                                     #Template Rule 2 WORKING ALTERNATIVE (Line2/2)

Everyone should feel free open the script with Notepad++ and take a look inside the script there as there is additional documentation not present in the first posts of this thread that can be beneficial depending on user need).

As for mis-categorization, it's simply deficiency in the trend micro rules responsible for deep packet analysis. This is the same reason I have custom rules to identify my VoIP traffic.

What I am really interested in how that traffic is getting into "Net Control". Can you turn on "App Analysis" and tell me the app name that the traffic is being identified as?

---

Also new update for gaming rules is coming. Currently the optional gaming rules direct 80/443 traffic into downloads.

Users have reported that actual download traffic is drowning out port 80/443 gaming traffic.

Going forward, the default configuration will be to place 80/443 gaming traffic "Defaults" category. (This is the non WebUI adjustable category that is locked in the bottom place.)
This will allow gaming port 80/443 traffic to have it's own guaranteed traffic allotment making it so file downloads, torrent, etc will not drown it out anymore.
(This is necessary since log-in + matchmaking also operates on these ports in addition to the bulky game updates, This traffic was being heavily drown out by downloads leading to a poor experience logging in/matchmaking )

I will also move the priority of "Defaults" up one position. This means that if you last traffic category in WebUI is "Downloads", then the actual QOS traffic priority will be "Defaults" and then "Downloads".

I am a little busy right now, but the update will come soon.

The traffic analyzer says the traffic is coming from Akamai.net. I tried the suggestion above but am not sure what to put in place of 'VOIP". I tried using "Streaming" but that still shows the traffic as "web surfing".

 

[FreshJR]

The traffic analyzer says the traffic is coming from Akamai.net. I tried the suggestion above but am not sure what to put in place of 'VOIP". I tried using "Streaming" but that still shows the traffic as "web surfing".

${Streaming} and ${Streaming_Mark}

These, and all the available variables, are listed in the header of the script!!

Don't forget to change the rule IP to match your box. The quoted rules were just a template!!

It wouldn't matter what the traffic will be identified as, since the this rule will override the original destination.

 

[skeal]

${Streaming} and ${Streaming_Mark}

These, and all the available variables, are listed in the header of the script!!

Don't forget to change the IP the match your box. The quoted rules were just a template!!

It wouldn't matter what the traffic will be identified as, since the this rule will override the original destination.

Gotcha thanks again @FreshJR !

 

[skeal]

${Streaming} and ${Streaming_Mark}

These, and all the available variables, are listed in the header of the script!!

Don't forget to change the rule IP to match your box. The quoted rules were just a template!!

It wouldn't matter what the traffic will be identified as, since the this rule will override the original destination.

Should this notation "${tc}" actually be "realtc" because I get this error in the logs if it isn't changed to "realtc". The log entry I get is:

May  6 17:12:14 adaptive QOS: /jffs/scripts/FreshJR_QOS: line 1: filter: not found

 

[FreshJR]

Should this notation "${tc}" actually be "realtc" because I get this error in the logs if it isn't changed to "realtc". The log entry I get is:

May  6 17:12:14 adaptive QOS: /jffs/scripts/FreshJR_QOS: line 1: filter: not found

No, ${tc} automatically adjusts itself into realtc OR tc depending on the users firmware.

You most likely placed the rules in a wrong location.

There is a specific section for download and upload rules. Read the green comments !!

The rules/templates are in the FreshJR_QOS file. I can see you being confused if you are looking at the fakeTC file.

 

[skeal]

No, ${tc} automatically adjusts itself into realtc OR tc depending on the users firmware.

You most likely placed the rules in a wrong location.

There is a specific section for download and upload rules. Read the green comments !!

All the green areas in download section of the top of the script use "realtc" so not sure what you mean there the help section gives examples using "realtc".

 

[FreshJR]

All the green areas in download section of the top of the script use "realtc" so not sure what you mean there the help section gives examples using "realtc".

Gotcha!!

You were correct, change it into realtc.

The templates in the "compat" & "fast" version differ slightly. I grabbed the templates from the compatible version.