[Release] FreshJR Adaptive QOS (Improvements / Custom Rules / and Inner workings)

[FreshJR]

Were there any recent changes incorporated to the legacy script in the last few days ?

Yes complete rewrite backporting everything in recent version to work with legacy fw. Experiencing issues?

@Sinner

https://www.snbforums.com/threads/r...-and-inner-workings.36836/page-50#post-387657

Try creating a rule using information in this post. I am here for auxiliary support.

 

[Sinner]

Wow ok thanx bro. Sorry to bother you

 

[FreshJR]

Wow ok thanx bro. Sorry to bother you

I could give you the rule or you can go through the process of creating your own rules so you can make whatever changes you want in the future.

If something is not working as expected, you could evaluate what rule might be the issue.

If that was sarcasm that's disrespectful. I provide you a lot of my time just to create the script. Let alone all the support in this thread. I offer all this free of charge. In addition I even have a 5page pm conversation with you answering all your questions. I had to piece together the functionality without information presented in a centralized place and explained in a simple manner. I have never even programmed when I started this thread. Smh
I even have to write the same the same thing twice or more within a few pages of each other.

 

[MysticGold04]

Yes complete rewrite backporting everything in recent version to work with legacy fw. Experiencing issues?

No issues, Im just using the older 1.92 version I downloaded a while ago. When I get some time this weekend I'll update it. Thank you!

 

[Sinner]

I could give you the rule or you can go through the process of creating your own rules so you can make whatever changes you want in the future.

If something is not working as expected, you could evaluate what rule might be the issue

actually its ok.. you were right and I needed a little push to figure out that hex codes stuff.. that windows calc was what I was needing. heres what ive gote and appears to be working I had one small question though. the bolded section I just copied from all the other lines that used more or less the same one without understanding it but appears to be working.. is that part correct?

incoming:
realtc filter add dev br0 protocol all prio 2 u32 match mark 0x8004000A 0xc03f00ff flowid ${Downloads} #Apple iTunes Downloads TEST

outgoing:
realtc filter add dev eth0 protocol all prio 2 u32 match mark 0x8004000A 0xc03f00ff flowid ${Downloads} #Apple iTunes Downloads TEST

 

[FreshJR]

actually its ok.. you were right and I needed a little push to figure out that hex codes stuff.. that windows calc was what I was needing. heres what ive gote and appears to be working I had one small question though. the bolded section I just copied from all the other lines that used more or less the same one without understanding it but appears to be working.. is that part correct?

incoming:
realtc filter add dev br0 protocol all prio 2 u32 match mark 0x8004000A 0xc03f00ff flowid ${Downloads} #Apple iTunes Downloads TEST

outgoing:
realtc filter add dev eth0 protocol all prio 2 u32 match mark 0x8004000A 0xc03f00ff flowid ${Downloads} #Apple iTunes Downloads TEST

Correct for the most part. Outgoing should always start with a 4 --> 0x4004000A

If total rules in that section go upto 255 mask can be 0xc03f00ff
If total rules in that section go past 255+ mask can be 0xc03fffff
(Only section I saw go past 255+ entries was "gaming" which is located at decimal 8)

Always using mask 0xc03fffff for custom rules should work in theory, but I didn't test it, that's why my rules have the smaller mask.

Enjoy this table for quick reference when looking at the DB

dec  (hex)  destination
0    (0x0)  ----VOIP
1    (0x1)  ---- FILE TRANSFERS
3    (0x3)  ----DOWNLOADS
4    (0x4)  ----STREAMING
5    (0x5)  ----VOIP
6    (0x6)  ----VOIP
7    (0x7)  ----OTHERS
8    (0x8)  ----GAMING
9    (0x9)  ----NET
10   (0xA)  ----OTHERS
11   (0xB)  ----OTHERS
12   (0xC)  ----STREAMING
13   (0xD)  ----WEB
14   (0xE)  ----DOWNLOADS
15   (0xF)  ----VOIP
17   (0x11) ----VOIP
18   (0x12) ----NET
19   (0x13) ----NET  ----->  OTHERS
20   (0x14) ----NET  -----> OTHERS
21   (0x15) ----OTHERS       (EMPTY SECTION)
22   (0x16) ----NOT PRESENT  (ZERO RATED)
23   (0x17) ----OTHERS
24   (0x18) -----WEB
26   (0x1A) ----NOT PRESENT  (ZERO RATED)

 

[iaTa]

I'm using the legacy script on 380.65 and trying to associate traffic to and from port 563 to downloads:

Down: realtc filter add dev br0 protocol all prio 1 u32 match ip dport 563 0xffff flowid ${Downloads} #NZBGet
Up: realtc filter add dev eth0 protocol all prio 1 u32 match ip sport 563 0xffff flowid ${Downloads} #NZBGet

But it's still being classified as VOIP. Why would that be?

 

[FreshJR]

I'm using the legacy script on 380.65 and trying to associate traffic to and from port 563 to downloads:

Down: realtc filter add dev br0 protocol all prio 1 u32 match ip dport 563 0xffff flowid ${Downloads} #NZBGet
Up: realtc filter add dev eth0 protocol all prio 1 u32 match ip sport 563 0xffff flowid ${Downloads} #NZBGet

But it's still being classified as VOIP. Why would that be?

Are you sure it's destined to port 563 on your computer, rather than coming from port 563 on the server?

switch sport & dport around if that's the case.

 

[Sinner]

Correct for the most part. Outgoing should always start with a 4 --> 0x4004000A

If total rules in that section go upto 255 mask can be 0xc03f00ff
If total rules in that section go past 255+ mask can be 0xc03fffff
(Only section I saw go past 255+ entries was "gaming" which is located at decimal 8)

Always using mask 0xc03fffff for custom rules should work in theory, but I didn't test it, that's why my rules have the smaller mask.

ty good catch on the outgoing "4". ive only added 3 rules and am not using the gaming ones. ive mostly redirected a couple you already had into voip. communication/gaming highest priority here so to ensure all communication possible was top I use VOIP as my top priority container including the actual pc-gaming which uses almost no data really. the xbox/playstation stuff for the moment is ok where it is. media streaming etc are mid range of importance and windows updates/device downloads etc are totally least of importance so this has made me a happy camper! TY soo much for all your hard work, you've done an incredible job making adaptive QoS amazing to use! Personally I don't think I can even think of anything more Id like out of it

 

[Sinner]

ty good catch on the outgoing "4". ive only added 3 rules and am not using the gaming ones. ive mostly redirected a couple you already had into voip. communication/gaming highest priority here so to ensure all communication possible was top I use VOIP as my top priority container including the actual pc-gaming which uses almost no data really. the xbox/playstation stuff for the moment is ok where it is. media streaming etc are mid range of importance and windows updates/device downloads etc are totally least of importance so this has made me a happy camper! TY soo much for all your hard work, you've done an incredible job making adaptive QoS amazing to use! Personally I don't think I can even think of anything more Id like out of it

before I save/install new change.. is the prio 2 correct for this rule? that also a little unclear for me as I notice half of them you have in there are prio 15 particularly the other google/apple rules you've added made me curious. or knowing what that actually means might also help

 

[iaTa]

Are you sure it's destined to port 563 on your computer, rather than coming from port 563 on the server?

The server I'm downloading from is running on port 563. I've just tried switching them around and it's still showing as VOIP.

 

[FreshJR]

before I save/install new change.. is the prio 2 correct for this rule? that also a little unclear for me as I notice half of them you have in there are prio 15 particularly the other google/apple rules you've added made me curious.

Ahh you noticed!

So what happens is that every packet gets checked against the rule list in TC from top to bottom3

So it goes

Pref1) Is this LAN
Pref2) Is this Unidentified
Pref3) Is it VOIP <--- this is an exception, but idea is to immediately identify this traffic
Pref4) Is it P2P
Pref6) Is it Web Downloads
Pref7) Is it Streaming

The idea is the sort traffic with the least ammount of questions, as each question uses CPU power and takes a finite amount of time. The most efficient way to ask the question to ask them by most commonly to least common statistically observed traffic.

For custom rules, its better most ideal to make the prio (1) less than the original rule.

The original rules are available by

"tc filter show dev br0 | grep "flowid" -A1"

Here's an unmolested copy free from custom rules.

filter parent 1: protocol all pref 1 u32 fh 800::800 order 2048 key ht 800 bkt 0 flowid 1:2
  mark 0x0000 0xc0000000
--
filter parent 1: protocol all pref 2 u32 fh 825::804 order 2052 key ht 825 bkt 0 flowid 1:17
  mark 0x80000000 0x8000ffff
--
filter parent 1: protocol all pref 3 u32 fh 803::800 order 2048 key ht 803 bkt 0 flowid 1:11
  mark 0x80000000 0xc03f0000
--
filter parent 1: protocol all pref 4 u32 fh 822::800 order 2048 key ht 822 bkt 0 flowid 1:16
  mark 0x80010000 0xc03f0000
--
filter parent 1: protocol all pref 6 u32 fh 823::800 order 2048 key ht 823 bkt 0 flowid 1:16
  mark 0x80030000 0xc03f0000
--
filter parent 1: protocol all pref 7 u32 fh 821::800 order 2048 key ht 821 bkt 0 flowid 1:15
  mark 0x80040000 0xc03f0000
--
filter parent 1: protocol all pref 8 u32 fh 804::800 order 2048 key ht 804 bkt 0 flowid 1:11
  mark 0x80050000 0xc03f0000
--
filter parent 1: protocol all pref 9 u32 fh 805::800 order 2048 key ht 805 bkt 0 flowid 1:11
  mark 0x80060000 0xc03f0000
--
filter parent 1: protocol all pref 10 u32 fh 809::800 order 2048 key ht 809 bkt 0 flowid 1:13
  mark 0x80070000 0xc03f0000
--
filter parent 1: protocol all pref 11 u32 fh 808::800 order 2048 key ht 808 bkt 0 flowid 1:12
  mark 0x80080000 0xc03f0000
--
filter parent 1: protocol all pref 12 u32 fh 802::800 order 2048 key ht 802 bkt 0 flowid 1:10
  mark 0x80090000 0xc03f0000
--
filter parent 1: protocol all pref 13 u32 fh 80a::800 order 2048 key ht 80a bkt 0 flowid 1:13
  mark 0x800a0000 0xc03f0000
--
filter parent 1: protocol all pref 14 u32 fh 80b::800 order 2048 key ht 80b bkt 0 flowid 1:13
  mark 0x800b0000 0xc03f0000
--
filter parent 1: protocol all pref 15 u32 fh 820::800 order 2048 key ht 820 bkt 0 flowid 1:15
  mark 0x800c0000 0xc03f0000
--
filter parent 1: protocol all pref 16 u32 fh 81e::800 order 2048 key ht 81e bkt 0 flowid 1:14
  mark 0x800d0000 0xc03f0000
--
filter parent 1: protocol all pref 17 u32 fh 824::800 order 2048 key ht 824 bkt 0 flowid 1:16
  mark 0x800e0000 0xc03f0000
--
filter parent 1: protocol all pref 18 u32 fh 806::800 order 2048 key ht 806 bkt 0 flowid 1:11
  mark 0x800f0000 0xc03f0000
--
filter parent 1: protocol all pref 20 u32 fh 807::800 order 2048 key ht 807 bkt 0 flowid 1:11
  mark 0x80110000 0xc03f0000
--
filter parent 1: protocol all pref 21 u32 fh 801::800 order 2048 key ht 801 bkt 0 flowid 1:10
  mark 0x80120000 0xc03f0000
--
filter parent 1: protocol all pref 22 u32 fh 826::800 order 2048 key ht 826 bkt 0 flowid 1:14
  mark 0x80130000 0xc03f0000
--
filter parent 1: protocol all pref 23 u32 fh 827::800 order 2048 key ht 827 bkt 0 flowid 1:14
  mark 0x80140000 0xc03f0000
--
filter parent 1: protocol all pref 24 u32 fh 80c::800 order 2048 key ht 80c bkt 0 flowid 1:13
  mark 0x80150000 0xc03f0000
--
filter parent 1: protocol all pref 26 u32 fh 80d::800 order 2048 key ht 80d bkt 0 flowid 1:13
  mark 0x80170000 0xc03f0000
--
filter parent 1: protocol all pref 27 u32 fh 81f::800 order 2048 key ht 81f bkt 0 flowid 1:14
  mark 0x80180000 0xc03f0000
--

I was thinking of compiling a table for most optimal prio depending on where it came from the DB but I didn't know if anyone would be making custom rules.

Technically all custom rules as prio2 will work, but why be wasteful.

When I was redirecting "Speedtest/GooglePlay/AppStore" they were originally 0x800D.
0x800D is a PREF16 question.
Since I wanted to ask my question before, I inserted by question at PREF15.

 

[Sinner]

oh I have been because my needs are custom. would be retarded to ask you to update this for me personally which is why ive been trying to figure it out myself heres what I'm working with to give you an idea:

not sure as of yet if I require the snapchat rule but left it for the time being.

##DOWNLOAD (INCOMMING TRAFFIC) CUSTOM RULES START HERE
 
   realtc filter add dev br0 protocol all prio 2 u32 match ip dport 500 0xffff flowid ${VOIP}            #Wifi Calling (1/2) - (All incoming traffic w/ LAN destination port 500  goes to "VOIP" Traffic Container)       
   realtc filter add dev br0 protocol all prio 2 u32 match ip dport 4500 0xffff flowid ${VOIP}            #Wifi Calling (2/2) - (All incoming traffic w/ LAN destination port 4500 goes to "VOIP" Traffic Container)
   realtc filter add dev br0 protocol all prio 2 u32 match ip dport 16384 0xffe0 flowid ${VOIP}           #Facetime
   realtc filter add dev br0 protocol all prio 2 u32 match mark 0x8000006B 0xc03f00ff flowid ${Others}          #Snapchat
   realtc filter add dev br0 protocol all prio 15 u32 match mark 0x800D0007 0xc03f00ff flowid ${Downloads}         #Speedtest.net
   realtc filter add dev br0 protocol all prio 15 u32 match mark 0x800D0086 0xc03f00ff flowid ${Downloads}         #Google Play
   realtc filter add dev br0 protocol all prio 15 u32 match mark 0x800D00A0 0xc03f00ff flowid ${Downloads}         #Apple AppStore
   realtc filter add dev br0 protocol all prio 2 u32 match mark  0x8004000A 0xc03f00ff flowid ${Downloads}         #Apple iTunes Downloads TEST
   realtc filter add dev br0 protocol all prio 2 u32 match ip dport 30584 0xffff flowid ${VOIP}           #Entropia Universe (All incoming traffic goes into VOIP)
   realtc filter add dev br0 protocol all prio 2 u32 match ip dport 9501 0xffff flowid ${VOIP}            #WOW NightHaven (all incoming traffic into VOIP)
   #realtc filter add dev br0 protocol all prio 10 u32 match ip dst 192.168.1.100/30 match mark 0x80000000 0x8000ffff flowid ${Gaming}     #Gaming (1/3) - Routes Unidentified Traffic into "Gaming", instead of "Others", for LAN clients specified
   realtc filter add dev br0 protocol all prio 10 u32 match mark 0x80080000 0xc03f0000 match ip sport 443 0xffff flowid ${Downloads}       #Gaming (2/3) - Routes "Gaming" traffic coming from port 443 into "Downloads"
   realtc filter add dev br0 protocol all prio 10 u32 match mark 0x80080000 0xc03f0000 match ip sport 80 0xffff flowid ${Downloads}   #Gaming (3/3) - Routes "Gaming" traffic coming from port 80 into "Downloads"
 
  ##DOWNLOAD (INCOMMING TRAFFIC) CUSTOM RULES END HERE
  realtc filter add dev br0 protocol all prio 2 u32 match mark 0x80000000 0x8000ffff flowid ${VOIP}           #Unidentified traffic - Routes Unidentified Traffic into webUI adjustable "Others" traffic container, instead of "Default" traffic container
  realtc filter add dev br0 protocol all prio 22 u32 match mark 0x80130000 0xc03f0000 flowid ${Web}           #HTTPS
  realtc filter add dev br0 protocol all prio 23 u32 match mark 0x80140000 0xc03f0000 flowid ${Web}           #HTTPS
 }

 custom_up_rules() {
  set_variables
  echo "Applying ------ Up Rules"
  ##UPLOAD (OUTGOING TRAFFIC) CUSTOM RULES START HERE
   realtc filter add dev eth0 protocol all prio 2 u32 match ip sport 500 0xffff flowid ${VOIP}           #Wifi Calling (1/2) - (All outgoing traffic w/ LAN source port 500  goes to "VOIP" Traffic Container)         
   realtc filter add dev eth0 protocol all prio 2 u32 match ip sport 4500 0xffff flowid ${VOIP}          #Wifi Calling (2/2) - (All outgoing traffic w/ LAN source port 4500 goes to "VOIP" Traffic Container)
   realtc filter add dev eth0 protocol all prio 2 u32 match ip sport 16384 0xffe0 flowid ${VOIP}          #Facetime
   realtc filter add dev eth0 protocol all prio 2 u32 match mark 0x4000006B 0xc03f00ff flowid ${Others}        #Snapchat
   realtc filter add dev eth0 protocol all prio 15 u32 match mark 0x400D0007 0xc03f00ff flowid ${Downloads}       #Speedtest.net
   realtc filter add dev eth0 protocol all prio 15 u32 match mark 0x400D0086 0xc03f00ff flowid ${Downloads}       #Google Play
   realtc filter add dev eth0 protocol all prio 15 u32 match mark 0x400D00A0 0xc03f00ff flowid ${Downloads}       #Apple AppStore
   realtc filter add dev eth0 protocol all prio 2 u32 match mark  0x4004000A 0xc03f00ff flowid ${Downloads}       #Apple iTunes Downloads TEST
   realtc filter add dev eth0 protocol all prio 2 u32 match ip sport 30584 0xffff flowid ${VOIP}          #Entropia Universe (All incoming into VOIP)
   realtc filter add dev eth0 protocol all prio 2 u32 match ip sport 9501 0xffff flowid ${VOIP}          #WOW NightHaven (all outgoing into VOIP)
   realtc filter add dev eth0 protocol all prio 10 u32 match mark 0x40080000 0xc03f0000 match ip dport 443 0xffff flowid ${Downloads}  #Gaming (1/4) - Routes "Gaming" traffic going to port 443 into "Downloads"
   realtc filter add dev eth0 protocol all prio 10 u32 match mark 0x40080000 0xc03f0000 match ip dport 80 0xffff flowid ${Downloads} #Gaming (2/4) - Routes "Gaming" traffic going to port 80 into "Downloads"
  ##UPLOAD (OUTGOING TRAFFIC) CONTINUED BELOW
  realtc filter add dev eth0 protocol all prio 2 u32 match mark 0x40000000 0x4000ffff flowid ${VOIP}          #Unidentified traffic - Routes Unidentified Traffic into webUI adjustable "Others" traffic container, instead of "Default" traffic container
  realtc filter add dev eth0 protocol all prio 22 u32 match mark 0x40130000 0xc03f0000 flowid ${Web}          #HTTPS
  realtc filter add dev eth0 protocol all prio 23 u32 match mark 0x40140000 0xc03f0000 flowid ${Web}          #HTTPS

 

[Sinner]

#Percent of download speed guaranteed per QOS category, change below as desired (sum should equal 100)
  NetControl_DownBandPercent=5     #This value can be adjust as desired
  VoIP_DownBandPercent=20       #This value can be adjust as desired
  Gaming_DownBandPercent=10      #This value can be adjust as desired
  Others_DownBandPercent=10      #This value can be adjust as desired  #Note: New destination for all unidentified traffic per script default
  WebSurfing_DownBandPercent=20     #This value can be adjust as desired
  Video_DownBandPercent=20      #This value can be adjust as desired
  FileTransfer_DownBandPercent=10     #This value can be adjust as desired
  Default_DownBandPercent=5      #This value can be adjust as desired  #Note: Original destination all for unidentified traffic, no traffic should flow here
  #Percent of upload speed guaranteed per QOS category, change below as desired (sum should equal 100)
  NetControl_UpBandPercent=5      #This value can be adjust as desired
  VoIP_UpBandPercent=20       #This value can be adjust as desired
  Gaming_UpBandPercent=10       #This value can be adjust as desired
  Others_UpBandPercent=10       #This value can be adjust as desired  #Note: New destination for all unidentified traffic per script default
  WebSurfing_UpBandPercent=20      #This value can be adjust as desired
  Video_UpBandPercent=20       #This value can be adjust as desired
  FileTransfer_UpBandPercent=10     #This value can be adjust as desired
  Default_UpBandPercent=5       #This value can be adjust as desired

also using modified %'s as well since me personally and most other have internet connections that are 10/1 it made more sense to me to just keep the ul/dl matching and closer to my avg needs so the router is shaping only when required not conststantly.

 

[FreshJR]

oh I have been because my needs are custom. would be retarded to ask you to update this for me personally which is why ive been trying to figure it out myself heres what I'm working with to give you an idea:

What's wrong with it?

Also routers always shaping. The %'s don't matter.
They are only there for you to determine a guaranteed rate.
Even if you are using less than guaranteed it's still shaping.

Insert code between

[c0de]
[/c0de] blocks or use pastebin!!

The forums will be hard to read if you spam it like that.

--

Your itunes rule would be more efficient at prio 6.

 

[FreshJR]

The server I'm downloading from is running on port 563. I've just tried switching them around and it's still showing as VOIP.

Switch only the words sport dport around.

Did you wait until the script applies changes as shown in the system log?

If the connection was already active before the script changes have been applied, you have to stop and start the connection, so it get's a new connection mark. The connection is only marked once upon starting a believe, so it might have the outdated initial mark.

Another user here has nzb rules. I think it was @Jack Yaz

 

[Sinner]

lol I was wondering how you did that ok I'm done with that now.

What's wrong with it?

Also routers always shaping. The %'s don't matter.
They are only there for you to determine a guaranteed rate.
Even if you are using less than guaranteed it's still shaping.

Insert code between

[c0de]
[/c0de] blocks or use pastebin!!

The forums will be hard to read if you spam it like that.

ok ty for that.. updated the to code.. also ty for the prio explantion.. ill update it to pri15 is then as well.. because its not common but when it happens needs to be deal with.. ty

 

[FreshJR]

lol I was wondering how you did that ok I'm done with that now.


ok ty for that.. updated the to code.. also ty for the prio explantion.. ill update it to pri15 is then as well.. because its not common but when it happens needs to be deal with.. ty

No iTunes need prio6 or lower because it will get detected at the prio7 rule which asks "Is this streaming traffic"

If you put it in prio15, it will not make it that far down the questioning list.

 

[Sinner]

No iTunes need prio6 or lower because it will get detected at the prio7 rule which asks "Is this streaming traffic"

If you put it in prio15, it will not make it that far down the questioning list.

Sorry didnt fully understand that.

Theres no need to put it 6 or lower? Because it gets detected anyway at 7?

*** nm that i think i undersand what you meant now. It needs to be lower than 6 or 7 because its getting detetced sooner at 6 where i have it redirected?

 

[iaTa]

Switch only the words sport dport around.

Did you wait until the script applies changes as shown in the system log?

If the connection was already active before the script changes have been applied, you have to stop and start the connection, so it get's a new connection mark. The connection is only marked once upon starting a believe, so it might have the outdated initial mark.

Another user here has nzb rules. I think it was @Jack Yaz

Yeah sorry I only switch around dport and sport.

Yes the syslog shows applying up and down rules.

That's actually how I found this thread, from Jack's posts regarding NZB rules.

I can't see why it wouldn't be classifying the traffic as downloads.