[Release] FreshJR Adaptive QOS (Improvements / Custom Rules / and Inner workings)

[dave14305]

1- outbund connections are being shown as UNTRACKED (green category) and marked as 00000. is there a way to properly identify them on the purple category?

I might suggest you change the rule for 51415 to be !51416 so that only 51416 goes to Streaming and everything else goes to File Downloads.

2- i see a lot of activity on the brown category but i dont see any connections marked with that color on the list.

Let’s confirm that the default rule is in place:

tc filter show dev eth0 | grep -B1 0x4000ffff

The connection list coloring is independent of the actual underlying traffic flows. The connection table shows what is expected to happen with traffic classes, based on the rules defined, but if the rules do not apply correctly, the colors will not match where we see the traffic in the pie chart.

 

[andresmorago]

I might suggest you change the rule for 51415 to be !51416 so that only 51416 goes to Streaming and everything else goes to File Downloads.

Let’s confirm that the default rule is in place:

tc filter show dev eth0 | grep -B1 0x4000ffff

The connection list coloring is independent of the actual underlying traffic flows. The connection table shows what is expected to happen with traffic classes, based on the rules defined, but if the rules do not apply correctly, the colors will not match where we see the traffic in the pie chart.

hello dave

Regarding the ports, let me put you in a better context before i do any changes. sorry, i just wanna make sure!
port 51416 is for emby server running on the synology server (10.0.0.2). this traffic is set to Streaming
port 51415 is for incoming torrent connections on the synology server (10.0.0.2). this traffic is set File downloads

Regarding the rule check, the code yields no output

andresmorago@RT-AC3100-0548:/tmp/home/root# tc filter show dev eth0 | grep -B1 0x4000ffff
andresmorago@RT-AC3100-0548:/tmp/home/root#

 

[dave14305]

Regarding the rule check,

tc filter show dev eth0 | grep -B1 0x4000ffff

yields no output

It would help if you run the correct command instead of the one I gave you.

tc filter show dev eth0 | grep -B1 0xc000ffff

And still running 384.15?

 

[andresmorago]

It would help if you run the correct command instead of the one I gave you.

tc filter show dev eth0 | grep -B1 0xc000ffff

thanks! got an output

andresmorago@RT-AC3100-0548:/tmp/home/root# tc filter show dev eth0 | grep -B1 0xc000ffff

filter parent 1: protocol all pref 2 u32 fh 827::800 order 2048 key ht 827 bkt 0 flowid 1:17
  mark 0x40000000 0xc000ffff (success 398155)

And still running 384.15?

yes. i wish i could update now but im out of my country and dont want to risk my router stability by trying to update it remotely

 

[dave14305]

thanks! got an output

andresmorago@RT-AC3100-0548:/tmp/home/root# tc filter show dev eth0 | grep -B1 0xc000ffff

filter parent 1: protocol all pref 2 u32 fh 827::800 order 2048 key ht 827 bkt 0 flowid 1:17
  mark 0x40000000 0xc000ffff (success 398155)

Ok, something restarted QoS at some point. Run:

/jffs/scripts/FreshJR_QOS -check

to reapply the tc rules.

 

[webs0r]

1- outbund connections are being shown as UNTRACKED (green category) and marked as 00000. is there a way to properly identify them on the purple category?

Another alternative is what I've done on my setup - if your torrent client lets you specify the local port range for outbound connections.

Whilst you have set a specific port & rule that captures torrent traffic triggered by an inbound request, there is the matter of when your torrent client initiates a
outbound request.

As you can see in your connection list, the system generally assigns a random local port >49152 for these connections.

So - set options in your torrent client to use a defined range of ports for outbound traffic. Then define a rule to classify this traffic as file downloads.
In my case I've set 60010 - 60300 to be this range.

In the cap below, you can see where the engine auto-magically-detects Bittorrent traffic and tags it at "BitTorrent Series".
Then for the traffic it doesn't know (this would be the "Untracked" in your shot), my Rule1 captures it.

 

[andresmorago]

Ok, something restarted QoS at some point. Run:

/jffs/scripts/FreshJR_QOS -check

to reapply the tc rules.

ran it and waited, just in case, 5 minutes. before i ran the rule check and generated no output

andresmorago@RT-AC3100-0548:/tmp/home/root# /jffs/scripts/FreshJR_QOS -check
adaptive QOS: Scheduled Persistence Check -> Reapplying Changes
andresmorago@RT-AC3100-0548:/tmp/home/root# tc filter show dev eth0 | grep -B1 0xc000ffff
andresmorago@RT-AC3100-0548:/tmp/home/root#

 

[andresmorago]

Another alternative is what I've done on my setup - if your torrent client lets you specify the local port range for outbound connections.

Whilst you have set a specific port & rule that captures torrent traffic triggered by an inbound request, there is the matter of when your torrent client initiates a
outbound request.

As you can see in your connection list, the system generally assigns a random local port >49152 for these connections.

So - set options in your torrent client to use a defined range of ports for outbound traffic. Then define a rule to classify this traffic as file downloads.
In my case I've set 60010 - 60300 to be this range.

In the cap below, you can see where the engine auto-magically-detects Bittorrent traffic and tags it at "BitTorrent Series".
Then for the traffic it doesn't know (this would be the "Untracked" in your shot), my Rule1 captures it.
View attachment 24154

interesting implementation. ill have to do some research on how can i configure the outbound ports on synology torrent client

 

[dave14305]

ran it and waited, just in case, 5 minutes. before i ran the rule check and generated no output

andresmorago@RT-AC3100-0548:/tmp/home/root# /jffs/scripts/FreshJR_QOS -check
adaptive QOS: Scheduled Persistence Check -> Reapplying Changes
andresmorago@RT-AC3100-0548:/tmp/home/root# tc filter show dev eth0 | grep -B1 0xc000ffff
andresmorago@RT-AC3100-0548:/tmp/home/root#

Ok, I see why I was confused earlier. Run:

tc filter show dev eth0 | grep -B1 000ffff

 

[andresmorago]

Ok, I see why I was confused earlier. Run:

tc filter show dev eth0 | grep -B1 000ffff

andresmorago@RT-AC3100-0548:/tmp/home/root# tc filter show dev eth0 | grep -B1 000ffff

filter parent 1: protocol all pref 2 u32 fh 82a::801 order 2049 key ht 82a bkt 0 flowid 1:13
  mark 0x40000000 0x4000ffff (success 3825)

andresmorago@RT-AC3100-0548:/tmp/home/root#

 

[Audionut]

Your Resource Monitor screenshot shows that the upload is going to a host that does not match your rule. So not an issue with the script.

# dig +short syd09s17-in-f10.1e100.net
172.217.167.106

This should be covered by 172.217.0.0/16.
Added a screenshot of just the iprules since the other SS is hard to read.

edit: and the app_rules screenshot in the previous post shows that IP being covered by rule 2.

 

[Audionut]

This next upload its hitting the 142.250.0.0/15 (rule 3) and is being sent to the correct queue.

What is the Mark associated with the uploads?

1400BA

iptables -t mangle -nvL POSTROUTING

Chain POSTROUTING (policy ACCEPT 1272K packets, 1057M bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 MARK       udp  --  *      br0     0.0.0.0/0            0.0.0.0/0            multiport sports 500,4500 MARK set 0x80060001
    7   582 MARK       udp  --  *      br0     0.0.0.0/0            0.0.0.0/0            udp dpts:16384:16415 MARK set 0x80060001
    0     0 MARK       tcp  --  *      br0     0.0.0.0/0            0.0.0.0/0            multiport sports 119,563 MARK set 0x80030001
 1770  162K MARK       all  --  *      br0     0.0.0.0/0            0.0.0.0/0            mark match 0x40000000/0xc0000000 MARK xset 0x80000000/0xc0000000
76181  104M MARK       tcp  --  *      br0     0.0.0.0/0            0.0.0.0/0            mark match 0x80080000/0xc03f0000 multiport sports 80,443 MARK set 0x803f0001
    0     0 MARK       tcp  --  *      br0     0.0.0.0/0            192.168.50.51        mark match 0x80000000/0x8000ffff multiport sports  !80,443 MARK set 0x80080001
 277K   40M MARK       udp  --  *      br0     0.0.0.0/0            192.168.50.51        mark match 0x80000000/0x8000ffff multiport sports  !80,443 MARK set 0x80080001
 177M  263G MARK       tcp  --  *      br0     18.205.93.0/29       0.0.0.0/0            tcp spt:443 mark match 0x8013005e/0xc03fffff MARK set 0x800a0001
    0     0 MARK       udp  --  *      br0     18.205.93.0/29       0.0.0.0/0            udp spt:443 mark match 0x8013005e/0xc03fffff MARK set 0x800a0001
20899 7496K MARK       all  --  *      br0     172.217.0.0/16       0.0.0.0/0            MARK set 0x80030001
 133K   19M MARK       all  --  *      br0     142.250.0.0/15       0.0.0.0/0            MARK set 0x80030001
 150K  182M MARK       all  --  *      br0     216.58.192.0/19      0.0.0.0/0            MARK set 0x80030001
    0     0 MARK       udp  --  *      eth0    0.0.0.0/0            0.0.0.0/0            multiport dports 500,4500 MARK set 0x40060001
    0     0 MARK       udp  --  *      eth0    0.0.0.0/0            0.0.0.0/0            udp spts:16384:16415 MARK set 0x40060001
    0     0 MARK       tcp  --  *      eth0    0.0.0.0/0            0.0.0.0/0            multiport dports 119,563 MARK set 0x40030001
    0     0 MARK       tcp  --  *      eth0    0.0.0.0/0            0.0.0.0/0            mark match 0x40080000/0xc03f0000 multiport sports 80,443 MARK set 0x403f0001
    0     0 MARK       tcp  --  *      eth0    192.168.50.51        0.0.0.0/0            mark match 0x40000000/0x4000ffff multiport dports  !80,443 MARK set 0x40080001
 617K   67M MARK       udp  --  *      eth0    192.168.50.51        0.0.0.0/0            mark match 0x40000000/0x4000ffff multiport dports  !80,443 MARK set 0x40080001
 150M 7894M MARK       tcp  --  *      eth0    0.0.0.0/0            18.205.93.0/29       tcp dpt:443 mark match 0x4013005e/0xc03fffff MARK set 0x400a0001
    0     0 MARK       udp  --  *      eth0    0.0.0.0/0            18.205.93.0/29       udp dpt:443 mark match 0x4013005e/0xc03fffff MARK set 0x400a0001
19337 5713K MARK       all  --  *      eth0    0.0.0.0/0            172.217.0.0/16       MARK set 0x40030001
 260K  344M MARK       all  --  *      eth0    0.0.0.0/0            142.250.0.0/15       MARK set 0x40030001
92708   20M MARK       all  --  *      eth0    0.0.0.0/0            216.58.192.0/19      MARK set 0x40030001

I'll do some more testing and see what happens when it chooses the 172.217.0.0/16 again.

 

[dave14305]

This should be covered by 172.217.0.0/16.
Added a screenshot of just the iprules since the other SS is hard to read.

edit: and the app_rules screenshot in the previous post shows that IP being covered by rule 2.

The new screenshot helps. So yes, that particular upload was captured in rule 2. You should observe what connections are in Web Surfing the next time you see high uploads. It may be from another device on your network.

 

[Mohammad Wahiduzzaman]

anyway to add more than 4 iptable rules ?i want to add more rules but i ran out of space on the iptable. need help

 

[maghuro]

anyway to add more than 4 iptable rules ?i want to add more rules but i ran out of space on the iptable. need help

You can define a subnet for all your streaming devices.

For example, I have all my streaming devices (mostly Chromecast) manually assigned from 10.0.0.65 to 10.0.0.70

On iptables, I have set 10.0.0.65/29 to streaming. And all of them are inside that subnet

 

[Audionut]

The new screenshot helps. So yes, that particular upload was captured in rule 2. You should observe what connections are in Web Surfing the next time you see high uploads. It may be from another device on your network.

I realise I'm not the smartest cookie in the cookie jar, but I'm not the dumbest one in the jar either.

Windows reporting 4,000,000 B/sec (connection limit), and QOS reporting 3,937 kB/s. This occurred while QOS was reporting the data being captured by rule 2.

 

[Audionut]

Here's a new rule. Waited well in excess of 5 minutes after applying.

https://i.postimg.cc/qRxvYqJb/rule.png

I wonder if persistent connections remain in the default category, and it's only new connections that hit the rule that get directed correctly?

 

[dave14305]

I realise I'm not the smartest cookie in the cookie jar, but I'm not the dumbest one in the jar either.

I realize I’m not the dumbest cookie in the jar, but I’m not the smartest one either. I intended to convey that without seeing any connections showing (or not) in the Web Surfing class, it’s hard to tell whether it’s a problem in the script.

I wonder if persistent connections remain in the default category, and it's only new connections that hit the rule that get directed correctly?

Here’s some wisdom from the past:
[Release] FreshJR Adaptive QOS (Improvements / Custom Rules / and Inner workings)

 

[dave14305]

anyway to add more than 4 iptable rules ?i want to add more rules but i ran out of space on the iptable. need help

Immediate option is to manually add your rules in the script in the “legacy” sections. Near-future option is to wait until I am ready to release a new version of the script that allows for more rules in the webui. It will require 384.18 or higher though.

 

[Audionut]

Here’s some wisdom from the past:

There it is, thankyou.

"Ports explained" a few posts down is linked in the OP.