[Release] FreshJR Adaptive QOS (Improvements / Custom Rules / and Inner workings)

[skeal]

@Testscript

Just keep the default https rules. Those two entire categories I redirected get many hits under different names when transferring https traffic instead of just that one entry you have listed.

@skeal

Lowered bursts? Or router change?

I can't figure out how to adjust burst. I would like to adjust it down, as I have a spike at the beginning of my tests.

 

[FreshJR]

@shelleyevans only ONCE on the main router that receives the internet connection.

The main router will be applying QOS for the entire remainder of the switches included.

 

[skeal]

I can't find where to modify burst, can someone explain please?

 

[Kingp1n]

Thank you for having me patience in my confusing explanation

Download rules: (Now does not limit the LAN traffic and It is not necessary to add "IP")

iptables -D POSTROUTING -t mangle -o br0 -m mark --mark 0x40000000/0xc0000000 -j MARK --set-xmark 0x80000000/0xC0000000 &> /dev/null       #Fix traffic when use VPN Client
iptables -A POSTROUTING -t mangle -o br0 -m mark --mark 0x40000000/0xc0000000 -j MARK --set-xmark 0x80000000/0xC0000000                    #Fix traffic when use VPN Client

Upload rules: (It is not necessary to add "IP", "Protocol" and "Port")

iptables -D OUTPUT -t mangle -o $wan -j MARK --set-mark ${Downloads_mark_up} &> /dev/null       #Fix traffic when use VPN Client
iptables -A OUTPUT -t mangle -o $wan -j MARK --set-mark ${Downloads_mark_up}                    #Fix traffic when use VPN Client

@FreshJR are these rules better to use than your original post thatvhad the ip address under the download within script when using openvpn?

 

[FreshJR]

@Kingp1n yes

 

[Testscript]

Link: https://www.dropbox.com/s/emit5ajgmo0a5cj/FreshJR_QOS.sh?dl=0

@FreshJR Can you look at my script if everything is ok? (Gaming rule #3 focused only for consoles)

My changes:

1. I use the "Default Category" for Unidentified Traffic and this is the priority 2. (under the "Gaming category")
   
   
   
   
2. Redirect the "Gaming" traffic from the ports 80/443 to the Category "Other" to be able to move the category where I want.
   Example: Download a game update quickly if I move the Category "Others" to the second position (Remember the VPN Traffic on the computer it is also in this category)
   
3. I kept bursts Original for "Net Control, Gaming, Default and VoIP". Minimum burst (1600b) for "Video Streaming, Others, Web Surfing and File Transferring".
4. My Custom Order:
   Standard
   
   
   
   
   
   Download Games
   
   

 

[FreshJR]

@Testscript

All your modifications are working as you wanted them to. Congrats.

Seems like you understand the internal configuration pretty well.

 

[shelleyevans]

I hope this isn't against forum rules-- I would love to make a donation, to thank you, @FreshJR, for your hard work and thoughtful support. I searched for a link, like @RMerlin has to his website, but can't find anything. How can we thank you?

 

[FreshJR]

No donations. Enjoy.

 

[shelleyevans]

I am. As you can see.

 

[vw-kombi]

I have this all setup for the xbox gaming traffic fine and that works with no issues.
I have also hardcoded my main PC's IP address (192.168.1.10) for uploads and downloads to go to downloads (for the odd manual torrent etc) - working correctly.

However I have an unraid server (ip 192.168.1.7) with a docker installed for nginx reverse proxy so all incoming wan traffic is destination 443, which has a port forward to the nginx docker, which reroutes services based on subdomains to the backend services (emby server, sabnzdb, hosted personal web server etc) - hence I don't believe I can specify a dedicated port or dedicated ip address to direct sabnzbd downloads to the downloads mark as it is all destination port 443 and destination IP 192.168.1.7.
Am I correct in this ?

I have this coded for uploads from unraid for the 'streaming' - when people are streaming video from my system, this works fine (but personal web traffic is grouped in also - which i can liove with as it is minimal):
iptables -D POSTROUTING -t mangle -o $wan -s 192.168.1.7/32 -j MARK --set-mark ${Streaming_mark_up} &> /dev/null
iptables -A POSTROUTING -t mangle -o $wan -s 192.168.1.7/32 -j MARK --set-mark ${Streaming_mark_up}

I have this for the downloads for unraid for the sabnzbd 'downloads' :
iptables -D POSTROUTING -t mangle -o br0 -d 192.168.1.7/32 -j MARK --set-mark ${Downloads_mark_down} &> /dev/null
iptables -A POSTROUTING -t mangle -o br0 -d 192.168.1.7/32 -j MARK --set-mark ${Downloads_mark_down}

When sab is doing a download however, I do see large packets in downloads (which is correct), but also an increase in the packets for streaming uploads - I dont know why there is an increase in uploads at the same time - and these will affect the QOS for the above video streaming ?

Am I doing this correctly ? Is there another way ?

Thanks.

 

[FreshJR]

I can specify a dedicated port or dedicated ip address to direct sabnzbd downloads to the downloads mark as it is all destination port 443 and destination IP 192.168.1.7.
Am I correct in this ?

....

I do see large packets in downloads (which is correct), but also an increase in the packets for streaming uploads - I dont know why there is an increase in uploads at the same time - and these will affect the QOS for the above video streaming ?

Am I doing this correctly ? Is there another way ?

Thanks.

Externally, WAN SIDE, all your hosted servers are accessibly and served from 443.
Internally all those server you have hosted on a PC are not hosted on 443, but rather get forwarded to/from it.

You can use ports rules defining the local ports used by those servers.

--

For your second question, TCP data transfer is not one-way communication.

For a server to upload files, it has to listen for "package recieved messages" from the client doing the downloading.
The same is true in reverse.

What seems like 1-way data transfer is actually an ongoing 2-way communication. Listening or sending for "awk" messages shouldn't be too much data compared to the file transfer, but it will still be there. That is the noise you are seeing when you are looking the upload/download statistics. If you want better accuracy you have to use ports.

 

[vw-kombi]

I cant get it to work using any of my internet ports configured for the end services - traffic always goes to default if I remove the IP address and try to add destination ports in downloads and source ports in uploads.
Based on your answer to the second question however, the traffic in the upload streaming from sab downloads is just light ack packets, and hence a non issue so I will leave it at that.
Thanks.

 

[FreshJR]

Make sure you are using correct src or dst labeling. This was explained in great deal in the port rules section.

Go check the first posts.

 

[FreshJR]

@vw-kombi, I had an email where you listed you netstat-nat results but that post is missing.

On the netstat-nat results, the local ports were listed on the left side.

In those results the servers were locally hosted in the dynamic ephemeral range (49152 - 65535) and they were externally listening on 443.
You should be able to configure them to a static port outside the ephemeral range, and use port rules on the configured local port.

Just wanted to let you know it is possible. You can then create both upload/download rules on local ports to even get those AWK's sorted as you wish.

 

[xman0980]

How would I go about changing Plex traffic? It gets put under default, but besides gaming, its the second most used thing in my house. Should I just change the default in the master script?

 

[Sinner]

Use the fresh jr qos. It will move all the default traffic to other which you can move up to where youd like. Then if needed its possible to customize the script even further but may not be ness for you.

 

[xman0980]

Use the fresh jr qos. It will move all the default traffic to other which you can move up to where youd like. Then if needed its possible to customize the script even further but may not be ness for you.

I am. I have it installed. Its showing the default on the chart, and when plex is running, it just goes sky high. When I have FreshJr disabled, it just put it under Other.

 

[FreshJR]

I am. I have it installed. Its showing the default on the chart, and when plex is running, it just goes sky high. When I have FreshJr disabled, it just put it under Other.

You sure you don't have the categories mixed up? The reverse should be happening.

The plex server can have a custom rule created for it by either its IP or PORT if you need it in a different category.

 

[Sptz]

Hi all,

I'm very inexperienced with QoS and it's inner works. I'm on Virgin in the UK 100/6 (Hub3 modem mode) with an AC68U and the latest merlin firmware. I turned on QOS, adaptive, fq_codel. My main objective is for games (from steam like rocket league etc) to be prioritised, then streaming then anything else.

Unfortunately, I'm using Wi-Fi at the moment, router is next room, full bars, 5Ghz, 2.4Ghz is turned off. In Rocket League I get a lot of "latency" warnings, with rubber banding etc. DSLreports speedtest always shows A or A+ for everything including buffer bloat (with QOS on) and speeds are mostly steady (92-99/4-5). So is the issue I'm getting due to using Wi-Fi or could it be the router?
Would this script help in any shape or form or any other settings?