[Release] FreshJR Adaptive QOS (Improvements / Custom Rules / and Inner workings)

[pattiri]

sounds like steam is not being detected as downloads heres the ports

Required Ports for Steam
Which ports do I need to open on my router or firewall for Steam?

To log into Steam and download content:

• HTTP (TCP port 80) and HTTPS (443)
• UDP 27015 through 27030
• TCP 27015 through 27030

Steam Client

• UDP 27000 to 27015 inclusive (Game client traffic)
• UDP 27015 to 27030 inclusive (Typically Matchmaking and HLTV)
• UDP 27031 and 27036 (incoming, for In-Home Streaming)
• TCP 27036 and 27037 (incoming, for In-Home Streaming)
• UDP 4380

Dedicated or Listen Servers

• TCP 27015 (SRCDS Rcon port)

Steamworks P2P Networking and Steam Voice Chat

• UDP 3478 (Outbound)
• UDP 4379 (Outbound)
• UDP 4380 (Outbound)

https://support.steampowered.com/kb_article.php?ref=8571-GLVN-8711

but when I'm downloading a game I can see it on QoS Statistics page. as here;

https://www.snbforums.com/attachments/screenshot_1-jpg.10175/

 

[Vexira]

Thank you. let me check this

your welcome there may or may not be a need for a custom rule in the script to adjust for steam downloads based on the port numbers, check the first post to see how to create a custom rule, I've bee looking for the ports or port, that Xbox one and ps4 use for downloads and make them into custom rules if its not classified correctly.

From what I can see is that steam is being classified as gaming, including the port number used for downloads, seems that ill have to fiddle with custom rules so that the port for steam download goes to download traffic.

#${tc} filter add dev br0 protocol all prio 1 u32 match ip dport 1234 0xffff flowid ${Downloads}
#${tc} filter add dev eth0 protocol all prio 1 u32 match ip sport 1234 0xffff flowid ${Downloads}

I think these template rules just need to be modified for steam download ports.
58000-60999
this might be the right port range
http://steamcommunity.com/discussions/forum/1/540744937317858314/

 

[Vexira]

sadly I'm not sure how to specify tcp

 

[beboptrumpet]

Just wanted to suggest turning qos off and back on after a firmware update to anyone who has issues blufferboat issues after updating

I just updated to 368.68 from the alpha. I power cycled afterward, but was till getting blufferboat issues. I checked the syslof and the qos stats page to make sure this was still working--it was. Finally, I turned qos off & on again--just like you do for the last step in installing the script. Now I'm back to great results

 

[Vexira]

but when I'm downloading a game I can see it on QoS Statistics page. as here;

https://www.snbforums.com/attachments/screenshot_1-jpg.10175/

Did you want me to try mod ding the script to get it working, see if it's fine on my steam, I can try to add a rule but I'm not sure to specify protocols like UDP or tcp.

 

[FreshJR]

@pattiri I did not upgrade yet, but I will do so soon.

Your QOS statistics page looks correct, which is terrible news since steam SHOULD be getting bandwidth last! I hope it's something simple!

You can try this command in ssh

tc class show dev br0 | grep "parent 1:1"

To make sure minimums are still defined after the upgrade.

@Vexira by default the rule will account for both TCP & UDP traffic. Do not make a custom rule for port 80 or 443, or anything under 1024, ever for that matter.
80 is regular web browsing
443 is https web browing

You will wrongly sort regular traffic.

 

[pattiri]

@pattiri I did not upgrade yet, but I will do so soon.

Your QOS statistics page looks correct, which is terrible news since steam SHOULD be getting bandwidth last! I hope it's something simple!

You can try this command in ssh

tc class show dev br0 | grep "parent 1:1"

To make sure minimums are still defined after the upgrade.

@Vexira by default the rule will account for both TCP & UDP traffic. Do not make a custom rule for port 80 or 443, or anything under 1024, ever for that matter.
80 is regular web browsing
443 is https web browing

You will wrongly sort regular traffic.

All seems fine I think.

admin@Fatiii:/tmp/home/root# tc class show dev br0 | grep "parent 1:1"
class htb 1:11 parent 1:1 leaf 11: prio 1 rate 25600Kbit ceil 51200Kbit burst 1596b cburst 1593b
class htb 1:10 parent 1:1 leaf 10: prio 0 rate 2560Kbit ceil 51200Kbit burst 1600b cburst 1593b
class htb 1:13 parent 1:1 leaf 13: prio 3 rate 5120Kbit ceil 51200Kbit burst 1599b cburst 1593b
class htb 1:12 parent 1:1 leaf 12: prio 2 rate 15360Kbit ceil 51200Kbit burst 1597b cburst 1593b
class htb 1:15 parent 1:1 leaf 15: prio 5 rate 2560Kbit ceil 51200Kbit burst 1600b cburst 1593b
class htb 1:14 parent 1:1 leaf 14: prio 4 rate 5120Kbit ceil 51200Kbit burst 1599b cburst 1593b
class htb 1:17 parent 1:1 leaf 17: prio 7 rate 2560Kbit ceil 51200Kbit burst 1600b cburst 1593b
class htb 1:16 parent 1:1 leaf 16: prio 6 rate 2560Kbit ceil 51200Kbit burst 1600b cburst 1593b

 

[Arsenal]

If you are sure that's the only mark they are using then these two rules should work

${tc} filter add dev br0 protocol all prio 1 u32 match u8 0x38 0xFC at 8 flowid ${VOIP}
${tc} filter add dev eth0 protocol all prio 1 u32 match u8 0x38 0xFC at 8 flowid ${VOIP}

Thanks again for this, as well as the detailed explanation as to why its worth checking.. it does seem odd that Discord would be working with a deprecated ruleset.

I've just had a chance to try installing your script and give this a go, however i seem to be doing something wrong as whenever i enable QoS with the script installed my ceiling seems to be limited to approximately 100KB/s for all types of traffic, even though i have it set to 13Mbit in the interface and running "tc class show dev br0" shows ceil 13312Kbit on everything.

If i remove the script and just run the standard Adaptive QoS, things return to normal with 10+Mbit throughput. (For the record my DSL connection is 13.8Mbit/1Mbit).

Is there something obvious i'm doing wrong with the install?

 

[Vexira]

@pattiri I did not upgrade yet, but I will do so soon.

Your QOS statistics page looks correct, which is terrible news since steam SHOULD be getting bandwidth last! I hope it's something simple!

You can try this command in ssh

tc class show dev br0 | grep "parent 1:1"

To make sure minimums are still defined after the upgrade.

@Vexira by default the rule will account for both TCP & UDP traffic. Do not make a custom rule for port 80 or 443, or anything under 1024, ever for that matter.
80 is regular web browsing
443 is https web browing

You will wrongly sort regular traffic.

I'm aware of ports below 1024 is privileged, I miss read the port range lol, so tired I was last night.

 

[Vexira]

Will this script suffer if firewall-start is called mutliple times during a router reboot, or does it already include provisions to prevent overlapping runs?

script works fine with multiple reboots, alt least in my case it does

 

[Vexira]

I think that's it for ports, I is it ok in the script ex for downloads if I did a port 27000-27020 (downloads) for example?, also I'm not sure what to classify match making traffic as either gaming or web. hmm seems that steam only uses TCP for downloads need a way to specify only TCP for the download ports to avoid a conflict with the UDP ports.
http://assemblyrequired.crashworks.org/wp-content/uploads/2008/12/tomatoqos.png
http://assemblyrequired.crashworks.org/router-qos-steam-games/

 

[pattiri]

I think that's it for ports, I is it ok in the script ex for downloads if I did a port 27000-27020 (downloads) for example?, also I'm not sure what to classify match making traffic as either gaming or web. hmm seems that steam only uses TCP for downloads need a way to specify only TCP for the download ports to avoid a conflict with the UDP ports.
http://assemblyrequired.crashworks.org/wp-content/uploads/2008/12/tomatoqos.png
http://assemblyrequired.crashworks.org/router-qos-steam-games/

Let me show you another interesting thing. here is two screenshot form wireshark while I was downloading a game. (172.24.5.2 is the LAN IP address of my pc and 146.66.155.6 possibly the IP address of the steam server that I download)

As you can see the my pc sending request with destination port 80 and server replies it with the source port as 80 as expected. It looks exactly same with a web download

 

[Vexira]

Let me show you another interesting thing. here is two screenshot form wireshark while I was downloading a game. (172.24.5.2 is the LAN IP address of my pc and 146.66.155.6 possibly the IP address of the steam server that I download)

View attachment 10191 View attachment 10192

As you can see the my pc sending request with destination port 80 and server replies it with the source port as 80 as expected. It looks exactly same with a web download

No not exactly, it's using a translation port 57144 but using the port to translate to port 80. Id say port 80 is internal port and 57144 is external that's the port I'll set as downloads for steam 57144.

 

[ColinTaylor]

No not exactly, it's using a translation port 57144 but using the port to translate to port 80. Id say port 80 is internal port and 57144 is external that's the port I'll set as downloads for steam 57144.

No I don't think so. All the Steam downloads I've done have used port 80 on the remote end and a dynamic (ephemeral) high-port on the client end. Exactly the same as a normal HTTP web session, HTTP download or a streaming service like Netflix. There is no "port translation" happening. So I believe @pattiri is correct.

 

[Vexira]

in

No I don't think so. All the Steam downloads I've done have used port 80 on the remote end and a dynamic (ephemeral) high-port on the client end. Exactly the same as a normal HTTP web session, HTTP download or a streaming service like Netflix. There is no "port translation" happening. So I believe @pattiri is correct.

that case take another glance at his sreen shot, and please clarfy the port 57144

 

[ColinTaylor]

in that case take another glance at his sreen shot, and please clarfy the port 57144

As I said, that is an ephemeral port. This is the way HTTP works.

Example: You want to visit http://google.com.
You type the URL into your web browser.
Your computer opens a random high-port (ephemeral) on your computer, in this case port 57144.
This port connects Google's web server which is listening on port 80.
You send a query from port 57144 on your computer to port 80 on Google's server.
Google's server responds by sending a reply back through the same connection, i.e. from Google's port 80 to your port 57144.

 

[pattiri]

Here is a new capture from download which confirms @ColinTaylor
This time port 63335 is used. It's chosen randomly

 

[maxsteel]

Thanks @FreshJR for the great script and your helpful support. I would like to extend my thanks to @Vexira and @Jack Yaz for their amazing work with me to help get the script working.

I have a question regarding Mssfix on Astrill Applet client, should I enable it as (1400 the default)" with the script and QoS working, or should I disable it with value (1500)?

 

[Vexira]

As I said, that is an ephemeral port. This is the way HTTP works.

Example: You want to visit http://google.com.
You type the URL into your web browser.
Your computer opens a random high-port (ephemeral) on your computer, in this case port 57144.
This port connects Google's web server which is listening on port 80.
You send a query from port 57144 on your computer to port 80 on Google's server.
Google's server responds by sending a reply back through the same connection, i.e. from Google's port 80 to your port 57144.

I see so that's how multiple devices can use port 80, hmmm interesting. Thanks for that.

 

[Vexira]

Here is a new capture from download which confirms @ColinTaylor
This time port 63335 is used. It's chosen randomly
View attachment 10204

Steams guide is out dated they should use a port other than 80 for downloading data, either that or asus need to update qos database.