[Release] FreshJR Adaptive QOS (Improvements / Custom Rules / and Inner workings)

[FreshJR]

Try it and see

 

[Sptz]

Try it and see

Should I install the default script and then can I customise the priority order in the GUI or leave it at default upon turning on?

 

[Sinner]

No you install the acript but still need to go in and move others gaming download etc to your needed priorities. In almost all scenarios video n download shud be at the bottom cuz they will hog all of your bandwidth if they can

 

[vw-kombi]

@FreshJR - thanks for the reply, I created a post, then deleted it while I did more investigation on my own. Spent a while on it and I have some enhancements over just the IP addresses as you say. The NAT stuff in the router was a great help (I read that so long ago I forgot about it). I figured out my sabnzbd stuff is using my vpn via Canada, and hence I have a port I could use for its news connection - so that is all travelling up and down well. I then figured out that as my random torrents is also using a socks5 proxy (to Holland) and just use that port for those - so the main traffic is now all in the downloads area and perfect.

My roku's in the house I have hardcoded with a /30 so when they do any download streaming of TV shows from the UK, they go to streaming - great.

My xboxes gaming traffic are sorted in the same way - with a /30 entry.

The only other traffic I wish to put into streaming upload is a tricky one however. This is for remote family members that watch media from my system. I am struggling with this one as when I get one of them to watch it, I only have 443 ports shown in the router nat table. This is configfured as such :

Emby is hosted on port 8096 on unraid at 192.168.1.7.
Remote users go to https://emby.mydomain.com, so come in on 443
The router port forwards 443 to 4443 on the NGINX docker on unraid, (which unraid remaps to 443 on the docker)
Nginx then pushes the emby.mydomain.com to unraid at 192.168.1.7:8096.
At no point when someone is watching something can I see anything NAT except 443 on the router.
I tried putting source port 8096 in the fresh script for upload, but it went to other.....

For now, I have a catchall upload streaming for 192.168.1.7 but it would be nice to manage this by port instead.

 

[Sinner]

I would keep an eye on your traffic and deal with the badwidth hogs or problems 1st. Work it from that direction and youll be happy quickly. Things like dowloads and netflix will use everything causing huge bufferbloats and ping times. Thing like discord vent teamspeak and online games need to be top priority cuz theyre ping sensitive and cant be delayed orbyir dead etc

 

[Sptz]

Wow. That's all I have to say, this script literally saved my multiplayer gaming performance even with just my laptop in the network I was getting both warnings for:

• 
  

  [*]Latency Variance: When your ping isn’t holding steady, typically fluctuating 40ms-60ms
  [*]High Latency: Commonly known as “lag,” usually indicated by a high ping to the server

And it was driving me crazy, after installing this script, checking it was definitely applied in about 5 games I played I got that warning for 2 seconds and that was it, never again. DSLReports test also shows how the picture has changed.

Latest Merlin on AC68U:

No QoS at all:

• Idle - Sample: 29ms / Minimum: 22ms
• Downloading - Sample: 61ms / Average: 53ms
• Uploading - Sample: 32ms / Average: 27ms

QoS On (NO Script, Adaptive, fq_codel)

• Idle - Sample: 23ms / Minimum: 18ms
• Downloading - Sample: 38ms / Average: 29ms
• Uploading - Sample: 58ms / Average: 25ms

QoS On (Freshjr Script, Adaptive, fq_codel)

• Idle - Sample: 25ms / Minimum: 19ms
• Downloading - Sample: 33ms / Average: 28ms
• Uploading - Sample: 29ms / Average: 23ms

 

[vw-kombi]

Are these graphs and stats from the merlin firmware somewhere ? I have not seen those ?

 

[Sptz]

Are these graphs and stats from the merlin firmware somewhere ? I have not seen those ?

http://www.dslreports.com/speedtest

 

[Spydawg]

Hey FreshJR, Been using your script now since you released it way back when, and never had any issues.. But now after increasing my connection speed I'm having problems.
I was at 300 down 300 up and now upgraded to 1000 down 1000 up also my ISP came and changed my modem, Its all setup likw it was before, the ISP modem is in bridge mode.
My ASUS 3100 had nothing changed on it excpt the QOS upload and download bandwidth from 250 down 250 up to 900 down 900 up.

Now when i speed test with DSLreports without QOS enabled i get great speeds 1100 down 950 up. but buffer bloat on upload is bad 250 ms +
With your QOS Script on the same test with DSLreports my speeds drop down to only 200 down 350 up. and buffer bloat is still 250 ms + on upload.

I checked all settings tried all WAN packet overheads tried with ATM on and off with eachWAN packet overhead options. same slow speeds..
I even removed your script and tried with the ASUS defalt QOS same thing happens.
I reistalled your QOS script to be sure and re tested, same slow speeds while enabled.

I had AMTM installed with Diversion 4.0 and pixelserv-tls 2.2.0-rc.6 and dnscrypt,
I removed them all and just tried your script and the default ASUS QOS to test and I get the same slow speeds with then enabled.

Any idea?

 

[D_Day]

Hey FreshJR, Been using your script now since you released it way back when, and never had any issues.. But now after increasing my connection speed I'm having problems.
I was at 300 down 300 up and now upgraded to 1000 down 1000 up also my ISP came and changed my modem, Its all setup likw it was before, the ISP modem is in bridge mode.
My ASUS 3100 had nothing changed on it excpt the QOS upload and download bandwidth from 250 down 250 up to 900 down 900 up.

Now when i speed test with DSLreports without QOS enabled i get great speeds 1100 down 950 up. but buffer bloat on upload is bad 250 ms +
With your QOS Script on the same test with DSLreports my speeds drop down to only 200 down 350 up. and buffer bloat is still 250 ms + on upload.

I checked all settings tried all WAN packet overheads tried with ATM on and off with eachWAN packet overhead options. same slow speeds..
I even removed your script and tried with the ASUS defalt QOS same thing happens.
I reistalled your QOS script to be sure and re tested, same slow speeds while enabled.

I had AMTM installed with Diversion 4.0 and pixelserv-tls 2.2.0-rc.6 and dnscrypt,
I removed them all and just tried your script and the default ASUS QOS to test and I get the same slow speeds with then enabled.

Any idea?

I don’t know that much about QOS but with your speeds you require NAT acceleration on and it comes into direct conflict with QOS.
It’s just a thought and I could be totally wrong.

 

[Spydawg]

I don’t know that much about QOS but with your speeds you require NAT acceleration on and it comes into direct conflict with QOS.
It’s just a thought and I could be totally wrong.

Thanks for the info, I'm thinking you are right.

 

[D_Day]

Thanks for the info, I'm thinking you are right.

I tend to use this guide to help me. Hope you can find some use for it.
https://routerguide.net/nat-acceleration-on-or-off/

 

[Spydawg]

I tend to use this guide to help me. Hope you can find some use for it.
https://routerguide.net/nat-acceleration-on-or-off/

nice link, thanks

 

[FreshJR]

@Spydawg

While adaptiveQOS does NOT turn off NAT acceleration (traditional QOS does), filtering does still take cpu power.

If your cpu hits 100% it will lagg and produce bufferbloat. Your internet could simply be too fast for cpu to handle.

At those speeds you don’t really need adaptiveQOS doing deep packet inspection and using chunks of cpu power.

A simpler lightweight QOS system would produce better results.

Eg. use htb to guarantee each client a ~50mbps fq-codel queue and use that htb tree to do 80-95% bandwidth limiting. That would cut cpu use, and with these speeds, traffic prioritization is irrelevant.

 

[Sinner]

I was thinking last night of a useful feature which I guess is more of a @RMerlin thing but going to post it here cuz as far as im concerned this is the official qos thread.

How hard would it be to modify the qos apps analysis details to show not only the type of data like "Netflix" httpd" etc to also show the container its being routed to like "games" "video" "downloads" etc? this might not only be very useful for those running the script but equally for those not running it and using stock qos.

 

[Spydawg]

@Spydawg

While adaptiveQOS does NOT turn off NAT acceleration (traditional QOS does), filtering does still take cpu power.

If your cpu hits 100% it will lagg and produce bufferbloat. Your internet could simply be too fast for cpu to handle.

Oh ok, I did see my CPU Max out while testing with QoS enabled..

Time for a new Router lol

 

[FreshJR]

@Spydawg

It should be simple to implement a lightweight QOS structure for gigabit connections.

I just need to ask around how to prep the interfaces for TC into eth0 / br0 which allows us to separate the download / upload streams for shapping without disabling NAT.

If I get an answer, I’ll try creating a solution for you.

If not, microtik does offer that simple lightweight QOS structure I mentioned that works well with gigabit connection, but the setup is not all that intuitive (maybe they have a youtube video).

@Sinner that should be simple to do if that portion had its source code available. So just rely on appdb.

 

[Spydawg]

@Spydawg

It should be simple to implement a lightweight QOS structure for gigabit connections.

I just need to ask around how to prep the interfaces for TC into eth0 / br0 which allows us to separate the download / upload streams for shapping without disabling NAT.

If I get an answer, I’ll try creating a solution for you.

If not, microtik does offer that simple lightweight QOS structure I mentioned that works well with gigabit connection, but the setup is not all that intuitive (maybe they have a youtube video).

@Sinner that should be simple to do if that portion had its source code available. So just rely on appdb.

Oh this a microtik script that I can install on my 3100? or you talking about microtik hardware?

 

[FreshJR]

Oh this a microtik script that I can install on my 3100? or you talking about microtik hardware?

microtik hardware comes with QOS capabilities. They have a very lightweight QOS implementation available that only does bandwidth limiting per connected device. It does not do deep packet analysis / inspection for traffic type priority.

At gigabit speeds simply ONLY dividing available bandwidth per connected device WITHOUT give priority to certain traffic types is a sufficient solution since speeds allotted per device will be very high.

While this type of QOS system is crude and simple, it will severely decrease CPU load and allows it to handle gigabit connection without choking.

The functional point is that solution will be capping speeds below ISP allowed speeds to eliminating bufferbloat.

--

Implementing a similar lightweight QOS setup on our ASUS routers, intended to limit CPU load should also be possible with a script that has not yet been created.

I can give it a shot when I get some time. Assuming I do not have to prep eth0 / br0 interfaces for upload/download segregation to use within TC manually, the creation should be simple.

If I do have to configure/prepare the interfaces before creating the TC qos system, then I am out of luck since I do not know which commands to issue to do so.

--

Side note: The QOS landscape is heating up between router manufacturers as a big marketing point recently. Competition is good as now users will have solutions available out of the box as it should of been from day1.

 

[Spydawg]

ok good to know thanks