[Release] FreshJR Adaptive QOS (Improvements / Custom Rules / and Inner workings)

[MarioCaires]

Agree mate, uTorrent is the worst in that aspect! Have tried others (qbittorrent and deluge) however the seeding performance is awful compared with uTorrent and that's way I kept it!

Regarding the issue, were you with encryption disabled btw? I have disabled and now 99% of the traffic is being correctly categorized as "Downloads". I can see there's still some traffic going to "Others" at around 2 and 5Kbps, but much better now!

 

[jpclarke]

I have implemented the script and is running perfectly. I decided to feed the upload and download values from a scheduled Speedtest running on a raspberry pi device and this seems to be working as expected.

The speedtest basically generates 2 text files which a script extracts and stores in nvram, I then restart the qos service then run this script to have automatic speed adjustment without using the auto feature (10g only links)

 

[FreshJR]

I have implemented the script and is running perfectly. I decided to feed the upload and download values from a scheduled Speedtest running on a raspberry pi device and this seems to be working as expected.

The speedtest basically generates 2 text files which a script extracts and stores in nvram, I then restart the qos service then run this script to have automatic speed adjustment without using the auto feature (10g only links)

Make sure you input a percentage of your max upload/download into QOS for proper QOS functionality.

If you input your MAX limits into QOS, you may still get buffer bloat. Look at the explanation a few pages back. I will be adding this reminder into the installation.

Note: Still not sure if the new WAN overhead is double accounting for packet size or not. Maybe now with WAN overhead as a configurable option, we may not have to reduce speedtest bandwidth results while imputing QOS limits. I did not perform enough testing to see if this is true or not.

I am partially confused since Speedtest will show my max advertised ISP limits. I do not know if the speedtest gauge includes packet overhead or just packet payload as part of the speed reading. If it is just packet payload, then it seems like my ISP does not throttle on WAN overhead but rather on payload.

Yet when I do a download, instead of a speedtest, I seem to hit 3.5 MB/S = 28,000 kbps. My speed is advertised as 30 mbps. When doing QOS without setting up packet overhead, I had to set my QOS rates at 28,000 instead of 30,000 so my router would be the one dropping/queuing packets instead of the ISP.

 

[jpclarke]

Yeah. I have the basics setup as it was getting late but will probably refine the figures to 95% of the maximum from the test.

I may also apply some logic to avoid low results being returned if the network is busy when the test is being run, this could be if the new figure is 20% lower than the last ignore or rerun in 10 mins and recheck/ignore.

 

[FreshJR]

Yeah. I have the basics setup as it was getting late but will probably refine the figures to 95% of the maximum from the test.

I may also apply some logic to avoid low results being returned if the network is busy when the test is being run, this could be if the new figure is 20% lower than the last ignore or rerun in 10 mins and recheck/ignore.

Also, do you have the rPI exempt from QOS throttling? If not, you would get lower and lower results each time.

 

[jpclarke]

Great point which I was going to raise. What's the best way to ensure the speedtest gets the full amount of bandwidth available?

I assume I can add a line to the script?

 

[FreshJR]

Great point which I was going to raise. What's the best way to ensure the speedtest gets the full amount of bandwidth available?

I assume I can add a line to the script?

You would have to look at the iptable command in the script. The mark would have to be something that doesn't match any rules. The template includes an example for upload traffic mark. You would need one for download traffic aswell.

Mark does not match any traffic rules = bypass qos.

As you can see, mark can be assigned to all packets from a specific device.

 

[jpclarke]

Hi,

Would the allocation to the highest priority for that specific traffic ensure the speedtest can consume 100% of the bandwidth even with other traffic flowing?

 

[FreshJR]

Hi,

Would the allocation to the highest priority for that specific traffic ensure the speedtest can consume 100% of the bandwidth even with other traffic flowing?

It would get highest priority within that container, but that QOS container has a shared amount of bandwidth in relation to the rest, so no.

Even if it didn't, you would still be subject to the bandwidth defined in your QOS rules, so your results would get smaller everytime. I suggest u do not set QOS at 100% of your max throughput, to squeeze some extra network performance, as then buffer bloat will not be handled properly (main reason for QOS).

When you get bufferbloat it's packets being delayed or dropped. You want your own router to choose which packets are delayed or dropped. If you are at max speeds, then it's your ISP dropping/delaying packets.

Two ways to do what you want. You can exclude that device from QOS throttling as I told you before. Even then it will have to share bandwidth with other devices but it won't be bandwidth capped.

Ultimately, the foolproof method would be for you to have a traffic container above netcontrol that would have a 1 gbps traffic cieling and rate. That way it would always get 100% bandwidth. I have not investigated changing class structure and if QOS would still work properly with auto generated sub class rules. If it did break, you can fix it by the further hacky method of intercepting all TC calls and adding 1 to each class destination for sub class rules.

As a workaround to changing class structure, to make it less hacky if sub classes didn't work properly without furthur modifications, you can try setting net control to an 1gbps rate and ceiling, (instead of a new class) and sitting the rPI in there.

So two options
- rPI gets a mark to not match any TC rules. This will not traffic shape it's traffic
-rPI goes into top container which would have to set setup to consume all bandwidth available, exceeding user defined limits. ( pretty hacky work around, hopefully other devices do not generate much net control traffic, as that would bring the rest of ur network to a stop)

 

[brummygit]

So I must admit to reading your post and taking a look at the script, but not really understanding what to do.

I have a Sam Knows white box on my LAN which is meant to monitor my internet connection and report data back for Ofcom. It would be great to get an idiots guide on how to get this device to bypass QoS so that it reports my true connection details. It runs as a transparent bridge and so is able to see my network traffic in order to avoid conflicting with it.

 

[MagikMark]

Hi!

Not sure if this has been answered.

Is there a way to throttle " Guest Network"? As in all devices connected to this network is automatically set to a particular bandwidth?

Right now I have to individually set the speed of each device connected. It would be nice if this is done on the fly

Thanks


Sent from my iPhone using Tapatalk

 

[FreshJR]

Hi!

Not sure if this has been answered.

Is there a way to throttle " Guest Network"? As in all devices connected to this network is automatically set to a particular bandwidth?

Right now I have to individually set the speed of each device connected. It would be nice if this is done on the fly

Thanks


Sent from my iPhone using Tapatalk

Did not look how guest network interacts with adaptive QOS.
Where are you setting individual speeds.

This could be something I'll look into after I finish more pressing issues.

 

[MagikMark]

It's in:

Adaptive QOS -> QOS -> Bandwith Limiter

I just look for the devices that are connected then set the upload & download speed limit


Sent from my iPhone using Tapatalk

 

[FreshJR]

It's in:

Adaptive QOS -> QOS -> Bandwith Limiter

I just look for the devices that are connected then set the upload & download speed limit


Sent from my iPhone using Tapatalk

Going forward I would like to inplement something like that. Easiest way would be to send all guest traffic (except net control) into the last container (default) and have that be container be bandwidth limited.

I still have to see how asus packet inspection interacts with guest traffic. If guest traffic is marked and flows through eth0 and br0 interfaces it's a walk in the park to implement.

Is it possible to do bandwidth limited and adaptive QoS at the same time ?

If you want to do it immediately you can use the bandwidth limiter I implemented for Medwatt in this thread.

It's the same exact commands that I would be using, you just need to change the ip ranges of the rule to match the guests dhcp assignment range. This will only work if the above condition is true about guest traffic and adaptive qos, I have not done any testing in this reguard.

Note, that crude version of bandwidth limiter also sends net control packets into the last container, and it really shouldn't. Feel free to fix that. Should be additional 2-3 lines of code to catch that exemption.

I will be upgrading the crude version properly later, but that is not a priority yet.

 

[MagikMark]

[emoji106]


Sent from my iPhone using Tapatalk

 

[lilstone87]

Well when it comes to running scripts, and such, I consider myself a rookie, as i don't use them. With that said, i plan to run this script, but i will fully read the OP of this thread before doing so.

I want to ask though, as i have between 5-7 people using multiple devices at times. I game online at times, so i want it to have highest priority. However i notice with Asus current Adaptive QoS. Even when consoles are doing game download's, and patch updates. That traffic is still classified as gaming, which it is in a sense. But in the end, it's bulk traffic. I noticed with PS4 at least, a lot of actual gaming packets are routed via port 3074.

So I'm wondering is there a way to set that port as a highest priority via the script? If so how i go about doing this?

One other question, i have noticed with Adaptive QoS enabled, fd-codel selected, and bandwidth limits manually inputted. When running certain test, I'm able to bypass the limits i set. So is there a way to make sure the limits i set, are actually honored via this script?

Again i appreciate any, and all help. I'm just trying to maximize my home setup to my liking, but also willing to admit I'm a rookie when it comes to scripts in general.

 

[lilstone87]

Well I did step by step, and installed the script. Which I appreciate everyone that has helped with this, and making QoS run better on these Asus routers. However I'm still curious on having things maximized for gaming, which I know FreshJR mentioned his next script version being focused on that. So I look forward to that, and maybe we can also fit into the script a way to keep the larger bulk gaming traffic to a lower priority, then gaming packet's themselves.

Because like I mentioned in my above post, currently with the way adaptive qos is setup. At least for gaming consoles, when doing game download's, and patch updates. They're getting the same priority as an actual gaming packet, which are using very little data in comparison. I mentioned giving certain port's higher priority, as I do know actual game traffic is normally passed over a certain port. Not 100% on xbox, but for PS4, I have often seen port 3074 used to pass gaming packets.

Again I'm pretty sure with the current script FreshJR has posted here for us, it helps with this compared to the general Adaptive QoS, minus this script. Just be nice to maximize things for gaming, which like myself has multiple gaming console's that run online at the same time, at times. Reason I mentioned trying to raise actual gaming packet's priority, over games/patches being download. As it's not the smartest to have "Console(A) downloading a game, plus multiple patches, having same priority as Console(B) online only playing a multiplayer game, without any bulk download's running".

I know it's possible just need to likely prioritize certain ports. Which honestly, like I said before... I'm a rookie to scripts in general, so I don't feel comfortable enough to go modifying FreshJR's script. I will wait to see what he comes up with, when he has a bit of time to do so. I want to make it clear I appreciate the one's who have been helping out with this. I love seeing effort put into trying to improve QoS to run as best as possible, for each one of us.

 

[FreshJR]

Well I did step by step, and installed the script. Which I appreciate everyone that has helped with this, and making QoS run better on these Asus routers. However I'm still curious on having things maximized for gaming, which I know FreshJR mentioned his next script version being focused on that. So I look forward to that, and maybe we can also fit into the script a way to keep the larger bulk gaming traffic to a lower priority, then gaming packet's themselves.

Because like I mentioned in my above post, currently with the way adaptive qos is setup. At least for gaming consoles, when doing game download's, and patch updates. They're getting the same priority as an actual gaming packet, which are using very little data in comparison. I mentioned giving certain port's higher priority, as I do know actual game traffic is normally passed over a certain port. Not 100% on xbox, but for PS4, I have often seen port 3074 used to pass gaming packets.

Again I'm pretty sure with the current script FreshJR has posted here for us, it helps with this compared to the general Adaptive QoS, minus this script. Just be nice to maximize things for gaming, which like myself has multiple gaming console's that run online at the same time, at times. Reason I mentioned trying to raise actual gaming packet's priority, over games/patches being download. As it's not the smartest to have "Console(A) downloading a game, plus multiple patches, having same priority as Console(B) online only playing a multiplayer game, without any bulk download's running".

I know it's possible just need to likely prioritize certain ports. Which honestly, like I said before... I'm a rookie to scripts in general, so I don't feel comfortable enough to go modifying FreshJR's script. I will wait to see what he comes up with, when he has a bit of time to do so. I want to make it clear I appreciate the one's who have been helping out with this. I love seeing effort put into trying to improve QoS to run as best as possible, for each one of us.

To prioritize ports it's just removing two hashtags in the script, changing the port on those lines, and picking your QOS container destination. Read the comments on every line and it will be obvious which ones it is.

Pretty simple.

Also see if you can find if the PS4 uses a different port for its download traffic compared to gaming. I would rather send that port data to downloads.

About your traffic that bypasses qos. I recently saw that happening too when another user pointed it out. It was https download traffic. That's a bug from asus that I can fix next release. To get more information, what traffic bypassed QOS in your use case.

 

[lilstone87]

About your traffic that bypasses qos. I recently saw that happening too when another user pointed it out. It was https download traffic. That's a bug from asus that I can fix next release. To get more information, what traffic bypassed QOS in your use case.

Well I think I noticed it twice... But I can only remember one off the top of my head right now, and that was while running dslreports.com speedtest. Which I know has been used on here at times for people trying to judge there buffer bloat. So I'm guessing that test is doing what you mentioned, still it would be nice to have this fixed, to keep bandwidth limit's respected, and have QoS working as intended.

Also see if you can find if the PS4 uses a different port for its download traffic compared to gaming. I would rather send that port data to downloads.

As for this, I dunno if I can do this. Wish I had my older router running tomato... As I could likely run QoS on it, and see what port/port's the ps4 is using when downloading data. With these Asus routers, I have no way to track usage across ports. So I wonder if anyone else on the forums here has a PS4, and has a router/device they can track data flow across active ports. While the PS4 downloads something, because then we can get an ideal if they're using a port, or two to push the data threw, or if it is a lot of different ports. Trust me I wish I could help with this, as it's important to me. I just have no current way to track data flow across active ports.

 

[FreshJR]

My dslreports speedtest is identified as a file transfer. Are you on the latest definitions? I was using chrome.

I only know of one on type of traffic that has bypassed QOS. I can't look into things without more information.

Note: I consitantly get an A for bufferbloat, but a C or a B for quality as my upload speed loves to decide not to be stable.