[Release] FreshJR Adaptive QOS (Improvements / Custom Rules / and Inner workings)

[Randown Question]

Watch the Minute 3:40 Web UI

 

[FreshJR]

Update: Bug fixes have all been completed. Thanks to everyone for the reports! (Was a little slow due to me taking a break)

The current delay is from me testing an experimental feature prior to release.
I still need more time testing it before pushing it public.

I don't really want to push two releases (1 with fixes and then 1 with feature)

So everyone affected by glitches or incompatibility with your router, just be patient a little longer.

 

[L&LD]

Update: Bug fixes have all been completed. Thanks to everyone for the reports! (Was a little slow due to me taking a break)

The current delay is from me testing an experimental feature prior to release.
I still need more time testing it before pushing it public.

I don't really want to push two releases (1 with fixes and then 1 with feature)

So everyone affected by glitches or incompatibility with your router, just be patient a little longer.

Do you need any more testers?

 

[FreshJR]

Do you need any more testers?

Testing post release was great.

I don't need pre-release testers since I am looking for specific results/metrics/functionality internally.

(Many revision are made during internal testing that you guys dont see)

 

[medwatt]

@FreshJR, do you think it's possible at this stage to add a way to bandwidth limit a particular IP in your script ?

 

[FreshJR]

@FreshJR, do you think it's possible at this stage to add a way to bandwidth limit a particular IP in your script ?

If wanting to do it from the UI you can bandwidth limit a relatively unused existing category (aka Game Transferring) and stick the users you want limited into that category.

Keep in mind that those users would have to share limits with traffic in that category.

I could probably create an additional "empty" category exlusively used for bandwidth limiting devices, but that isn't planned for this next release.

--

Creating an additional bandwidth limited category can easily be achieved by manual editing a few lines into the script. To reflect it in the UI, make it intractable, etc, takes a BIT more work

##upload
iptables -D POSTROUTING -t mangle -o eth0 -s 192.168.2.100/32 -m mark --mark 0x40000000/0xc0000000 -j MARK --set-mark 0x403FFFFF
iptables -A POSTROUTING -t mangle -o eth0 -s 192.168.2.100/32 -m mark --mark 0x40000000/0xc0000000 -j MARK --set-mark 0x403FFFFF
tc class add dev eth0 parent 1:1 classid 1:100 htb prio 8 rate 1500kbit ceil 1500kbit
tc qdisc add dev eth0 parent 1:100 handle 100: sfq
tc filter add dev eth0 parent 1: protocol all prio 50 u32 match mark 0x403FFFFF 0xC03FFFFF flowid 1:100

##download
iptables -D POSTROUTING -t mangle -o br0 -d 192.168.2.100/32 -m mark --mark 0x80000000/0xc0000000 -j MARK --set-mark 0x803FFFFF
iptables -A POSTROUTING -t mangle -o br0 -d 192.168.2.100/32 -m mark --mark 0x80000000/0xc0000000 -j MARK --set-mark 0x803FFFFF
tc class add dev br0 parent 1:1 classid 1:100 htb prio 8 rate 1500kbit ceil 1500kbit
tc qdisc add dev br0 parent 1:100 handle 100: sfq
tc filter add dev br0 parent 1: protocol all prio 50 u32 match mark 0x803FFFFF 0xC03FFFFF flowid 1:100

FlowID in this example is the same as @Martineau "Real-time Quote Monitoring Script".

Change it if you do not want conflicts with his script. More info from my post here:

https://www.snbforums.com/threads/e...-download-upload-data-used.50066/#post-446015

 

[medwatt]

If wanting to do it from the UI you can bandwidth limit a relatively unused existing category (aka Game Transferring) and stick the users you want limited into that category.

Keep in mind that those users would have to share limits with traffic in that category.

I could probably create an additional "empty" category exlusively used for bandwidth limiting devices, but that isn't planned for this next release.

--

Creating an additional bandwidth limited category can easily be achieved by manual editing a few lines into the script. To reflect it in the UI, make it intractable, etc, takes a BIT more work

##upload
iptables -D POSTROUTING -t mangle -o eth0 -s 192.168.2.100/32 -m mark --mark 0x40000000/0xc0000000 -j MARK --set-mark 0x403FFFFF
iptables -A POSTROUTING -t mangle -o eth0 -s 192.168.2.100/32 -m mark --mark 0x40000000/0xc0000000 -j MARK --set-mark 0x403FFFFF
tc class add dev eth0 parent 1:1 classid 1:100 htb prio 8 rate 1500kbit ceil 1500kbit
tc qdisc add dev eth0 parent 1:100 handle 100: sfq
tc filter add dev eth0 parent 1: protocol all prio 50 u32 match mark 0x403FFFFF 0xC03FFFFF flowid 1:100

##download
iptables -D POSTROUTING -t mangle -o br0 -d 192.168.2.100/32 -m mark --mark 0x80000000/0xc0000000 -j MARK --set-mark 0x803FFFFF
iptables -A POSTROUTING -t mangle -o br0 -d 192.168.2.100/32 -m mark --mark 0x80000000/0xc0000000 -j MARK --set-mark 0x803FFFFF
tc class add dev br0 parent 1:1 classid 1:100 htb prio 8 rate 1500kbit ceil 1500kbit
tc qdisc add dev br0 parent 1:100 handle 100: sfq
tc filter add dev br0 parent 1: protocol all prio 50 u32 match mark 0x803FFFFF 0xC03FFFFF flowid 1:100

FlowID in this example is the same as @Martineau "Real-time Quote Monitoring Script".

Change it if you do not want conflicts with his script. More info from my post here:

https://www.snbforums.com/threads/e...-download-upload-data-used.50066/#post-446015

Thanks for your reply. I will try what you suggested. I'm just wondering if there's a better way to do what I'm trying to accomplish.

A neighbour of mine uses my WiFi. What I want to do is for him to only have full speed when I'm not using the internet. For instance, at the moment, when he's streaming a movie, my download speed suffers. This is obviously the way your script is supposed to work, but I'd like my downloads to take priority irrespective of whatever he is doing.

I tried giving him "lowest priority" under Bandwidth Monitor, but that didn't seem to have any effect. What is the point of Device Priority anyway ?

I've also tried putting him under "Others" category and limiting his bandwidth. While that works, his speed never gets above the limit I've set unless I change it back to normal.

What I really want here is a have two types of priorities. I want devices to first be ranked into high and low priorities and then the traffic be ranked into high and low priorites. Is there an elegant way to accomplish this ?

 

[FreshJR]

@medwatt

Just stick him into file downloads, give his device lowest priority, but LEAVE the rate at 5%/100%,

Device priority works to assign priority **WITHIN** traffic classes for competing devices.

Eg

(Device A - Lowest Priority ) will still be higher in overall priority when "Streaming" compared to (Device B - Highest Priority) in "Downloads"

While this is working as expected, bandwidth allocation between devices of different priorities (in the same class) is generally terrible when they are competing for bandwidth.
This is why I tell users not to touch it / use it as the final results are generally undesired!

But in your case, lowest priority in file downloads should be fine for him.

(Tell him if he coughs up the dough you'll put him into into lowest priority video streaming haha)

--

With functioning QoS bandwidth limiting is kind of a waste.

 

[medwatt]

@medwatt

Just stick him into file downloads, give his device lowest priority, but LEAVE the rate at 5%/100%,

Device priority works to assign priority **WITHIN** a traffic classes for devices.

Eg

(Device - A) Lowest Priority in "Streaming" will still be higher than (Device - B) Highest Priority in "Downloads"

Even then the bandwidth allocation between devices of different priorities in the same class will not be good.
This is why I tell users not to touch it / use it as the results are generally undesired!

But in your case, lowest priority in file downloads should be fine for him.

(Tell him if he coughs up the dough you'll put him into into lowest priority video streaming haha)

Haha. I'll try putting him in downloads and see if it works.

 

[FreshJR]

update

I just checked what rates are allowed for depirotized devices when using device priority.
(just to see if the improvements have been made with the last few firmware updates to bring the feature into a usable state)

Deprioritized devices are still assigned rates of 16kbit (2kB/s). This means medwatts neighbors connection will completely cease to function whenever @medwatt has any open download instead of throttling his device into a reasonable rate.

The same logic extends to anyone else considering using the device priority as mentioned in the first post:
DO NOT USE DEVICE PRIORITY!! It does not produce sane results or anything remotely close to logical!

I don’t know what they were thinking when choosing these rates. !!! They don't work at all !
(There is a logical solution if they had the slightest care about functionality or fixing bugs)

--

Open letter to ASUS:

1) You only need ONE bandwidth shaper for the QOS structure, PERIOD.
The existing HTB at root sorting traffic into 8 sub-categories is the only ONE that is needed (with better rates of course).

2) The resulting 8 children should NOT be cascaded HTB's of their own (seriously, dont do that).....
Instead each subcategory should schedule traffic weighted round robin (weights assigned per device priority) and de-queue its incoming traffic into a codel queue.

Eg:
All devices medium priority: 1-packet per each device is dequeued per round. (again these will end up into individual per device fq-codel queues).

1 device high priority, 1 device medium priority: 2-packets de-queued for the high device & 1 packet de-queued for medium device per round.

1 device high priority, 1 device low priority: 4-packets dequeued for high device & 1 packets de-queued for low device after 2 rounds

This is HOW you get functional device priority! What is your intended functionality vs the actual end results we are seeing with these cascaded HTB's that have terribly assigned rates?!? Do you guys use this product?

--

Now if you are seriously lazy. Make 1 HTB the leads to 8 fq-codel queues ... that it.
It would perform better then these pointless cascaded HTB queues.
You would have no disfunctional device priority settings to worry about

--

I retract recommending Medwatt to use it for QOS'ing his neighbor earlier. He will have to create an additional queue or re-purpose an existing one as instructed earlier!

 

[sfortis]

sorry for asking something that maybe is already discussed but this thread is huge!

...is this normal that every https packet is classified as "net control packet" ? the script has been installed without any mod on ac56u / merlin 384.6

 

[L&LD]

sorry for asking something that maybe is already discussed but this thread is huge!

...is this normal that every https packet is classified as "net control packet" ? the script has been installed without any mod on ac56u / merlin 384.6

You are running an older firmware than what the script supports (I think).

 

[FreshJR]

sorry for asking something that maybe is already discussed but this thread is huge!

...is this normal that every https packet is classified as "net control packet" ? the script has been installed without any mod on ac56u / merlin 384.6

Script modifications have not taken effect in the screenshot.

Did you restart QoS after installing the script and then wait the 5 minutes as instructed?

No need to read the entire thread, just reading first three posts is sufficient.
I will add "You can stop reading here" to the end of the third post.

@L&LD

Incorrect, the script itself is still compatible and functioning on older firmwares.

Compatability has been maintained for ALL previous firmwares sans having the new WebUI page shown on anything pre 384.9

 

[FreshJR]

So ..... can someone test the following command on stock firmware?

service "restart_qos;restart_firewall"

I am ready for release but I still need to see if have to pull the "QoS restart after rule modification" feature for users on stock firmware prior to pushing the release.

(Other experimental feature that was in testing been pulled.)

 

[QuikSilver]

(Other experimental feature that was in testing been pulled.)

Hmmmmm.......interested in what that was.....

 

[sfortis]

Script modifications have not taken effect in the screenshot.

Did you restart QoS after installing the script and then wait the 5 minutes as instructed?

No need to read the entire thread, just reading first three posts is sufficient.
I will add "You can stop reading here" to the end of the third post.

@L&LD

Nope the script (modification) itself are still compatible and functioning on older firmwares.

Compatability has been maintained for ALL previous firmwares sans have the new WebUI page available on anything pre 384.9

thanks, a re-installation did the trick.

is it possible to get the config-only table webUI (without tha graphs etc that from what i understand are not compatible with older firmware) for pre-384.6 firmware? maybe as a user page in /www?

 

[FreshJR]

is it possible to get the config-only table webUI (without tha graphs etc that from what i understand are not compatible with older firmware) for pre-384.6 firmware

It is possible but I am not sure it’s worth the effort to backport an alternate webpage for outdated firmware’s.

Are you never planning to update?? Visually seeing your rules take effect in the connection table after creation is great.

The following takes you the a similar table like structure.

/jffs/scripts/FreshJR_QOS -rules

 

[FreshJR]

Update has been pushed

v8.1
-bug fixes
-backport UI to partially display on v382+ instead of v384.9+

 

[Ditch]

Update has been pushed

v8.1
-bug fixes
-backport UI to partially display on v382+ instead of v384.9+

FreshJR can you add the version number and the changelog in the first post? I have to always check the conversation only to know if anything has been released.

 

[FreshJR]

FreshJR can you add the version number and the changelog in the first post? I have to always check the conversation only to know if anything has been released.

Done.

Also, simply running the update command should tell you if you are on the current version.

I usually don’t push updates often so you rarely have to check.

I never kept track of all the changes so I don’t have a change log.

I guess I also made some UI tweaks to the pie chart area as reading the data was slightly difficult but it never hurts to be on the current version.

---

Mixed up the hard coded labels in v8.1 (whoops)

For the few people that quickly grabbed the v8.1 version hours after release —> update to v8.2.

Install guide and tool coming for users on stock firmware next.