Would moving VoIP further down be fine if you do not have dedicated VoIP? I take this statement to mean that enabling WiFi Calling on your cellphone does NOT fall into the VoIP category. What would be some real scenarios that would fall into that category for an average home user? Would this be things like FaceTime and Skype?
[Release] FreshJR Adaptive QOS (Improvements / Custom Rules / and Inner workings)
- Thread starter FreshJR
- Start date
- Status
NGRhodes
Regular Contributor
Hi,
It may be worth noting that there in release/src/asustools/squashfs-root/usr/sbin/ there is a svqos2 which is now utilising hfsc for the AC86U 3.0 kernel; the code looks pretty identical apart from this change, probably something that needs to be tested.
Also, I was looking at how we could achieve per-host isolation and stumbled on this (https://serverfault.com/questions/7...dwidth-per-ip-using-htb-a-cidr-range-in-linux), which also turns out how Gargoyle does it:
The bottom line is the cool bit.
fq_codel should do a good enough job (within each [per-ip] class), but I think it may be possible to then create sub classes in each bucket based on Adaptive QoS, might remove/reduce the need to rely on additional per application rules.
Over the winter months I am going to try and create a proof of concept script, then the step after this would be creating something more generic that correctly considers total ip addresses available (in-use or available) and setting suitable rates and incorporating into the FreshJR script.
It may be worth noting that there in release/src/asustools/squashfs-root/usr/sbin/ there is a svqos2 which is now utilising hfsc for the AC86U 3.0 kernel; the code looks pretty identical apart from this change, probably something that needs to be tested.
Also, I was looking at how we could achieve per-host isolation and stumbled on this (https://serverfault.com/questions/7...dwidth-per-ip-using-htb-a-cidr-range-in-linux), which also turns out how Gargoyle does it:
Code:
tc qdisc add dev eth0 root handle 1: htb default 12
tc class add dev eth0 parent 1: classid 1:1 htb rate 100mbit ceil 100mbit
# Here, you want a fair qdisc
tc qdisc add dev eth0 parent 1:1 handle 101: sfq perturb 10
# Put a range in the filter
tc filter add dev eth0 protocol ip parent 1:0 prio 1 u32 match ip src 192.168.1.0/24 flowid 1:1
# Create 254 bucket, each ip src will be attached to one bucket
tc filter add dev eth0 parent 1:1 protocol ip handle 10 flow hash keys nfct-src divisor 254The bottom line is the cool bit.
fq_codel should do a good enough job (within each [per-ip] class), but I think it may be possible to then create sub classes in each bucket based on Adaptive QoS, might remove/reduce the need to rely on additional per application rules.
Over the winter months I am going to try and create a proof of concept script, then the step after this would be creating something more generic that correctly considers total ip addresses available (in-use or available) and setting suitable rates and incorporating into the FreshJR script.
JohnSmith
Senior Member
Yes I am sure, as I tried both dport & sport , as well as TCP and UDP on both upload rules for testing purpose, and in all 8 tests (4 tests per 2 rules), Upload traffic still got classified into "Web Surfing".
When I ran "netstat -na" on the Window machine at a command prompt while the download was happening, it showed TCP
On the ASUS router, when I used the Network Tools, for "netstat-nat" and by "Source IP" it shows TCP
Are there any logging commands that I can run to determine where these Upload packets are being classified into?
FreshJR
Very Senior Member
I was planning to do something like this earlier to fix the device priority but I stopped.
Asus already dynamically modifies the TC structure as devices enter & leave the network.
Code:
1:1 - root HTB
1:10 - NetControl
10:1 - NetControl HTB
10:2 - device1
10:3 - device2
10:4 - device3
1:11 - VoOP
11:1 - VoIP HTB
11:2 - device1
11:3 - device2
11:4 - device3
1:12
1:13
1:14
1:15
1:16I was planning to intercept Asus's TC commands for the sub structure and fix the rate/ciels/ & priority for each device to my liking.
I find that the TC parameters Asus is using to dynamically implement the per device priority in the substructure leads to a terrible end result (this is my opinion).
Calculating and changing rate/priority per networked device via the existing structure wasn't an issue at all.
I was going to piggyback off RMerlins fakeTC to intercept the commands and run some calculations to make it work. (In doing that, I did find that the QOS commands issued from the bwdpi engine issues are time sensitive before it throws errors and wipes the entire TC structure and restarts, I did find a way to work around that).
Everything was going good but I ran into an issue.
If my iPhone was device2. Once it left the network, the next device that connected to the network would will replace my iPhone as device2 instead of being assigned device4.
They are filting devices via an ID assigned within the mark and not filtering by device IP. I will pull up a picture of the mark bitwise structure I created and post it here so it can be clear.
EDIT:
Since I couldn't figure out which mark corresponds with which device (as the correlation is random and tracked internally somewhere), I stopped trying at this point to fix it.
---
I see that you are trying to do it in the reverse order and that will work even better. (I was trying to maintain the existing structure with different parameters, but do perfer it in the reverse order as you mentioned).
The reason I perfer first by device and they by traffic, is that even if traffic identification is poor. Each device will always have its own guaranteed bandwidth and will use it like having a dedicated connection even if identification completely fails. The traffic prioritization that would be occuring in the subclass would be nice on per machine traffic performance but blips in performance wouldn't make or break QOS quality.
A word of advice is you will HAVE to use a different set of class IDs for each perIP class and also for the all the perTraffic class within each perIP class so Asus dynamic modification do not interfere with your structure.
(We could block all of Asus dynamic modifcations via fakeTC but you would then also have to cross your fingers and hope Asus doesn't try to wipe the new structure).
My approach would be to use fakeTC to block the Asus structure entirely and then create our own. The issue is that the bwdpi executable that issues the TC commands would definitely not be happy.
If someone can comment out the QOS check, I would be happy to join you in creating this new QOS structure that dynamically adjusts based on devices present on the network.
Both Wifi-Calling and FaceTime were unidentified. I created special rules for both those cases. Try making some calls and see if the VoIP category grows. I believe Skype was correctly identified during the time I last tested it, but that was a few years ago.
@JohnSmith
Why do you keep using so many filtering elements?
Eg.
Code:
iptables -D POSTROUTING -t mangle -o $wan -s X.X.X.X/32 -m mark --mark 0x801400c2/0xc03fffff -p tcp --dport 49152:65535 -j MARK --set-mark ${Downloads_mark_up} &> /dev/null
iptables -A POSTROUTING -t mangle -o $wan -s X.X.X.X/32 -m mark --mark 0x801400c2/0xc03fffff -p tcp --dport 49152:65535 -j MARK --set-mark ${Downloads_mark_up}Could be condensed into this
Code:
iptables -D POSTROUTING -t mangle -o $wan -m mark --mark 0x801400c2/0xc03fffff -j MARK --set-mark ${Downloads_mark_up} &> /dev/null
iptables -A POSTROUTING -t mangle -o $wan -m mark --mark 0x801400c2/0xc03fffff -j MARK --set-mark ${Downloads_mark_up}And has higher chance of the match not being void by a mismatch with the source IP, transfer protocol, or ports used.
Go ahead and use this to log packets.
Code:
iptables -D POSTROUTING -t mangle -o $wan -m mark --mark 0x801400c2/0xc03fffff -m limit --limit 1/s -j LOG &> /dev/null
iptables -A POSTROUTING -t mangle -o $wan -m mark --mark 0x801400c2/0xc03fffff -m limit --limit 1/s -j LOG
Last edited:
Sinner
Senior Member
The facetime discord teamspeak ventrillo and alot of online games being unidentified is why i find myself using voip for my unidentified at the top and manually moving anything out i need to. Others for me is unused
JohnSmith
Senior Member
Thanks for the log code FreshJR! So I used it to see if on the upload, the "Lets Encrypt" traffic would get logged to show why it is being classified under the "Web Surfing" and not the "File Transferring", and no logs ever showed up in the system log. Is this not where the "LOG" command puts this information if it matches your code, or is it somewhere else?
So I have gone back to my rule that puts this ONE computers "Lets Encrypt" traffic uploading under "File Transferring" (unlike your code above that puts ALL computers "Lets Encrypt" traffic under File Transferring, which I didn't want to happen yet unless necessary in the long run), and it works
Code:
iptables -D POSTROUTING -t mangle -o $wan -s X.X.X.X/32 -j MARK --set-mark ${Downloads_mark_up} &> /dev/null
iptables -A POSTROUTING -t mangle -o $wan -s X.X.X.X/32 -j MARK --set-mark ${Downloads_mark_up}I was just hoping to tie it down further to not only an IP address, but to its specific ports, in case other applications get classified under the "Lets Encrypt" classification, and I would then consider that traffic to be in "Web Surfing" (or higher, depending on what it is or does).
FreshJR
Very Senior Member
I messed up the log rule. 0x8 marks are download, 0x4 marks are upload. (The log rule was trying to look download traffic on upload interface, thats why nothing hit)
This is same reason why your original upload rule was not hitting at all. The mark prefix was not 0x4.
I will make the results of -appDB clearer in the next release.
Try these
Code:
iptables -D POSTROUTING -t mangle -o $wan -m mark --mark 0x401400c2/0xc03fffff -m limit --limit 1/s -j LOG &> /dev/null
iptables -A POSTROUTING -t mangle -o $wan -m mark --mark 0x401400c2/0xc03fffff -m limit --limit 1/s -j LOG
Last edited:
JohnSmith
Senior Member
Thanks FreshJR, that was exactly it (I missed that as well when looking at the "iptables -vL -t mangle", argh, it's always the smallest change), change "0x8" to "0x4" in uploads, (also used "dport" in the upload rule, as "sport" was used in the download rule) and classification is now working properly for that one computer......
Code:
iptables -D POSTROUTING -t mangle -o $wan -s X.X.X.X./32 -m mark --mark 0x401400c2/0xc03fffff -p tcp --dport 563 -j MARK --set-mark ${Downloads_mark_up} &> /dev/null
iptables -A POSTROUTING -t mangle -o $wan -s X.X.X.X/32 -m mark --mark 0x401400c2/0xc03fffff -p tcp --dport 563 -j MARK --set-mark ${Downloads_mark_up}
Vexira
Part of the Furniture
i remember ages ago fresh asking for game ports posting it here for anyone that wants it
https://us.battle.net/support/en/article/7842
https://us.battle.net/support/en/article/7842
WAN-Noob
New Around Here
@FreshJR
Hi, I am using this script on ASuS RT-AC68U with merlin's fw and it's working great. I get good enough bandwidth when my family is watching netflix. So thanks for doing all the hard work.
I'm very noob in regards to iptables and tc and was wondering if this script could be modified to use on custom linux router? Could you give any pointers if possible?
I use ppp to connect to internet on a debian router.
Hi, I am using this script on ASuS RT-AC68U with merlin's fw and it's working great. I get good enough bandwidth when my family is watching netflix. So thanks for doing all the hard work.
I'm very noob in regards to iptables and tc and was wondering if this script could be modified to use on custom linux router? Could you give any pointers if possible?
I use ppp to connect to internet on a debian router.
FreshJR
Very Senior Member
no, the update is just text updates for the rarely used AppDB command to make the output a little more clear.
It is so minor I have been holding off on pushing it unless I get more questions about it.
Yes and no. The deep packet inspection system that categorizes traffic by type that TC filters on will only work on the Asus router it was licensed for.
You can still use TC on the linux box to implement QOS by allocating bandwidth & priority per networked device instead of allocating bandwidth & priority per traffic type.
(Alloting good limits and priority per device will give equivalent performance to limiting by traffic type. It is just a different approach. This setup will even perform better than using DPI since bandwidth will always be allocated as you expect it to compared to the situations where DPI is performing poorly, which this script tries to mitigate).
Last edited:
WAN-Noob
New Around Here
Thanks for clarifying that, I will try to find more resources on tc and learn which is the best way to implement for a small home.
PS: Thanks again for your awesome script for Asus.
FreshJR
Very Senior Member
Both should be showing up.
You can try using AppAnalysis in the WebUI and then look it up with the AppDB command within the script. This method should have less guessing and even works when the network is busy.
Barebones you need to perform rate limiting and then each user should have either fq-codel or cake as their queuing discipline.
A step head is an HTB on the root where you filter each user into their own class that will have a minimum + maximum bandwidth & priority per user.
Should be many examples online as this is common.
Last edited:
Question....my paid Xfinity speeds are 250/10 mbps with blast included. My actual speeds from speed test range from 275-300 mbps and upload ranges from 10-12 mbps. When setting up the 90% of upload/download speeds Ive set them up anywhere from 210-245 and 10 mbps and my bufferbloat always shows a grade of C. Even doing 200mbps shows the same. I am doing the speed test on wireless (5g) on my mobile phone, is this the reason why I'm always getting a grade of C? Should I try a wired PC test? I'm using the DLS reports speed test website. Any help is greatly appreciated
FreshJR
Very Senior Member
Yes if your wifi link rate drops below your internet speeds that will also introduce bloat.
Bloat occurs when any link through the network path is saturated. The wired results should be your true results.
If wifi is constrained, maybe get another access point so it can connect at a higher link rate if the throughput is falling off due to lower signal.
Bloat occurs when any link through the network path is saturated. The wired results should be your true results.
If wifi is constrained, maybe get another access point so it can connect at a higher link rate if the throughput is falling off due to lower signal.
Thanks @FreshJR ...I'll connect my PC wired and will some get true readings. Thanks again for the assist as always..
Read somewhere with the new Merlin beta 384.8, VPN settings might have gotten better for download without making any mods, except keeping the upload mod using your script...will you be making updates to your 1st page ?
- Status
Similar threads
DomainVPNRouting
Domain VPN Routing v3.0.4 ***Release***
Similar threads
Similar threads
-
DomainVPNRouting Domain VPN Routing v3.0.4 ***Release***
- Started by Ranger802004
- Replies: 54
-
-
FlexQoS FlexQoS 1.4.4 - Flexible QoS Enhancement Script for Adaptive QoS
- Started by dave14305
- Replies: 62