[Release] FreshJR Adaptive QOS (Improvements / Custom Rules / and Inner workings)

[ugandy]

Telescola meu amigo?

TV rural!

 

[Kingp1n]

It's always been a "feature" of FreshJR to recategorize Snapchat to Other. What are you seeing differently?

@dave14305...speaking of snapchat categorized under "Others" as default, I know FreshJR recommended for those who used the script for gaming to recategorize snapchat under "web", I believe it was due to when gaming and using a mic & gaming using VOIP. I know you mentioned you're not a gamer but I just wanted to put that out of there!

 

[dave14305]

@dave14305...speaking of snapchat categorized under "Others" as default, I know FreshJR recommended for those who used the script for gaming to recategorize snapchat under "web", I believe it was due to when gaming and using a mic & gaming using VOIP. I know you mentioned you're not a gamer but I just wanted to put that out of there!

Any rule that isn’t tricky behind the scenes will be editable/removable in my fork. “Tricky” to me means it doesn’t follow the normal patterns, such as iptables OUTPUT chain vs POSTROUTING.

I’ve been studying Fresh’s code a lot. I need to write a lot on paper to grasp everything he is doing. I don’t want to mess too much with the tc stuff, but I would like to reduce the 5 minute wait based on some output from tc show, hopefully.

I also would like to spay or neuter the ssh menu system since it will become unworkable with my plans for more than 4 rules. We shall see. I’ve yet to ever publish anything.

 

[Kingp1n]

Any rule that isn’t tricky behind the scenes will be editable/removable in my fork. “Tricky” to me means it doesn’t follow the normal patterns, such as iptables OUTPUT chain vs POSTROUTING.

I’ve been studying Fresh’s code a lot. I need to write a lot on paper to grasp everything he is doing. I don’t want to mess too much with the tc stuff, but I would like to reduce the 5 minute wait based on some output from tc show, hopefully.

I also would like to spay or neuter the ssh menu system since it will become unworkable with my plans for more than 4 rules. We shall see. I’ve yet to ever publish anything.

Thanks a lot for the info. I recently updated my script and added "853/80" ports under tcp based from one your older post under the VPN fix:

iptables -D OUTPUT -t mangle -o $wan -p tcp -m multiport ! --dports 53,123,853,80 -j MARK --set-mark ${Downloads_mark_up} &> /dev/null #VPN Fix - (Fixes upload traffic not detected when the router is acting as a VPN Client)
iptables -A OUTPUT -t mangle -o $wan -p tcp -m multiport ! --dports 53,123,853,80 -j MARK --set-mark ${Downloads_mark_up}​

I also created a rule under the GUI for AI protection (23.47.79.0/24 /TCP /Remote port: 80).

Does this output look correct:

admin@RT-AX88U-F2E8:/tmp/home/root# iptables -t mangle -L OUTPUT -n -v
Chain OUTPUT (policy ACCEPT 314K packets, 260M bytes)
pkts bytes target prot opt in out source destination
248K 89M MARK udp -- * eth0 0.0.0.0/0 0.0.0.0/0 multiport dports !53,123,853 MARK set 0x40030001
3003 160K MARK tcp -- * eth0 0.0.0.0/0 0.0.0.0/0 multiport dports !53,123,853,80 MARK set 0x40030001

I'm testing it out now. I can't believe I missed this in the past! Thanks a lot!!!

 

[dave14305]

I can't believe I missed this in the past!

At least you didn’t miss it in the future...yet.

I would argue that putting 853 under udp is not necessary since we know DoT use TCP. Much smarter now than I was then.

 

[distilled]

At least you didn’t miss it in the future...yet.

"Dwight, at 8:00 a.m. today, someone poisons the coffee. Do not drink the coffee. More instructions will follow. Cordially, Future Dwight."

 

[QuikSilver]

"Dwight, at 8:00 a.m. today, someone poisons the coffee. Do not drink the coffee. More instructions will follow. Cordially, Future Dwight."

 

[Vexira]

@dave14305 I've been using your dev branch, and so far so good - ipv6 seems working good (although I have a 6in4 solution, not native. And I use it only sometimes, most for testing things)

What command did you use to opt into the dev branch if you don't mind me asking?

 

[maghuro]

What command did you use to opt into the dev branch if you don't mind me asking?

Changed master to dev inside script (using nano for example) and then force update

 

[Vexira]

Changed master to dev inside script (using nano for example) and then force update

Ahh gotcha thanks for that, which line exactly may i ask?

 

[Sinner]

Any rule that isn’t tricky behind the scenes will be editable/removable in my fork. “Tricky” to me means it doesn’t follow the normal patterns, such as iptables OUTPUT chain vs POSTROUTING.

I’ve been studying Fresh’s code a lot. I need to write a lot on paper to grasp everything he is doing. I don’t want to mess too much with the tc stuff, but I would like to reduce the 5 minute wait based on some output from tc show, hopefully.

I also would like to spay or neuter the ssh menu system since it will become unworkable with my plans for more than 4 rules. We shall see. I’ve yet to ever publish anything.

I've been happy with 30sec delay in mine but there was a reason it was extended so long. Im not sure if it was hardware delays or special scenarios but it was deemed enough to be 100% sure it would work.

 

[Vexira]

I've been happy with 30sec delay in mine but there was a reason it was extended so long. Im not sure if it was hardware delays or special scenarios but it was deemed enough to be 100% sure it would work.

It was to do with changes not applying correctly for some reason, he wanted to give the firewall enough time to process, I remember he explained it in depth some posts back.

Around that time he had two script versions one fast and the other was more stable.

 

[Vexira]

@dave14305 I've been trying to work out which line I need to change to the dev branch, if it's not too much to as if I could be directed to which it is if that's not too much trouble.

 

[Jack Yaz]

@dave14305 I've been trying to work out which line I need to change to the dev branch, if it's not too much to as if I could be directed to which it is if that's not too much trouble.

lines 1877,1906. 2379 probably, they reference https://raw.githubusercontent.com/

 

[pdc]

I'm seeing strange behavior where Steam download traffic is being shown under Net Control Packets, but only in the pie chart. In the Tracked connections list, there are no red applications at all, and at the very bottom in brown is Game Transferring Steam. The actual traffic appears to be generic File Transferring.

Is there a known issue where the pie chart shows something different than the list of connections? Which is correct?

I've attached a screenshot. Note the 8 MB/s in Net Control Packets, but in the Tracked connections the highest priority is Gmail. Some Steam traffic is showing in Web Surfing and File Transferring, but that's just a matter of finding the IP and reclassifying.



I only set this up a few days ago and have not configured anything aside from what's suggested in the first few posts in this thread.

Thanks!

 

[solstyce]

Kudos to FreshJR for creating this amazing improvement to Asus' QoS service!

I struggled to get this configured correctly, and in the end, ended up reading all 212 pages of this thread to better understand how the script works, and how to configure it. Much of what I learned I absorbed through tidbits repeated in different ways across many many posts. But here are a few specific posts that I found particularly informative, and saved for my reference.

• How to allocate all traffic from a specific system to a specific category
• How to set specific source and destination ports if you're editing the script manually
• Netflix's speed test service, Fast
• Using pingplotter to track your connection latency over time
• How to bandwidth limit specific devices

I hope others find this as useful as I did.

 

[solstyce]

My setup

• AC68U router (Merlin 384.16, TrendMicro 2.176)
  
• Pace 5268AC modem/router (DMZ+ to Asus)
  
• AT&T bonded FTTN through Sonic.net (~50Mbps down / ~6Mbps up)

I've been struggling with QoS the last 8 weeks of working from home with the whole family trying to work and learn at the same time - especially for Zoom/WebEx video calls. Tried Asus adaptive QoS, failed. Tried FreshJR Adaptive QoS failed (didn't read the instructions, especially about not using device priority, and having to wait 5 minutes). Read all 212 pages of this thread, success! Almost.

I've set bandwidth manually and tuned for bufferbloat. I'm using default allocation percentages. I've set up custom prioritization - Voip, Gaming, Others, Web Surfing, Streaming, File Transferring. Turn it on, wait 5 minutes, and it works! Eureka!

5 minutes later, I'm back to default Asus adaptive Q0S. Doh!

5 minutes later, it works!

Repeat ad nauseum.

Checking the logs, I see an endless loop of my WAN connection resetting, triggering QoS resetting. So it starts Asus QoS, then FreshJR QoS which waits 5 minutes, then starts up. So I have 5 minutes of good QoS and 5 minutes of horrible.

I've investigated, and apparently ATT sets a 10 minute DHCP lease, and every time it expires, it is causing this problem. If I disable Adaptive QoS, I don't see any mention of

dhcp_client: bound 108.228.12.xxx/255.255.252.0 via 108.228.12.1 for 600 seconds.

or

lldpd[298]: removal request for address of 108.228.12.xxx%4, but no knowledge of it

and the Adaptive QoS doesn't cycle back and forth between Asus and FreshJR. Factory reset of the router and modem didn't fix anything. Also tried reverting from 384.17 to 384.16 with factory reset just in case.

Pasting the syslog triggers a Cloudflare block, so I've posted it here - https://pastebin.com/jWSu4Cfb

I really really really want to get this to work (did I mention I read all 212 pages of this thread?). Sonic was useless - can't sell me a static IP to avoid the DHCP situation, and won't send me a different type of modem to try.

I'm desperately hoping someone here can suggest a way to fix this.

Thanks!

 

[Kingp1n]

My setup

• AC68U router (Merlin 384.16, TrendMicro 2.176)
  
• Pace 5268AC modem/router (DMZ+ to Asus)
  
• AT&T bonded FTTN through Sonic.net (~50Mbps down / ~6Mbps up)

I've been struggling with QoS the last 8 weeks of working from home with the whole family trying to work and learn at the same time - especially for Zoom/WebEx video calls. Tried Asus adaptive QoS, failed. Tried FreshJR Adaptive QoS failed (didn't read the instructions, especially about not using device priority, and having to wait 5 minutes). Read all 212 pages of this thread, success! Almost.

I've set bandwidth manually and tuned for bufferbloat. I'm using default allocation percentages. I've set up custom prioritization - Voip, Gaming, Others, Web Surfing, Streaming, File Transferring. Turn it on, wait 5 minutes, and it works! Eureka!

5 minutes later, I'm back to default Asus adaptive Q0S. Doh!

5 minutes later, it works!

Repeat ad nauseum.

Checking the logs, I see an endless loop of my WAN connection resetting, triggering QoS resetting. So it starts Asus QoS, then FreshJR QoS which waits 5 minutes, then starts up. So I have 5 minutes of good QoS and 5 minutes of horrible.

I've investigated, and apparently ATT sets a 10 minute DHCP lease, and every time it expires, it is causing this problem. If I disable Adaptive QoS, I don't see any mention of

dhcp_client: bound 108.228.12.xxx/255.255.252.0 via 108.228.12.1 for 600 seconds.

or

lldpd[298]: removal request for address of 108.228.12.xxx%4, but no knowledge of it

and the Adaptive QoS doesn't cycle back and forth between Asus and FreshJR. Factory reset of the router and modem didn't fix anything. Also tried reverting from 384.17 to 384.16 with factory reset just in case.

Pasting the syslog triggers a Cloudflare block, so I've posted it here - https://pastebin.com/jWSu4Cfb

I really really really want to get this to work (did I mention I read all 212 pages of this thread?). Sonic was useless - can't sell me a static IP to avoid the DHCP situation, and won't send me a different type of modem to try.

I'm desperately hoping someone here can suggest a way to fix this.

Thanks!

I use Comcast but I ended up buying my own modem. Are you able to use your own modem with ATT?

 

[brummygit]

I'm a long time user of Adaptive QoS on my RRT-AC68U and use FreshJRs script. I have a number of devices on my network that connect to an internet host and constantly send data - CCTV camera, ADSB Receiver etc. and I've noticed that these are often not shown in the QoS Monitoring and I believe aren't being included in bandwidth management. I think I understand why:

When the router starts up these devices connect prior to QoS getting properly started. With FreshJRs script this is even further complicated as his script has to wait until QoS is fully up and running before applying the changes some minutes later. Additionally my router reboots every night because I suffer the memory leak from Trend Micro's AI Protection code so I need to try to automate a fix.

I'm guessing that if I can find a way to reset all active connections on the router after FreshJRs changes have applied this would cause the re-established connections to be properly shaped by QoS.

 

[solstyce]

I use Comcast but I ended up buying my own modem. Are you able to use your own modem with ATT?

Hmm, interesting question. There was a period where AT&T screwed up a firmware update that dropped DMZ+ users' bandwidth by ~40% and people were switching to a different modem. Presumably I could look up that modem and buy one off ebay or Amazon.

I've been assuming the DHCP lease expiration is triggering this loop, so assuming it's not a failing modem, I'm not sure a different type of modem would fix the problem. I'll add it to the hail mary list once I've exhausted any other options...