FlexQoS FlexQoS 1.0 - Flexible QoS Enhancement Script for Adaptive QoS

[dave14305]

Hi, normally wait a moment before upgrading the firmware, but now applied it asap due to wanting to try your flexqos, thanks for this!

Question though: is it correct that there is a tab classification ánd a flexqos tab, basically showing the same info and only difference the customize button?
Or is there something wrong in my setup (asus rt-ac2900 384.18, with formerly freshqos installed)

In the original FreshJR days, the only option to customize the web UI was to overwrite an existing page. Since 384.15 it is no longer required, and so my plan was to not suppress Merlin's original page. My goal is to make the FlexQoS page look as much like Merlin's as possible, with the exception of the unique customization and special re-coloring of the application names.

 

[Bluemosh]

In the master branch for 384.17 and earlier, there are only bug fixes to FreshJR’s original script:

• Fixed AppDB rule categories not reflected in connection list
• Excluded DNS-over-TLS port 853 from VPN traffic fix
• Retrieve WAN interface name from nvram if script not invoked by firewall-start
• Fixed AppDB Custom Rule 4 being ignored in connection list
• Fixed AppDB rules to properly implement wildcard marks
• Fixed port 443 exclusion for Gaming rule

(Pulled this from the FreshJR thread)

As I'm currently running gnutons dsl firmware, is there anyway I can get this version with the backwards compatible fixes for my router? Its on 384.17, any suggestions?

 

[dave14305]

(Pulled this from the FreshJR thread)

As I'm currently running gnutons dsl firmware, is there anyway I can get this version with the backwards compatible fixes for my router? Its on 384.17, any suggestions?

Better asked in the FreshJR thread to avoid confusion here, but that FreshJR_QOS version is installable with:

curl "https://raw.githubusercontent.com/dave14305/FreshJR_QOS/master/FreshJR_QOS.sh" -o /jffs/scripts/FreshJR_QOS --create-dirs && curl "https://raw.githubusercontent.com/dave14305/FreshJR_QOS/master/FreshJR_QoS_Stats.asp" -o /jffs/scripts/www_FreshJR_QoS_Stats.asp && sh /jffs/scripts/FreshJR_QOS -install

 

[Bluemosh]

Better asked in the FreshJR thread to avoid confusion here, but that FreshJR_QOS version is installable with:

curl "https://raw.githubusercontent.com/dave14305/FreshJR_QOS/master/FreshJR_QOS.sh" -o /jffs/scripts/FreshJR_QOS --create-dirs && curl "https://raw.githubusercontent.com/dave14305/FreshJR_QOS/master/FreshJR_QoS_Stats.asp" -o /jffs/scripts/www_FreshJR_QoS_Stats.asp && sh /jffs/scripts/FreshJR_QOS -install

Great! Thanks for your help

 

[Huey11]

He literally just answered this few hours ago, 2 posts above. When kind people devote their time to freely produce stuff like FlexQoS, at least have the courtesy to try and not bog them down with questions already answered.

Well, I fully agree with you, and I can only say I try keep track of this thread and others, did not see an answer to this question and only posted it today without noticing the former post. Possibly due to my short sightness. It pains me a little, while always trying to be careful, getting rebuffed like this, but alas, it remains a fair point.

 

[QuikSilver]

Well, I fully agree with you, and I can only say I try keep track of this thread and others, did not see an answer to this question and only posted it today without noticing the former post. Possibly due to my short sightness. It pains me a little, while always trying to be careful, getting rebuffed like this, but alas, it remains a fair point.

Keep in mind that long ago we all were once "new" to the forum, but I can say for certainty that if you stick around long enough, you too will become annoyed at the comments and "first time" threads that have been answered over and over. They serve no purpose but to clutter up an already detailed forum. I've found that you can not only use the forum search but you can use google to search the forums as well. Take care, and hope you stick around!

Edit: I see now that you joined back in 2015 but only made 5 posts. I suggest becoming more involved in the community then.

 

[andresmorago]

I can't recall if I read this or imagined it, but I seem to think there is a performance hit with rendering clickable links in a list of (up to) 500 connections that refresh every 3 seconds by default. There's already a lot of extra processing in displaying the list with the large number of possible rules to determine the Application class color and sorting by Class.

In general, how many of these IPs do you ever wonder about? I'm curious if anyone else would find this useful or not?

thanks for the feedback. i guess im just curious when looking at the details of the active connections so i would find this somehow useful to me. i dont have any background on asp pages so i had no idea this could affect router performance. ill stick with the ctrl-c + ctrl-v

 

[Sinner]

I would love a more basic updated version. Like by basic im talking just untracted connections in OTHERS and webui with tracked connection features. Half these rules may be redundant and uneeded now particularly since we have the webui and its soo easy to add a tweak if you need one.

The webui is the real asset here now imo

 

[dave14305]

I would love a more basic updated version. Like by basic im talking just untracted connections in OTHERS and webui with tracked connection features. Half these rules may be redundant and uneeded now particularly since we have the webui and its soo easy to add a tweak if you need one.

The webui is the real asset here now imo

What script are we talking about? You can delete any rule in FlexQoS. But your sig says you're on 384.16 with FreshJR_QOS.

 

[dave14305]

I've found juicy beta bug that is a real problem. I never got around to testing it until today, but I have now learned the hard way that iptables MARK target is non-terminating, meaning it keeps on processing more rules even after it finds a match. So the LAST iptables rule that matches a connection will win the day. However, the GUI was coded (by me) to stop matching the re-coloring of the Application name after the FIRST rule hit. So things would appear mis-colored if more than one rule would satisfy it. So I need to check every rule for every connection now, and return the last matching Class as the one to override the Application color. Seemingly working now in my test case.

Then there was the issue that AppDB rules were evaluated (by me) AFTER the iptables rules, so if the original Mark for a connection was part of an AppDB redirection rule, but an iptables rule changes the actual final MARK, the connection would still show up colored with the AppDB rule Class color. So I need to evaluate the AppDB rules BEFORE the iptables rules so things appear correctly.

Still needs some testing but will be in the next 0.8.5 release. Let me know if you've observed things like that also.

Lesson: your most important rules should be at the bottom of the rules list if it's possible more than 1 rule can match your connection.

Spoiler: Background

Interestingly (to me), I discovered this after installing Unbound on my Raspberry Pi again and watching it flood my connection table with DNS requests mostly labeled as DNS. But some were labeled as YouTube and I thought that odd. Must be that the DPI engine identified a DNS lookup for youtube.com as a Youtube connection and classified it as Streaming. So I added a rule for my Pi IP address and dest port 53 to go to Net Control. Fixed, or so I thought.

Next, one of my outbound DNS connections happened to originate from port 16413 which also happens to match the default Facetime rule in the script. It was correctly labeled DNS, but it was colored with my Work-From-Home category. So I knew I had a problem...there goes the afternoon.

 

[Sinner]

I've found juicy beta bug that is a real problem. I never got around to testing it until today, but I have now learned the hard way that iptables MARK target is non-terminating, meaning it keeps on processing more rules even after it finds a match. So the LAST iptables rule that matches a connection will win the day. However, the GUI was coded (by me) to stop matching the re-coloring of the Application name after the FIRST rule hit. So things would appear mis-colored if more than one rule would satisfy it. So I need to check every rule for every connection now, and return the last matching Class as the one to override the Application color. Seemingly working now in my test case.

Then there was the issue that AppDB rules were evaluated (by me) AFTER the iptables rules, so if the original Mark for a connection was part of an AppDB redirection rule, but an iptables rule changes the actual final MARK, the connection would still show up colored with the AppDB rule Class color. So I need to evaluate the AppDB rules BEFORE the iptables rules so things appear correctly.

Still needs some testing but will be in the next 0.9.0 release. Let me know if you've observed things like that also.

Also why im interested in a clean version. Im finding it getting more and more complicated to remove the iptables etc from both files with so many IF statements everywhere

Lesson: your most important rules should be at the bottom of the rules list if it's possible more than 1 rule can match your connection.

Spoiler: Background

Interestingly (to me), I discovered this after installing Unbound on my Raspberry Pi again and watching it flood my connection table with DNS requests mostly labeled as DNS. But some were labeled as YouTube and I thought that odd. Must be that the DPI engine identified a DNS lookup for youtube.com as a Youtube connection and classified it as Streaming. So I added a rule for my Pi IP address and dest port 53 to go to Net Control. Fixed, or so I thought.

Next, one of my outbound DNS connections happened to originate from port 16413 which also happens to match the default Facetime rule in the script. It was correctly labeled DNS, but it was colored with my Work-From-Home category. So I knew I had a problem...there goes the afternoon.
View attachment 24412

This is why im on 384.16 and a script a couple versions back non beta I remember this learning curve with fresh and this is the time where everything that shud be simple goes haywire till u get it all figured out

 

[dave14305]

Version 0.8.4

• Differentiate between version change and hotfix during update (@maghuro)
• Check webui ASP page for differences during update check (@maghuro)
• Disable auto-refresh when total tracked connections exceeds 750. Allow user to re-enable auto-refresh if needed.
  
• Sort wildcard appdb rules at the bottom of the rules list when saving so rules apply properly to connection table
• Reorganize custom rule evaluation to ensure application in proper order
• Added version number to WebUI heading

NOTE: If more than one iptables rule matches a single connection, the last matching rule will determine the actual QoS class. Previous beta versions of FlexQoS always chose the first matching rule, which isn't in harmony with how iptables works.

sh /jffs/addons/flexqos/flexqos.sh update

 

[Sinner]

Version 0.8.4

• Differentiate between version change and hotfix during update (@maghuro)
• Check webui ASP page for differences during update check (@maghuro)
• Disable auto-refresh when total tracked connections exceeds 750. Allow user to re-enable auto-refresh if needed.
  
• Sort wildcard appdb rules at the bottom of the rules list when saving so rules apply properly to connection table
• Reorganize custom rule evaluation to ensure application in proper order
• Added version number to WebUI heading

NOTE: If more than one iptables rule matches a single connection, the last matching rule will determine the actual QoS class. Previous beta versions of FlexQoS always chose the first matching rule, which isn't in harmony with how iptables works.

sh /jffs/addons/flexqos/flexqos.sh update

Thats why i removed them all other than unidentified,wifi calling and https/ssl etc. Asus is working on qos finally i expect things in the appd to change drastically too. id prefer as few rules as possible.

Its proving to be a huge pain in the A$$ tho.. having an issue with an "}" you know the one =p

 

[andresmorago]

*speedtest.net is going to net control packets. is this normal? how can i re-categorize it? When running a speedtest i dont see any connections tagged with net control category (red), just the pie and class data increasing.

This is an indication that the router has reset the tc rules. Run the script with the -check parameter to see if it reapplies the rules (it will log in the syslog). This could also be an indication that my tc delay check isn’t detecting the right conditions and so it applies the tc rules too early and they get replaced.

hello dave.
i now see all speedtest traffic going to websurfing. mark seems to be 1400b9 (im not 100% sure). should i just add an appdb rule in this case?

 

[Sinner]

hello dave.
i now see all speedtest traffic going to websurfing. mark seems to be 1400b9 (im not 100% sure). should i just add an appdb rule in this case?

if you scroll down in the webui there should be a rule already

 

[Sinner]

Version 0.8.4

• Differentiate between version change and hotfix during update (@maghuro)
• Check webui ASP page for differences during update check (@maghuro)
• Disable auto-refresh when total tracked connections exceeds 750. Allow user to re-enable auto-refresh if needed.
  
• Sort wildcard appdb rules at the bottom of the rules list when saving so rules apply properly to connection table
• Reorganize custom rule evaluation to ensure application in proper order
• Added version number to WebUI heading

NOTE: If more than one iptables rule matches a single connection, the last matching rule will determine the actual QoS class. Previous beta versions of FlexQoS always chose the first matching rule, which isn't in harmony with how iptables works.

sh /jffs/addons/flexqos/flexqos.sh update

im running 384.16 fresh or flax 8.9. beta testing both the firmware and qos not an option for me atm.

 

[andresmorago]

if you scroll down in the webui there should be a rule already

yes. i saw it but its for mark 0D0007. im not seeing that mark when running speedtests but mark 1400b9

 

[dave14305]

yes. i saw it but its for mark 0D0007. im not seeing that mark when running speedtests but mark 1400b9

Filter the list on remote port 8080 and the device you run the test from. I see many HTTP connections when running a speedtest using port 8080.

 

[andresmorago]

Filter the list on remote port 8080 and the device you run the test from. I see many HTTP connections when running a speedtest using port 8080.

i have been able to identify 3 marks so far
1400c2
0d00de
1400b9

is there a way to change the name on the custom appdb rules?

 

[dave14305]

is there a way to change the name on the custom appdb rules?

No, but re-enter them in all CAPS and it should show the real name.