[Release] FreshJR Adaptive QOS (Improvements / Custom Rules / and Inner workings)

[charlie2alpha]

Hi FreshJR,

I looked through your script to see what might be broken. It looks like the match strings are slightly different in 382.. The second field has changed from 0x8000FFFF to 0xC000FFFF, as an example. In general all the of the second field values have a 4 OR'd in (0x8 OR 0x4 == 0xC) There are also now 6 or 7 different entries in the TC filters that detect the FFFF fields. On my system only the first one is used - not sure what triggers the Trend Micro code to use the other fields ...perhaps it's prio.

It also looks at first glance that ASUS has cleaned up their rate/ceil allocations a little bit. A quick glance at the current values in the tc class dump for 382 shows values that look pretty much like what your default math in the script does. I haven't stared at that so much so perhaps I've missed a detail, but the ASUS defaults might be ok.

I made some minor mods to your script to look for the first instance of the "Unidentified" Filter and ignore the rest as well as search for 0xC.... fields. It appears to do the right thing on my system. Generalizing these changes to support both 380 and 382 plus truly understanding what the other entries do would be some more work.

All that to say it's probably not too much work to get your script going for the 382 people again.

/dedd

I just finished checking the script on my own router and came to similar conclusions. But since I do not know exactly how to properly fix for now I have to disable it as it may do more harm than good with the classification.

 

[charlie2alpha]

Well scratch that, I couldn't let it go so easily after all so I think I have now fixed the script, nothing goes to the default class now.

 

[JDB]

Hi All,

Just a brief update to let you know that FreshJR has asked me to become the custodian of this project as he is no longer an active Asus user.

I’ve forked the repository and will look at getting a new release sorted soon. I’ve been a bit short of time since 382 came out as we’ve just had a baby!

@charlie2alpha - as you’ve clearly made some good progress here, would you mind PM’ing me so I can steal (and credit) your work [emoji4]

FreshJR will continue to support me in the background as I find my feet with the project but please address any issues towards myself going forwards (or indeed just post in this thread).

And I’ll sign off by just offering a thanks from myself and all the users of this great project to FreshJR for creating and managing it to date.


POSTED IN ERROR - see later post https://r.tapatalk.com/shareLink?sh...rums.com/index.php?posts/369775/&share_type=t

Sent from my iPhone using Tapatalk

 

[deddc23efb]

Hi All,

Just a brief update to let you know that FreshJR has asked me to become the custodian of this project as he is no longer an active Asus user.

I’ve forked the repository and will look at getting a new release sorted soon. I’ve been a bit short of time since 382 came out as we’ve just had a baby!

@charlie2alpha - as you’ve clearly made some good progress here, would you mind PM’ing me so I can steal (and credit) your work

FreshJR will continue to support me in the background as I find my feet with the project but please address any issues towards myself going forwards (or indeed just post in this thread).

And I’ll sign off by just offering a thanks from myself and all the users of this great project to FreshJR for creating and managing it to date.


Sent from my iPhone using Tapatalk

Hi @JBD and @charlie2alpha,

I think all you need to do is replace all instances of the mask fields that currently start with 0x8.. or 0x4 with 0xc..
So 0x80000000 0x8000ffff becomes 0x80000000 0xc000ffff or 0x40130000 0x403f0000 becomes 0x40130000 0xC03f0000 and so on.. make sure to change all of them.
If the plan is to make this work for releases for 380 and 382, then you'll need to make this conditional somehow ( look for both matches, have the script derive the right value to match, make two scripts ... up to you.)

I reset my router to get the config back to a clean state and realized that my earlier observations were confused by my testing. It looks like there is only one instance of Unknown (as opposed to the multiple extras I had added.)

The traffic rates are not exactly as described by FreshJR in his original posts, however all classes appear to share the same bandwidth allocation. I'm thinking it still makes sense to run the new traffic rate and ceiling config in FreshJR's script.


/dedd

 

[charlie2alpha]

Hi @JBD and @charlie2alpha,

I think all you need to do is replace all instances of the mask fields that currently start with 0x8.. or 0x4 with 0xc..
So 0x80000000 0x8000ffff becomes 0x80000000 0xc000ffff or 0x40130000 0x403f0000 becomes 0x40130000 0xC03f0000 and so on.. make sure to change all of them.
If the plan is to make this work for releases for 380 and 382, then you'll need to make this conditional somehow ( look for both matches, have the script derive the right value to match, make two scripts ... up to you.)

I reset my router to get the config back to a clean state and realized that my earlier observations were confused by my testing. It looks like there is only one instance of Unknown (as opposed to the multiple extras I had added.)

The traffic rates are not exactly as described by FreshJR in his original posts, however all classes appear to share the same bandwidth allocation. I'm thinking it still makes sense to run the new traffic rate and ceiling config in FreshJR's script.


/dedd

Hi @deddc23efb , your observations are in agreement with mine, what you describe is exactly what I did more or less. I suppose it can be made conditional but IMHO it will be simpler to just develop a new fixed version for the new Merlin versions starting with 382.x.

 

[JDB]

Thanks dedd. Your input is greatly appreciated.

I’m going to have a proper look at it all and then make a new post along with a new release I can control (and probably have FreshJR lock this one with a final post pointing at my new one).


Sent from my iPhone using Tapatalk

 

[deddc23efb]

Thanks dedd. Your input is greatly appreciated.

I’m going to have a proper look at it all and then make a new post along with a new release I can control (and probably have FreshJR lock this one with a final post pointing at my new one).


Sent from my iPhone using Tapatalk

Good to hear. Thanks for taking on the work of supporting this.

Note: I just tried my suggested changes, keep in mind the script is doing strict text matches and the letter case matters. I used uppercase 'C' and nothing worked, make sure to use lower case 'c'.

/dedd

 

[charlie2alpha]

Good to hear. Thanks for taking on the work of supporting this.

Note: I just tried my suggested changes, keep in mind the script is doing strict text matches and the letter case matters. I used uppercase 'C' and nothing worked, make sure to use lower case 'c'.

/dedd

That's the way things work in *nix type systems, everything is case sensitive. I used the lowercase, exactly as it appeared in tc output.

 

[deddc23efb]

That's the way things work in *nix type systems, everything is case sensitive. I used the lowercase, exactly as it appeared in tc output.

Thanks charlie2alpha,

I knew that, but I'm used to coding in C and like my hex constants in big letters... force of habit.

Since we are hopping on this, I decided to look at the filter config and traffic config a little harder. I see a few technical problems with the script as it is operating on my system. I don't know if we should wait for a new thread to discuss but here goes....

I'll list them so they can be group-thinked...

1) BUG Line 145 and 154, tc filter prio 1 is deleted, however the new Unidentified catch-all is injected as prio 2. Prio 1 on my system at startup is matching and sending traffic to a different bucket ( 1:2 ) that appears to be an aggregate flow of some sort. I think this is a bug. That entry should not be deleted and PRIO 2 should be deleted since that is where the catch-all is configured by ASUS and by the script. I don't know if it has the same behaviour on version 380 (or whether that PRIO is stable from release to release.)

2) ENHANCEMENT: Custom Filter Rules are being injected in at PRIO 1 - the rule priority that is being deleted. I think for consistency they should be moved to PRIO2. Additionally I would suggest that the catch-all at PRIO 2 should be added LAST in set of rules at PRIO2 to avoid if being over-greedy. If BUG1 is fixed, then moving the custom rules to PRIO2 becomes a required change to cleanup any old custom rules (or something similar) on a config change.

3) ENHANCEMENT: ASUS has placed the Unidentified catch-all filter as TOP priority by putting it essentially first in line at PRIORITY 2. It would seem if they trusted Trend Micro to "GET IT RIGHT" more often than "GET IT WRONG" that catch-all should be last. It's possible that it might be more efficient to move the catch-all to PRIORITY 255 so that it is only hit when NOTHING else matches. Tough to say what to do now that we are using it for OTHER.

3) BUG In the bandwidth config, none of the code that attempts to parse the existing rates/ceiling/burst sizes works correctly. The parsing fails to remove "kbit" and "b" from the tail of strings. The result is that the bandwidth updates are always applied, even if not required. The trailing stuff can be stripped with a simple sed command added to the end of the parsing.

4) DOUBT: I think the burst and cburst computations in FreshJR's original script are not quite right. The same burst and cburst sizes are applied to all the traffic categories regardless of expected rate.

 

[Jack Yaz]

Out of interest - does VPN client traffic in 382 get registered as upload by QoS? If so, is there anything that this script can do to amend it?

 

[JDB]

I’ve just realised I’ve made a complete tit of myself!!!
Posted all of this on the wrong thread without actually reading which one I was on!
Many apologies @FreshJR!
It’s another NTP project I’m taking on.
Saw notifications of loads of people commenting on this asking for updates and got my wires crossed not thinking straight, not enough sleep!!!
Sorry again.
Still happy to help in any way I can! It was on my list to look at getting this working for 382 anyway.


Sent from my iPhone using Tapatalk

 

[lalala2]

What am I doing wrong? The script doesn't seem to be starting although firewall-start runs properly.

-rwxr-xr-x    1 admin    root         18029 Jan  6 14:07 FreshJR_QOS
-rwxr-xr-x    1 admin    root            36 Jan  6 13:55 firewall-start
-rwxrwxrwx    1 admin    root           206 Dec  9  2016 services-start
admin@RT-AC68U-DC20:/jffs/scripts .\FreshJR_QOS
-sh: .FreshJR_QOS: not found
admin@RT-AC68U-DC20:/jffs/scripts ./FreshJR_QOS
-sh: ./FreshJR_QOS: not found

 

[deddc23efb]

What am I doing wrong? The script doesn't seem to be starting although firewall-start runs properly.

-rwxr-xr-x    1 admin    root         18029 Jan  6 14:07 FreshJR_QOS
-rwxr-xr-x    1 admin    root            36 Jan  6 13:55 firewall-start
-rwxrwxrwx    1 admin    root           206 Dec  9  2016 services-start
admin@RT-AC68U-DC20:/jffs/scripts .\FreshJR_QOS
-sh: .FreshJR_QOS: not found
admin@RT-AC68U-DC20:/jffs/scripts ./FreshJR_QOS
-sh: ./FreshJR_QOS: not found

The second one should work. Do you have an errant "FreshJR_QOS" accidentally invoked at the top of your script (assuming you modified it?)
You can also try

/bin/sh FreshJR_QOS

to eliminate any path and permission funnies.

 

[lalala2]

I haven't modified the .92 version script. Maybe it's down to the OS version, as I'm running 380.68_4. I was able to run part of it successfully with the direct command.

admin@RT-AC68U-DC20:/jffs/scripts# /bin/sh FreshJR_QOS
: not found: line 9:
: not found: line 34:
: not found: line 36:
FreshJR_QOS: line 119: syntax error: unexpected "done" (expecting "fi")
admin@RT-AC68U-DC20:/jffs/scripts# /jffs/scripts/FreshJR_QOS
-sh: /jffs/scripts/FreshJR_QOS: not found
admin@RT-AC68U-DC20:/jffs/scripts# /jffs/scripts/FreshJR_QOS
-sh: /jffs/scripts/FreshJR_QOS: not found
admin@RT-AC68U-DC20:/jffs/scripts#

 

[deddc23efb]

I haven't modified the .92 version script. Maybe it's down to the OS version, as I'm running 380.68_4. I was able to run part of it successfully with the direct command.

admin@RT-AC68U-DC20:/jffs/scripts# /bin/sh FreshJR_QOS
: not found: line 9:
: not found: line 34:
: not found: line 36:
FreshJR_QOS: line 119: syntax error: unexpected "done" (expecting "fi")
admin@RT-AC68U-DC20:/jffs/scripts# /jffs/scripts/FreshJR_QOS
-sh: /jffs/scripts/FreshJR_QOS: not found
admin@RT-AC68U-DC20:/jffs/scripts# /jffs/scripts/FreshJR_QOS
-sh: /jffs/scripts/FreshJR_QOS: not found
admin@RT-AC68U-DC20:/jffs/scripts#

The lines you are referencing are all blank lines in FreshJR's script. Is your script in DOS format rather than UNIX? The line endings do matter and can make things go wrong.
You can experiment with fixing this on your router. From cli run:
/usr/bin/dos2unix FreshJR_QOS

If you have messed up line endings that will fix them. After that try to run your script manually again.

 

[brummygit]

I just had a quick read through the recent posts and see that FreshJR is no longer an active user and JDB had his wires crossed. Those are both perfectly understandable but I'm suffering with broken upload QoS since moving to 382.2 Beta 2 on my RT-AC68U.

I wonder if either @deddc23efb or @charlie2alpha have a working version of the script which they would be willing to let me have a copy of to help with my troubles? I understand if not as you could easily slip in to picking up the ongoing support of FreshJRs script.

 

[JDB]

I just had a quick read through the recent posts and see that FreshJR is no longer an active user and JDB had his wires crossed. Those are both perfectly understandable but I'm suffering with broken upload QoS since moving to 382.2 Beta 2 on my RT-AC68U.

I wonder if either @deddc23efb or @charlie2alpha have a working version of the script which they would be willing to let me have a copy of to help with my troubles? I understand if not as you could easily slip in to picking up the ongoing support of FreshJRs script.

FreshJR is still an active Asus user (just less here these days). Ignore everything I said! As you say, my wires were well crossed!




Sent from my iPhone using Tapatalk

 

[deddc23efb]

Hi @brummygit,

JDB, charlie2alpha and I are working on some changes to support 382. It is in .. lets call it alpha... right now. I'm expecting that we will have something that would be useful for beta testing soon.

/Dedd

 

[brummygit]

Hi @brummygit,

JDB, charlie2alpha and I are working on some changes to support 382. It is in .. lets call it alpha... right now. I'm expecting that we will have something that would be useful for beta testing soon.

/Dedd

Thanks, that really is great news. FreshJR had done some great work fixing Asus's mistakes and it was looking like we might loose them in order to move forward with releases. For people like myself who aren't competent scripters it's really great that you are all on the case as the difference his script made to my network was very noticeable.

 

[lalala2]

Hi, it looks like the DOS2UNIX program fixed all of the issues. I'm surprised because I pressed the download button directly on the pastebin and didn't resave it or do anything to mess up the original format. Thank you for the help!

admin@RT-AC68U-DC20:/jffs/scripts# /usr/bin/dos2unix FreshJR_QOS
admin@RT-AC68U-DC20:/jffs/scripts# ./FreshJR_QOS
admin@RT-AC68U-DC20:/jffs/scripts#

https://www.koszek.com/blog/2015/10/05/pastebin-adds-dos-newline-on-snippet/