[Release] FreshJR Adaptive QOS (Improvements / Custom Rules / and Inner workings)

[Sinner]

It have been replaced by v382

I realize that but have been lead to believe its not compatible with 382+

 

[FreshJR]

I realize that but have been lead to believe its not compatible with 382+

It will receive an minor update to make the template rules current and be posted on the first page soon enough.

Functionality wise, it never broke v382+.

It was supposed to be superceeded by this beta version making all changes instantaneous.
The beta version also was to have a significantly easier install process.

Unfortunately we ran into some nasty bugs along the way.
For the most part, all have been squashed except for one.

 

[Sinner]

Youve done a great job with both! I'm honestly torn between the two but i think id much prefer to use a songle script myself.. its a single script I can figure out and tweak as needed without all the guesswork behind the whole faketc thing going on I can live with a delayed start on it. but that's just me I guess.. let the masses decide =p

** Need a vote thing in here for this! **

 

[elintseeker]

is this normal? I've just installed the script.

https://pastebin.com/bC2YBCQC

after a couple of seconds... none of that type of messages on the logs

Mar  8 12:58:06 adaptive QOS: Warming Up
Mar  8 12:58:09 adaptive QOS: Warming Up
Mar  8 12:58:09 adaptive QOS: Warming Up
Mar  8 12:58:10 adaptive QOS: Warming Up
Mar  8 12:58:26 adaptive QOS: Applying Custom Upload Rules
Mar  8 12:58:28 adaptive QOS: Applying Custom Download Rules
Mar  8 12:59:31 adaptive QOS: Parsing User QOS Rates -> (Updating)
Mar  8 12:59:31 rc_service: udhcpc 8844:notify_rc start_firewall
Mar  8 12:59:32 miniupnpd[8911]: shutting down MiniUPnPd
Mar  8 12:59:32 dhcp_client: bound xxx.xx.xx.116 via xxx.xx.0.1 during 3600 seconds.
Mar  8 12:59:32 nat: apply nat rules (/tmp/nat_rules_eth0_eth0)
Mar  8 12:59:33 custom_script: Running /jffs/scripts/firewall-start (args: eth0)
Mar  8 12:59:33 miniupnpd[16461]: HTTP listening on port 33239
Mar  8 12:59:33 miniupnpd[16461]: Listening for NAT-PMP/PCP traffic on port 5351
Mar  8 12:59:51 kernel: HTB: quantum of class 10010 is small. Consider r2q change.
Mar  8 12:59:52 adaptive QOS: Applying Iptables Rules
Mar  8 12:59:52 adaptive QOS: Adaptive QOS: Changing 1:10 class rate
Mar  8 12:59:53 adaptive QOS: Adaptive QOS: Changing 1:11 class rate
Mar  8 12:59:55 adaptive QOS: Adaptive QOS: Changing 1:12 class rate
Mar  8 12:59:55 kernel: HTB: quantum of class 10012 is small. Consider r2q change.
Mar  8 12:59:57 adaptive QOS: Adaptive QOS: Changing 1:13 class rate
Mar  8 12:59:58 kernel: HTB: quantum of class 10014 is small. Consider r2q change.
Mar  8 13:00:00 adaptive QOS: Adaptive QOS: Changing 1:14 class rate
Mar  8 13:00:02 kernel: HTB: quantum of class 10015 is small. Consider r2q change.
Mar  8 13:00:03 adaptive QOS: Adaptive QOS: Changing 1:15 class rate
Mar  8 13:00:05 adaptive QOS: Adaptive QOS: Changing 1:16 class rate
Mar  8 13:00:06 kernel: HTB: quantum of class 10017 is small. Consider r2q change.
Mar  8 13:00:08 adaptive QOS: Adaptive QOS: Changing 1:17 class rate
Mar  8 13:00:44 disk_monitor: Got SIGALRM...

 

[Quoc Huynh]

That has to be enabled under administration in the web interface. You will know which one it is once you see it.

Use manual bandwidth with 90-95% of your achievable.

I am sorry if my question was answered somewhere. I just wonder that 90% is for upload and 95% is for download bandwidth, right? Can I adjust it to 85% - 95% depends on dslreports test result?

Moreover, thank you @FreshJR for your great script, especially the beta 8! It is the cleanest and most working QoS stats version so far.

 

[chris.at]

Ok, everyone, here is BETA8.

Hi FreshJR,

Sorry for being late, upgraded to Beta8 right now and some strange things happen. Restarted router to be sure it's not a problem coming from playing with iptables but still the same:

Mar  8 06:59:56 rc_service: httpd 283:notify_rc restart_qos;restart_firewall
Mar  8 06:59:56 adaptive QOS: Warming Up
Mar  8 07:00:15 adaptive QOS: Warming Up
Mar  8 07:00:15 adaptive QOS: Warming Up
Mar  8 07:00:29 adaptive QOS: Applying Custom Upload Rules
Mar  8 07:00:29 adaptive QOS: iptables v1.4.15: unknown option "--set-mark"
Mar  8 07:00:29 adaptive QOS: Try `iptables -h' or 'iptables --help' for more information.
Mar  8 07:00:29 adaptive QOS: iptables v1.4.15: unknown option "--set-mark"
Mar  8 07:00:29 adaptive QOS: Try `iptables -h' or 'iptables --help' for more information.
Mar  8 07:00:29 adaptive QOS: Applying Custom Download Rules
Mar  8 07:01:37 adaptive QOS: Parsing User QOS Rates -> (No Changes)
Mar  8 07:01:37 nat: apply nat rules (/tmp/nat_rules_ppp0_eth0)
Mar  8 07:01:38 custom_script: Running /jffs/scripts/firewall-start (args: ppp0)
Mar  8 07:01:57 adaptive QOS: Applying Iptables Rules
Mar  8 07:01:57 adaptive QOS: Adaptive QOS: Changing 1:10 class rate
Mar  8 07:01:58 adaptive QOS: Adaptive QOS: Changing 1:11 class rate
Mar  8 07:01:59 adaptive QOS: Adaptive QOS: Changing 1:12 class rate
Mar  8 07:02:01 adaptive QOS: Adaptive QOS: Changing 1:13 class rate
Mar  8 07:02:04 adaptive QOS: Adaptive QOS: Changing 1:14 class rate
Mar  8 07:02:06 adaptive QOS: Adaptive QOS: Changing 1:15 class rate
Mar  8 07:02:08 adaptive QOS: Adaptive QOS: Changing 1:16 class rate
Mar  8 07:02:09 adaptive QOS: Adaptive QOS: Changing 1:17 class rate

When I copy the line and insert some fake values for the variable ${Net_mark} to paste it manually iptables accepts and installs it:

iptables -A POSTROUTING -t mangle -o ppp0 -s 192.168.1.20/32 -j MARK --set-mark 0x1/0x7

With previous betas there were no complaints about set-mark in the script, maybe the variable isn't completely initialized when the script tries to setup iptables?

Regards,
Chris

 

[FreshJR]

@chris.at

Your log is a little weird.

Your two errors appear after "Applying Custom Upload Rules"
If you read lower, no errors appear after "Applying Iptables Rules"

You probably either your have all your iptable rules in the wrong section OR have old remants that were not deleted in the custom upload rule section.
No iptables rules should be in that section.

All iptable rules have to be in the iptable rule section.

I made all mark variables invalid in the custom upload rule section.
This was done so an error is produced for users not paying close attention to the sections, so they can see something is wrong in the system log.

I hope this makes sense.

--

@elintseeker

Yes it is normal, it occures when one of your class rate allocations exceeds 16mbps.
It is a harmless error message .

--

@Quoc Huynh

Yes it can be anything. The range is just a recommendation.

--

As an update to everyone else.

The first post will be updated soon with the current release candidates.
I have reached a point where I am VERY satisfied with the two versions.

1) The latest fakeTC method has tight timings, and all changes are applied almost instantly after clicking "Apply"
2) The long sleep firewall start script version has been updated with a simplified install procedure and updates to the rule templates.

The only thing preventing me from doing so is the bug @skeal has mentioned.
Honestly speaking, I am too lazy to install all the applications he has running to recreate his bug.
I hope he will agree to run a debug version.

If I squash this bug, we should be ready to roll with no more constant updates!

--

@skeal

Since I cannot attach files in a PM. Would you be willing to try out these two files for me?
I really want to see what is happening with your mount issue you mentioned.
**all other users, this is a debug version with lots of logging enabled intended for @skeal, skip it and wait for release**

 

[chris.at]

You probably either your have all your iptable rules in the wrong section

Thank you, you were right. The splitted section is new to beta8 so I didn't recognize it and pasted the rules right below realtc filters as it was before. Sorry for my blindness!
Now everything runs fine!

Best wishes,
Chris

 

[Quoc Huynh]

@Quoc Huynh

Yes it can be anything. The range is just a recommendation.

Thank you, @FreshJR. May I ask again that the first percentage is for upload and the second one is for download bandwidth?

 

[FreshJR]

Thank you, @FreshJR. May I ask again that the first percentage is for upload and the second one is for download bandwidth?

no its general range recommended for both.

Its supposed to be lower than 100% of your max bandwidth on both.
How much lower depends on your isp. So 85-95% is just a general range.

 

[Quoc Huynh]

no its general range recommended for both.

Its supposed to be lower than 100% of your max bandwidth on both.
How much lower depends on your isp. So 85-95% is just a general range.

Oh, I see. I thought they are strict percentage applied for upload and download bandwidth, respectively. Thank you very much

 

[Vexira]

95% if using speedtest.net values 85% for ISP values.

 

[Quoc Huynh]

95% if using speedtest.net values 85% for ISP values.

Thanks @Vexira! I will try both values to see which are the most suitable ones for my Internet speed

 

[elintseeker]

@FreshJR
After playing with the console (PS4) i'm getting random router disconnects and a bunch of logs seemingly restarting the router. I've formatted the jffs and removed the script momentarilly till I get the time to check what's happening in the logs.

edit: reinstalled script

 

[chris.at]

@FreshJR , I noticed that although I set up iptables and it gets matches traffic from the classified ip address gets classified as other instead of netcontrol, download gets classified correctly. I can't say if this worked with other betas before because I only checked iptables matches before (sorry for that). When I change Net_mark to Downloads_mark upload traffic gets classified correctly do downloads category so it seems as if you are using a wrong value for Net_mark (0x40090000/0xffffffff) while Net for download classification is correct.

Regards,
chris

Edit: Solved, you set the following in your script:
admin@router:/tmp/home/root# cat /jffs/scripts/FreshJR_QOS | grep _mark=
VOIP_mark="0x40060001" # Marks to be used if using iptables version of rules
Gaming_mark="0x40080001" # Note these marks are same as filter match/mask combo but have a 1 at the end. That trailing 1 prevents them from being caught by unidentified mask
Others_mark="0x400a0001"
Web_mark="0x400d0001"
Streaming_mark="0x40040001"
Downloads_mark="0x40030001"
Default_mark="0x40000000"
Net_mark="0x40090000"

You write about the 1 at the end but you set Default and Net with 0 at the end. I changed Net to 1 and it works now.
Net_mark="0x40090001"

With Default it's more difficult, with 0 at the end its classified as Others again, with 1 as Streaming.

 

[FreshJR]

@FreshJR , I noticed that although I set up iptables and it gets matches traffic from the classified ip address gets classified as other instead of netcontrol

Correct, net control should of had a trailing 1, or else it would go into the unidentified traffic catagory which then gets redirected to "others" per one of the script rules, as you have experienced.
Nice catch.

If you need a working rule for "defaults" that wont get redirected into "others" just ask.
I can post an example.

-- I also thought of a good application for "Default" catagory.
I will place all my IOT devices into default.
This way I will see really quick if they are scheming some nasty telemetry data in the background!

 

[chris.at]

If you need a working rule for "defaults" that wont get redirected into "others" just ask.
I can post an example.

Yes, that would be fine (not only for me), thank you in advance!

 

[FreshJR]

Yes, that would be fine (not only for me), thank you in advance!

already included in the upcoming release candidate. Just give it a another day (maybe a few hours) if I dont fall asleep.

 

[chris.at]

No problem, go to bed, this can wait! A new day just began here that's why I answered now, not because I need it now.

 

[Sinner]

already included in the upcoming release candidate. Just give it a another day (maybe a few hours) if I dont fall asleep.

Could you include the old rules 22/23 i think going into web again i think? Even comented out if your not planning on using that anymore? I prefered only high priority dns etc traffic in netcontrol

Im out again for one last day of hell this week so i cant check but you know the ones

Oh and ios/google appstore downloads if possible in downloads would be dreamy as well if its not going there already