AutoBW: Automatically set QoS bandwidth using spdMerlin

[peepsnet]

for example here is a list of speed test conducted by spdmerlin
View attachment 22454
here is an example speed test done on ookla from a client,
View attachment 22455

From the spdMerlin OP:
spdMerlin uses SpeedTest CLI
If the speeds from this utility are significantly lower than those you see from the desktop app/browser test, the limiting factor will be your router's CPU. That being said, there is probably room for improvement/optimisation by the Ookla team in the CLI binary itself. Please direct feedback about speed issues to Ookla via the above link. There is nothing that I can do about them.
https://www.snbforums.com/threads/spdmerlin-automated-speedtests-with-graphs.55904/

 

[SomeWhereOverTheRainBow]

From the spdMerlin OP:
spdMerlin uses SpeedTest CLI
If the speeds from this utility are significantly lower than those you see from the desktop app/browser test, the limiting factor will be your router's CPU. That being said, there is probably room for improvement/optimisation by the Ookla team in the CLI binary itself. Please direct feedback about speed issues to Ookla via the above link. There is nothing that I can do about them.
https://www.snbforums.com/threads/spdmerlin-automated-speedtests-with-graphs.55904/

I hear your argument, that is why i raise the question of the usefulness of the script until issues with ookla binaries gets resolved , or it is determined to be a router cpu issue. (dont know if it is cpu as it bearly breaks a sweat performing the task- RT-AX88U)

 

[squidbilly]

Working on v2.0 at the moment ... with some help from peepsnet

 

[Jack Yaz]

Working on v2.0 at the moment ... with some help from peepsnet

looks like you're working to remove the dependency on FreshJR_QoS and manage the correct ceilings and bursts etc.?

I was going to ask you about merging it into spdMerlin, but looks like you have a fully fledged project developing. I'll PM you if you need to make any changes, I'm looking at reworking how spdMerlin generates output data for charting which may involve breaking changes for your script.

 

[peepsnet]

I'm looking at reworking how spdMerlin generates output data for charting which may involve breaking changes for your script.

@Jack Yaz Fun...

Let us know.

Right now we just read the last X lines from:
/jffs/addons/spdmerlin.d/csv/Uploaddaily_WAN.htm
and
/jffs/addons/spdmerlin.d/csv/Downloaddaily_WAN.htm

To get the values. We will just need to know where you put the data in the future

 

[maghuro]

I still think that it was a good ideia to merge this project into spdmerlin
Because if AutoBW doesn't survive without spdmerlin... I think it's a good idea to merge it.

Just my two cents

 

[SomeWhereOverTheRainBow]

I still think that it was a good ideia to merge this project into spdmerlin
Because if AutoBW doesn't survive without spdmerlin... I think it's a good idea to merge it.

Just my two cents

Maybe merge it with an on off switch

 

[maghuro]

Maybe merge it with an on off switch

Yes of course

 

[peepsnet]

Maybe merge it with an on off switch

Well... Specifically it would just be a feature...

spdMerlin
1) All the original functionality of spdMerlin
2) Modify QOS values based on Speed tests
3) Modity Traffic Control based on... (I dont know how that works... I am just working on the surrounding code)

OR

AutoBW could run its own speedtest...

OR

just ask you for the values you get from a speed test

I see issues with all the options because of the "false" speedtest values you can get from running a speed test.

I am a perfect example....

I can plug my PC directly into my Cable Modem and get 850-980Mbps( I have 1Gbps internet)
Through the router I can only get <500Mbps

So setting the QOS from a speedtest would be below the true limit of my internet speed!!!


I don't know what to do with all this!!!

 

[immi803]

@squidbilly @peepsnet
I can confirm AutoBW works with ipv4 and not taking care of internet supporting Dual stack (ipv6 with fall back of ipv4)

Whereas spdMerlin does take ipv6 first if available

 

[juched]

spdMerlin use to measure 880 - 900 Mbps for my on my gigbit connection. The same or very close to my clients. As soon as I turned on AiProtection, and now with QoS enabled, it cannot get higher than 470Mbps.

Clients on my network can get still >800Mbps. I have a feeling the speedtest cli is single threaded or something, and the extra overhead of processing network traffic is throttling it. This is with a RT-AX88U, so a fast CPU.

In top command I see speedtest-cli go up to ~25%.

 

[Jack Yaz]

@squidbilly @peepsnet
I can confirm AutoBW works with ipv4 and not taking care of internet supporting Dual stack (ipv6 with fall back of ipv4)

Whereas spdMerlin does take ipv6 first if available

you know its just calling spdmerlin's generate commands right?

 

[immi803]

you know its just calling spdmerlin's generate commands right?

I know, this confused me also
Can't figure out why spdMerlin calling out is ipv4 and spdMerlin independently uses ipv6, there must be something with the code

 

[squidbilly]

you know its just calling spdmerlin's generate commands right?

Yep, sure wouldn't work without it.

 

[squidbilly]

curl "https://raw.githubusercontent.com/squidbillyms/AutoBW/development/AutoBW" -o /jffs/scripts/AutoBW && chmod 755 AutoBW
Might need 'dos2unix AutoBW'

There is lots of new code, so it will need lots of testing I tried to test the best I could but you know how that goes.
In addition to options in the config file, AutoBW can now be run with command-line arguments like:
AutoBW -sim (simulation mode; do not change anything but simulate what would happen)
AutoBW -num_tests 3 (average speeds over 3 spdMerlin tests; set QoS upload/download bandwidth)
AutoBW -read_bw -num_tests 10 (read in the last 10 spdMerlin tests from daily csv files and calculate average values; set QoS upload/download bandwidth)
and so on ....
Try AutoBW -help for all the options (it can still be run without any options; the options are pulled in from the AutoBW config file)

 

[JonEngland]

curl "https://raw.githubusercontent.com/squidbillyms/AutoBW/development/AutoBW" -o /jffs/scripts/AutoBW && chmod 755 AutoBW
Might need 'dos2unix AutoBW'

There is lots of new code, so it will need lots of testing I tried to test the best I could but you know how that goes.
In addition to options in the config file, AutoBW can now be run with command-line arguments like:
AutoBW -sim (simulation mode; do not change anything but simulate what would happen)
AutoBW -num_tests 3 (average speeds over 3 spdMerlin tests; set QoS upload/download bandwidth)
AutoBW -read_bw -num_tests 10 (read in the last 10 spdMerlin tests from daily csv files and calculate average values; set QoS upload/download bandwidth)
and so on ....
Try AutoBW -help for all the options (it can still be run without any options; the options are pulled in from the AutoBW config file)

Nice, was thinking it could do with a separate config file as when it previously updated it lost the settings

Working ok, apart from some issue with the update check, e.g.

Checking prerequisites:
 [ X ] AsusWRT-Merlin version checked!
 [ X ] bc installed
 [ X ] spdMerlin is available.
 [ X ] spdMerlin is >=3.3.2
 [ X ] spdMerlin is not currently running.
 [ X ] spdMerlin Stats found
 [ X ] Adaptive QOS Enabled
 [ X ] AutoBW config file found
 Checking for update ...
 WARNING: This version is v2.0
 WARNING: Latest version is v1.5
 WARNING: The file hashes don't match.

UPDATE:
it ran ok first time but subsequent times seem to give same result below

               --------------- SPDMERLIN --------------
                 Download (Kbps)       Upload (Kbps)
               -------------------  -------------------
Test  1 of  2      111769.6              10260.5
Test  2 of  2      111759.4              10137.6
Average            111764.5              10199.0
Scale Factors          0.85                 0.85
Scaled Speeds       94999.8               8669.2
               ----------------- QOS -----------------
                Download (Kbps)        Upload (Kbps)
               -----------------     -----------------
Previous              94734.0              8595.0
New (from above)      94999.8              8669.2
Scaling by              1.003               1.009
              ====================================== DOWNLOAD ======================================
               ----------------- RATE -----------------    ----------------- CEIL -----------------
Class               Previous                New                 Previous                New
-----          -------------------  -------------------    -------------------  -------------------
 ERROR: Did not recognize TC ceil unit:

previous version 1.5 is still working ok

UPDATE2:
I do see entries as below in logs and bandwidth values are updated in gui, so what is the impact of the ERROR above?

Apr  9 19:04:59 RT-AC86U-D318 AutoBW:  -----> Setting QoS Download Speed to  97175.8 Kbps ...
Apr  9 19:04:59 RT-AC86U-D318 AutoBW:  -----> Setting QoS Upload Speed to   8782.3 Kbps ...

 

[figorr]

Well... Specifically it would just be a feature...

spdMerlin
1) All the original functionality of spdMerlin
2) Modify QOS values based on Speed tests
3) Modity Traffic Control based on... (I dont know how that works... I am just working on the surrounding code)

OR

AutoBW could run its own speedtest...

OR

just ask you for the values you get from a speed test

I see issues with all the options because of the "false" speedtest values you can get from running a speed test.

I am a perfect example....

I can plug my PC directly into my Cable Modem and get 850-980Mbps( I have 1Gbps internet)
Through the router I can only get <500Mbps

So setting the QOS from a speedtest would be below the true limit of my internet speed!!!


I don't know what to do with all this!!!

I am having the same issue.

Running a 600/600Mbps PPPoE fiber connection under a RT-AX88U. QOS and FreshJR script enabled (90% bandwidth limit fixed)

When I run the speedtest with my PC directly connected to the router I got around 560/560Mbps (withQOS disabled I got around 620/620Mbps). That's Ok. But when I ran the spdMerlin script ... I didn't get any test result higher than 350/350Mbps, even with QOS disabled.

I don't know why the speed tests directly from the router are too low.

 

[StefanoN]

Hi
With the version 2.0 after the test in my log I see this message:

Apr 9 18:58:11 RT-AC86U-81D8 kernel: HTB: quantum of class 10010 is small. Consider r2q change.
Apr 9 18:58:11 RT-AC86U-81D8 kernel: HTB: quantum of class 10013 is small. Consider r2q change.
Apr 9 18:58:11 RT-AC86U-81D8 kernel: HTB: quantum of class 10015 is small. Consider r2q change.
Apr 9 18:58:11 RT-AC86U-81D8 kernel: HTB: quantum of class 10014 is small. Consider r2q change.
Apr 9 18:58:11 RT-AC86U-81D8 kernel: HTB: quantum of class 10017 is small. Consider r2q change.
Apr 9 18:58:11 RT-AC86U-81D8 kernel: HTB: quantum of class 10016 is small. Consider r2q change.

What is it ?

Thanks

 

[squidbilly]

Working ok, apart from some issue with the update check, e.g.

Checking prerequisites:
 [ X ] AsusWRT-Merlin version checked!
 [ X ] bc installed
 [ X ] spdMerlin is available.
 [ X ] spdMerlin is >=3.3.2
 [ X ] spdMerlin is not currently running.
 [ X ] spdMerlin Stats found
 [ X ] Adaptive QOS Enabled
 [ X ] AutoBW config file found
 Checking for update ...
 WARNING: This version is v2.0
 WARNING: Latest version is v1.5
 WARNING: The file hashes don't match.

No problem. That's is just a warning that you are running a version that is different (it is in development) from the official release (v2.0 vs. v1.5). Obviously, you don't want to update to the official release since it is an earlier version (v1.5) so it just warns you. If you had been running an earlier version (say v1.4) it would have updated you automatically.

 

[squidbilly]

UPDATE:
it ran ok first time but subsequent times seem to give same result below

               --------------- SPDMERLIN --------------
                 Download (Kbps)       Upload (Kbps)
               -------------------  -------------------
Test  1 of  2      111769.6              10260.5
Test  2 of  2      111759.4              10137.6
Average            111764.5              10199.0
Scale Factors          0.85                 0.85
Scaled Speeds       94999.8               8669.2
               ----------------- QOS -----------------
                Download (Kbps)        Upload (Kbps)
               -----------------     -----------------
Previous              94734.0              8595.0
New (from above)      94999.8              8669.2
Scaling by              1.003               1.009
              ====================================== DOWNLOAD ======================================
               ----------------- RATE -----------------    ----------------- CEIL -----------------
Class               Previous                New                 Previous                New
-----          -------------------  -------------------    -------------------  -------------------
 ERROR: Did not recognize TC ceil unit:

previous version 1.5 is still working ok

It has to do with picking out the units (b, Kbps, Mbps, etc) for the class ceil/rates. What does tc class show dev br0 | grep "parent 1:1" output? I only have Kbps speeds so it is hard for me to adequately test that section.