CakeQOS CakeQOS-Merlin

[heysoundude]

Interesting unintended consequences for people (ab)using betterspeedtest.sh.

[Bloat] netperf server news

LOL - how many of the 470-odd blacklisted IPs could be tied to people from these parts would you reckon?
(I have no shame using @Jack Yaz 's spdmerlin because it hits a purpose-built and maintained commercial server; this is a guy on a 7Mbps connection somewhere...people: they're just GUIDELINES)

 

[askan7]

Hey, anyone having bad jitter and idle latency with cake? If I use fq-codel my idle latency on dslreports is perfect and no jitter but with cake I have ~10ms idle ping spikes and bad jitter.

This is on a AC86U, 384.19, fresh install everything.

 

[heysoundude]

Hey, anyone having bad jitter and idle latency with cake? If I use fq-codel my idle latency on dslreports is perfect and no jitter but with cake I have ~10ms idle ping spikes and bad jitter.

This is on a AC86U, 384.19, fresh install everything.

you need to spend some time playing with config, it seems...
tell us more about your connection and network; we're probably able to help get you aimed in the right direction

 

[askan7]

you need to spend some time playing with config, it seems...
tell us more about your connection and network; we're probably able to help get you aimed in the right direction

I have ADSL2+ 12/1 . Shouldn't take long to get fiber I hope, have the cables on the street...
Latency under load/buffer bloat is actually pretty good, way better than fq-codel, losing only about 5% bandwidth. But idle latency seems less stable, it's not unstable but i have ping spikes that i didn't have before while there's no load on the network at all.
Download limit 11.8mbps
Upload limit 0.9mbps
extra download: atm
extra upload: atm.

For fq-codel I have to set limits to 10/0.8 in order to have the same bandwidth and avoid massive buffer bloat (about 5% from max bandwidth).
AC86U is connected to the ISP router that I've set to bridge mode.

 

[heysoundude]

I have ADSL2+ 12/1 . Latency under load/buffer bloat is actually pretty good, way better than fq-codel, losing only about 5% bandwidth. But idle latency seems less stable, it's not unstable but i have ping spikes that i didn't have before while there's no load on the network at all.
Download limit 11.8mbps
Upload limit 0.9mbps
extra download: atm
extra upload: atm.

AC86U is connected to the ISP router that I've set to bridge mode.

consider removing your speed limiting - selecting unlimited for cake to go up against is often better below 100Mbps; your ISP seems to be taking care of speed limiting already.
what's your ping time to your ISP's server? You may want to set rtt regional if it is usually under 30ms or get really crazy and use metro if it is usually below 10ms (the connmon script can help you get this narrowed down), but you can also input a value more appropriate to you: if your average ping is steady at around 18ms +/-1, set your rtt at 20000 which is 20ms. {the people who wrote cake were rather specific about setting RTT along with the extra up and download overhead parameters everyone focuses on - a read of the manpage can help clarify things}
I would also recommend setting besteffort and flowblind but also raw (the default/fallback) before those
hopefully this helps you.

 

[askan7]

consider removing your speed limiting - selecting unlimited for cake to go up against is often better below 100Mbps; your ISP seems to be taking care of speed limiting already.
what's your ping time to your ISP's server? You may want to set rtt regional if it is usually under 30ms or get really crazy and use metro if it is usually below 10ms (the connmon script can help you get this narrowed down), but you can also input a value more appropriate to you: if your average ping is steady at around 18ms +/-1, set your rtt at 20000 which is 20ms. {the people who wrote cake were rather specific about setting RTT along with the extra up and download overhead parameters everyone focuses on - a read of the manpage can help clarify things}
I would also recommend setting besteffort and flowblind but also raw (the default/fallback) before those
hopefully this helps you.

If I don't limit it I get high loaded ping. 100ms while downloading and 500ms upload.
I have ~10ms to ISP and 1.1.1.1. But setting rtt below 100ms gives insane amount of re transmits on higher latency servers, I play games and on EU servers I get 50-70ms.

Just tried flowblind, makes things worse at least on dslreports.

 

[heysoundude]

wow...gaming on a 12/1 connection.

alrighty...so you know what doesn't work for you so far. read the manpage, refer to this thread...you'll get it locked in.
(have you tried (dual-)src/dsthost on the appropriate legs?...really not necessary with besteffort, but if we're lobbing grenades, we might as well try these)

 

[Audionut]

extra download: atm
extra upload: atm.

I would suggest to add the appropriate ADSL overhead parameters.

tc-cake(8) - Linux manual page

ADSL Overhead Keywords
       Most ADSL modems have a way to check which framing scheme is in use.
       Often this is also specified in the settings document provided by the
       ISP.  The keywords in this section are intended to correspond with
       these sources of information.  All of them implicitly set the atm
       flag.

       pppoa-vcmux
            Equivalent to overhead 10 atm

       pppoa-llc
            Equivalent to overhead 14 atm

       pppoe-vcmux
            Equivalent to overhead 32 atm

       pppoe-llcsnap
            Equivalent to overhead 40 atm

       bridged-vcmux
            Equivalent to overhead 24 atm

       bridged-llcsnap
            Equivalent to overhead 32 atm

       ipoa-vcmux
            Equivalent to overhead 8 atm

       ipoa-llcsnap
            Equivalent to overhead 16 atm

How are you measuring the idle latency, and are you sure that there are not background tasks going at the time?

 

[askan7]

I would suggest to add the appropriate ADSL overhead parameters.

tc-cake(8) - Linux manual page

ADSL Overhead Keywords
       Most ADSL modems have a way to check which framing scheme is in use.
       Often this is also specified in the settings document provided by the
       ISP.  The keywords in this section are intended to correspond with
       these sources of information.  All of them implicitly set the atm
       flag.

       pppoa-vcmux
            Equivalent to overhead 10 atm

       pppoa-llc
            Equivalent to overhead 14 atm

       pppoe-vcmux
            Equivalent to overhead 32 atm

       pppoe-llcsnap
            Equivalent to overhead 40 atm

       bridged-vcmux
            Equivalent to overhead 24 atm

       bridged-llcsnap
            Equivalent to overhead 32 atm

       ipoa-vcmux
            Equivalent to overhead 8 atm

       ipoa-llcsnap
            Equivalent to overhead 16 atm

How are you measuring the idle latency, and are you sure that there are not background tasks going at the time?

Adding bridged-llcsnap helped a bit, thank you.
I get AAA on dslreports and can get A+ quality of I set rtt to 300ms. Servers available for testing are far away, that's why...
Anyway I prefer rtt 100ms as latency seems a bit better as most of the traffic is below 100ms.
Fast.com gives really good results, around 20-25ms unloaded and 30-35ms loaded latency with these settings.

As for idle latency it seems to be about the same, maybe an issue with background tasks/devices, not sure. Loading some websites causes single random ping spike higher than loaded latency. Spike to 60-100ms sometimes then stabilizes at normal 20-30ms.
Just starting a youtube video sometimes does the same, spike and then stable.

 

[Audionut]

I get AAA on dslreports and can get A+ quality of I set rtt to 300ms. Servers available for testing are far away, that's why...

In which case I would suggest to measure performance as I have described here: https://www.snbforums.com/threads/cakeqos-merlin.64800/post-622003

As for idle latency it seems to be about the same, maybe an issue with background tasks/devices, not sure. Loading some websites causes single random ping spike higher than loaded latency. Spike to 60-100ms sometimes then stabilizes at normal 20-30ms.
Just starting a youtube video sometimes does the same, spike and then stable.

This suggests to me that there is still a buffer impacting on performance. I would suggest to lower the "download limit", perhaps to 10mbps, and see if that has the desired affect.
edit: and if so, you can then raise the download limit to find the optimal balance between speed and latency.

edit2: I misread the adsl speeds. With a 12/1 ratio, you may want to use "ack-filter" or even "ack-filter-aggressive". I can't recall where the documentation is for these parameters, but if memory serves correctly, this will discard unnecessary acknowledgement packets and free up some bandwidth.

 

[askan7]

In which case I would suggest to measure performance as I have described here: https://www.snbforums.com/threads/cakeqos-merlin.64800/post-622003




This suggests to me that there is still a buffer impacting on performance. I would suggest to lower the "download limit", perhaps to 10mbps, and see if that has the desired affect.
edit: and if so, you can then raise the download limit to find the optimal balance between speed and latency.

edit2: I misread the adsl speeds. With a 12/1 ratio, you may want to use "ack-filter" or even "ack-filter-aggressive". I can't recall where the documentation is for these parameters, but if memory serves correctly, this will discard unnecessary acknowledgement packets and free up some bandwidth.

I was already using multiple ping -t at the same time. Also I can replicate the cake-qos idle latency issue now with this test. No load on connection at all, just ping. As soon as I disable cake, stable 7-8ms to 1.1.1.1 while running test.

Now I know why the new ping spikes while gaming with not load on connection at all... Maybe it's related to running HW acceleration off and cpu not being as effective or even firmware issue?

Edit: Just did a factory reset using the WPS+power buttons. Simple config with only SSID, enable SSH and jffs scripts, installed cake. Problem remains.

 

[Morris]

Hey, anyone having bad jitter and idle latency with cake? If I use fq-codel my idle latency on dslreports is perfect and no jitter but with cake I have ~10ms idle ping spikes and bad jitter.

This is on a AC86U, 384.19, fresh install everything.

Try testing yourself rather than using a script. Do this

Set up a repeating ping to your closest router
ping -t x.x.x.x

Then go to speedtest.net

if cake is working properly, there should be little or now change in the ping times. The changes are the jitter. When I do this I get zero jitter. When I use dslreports test I get up to 10 ms of jitter and sometimes more. I don't know what there pinging during the load yet it's not the nearest router. I've even run the ping while running dslreports test and the ping shows perfect.

Morris

 

[Morris]

I was already using multiple ping -t at the same time. Also I can replicate the cake-qos idle latency issue now with this test. No load on connection at all, just ping. As soon as I disable cake, stable 7-8ms to 1.1.1.1 while running test.
View attachment 26881


Now I know why the new ping spikes while gaming with not load on connection at all... Maybe it's related to running HW acceleration off and cpu not being as effective or even firmware issue?

Edit: Just did a factory reset using the WPS+power buttons. Simple config with only SSID, enable SSH and jffs scripts, installed cake. Problem remains.

Do the same thing and ping your next hop router. 1.1.1.1 may be too far away and on the other side of a congested link.

 

[Audionut]

Do you still get the same random ping spike during initial load with fq-codel?

If not, and without the random ping spike at idle, it's probably best just to stick with fq-codel and the lower throughput (read: I can't help any further ).

 

[askan7]

Do you still get the same random ping spike during initial load with fq-codel?

If not, and without the random ping spike at idle, it's probably best just to stick with fq-codel and the lower throughput (read: I can't help any further ).

During initial load I get the same ping spikes with cake or fq-codel. Maybe caused by my isp router? It's in bridge mode, I get public IP on asus.
On idle just using ping -t to first hop or 1.1.1.1 is stable with only 1ms jitter. But as soon as I start meter.net/ping-test ping spikes only with cake, if cake is off, or if I'm using fq-codel I have no such ping spikes.
The same happens with certain games, they cause the same ping spikes as that test.

I really like how cake balances load much better than the Asus QOS but ping spikes while gaming, with no load on connection, bother me.

 

[Audionut]

I would trust a command line ping or continuous ping from the ASUS (/Main_Analysis_Content.asp) more then some random browser based test every single day of the week.

Granted, the differing results with the different bandwidth shapers is interesting, and you seem to have subjective evidence to back up the results of that browser based test, but as an old bastard that's spent far to much time in his earlier years chasing minute things, I would be questioning how noticeable the spikes are to actual game play. Try not to look to hard at it. Don't try and find a problem with the ping spikes with cake while gaming. Just game away and go by your gut feeling afterwards.

Then weighing up the pros and cons. Only you can do that, because it's your connection.

Back in my younger days, I would read the instructions 4 or 5 times, spend countless hours google searching each option trying to understand the thing, then just start playing with the switches until the thing did what I wanted or I gave up.

 

[askan7]

I would trust a command line ping or continuous ping from the ASUS (/Main_Analysis_Content.asp) more then some random browser based test every single day of the week.

Granted, the differing results with the different bandwidth shapers is interesting, and you seem to have subjective evidence to back up the results of that browser based test, but as an old bastard that's spent far to much time in his earlier years chasing minute things, I would be questioning how noticeable the spikes are to actual game play. Try not to look to hard at it. Don't try and find a problem with the ping spikes with cake while gaming. Just game away and go by your gut feeling afterwards.

Then weighing up the pros and cons. Only you can do that, because it's your connection.

Back in my younger days, I would read the instructions 4 or 5 times, spend countless hours google searching each option trying to understand the thing, then just start playing with the switches until the thing did what I wanted or I gave up.

You didn't get it. That browser test causes the command line ping spike at the same time. But some games do the exact same... That's why I use the browser test, easy to reproduce. I even had a second command line to the router (192.168.1.1) and that is stable <1ms.

 

[Morris]

After reading the recent observations on raw and unlimited download speeds I have revisited and adjusted my cake settings. My notional connection is 80/20, VDSL/PPPOE, I am in a metropolitan area and all my tests are subject to the usual variances to be expected from external noise, so YMMV.

My previous settings were

[1] --> Download Speed | [78 Mbit]
[2] --> Upload Speed | [19 Mbit]
[3] --> Queue Priority | [besteffort]
[4] --> Extra Download Options | [ack-filter pppoe-ptm diffserv4 rtt 80ms]
[5] --> Extra Upload Options | [ack-filter pppoe-ptm diffserv4 rtt 80ms]

They are now

[1] --> Download Speed | [77 Mbit]
[2] --> Upload Speed | [19 Mbit]
[3] --> Queue Priority | [besteffort]
[4] --> Extra Download Options | [raw diffserv4 rtt 80ms]
[5] --> Extra Upload Options | [raw diffserv4 rtt 80ms]

Observations

Adding raw removes the need to have either ack-filter or pppoe-ptm and provides slightly better throughput, latency and jitter. Adding either or both of them back (e.g. ack-filter raw pppoe-ptm diffserv4 rtt 80ms) gives a worse result.

Setting Speed to unlimited results in a very marginal increase in throughput at the expense of a substantial increase in both latency and jitter.

While reducing rtt marginally improves latency and jitter on for devices routed directly, after I go below 80ms, the throughput drops perceptibly on devices routed through a VPN on the router.

I tested using spdMerlin and betterspeedtest run on the router and DSL reports run on devices going direct and via VPN.

I just got a chance to read back. Your configuration is has conflicting Queue Priority. Dffserv4 is a queue priority and dose not belong in the extra upload options. Furthermore, if you want cake, then don't use diffserv4 as it's classified QOS.

A lot of people are over tuning and this is contrary to the principles of Cake. Proper starting point:

[1] --> Download Speed | enter 90% of measured speed with no QOS
[2] --> Upload Speed | [ enter 50% of measured speed with no QOS
[3] --> Queue Priority | [besteffort] Don't use anything else!
[4] --> Extra Download Options | Ethernet (or what ever your connection type is from the table. Don't play with other options. There not for the typical home NAT router
[5] --> Extra Upload Options | Ethernet (or what ever your connection type is from the table. Don't play with other options. There not for the typical home NAT router

Tuning must be done on a quiet link (no other users)

Download Tuning
1) Run bufferbloat test.
2) If good increase download speed by 1 and test till good. Then decrease by 1 and preceded to Upload Tuning
3) if bad decrease download speed by 1 and test till good. next proceed to Upload Tuning

Upload Tuning
2) If good increase upload speed by 1 and test till good. Then decrease by 1 and preceded to Done Tuning
3) if bad decrease upload speed by 1 and test till good. next proceed to Done Tuning

Done Tuning
The above procedure works best on a link to the next hop router that is not shared by other users. The is ISP configuration dependent. If you experience variation during the day you may have to reduce your upload and/or download speed a bit to cover this. This is in the spirit of cake that works by giving up a little bit of bandwidth to avoid congestion that results in jitter.

If your link is over subscribed by real time applications such as video streaming, video conferencing, VOIP, QOS is not the going to help till you address the over subscription by increasing bandwidth or decreasing quality settings for the streaming applications and/or video conferencing.

Adjusting the other parameters will not help in a significant way. The best tuning is done by adjusting the bandwidth parameters. This is more important that the type of connection option.

Have fun,

Morris

 

[askan7]

If I leave ping -t while I game it's the same result as meter.com test. Seems like many small packets, like playing apex legends while shooting bots with fast automatic guns causes cake to buffer. No load on network at all, only 1 device and no background services.

Ping to first hop and router while moving and shooting around on Apex Legends.
Cake on:


Cake off:


Can anyone reproduce this? meter.com/ping-test causes the same buffer as certain games as it sends a lot of small packets.
Cake still a lot better with load....

 

[heysoundude]

During initial load I get the same ping spikes with cake or fq-codel. Maybe caused by my isp router? It's in bridge mode, I get public IP on asus.
On idle just using ping -t to first hop or 1.1.1.1 is stable with only 1ms jitter. But as soon as I start meter.net/ping-test ping spikes only with cake, if cake is off, or if I'm using fq-codel I have no such ping spikes.
The same happens with certain games, they cause the same ping spikes as that test.

I really like how cake balances load much better than the Asus QOS but ping spikes while gaming, with no load on connection, bother me.

never mind pinging Cloudflare's server - what's the ping to your ISP's servers (always the first hop)? and to the gaming server you connect to? is the gaming server returning an IPv6 address?
If so, is your connection (especially the computer that you game on) configured to use IPv6 and thus bypass NAT, possibly removing a hint of latency?

CakeQOS - CakeQOS-Merlin

Interesting unintended consequences for people (ab)using betterspeedtest.sh. https://lists.bufferbloat.net/pipermail/bloat/2020-October/015941.html LOL - how many of the 470-odd blacklisted IPs could be tied to people from these parts would you reckon? (I have no shame using @Jack Yaz 's...

www.snbforums.com

I agree completely w/ @Morris ^ when he says people are overtuning...raw, besteffort, unlimited (has brought me the best results) and rtt metro. (Read the ManPage for cake)
In combination with unbound, I am seeing much improvement on my 50/10 vDSL, and I don't doubt you can as well.

Quiet LAN - "but that's when I game!!!" the game will be there tomorrow. fix your network: you'll play better/get better results/have more fun/win more