Adaptive CAKE QoS

[dave14305]

Ive got to do some more testing on the AX. noticed a weird thing playing with your script this morning. If FC was enabled , speedtests were only showing around 15MBit, the router speedtest showed the full 25. But when I disabled FC. speedtests jumped right up. Wth would cause that with the rules. Normal cake is fine. I swear this AX is a bastard router.
I hooked up a AC86u to the modem also I can play around with. Curious where your able to take this.

The router speedtest won't be limited on the download, because there is no limiting on the incoming WAN traffic; only when it leaves the LAN interface does it get limited.

You've had lots of horror stories with fc and the AX86U. Wait for the next ASUS GPL and see how it goes.

 

[Wade Coxon]

Have installed and verified that fast.com results are being placed in the video tin.

Ran bufferbloat tests on both Waveform and DSL reports.
I am getting B results with a pretty nasty 20-40ms bufferbloat on Waveform which is placed into the besteffort tin.
I am getting A+ results with 5-10ms bufferbloat on DSLReports, which is quite counterintuitively being placed into the bulk tin.

So I will keep running with this for now, but that is a very odd result.

 

[RMerlin]

I hear you, but I’m not yet seeing any ill effects of this experiment. The router thinks Adaptive QoS is running, so it’s doing the usual marking. How would fq_codel have kept track of flows? Was it a modified version with API access?

I’m not doing anything in iptables with the traffic, where I know HW acceleration could cause issues.

See the post just above.

fq_codel isn't doing all its own flow tracking internally unlike Cake. That's why fq_codel requires you to configure all the various queues, and these queues are used as per marking. With Cake, the goal was to work without requiring users configuring separate queues, meaning flow management is entirely done internally.

 

[Morris]

See the post just above.

fq_codel isn't doing all its own flow tracking internally unlike Cake. That's why fq_codel requires you to configure all the various queues, and these queues are used as per marking. With Cake, the goal was to work without requiring users configuring separate queues, meaning flow management is entirely done internally.

I agree with this statement Eric. It makes me wonder why you did not chose besteffort in both directions as the default. Besteffort in both directions works great. Possibly would avoid all the tinkering that's getting many bad results.

 

[dave14305]

See the post just above.

Our friend the Albino Man posts some wacky things sometimes. I wouldn't point to him as empirical evidence (no offense @albinoman887!).

fq_codel isn't doing all its own flow tracking internally unlike Cake. That's why fq_codel requires you to configure all the various queues, and these queues are used as per marking. With Cake, the goal was to work without requiring users configuring separate queues, meaning flow management is entirely done internally.

You've kinda lost me. A single fq_codel qdisc manages the tracking of flows just like CAKE does based on the 5-tuple of proto,srcIP,srcPrt,dstIP,dstPrt. Both have to be able to hash flows into one of many internal queues. HTB does the shaping and "tinning" in htb+fq_codel. fq_codel does the fairness scheduling. CAKE combines it all in one qdisc.

I just haven't seen any technical explanation of why this cannot work compared to htb+fq_codel. I'm not declaring it a success, but I want to understand it more than the general surrender of, "it's the Broadcom/Trend Micro boogey-man's fault." I may shout that exact phrase in the end, but for now I will persevere. Sometimes you have to break some sh*t to see what's possible.

EDIT: I used the s-h-i-t word in my post but the forum was instead displaying it as "shirt". I went to edit the post and there was my unadulterated 4-letter word. But it was still being displayed as "shirt". So I edited it to be sh*t since shirt made no sense.

 

[RMerlin]

You've kinda lost me. A single fq_codel qdisc manages the tracking of flows just like CAKE does based on the 5-tuple of proto,srcIP,srcPrt,dstIP,dstPrt. Both have to be able to hash flows into one of many internal queues. HTB does the shaping and "tinning" in htb+fq_codel. fq_codel does the fairness scheduling. CAKE combines it all in one qdisc.

I'm not an expert, so I can't explain the details on how it works. But both an Asus engineer and Dave Taht separately told me that with hardware acceleration enabled, flow handling won't work properly.

 

[dave14305]

I'm not an expert, so I can't explain the details on how it works. But both an Asus engineer and Dave Taht separately told me that with hardware acceleration enabled, flow handling won't work properly.

OK, now I've got some evidence to prove my hypothesis wrong.

This is the list of cake flows (classes) during a Speedtest.net test, using normal CAKE configured by the firmware without addons. HW Acceleration is disabled. There are many flows for all the concurrent connections being used by the test:

Here's the same speedtest run with Adaptive QoS enabled with HW acceleration and the flexcake script in this thread running CAKE on br0. Only 1 flow seems to persist throughout the test. Some flash in briefly, but not like in the first test where they were present throughout.

I'm also starting to doubt if fq_codel has ever worked 100% with Adaptive QoS.

 

[Wade Coxon]

See the post just above.

@dave14305 Any chance that my uneven results with the two bufferbloat tests is a result of this complication.
I am running the RT-AC86U like yourself, so I don't have any controls for HW acceleration (both showing enabled in the web interface though).

 

[dave14305]

@dave14305 Any chance that my uneven results with the two bufferbloat tests is a result of this complication.
I am running the RT-AC86U like yourself, so I don't have any controls for HW acceleration (both showing enabled in the web interface though).

You can disable it over SSH, but that would have defeated the purpose of trying to run CAKE with HW acceleration enabled.

runner disable
fc disable
fc flush
---
runner enable
fc enable
fc flush

 

[Kingp1n]

Here's the same speedtest run with Adaptive QoS enabled with HW acceleration and the flexcake script in this thread running CAKE on br0. Only 1 flow seems to persist throughout the test. Some flash in briefly, but not like in the first test where they were present throughout.

View attachment 36695

I'm also starting to doubt if fq_codel has ever worked 100% with Adaptive QoS.

This is probably unrelated to what you're currently testing but I can honestly say, my bufferbloat definitely is affected when I use 'sfq' versus fq_codel. At least for me, fq_codel is definitely beneficial when using A. QoS (FlexQoS). Again, sorry is this not related but just thought I would share my results.

 

[dave14305]

This is probably unrelated to what you're currently testing but I can honestly say, my bufferbloat definitely is affected when I use 'sfq' versus fq_codel. At least for me, fq_codel is definitely beneficial when using A. QoS (FlexQoS). Again, sorry is this not related but just thought I would share my results.

It’s a valid observation, but all this talk about HW acceleration has me challenging all assumptions now.

 

[Wade Coxon]

It’s a valid observation, but all this talk about HW acceleration has me challenging all assumptions now.

You can disable it over SSH, but that would have defeated the purpose of trying to run CAKE with HW acceleration enabled.

runner disable
fc disable
fc flush
---
runner enable
fc enable
fc flush

Thanks for that.

I can confirm that disabling hw acceleration absolutely fixes the bufferbloat on the Waveform test. 0ms bufferbloat and A+ result after disabling.

Maybe that result was because of the way you handle the besteffort tin?

 

[dave14305]

This ends the experiment. Please resume using whatever QoS setup you were using before. Thanks for your participation!

 

[Wade Coxon]

This ends the experiment. Please resume using whatever QoS setup you were using before. Thanks for your participation!

You don't think this arrangement still has value?
After all, we have Trend's DSCP marking and software driven cake, as opposed to just software cake-only.

Plus, I was still getting good bufferbloat results on DSLreports when it's traffic was being sent to bulk.

 

[dave14305]

You don't think this arrangement still has value?
After all, we have Trend's DSCP marking and software driven cake, as opposed to just software cake-only.

Plus, I was still getting good bufferbloat results on DSLreports when it's traffic was being sent to bulk.

It has merit for tinning traffic, but if we lose proper flow fairness in the process, that's also half the bufferbloat battle -- getting those small packets through in the face of larger flows.

Personally, I bit off a lot more than I care to manage these past few days. I was very happy with finally adding skbedit/ipset support to CakeQOS-Merlin 2.20 beta. Then I distracted myself with this experiment and then CAKE on br0. I've ended up scatter-brained as a result. I usually end up frustrated anyhow when trying to do anything that touches the Broadcom/Trend black box.

Maybe I'll revisit this again in the dark days of winter. But for now I want to refocus on the primary QoS addons.

Notes to my future self:

• Does the bwdpi engine still work properly when runner/fc is disabled?
• What issues are there trying to do QoS on a bridge interface? Why does OpenWrt discourage it?
• Morris always shows up to scold anyone who uses anything but besteffort!

 

[Wade Coxon]

Maybe I'll revisit this again in the dark days of winter. But for now I want to refocus on the primary QoS addons.

Fair enough

I might continue to use this variant for a while yet though. Now that hw acceleration is off, I am seeing great results on the tests, and the tinning appears to be working and categorising flows accurately. I'll take the "free" DSCP from Trend as a bonus .

(And that satisfies Morris's "keep it simple" principle since I will leave all the messy tagging to the third party binary blob .)

My connection is slow enough that hw acceleration is never going to provide a benefit to me .

 

[rlj2]

Fair enough

I might continue to use this variant for a while yet though. Now that hw acceleration is off, I am seeing great results on the tests, and the tinning appears to be working and categorising flows accurately. I'll take the "free" DSCP from Trend as a bonus .

(And that satisfies Morris's "keep it simple" principle since I will leave all the messy tagging to the third party binary blob .)

My connection is slow enough that hw acceleration is never going to provide a benefit to me .

Wade, any chance you have a AX router?

 

[Wade Coxon]

Wade, any chance you have a AX router?

No, the RT-AC86U.

Ahh, I did just remember one downside to this.

Any time the Trend engine updates, cake is disabled again (and likely hw accel would turn on again, though I haven't checked that).
I need to set up a cron job or something to make sure this all remains activated.

 

[rlj2]

No, the RT-AC86U.

Ahh, I did just remember one downside to this.

Any time the Trend engine updates, cake is disabled again (and likely hw accel would turn on again, though I haven't checked that).
I need to set up a cron job or something to make sure this all remains activated.

gotcha on the ac86. Ive never seen cake disabled over trend updates. but try "fc disable" in /jffs/scripts/firewall-start

 

[Kingp1n]

Fair enough

I might continue to use this variant for a while yet though. Now that hw acceleration is off, I am seeing great results on the tests, and the tinning appears to be working and categorising flows accurately. I'll take the "free" DSCP from Trend as a bonus .

(And that satisfies Morris's "keep it simple" principle since I will leave all the messy tagging to the third party binary blob .)

My connection is slow enough that hw acceleration is never going to provide a benefit to me .

What are your speeds? I might test this just to see how it does in my environment!