Interaction between Cake-QOS and OpenVPN

[Lynx]

I spent some time trying to figure out why I could not control the buffer bloat on my connection using the default cake implementation in 386.2_6 until I identified that switching off my VPN (OpenVPN: NordVPN) removes the buffer bloat problem.
What is the interaction between cake and OpenVPN? Do the eth0 and ifb4eth0 interfaces still make sense when using a VPN?

admin@RT-AX86U-4168:/tmp/home/root# tc qdisc | grep cake
qdisc cake 8001: dev eth0 root refcnt 2 bandwidth 5120Kbit diffserv3 dual-srchost nat nowash no-ack-filter split-gso rtt 100ms noatm overhead 0
qdisc cake 8002: dev ifb4eth0 root refcnt 2 bandwidth 5120Kbit besteffort dual-dsthost nat wash ingress no-ack-filter split-gso rtt 100ms noatm overhead 0

Otherwise how can I get cake to properly work with NordVPN?
What settings do I need to configure?

 

[dave14305]

At the point where the encrypted tunnel leaves the router on eth0, it is just seen as a single flow to (likely) a single remote IP and port. Everything in the tunnel will look the same to CAKE. If any other traffic is allowed outside the VPN tunnel, it will get better fairness treatment than traffic within the tunnel.

 

[Lynx]

Should the cake interface be altered to tun11? How would I do that using stock 386.2_6 + stock cake that Merlin put into the firmware?

What is the solution here to get cake + OpenVPN to work together?

With default cake 5/5 I get A+ buffer bloat.

With default cake 5/5 + NordVPN (OpenVPN / UDP), I get bad download, albeit upload looks OK:

Bufferbloat and Internet Speed Test - Waveform

View the full results, and test your own bufferbloat

www.waveform.com

 

[dave14305]

You could certainly experiment with putting cake on tun11 for upload and br0for download. You’d want to stop the firmware cake with /tmp/qos stop

The idea would be to shape the traffic before it enters the tunnel (tun11) from the LAN and shape it again after it exits the tunnel heading to the LAN (br0).

No idea if/how it will work.

 

[Lynx]

This feels like a fairly basic use case that should be configurable from GUI, no? I mean how uncommon is it to want to use QOS and a VPN?

Would the correct way to do this be to set up the scripts as outlined by RMerlin here:

[Cake test] Cake experiments for 386.2 Beta

Hi folks, If you are currently testing Cake on 386.2 Beta, could you guys test the following customization, and see if there is any measurable improvement or issues? Make sure you have custom JFFS scripts enabled (Administration -> System). Then, create /jffs/configs/cake-qos.conf.add with...

www.snbforums.com

And specify tun11 for upload and br0 for download? Wouldn't that apply QOS to LAN stuff?

Getting QOS on an unencrypted point seems to make sense right?

 

[dave14305]

This feels like a fairly basic use case that should be configurable from GUI, no? I mean how uncommon is it to want to use QOS and a VPN?

Not everyone wants to send ALL traffic through a VPN. If it’s not all traffic, then what do you do about shaping the traffic that bypasses the VPN? It gets complicated quickly.

Would the correct way to do this be to set up the scripts as outlined by RMerlin here:

[Cake test] Cake experiments for 386.2 Beta

Hi folks, If you are currently testing Cake on 386.2 Beta, could you guys test the following customization, and see if there is any measurable improvement or issues? Make sure you have custom JFFS scripts enabled (Administration -> System). Then, create /jffs/configs/cake-qos.conf.add with...

www.snbforums.com

And specify tun11 for upload and br0 for download?

I was thinking you should just type the tc qdisc replace commands directly over SSH as a test to see how it behaves. If it achieves your desired results, then worry about putting it in a script later.

…br0 for download? Wouldn't that apply QOS to LAN stuff?

Getting QOS on an unencrypted point seems to make sense right?

Maybe. That’s why I labeled this an “experiment”.

 

[Lynx]

Thanks a lot for your help here.

So would it be fair to say that at the moment QOS with VPN just isn't handled? Or am I missing something? I just assumed enabling QOS and VPN would be handled. But it seems then for everyone using a VPN and expecting QOS to work properly it actually will not.

Doesn't it make sense to apply QOS in the manner we are contemplating here for VPN traffic? Otherwise QOS is operating in a very handicapped manner right?

Feels to me like many will want to use QOS + VPN?

I will try to test as soon as I can. Sorry to be so thick here, but what would the correct replace commands be to apply based on my present settings:

admin@RT-AX86U-4168:/tmp/home/root# tc qdisc | grep cake
qdisc cake 8001: dev eth0 root refcnt 2 bandwidth 5120Kbit diffserv3 dual-srchost nat nowash no-ack-filter split-gso rtt 100ms noatm overhead 0
qdisc cake 8002: dev ifb4eth0 root refcnt 2 bandwidth 5120Kbit besteffort dual-dsthost nat wash ingress no-ack-filter split-gso rtt 100ms noatm overhead 0

 

[dave14305]

Just the basics:

/tmp/qos stop
tc qdisc replace dev tun11 root cake bandwidth 5Mbit
tc qdisc replace dev br0 root cake bandwidth 5Mbit

When not using on the WAN interface, your existing settings don’t mean much. Can’t help much more since I’m not running any VPN and I’m not even running Merlin at the moment, so no Cake.

 

[Lynx]

Thanks. I think I read somewhere that policy based routing uses a secondary routing table. Would that affect things here?

 

[Lynx]

I think this thread suggests that the default CAKE implementation in Asus-Merlin should be changed to allow proper handling of the use of CAKE with a VPN. Perhaps the interfaces to which QoS is applied should be user configurable via the GUI?

Just the basics:

/tmp/qos stop
tc qdisc replace dev tun11 root cake bandwidth 5Mbit
tc qdisc replace dev br0 root cake bandwidth 5Mbit

When not using on the WAN interface, your existing settings don’t mean much. Can’t help much more since I’m not running any VPN and I’m not even running Merlin at the moment, so no Cake.

Thanks again for this dave14305. Having finally tested, and as expected, I see a drastic improvement using 'tun11' and 'br0' - CAKE 5/5 now works properly with the VPN, see e.g.:

Bufferbloat and Internet Speed Test - Waveform

View the full results, and test your own bufferbloat

www.waveform.com

I also tried with CAKE 35/20 otherwise with the same settings and things still look great:

Bufferbloat and Internet Speed Test - Waveform

View the full results, and test your own bufferbloat

www.waveform.com

Compare the above application of CAKE 5/5 on interfaces 'tun11' and 'br0' with the broken application of CAKE 5/5 on 'eth0', as is the default in 386.2_6, when using VPN:

Bufferbloat and Internet Speed Test - Waveform

View the full results, and test your own bufferbloat

www.waveform.com

I think something along these lines, i.e. CAKE on 'tun11' and 'br0' should be the default for anyone wanting to use CAKE + OpenVPN.
On IRC RMerlin indicated that this made no sense, but it makes a lot of sense to me, and it makes a lot of sense to many others such as the OpenWRT guys, e.g. see here:

OpenWrt Forum Archive

I think a reason for why this works so much better was provided in the link above by 'moeller0'. He seems particularly active in respect of SQM on OpenWRT. He writes:

Okay, I guess I am a bit daft today, but your intuition seems about right, before all SQM saw was the outer UDP packets all directed to the same IP address and port, so effectively you have been running with pure codel as the fq bit in fq_codel, the fair queueing part works by spreading out the flows by src/dest IP and port/protocol information all packets have been sharing one queue. Now with SQM active on an interface where it sees the real IP headers things work better. I believe you will need to make sure that you set the correct overhead to account for the added IP/UDP headers (so probably 20+20+14 bytes, assuming IPv4), might be worthwhile to play around/test a few different overhead values and see whether it affects bufferbloat...

Best Regards
M.

Also see this link here, which discusses the correct interface(s) to use and the determination of overhead:

[Solved] SQM with a VPN: What interface name? What packet overhead?

I currently use both SQM and an OpenVPN client on a 19.07.2 install of OpenWRT. Should I use pppoe-wan or tun0? Any pros or cons maybe? What per packet overhead should I add? All my devices use the VPN client, with no exception. Otherwise my network interfaces set up is pretty standard with...

forum.openwrt.org

In terms of overhead calculation, the above link suggests:

1. tcpdump -vpni tun0, ping 8.8.8.8 while tcpdump is running, see the resulting packet size.
2. tcpdump -vpni pppoe-wan, ping 8.8.8.8 (note that you will see encrypted openvpn packet

Would there be an alternative way to carry this test out to determine the correct overhead without me having to install tcpdump?
What would be the most elegant way to make this my new default with the stock 386.2_6 CAKE implementation?

 

[RMerlin]

I think something along these lines, i.e. CAKE on 'tun11' and 'br0' should be the default for anyone wanting to use CAKE + OpenVPN.
On IRC RMerlin indicated that this made no sense, but it makes a lot of sense to me,

One of the goals of Cake is to ensure that buffers don't get filled up, causing additional latency (the so-called bufferbloat effect). If you handle that inside the tunnel, the VPN packet gets overhead added to it before being sent over the Internet. That overhead isn't taken into account, so you will end up with bufferbloat issues. You HAVE to handle that at the WAN level.

A tunnel isn't a magical route that bypasses your ISP. You are still effectively sending data to your ISP, through the usual route.

And running bufferbloat tests within a tunnel is unreliable, because you aren't just dealing with bufferbloat here, you are dealing with VPN latency, VPN providers are generally oversubscribed. Also, Cake requires you to provide the maximum expected throughput numbers to properly manage traffic. That maximum throughput is generally impossible to determine with a VPN provider, as it will constantly vary based on their server load.

Bottom line is, if latency is important, then DON'T use a VPN. Because you cannot accurately manage VPN traffic.

 

[Lynx]

RMerlin I would urge you to reconsider your position here. At least let me try, OK?

If you handle that inside the tunnel, the VPN packet gets overhead added to it before being sent over the Internet. That overhead isn't taken into account, so you will end up with bufferbloat issues. You HAVE to handle that at the WAN level.

The overhead can be readily taken into account. The links I provided above provide the means to do so if desired. What's the problem?

And running bufferbloat tests within a tunnel is unreliable, because you aren't just dealing with bufferbloat here, you are dealing with VPN latency, VPN providers are generally oversubscribed. Also, Cake requires you to provide the maximum expected throughput numbers to properly manage traffic. That maximum throughput is generally impossible to determine with a VPN provider, as it will constantly vary based on their server load.

Bottom line is, if latency is important, then DON'T use a VPN. Because you cannot accurately manage VPN traffic.

It is as if you are implying that there is never a valid use case for combining QoS with a VPN.
You argue applying CAKE in the manner suggested in this thread makes no sense. But applying CAKE in a VPN context on 'eth0' does not work. It does not work on a theoretical level (CAKE is operating blind) and it demonstrably does not work well on a practical level (CAKE demonstrably struggles).
Applying CAKE in a VPN context in the manner suggested in this thread makes more sense than the default setup allows for at the moment. CAKE can actually see what it is doing! It demonstrably works more effectively. Throughput issues can be managed. Minimal internet searching reveals numerous examples (on OpenWRT, etc.) of traffic shaping in this way - and positive reports in respect of performance.
I realise you are on a super fast connection and QoS may not be a big issue, but here is my story. I have a 4G LTE connection and wanted to buy a router to simultaneously apply a VPN (to circumvent ISP throttling and restrictions) and QoS (to deal with huge bufferbloat on saturating my limited bandwidth). I researched routers and came to the view that with your excellent work the ASUS router with Asus-Merlin firmware was the way to go! A good compromise between 'it just works' and flexibility. It has been exciting getting everything up and running so far. I am really hopeful that with your blessing a good solution can be managed. Surely this is not an edge case. Surely there are many other users who will want to use QoS with a VPN in the manner described in this thread?
Are you at all persuaded? I've given this my best shot.
In any case, for those of us who really want CAKE and a VPN to work well together, please can you consider at least providing an indication of the most elegant way to change the interface from 'eth0' to 'br0'/'tun11', if not make the interfaces to which CAKE is applied user configurable in the GUI or otherwise?

 

[RMerlin]

It is as if you are implying that there is never a valid use case for combining QoS with a VPN.

No. What I'm saying is changing the QoS configuration to handle the VPN traffic instead of the WAN traffic won't work.

VPN traffic would need to have its own QoS in addition to the WAN QoS. Feel free to add your own traffic queues to the tun interface, but be prepared to be very disappointed by the results. Using a public VPN providers will still leave you with the problem that speed cannot be reliably predicated, making it impossible to finetune the QoS configuration. And no amount of QoS can compensate for the latency issues associated with using a public VPN provider.

Also, prioritizing packets by size within the VPN won't do much good when ultimately, you are still sending the same encapsulated packets on your WAN interface, with no size prioritization.

 

[Lynx]

Feel free to add your own traffic queues to the tun interface, but be prepared to be very disappointed by the results.

But I am already happy with the results in just applying to 'tun11' and 'br0' even without yet properly taking into account the overhead. My bufferbloat issues have gone, and this mirrors the experiences of the OpenWRT guys.

I have seen examples of a kind of two stage shaping (global and per tunnel) and I wonder what the benefits might be in further applying shaping to the already ordered encapsulated packets? Would the idea be to have per tunnel shaping set with unlimited upload and download and then global shaping with the throttled upload and download? Would the shaping still work then? I thought shaping needed some sense of throttling constraints to work effectively?

Tick box in VPN Director? How about it?

In any case, please can you provide an indication of how the interface change (or addition!) might be most elegantly achieved in 386.2_6?

I am ready and eager to test any suggestions!

 

[dave14305]

How did you leap from testing an experiment to lobbying for GUI changes? Are those your results in post #10 under a different username? Confusing.

I don’t trust the results that show 0ms added latency for this experimental setup. Is that consistently 0ms?

What are your Cake stats?

tc -s qdisc show dev tun11
tc -s qdisc show dev br0

Lower your expectations about this scenario of yours being accommodated in the GUI. The GUI handles only the basic case for the majority of users. You will need to achieve what you want through the available custom scripting hooks (qos-start and cake.conf.add).

VPN Director teaches us that some people run multiple VPN clients, so this test does not scale well. I wonder if cake is the wrong qdisc in this scenario. Maybe fq_codel on the tun interfaces (for flow fairness) and a simple Token Bucket filter on the WAN interface (for shaping).

But again this gets farther off the beaten path and less likely for broad adoption.

 

[Lynx]

dave14305 you are right about jumping the gun - my enthusiasm got the better of me. I have been reading about SQM implementations in OpenWRT involving shaping along the lines discussed in this thread and configuring appropriate interface(s) in the context of VPN use.

For Asus-Merlin I have struggled to find the right information on just changing the interface for QoS application - is this documented anywhere? The only thing I could find was the post from RMerlin involving scripting as part of testing CAKE for beta testers and I don't even know if that is still applicable.

I am open minded - as you say CAKE may not be the best for me. I have a 4G LTE connection that gives around 50-60 download and 20-25 upload with a latency of around 55ms and unloaded jitter about 5ms. Impact of VPN (NordVPN) seems pretty minimal from the testing I have done, which I presume is because of my relatively poor connection. I need to use a VPN because my ISP (well, mobile phone provider) heavily and consistently throttles to 10Mbit/s on port 443 (VPN circumvents this) and I am keen to keep Microsoft Teams and Zoom calls working as well as possible despite background downloads. Since I am especially bothered about Teams and Zoom the adaptive QoS classification may make sense. I realise that my connection presents rather a challenge in terms of QoS - the bandwidth admittedly varies somewhat (especially on download), albeit the unloaded latency is pretty consistent.

So far all I have done is implement CAKE in accordance with your suggestion:

/tmp/qos stop
tc qdisc replace dev tun11 root cake bandwidth 5Mbit
tc qdisc replace dev br0 root cake bandwidth 5Mbit

Except that I have changed this to 30/20 (as a very crude initial increase from the 5/5 level). Overhead is not being properly taken into account, but I understand that has a pretty minimal effect in terms of the bandwidth calculation used in CAKE anyway. The overhead stuff seems a little academic given that most users just converge upon limits by trial and error.

What I can say is that the application of CAKE in the way you have suggested above seems to work in terms of my latency not spiking upon downloads, whereas on 'eth0' it did not appear to work. And I understand that is because CAKE is operating blind.

As you have asked for dave14305, here is the output from the commands:

admin@RT-AX86U-4168:/tmp/home/root# tc -s qdisc show dev tun11

qdisc cake 8003: root refcnt 2 bandwidth 20Mbit diffserv3 triple-isolate nonat nowash no-ack-filter split-gso rtt 100ms raw overhead 0
Sent 1010237661 bytes 2063238 pkt (dropped 3461, overlimits 2193762 requeues 0)
backlog 0b 0p requeues 0
memory used: 668160b of 4Mb
capacity estimate: 5Mbit
min/max network layer size:           29 /    1408
min/max overhead-adjusted size:       29 /    1408
average network hdr offset:            0

                   Bulk  Best Effort        Voice
  thresh       1250Kbit       20Mbit        5Mbit
  target         14.4ms          5ms          5ms
  interval        109ms        100ms        100ms
  pk_delay          0us        770us       1.11ms
  av_delay          0us         35us        159us
  sp_delay          0us          3us          4us
  backlog            0b           0b           0b
  pkts                0      2066202          497
  bytes               0   1014820004        77366
  way_inds            0        18588           31
  way_miss            0        20576           58
  way_cols            0            0            0
  drops               0         3461            0
  marks               0            1            0
  ack_drop            0            0            0
  sp_flows            0            0            2
  bk_flows            0            1            0
  un_flows            0            0            0
  max_len             0         1408          576
  quantum           300          610          300

admin@RT-AX86U-4168:/tmp/home/root# tc -s qdisc show dev br0

qdisc cake 8004: root refcnt 2 bandwidth 30Mbit diffserv3 triple-isolate nonat nowash no-ack-filter split-gso rtt 100ms raw overhead 0
Sent 2765966059 bytes 2873292 pkt (dropped 98902, overlimits 3814963 requeues 0)
backlog 0b 0p requeues 0
memory used: 1431040b of 4Mb
capacity estimate: 5Mbit
min/max network layer size:           42 /    1514
min/max overhead-adjusted size:       42 /    1514
average network hdr offset:           14

                   Bulk  Best Effort        Voice
  thresh       1875Kbit       30Mbit     7500Kbit
  target         9.69ms          5ms          5ms
  interval        105ms        100ms        100ms
  pk_delay        537us       2.41ms        227us
  av_delay         30us        229us         25us
  sp_delay          1us          0us          2us
  backlog            0b           0b           0b
  pkts             1400      2864113       106681
  bytes          167097   2792627527    109345457
  way_inds            0        47220            0
  way_miss           55        24824           49
  way_cols            0            0            0
  drops               0        98882           20
  marks               0           10            0
  ack_drop            0            0            0
  sp_flows            0            1            0
  bk_flows            0            1            1
  un_flows            0            0            0
  max_len          1379         7570        34010
  quantum           300          915          300

I also repeated above for 5/5:
admin@RT-AX86U-4168:/tmp/home/root# tc -s qdisc show dev tun11

qdisc cake 8009: root refcnt 2 bandwidth 5Mbit diffserv3 triple-isolate nonat nowash no-ack-filter split-gso rtt 100ms raw overhead 0
Sent 407110198 bytes 893164 pkt (dropped 2163, overlimits 1083153 requeues 0)
backlog 0b 0p requeues 0
memory used: 830Kb of 4Mb
capacity estimate: 20Mbit
min/max network layer size:           40 /    1378
min/max overhead-adjusted size:       40 /    1378
average network hdr offset:            0

                   Bulk  Best Effort        Voice
  thresh      312496bit        5Mbit     1250Kbit
  target         57.6ms          5ms       14.4ms
  interval        153ms        100ms        109ms
  pk_delay          0us       1.84ms       1.34ms
  av_delay          0us        112us        158us
  sp_delay          0us          1us         10us
  backlog            0b           0b           0b
  pkts                0       895183          144
  bytes               0    410035450        12425
  way_inds            0        10308            0
  way_miss            0         1234           12
  way_cols            0            0            0
  drops               0         2163            0
  marks               0            3            0
  ack_drop            0            0            0
  sp_flows            0            1            1
  bk_flows            0            1            0
  un_flows            0            0            0
  max_len             0         1378          576
  quantum           300          300          300

admin@RT-AX86U-4168:/tmp/home/root# tc -s qdisc show dev br0

qdisc cake 8004: root refcnt 2 bandwidth 5Mbit diffserv3 triple-isolate nonat nowash no-ack-filter split-gso rtt 100ms raw overhead 0
Sent 4082401638 bytes 4095713 pkt (dropped 146691, overlimits 5464741 requeues 0)
backlog 0b 0p requeues 0
memory used: 1431040b of 4Mb
capacity estimate: 5Mbit
min/max network layer size:           42 /    1514
min/max overhead-adjusted size:       42 /    1514
average network hdr offset:           14

                   Bulk  Best Effort        Voice
  thresh      312496bit        5Mbit     1250Kbit
  target         58.1ms          5ms       14.5ms
  interval        153ms        100ms        110ms
  pk_delay        375us       8.77ms        2.3ms
  av_delay         29us       6.33ms        193us
  sp_delay          2us          2us         20us
  backlog            0b           0b           0b
  pkts             1576      4132619       108209
  bytes          183085   4174802673    109445618
  way_inds            0        51839            0
  way_miss           68        28699           58
  way_cols            0            0            0
  drops               0       146671           20
  marks               0           11            0
  ack_drop            0            0            0
  sp_flows            1            0            1
  bk_flows            0            1            0
  un_flows            0            0            0
  max_len          1379         7570        34010
  quantum           300          300          300

I wonder what this reveals?
On 5/5 here is a high-resolution dslreports.com/speedtest (with settings as recommended by the OpenWRT guys: high-resolution buffer bloat and 30 seconds ul/dl):

Speed test result

I tested my connection at DSLReports.com/speedtest ..

www.dslreports.com

and here is the waveform bufferbloat test result:

Bufferbloat and Internet Speed Test - Waveform

View the full results, and test your own bufferbloat

www.waveform.com

Regarding:

Maybe fq_codel on the tun interfaces (for flow fairness) and a simple Token Bucket filter on the WAN interface (for shaping).

I would be very eager to try this out.

 

[dave14305]

For Asus-Merlin I have struggled to find the right information on just changing the interface for QoS application - is this documented anywhere? The only thing I could find was the post from RMerlin involving scripting as part of testing CAKE for beta testers and I don't even know if that is still applicable.

You can put that script in /jffs/scripts/qos-start and then put your desired options in /jffs/configs/cake-qos.conf.add:

ULIF='tun11'
DLIF='br0'

Once you do that, you should be able to have your desired setup be applied from the normal starting of qos.

Overhead is not being properly taken into account, but I understand that has a pretty minimal effect in terms of the bandwidth calculation used in CAKE anyway. The overhead stuff seems a little academic given that most users just converge upon limits by trial and error.

It's still important to figure out the right overhead, especially if your configured bandwidth is anywhere close to actual bandwidth. The tcpdump test documented at openwrt.org seems like a good idea. You can install tcpdump via Entware.

As you have asked for dave14305, here is the output from the commands:

The output is rather unreadable because it wasn't put into a code block, so the spacing is too difficult to follow-along left to right. But what I could make out didn't raise any red flags.

You're being given enough rope to hang yourself. Don't slip.

 

[Lynx]

You're being given enough rope to hang yourself. Don't slip.

Thank you for your helpful insight and forbearance.

It's still important to figure out the right overhead, especially if your configured bandwidth is anywhere close to actual bandwidth. The tcpdump test documented at openwrt.org seems like a good idea. You can install tcpdump via Entware.

Was quite hard to capture amongst everything else, but I am pretty confident the entries below relate to the 'ping 8.8.8.8' calls:

tcpdump -vpni tun11

 12:28:09.320900 IP (tos 0x0, ttl 64, id 37907, offset 0, flags [DF], proto ICMP (1), length 84)
    10.8.2.5 > 8.8.8.8: ICMP echo request, id 53794, seq 1, length 64
12:28:09.371879 IP (tos 0x0, ttl 120, id 0, offset 0, flags [none], proto ICMP (1), length 84)
    8.8.8.8 > 10.8.2.5: ICMP echo reply, id 53794, seq 1, length 64

tcpdump -vpni eth0

12:25:43.753390 IP (tos 0x0, ttl 64, id 32446, offset 0, flags [DF], proto UDP (17), length 137)
    10.1.168.205.34090 > 195.206.183.101.1194: UDP, length 109
12:25:43.801997 IP (tos 0x28, ttl 54, id 3721, offset 0, flags [DF], proto UDP (17), length 137)
    195.206.183.101.1194 > 10.1.168.205.34090: UDP, length 109

So my WAN packet overhead is: 137-84=53, right?

The output is rather unreadable because it wasn't put into a code block, so the spacing is too difficult to follow-along left to right. But what I could make out didn't raise any red flags.

Oops, sorry about that. I have changed to code format above.

Thanks for confirming about changing the interface on stock 386.2_6. The steps seem to have worked OK. Having added the script and config file, when I reboot I see:

admin@RT-AX86U-4168:/tmp/home/root# tc qdisc ls

qdisc pfifo_fast 0: dev eth0 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: dev eth1 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: dev eth2 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: dev eth3 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: dev eth4 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: dev eth5 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: dev spu_us_dummy root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: dev spu_ds_dummy root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: dev eth6 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: dev eth7 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc cake 8001: dev br0 root refcnt 2 bandwidth 30720Kbit besteffort dual-dsthost nat wash ingress no-ack-filter split-gso rtt 100ms noatm overhead 53
qdisc pfifo_fast 0: dev tun11 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1

I have to manually enter: service restart_qos, and then I finally get:

admin@RT-AX86U-4168:/tmp/home/root# tc qdisc ls

qdisc pfifo_fast 0: dev eth0 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: dev eth1 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: dev eth2 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: dev eth3 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: dev eth4 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: dev eth5 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: dev spu_us_dummy root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: dev spu_ds_dummy root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: dev eth6 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: dev eth7 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc cake 8003: dev br0 root refcnt 2 bandwidth 30720Kbit besteffort dual-dsthost nat wash ingress no-ack-filter split-gso rtt 100ms noatm overhead 53
qdisc cake 8002: dev tun11 root refcnt 2 bandwidth 20480Kbit diffserv3 dual-srchost nat nowash no-ack-filter split-gso rtt 100ms noatm overhead 53

Is this as expected? How can I avoid having to ssh and enter 'service restart_qos' every time I reboot my router?

What would the equivalent process be to change the interface in respect of the full installation script(?):

mkdir -p /jffs/addons/cake-qos && /usr/sbin/curl -s "https://raw.githubusercontent.com/ttgapers/cakeqos-merlin/master/cake-qos.sh" -o "/jffs/addons/cake-qos/cake-qos" && chmod 755 /jffs/addons/cake-qos/cake-qos && sh /jffs/addons/cake-qos/cake-qos install

 

[dave14305]

What would the equivalent process be to change the interface in respect of the full installation script(?):

Use the same qos-start script but add the ULIF and DLIF lines to qos-start instead of the .add file, before the case statement.

 

[Lynx]

Thanks. Seems to be working very well:

Bufferbloat and Internet Speed Test - Waveform

View the full results, and test your own bufferbloat

www.waveform.com

At the moment on reboot with stock 386.2_6 either upload is not getting set or something appears to be overwriting such that I end up with:

qdisc pfifo_fast 0: dev tun11 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1

until I manually issue 'service restart_qos'

(see my updated post above for details)

Any idea what is causing this and the fix?

Update: ah, I suppose VPN is not up at this point? How should I address that?