WG Server test with flowcache bypass

[ColinTaylor]

Oh ok, so its a long standing bug then with the AX86U. I've reported this a long time now but i guess no one uses Cake QOS on the AX86U and have a NAS.

I can confirm your findings on my RT-AX86U also. Enabling Cake reduces the WiFi to (Ethernet) LAN transfer speed from ~112MB/s to ~80MB/s, even though the traffic is only traversing br0 and CPU usage is ~10%.

EDIT: Corrected typo.
EDIT2: Corrected Mbps to MB/s.

 

[Morris]

I can confirm your findings on my RT-AX86U also. Enabling Cake reduces the WiFi to (Ethernet) LAN transfer speed from ~112Mbps to ~80Mbps, even though the traffic is only traversing br0 and CPU usage is ~10%.

EDIT: Corrected typo.

I wonder why I get WiFi to Ethernet transfers much faster than you do with cake enabled?

 

[supe]

I wonder why I get WiFi to Ethernet transfers much faster than you do with cake enabled?

An alpha version pertaining to this thread has been released, so I guess I can post here just for clarity

I don't know what you are trying to convey, is it that your WiFi to Ethernet in your LAN is faster with cake enabled than with it disabled, or that you simply have faster then ~80 MB/s with cake enabled?

 

[dave14305]

Enabling Cake reduces the WiFi to (Ethernet) LAN transfer speed from ~112Mbps to ~80Mbps, even though the traffic is only traversing br0 and CPU usage is ~10%.

Out of curiosity, does the 80 Mbps relate at all to the configured upload bandwidth when CAKE was enabled?

 

[ColinTaylor]

Out of curiosity, does the 80 Mbps relate at all to the configured upload bandwidth when CAKE was enabled?

No. I've tried experimenting with wildly different upload and download values but they made no difference.

To be clear, this appears to be related to Cake (I haven't tried other QoS types) rather than just Flow Cache. With QoS turned off I can directly enable or disable flow cache and this problem does not appear to be as noticeable.

@dave14305 If there's something else you'd like me to test I'm more than happy to. But I suggest you send me a PM rather than us polluting this thread with off-topic posts as the problem doesn't appear to be directly related to WireGuard or flow cache.

EDIT: Corrected my previous post from Mbps to MB/s.

 

[Morris]

An alpha version pertaining to this thread has been released, so I guess I can post here just for clarity

I don't know what you are trying to convey, is it that your WiFi to Ethernet in your LAN is faster with cake enabled than with it disabled, or that you simply have faster then ~80 MB/s with cake enabled?

CAKE enabled with 116 Mbs up and 107 Mbs down for my FIOS WAN which is 100-Mbs. I have Besteffort configured in both directions and NAT lookup for both directions as well. Testing from my phone to my wired DVR I just got 241 Mbs down and from my laptop I get 560 to 600 Mbs down. File transfers are similar in both directions.

CAKE is only for the LAN and has no effect on the LAN. I believe you that your transfer rates are not as good as they should be yet CAKE is not the issue

 

[ColinTaylor]

@Morris See my correction above. We were talking about mega-bytes, not mega-bits.

 

[Morris]

No. I've tried experimenting with wildly different upload and download values but they made no difference.

To be clear, this appears to be related to Cake (I haven't tried other QoS types) rather than just Flow Cache. With QoS turned off I can directly enable or disable flow cache and this problem does not appear.

@dave14305 If there's something else you'd like me to test I'm more than happy to. But I suggest you send me a PM rather than us polluting this thread with off-topic posts as the problem doesn't appear to be directly related to WireGuard or flow cache.

EDIT: Corrected my previous post from Mbps to MB/s.

So your experience is 8 times worse.

When you turn off flow cache, the flows will not go through the CAKE code so you are effectively disabling it. Why not just disable CAKE? If you do disable CAKE, dose it make a difference for your pore WiFi to LAN experience?

 

[ColinTaylor]

Why not just disable CAKE?

I don't normally use Cake or any other type of QoS. I only enabled it temporarily as an academic exercise to confirm/deny @supe's findings.

If you do disable CAKE, dose it make a difference for your pore WiFi to LAN experience?

Yes. As stated above, with QoS disabled I don't see this issue at all.

 

[Morris]

I don't normally use Cake or any other type of QoS. I only enabled it temporarily as an academic exercise to confirm/deny @supe's findings.


Yes. As stated above, with QoS disabled I don't see this issue at all.

When you have CAKE enabled, have you loaded the add on and set both directions to Besteffort? IMHO, tins should not be used with CAKE as they interfere with the process. This difference could be resulting is queuing issues.

 

[dave14305]

When you have CAKE enabled, have you loaded the add on and set both directions to Besteffort? IMHO, tins should not be used with CAKE as they interfere with the process. This difference could be resulting is queuing issues.

CAKE qdisc on the WAN interface shouldn’t be seeing LAN traffic at all, so I don’t think this notion is applicable.

 

[Morris]

CAKE qdisc on the WAN interface shouldn’t be seeing LAN traffic at all, so I don’t think this notion is applicable.

It probably is not seeing the traffic on the WAN interface. I suspect the issue is about scheduling of the traffic mover tasks and some sort of starvation is going on.

 

[sfx2000]

@dave14305 If there's something else you'd like me to test I'm more than happy to. But I suggest you send me a PM rather than us polluting this thread with off-topic posts as the problem doesn't appear to be directly related to WireGuard or flow cache.

I would recommend starting a new thread as this one has gone well off-topic

 

[johndoe85]

Yes, tho still don't expect WG throughput anywhere close to a desktop CPU.


Cake applies to the entire LAN, therefore the entire LAN would need to bypass NAT acceleration, which would provide the exact same result as disabling NAT acceleration.

Flow cache bypass doesn't magically make WireGuard NAT accelerated. It just allows clients/servers that use WireGuard to be excluded from NAT acceleration without having to disable it for all clients that do not go through WireGuard.

This NAT acceleration problem is only for WireGuard server, right? If i have a WG client in the router connected to an outside WG server i won't suffer NAT acceleration problems?

 

[ColinTaylor]

This NAT acceleration problem is only for WireGuard server, right? If i have a WG client in the router connected to an outside WG server i won't suffer NAT acceleration problems?

No, it affect clients and servers.

 

[johndoe85]

No, it affect clients and servers.

Ah ok! Thank you

 

[Mikii]

This NAT acceleration problem is only for WireGuard server, right? If i have a WG client in the router connected to an outside WG server i won't suffer NAT acceleration problems?

I understand this is behind Merlin reach to fix it, but is this NAT acceleration Bug/problem something that we can expect to be fixed in the future or do we have to cope with it forever? I am asking so that I can better weight my options with future scenario in mind. Thanks.

 

[ColinTaylor]

I understand this is behind Merlin reach to fix it, but is this NAT acceleration Bug/problem something that we can expect to be fixed in the future or do we have to cope with it forever? I am asking so that I can better weight my options with future scenario in mind. Thanks.

See post #6.

 

[Mikii]

I see... thanks for pointing that out... and thanks of course Merlin. So something is coming, we just have to wait and see how effective it will be.

 

[sfx2000]

All thanks should go to Asus or Broadcom (not sure which of them implemented the bypass). I'm just backporting it ahead of the next GPL release which probably won't be until February.

I'm not sure either what kind of performance will be provided from Client mode, it may be different.

I've been following this thread for a while, but have reserved comment...

Nice to see that the WG flow can do the fast path through flow cache - makes sense as that traffic can be easily classified in any case.

Recall with HW acceleration, there is the commitment that no packets get dropped - and esp with WG, as the tunnel is UDP only, so we cannot afford to drop packets without impacting the application layers...

As some in this thread have mentioned QoS - recall that QoS (whether Cake SQM or other) is tolerant of dropping packets to ensure that the flows are treated according to the qdisc properties...

Would be interesting to see if WG is treated as a QOS managed flow or just allowed the fast path thru flowcache as an exception...