Wireguard Session Manager - Discussion (2nd) thread
- Thread starter Martineau
- Start date
Martineau
Part of the Furniture
With Flow Cache ENABLED; if WireGuard Peer speed/throughput performance is deemed acceptable, then there is probably no need to DISABLE Flow Cache (despite ASUS/Broadcom seemingly insisting that it should be DISABLED for specific AX routers?).
If however WireGuard Peer speed/throughput dramatically increases when Flow Cache is DISABLED in your environment, then you can decide if it warrants a script to DISABLE Flow Cache when the first WireGuard Peers is made ACTIVE then ENABLE Flow Cache when ALL WireGuard Peers are no longer ACTIVE.
I can't use Wireguard with fc enabled at all (throughput is about 2-3 mbps and router's cores at almost 100 %). If fc disabled Wireguard performance is great. However, even if there are none active Wireguard peers, Speedtest download results won't reach the values I have with fc enabled (300 - 500 mbps down and 100mbps up instead of 940 mbps down and 100 mpbs up).
Can we expect better Wireguard performance with official Asus support? Or you should I look for more powerful router?
Can we expect better Wireguard performance with official Asus support? Or you should I look for more powerful router?
Martineau
Part of the Furniture
There are many threads discussing the impact of Flow Cache on 1Gbps connections.
You can pose this question to ASUS
However, RMerlin presumably provides the same WireGuard Kernel module version 1.0.20210124 as used by ASUS in their v386 RC Beta release.
@ZebMcKayhan has kindly compiled newer WireGuard Kernel modules for a couple of AX routers
GitHub - ZebMcKayhan/Wireguard: Wireguard kernel module and userspace-tools for asus routers running Entware 3.10
Wireguard kernel module and userspace-tools for asus routers running Entware 3.10 - ZebMcKayhan/Wireguard
github.com
I guess you'll have to see if anyone running WireGuard thru' a 1Gbps connection can provide their WireGuard throughput metrics with Flow Cache ENABLED/DISABLED for hardware recommendations.
JGrana
Very Senior Member
And I can personally verify that if you don’t disable flow cash, your log file will fill with lots of “blog_emit” warnings.
In my case no 'blog_emit', but if I run
Code:
tcpdump -i eth0 -vv icmp6
Code:
protocol 0800 is buggy, dev eth0
protocol 0800 is buggy, dev eth0
protocol 0800 is buggy, dev eth0
net_ratelimit: nnnn callbacks suppressed
protocol 0800 is buggy, dev eth0
etcThese go away if flow cache is disabled, but I have no idea if they actually mean anything.
Last edited:
I went in a slightly different direction, not sure if it helpscrap... I dont really find anything that would do this, and without having this myself, it is extremely hard to debug.
The only thing left could be to find the scope of the problem. meaning how much of you prefix could you change. while you accidently encountered the bug, you used a /48 prefix instead of your /64, so you changed 4 more numbers than your prefix allowed. this package got forwarded to eth0 successfully. so could you change more, like
2a02:c7f:: -->?
2a02:: -->?
to speed up testing, just check if the packages arrives out on eth0 (using tcpdump), but dont expect any reply. to create an ULA with 2 as prefix is risky as it could cause conflict with ips on the internet. but A.F.A.I.K only ips starting with 2 is used today, so if we could just change something:
2 = 0b0010 /4
so if the limit is /3 then perhaps 3 could be used as well as:
3 = 0b0011 /3
or can we change just one more bit:
4 = 0b0000 /2
best case, maybee even another:
6 = 0b0110 /1
finally:
a = 0b1010 (already checked this to not work)
so the full list to test would be something like:
2a02:c7f:: -->?
2a02:: -->? (/16)
2a00:: -->? (/12)
2af0:: --> ? (/8)
2000:: -->? (/4)
3000:: -->? (/3)
4000:: --> ? (/2)
6000:: --> ? (/1)
a000:: --> dont work /0
I know it is a lot of testing and will probably take like half an hour to go through. on the other hand, you could do by successive approximation, start in the middle (2000:and if it works, continue down if not continue up. Hopefully we could find a prefix that we could use that is not used on the internet.
whenever you found the smallest amount, try to use this with your masquarading rule (change the -s address accordingly) and hopefully it should work so you get a reply.
To finally resolve this, we probably need someone who has deeper knowledge on kernel routing than me.
Before changing the range, I looked at adding a public IPv6 address which fell outside the allocated range 2a02:c7f:f0c3:1000::1/56
So I tried peer new ip=10.50.5.1/24 ipv6=2a02:c7f:f0c3:2010::1/64 and as with the aa, fa, & fd ranges, I had no IPv6
I then tried peer new ip=10.50.3.1/24 ipv6=2a02:c7f:f0c3:1010::1/48, which worked with IPv6 but note the the designated address is inside the allocated range, whereas peer new ip=10.50.4.1/24 ipv6=2a02:c7f:f0c3:2010::1/48 failed on IPv6, presumably because the designated address is outside.
It therefore seems possible that the routing rule is not checking to see the /CIDR range (I could test this further if you think it would help) but rather whether the selected IPv6 address falls inside the allocated range, so that IPv6 works if it does, and fails if it doesn't.
@Martineau I also looked at creating server using an ipv6 /16 address range peer new ip=10.50.6.1/24 ipv6=2a02:c7f:f0c3:1080::1/16 and ended up with IPv6 working and IPv4 broken. Noted that the device addresses were showing as
Address = 10.50.6.1/32,2a02:c7f:f0c3:1080::2/128, rather than the expected
Address = 10.50.6.2/32,2a02:c7f:f0c3:1080::2/128
and in the server conf
AllowedIPs = 10.50.6.1/32,2a02:c7f:f0c3:1080::2/128 rather than the expected
AllowedIPs = 10.50.6.2/32,2a02:c7f:f0c3:1080::2/128
After editing the server and device conf on the router and the device conf on the phone, IPv4 and IPv6 were fine.
Last edited:
ZebMcKayhan
Very Senior Member
I'm not sure I'm following you, but I agree that the cidr doesn't matter. For wg21 it should always be 64 or higher. My point was to see how far back you could change the ip while packages are still forwarded to eth0.
There could be any number of resons that your ipv6 is not working so right now only use tcpdump to see if the pings reaches out on eth0. Thats all that matters. You will never get a reply until we get the masquarading to work but lets focus on one problem at the time.
You prefix is 56 and I read your post as if you make changes outside this, it doesn't work. Altough in your previous test
In post #732:
Here, packages were forwarded out Altough only /48 is kept the same and packages are forwarded to eth0. You never get any reply since the source address is not yours, but we can setup masquarading or snat later.
What if you try
2a02::1/64 as wg21 adress. Do packages still gets forwarded (only /16 kept)?
Whenever you change wg21+client ip you also need to update the masquarade rule if there should be any chance for you to actually get a reply. But Im still not sure the masquarade rule will work on interface with only link local address.
Another thought:
Any interface could have many ipv6 addresses, so alias doesn't exist, you just add more addresses. What if you add an address within your ULA to eth0? Like:
Code:
ip -6 address add dev eth0 fd36:7ef1:2add:aa88::100/128
Last edited:
Martineau
Part of the Furniture
Whoops
- extremely embarrassing ....concentrating on the IPv6 'server' Peer Base IP stuff and I borked the change for IPv4I've uploaded
wireguard_manager Beta v4.16b4Update wg_manager.sh · MartineauUK/wireguard@3204469
FIX: Creating a new Road-Warrior 'device' Peer does not get correct IPv4 address assigned but instead uses the 'server' Peer base IPv4 address - Thanks SNB Forum member @archiel
github.com
Code:
e = Exit Script [?]
E:Option ==> uf devMartineau
Part of the Furniture
I've uploaded
github.com
Effectively
Unfortunately, the existing
e.g. In the following example
****Don't forget to restart the converted Peers!!!****
Worst case scenario would be to delete ALL Peers and re-import (
To upgrade use
wireguard_manager Beta v4.16b5Update wg_manager.sh · MartineauUK/wireguard@d3583f7
CHANGE: Peer configuration .conf files must be in (cross-platform supported) 'wg/wg-quick' format i.e. comments are now TRUE comments i.e. '#Address = ' is now ignored, ...
github.com
wireguard_manager Peer .conf files are now fully WYSIWYG and wireguard_manager expects them now to be in wg/wg-quick formatUnfortunately, the existing
wireguard_manager Peer .conf files will need to be converted either manually, or I have created a crude command to attempt an auto conversion.e.g. In the following example
Code:
e = Exit Script [?]
E:Option ==> formatwg-quick
Checking Peer Config for conversion to wg-quick format:
wg11.conf
wg12.conf
wg21.conf
wg22.conf
[✔] 'wg12.conf' converted to 'wg-quick' format
Press y to convert 'wg22.conf' multiple '#Pre/#Post' directives or press [Enter] to SKIP:- wg11.conf and wg21.conf are deemed to be already in
wg/wg-quickcompatible format. - wg12.conf was auto converted - 'client' Peers are usually imported so it is safe to assume that only the unique '#Address =' and '#DNS =' directives were converted.
- wg22.conf contains several '#Pre*'/'#Post*' directives so whilst replying 'y' should be safe, it is flagged to remind you to check the conversion!
formatwg-quick Cabin.conf.Imported****Don't forget to restart the converted Peers!!!****
Worst case scenario would be to delete ALL Peers and re-import (
peer import xxxxxx) the appropriate 'client' Peer(s), and recreate (peer new) the necessary 'server' Peer(s)e.g. A dump of running Peer .conf
Code:
e = Exit Script [?]
E:Option ==> raw wg22
================Config===============
# RT-AX58U Server 2
[Interface]
PrivateKey = oCb9xgcgKFuqWn/O0gpyTAht7jlUF7RUX/Of2fOG2UM=
Address = 10.50.2.1/24
ListenPort = 11502
Code:
e = Exit Script [?]
E:Option ==> wg showconf wg22
WireGuard Userspace Tool:
[Interface]
ListenPort = 11502
PrivateKey = oCb9xgcgKFuqWn/O0gpyTAht7jlUF7RUX/Of2fOG2UM=
Code:
e = Exit Script [?]
E:Option ==> wg
WireGuard Userspace Tool:
interface: wg21
public key: rRZLITFAHGq6grFgg4gi8IMf4ti7lngZL2H4qDz78SA=
private key: (hidden)
listening port: 51820
interface: wg22
public key: fW/5OE46hXmEHi2aGUtZx0L1X6rrTfA4Y+pSgnye6Xg=
private key: (hidden)
listening port: 11502
Code:
e = Exit Script [?]
E:Option ==> uf dev
Last edited:
ZebMcKayhan
Very Senior Member
Soo, moving to generate temporary files for the wg setconf? I think this a great choice for the future!I've uploadedwireguard_managerBeta v4.16b5
EffectivelyUpdate wg_manager.sh · MartineauUK/wireguard@d3583f7
CHANGE: Peer configuration .conf files must be in (cross-platform supported) 'wg/wg-quick' format i.e. comments are now TRUE comments i.e. '#Address = ' is now ignored, ...wireguard_managerPeer .conf files are now fully WYSIWYG andwireguard_managerexpects them now to be inwg/wg-quickformat
Unfortunately, the existingwireguard_managerPeer .conf files will need to be converted either manually, or I have created a crude command to attempt an auto conversion.
e.g. In the following example
Code:e = Exit Script [?] E:Option ==> formatwg-quick Checking Peer Config for conversion to wg-quick format: wg11.conf wg12.conf wg21.conf wg22.conf [✔] 'wg12.conf' converted to 'wg-quick' format Press y to convert 'wg22.conf' multiple '#Pre/#Post' directives or press [Enter] to SKIP:NOTE: You may also explicitly specify the .conf file (doesn't have to physically have .conf suffix) such as
- wg11.conf and wg21.conf are deemed to be already in
wg/wg-quickcompatible format.- wg12.conf was auto converted - 'client' Peers are usually imported so it is safe to assume that only the unique '#Address =' and '#DNS =' directives were converted.
- wg22.conf contains several '#Pre*'/'#Post*' directives so whilst replying 'y' should be safe, it is flagged to remind you to check the conversion!
formatwg-quick Cabin.conf.Imported
Worst case scenario would be to delete ALL Peers and re-import (peer import xxxxxx) the appropriate 'client' Peer(s), and recreate (peer new) the necessary 'server' Peer(s)
To upgrade usee.g. A dump of running Peer .conf
Code:e = Exit Script [?] E:Option ==> raw wg22 ================Config=============== # RT-AX58U Server 2 [Interface] PrivateKey = oCb9xgcgKFuqWn/O0gpyTAht7jlUF7RUX/Of2fOG2UM= Address = 10.50.2.1/24 ListenPort = 11502Code:e = Exit Script [?] E:Option ==> wg showconf wg22 WireGuard Userspace Tool: [Interface] ListenPort = 11502 PrivateKey = oCb9xgcgKFuqWn/O0gpyTAht7jlUF7RUX/Of2fOG2UM=Code:e = Exit Script [?] E:Option ==> wg WireGuard Userspace Tool: interface: wg21 public key: rRZLITFAHGq6grFgg4gi8IMf4ti7lngZL2H4qDz78SA= private key: (hidden) listening port: 51820 interface: wg22 public key: fW/5OE46hXmEHi2aGUtZx0L1X6rrTfA4Y+pSgnye6Xg= private key: (hidden) listening port: 11502
Code:e = Exit Script [?] E:Option ==> uf dev
Hazzle-free update, conversion (formatwg-quick) of the conf files (wg11, wg21 and wg21) and starting them again. All works!
Thanks!
Martineau
Part of the Furniture
"A wise man changes his mind, a fool never will"
I assume that's why this exists?
Code:
wg addconf
Usage: wg addconf <interface> <configuration filename>Hassle free?...surely I've left some dumb coding mistake....you've probably not looked hard enough!
Q. Can you please do me a favour and check if the following commands show a priority 220 rule to table 220
Code:
ip rule
0: from all lookup local
220: from all lookup 220
Code:
ip -6 rule
0: from all lookup local
220: from all lookup 220
32766: from all lookup mainZebMcKayhan
Very Senior Member
Uuh, no:
Code:
admin@RT-AC86U-D7D8:/tmp/home/root# ip rule
0: from all lookup local
9810: from all fwmark 0xd2 lookup 210
9900: from 192.168.1.1/24 fwmark 0x8000 lookup main
9910: from all to 192.168.1.1/16 lookup main
9911: from 192.168.1.1/24 lookup 121
9921: from 192.168.6.0/24 lookup 122
9991: from all fwmark 0x1000/0x1000 lookup 121
32766: from all lookup main
32767: from all lookup default
admin@RT-AC86U-D7D8:/tmp/home/root# ip -6 rule
0: from all lookup local
9810: from all fwmark 0xd2 lookup 210
9900: from fdff:a37f:fa75:1::/64 fwmark 0x8000 lookup main
9910: from all to fdff:a37f:fa75::/48 lookup main
9911: from fdff:a37f:fa75:1::/64 lookup 121
9921: from fdff:a37f:fa75:6::/64 lookup 122
9991: from all fwmark 0x1000/0x1000 lookup 121
32766: from all lookup main
admin@RT-AC86U-D7D8:/tmp/home/root#Who would want to redirect ALL Traffic to another routing table? Well, not my business.
Martineau
Part of the Furniture
OK thanks.
So just one single GT-AXE11000 user (out of 190 downloads?) eventually raised a ticket yesterday on my Github asking why my script complained about the lack of WireGuard modules.
I'm sure we'll manage to cope with this magnitude of potential new user support issues!
Last edited:
Okay so I went back to the beginning testingcrap... I dont really find anything that would do this, and without having this myself, it is extremely hard to debug.
The only thing left could be to find the scope of the problem. meaning how much of you prefix could you change. while you accidently encountered the bug, you used a /48 prefix instead of your /64, so you changed 4 more numbers than your prefix allowed. this package got forwarded to eth0 successfully. so could you change more, like
2a02:c7f:: -->?
2a02:: -->?
to speed up testing, just check if the packages arrives out on eth0 (using tcpdump), but dont expect any reply. to create an ULA with 2 as prefix is risky as it could cause conflict with ips on the internet. but A.F.A.I.K only ips starting with 2 is used today, so if we could just change something:
2 = 0b0010 /4
so if the limit is /3 then perhaps 3 could be used as well as:
3 = 0b0011 /3
or can we change just one more bit:
4 = 0b0000 /2
best case, maybee even another:
6 = 0b0110 /1
finally:
a = 0b1010 (already checked this to not work)
so the full list to test would be something like:
2a02:c7f:: -->?
2a02:: -->? (/16)
2a00:: -->? (/12)
2af0:: --> ? (/8)
2000:: -->? (/4)
3000:: -->? (/3)
4000:: --> ? (/2)
6000:: --> ? (/1)
a000:: --> dont work /0
I know it is a lot of testing and will probably take like half an hour to go through. on the other hand, you could do by successive approximation, start in the middle (2000:
whenever you found the smallest amount, try to use this with your masquarading rule (change the -s address accordingly) and hopefully it should work so you get a reply.
To finally resolve this, we probably need someone who has deeper knowledge on kernel routing than me.
ping from my phone using PingTools for Android and testing on the router with tcpdump
and then adding routing via
where 2a02:c7f:f0c3:1000::1 is may current ipv6 WAN and testing the ping again.
In the vast majority of cases I could see the ping immediately on tcpdump and after adding the routing the ping and the echo. In each of these cases I also had full IPv6 connectivity on the phone.
I started with a /48 and was able to reduce down to /8 without problem. However 2000::1/4 was a problem (though 1000::1/4, 3000::1/4 to e000::1/4 were fine) as was f000::1/4. I then tested some fc.../8 and fd.../8 addresses and as previously noted, no ping and no IPv6.
In fact with both 2600::1/8 and f000::1/4 both started okay but quickly messed up, stopping unbound, messing up the ipv6 routing and generating this log
Code:
Mar 14 20:48:30 RT-AX88U-5050 kernel: <unknown>: hw csum failure
Mar 14 20:48:30 RT-AX88U-5050 kernel: CPU: 0 PID: 4477 Comm: unbound Tainted: P O 4.1.51 #2
Mar 14 20:48:30 RT-AX88U-5050 kernel: Hardware name: Broadcom-v8A (DT)
Mar 14 20:48:30 RT-AX88U-5050 kernel: Call trace:
Mar 14 20:48:30 RT-AX88U-5050 kernel: [<ffffffc000087658>] dump_backtrace+0x0/0x150
Mar 14 20:48:30 RT-AX88U-5050 kernel: [<ffffffc0000877bc>] show_stack+0x14/0x20
Mar 14 20:48:30 RT-AX88U-5050 kernel: [<ffffffc00052ae48>] dump_stack+0x90/0xb0
Mar 14 20:48:30 RT-AX88U-5050 kernel: [<ffffffc0003dd194>] netdev_rx_csum_fault+0x3c/0x50
Mar 14 20:48:30 RT-AX88U-5050 kernel: [<ffffffc0003d2408>] skb_copy_and_csum_datagram_msg+0xe8/0xf8
Mar 14 20:48:30 RT-AX88U-5050 kernel: [<ffffffc0004d5aa8>] udpv6_recvmsg+0x2a0/0x7d8
Mar 14 20:48:30 RT-AX88U-5050 kernel: [<ffffffc000484654>] inet_recvmsg+0xa4/0xc8
Mar 14 20:48:30 RT-AX88U-5050 kernel: [<ffffffc0003c36b0>] SyS_recvfrom+0xa8/0x128With 2600::/1 I was taking up part of the Sprint range 2600::/29 and for 2000::/4 and f000::/4 the range was straddling the reserved / restricted addresses and I assume that this was causing the problem. For more detail on restricted addresses see IANA IPv6 Special-Purpose Address Registry, IPv6 Global Unicast Address Assignments (iana.org) and RFC 4193: Unique Local IPv6 Unicast Addresses (rfc-editor.org) which provides details on the routability of Unique-Local addresses. The Unique-Local prefix is drawn from the IPv6 Global Unicast Address range, but is specified as not globally routed.
If I have understood the position correctly it would seem that for users like me who have dynamic IPv6 from their ISP
1. ULAs (fc00::/7) will not work for internet IPv6
2. Link Local addresses (fe80::/10) cannot work with WireGuard
2. If you are choosing a random address, check the IANA preassigned and reserved addresses and make sure your range does not straddle any of these.
3. A better solution may be to find out your ISP's reserved range and then choose a random address within it. You 'might' have a clash with another user for the same ISP, but the odds are extremely small. For SKY (my ISP approximately I in 1.2676506002282*e^30)
And on an RT-AX88U, adding entware iptables (to allow SNAT) does not (appear) to cause any issues.
@ZebMcKayhan let me know if there any other tests that you think may be helpful.
ZebMcKayhan
Very Senior Member
Wow, great testing! Well, one last thing, maybee youve already tested this, but do you HAVE to use SNAT for it to work? MASQUARADE dont work for you?
The problem with SNAT is that you would need to lookup the br0 ip address each time and dont know if firewall gets restarted whenever your (or any other) isp decides to change ip. Masquarade would be much more convenient if it is working.
What would be the format to test MASQUARADE (I have lost track of which of the various commands I should be using)? Otherwise I agree you would need to have a script for up/down which checks the current WAN IP and updates it.
ZebMcKayhan
Very Senior Member
To whomever it concerns:
Just found out that NPT6 (network prefix translation ipv6) is available in our routers.
NPT6 is a stateless (no connection tracking involved) address translation that only translates the prefix but keeps the device-suffix. This is the prefferred translation method whenever possible.
To translate wg21 (fc00:192:168:100::/64) to wg11 (fdab:1337:1337:69::/64) as it goes out wg11:
and to translate the replies back:
only works with /64 and would in theory make it possible to run ULA inside your network and global network on outside while still comply to rfc's as all nodes are still reachable from outside.
Have only tested on outgoing packages and it works:
Just found out that NPT6 (network prefix translation ipv6) is available in our routers.
NPT6 is a stateless (no connection tracking involved) address translation that only translates the prefix but keeps the device-suffix. This is the prefferred translation method whenever possible.
To translate wg21 (fc00:192:168:100::/64) to wg11 (fdab:1337:1337:69::/64) as it goes out wg11:
Code:
ip6tables -t mangle -I POSTROUTING -s fc00:192:168:100::/64 -o wg11 -j SNPT --src-pfx fc00:192:168:100::/64 --dst-pfx fdab:1337:1337:69::/64and to translate the replies back:
Code:
ip6tables -t mangle -I PREROUTING -i wg11 -d fdab:1337:1337:69::/64 -j DNPT --src-pfx fdab:1337:1337:69::/64 --dst-pfx fc00:192:168:100::/64only works with /64 and would in theory make it possible to run ULA inside your network and global network on outside while still comply to rfc's as all nodes are still reachable from outside.
Have only tested on outgoing packages and it works:
Code:
admin@RT-AC86U-D7D8:/tmp/home/root# tcpdump -i wg11 icmp6
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on wg11, link-type RAW (Raw IP), capture size 262144 bytes
22:03:55.120948 IP6 fc00:192:168:100::2 > arn11s11-in-x0e.1e100.net: ICMP6, echo request, seq 1, length 64
^C
1 packet captured
1 packet received by filter
0 packets dropped by kernel
admin@RT-AC86U-D7D8:/tmp/home/root# ip6tables -t mangle -I POSTROUTING -s fc00:192:168:100::/64 -o wg11 -j SNPT --src-pfx fc00:192:168:100::/64 --dst-pfx fdab:1337:1337:69::/64
admin@RT-AC86U-D7D8:/tmp/home/root# tcpdump -i wg11 icmp6
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on wg11, link-type RAW (Raw IP), capture size 262144 bytes
22:04:13.806984 IP6 fdab:1337:1337:69:db77::2 > arn11s11-in-x0e.1e100.net: ICMP6, echo request, seq 1, length 64
^C
1 packet captured
1 packet received by filter
0 packets dropped by kernel
Last edited:
Similar threads
Similar threads
Similar threads
-
New Addon - ChatAI - have a chat session with Googles Gemini AI Bot- Started by JGrana
- Replies: 2
-
Solved How do I remove WG Manager from amtm without resetting amtm?
- Started by DJones
- Replies: 2
Latest threads
-
Ubuntu Linux impacted by decade-old 'needrestart' flaw that gives root- Started by PR3MIUM
- Replies: 0
-
-
-
-
Upgrading from wifi 5 to wifi 6 and looking for suggestions
- Started by Listedguru2
- Replies: 6
Support SNBForums w/ Amazon
If you'd like to support SNBForums, just use this link and buy anything on Amazon. Thanks!
Sign Up For SNBForums Daily Digest
Get an update of what's new every day delivered to your mailbox. Sign up here!