Wireguard Session Manager - Discussion (2nd) thread

[ZebMcKayhan]

Whoops!

In the Beta4.14b3 I took note of your suggestion to differentiate between the actual IPv6 configuration status and changed

 [ "$(nvram get ipv6_service != "disabled")" ] && USE_IPV6="Y"; IPV6_TXT="(IPv6) "

to

IPV6_SERVICE=$(nvram get ipv6_service)

if [ "$IPV6_SERVICE" != "disabled" ];then      
        ipv6pt|dhcp6)
            # ip -6 addr | grep "scope global"
            USE_IPV6="Y"; IPV6_TXT="(IPv6) "    # 4.08
        ;;
        6to4|6in4|6rd)
            :
        ;;
        other)
            :
        ;;
        spoof|simulate)
            USE_IPV6="Y"; IPV6_TXT="(IPv6) Simulate "   # v4.14
        ;;
    esac
fi

I've pushed the patch

Update wg_client · MartineauUK/wireguard@0a41e12

FIX: Add detection of IPv6 native

github.com

to Github Dev Branch.


- Doh!

Great!

Altough while experimenting I'm keeping wgm to ipv4 only.

Still cant wrap my head around why its not accepting the default route other then for local packages. Perhaps I need to add the [scope: global] to it. Or perhaps I need to add a proper default route since my system is missing this:

ip -6 route add default dev wg11

If the system somehow requires this.

I will probably need to generate a proper ULA to use within my local network, but I guess a made-up one will do for the moment.

There is a chance that the wg server does not accept packages with a different ipv6 source adress than assigned and we might need to find out if asus/ @RMerlin has included some kind of NPTv6 and how to use it.

//Zeb

 

[ZebMcKayhan]

Continued wireguard ipv6 testing...

Figuring in real life one would have unique globally routable adresses throughout LAN so I generated my own Unique Local Adress (ULA) (kind-of-like 192.168.x.x) which turned out to be:
fdff:a37f:fa75::/48

Subdivided into a first subnet:
fdff:a37f:fa75:1::/64

And set it up in GUI (ipv6 tab, Native, DHCP-PD off) for my LAN.

made a tcpdump of ipv6 packages over wg11 when I was trying to ping 2600:: from my phone:

admin@RT-AC86U-D7D8:/tmp/home/root# tcpdump ip6 -i wg11
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on wg11, link-type RAW (Raw IP), capture size 262144 bytes
00:57:31.694248 IP6 fdff:a37f:fa75:1:bc68:f8f:8dea:30fa > 2600::: ICMP6, echo
 request, seq 1, length 64
00:57:35.712388 IP6 fdff:a37f:fa75:1:bc68:f8f:8dea:30fa > 2600::: ICMP6, echo
 request, seq 1, length 64
00:57:39.768174 IP6 fdff:a37f:fa75:1:bc68:f8f:8dea:30fa > 2600::: ICMP6, echo
 request, seq 1, length 64

3 packets captured
3 packets received by filter
0 packets dropped by kernel

So the ping actually goes out wg11 but nothing comes back.
Trying the same on the router:

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on wg11, link-type RAW (Raw IP), capture size 262144 bytes
00:46:46.870240 IP6 fdab:xxxx:xxxx:69::214 > 2600::: ICMP6, echo request, seq
0, length 64
00:46:47.010364 IP6 2600:: > fdab:xxxx:xxxx:69::214: ICMP6, echo reply, seq 0
, length 64
00:46:47.877710 IP6 fdab:xxxx:xxxx:69::214 > 2600::: ICMP6, echo request, seq
1, length 64
00:46:48.018255 IP6 2600:: > fdab:xxxx:xxxx:69::214: ICMP6, echo reply, seq 1
, length 64
00:46:48.887695 IP6 fdab:xxxx:xxxx:69::214 > 2600::: ICMP6, echo request, seq
2, length 64
00:46:49.026471 IP6 2600:: > fdab:xxxx:xxxx:69::214: ICMP6, echo reply, seq 2
, length 64

6 packets captured
6 packets received by filter
0 packets dropped by kernel

My enterpretation of this is that it only works on router locally since that is the only time the source adress is the intended wg11 adress..

I also tested with using the same ip-prefix as wg11 but the result is the same, so NPTv6 (only translates the prefix) won't work either, we need to masquarade.

admin@RT-AC86U-D7D8:/tmp/home/root# ip6tables -t nat -I POSTROUTING -s fdff:a37f:fa75:1::/64 -o wg11 -j MASQUERADE -m comment --comment "WireGuard 'client'"
ip6tables v1.4.15: can't initialize ip6tables table `nat': Table does not exist (do you need to insmod?)
Perhaps ip6tables or your kernel needs to be upgraded.

Anyone have any idea how to do this?

 

[JGrana]

Thanks @Martineau ! I will do some testing later this week when I am at the remote site (cabin).
In the mean time, I am going to "clean out" both routers with respect to wireguard. I have been doing too much "experimenting". I think I have at least 2 wireguard.ko on both routers and a number of wg-quick files...
BTW, As @RMerlin has noted, there is a wireguard.ko module now in 386.4 beta 2 (at least on my AX88U):

/lib/modules/4.1.51/kernel/net/wireguard/wireguard.ko

Should I use this one (from Asus) or the one from wireguard-manager?

@Martineau

I was able to do some initial testing today. I am able to login to my local (home) locally and my remote (cabin) via ssh.
I am presently running OpenVPN between the 2 sites. Works well except my speed is not great and I have a chron that runs an rsync nightly of off-home backup to a NAS at the cabin.
Home is 200/10 (cable) and cabin is 300/300 (FiOS). So, the weak link is home->10Mb->cabin.

I disabled the VPN's. Rebooted and then downloaded wg-manager via amtm.
I created a home conf and a cabin conf (both called wg11).
BTW, nice job on noticing when Asuswrt-merlin has already installed the wg app. It didn't install on home (AX88U running Beta 386.4_beta2) and did install at cabin (AX86U running Beta 386.4_beta1)

I used this link as my "guide":

WireGuard Site to Site Configuration

How to set up two WireGuard peers in a Site to Site topology.

www.procustodibus.com

As I mentioned earlier, I added the Wireguard serves wg IP to the AllowedIP.
For example:
Home = lan:192.168.1.0/24, wg IP: 10.0.0.1/32, hostname: foo.asuscomm.com
Cabin = lan:192.168.2.0/24, wg IP: 10.0.0.2/32 hostname: bar.asuscomm.com

In the AllowedIPs:
home.conf: 192.168.2.0/24, 10.0.0.2/32
cabin.conf: 192.168.1.0/24, 10.0.0.1/32

I did the conf import into wg_manager. I also deleted the original server peer since the confs I created should have both server and peer.
I was able to start both sides - but pings in either direction failed.
When I did a wg show, I can see that both sides are transmitting but neither side is receiving.

Time to take a step back. I think wg_manager is ideal for connecting a client peer to a 3rd party server (i.e. Muldav) or setting up mobile clients to tunnel to a router as a server peer. But for "plain jane" site to site it might be attempting too much. Its much (IMHO) simpler.

In my scenario, each site has it's own WAN feed to the internet. Both sides have their own DNS server (unbound - thanks again @Martineau ). Each site takes care of routing their LAN clients to the internet.
Neither site really needs firewall or masq settings added - each sites Asus router takes care of that.
A client on one site never routes through the other site to the internet. They just route lan to lan.

It actually a very simple setup. Site A lan devices (192.168.1.0) can VPN to Site B lan devices (192.168.2.0) and vice versa.

I will be traveling to the cabin tomorrow and will spend (some time ;-) over the next few days trying to get this working.

 

[Martineau]

Time to take a step back. I think wg_manager is ideal for connecting a client peer to a 3rd party server (i.e. Muldav) or setting up mobile clients to tunnel to a router as a server peer. But for "plain jane" site to site it might be attempting too much. Its much (IMHO) simpler.

Indeed, the site-to-site topology you desire is not (currently) fully supported by wireguard_manager.

However, you can currently use wireguard_manager to set up the 'server' home Peer 'wg21' to listen to the 'site-to-site' (aka 'Road-Warrior' ) cabin Peer; then simply replicate the 'wg21.conf' to the remote site as 'wg21.conf' and obviously manually edit to change the appropriate directives, but the missing routes will still need to be manually implemented.

Good luck.

 

[ZebMcKayhan]

@Martineau

I was able to do some initial testing today. I am able to login to my local (home) locally and my remote (cabin) via ssh.
I am presently running OpenVPN between the 2 sites. Works well except my speed is not great and I have a chron that runs an rsync nightly of off-home backup to a NAS at the cabin.
Home is 200/10 (cable) and cabin is 300/300 (FiOS). So, the weak link is home->10Mb->cabin.

I disabled the VPN's. Rebooted and then downloaded wg-manager via amtm.
I created a home conf and a cabin conf (both called wg11).
BTW, nice job on noticing when Asuswrt-merlin has already installed the wg app. It didn't install on home (AX88U running Beta 386.4_beta2) and did install at cabin (AX86U running Beta 386.4_beta1)

I used this link as my "guide":

WireGuard Site to Site Configuration

How to set up two WireGuard peers in a Site to Site topology.

www.procustodibus.com

As I mentioned earlier, I added the Wireguard serves wg IP to the AllowedIP.
For example:
Home = lan:192.168.1.0/24, wg IP: 10.0.0.1/32, hostname: foo.asuscomm.com
Cabin = lan:192.168.2.0/24, wg IP: 10.0.0.2/32 hostname: bar.asuscomm.com

In the AllowedIPs:
home.conf: 192.168.2.0/24, 10.0.0.2/32
cabin.conf: 192.168.1.0/24, 10.0.0.1/32

I did the conf import into wg_manager. I also deleted the original server peer since the confs I created should have both server and peer.
I was able to start both sides - but pings in either direction failed.
When I did a wg show, I can see that both sides are transmitting but neither side is receiving.

Time to take a step back. I think wg_manager is ideal for connecting a client peer to a 3rd party server (i.e. Muldav) or setting up mobile clients to tunnel to a router as a server peer. But for "plain jane" site to site it might be attempting too much. Its much (IMHO) simpler.

In my scenario, each site has it's own WAN feed to the internet. Both sides have their own DNS server (unbound - thanks again @Martineau ). Each site takes care of routing their LAN clients to the internet.
Neither site really needs firewall or masq settings added - each sites Asus router takes care of that.
A client on one site never routes through the other site to the internet. They just route lan to lan.

It actually a very simple setup. Site A lan devices (192.168.1.0) can VPN to Site B lan devices (192.168.2.0) and vice versa.

I will be traveling to the cabin tomorrow and will spend (some time ;-) over the next few days trying to get this working.

Since wgm assumes you are setting up internet clients, it will not setup any reverse access to your system, but only RELATED, ESTABLISHED are allowed back in.

Looks like, in the latest beta, @Martineau have implemented the post/pre-up/down directive which could be used for this (please correct me if I'm wrong)

Assuming your client (cabin) is on wg11, you could add this in the cabin.conf file:

PostUp = iptables -I INPUT -i %i -j ACCEPT; iptables -I FORWARD -i %i -j ACCEPT
PreDown = iptables -D INPUT -i %i -j ACCEPT; iptables -D FORWARD -i %i -j ACCEPT

%i could/should be replaced by wg11 (wg replaces this with interface names but don't know if this is implemented in wgm). Also don't know if multiple rules in same line is supported (@Martineau ?).

Regarding the routes, if you put the peer in policy mode with rule:

peer wg11 rule add vpn dst=192.168.1.1/24 comment ToHome
peer wg11 rule add vpn dst=10.0.0.1/32 comment ToHomeRouter

To remove the masquarading you would need to put

iptables -t nat -D POSTROUTING -s 192.168.2.1/16 -o wg11 -j MASQUERADE -m comment --comment "WireGuard 'client'"

(Could be /24 depending on your system setup). Put either in PostUp or in wg11-up.sh.

This would be a start for the client(cabin) side.

the server would not use any masquarading, so you will only need to add the routes in the main table:

ip route add 192.168.2.1/24 dev wg21

Don't think the server has any .conf file (?) So maybee put this in wg21-up.sh

//Zeb

 

[DreaZ]

well, atleast good news that the kernel module is not immediately affecting your system... then I dont get why uninstalling wgm would make any difference. besides this and stopping all peers it just removes various files...

maybee you could try to update the the new kernel module to see if it makes any difference. there are some changes that affect compatibility, but it is a long shot.

//Zeb

Hey! My issue disappeared like 2 weeks ago. It was after I did a some module update.

My speed has been consistently around 800-900 now.

 

[ZebMcKayhan]

Hey! My issue disappeared like 2 weeks ago. It was after I did a some module update.

My speed has been consistently around 800-900 now.

Really! Thats fantastic news! Thanks for the feedback!

 

[Martineau]

Hey! My issue disappeared like 2 weeks ago. It was after I did a some module update.

My speed has been consistently around 800-900 now.

WOW! - for the speed declaration

Hmmmm - for the module update declaration ( i.e. was this a WireGuard Kernel module i.e. using @ZebMcKayhan's rather than the firmware?)

 

[Martineau]

Since wgm assumes you are setting up internet clients, it will not setup any reverse access to your system, but only RELATED, ESTABLISHED are allowed back in.

When using wireguard_manager it would probably be best to configure a 'server' Peer wg2x at both ends then tweak as necessary - although probably still too messy.

Looks like, in the latest beta, @Martineau have implemented the post/pre-up/down directive which could be used for this (please correct me if I'm wrong)

Indeed - with caveats....simply to assist cross-platform portability and makes the WireGuard web tutorials/examples meaningful if not actually 100% compatible with the ASUS router architecture.

PostUp = iptables -I INPUT -i %i -j ACCEPT; iptables -I FORWARD -i %i -j ACCEPT
PreDown = iptables -D INPUT -i %i -j ACCEPT; iptables -D FORWARD -i %i -j ACCEPT

%i could/should be replaced by wg11
(wg replaces this with interface names but don't know if this is implemented in wgm). Also don't know if multiple rules in same line is supported (@Martineau ?).

Multiple 'Pre*' .conf directives per line is not supported by wireguard_manager (does wg-quick support it?) but multiple directives are.

However wireguard_manager Beta v4.14b5 does now allow/support the use of the special '%i' interface placeholder substitution.

e.g. 'wg99.conf' contains (NOTE: two '##' are required to denote that the directive is not to be processed )

# TorGuard USA, Miami
[Interface]

<snip>

#PreUp = iptables -N wg99-input
#PostUp = wg show %i

Dec 29 13:16:37 RT-AC86U-6160 (wg_manager.sh): 31738 v4.14b5 Initialising Wireguard VPN 'client' Peer (wg99)
Dec 29 13:16:37 RT-AC86U-6160 wireguard-clientwg99: Initialising Wireguard VPN client Peer (wg99) to 146.70.51.178:1443 (# TorGuard USA, Miami)
Dec 29 13:16:37 RT-AC86U-6160 wireguard-clientwg99: Executing PreUp: 'iptables -N wg99-input'
Dec 29 13:16:37 RT-AC86U-6160 wireguard-clientwg99: Executing PostUp: 'wg show wg99'
Dec 29 13:16:37 RT-AC86U-6160 wireguard-clientwg99: Initialisation complete.

Upgrade using

e  = Exit Script [?]

E:Option ==> uf dev

 

[JGrana]

@Martineau and @ZebMcKayhan . I will do some experimenting today on site to site.
Both routers (AX88U home; AX86U cabin) are now running 386.4 beta 2. As a result, I have the Asus supplied wg kernel modules.

My plan is to start simple. Just use wg-quick to bring up the interfaces.
Then, look at the routing tables.

A quick question before I dig in.

I am running an OpenVPN server on the cabin and OpenVPN client on home.
Is just turning both off before doing any wireguard attempts or should I disable both and also reboot?

 

[JGrana]

When using wireguard_manager it would probably be best to configure a 'server' Peer wg2x at both ends then tweak as necessary - although probably still too messy.

Yes, that’s the issue with site-to-site. Each site is a server and a client to the other…
In theory, assuming you simply want to access any device on each lan, the wgX.conf should be simple.
I will experiment with a simple site-to-site on these routers and see how it goes…

 

[ZebMcKayhan]

'Pre*' .conf directives per line is not supported by wireguard_manager (does wg-quick support it?) but multiple directives are.

Not really sure, but I have seen this on some examples.
Looking into wg-quick:

PostUp) POST_UP+=( "$value" ); continue ;;

execute_hooks "${POST_UP[@]}"

for hook in "$@"; do         
hook="${hook//%i/$INTERFACE}"         
echo "[#] $hook" >&2         
(eval "$hook")     
done

This probably means more to you than it does for me, but to me it looks like an indexed list, either created by multiple commands or embedded in the string read (perhaps the usage of "," would be more appropriate).
Never the less, multiple commands are probably a better solution:

PostUp = iptables -I INPUT -i %i -j ACCEPT
PostUp = iptables -I FORWARD -i %i -j ACCEPT
PreDown = iptables -D INPUT -i %i -j ACCEPT
PreDown = iptables -D FORWARD -i %i -j ACCEPT

 

[Martineau]

Not really sure, but I have seen this on some examples.
Looking into wg-quick:

PostUp) POST_UP+=( "$value" ); continue ;;

execute_hooks "${POST_UP[@]}"

for hook in "$@"; do      
hook="${hook//%i/$INTERFACE}"      
echo "[#] $hook" >&2      
(eval "$hook")

This probably means more to you than it does for me, but to me it looks like an indexed list, either created by multiple commands or embedded in the string read (perhaps the usage of "," would be more appropriate).

I'm probably wrong, but my understanding is that BASH variable POST_UP is defined as an array, therefore there are no delimiters used/required (neither ';' nor ',' etc. ) to process the collated (as a space separated concatenated string) individual directives (read line-by-line from '.conf')

Never the less, multiple commands are probably a better solution:

PostUp = iptables -I INPUT -i %i -j ACCEPT
PostUp = iptables -I FORWARD -i %i -j ACCEPT
PreDown = iptables -D INPUT -i %i -j ACCEPT
PreDown = iptables -D FORWARD -i %i -j ACCEPT

Indeed

 

[Martineau]

Yes, that’s the issue with site-to-site. Each site is a server and a client to the other…

Technically there are only WireGuard Peers - there is not a 'Server/Client' relationship

 

[jobhax]

I noticed for a while now that when attempting to delete txt while on the wg manager it deletes but keeps the txt onscreen. Unsure what causes it. amtm and other utils do not have this problem. Just a curious thing. I usually exit and start over since I don't remember what's been typed. In the screenshot i typed deleted all the text and typed 3 and hit enter but the txt still shows.

 

[Martineau]

I noticed for a while now that when attempting to delete txt while on the wg manager it deletes but keeps the txt onscreen.

Without knowing which SSH client/platform/codepage you are using I can't quickly tell you how to 'fix' this.
Have you tried a different SSH client?

Unsure what causes it.

Unlike amtm etc. (with its simplistic maximum 1 or 2 character submenu prompt/reply input method), in lieu of the wireguard_manager GUI, the command line syntax/options can be verbose, so during development/testing I wanted to use the 'Pg-Up Key' to retrieve any of the last 5 commands and quickly repeat say peer wg11 config as peer wg12 config by retrieving the command using the Pg-Up key and only needing to overtype the single character '1'

With my fat fingers, retyping a lengthy command such as

peer wg13 rule add wan 52.97.133.162 comment smtp.office365.com

in its entirety from scratch to fix a single typo gets tedious very quickly!

e.g. Best viewed full screen (click on the full screen icon button to the left of the vimeo text in the bottom right corner) , but it works for me....

Backspace    deletes the character to the left of the cursor
Del          deletes the character to the right of the cursor
Pg-Up        retrieves the last command entered (up to a maximum of 5 previous commands stored.)
Cursor-Left  moves cursor to Left
Cursor-Right moves cursor to right

amtm and other utils do not have this problem.

see above

 

[ZebMcKayhan]

After searching for a way to do nat66 to allow my ipv6 network to connect over wg vpn I'm comming up almost blank.

I wouldn't expect Asus to update iptables anytime soon as it ties too much into the system.
Soo, been trying to find userspace implementation (which I could hope to compile to the routers) or kernelmodules that does this.. the closest I could find is:

NAT66: A first implementation
Really old and requires changes to netfilter before compilation which would require a complete custom firmware, so not really feasable.

Xtables-extension-rawnat
Dont really get to know what this is but looks maybee possible to compile. However the raw table does not exist in our ip6tables.

Tayga
Userspace implementation of NAT64 currently available in Entware. Only does ipv6 to ipv4 via tun- interface.

No one that knows any NAT66 userspace implementation that we could hope to compile?

 

[Martineau]

After searching for a way to do nat66 to allow my ipv6 network to connect over wg vpn I'm comming up almost blank.

I wouldn't expect Asus to update iptables anytime soon as it ties too much into the system.

Your diligent research alas hasn't yet revealed an easy solution for older models

....but some models e.g. RT-AX86U apparently have the missing support?

admin@RT-AX86U:/tmp/home/root# ip6tables -V

ip6tables v1.4.15

 ip6tables -nvL -t raw

Chain PREROUTING (policy ACCEPT 538 packets, 144K bytes)
pkts bytes target     prot opt in     out     source               destination        

Chain OUTPUT (policy ACCEPT 172 packets, 103K bytes)
pkts bytes target     prot opt in     out     source               destination

ip6tables -nvL -t nat

Chain PREROUTING (policy ACCEPT 177 packets, 33633 bytes)
pkts bytes target     prot opt in     out     source               destination        

Chain INPUT (policy ACCEPT 34 packets, 20945 bytes)
pkts bytes target     prot opt in     out     source               destination        

Chain OUTPUT (policy ACCEPT 20 packets, 27494 bytes)
pkts bytes target     prot opt in     out     source               destination        

Chain POSTROUTING (policy ACCEPT 20 packets, 27494 bytes)
pkts bytes target     prot opt in     out     source               destination

 

[jobhax]

Without knowing which SSH client/platform/codepage you are using I can't quickly tell you how to 'fix' this.
Have you tried a different SSH client?

Unlike amtm etc. (with its simplistic maximum 1 or 2 character submenu prompt/reply input method), in lieu of the wireguard_manager GUI, the command line syntax/options can be verbose, so during development/testing I wanted to use the 'Pg-Up Key' to retrieve any of the last 5 commands and quickly repeat say peer wg11 config as peer wg12 config by retrieving the command using the Pg-Up key and only needing to overtype the single character '1'

With my fat fingers, retyping a lengthy command such as

peer wg13 rule add wan 52.97.133.162 comment smtp.office365.com

in its entirety from scratch to fix a single typo gets tedious very quickly!

e.g. Best viewed full screen (click on the full screen icon button to the left of the vimeo text in the bottom right corner) , but it works for me....

Backspace    deletes the character to the left of the cursor
Del          deletes the character to the right of the cursor
Pg-Up        retrieves the last command entered (up to a maximum of 5 previous commands stored.)
Cursor-Left  moves cursor to Left
Cursor-Right moves cursor to right

see above

ahh should have mentioned that. Sorry bout that. I have used Windows Terminal and have tried Bitvise SSH client. Both have the same problem. Up and down arrows work but anything with backspace/del does not work which is weird.

 

[ZebMcKayhan]

but some models e.g. RT-AX86U apparently have the missing support?

Ooh, Thats not fair

What firmware are you running? Latest beta from @RMerlin? or stock beta?
Then maybee, just maybee, this will be in a future AC86U firmware as well (fingers crossed)