Wireguard Session Manager - Discussion (2nd) thread

[Martineau]

ahh should have mentioned that. Sorry bout that. I have used Windows Terminal and have tried Bitvise SSH client. Both have the same problem. Up and down arrows work but anything with backspace/del does not work which is weird.

I highly recommend/use Xshell 7 (FREE for Personal use) with these key mappings

However, I have pushed wireguard_manager Beta v4.14b6 which allows DISABLING the command buffer 'Pg-Up' feature, if the key mappings can't be altered.

To upgrade use

e  = Exit Script [?]

E:Option ==> uf dev

e  = Exit Script [?]

E:Option ==> createconfig

then you may toggle the feature

e  = Exit Script [?]

E:Option ==> pgupkey off

    [✖]  Use of 'PG-Up' key for command retrieval DISABLED

 

[Martineau]

Ooh, Thats not fair

What firmware are you running? Latest beta from @RMerlin? or stock beta?
Then maybee, just maybee, this will be in a future AC86U firmware as well (fingers crossed)

Currently running

ASUSWRT-Merlin RT-AX86U 386.4_beta2

Linux RT-AX86U 4.1.52 #2 SMP PREEMPT Wed Dec 22 13:54:38 EST 2021 aarch64 ASUSWRT-Merlin

 

[ZebMcKayhan]

Currently running

ASUSWRT-Merlin RT-AX86U 386.4_beta2

Linux RT-AX86U 4.1.52 #2 SMP PREEMPT Wed Dec 22 13:54:38 EST 2021 aarch64 ASUSWRT-Merlin

Hoping that this was something asus been working on on the latest release since there are alot of ipv6 related fixes so I updated to 386.4 beta 3, and the ip6tables NAT table were there!!!!

Giving it a spin:

admin@RT-AC86U-D7D8:/tmp/home/root# ip6tables -t nat -I POSTROUTING -s fdff:a37f:fa75:1::/64 -o wg11 -j MASQUERADE -m comment --comment "WireGuard 'client'"

The rule is accepted by the system. Trying to log pings from my android client:

admin@RT-AC86U-D7D8:/tmp/home/root# tcpdump ip6 -i wg11
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on wg11, link-type RAW (Raw IP), capture size 262144 bytes
14:12:44.027822 IP6 fdab:xxxx:xxxx:69::214 > 2600::: ICMP6, echo request, seq 1, length 64
14:12:44.167776 IP6 2600:: > fdab:xxxx:xxxx:69::214: ICMP6, echo reply, seq 1, length 64
14:12:47.167281 IP6 fdab:xxxx:xxxx:69::214 > 2600::: ICMP6, destination unreachable, unreachable address fdab:xxxx:xxxx:69::214, length 112

A reply atleast. So the adress is translates but it currently does not seem to work the other way.

 

[ZebMcKayhan]

Ok, now it starts to happen...
For some (unknown) reason I had to add a subnet route to ip -6 route table main:

admin@RT-AC86U-D7D8:/tmp/home/root# ip -6 route
fdab:xxxx:xxxx:69::/64 dev wg11 proto kernel metric 256 pref medium
fdff:a37f:fa75:1::/64 dev br0 proto kernel metric 256 pref medium
fdff:a37f:fa75:1::/64 dev br0 metric 1024 pref medium

The "proto kernel" was there since before and return packets were dropped, but by adding the exact same route it just works... wierd...

Now I can ping pretty much every ipv6 I can think of from my Android phone (yippie).

But... for some reason I cannot do ipv6 lookup. Clients recieve router br0 adress as dns, so ending up at dnsmasq I guess (which should be redirecting to Unbound)

so looking at unbound config:

#########################################
# integration IPV6
#
# do-ip6: no
# private-address: ::/0
do-ip6: yes
# edns-buffer-size: 1232
interface: ::0
# access-control: ::0/0 refuse
access-control: ::1 allow
# private-address: fd00::/8
# private-address: fe80::/10
#########################################
#module-config: "dns64 respip validator iterator"      
#dns64-prefix: 64:FF9B::/96

Tried to enter dns manually in gui but still I only get "Unknown host"

Edit: probably not unbound related... turned off unbound and it still doesn't work, even put in my wg dns in the ipv6 gui but same result... hmmmmm....

 

[ZebMcKayhan]

Well, Im not really sure, but I think I'm there. It was just to reconnect my wifi to get a new dns advertisement and I can now access sites like ipv6.google.com
I can ping the same adress with "PingTools" for Android (wonderful tool btw).
However, my ipv6-test looks like this (see pics) Is this because I use unbound or vpn or am I still missing something?

Dnsleaktest.com still only shows my public ipv4...

Edit: DNS6+IP4 and DNS6+IP6 goes "Reachable" and Green if I stop unbound in unbound_manager using 3 is this the expected behavior?

Edit2: turns out Unbound does not do lookup over ipv6 if outgoing-interface: is set to an ipv4 adress (192.168.1.1 in my case) (kind of makes sense). So adding:

outgoing-interface: 192.168.1.1
outgoing-interface: fdff:a37f:fa75:1::1

Does the trick!

 

[Martineau]

Edit: DNS6+IP4 and DNS6+IP6 goes "Reachable" and Green if I stop unbound in unbound_manager using 3 is this the expected behavior?

Edit2: turns out Unbound does not do lookup over ipv6 if outgoing-interface: is set to an ipv4 adress (192.168.1.1 in my case) (kind of makes sense). So adding:

outgoing-interface: 192.168.1.1
outgoing-interface: fdff:a37f:fa75:1::1

Does the trick!

Well someone has spent the festive period being extremely focused and productive ...myself I have been in a merry stupor which is always dangerous when scripting!

Thanks for the heads-up about the missing unbound IPv6 outgoing-interface: for compatibility for these hybrid DNS6+IP4 and DNS6+IP6 environments and will get around to incorporating the directive to unbound_manager sometime in 2022!

Many thanks for documenting/supporting wireguard_manager in 2021, and may it continue in 2022; if there is still a need for its implementation.

Happy New Year.

 

[ZebMcKayhan]

Many thanks for documenting/supporting wireguard_manager in 2021, and may it continue in 2022; if there is still a need for its implementation.

Thank you for a great script and Happy New Year!

I will kick-off 2022 by trying to re-import wg11 in wgm as ipv4/ipv6 and see if everything turns out correct now that I know what it takes and do some documentation about it.

 

[Martineau]

Thank you for a great script and Happy New Year!

I will kick-off 2022 by trying to re-import wg11 in wgm as ipv4/ipv6 and see if everything turns out correct now that I know what it takes and do some documentation about it.

If you can summarise the hoops you went thru', then I'll replace any of the IPv6 dross I guessed at in wireguard_manager

 

[JGrana]

I wish I was as productive as @ZebMcKayhan at my site-site testing.
I tried simple at first. No PostUp/Down, just a Server and single Peer directive
Used Entwares wq-quick to bring wg up and down.
wg showed links up and even some bytes flowing. But no ping to either side worked.
I then tried various iptables settings (4 different from various web sites)

Here is the wg0.conf from one side (the other peer, wg0, had the same iptables with appropriate changes to its port/wg:

[Interface]
PrivateKey = wEof+SBziREPd8mi6XaLyeHjv9fXxAjZ5vb6bDzTmGk=
Address = 10.0.0.2/32
ListenPort = 51822

#PreUp = iptables -I INPUT -p udp --dport 51821 -j ACCEPT
#PostDown = iptables -D INPUT -p udp --dport 51821 -j ACCEPT

#PreUp = iptables -I INPUT -i wg11 -j ACCEPT; iptables -I FORWARD -i wg11 -j ACCEPT
#PreDown = iptables -D INPUT -i wg11 -j ACCEPT; iptables -D FORWARD -i wg11 -j ACCEPT

#PostUp = iptables -A FORWARD -i wg11 -j ACCEPT; iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
#PostDown = iptables -D FORWARD -i wg11 -j ACCEPT; iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE

PostUp = iptables -A FORWARD -i %i -j ACCEPT; iptables -A FORWARD -o %i -j ACCEPT; iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
PostDown = iptables -D FORWARD -i %i -j ACCEPT; iptables -D FORWARD -o %i -j ACCEPT; iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE
# Peer is Homer

[Peer]
PublicKey = YKfJ1qPRrJOuTqfrPVagDG2S43dMkCXPfyp7XXXX
Endpoint = XXXXXX.asuscomm.com:51821
AllowedIPs = 192.168.1.0/24, 10.0.0.1/32

BTW, for the first three, I didn’t put the tunnel IP in AllowedIP’s.

In most of the above cases, wg showed the peer sending but not receiving.

OpenVPN works fine, here is the routing table when its active:

Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
default lo0-100.SYRCNY- 0.0.0.0 UG 0 0 0 eth0
10.8.1.0 * 255.255.255.0 U 0 0 0 tun21
68.237.161.xxx lo0-100.SYRCNY- 255.255.255.255 UGH 1 0 0 eth0
71.250.0.12 lo0-100.SYRCNY- 255.255.255.255 UGH 1 0 0 eth0
96.238.146.0 * 255.255.255.0 U 0 0 0 eth0
96.238.146.1 * 255.255.255.255 UH 0 0 0 eth0
127.0.0.0 * 255.0.0.0 U 0 0 0 lo
192.168.1.0 10.8.1.2 255.255.255.0 UG 0 0 0 tun21
192.168.2.0 * 255.255.255.0 U 0 0 0 br0
default via 96.238.146.1 dev eth0
10.8.1.0/24 dev tun21 proto kernel scope link src 10.8.1.1
68.237.161.xxx via 96.238.146.1 dev eth0 metric 1
71.250.0.12 via 96.238.146.1 dev eth0 metric 1
96.238.146.0/24 dev eth0 proto kernel scope link src 96.238.146.126
96.238.146.1 dev eth0 proto kernel scope link
127.0.0.0/8 dev lo scope link
192.168.1.0/24 via 10.8.1.2 dev tun21
192.168.2.0/24 dev br0 proto kernel scope link src 192.168.2.1

I cant believe I didn’t capture the table after wireguard is started…

Anyway, I believe there is some kind of route issue. Over my head….

Have a good Holiday folks - and thanks for all your efforts!

 

[ZebMcKayhan]

If you can summarise the hoops you went thru', then I'll replace any of the IPv6 dross I guessed at in wireguard_manager

Most commands are from the scripts, so I hope were good:

wg11-route-up.sh:

### ipv6 ###
ip -6 address add dev wg11 <MyWgIpv6>/64 2>/dev/null
ip link set up dev wg11
############

wg11-up.sh

##### ipv6 config ######
ip -6 route add ::/0 dev wg11 2>/dev/null

ip6tables -t mangle -I FORWARD -o wg11 -p tcp -m tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu -m comment --comment "WireGuard 'client'" 2>/dev/null
ip6tables -t mangle -I FORWARD -i wg11 -p tcp -m tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu -m comment --comment "WireGuard 'client'" 2>/dev/null
ip6tables -t mangle -I FORWARD -o wg11 -j MARK --set-xmark 0x01/0x7 -m comment --comment "WireGuard 'client'" 2>/dev/null
ip6tables -t mangle -I PREROUTING -i wg11 -j MARK --set-xmark 0x01/0x7 -m comment --comment "WireGuard 'client'" 2>/dev/null
ip6tables -t filter -I FORWARD -i br0 -o wg11 -j ACCEPT 2>/dev/null

ip6tables -t nat -I POSTROUTING -s fdff:a37f:fa75:1::/64 -o wg11 -j MASQUERADE -m comment --comment "WireGuard 'client'"

ip -6 route del fdff:a37f:fa75:1::/64 dev br0 2>/dev/null
ip -6 route add fdff:a37f:fa75:1::/64 dev br0
#################

The default route could probably be as wgm has it. The route to br0 I dont understand why it is needed so wgm should probably not have this

 

[ZebMcKayhan]

Just converted wg11 to ipv4/ipv6 in wgm

For some reason wgm does not apply this rule:

ip6tables -t filter -I FORWARD -i br0 -o wg11 -j ACCEPT

All other rules to NAT and MANGLE is applied correctly.

DNS seems to be expected to be ipv6, but only ipv4 is imported. should probably be both, one dns for ipv4 and another for ipv6:

E:Option ==> restart wg11

        Requesting WireGuard VPN Peer restart (wg11)

        Restarting Wireguard 'client' Peer (wg11)
        wireguard-clientwg11: Wireguard VPN 'client' Peer (wg11) to xxxxxxxx:48574 (# Integrity Swe) Terminated

        wireguard-clientwg11: Initialising Wireguard VPN 'client' Peer (wg11)
in Policy Mode to xxxxxxxxx:48574 (# Integrity Swe) DNS=192.168.1.
1
ip6tables v1.4.21: Bad IP address "192.168.1.1"

Try `ip6tables -h' or 'ip6tables --help' for more information.
        wireguard-clientwg11: Initialisation complete.


        WireGuard ACTIVE Peer Status: Clients 2, Servers 0

Edit: just double-checked the forward rule but without it my clients are not allowed access. Iptables contains a br0 to everything rule but ip6tables does not:

admin@RT-AC86U-D7D8:/jffs/addons/wireguard/Scripts# ip6tables -nvL FORWARD -t
filter
Chain FORWARD (policy DROP 0 packets, 0 bytes)
pkts bytes target     prot opt in     out     source               destination
122K  137M ACCEPT     all      *      *       ::/0                     ::/0
            state RELATED,ESTABLISHED
    0     0 ACCEPT     all      br0    eth0    ::/0                 ::/0

    0     0 ACCEPT     all      br0    br0     ::/0                 ::/0

    0     0 logdrop    all      *      *       ::/0                 ::/0
            state INVALID
    0     0 ACCEPT     59       *      *       ::/0                 ::/0
            length 40
    3   312 ICMP_V6    icmpv6    *      *       ::/0                 ::/0

   34 15595 OVPN       all      *      *       ::/0                 ::/0
            state NEW
   34 15595 logdrop    all      *      *       ::/0                 ::/0

Edit2:
Another thing, if the endpoint is looked-up as AAAA then the tunnel might connect over ipv6 and if that peer is started in default mode then a specific route to that endpoint would be needed and looks like it is currently missing:

host="$(wg show $VPN_ID endpoints | sed -n 's/.*\t\(.*\):.*/\1/p')"
if [ -n "$(echo "$host" | Is_IPv4)" ];then # v4.08
ip route add $(ip route get $host | sed '/ via [0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}/{s/^\(.* via [0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\).*/\1/}' | head -n 1) 2>/dev/null
else
 :
# ip -6 route add
fi

 

[heysoundude]

well Zippitydoodahday...now we're cooking with gas!

Well someone has spent the festive period being extremely focused and productive ...myself I have been in a merry stupor which is always dangerous when scripting!

Thanks for the heads-up about the missing unbound IPv6 outgoing-interface: for compatibility for these hybrid DNS6+IP4 and DNS6+IP6 environments and will get around to incorporating the directive to unbound_manager sometime in 2022!

Many thanks for documenting/supporting wireguard_manager in 2021, and may it continue in 2022; if there is still a need for its implementation.

Happy New Year.

@thelonelycoder has released 2 updates to amtm in <24hrs, @Martineau - once the champagne has worn off, it appears you have some catching up to do
This may permit me to skip the DDNS shenanigans I thought I'd have to engage in to make my AC86 a WG server for my devices that roam off-wifi to connect to for the benefits of Merlin!

 

[ZebMcKayhan]

well Zippitydoodahday...now we're cooking with gas!

@thelonelycoder has released 2 updates to amtm in <24hrs, @Martineau - once the champagne has worn off, it appears you have some catching up to do
This may permit me to skip the DDNS shenanigans I thought I'd have to engage in to make my AC86 a WG server for my devices that roam off-wifi to connect to for the benefits of Merlin!

Perhaps a better way to stimulate progression is to setup your ipv6 server and report whatever bugs you find and populate the appropriate fixes in userscripts. Neither me or @Martineau (?) has the possibility to do this, so your testing are most welcome!

 

[Martineau]

Also don't know if multiple rules in same line is supported (@Martineau ?).

FYI,

wireguard_manager Beta v4.14b7 now supports multiple 'Pre*' .conf directives per line, and also now uniquely allows/supports the use of my special '%w' WAN interface placeholder substitution.

NOTE: Full IPv6 Selective Routing support is still in progress...so wouldn't hold your breath!

 

[ZebMcKayhan]

NOTE: Full IPv6 Selective Routing support is still in progress...so wouldn't hold your breath!

No worries, it'll be here when its done.

Did a feeble attempt to assign a separate ipv6 subnet to YazFi client by adding in dnsmasq.conf.add:

interface=wl1.2
#setup stateless assignment based on if prefix:
dhcp-range=wl1.2,::,constructor:wl1.2,ra-stateless,64,600
#assign DNS:
dhcp-option=wl1.2,option6:23,[2620:fe::fe],[2620:fe::9]

But it turns out that YazFi client gets both br0 adress and wl1.2 address, probably a side effect of an interface could have many ipv6 addresses and wl1.2 is inside br0... back to the drawing board.

Also messed around with ipsets... turns out ipsets are either ipv4 or ipv6, never both. Why can't stuff just work seamless!
In order to create an ipv6:

ipset create NETFLIX-DNS6 hash:net family inet6

So I added it in dnsmasq with some domain names but no ips were ever populated in it even if I ping them or do dnslookup from my phone...

Turns out you can create a list:set type and add both ipv4 ipsets and ipv6 ipsets in this ipset list. This way you could use the same firewall rule for both iptables and ip6tables.

ipset create dns4 hash:net family inet
ipset create dns6 hash:net family inet6
ipset create dns list:set
ipset add dns dns4
ipset add dns dns6

 

[Martineau]

In order to create an ipv6:

ipset create NETFLIX-DNS6 hash:net family inet6

So I added it in dnsmasq with some domain names but no ips were ever populated in it even if I ping them or do dnslookup from my phone...

Seems to work here (no Unbound)

'/jffs/configs/dnsmasq.conf.add'

ipset=/.netflix.com/NETFLIX-DNS4,NETFLIX-DNS6

ipset list NETFLIX-DNS4

Name: NETFLIX-DNS4
Type: hash:net
Revision: 6
Header: family inet hashsize 1024 maxelem 65536
Size in memory: 928
References: 0
Number of entries: 9
Members:
18.200.8.190
52.211.24.99
54.170.196.176
54.171.66.117
54.73.148.110
54.246.79.9
54.155.246.232
52.214.181.141
52.214.70.222

nslookup netflix.com '127.0.0.1' | awk 'NR>2&&/^Address/{print $(NF==2?2:3);exit}'

ipset list NETFLIX-DNS6

Name: NETFLIX-DNS6
Type: hash:net
Revision: 6
Header: family inet6 hashsize 1024 maxelem 65536
Size in memory: 1472
References: 0
Number of entries: 3
Members:
2a05:d018:76c:b685:3b38:679d:2640:1ced
2a05:d018:76c:b684:8e48:47c9:84aa:b34d
2a05:d018:76c:b683:f711:f0cf:5cc7:b815

 

[ZebMcKayhan]

Seems to work here

Thanks! I had 2 different lines in dnsmasq.conf.add containing the same domains, that did not work, but if I do like you and put both ipsets on the same line it works.

I also removed the YazFi guest networks from br0, like:

brctl delif br0 wl1.2

So it becomes a stand-alone port.
This seems to do the trick since all rules are based on the physical interface. Don't get why they are in br0 in the first place.

 

[Martineau]

Thanks! I had 2 different lines in dnsmasq.conf.add containing the same domains, that did not work, but if I do like you and put both ipsets on the same line it works.

I think you can have multiple lines referencing/populating a single IPSET; but not sure if the domains defined must be unique?

Also should you wish to bypass dnsmasq, not 100% sure if unbound now also allows auto-populating IPSETs - both IPv4 and IPv6.?

 

[Jack Yaz]

Thanks! I had 2 different lines in dnsmasq.conf.add containing the same domains, that did not work, but if I do like you and put both ipsets on the same line it works.

I also removed the YazFi guest networks from br0, like:

brctl delif br0 wl1.2

So it becomes a stand-alone port.
This seems to do the trick since all rules are based on the physical interface. Don't get why they are in br0 in the first place.

wifi client authentication might start failing, unless eapd doesn't need the same fudging that it used to in older firmwares

 

[ZebMcKayhan]

I think you can have multiple lines referencing/populating a single IPSET; but not sure if the domains defined must be unique?

I had 2 lines with the same domains and the second line ipset were NOT populated, so I would say they must be unique.

Also should you wish to bypass dnsmasq, not 100% sure if unbound now also allows auto-populating IPSETs - both IPv4 and IPv6.?

I don't see any point in bypassing dnsmasq so someone else could look into this.

wifi client authentication might start failing, unless eapd doesn't need the same fudging that it used to in older firmwares

Good to know! I'm typically not using my guest networks daily so I can keep this on experimental for now. I was connected 1h yesterday with no problems at all and nothing in syslog. Will keep testing.