Wireguard Session Manager - Discussion (3rd) thread

[Ellenswamy]

Do you not see the command in the post?

ha! sorry my mistake

 

[archiel]

Looks mostly like typo/concatenation errors....

But while following the sequence of commands, I don't see any route added to wg21 clients in policy route table, nor any "to wg21 use main" rules. If none of these are added there might be issues for any client policy routed to wg11 to contact wg21 clients.
This may not always be a problem, but you will not be able to contact your lan device, which is also routed to wg11, from wg21 clients (actually you could but it wont be able to reply)
Maybee the most obvious problem is if entire wg21 (including wg21 own ip) is used in passthrough any router local process (chrony, ntpd, dnsmasq a.s.o) will use wg21 ip to contact wg21 clients, which leads to routing table 121 where no routes exist so connection may be broken. If wg21 clients are then setup to use router for dns and wg11 too, it is not going to work.
Dns is not really an issue for a vanilla setup where wg21 clients uses wan dns and for passthru gets redirected to wg11 dns but you are quite far from vanilla setup.

I would thought that wgm adds route to wg21 in table 121 to get rid of this issue for passthru, but if it don't you still might need the 2 "To Wg21 use Main" rules even with passthru.

@ZebMcKayhan I am assuming your comments above are directed at @Martineau, but it does remind me that I still need to check my road-warrior to LAN connectivity once passthru is setup - not an issue for my phone, but with a laptop I may want to RDP to another desktop on the lan. In theory I can setup two 'device.conf' settings, one with passthru and one without, but I still think I should see what happens.

 

[ZebMcKayhan]

@ZebMcKayhan I am assuming your comments above are directed at @Martineau,

No, not really. I would not consider this to be a bug and I always assume there is a point as to why wgm is designed as it is although I might not always understand it. I may hope that @Martineau reads it and consider it but there may be perfectly good reasons for why wgm does what it does. The point was to you that you still might need some of the rules for your system even though the typos are fixed in wgm.

 

[archiel]

No, not really. I would not consider this to be a bug and I always assume there is a point as to why wgm is designed as it is although I might not always understand it. I may hope that @Martineau reads it and consider it but there may be perfectly good reasons for why wgm does what it does. The point was to you that you still might need some of the rules for your system even though the typos are fixed in wgm.

I don't follow what it is that you 'would not consider this to be a bug' I can add all of wg11 or specific devices using the rules you provided and the 'road-warrior' devices work as expected (the only IPs and DNS they expose are those of the VPN provider). If I use the latest beta, this does not happen, the IPv4 address are those of the VPN provider, but the IPv6 are from my ISP range. For passthru I was testing with just one device, I will test ALL and see if makes any difference.

Also looking at the errors in start wg11 debug they appear to arise from not parsing the wg21 IPv6 address correctly and so I thought that this could be connected to the IPv6 leaks in passthru.

 

[Martineau]

Also looking at the errors in start wg11 debug they appear to arise from not parsing the wg21 IPv6 address correctly and so I thought that this could be connected to the IPv6 leaks in passthru.

Abject apologies....I'm sure you've had more than enough of trying to overcome the seemingly never ending wireguard_manager inadequacies within your true Dual-stack IPv4+IPv6 environment, without the added frustrations of my idiotic hastily applied (and totally inexcusable) shoddy typos.

However, not having an IPv6 stack to test with, slightly mitigates the fact that the code execution path cannot be fully tested by me, but it may be why the following appears to be invalid....but works for IPv4?

ip -6 route show cache

ip -6 route flush cache

Consequently, with hindsight, I concur, you are correct to probably eschew the use of the Passthru feature, as it would be preferable to simply have the Road-Warrior 'client' Peer connect directly to the VPN ISP WireGuard 'server' Peer endpoint for better throughput/performance anyway, but I am truly grateful for your diligence and patience when offering your time/resource to improve the script's IPv6 compatibility - despite the many tedious interruptions to service endured by your family.

Nonetheless, I'll persevere with the attempted removal of the embarrassing typos etc., and try again....

• wg_client v4.16.15
• wg_server v4.16.11

To upgrade use

e  = Exit Script [?]

E:Option ==> uf dev

 

[ZebMcKayhan]

I don't follow what it is that you 'would not consider this to be a bug

The typo errors are of course bugs, which you so nicely reported. I choose not to write more about it than that, you already referred to @Martineau and he knows so it's no point in me repeating it.

But the behavioral part for passthru and "use Main" rules I would not consider a bug. Although now that @Martineau added

E:Option ==> create Samsung-S10 wg21 dns=local

One could argue that it might be broken functionality. But the method for implementation could be many. And for the structural parts we do best at just leave it to @Martineau to decide. I must also admit that my speculations and observations may not always be correct and there is a reasonable chance that I have gotten things backwards as I have not heard many people needing to use these rules (except for when using YazFi, but we cant expect wgm to be aware of all custom local networks we have created). This will not be fool-prof, as combining OpenVPN and Wireguard Policy routing and expect all policy routed clients to be able to communicate amongst each other will be close to impossible. Either each "from xyz use vpn" rule will have to be match with a "to xyz use main" or all various routing information in main routing table needs to be copied over to the policy routing table.

There is a good reason for not copying the entire main routing table for the policy table as wgm needs to be more in control over it than that so we dont get wierd routing paths with us that router sometimes has in the main routing table. it adds all routing path's that points to br0, which makes sense since it is a known local network on all routers. but as wgm creates wg21 it could also be considered a known network but I respect the difficulties since wg11 normally populates the policy table but wg21 might be started later, which means the functionality of adding routes might need to be incorporated into wg21 function and/or passthrough for all policy peers started.

Silently adding companion "To Main" rules quickly creates a messy set of rules, and may only be actually needed in some use cases.

As long as it is easily solved by adding some rules, I'm happy either way @Martineau decides to implement.

 

[archiel]

Abject apologies....I'm sure you've had more than enough of trying to overcome the seemingly never ending wireguard_manager inadequacies within your true Dual-stack IPv4+IPv6 environment, without the added frustrations of my idiotic hastily applied (and totally inexcusable) shoddy typos.

However, not having an IPv6 stack to test with, slightly mitigates the fact that the code execution path cannot be fully tested by me, but it may be why the following appears to be invalid....but works for IPv4?

ip -6 route show cache

ip -6 route flush cache

Consequently, with hindsight, I concur, you are correct to probably eschew the use of the Passthru feature, as it would be preferable to simply have the Road-Warrior 'client' Peer connect directly to the VPN ISP WireGuard 'server' Peer endpoint for better throughput/performance anyway, but I am truly grateful for your diligence and patience when offering your time/resource to improve the script's IPv6 compatibility - despite the many tedious interruptions to service endured by your family.

Nonetheless, I'll persevere with the attempted removal of the embarrassing typos etc., and try again....

• wg_client v4.16.15
• wg_server v4.16.11

To upgrade use

e  = Exit Script [?]

E:Option ==> uf dev

Firstly, thank you and @ZebMcKayhan for bearing with me as I keep trying to see where I can break this. Passthru now works , but not quite as expected - though given @ZebMcKayhan's comments on trying to cover all perms and comms this may well be a result of my particular setup (running unbound and then obscuring the DNS servers via the VPN tunnel).

However, what I have found is that although I just select one device for passthru

peer wg21 passthru add wg11 laptop

and get

Server Auto Subnet Port Annotate
wg21 N 10.50.1.1/24,aa36:7ef1:2add:aa88:100::1/120 11501 # RT-AX88U (IPv4/IPv6) Server 1


Server Client Passthru
wg21 wg11 laptop

any other device connected to wg21 will now also passthru

I can circumvent this by creating a second - no passthru server (wg23), so please feel free to tell me to be happy with what I have (and I am!!), but I thought I should report back. It would probably be helpful if someone else with a less convoluted setup also checks whether single / specified passthru is picking up all the server devices or is working as intended.

 

[ZebMcKayhan]

Firstly, thank you and @ZebMcKayhan for bearing with me as I keep trying to see where I can break this. Passthru now works , but not quite as expected - though given @ZebMcKayhan's comments on trying to cover all perms and comms this may well be a result of my particular setup (running unbound and then obscuring the DNS servers via the VPN tunnel).

However, what I have found is that although I just select one device for passthru

and get

any other device connected to wg21 will now also passthru

I can circumvent this by creating a second - no passthru server (wg23), so please feel free to tell me to be happy with what I have (and I am!!), but I thought I should report back. It would probably be helpful if someone else with a less convoluted setup also checks whether single / specified passthru is picking up all the server devices or is working as intended.

From the look of it, it doesn't seem to be your setup. Is both ipv4 and ipv6 from another device go out wg11 as well?

Perhaps try adding passthrough with ip instead:

E:Option ==> peer wg21 passthru add wg11 ip=10.50.1.2/32 ipv6=aa36:7ef1:2add:aa88:100::2/128

not 100% on the syntax though and assuming laptop is your first wg21 device peer.

Even though your command should have worked the above might bring some more information to the table.

 

[Martineau]

However, what I have found is that although I just select one device for passthru

Server Client Passthru
wg21   wg11   laptop

any other device connected to wg21 will now also passthru

Can you please stop wg11, then start wg11 debug and post the result.

Also can you please PM the contents of the diag command.

 

[archiel]

From the look of it, it doesn't seem to be your setup. Is both ipv4 and ipv6 from another device go out wg11 as well?

Perhaps try adding passthrough with ip instead:

E:Option ==> peer wg21 passthru add wg11 ip=10.50.1.2/32 ipv6=aa36:7ef1:2add:aa88:100::2/128

not 100% on the syntax though and assuming laptop is your first wg21 device peer.

Even though your command should have worked the above might bring some more information to the table.

The issue is on ipv4 and ipv6 and manually adding as above doesn't help (I edited as laptop is the second device) - the first (non-specified) device still gets sent through passthrough

 

[archiel]

Can you please stop wg11, then start wg11 debug and post the result.

Also can you please PM the contents of the diag command.

Somewhere in the middle of adding, removing, modifying, etc I have manged to type the wrong command and SNAFU! Please bear with me while I remove and rebuild my server, client and devices. Normal service will be resumed....

 

[Martineau]

Somewhere in the middle of adding, removing, modifying, etc I have manged to type the wrong command and SNAFU! Please bear with me while I remove and rebuild my server, client and devices. Normal service will be resumed....

Oh dear a bit unfortunate.

EDIT: However, I think I can replicate the issue.. the 'server' Peer IP subnet is always being used to build the RPDB rule, rather than the expected Road-Warrior 'client' Peer IP address

The use of the subnet was a remnant of my previous testing

ip rule
0: from all lookup local
9810: from all fwmark 0xd2 lookup 210
9981: from 10.50.1.1/24 lookup 121
9982: from 10.50.1.1/24 lookup 122
9982: from 10.50.1.1/24 lookup 122
9982: from 192.168.100.2 lookup 122
9982: from 10.50.1.4 lookup 122

I'll take a look tomorrow.

 

[archiel]

Oh dear a bit unfortunate.

EDIT: However, I think I can replicate the issue.. the 'server' Peer IP subnet is always being used to build the RPDB rule, rather than the expected Road-Warrior 'client' Peer IP address

The use of the subnet was a remnant of my previous testing

ip rule
0: from all lookup local
9810: from all fwmark 0xd2 lookup 210
9981: from 10.50.1.1/24 lookup 121
9982: from 10.50.1.1/24 lookup 122
9982: from 10.50.1.1/24 lookup 122
9982: from 192.168.100.2 lookup 122
9982: from 10.50.1.4 lookup 122

I'll take a look tomorrow.

...and we are back

Spoiler: peer wg11

Client  Auto  IP                                        Endpoint                   DNS                                   MTU  Annotate
wg11    P     <vpn_ipv4>/19,         <vpn_ipv6>/64  nl1.wg.azirevpn.net:nnnnn  10.50.60.1,fe80::aa5e:45ff:feae:5050       # N/A

        Selective Routing RPDB rules
ID  Peer  Interface  Source                      Destination  Description
10  wg11  VPN        fd36:7ef1:2add:aa88:100::1  Any          Unbound6VPN
9   wg11  VPN        192.168.3.1                 Any          Unbound4VPN

IPSet     Enable  Peer  FWMark  DST/SRC
wg11-mac  Y       wg11  0x1000  src

Server  Client  Passthru
wg21    wg11    pho21

Spoiler: start wg11 debug

        Requesting WireGuard VPN Peer start (wg11)

        WireGuard-clientwg11: Initialising WireGuard VPN 'client' Peer (wg11) in Policy Mode to nl1.wg.azirevpn.net:51820 (# N/A) DNS=10.50.60.1,fe80::aa5e:45ff:feae:5050
[#] iptables -t nat -N WGDNS1
[#] ip6tables -t nat -N WGDNS1
[#] ip link add dev wg11 type wireguard
[#] wg setconf wg11 /tmp/wg11.10547 #(/opt/etc/wireguard.d/wg11.conf)
[#] ip address add dev wg11 <vpn_ipv4>/19
[#] ip -6 address add dev wg11 <vpn_ipv6>/64
[#] ip link set up dev wg11
[#] ip -6 link set up dev wg11
[#] ifconfig wg11 mtu 1420
[#] ifconfig wg11 txqueuelen 1000
[+] wg11-route-up.sh
[#] ip route add <vpn_endpoint> via <ISP_WAN>
[#] ip rule add from 0/0 fwmark 0x1000/0x1000 table 121 prio 9991
[#] ip -6 rule add from ::/0 fwmark 0x1000/0x1000 table 121 prio 9991
[#] iptables -t mangle -A PREROUTING -m set --match-set wg11-mac src -j MARK --set-mark 0x1000/0x1000 -m comment --comment WireGuard 'client'
[#] ip6tables -t mangle -A PREROUTING -m set --match-set wg11-mac src -j MARK --set-mark 0x1000/0x1000 -m comment --comment WireGuard 'client'
[#] ip route add 0/1 dev wg11 table 121
[#] ip route add 128/1 dev wg11 table 121
[#] ip -6 route add 0::/1 dev wg11 table 121
[#] ip -6 route add 8000::/1 dev wg11 table 121
[#] ip route add table 121 10.0.0.0/8 proto kernel scope link src 10.50.60.10 dev br0
[#] ip route add table 121 10.50.60.0/24 proto kernel scope link src 10.50.60.1 dev br0
[#] ip -6 route add table 121 <LAN IPv6 Address>/56 proto kernel metric 256 pref medium dev br0
[#] ip -6 route add table 121 fe80::/64 proto kernel metric 256 pref medium dev br0
[#] ip rule add from 10.50.1.2/32 table 121 prio 9981
[#] iptables -t nat -I POSTROUTING -s 10.50.1.2/32 -o wg11 -j MASQUERADE
[#] ip -6 rule add from aa36:7ef1:2add:aa88:100::2/128 table 121 prio 9981
[#] ip6tables -t nat -I POSTROUTING -s aa36:7ef1:2add:aa88:100::2/128 -o wg11 -j MASQUERADE
[#] ip route flush cache
[?] ip -6 route flush cache >>>>'Failed to send flush request: No such process'.....ALWAYS FAILS!!!!
[+] wg11-up.sh
[#] iptables -t mangle -I FORWARD -o wg11 -p tcp -m tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu -m comment --comment WireGuard 'client'
[#] iptables -t mangle -I FORWARD -i wg11 -p tcp -m tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu -m comment --comment WireGuard 'client'
[#] iptables -t mangle -I FORWARD -o wg11 -j MARK --set-xmark 0x01/0x7 -m comment --comment WireGuard 'client'
[#] iptables -t mangle -I PREROUTING -i wg11 -j MARK --set-xmark 0x01/0x7 -m comment --comment WireGuard 'client'
[#] iptables -t nat -I POSTROUTING -s 10.50.60.1/24 -o wg11 -j MASQUERADE -m comment --comment WireGuard 'client'
[#] iptables -t nat -I PREROUTING -p tcp -m tcp --dport 53 -j WGDNS1 -m comment --comment WireGuard 'client1 DNS'
[#] iptables -t nat -I PREROUTING -p udp -m udp --dport 53 -j WGDNS1 -m comment --comment WireGuard 'client1 DNS'
[#] ip6tables -t mangle -I FORWARD -o wg11 -p tcp -m tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu -m comment --comment WireGuard 'client'
[#] ip6tables -t mangle -I FORWARD -i wg11 -p tcp -m tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu -m comment --comment WireGuard 'client'
[#] ip6tables -t mangle -I FORWARD -o wg11 -j MARK --set-xmark 0x01/0x7 -m comment --comment WireGuard 'client'
[#] ip6tables -t mangle -I PREROUTING -i wg11 -j MARK --set-xmark 0x01/0x7 -m comment --comment WireGuard 'client'
[#] ip6tables -t nat -I POSTROUTING -s <WAN IPv6 Address>/64 -o wg11 -j MASQUERADE -m comment --comment WireGuard 'client'
[#] ip6tables -t nat -I PREROUTING -p tcp -m tcp --dport 53 -j WGDNS1 -m comment --comment WireGuard 'client1 DNS'
[#] ip6tables -t nat -I PREROUTING -p udp -m udp --dport 53 -j WGDNS1 -m comment --comment WireGuard 'client1 DNS'
        WireGuard-clientwg11: Initialisation complete.

Also while rebuilding wg11.conf (adding the user defined DNS) I noted that it is now retaining the vpn IPv6 DNS

Before

[Interface]
PrivateKey = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx=
Address = <vpn_ipv4>/19, <vpn_ipv6>/64
DNS = <vpn_DNS4>, <vpn_DNS6>

run

peer wg11 dns=10.55.63.1,fe80::aa5e:45ff:feae:5050

After

[Interface]
PrivateKey = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx=
Address = <vpn_ipv4>/19, <vpn_ipv6>/64
DNS = 10.55.63.1,fe80::aa5e:45ff:feae:5050 <vpn_DNS6>

this was not happening before. I am also PMing the diag output

 

[Martineau]

...and we are back

Server Client Passthru
wg21 wg11 pho21[/CODE]

[#] ip rule add from 10.50.1.2/32 table 121 prio 9981
[#] iptables -t nat -I POSTROUTING -s 10.50.1.2/32 -o wg11 -j MASQUERADE
[#] ip -6 rule add from aa36:7ef1:2add:aa88:100::2/128 table 121 prio 9981
[#] ip6tables -t nat -I POSTROUTING -s aa36:7ef1:2add:aa88:100::2/128 -o wg11 -j MASQUERADE

So Road-Warrior 'device' Peer pho21 is both 10.50.1.2/32 and aa36:7ef1:2add:aa88:100::2/128 and from the above, no wg21 subnet has been added to the RPDB rules..... so only the one device (as desired) should be subject to the 'Passthru' ?

Also while rebuilding wg11.conf (adding the user defined DNS) I noted that it is now retaining the vpn IPv6 DNS

Before

[Interface]
PrivateKey = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx=
Address = <vpn_ipv4>/19, <vpn_ipv6>/64
DNS = <vpn_DNS4>, <vpn_DNS6>

run

After

[Interface]
PrivateKey = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx=
Address = <vpn_ipv4>/19, <vpn_ipv6>/64
DNS = 10.55.63.1,fe80::aa5e:45ff:feae:5050 <vpn_DNS6>

Wasn't aware that this was even broken?

this was not happening before. I am also PMing the diag output

No need now....you can just confirm/check

ip rule

ip -6 rule

So if things have magically fixed themselves then that is puzzling indeed, however I have uploaded wireguard_manager Beta v4.16bD

In truth, there are no changes to the code that inserts the Passthru RPDB rules..except it now displays the description tag when the passthru rule is added i.e for your pho21 'device' Peer.

The changes are documented here

Update wg_manager.sh · MartineauUK/wireguard@d83dcc4

FIX: Command 'peer wg21 passthru add wg12 Garbage' would incorrectly add non-existent 'device' Peer 'Garbage' to SQL database. CHANGE: Command 'peer wg2X passthru add...

github.com

and effectively simply enhances Passthru Road-Warrior 'device' Peer validation, and now also prompts to restart any Peer when the Passthru feature has been requested.


EDIT: If the Passthru issue is still present, I can dig deeper but hopefully the Passthru subnet issue has definitely resolved itself?...sadly it is

Doh! - I'll need to investigate and upload a fix to wg_server

I've uploaded a patched wg_server v4.16.12

Upgrade using:

e  = Exit Script [?]

E:Option ==> uf dev

 

[archiel]

So Road-Warrior 'device' Peer pho21 is both 10.50.1.2/32 and aa36:7ef1:2add:aa88:100::2/128 and from the above, no wg21 subnet has been added to the RPDB rules..... so only the one device (as desired) should be subject to the 'Passthru' ?

Wasn't aware that this was even broken?

No need now....you can just confirm/check

ip rule

ip -6 rule

So if things have magically fixed themselves then that is puzzling indeed, however I have uploaded wireguard_manager Beta v4.16bD

In truth, there are no changes to the code that inserts the Passthru RPDB rules..except it now displays the description tag when the passthru rule is added i.e for your pho21 'device' Peer.

The changes are documented here

Update wg_manager.sh · MartineauUK/wireguard@d83dcc4

FIX: Command 'peer wg21 passthru add wg12 Garbage' would incorrectly add non-existent 'device' Peer 'Garbage' to SQL database. CHANGE: Command 'peer wg2X passthru add...

github.com

and effectively simply enhances Passthru Road-Warrior 'device' Peer validation, and now also prompts to restart any Peer when the Passthru feature has been requested.


EDIT: If the Passthru issue is still present, I can dig deeper but hopefully the Passthru subnet issue has definitely resolved itself?...sadly it is

Doh! - I'll need to investigate and upload a fix to wg_server

I've uploaded a patched wg_server v4.16.12

Upgrade using:

e  = Exit Script [?]

E:Option ==> uf dev

just updated

 ip rule
0:      from all lookup local
9810:   from all fwmark 0xd2 lookup 210
9911:   from 192.168.3.1 lookup 121
9981:   from 10.50.1.1/24 lookup 121
9981:   from 10.50.1.1/24 lookup 121
9981:   from 10.50.1.1/24 lookup 121
9981:   from 10.50.1.1/24 lookup 121
9981:   from 10.50.1.1/24 lookup 121
9981:   from 10.50.1.2 lookup 121
9981:   from 10.50.1.1/24 lookup 121
9991:   from all fwmark 0x1000/0x1000 lookup 121
32766:  from all lookup main
32767:  from all lookup default

 ip -6 rule
0:      from all lookup local
9810:   from all fwmark 0xd2 lookup 210
9911:   from fd36:7ef1:2add:aa88:100::1 lookup 121
9981:   from aa36:7ef1:2add:aa88:100::1/120 lookup 121
9981:   from aa36:7ef1:2add:aa88:100::1/120 lookup 121
9981:   from aa36:7ef1:2add:aa88:100::1/120 lookup 121
9981:   from aa36:7ef1:2add:aa88:100::1/120 lookup 121
9981:   from aa36:7ef1:2add:aa88:100::1/120 lookup 121
9981:   from aa36:7ef1:2add:aa88:100::2 lookup 121
9981:   from aa36:7ef1:2add:aa88:100::1/120 lookup 121
9991:   from all fwmark 0x1000/0x1000 lookup 121
32766:  from all lookup main

running start wg11 debug - new error

....
[#] ip route add table 121 10.0.0.0/8 proto kernel scope link src 10.50.60.10 dev br0
[#] ip route add table 121 10.50.60.0/24 proto kernel scope link src 10.50.60.1 dev br0
[#] ip -6 route add table 121 <ISP_IPv6 Addresss> proto kernel metric 256 pref medium dev br0
[#] ip -6 route add table 121 fe80::/64 proto kernel metric 256 pref medium dev br0
grep: /opt/etc/wireguard.d/ip=10.50.1.3/32.conf: No such file or directory
[#] ip route flush cache
....

passthru still applies to all wg11 devices, selected or othersise.

 

[Martineau]

just updated

 ip rule
0:      from all lookup local
9810:   from all fwmark 0xd2 lookup 210
9911:   from 192.168.3.1 lookup 121
9981:   from 10.50.1.1/24 lookup 121
9981:   from 10.50.1.1/24 lookup 121
9981:   from 10.50.1.1/24 lookup 121
9981:   from 10.50.1.1/24 lookup 121
9981:   from 10.50.1.1/24 lookup 121
9981:   from 10.50.1.2 lookup 121
9981:   from 10.50.1.1/24 lookup 121
9991:   from all fwmark 0x1000/0x1000 lookup 121
32766:  from all lookup main
32767:  from all lookup default

 ip -6 rule
0:      from all lookup local
9810:   from all fwmark 0xd2 lookup 210
9911:   from fd36:7ef1:2add:aa88:100::1 lookup 121
9981:   from aa36:7ef1:2add:aa88:100::1/120 lookup 121
9981:   from aa36:7ef1:2add:aa88:100::1/120 lookup 121
9981:   from aa36:7ef1:2add:aa88:100::1/120 lookup 121
9981:   from aa36:7ef1:2add:aa88:100::1/120 lookup 121
9981:   from aa36:7ef1:2add:aa88:100::1/120 lookup 121
9981:   from aa36:7ef1:2add:aa88:100::2 lookup 121
9981:   from aa36:7ef1:2add:aa88:100::1/120 lookup 121
9991:   from all fwmark 0x1000/0x1000 lookup 121
32766:  from all lookup main

running start wg11 debug - new error

....
[#] ip route add table 121 10.0.0.0/8 proto kernel scope link src 10.50.60.10 dev br0
[#] ip route add table 121 10.50.60.0/24 proto kernel scope link src 10.50.60.1 dev br0
[#] ip -6 route add table 121 <ISP_IPv6 Addresss> proto kernel metric 256 pref medium dev br0
[#] ip -6 route add table 121 fe80::/64 proto kernel metric 256 pref medium dev br0
grep: /opt/etc/wireguard.d/ip=10.50.1.3/32.conf: No such file or directory
[#] ip route flush cache
....

passthru still applies to all wg11 devices, selected or othersise.

Does wg_server show version v4.16.12 ?

If not; you need to issue wgm stop; upgrade, then wgm start wg21 and see if ip rule still shows the incorrect subnet RPDB rule.

 

[ZebMcKayhan]

just updated

 ip rule
0:      from all lookup local
9810:   from all fwmark 0xd2 lookup 210
9911:   from 192.168.3.1 lookup 121
9981:   from 10.50.1.1/24 lookup 121
9981:   from 10.50.1.1/24 lookup 121
9981:   from 10.50.1.1/24 lookup 121
9981:   from 10.50.1.1/24 lookup 121
9981:   from 10.50.1.1/24 lookup 121
9981:   from 10.50.1.2 lookup 121
9981:   from 10.50.1.1/24 lookup 121
9991:   from all fwmark 0x1000/0x1000 lookup 121
32766:  from all lookup main
32767:  from all lookup default

 ip -6 rule
0:      from all lookup local
9810:   from all fwmark 0xd2 lookup 210
9911:   from fd36:7ef1:2add:aa88:100::1 lookup 121
9981:   from aa36:7ef1:2add:aa88:100::1/120 lookup 121
9981:   from aa36:7ef1:2add:aa88:100::1/120 lookup 121
9981:   from aa36:7ef1:2add:aa88:100::1/120 lookup 121
9981:   from aa36:7ef1:2add:aa88:100::1/120 lookup 121
9981:   from aa36:7ef1:2add:aa88:100::1/120 lookup 121
9981:   from aa36:7ef1:2add:aa88:100::2 lookup 121
9981:   from aa36:7ef1:2add:aa88:100::1/120 lookup 121
9991:   from all fwmark 0x1000/0x1000 lookup 121
32766:  from all lookup main

running start wg11 debug - new error

....
[#] ip route add table 121 10.0.0.0/8 proto kernel scope link src 10.50.60.10 dev br0
[#] ip route add table 121 10.50.60.0/24 proto kernel scope link src 10.50.60.1 dev br0
[#] ip -6 route add table 121 <ISP_IPv6 Addresss> proto kernel metric 256 pref medium dev br0
[#] ip -6 route add table 121 fe80::/64 proto kernel metric 256 pref medium dev br0
grep: /opt/etc/wireguard.d/ip=10.50.1.3/32.conf: No such file or directory
[#] ip route flush cache
....

passthru still applies to all wg11 devices, selected or othersise.

seems like there are some old routing rules stuck in your routing table.... it may/may not solve your problem, but a reboot might clean things up. you have5 redundant rules which have been added by wgm but never removed, so added again, and again when peers restart (I presume)... since they are not removed they could be carried on passed your upgrade and still cause issues (altough wgm might be ok now).

if you dont want a reboot you could remove all

ip rule del prio 9981
ip -6 rule del prio 9981

until there are none left

and then restart all peers, hopefully this is remnant from some crash or temporarily misbehavior, or before the last update? or it is an active problem.

 

[Martineau]

running start wg11 debug - new error

grep: /opt/etc/wireguard.d/ip=10.50.1.3/32.conf: No such file or directory

Weirdly (given the context of the error message) it seems script wg_client has somehow assumed text string 'ip=10.50.1.3/32' to be a WireGuard interface? such as wg11 and possibly extracted from the SQL database?

Could you please wgm stop wg11 then start it manually in full debug/trace mode using the appropriate command

i.e. If wg11 has auto=p issue

sh -x /jffs/addons/wireguard/wg_client wg11 policy

otherwise issue

sh -x /jffs/addons/wireguard/wg_client wg11

then PM me the top half of the debug trace output upto the point where the error message 'grep: /opt/etc/wireguard.d/ip=10.50.1.3/32.conf: No such file or directory' occurs.

 

[archiel]

Does wg_server show version v4.16.12 ?

If not; you need to issue wgm stop; upgrade, then wgm start wg21 and see if ip rule still shows the incorrect subnet RPDB rule.

if I run wgm I see

Version v4.16bD by Martineau

where can I see wg_server ?

After a reboot (wg11 is set to Policy Mode=P) and wg21 is not set to Auto

 ip rule
0:      from all lookup local
9911:   from 192.168.3.1 lookup 121
9991:   from all fwmark 0x1000/0x1000 lookup 121
32766:  from all lookup main
32767:  from all lookup default

 ip -6 rule
0:      from all lookup local
9911:   from fd36:7ef1:2add:aa88:100::1 lookup 121
9991:   from all fwmark 0x1000/0x1000 lookup 121
32766:  from all lookup main

After starting wg21

 ip rule
0:      from all lookup local
9810:   from all fwmark 0xd2 lookup 210
9911:   from 192.168.3.1 lookup 121
9991:   from all fwmark 0x1000/0x1000 lookup 121
32766:  from all lookup main
32767:  from all lookup default

0:      from all lookup local
9810:   from all fwmark 0xd2 lookup 210
9911:   from fd36:7ef1:2add:aa88:100::1 lookup 121
9991:   from all fwmark 0x1000/0x1000 lookup 121
32766:  from all lookup main

But now passthru was not working for the laptop via specified IP

peer wg21 passthru add wg11 ip=10.50.1.3/32 ipv6=aa36:7ef1:2add:aa88:100::3/128

or any other device
removing the above and replacing with a named device

peer wg21 passthru del all
peer wg21 passthru add wg11 pho21

then pho21 had passthru and laptop did not. Reversing the setup

peer wg21 passthru del all
peer wg21 passthru add wg11 laptop

left both devices with passthru and

 ip rule
0:      from all lookup local
9810:   from all fwmark 0xd2 lookup 210
9911:   from 192.168.3.1 lookup 121
9981:   from 10.50.1.2 lookup 121
9981:   from 10.50.1.3 lookup 121
9991:   from all fwmark 0x1000/0x1000 lookup 121
32766:  from all lookup main
32767:  from all lookup default

 ip -6 rule
0:      from all lookup local
9810:   from all fwmark 0xd2 lookup 210
9911:   from fd36:7ef1:2add:aa88:100::1 lookup 121
9981:   from aa36:7ef1:2add:aa88:100::2 lookup 121
9981:   from aa36:7ef1:2add:aa88:100::3 lookup 121
9991:   from all fwmark 0x1000/0x1000 lookup 121
32766:  from all lookup main

Removing the rules for pho21 (from 10.50.1.2 lookup 121 & from aa36:7ef1:2add:aa88:100::2 lookup 121)
removed pho21 from passthru as expected.

So it seem that adding a device adds the required rules but removing it (or at least removing all) leaves the rules intact.

 

[Martineau]

where can I see wg_server ?

Wireguard - Session Manager - Discussion (3rd) thread

another 6 months are up so, details of @Martineau's WireGuard manager can be found here and @ZebMcKayhan's invaluable setup guide here. Following on from #938 in the second thread Having only a single computer over vpn could prove difficult with ipv6. ideally you should make an ipv6 rule for...

www.snbforums.com