WireGuard Client & WireGuard Server/InstantGuard at the same time?

[ZebMcKayhan]

Probably relevant but can't understand any of the Google results

I remember seing something like this trying force route a packet over Wireguard while destination not included in AllowedIPs.

Could you confirm Avahi registers on wgs1 network upon Avahi start/restart? Check syslog.

Try to add multicast address to AllowedIPs (server) on your wg server for your client (VPN -> VPN Server -> Others -> Wireguard -> select client info -> more settings for site-2-site usage -> AllowedIPs (server)).
This is probably

10.6.0.2/32

try adding multicast address here as well:

10.6.0.2/32, 224.0.0.251/32

As this destination is also reachable over the tunnel altough not explicitly but perhaps Wireguard doesn't understand any better. It would explain why multicast doesn't make it across the tunnel.
You will need to restart wg server after this to apply the change.

 

[privacyguy123]

I remember seing something like this trying force route a packet over Wireguard while destination not included in AllowedIPs.

Could you confirm Avahi registers on wgs1 network upon Avahi start/restart? Check syslog.

Try to add multicast address to AllowedIPs (server) on your wg server for your client (VPN -> VPN Server -> Others -> Wireguard -> select client info -> more settings for site-2-site usage -> AllowedIPs (server)).
This is probably

10.6.0.2/32

try adding multicast address here as well:

10.6.0.2/32, 224.0.0.251/32

As this destination is also reachable over the tunnel altough not explicitly but perhaps Wireguard doesn't understand any better. It would explain why multicast doesn't make it across the tunnel.
You will need to restart wg server after this to apply the change.

I'm seeing it register with the wgs1 adapter in the log yes, and on the avahi-daemon --debug output also.

I have both my internal VPN Client and Server WireGuard configs set to 0.0.0.0/0 ... would this cause any issue? Surely I don't need to specify multicast addresses if I tell everything to go through the tunnel.

 

[privacyguy123]

Oh I see the option menu you mean now. Adding that to server config gives this new error ...

sendmsg() to 224.0.0.251 failed: Destination address required
sendmsg() to 224.0.0.251 failed: Destination address required
sendmsg() to 224.0.0.251 failed: Destination address required

 

[ZebMcKayhan]

I have both my internal VPN Client and Server WireGuard configs set to 0.0.0.0/0 ... would this cause any issue? Surely I don't need to specify multicast addresses if I tell everything to go through the tunnel.

AllowedIPs on either side is not only used for internal Wireguard routing, the firmware (or whatever app setting up Wireguard) uses this to setup routes.

The client can be set AllowedIPs to 0.0.0.0/0 if you want ALL data to go over VPN. For a typicall client that would be wanted in most cases. This setting has no meaning on the server side except to add it to the generated config for the client. So any update to this setting won't do anything until you generate and import the new config file on your client.

But the AllowedIPs (server) shall NOT be 0.0.0.0/0. You wouldn't want your router default route to be to wgs1. It would basically mean all your lan data to internet is sent via wgs1 client. It would likely break thing and not work as you intended. Hopefully the firmware caught this and refuse to add this route as I havnt heard you complaining about broken internet.

AllowedIPs (server) should be DESTINATIONS reached over vpn and on wgs1, client 1 there is only 1 ip, namely the client ip: 10.6.0.2/32. Only packets with this ip as destination should be routed to this peer. Usually unless tweaked if you are running multple sites so this peer is another router with another lan that you wish to connect to via this peer. Hence the site-2site hint on the setting. If this is not set correctly you would cause routing conflicts in your router and wg may not be able to send the right packets to the right peer.

 

[privacyguy123]

This new error also in the logs

avahi-daemon[3124]: IP_DROP_MEMBERSHIP failed: No such device

 

[ZebMcKayhan]

This new error also in the logs

I'm running out of ideas...

Perhaps you need to restart Avahi after you restarted wgs1??
Perhaps firmware tries to add 224.0.0.251/32 to ip route table (I tested adding it manually and it worked in the sense it shows in the route table). Multicast is not routed this way but wonder if it breaks something?
It is possible to update AllowedIPs after the firmware have setup the routes, so it wouldn't add this (see my previous link how I added Endpoint to a server config). But its far fetched.

 

[privacyguy123]

I'm running out of ideas...

Perhaps you need to restart Avahi after you restarted wgs1??
Perhaps firmware tries to add 224.0.0.251/32 to ip route table (I tested adding it manually and it worked in the sense it shows in the route table). Multicast is not routed this way but wonder if it breaks something?
It is possible to update AllowedIPs after the firmware have setup the routes, so it wouldn't add this (see my previous link how I added Endpoint to a server config). But its far fetched.

Adding 192.168.50.0/24 to AllowedIPs makes this error go away it seems ... I guess that is the destination it needs.

Here is avahi-daemon --debug output, still no devices showing up on my phone tunneled in though :|

Daemon already running on PID 10143
alex@RT-AX58U-5468:/jffs/scripts# avahi-daemon --kill
alex@RT-AX58U-5468:/jffs/scripts# avahi-daemon --debug
Found user 'nobody' (UID 65534) and group 'nogroup' (GID 5).
Successfully dropped root privileges.
avahi-daemon 0.8 starting up.
No service file found in /opt/etc/avahi/services.
Joining mDNS multicast group on interface wgc1.IPv4 with address 10.2.0.2.
New relevant interface wgc1.IPv4 for mDNS.
Joining mDNS multicast group on interface wgs1.IPv4 with address 10.6.0.1.
New relevant interface wgs1.IPv4 for mDNS.
Joining mDNS multicast group on interface br0.IPv4 with address 192.168.50.1.
New relevant interface br0.IPv4 for mDNS.
Joining mDNS multicast group on interface lo.IPv4 with address 127.0.1.1.
New relevant interface lo.IPv4 for mDNS.
Network interface enumeration completed.
Registering new address record for 10.2.0.2 on wgc1.IPv4.
Registering new address record for 10.6.0.1 on wgs1.IPv4.
Registering new address record for 192.168.50.1 on br0.IPv4.
Registering new address record for 127.0.1.1 on lo.IPv4.
Registering new address record for 127.0.0.1 on lo.IPv4.
Registering HINFO record with values 'ARMV7L'/'LINUX'.
Server startup complete. Host name is RT-AX58U-5468.local. Local service cookie is 3529910932.

Some more info I found here that needs translated to beginner lol

https://www.reddit.com/r/WireGuard/comments/15qpk46

VXLAN & Linux

VXLAN is an overlay network for L2 traffic over an existing IP network. Let's explore how to configure it on Linux.

vincent.bernat.ch

 

[ZebMcKayhan]

Adding 192.168.50.0/24 to AllowedIPs makes this error go away it seems ... I guess that is the destination it needs.

Adding 192.168.50.0/24 to AllowedIPs (server) would create a routing conflict with your lan. Altough as the route already exist the firmware may fail to add it to the routing table. Dont understand why it would be needed though. Packets with those ips are not available on the other side of the tunnel.

 

[ZebMcKayhan]

Here is avahi-daemon --debug output, still no devices showing up on my phone tunneled in though :|

Are you sure the problem isn't on the client side, somewere between the app and the wireguard interface? Could you sniff with tcpdump or similar to see the mDNS packets on either side?

 

[privacyguy123]

I think I'm wrong with the 192.168.50.0 thing ... error has randomly went away and came back, not sure how to reproduce ...

Digging deeper into these writeups, I decided to try igmpproxy, which requires mcpd to be killed (?)

Here is the output, similar to what avahi reflector is complaining about

igmpproxy -d -v /tmp/mnt/USB/entware/etc/igmpproxy.conf
adding VIF, Ix 0 Fl 0x0 IP 0x0132a8c0 br0, Threshold: 1, Ratelimit: 0
adding VIF, Ix 1 Fl 0x0 IP 0x0100060a wgs1, Threshold: 1, Ratelimit: 0
Joining group 224.0.0.2 on interface wgs1
Joining group 224.0.0.22 on interface wgs1
sendto to 224.0.0.1 on 10.6.0.1; Errno(126): Required key not available

Inserted route table entry for 239.255.255.250 on VIF #-1
RECV V2 member report from 192.168.50.63 to 239.255.255.250
Mebership report was received on the upstream interface. Ignoring.
sendto to 224.0.0.1 on 10.6.0.1; Errno(126): Required key not available

 

[ZebMcKayhan]

Digging deeper into these writeups, I decided to try igmpproxy, which requires mpcd to be killed (?)

Here is the output, similar to what avahi reflector is complaining about

We only added multicast ip for mDNS to AllowedIP (server). Not 224.0.0.1, 224.0.0.2 or 224.0.0.22. But you could change that to entire multicast (local network) range: 224.0.0.0/24 to also allow for whatever this is.

 

[privacyguy123]

We only added multicast ip for mDNS to AllowedIP (server). Not 224.0.0.1, 224.0.0.2 or 224.0.0.22. But you could change that to entire multicast (local network) range: 224.0.0.0/24 to also allow for whatever this is.

Perhaps you notice something wrong in my igmpproxy.conf?

quickleave

phyint br0 upstream ratelimit 0 threshold 1
altnet 192.168.0.0/24
altnet 10.6.0.0/24

phyint wgs1 downstream ratelimit 0 threshold 1
altnet 10.6.0.0/24

Adding multicast range to WG Server gets us further with more errors - I see the 10.6.0.2 server IP trying to do something here at least ...

Removing MFC: 10.6.0.2 -> 239.255.255.250, InpVIf: 0
MRT_DEL_MFC; Errno(2): No such file or directory
Inserted route table entry for 239.255.255.250 on VIF #-1
RECV V2 member report from 192.168.50.71 to 239.255.255.250
Mebership report was received on the upstream interface. Ignoring.
Removing MFC: 192.168.50.63 -> 239.255.255.250, InpVIf: 0
MRT_DEL_MFC; Errno(2): No such file or directory
Removing MFC: 192.168.50.71 -> 239.255.255.250, InpVIf: 0
MRT_DEL_MFC; Errno(2): No such file or directory
Inserted route table entry for 239.255.255.250 on VIF #-1
RECV V2 member report from 192.168.50.63 to 239.255.255.250
Mebership report was received on the upstream interface. Ignoring.
^Cselect() failure; Errno(4): Interrupted system call
Got a interrupt signal. Exiting.
Removing MFC: 192.168.50.71 -> 239.255.255.250, InpVIf: 0
MRT_DEL_MFC; Errno(2): No such file or directory
All routes removed. Routing table is empty.
Shutdown complete....

 

[privacyguy123]

I don't know man, I see post after post after post outlining the steps that they got this thing working just fine. (https://www.linksysinfo.org/index.p...figuring-a-reflector-aka-mdns-repeater.75706/) I guess none of them state they are using an Asus router, so whether it's down to how the firmware is set up on these routers OR the changes that Merlin has possibly made I think it could be time to take this back to the shop ... if other routers are able to support this kind of set up then I don't need this thing I need one of them.

Trying a full nuke this weekend to rule out any glitches ... if you're all out of ideas it's getting packaged back up.

 

[privacyguy123]

Multicast forwading
With IGMP snooping, multicast forwarding is disabled for bridges. One pure bridge solution is to disable multicast_snooping.

? I have this on, as it's turned on by default. Do you think this plays a part?

 

[privacyguy123]

/var/mcpd.conf - the in built IGMP proxy, seems to be on permanently regardless of settings - something in here needs tweaked perhaps?

igmp-query-response-interval 100
igmp-last-member-query-interval 10
igmp-robustness-value 2
igmp-max-groups 25
igmp-max-sources 25
igmp-max-members 25
igmp-fast-leave 1
igmp-admission-required 0
igmp-admission-bridging-filter 0
igmp-proxy-interfaces eth4
igmp-snooping-interfaces br0
igmp-mcast-interfaces eth4
#
#Begin MLD configuration
#
mld-default-version 2
mld-query-interval 125
mld-query-response-interval 10
mld-last-member-query-interval 10
mld-robustness-value 2
mld-max-groups 10
mld-max-sources 10
mld-max-members 10
mld-fast-leave 1
mld-admission-required 0
mld-admission-bridging-filter 0
mld-proxy-enable 1
mld-snooping-enable 1
mld-proxy-interfaces eth4
mld-snooping-interfaces br0
mld-mcast-interfaces eth4
#
#End MLD configuration
#
##### MCAST configuration #####
igmp-mcast-snoop-exceptions 239.255.255.250/255.255.255.255 224.0.255.135/255.255.255.255
mld-mcast-snoop-exceptions ff05::0001:0003/ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff

 

[privacyguy123]

Today I found "pimd" which also claims to fix my issue and doesn't work. truly mindblowing.

Candidate Rendezvous-Point Set ===============================================
RP address Incoming Group Prefix Priority Holdtime
--------------- -------- ------------------ -------- ---------------------
192.168.50.1 4 224/4 1 65535
169.254.0.1 0 232/8 1 65535
------------------------------------------------------------------------------
Current BSR address: 192.168.50.1

10:25:53.078 Received IGMP v2 Membership Report from 192.168.50.63 to 224.0.0.251
10:25:53.078 accept_group_report(): igmp_src 192.168.50.63 ssm_src 224.0.0.251 group 224.0.0.251 report_type 22
10:25:53.078 accept_group_report(): al_pv=3
10:25:53.078 Change IGMP compatibility mode to v2 for group 224.0.0.251
10:25:53.078 Set delete timer for group: 224.0.0.251
10:25:53.078 Not creating routing entry for LAN scoped group 224.0.0.251
10:25:53.680 Cache miss, src 192.168.50.71, dst 239.255.255.250, iif 1
10:25:53.680 create group entry, group 239.255.255.250
10:25:53.680 create source entry, source 192.168.50.71
10:25:53.680 move_kernel_cache: SG
10:25:54.340 Received IGMP v2 Membership Report from 192.168.50.71 to 239.255.255.250
10:25:54.340 accept_group_report(): igmp_src 192.168.50.71 ssm_src 239.255.255.250 group 239.255.255.250 report_type 22
10:25:54.340 accept_group_report(): al_pv=2
10:25:54.340 Set delete timer for group: 239.255.255.250
10:25:54.340 Adding vif 1 for group 239.255.255.250
10:25:55.184 Received IGMP v3 Membership Report from 192.168.50.100 to 224.0.0.22
10:25:55.184 accept_membership_report(): IGMP v3 report, 32 bytes, from 192.168.50.100 to 224.0.0.22 with 3 group records.
10:25:55.184 accept_group_report(): igmp_src 192.168.50.100 ssm_src 0.0.0.0 group 239.192.0.199 report_type 34
10:25:55.184 Set delete timer for group: 239.192.0.199
10:25:55.184 SM group order from 192.168.50.100 (*,239.192.0.199)
10:25:55.184 create group entry, group 239.192.0.199
10:25:55.184 Adding vif 1 for group 239.192.0.199
10:25:55.184 accept_group_report(): igmp_src 192.168.50.100 ssm_src 0.0.0.0 group 224.0.0.199 report_type 34
10:25:55.184 Set delete timer for group: 224.0.0.199
10:25:55.184 SM group order from 192.168.50.100 (*,224.0.0.199)
10:25:55.184 Not creating routing entry for LAN scoped group 224.0.0.199
10:25:55.184 accept_group_report(): igmp_src 192.168.50.100 ssm_src 0.0.0.0 group 224.0.0.251 report_type 34
10:25:55.184 Set delete timer for group: 224.0.0.251
10:25:55.184 Not creating routing entry for LAN scoped group 224.0.0.251

 

[privacyguy123]

iperf -u -s -l 1000 -i 1 -B 224.0.0.251
------------------------------------------------------------
Server listening on UDP port 5001
Joining multicast group 224.0.0.251
Server set to single client traffic mode (per multicast receive)
UDP buffer size: 512 KByte (default)
------------------------------------------------------------
Multicast not supported

???

 

[privacyguy123]

Iconsistencies here with which multicast groups are joined (I don't use ipv6 for the record)

br0
link 33:33:00:00:00:01
link 33:33:00:00:00:02
link 01:00:5e:00:00:01
link 01:00:5e:00:00:16
link 33:33:00:00:00:16
link 01:00:5e:00:00:fb
inet 224.0.0.251
inet 224.0.0.22
inet 224.0.0.1 users 2
inet6 ff02::16
inet6 ff02::2 users 2
inet6 ff02::1
inet6 ff01::1
31: wgs1
inet 224.0.0.251
inet 224.0.0.1
inet6 ff02::1
inet6 ff01::1
32: wgc1
inet 224.0.0.251
inet 224.0.0.1
inet6 ff02::1
inet6 ff01::1

 

[ZebMcKayhan]

I have this on, as it's turned on by default. Do you think this plays a part?

This is only for IPTV forwarding mcast between wan and a specific lan port if I remember correctly. These setting will be input to mcpd.conf (among other things). You could turn it on or off and see how mcpd.conf changes. Very little info about mscd online, it seems to be Asus/Broadcom own module.

/var/mcpd.conf - the in built IGMP proxy, seems to be on permanently regardless of settings - something in here needs tweaked perhaps?

Perhaps its settings could be tweaked to work between wgs1 and br0 instead of WAN to EthX. But I can hardly find any info at all on it.

Today I found "pimd" which also claims to fix my issue and doesn't work. truly mindblowing.

It may not be that easy to just install and run 3rd party sw on our routers. There may be other issues unrelated to your current.

Again, are you sure your issue is on the router side and not on the client side?
I tried "Service Browser" on Android and disconnected from wifi and joined my router via wifi and the app wouldn't even start. It refused to do anything if I'm not connected to wifi network. I know this is not the case for you but your app may refuse to use the vpn interface. How are you sure this part is working?

 

[privacyguy123]

This is only for IPTV forwarding mcast between wan and a specific lan port if I remember correctly. These setting will be input to mcpd.conf (among other things). You could turn it on or off and see how mcpd.conf changes. Very little info about mscd online, it seems to be Asus/Broadcom own module.

Should it be off or on for this setup then? Along with IGMP Snooping?

Again, are you sure your issue is on the router side and not on the client side?
I tried "Service Browser" on Android and disconnected from wifi and joined my router via wifi and the app wouldn't even start. It refused to do anything if I'm not connected to wifi network. I know this is not the case for you but your app may refuse to use the vpn interface. How are you sure this part is working?

Service Browser app (if it's the same one) shows nothing which is just the same as everything else: not working. Is there some underlying problem with Android not working in this kind of set up then? :S

My guess is router level because no device other than Windows can ping eachothers .local address no matter what setting I mess with.