Solved No ping on WG - BUG?

[Mikii]

Can someone explain this? Or is this a bug?

config: ASUSWRT-Merlin RT-AX88U_PRO 3004.388.8_2 Wed Jul 31 23:48:50 UTC 2024

I have two working wg interfaces:

Michele@RT-AX88U_Pro-D228:/tmp/home/root# wg

interface: wgc3

public key: xxx

private key: (hidden)

listening port: 58933



peer: xxxx

endpoint: 64.156.192.22:1443

allowed ips: 0.0.0.0/0

latest handshake: 1 minute, 27 seconds ago. (sec:87)

transfer: 30.09 KiB received, 22.44 KiB sent

persistent keepalive: every 25 seconds


interface: wgc4

public key: xxx

private key: (hidden)

listening port: 46061



peer: xxxxx

endpoint: 206.217.216.11:1337

allowed ips: 0.0.0.0/0

latest handshake: 1 minute, 33 seconds ago. (sec:93)

transfer: 97.77 KiB received, 30.85 KiB sent

persistent keepalive: every 25 seconds

Michele@RT-AX88U_Pro-D228:/tmp/home/root#

after a fresh reboot, devices connected over Ethernet are working fine when directed to any of the two WireGuard interfaces.

However, if I try to run

Michele@RT-AX88U_Pro-D228:/tmp/home/root# ping -I wgc4 1.1.1.1


PING 1.1.1.1 (1.1.1.1): 56 data bytes

^C

--- 1.1.1.1 ping statistics ---

6 packets transmitted, 0 packets received, 100% packet loss

But, if I also activate any openvpn, and I ping again via wgc4 I get:

Michele@RT-AX88U_Pro-D228:/tmp/home/root# ping -I wgc4 1.1.1.1

PING 1.1.1.1 (1.1.1.1): 56 data bytes

64 bytes from 1.1.1.1: seq=0 ttl=55 time=85.008 ms

64 bytes from 1.1.1.1: seq=1 ttl=55 time=84.396 ms

64 bytes from 1.1.1.1: seq=2 ttl=55 time=89.815 ms

64 bytes from 1.1.1.1: seq=3 ttl=55 time=84.243 ms

^C

--- 1.1.1.1 ping statistics ---

4 packets transmitted, 4 packets received, 0% packet loss

round-trip min/avg/max = 84.243/85.865/89.815 ms

Is this a bug?

Can someone replicate this?

Do you have any suggestion (apart from starting and stopping the openvpn at every boot?)

thank you.

 

[eibgrad]

Just a guess (since I don't have access to WG on my RT-AC68U).

What you've apparently done by having 0.0.0.0/0 for multiple WG connections is create *multiple* default gateways! And any WG connection that becomes the default gateway will eventually become unreachable as soon as the next WG connection gets established and makes itself the new default gateway!

I suspect when you establish OpenVPN, you're using the VPN Director, which causes the default gateway to return to the WAN, and now access to each WG connection becomes reachable again because neither is capable of becoming the default gateway.

In short, you can't create a situation where you have multiple default gateways. Only *one* can serve as a default gateway. All others must be specific as to what is allowed as a destination IP over that WG connection.

P.S. Even if it worked, you'd probably end up w/ nested WG connections, which is NOT good.

 

[Mikii]

Thank you very much for your explanation... I'll investigate the *multiple* default gateways issue, but how does this explain the fact that all the devices that I connect over the Ethernet work fine with the two different WG? I am bit lost here....

 

[eibgrad]

I don't understand the distinction you're making wrt "over the Ethernet". All I see is a failure to access one of the WG connections when ping is told to use one specifically. That's all I can see.

Once you mess up the routing system w/ multiple default gateway, ALL bets are off. I can't even be sure which WG connection got established first, and perhaps had the other nested within it. Then you added the fact that OpenVPN changed things, but without any information as to client or server, if client, how it was configured, etc.

IOW, there's simply too few details within a configuration that is NOT correct to begin with to explain what's happening. FIRST fix the multiple default gateway problem, THEN reevaluate the situation.

 

[Mikii]

hi. I disabled one of the two WG connection, and rebooted, but the problem is still there, with the same behaviour....
I'll keep investigating.... Thanks for your comments, though.

 

[eibgrad]

You could very well have multiple problems, but at least now you've eliminated one of them, so we have something much more predictable and potentially explainable.

"after a fresh reboot, devices connected over Ethernet are working fine when directed to any of the two WireGuard interfaces."

I'm going to venture a guess that what you mean by the above is, certain LAN devices behind the router are bound to one or the other WG connections using the VPN Director, and they work as expected. But when you're connected to the router itself (e.g., ssh), the *router* can NOT ping the individual network interface(s) of the WG clients.

If that's the case, it just wasn't clear from your original description.

And if it is the case, now explain exactly how you configured the OpenVPN client such that it is making it possible to finally access the WG connections while on the router.

 

[Mikii]

Yes, that is what I meant. Certain LAN devices behind the router are bound to one or the other WG connections using the VPN Director, and they work as expected. My apology for not bing clear, English is a second language to me.

 

[ZebMcKayhan]

Do you have any suggestion (apart from starting and stopping the openvpn at every boot?)

Turn off rp-filter (or set it to loose) and I believe you should get a response, but I'm not sure it will actually do what you want.

echo 2 > /proc/sys/net/ipv4/conf/wgc3/rp_filter
echo 2 > /proc/sys/net/ipv4/conf/wgc4/rp_filter

Frankly, I don't see how this would ever work as I assume there are no interfaces local to the router set to use any wgc gateway. WG is vpn-director only.

An idea is to set each local wg interface IP in vpn director to use respective interface. Not the endpoint, the local interface IP you get from

ifconfig wgc3
Ifconfig wgc4

Or look in the webgui under each client. Then setup a vpn-director rule for each ip as local IP and set to use it's own interface.

 

[Mikii]

Turn off rp-filter (or set it to loose) and I believe you should get a response, but I'm not sure it will actually do what you want.

echo 2 > /proc/sys/net/ipv4/conf/wgc3/rp_filter
echo 2 > /proc/sys/net/ipv4/conf/wgc4/rp_filter

Frankly, I don't see how this would ever work as I assume there are no interfaces local to the router set to use any wgc gateway. WG is vpn-director only.

An idea is to set each local wg interface IP in vpn director to use respective interface. Not the endpoint, the local interface IP you get from

ifconfig wgc3
Ifconfig wgc4

Or look in the webgui under each client. Then setup a vpn-director rule for each ip as local IP and set to use it's own interface.

Thank you so much @ZebMcKayhan !

Actually, your rp-filter suggestion works just fine! After

echo 2 > /proc/sys/net/ipv4/conf/wgc3/rp_filter,
a

ping -I wgc3 8.8.8.8

will work!

the mystery is why turning any openvpn connection on and off would also fix my problem. I suspect this has to do with the "accept DNS settings" of the openVPN ?

 

[Mikii]

Interesting, I got this behaviour:

If you boot up with only WireGuard active and no openvpn, you have:

cat /proc/sys/net/ipv4/conf/wgc4/rp_filter
1

however, if you also activate any openvopn

service start_vpnclient2

you then have:

cat /proc/sys/net/ipv4/conf/tun12/rp_filter
0

and also:

cat /proc/sys/net/ipv4/conf/wgc4/rp_filter
0

and this sticks even if you turn off the previous vpn:

service stop_vpnclient2
cat /proc/sys/net/ipv4/conf/wgc4/rp_filter
0

 

[ZebMcKayhan]

Interesting, I got this behaviour:

Wierd, that means openvpn changes

/proc/sys/net/ipv4/conf/all/rp_filter

Or possibly

/proc/sys/net/ipv4/conf/default/rp_filter

And then restarts wgc interfaces.

The reverse-path filtering is a security feature on the router and if a specific feature/function won't work unless you turn it off it means you are probably doing something wrong. However, it's fairly common that people turn it off and it's not a big deal.

Having vpn-director and policy based routing increases flexibility but it also complicates use. These usually go hand-in-hand.

In your case, you have atleast 3 different paths to internet, wan, wgc3 & wgc4. The 2 latter are only used when rules in vpndirector matches. And I guess you did not add wgc3 and wgc4 own addresses in there. This is normally OK, but it means the router itself cannot really use these interfaces (only to own subnet).

So, what ping does, it sets source interface to what you request and thereby manages to get this routed out wgc3/4 (or not) but when a reply comes, the rp-filter will change place on source and destination if and ip to see if a reply to this packet would get routed out the same way it came in. Clearly it would not, so it will be blocked. Whither or not the ping was ever sent is not really relevant.

I have not tried this myself but I would assume if you add wgc3,4 ip to vpndirector and set to each respective interface would just solve your problems and you could leave rp-filter enabled (that is until you enable openvpn, appearantly).

 

[Mikii]

Wierd, that means openvpn changes

/proc/sys/net/ipv4/conf/all/rp_filter

Or possibly

/proc/sys/net/ipv4/conf/default/rp_filter

And then restarts wgc interfaces.

The reverse-path filtering is a security feature on the router and if a specific feature/function won't work unless you turn it off it means you are probably doing something wrong. However, it's fairly common that people turn it off and it's not a big deal.

Having vpn-director and policy based routing increases flexibility but it also complicates use. These usually go hand-in-hand.

In your case, you have atleast 3 different paths to internet, wan, wgc3 & wgc4. The 2 latter are only used when rules in vpndirector matches. And I guess you did not add wgc3 and wgc4 own addresses in there. This is normally OK, but it means the router itself cannot really use these interfaces (only to own subnet).

So, what ping does, it sets source interface to what you request and thereby manages to get this routed out wgc3/4 (or not) but when a reply comes, the rp-filter will change place on source and destination if and ip to see if a reply to this packet would get routed out the same way it came in. Clearly it would not, so it will be blocked. Whither or not the ping was ever sent is not really relevant.

I have not tried this myself but I would assume if you add wgc3,4 ip to vpndirector and set to each respective interface would just solve your problems and you could leave rp-filter enabled (that is until you enable openvpn, appearantly).

Hi @ZebMcKayhan,

your suggestion sounds very interesting, I had not thought of that.

If I understand correctly, you are suggesting to do something like this on vpndirector:

VPN Director Custom Configuration

Interface: WGC3
Description     test
Local IP    192.168.1.1
Remote IP    <the VPN external IP>

If this is the suggestion, this does not seem to be working...

 

[ZebMcKayhan]

If I understand correctly, you are suggesting to do something like this on vpndirector:

No, not really like this.

If I go into my wgc3 in the gui, I can find the wgc3 interface address:

Disregard the ipv6. My wgc3 interface Address is 10.0.186.203

So my vpndirector rule would be:

Interface: WGC3
Description     test
Local IP    10.0.186.203
Remote IP    <leave blank>

 

[Mikii]

ah I see... thanks but it does not seem to work.

I will stick with your echo 2 > /proc/sys/net/ipv4/conf/wgc3/rp_filter which is working just fine.

What I could do is to reset rp_filter to 1 after I run my script, but since this is on a crontab, I would not want risk affecting the wgc3 connection in the process... I believe that setting rp_filter to 2 at boot is something I can live with. Probably all those using an openvpn connection have the same values without even knowing it.

Cheers.

 

[ZebMcKayhan]

ah I see... thanks but it does not seem to work.

That's to bad, I would assume the kernel to fill-in both if and ip, but maybe not. You may then succeed by specifying source ip instead of interface:

ping -I 10.0.186.203 8.8.8.8

It should provide the same result if your vpndirector rule is in place. As wg interface IP never changes it would be fairly future proof, unless you upload a new config.

However, it may make you less secure about what interface you are actually testing?

I will stick with your echo 2 > /proc/sys/net/ipv4/conf/wgc3/rp_filter which is working just fine.

Sure, up to you.

What I could do is to reset rp_filter to 1 after I run my script, but since this is on a crontab, I would not want risk affecting the wgc3 connection in the process... I believe that setting rp_filter to 2 at boot is something I can live with. Probably all those using an openvpn connection have the same values without even knowing it.

You could set it to 2 in wgclient-start when each wg client is started.

 

[ZebMcKayhan]

ah I see... thanks but it does not seem to work.

I had to test for myself and it does indeed work for me, with strict rp_filtering:

admin@RT-AX86U_Pro:/proc/sys/net/ipv4/conf/wgc2# ping -I wgc2 1.1.1.1 -c1
PING 1.1.1.1 (1.1.1.1): 56 data bytes
^C
--- 1.1.1.1 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss

admin@RT-AX86U_Pro:/proc/sys/net/ipv4/conf/wgc2# ip rule add from 10.0.186.203 lookup wgc2 prio 11300 #vpndirector rule
admin@RT-AX86U_Pro:/proc/sys/net/ipv4/conf/wgc2# ping -I wgc2 1.1.1.1 -c1
PING 1.1.1.1 (1.1.1.1): 56 data bytes
64 bytes from 1.1.1.1: seq=0 ttl=56 time=119.398 ms

--- 1.1.1.1 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 119.398/119.398/119.398 ms

Is it possible you forgot to hit "apply" on vpndirector page after adding the rule? Happens to me all the time...

 

[Mikii]

Yes, using the interface ip address in ping does work! thank you again.

and also this works for me:
ip rule del from 10.13.128.229 lookup wgc3 prio 11300

however, setting the rule via the vpndirector web interface does not work. Have you tried that?
Thanks again for helping me to fix this, I appreciate your time and patience.

 

[ZebMcKayhan]

Yes, using the interface ip address in ping does work! thank you again.

You just need to make sure it's going out your intended interface. Without your rule the ping will be successful but actually over WAN interface.

however, setting the rule via the vpndirector web interface does not work. Have you tried that?

No, I have not. When doing so, it restarts all wgc Interface and there will be instant screams from family. But it should produce the same rule altough possibly with a different prio number. As I don't have any other rules that covers this interface IP the prio order should not matter for me. But perhaps it does for you if you have some funky rules?
If you post them, I could take a look:

ip rule

 

[Mikii]

this is what i have now, but I need to do some clean-up I do not need them all:

(mybe I got the syntax wrong on the web interface for vpndirector? I will try and invert it when I have time.)

thanks.

ip rule

0:    from all lookup local


11300:    from 10.13.128.229 lookup wgc3


11300:    from 10.13.128.105 lookup wgc4


11610:    from 192.168.1.128 lookup wgc3


11611:    from 192.168.1.241 lookup wgc3


11612:    from 192.168.1.13 lookup wgc3


11613:    from all to 10.13.128.229 lookup wgc3


11810:    from 192.168.1.100 lookup wgc4


11811:    from 192.168.1.14 lookup wgc4


12218:    from 192.168.1.100 prohibit


12218:    from 192.168.1.14 prohibit


32766:    from all lookup main


32767:    from all lookup default

Edit: Stupid me, it works with the correct setup even on vpn director..... thanks again!

 

[ZebMcKayhan]

Edit: Stupid me, it works with the correct setup even on vpn director..... thanks again!

11613: from all to 10.13.128.229 lookup wgc3

Thats great!
so, just for my understanding, the 11613 rule was your VPN Director attempt, which appears to have the wgc3 ip in the "Remote IP" box in vpn-director which was why it did not work for you. when you moved it to "Local IP" box the rules looks the same and it works the same?

This is good information and will hopefully help others who attempts to make ping-tests and watchdog scripts to look out for their Wireguard connections.