[Experimental] WireGuard for HND platform (4.1.x kernels)

[Ubimo]

Thank you for your work and effort!
I'm trying to use Cloudflare WARP and was hoping, that the updated modules/kernel will solve my problem.
I cannot resolve most domain names like reddit.com or speedtest.net after starting the WARP client/interface.
Some domains, like snbforums.com, are loading fine...

Edit:
I was so desperate to figure out my problem, so I bought a mobile data sim and LTE stick.
I plugged this stick into my AC86U and now I'm using the mobile LTE data sim as my primary WAN.
Now, WARP is working without problems. So I guess the problem is my ISP or the special setup I use. (Router->LTU Pro->LTU Rocket->ISP)

My ISP told me, in order to properly use WARP I have to do something:

/ip firewall mangle
add out-interface=pppoe-out protocol=tcp tcp-flags=syn action=change-mss new-mss=1300 chain=forward tcp-mss=1301-65535

But I don't know what this is or how to use this?
Maybe someone can explain me, how I can use this or where I have to add this?

Edit:
I think it has something to do with MTU?
Is there a way to configure MTU for my warp.conf interface?

Edit2:
FINALLY!
I had to lower the MTU for my warp interface to 1280, now the tunnel is working!

How can I set the MTU to permanently is 1280 for my warp interface? Even after a reboot?

 

[noah]

Does anyone have the solution for this problem when i start the wireguard?

Device: RT-AC86U
Firmware: RT-AC86U_386.2_2
Entware verison: Entware-aarch64-3.10
wireguard-kernel_1.0.20210219-k27_1_aarch64-3.10.ipk
wireguard-tools_1.0.20210315-1_aarch64-3.10.ipk

admin@RT-AC86U-8B38:/tmp/home/root# /opt/etc/wireguard/S50wireguard start
RTNETLINK answers: Operation not supported

 

[Martineau]

Does anyone have the solution for this problem when i start the wireguard?

Device: RT-AC86U
Firmware: RT-AC86U_386.2_2
Entware verison: Entware-aarch64-3.10
wireguard-kernel_1.0.20210219-k27_1_aarch64-3.10.ipk
wireguard-tools_1.0.20210315-1_aarch64-3.10.ipk

admin@RT-AC86U-8B38:/tmp/home/root# /opt/etc/wireguard/S50wireguard start
RTNETLINK answers: Operation not supported

Without seeing the actual command that is causing the error

RTNETLINK answers: Operation not supported

then it could potentially impact the creation of the WIreGuard interface, or it is simply as a result of a (prudent) request to delete say a non-existent firewall rule/interface in order to prevent duplicates.

Perhaps you may wish to try

wireguard/README.md at main · MartineauUK/wireguard

Manage/Install WireGuard on applicable ASUS routers - MartineauUK/wireguard

github.com

which eliminates the need to use/modify S50wireguard.

 

[Jeffrey Young]

Does anyone have the solution for this problem when i start the wireguard?

Device: RT-AC86U
Firmware: RT-AC86U_386.2_2
Entware verison: Entware-aarch64-3.10
wireguard-kernel_1.0.20210219-k27_1_aarch64-3.10.ipk
wireguard-tools_1.0.20210315-1_aarch64-3.10.ipk

admin@RT-AC86U-8B38:/tmp/home/root# /opt/etc/wireguard/S50wireguard start
RTNETLINK answers: Operation not supported

Are you using "postUp" or "PostDown" directives? They are not supported in the entware version.

If you are ok with it, post your wg*.conf file and S50wireguard files (edit out your keys). I had your same issue as well when I first set up wireguard and it turned out that I was using a directive that works in the standard implementation which is not supported in entware.

Cheers

 

[noah]

Are you using "postUp" or "PostDown" directives? They are not supported in the entware version.

If you are ok with it, post your wg*.conf file and S50wireguard files (edit out your keys). I had your same issue as well when I first set up wireguard and it turned out that I was using a directive that works in the standard implementation which is not supported in entware.

Cheers

I'm using the default setting for S50wireguard.

The wg*.conf and S50wireguard files you can rename attach file from wireguardconf.txt to zip

 

[noah]

Without seeing the actual command that is causing the error

RTNETLINK answers: Operation not supported

then it could potentially impact the creation of the WIreGuard interface, or it is simply as a result of a (prudent) request to delete say a non-existent firewall rule/interface in order to prevent duplicates.

Perhaps you may wish to try


which eliminates the need to use/modify S50wireguard.

I try it, but did not work for me.
it shows "Initialisation complete", but the connection isn't working.

1  = Update Wireguard modules                        7  = Display QR code for a Peer {device} e.g. iPhone
2  = Remove WireGuard/wg_manager                    8  = Peer management [ "list" | "category" | "new" ] | [ {Peer | category} [ del | show | add [{"auto="[y|n|p]}] ]
                                    9  = Create Key-pair for Peer {Device} e.g. Nokia6310i (creates Nokia6310i.conf etc.)
3  = List ACTIVE Peers Summary [Peer...] [full]                10 = IPSet management [ "list" ] | [ "upd" { ipset [ "fwmark" {fwmark} ] | [ "enable" {"y"|"n"}] | [ "dstsrc"] ] } ]
4  = Start   [ [Peer [nopolicy]...] | category ] e.g. start clients                                 
5  = Stop    [ [Peer... ] | category ] e.g. stop clients                                   
6  = Restart [ [Peer... ] | category ] e.g. restart servers                                   

?  = About Configuration                   
v  = View ('/jffs/addons/wireguard/WireguardVPN.conf')       

e  = Exit Script [?]

E:Option ==> 4 3310

    Requesting WireGuard VPN Peer start (3310)

    wireguard-client3310: Initialising Wireguard VPN 'client' Peer (3310) to publicip:51820 (# Unidentified) DNS=
    wireguard-client3310: Initialisation complete.


     WireGuard ACTIVE Peer Status: Clients 1, Servers 0

 

[Jeffrey Young]

I'm using the default setting for S50wireguard.

The wg*.conf and S50wireguard files you can rename attach file from wireguardconf.txt to zip

When you say the "default" settings, did you at least define the mode and server/client settings? You do need to set those specific to your setup. I forget what the default "as-installed" settings were, but I did have to change it to my use. i.e. I use WireGuard as a server, so the top portion of of my S50wireguard file looks like this;

#!/bin/sh

PATH=/opt/sbin:/opt/bin:/opt/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

Mode=server   #server or client

#server
export Subnet=10.100.10.1/24   #e.g.)10.50.50.1/24
export wgport=51006

#client
export LocalIP=   #e.g.)10.50.50.2
Route=default   #default or policy
export wgdns=
export Nipset=wgvpn

As I am using WireGuard as a Server, the client directives means nothing.

For my wg1.conf file (wg1.conf is for a server), I have;

## Set Up WireGuard VPN ##

[Interface]
#Address directive is commented out here as not supported.  It is defined instead in S50wiregurad
#Address = 10.100.10.1/24
ListenPort = 51006
PrivateKey = <Key>

## Setup NAT to the rest of the net ##
## Commented out as not supported in this version

#PostUp = /etc/wireguard/wg-up.sh %i
#PostDown = /etc/wireguard/wg-down.sh %i

[Peer]
## Peer 1 ##
PublicKey = <Key>
PresharedKey = <Key>
AllowedIPs = 10.100.10.10/32

 

[Martineau]

I try it, but did not work for me.
it shows "Initialisation complete", but the connection isn't working.

1  = Update Wireguard modules                        7  = Display QR code for a Peer {device} e.g. iPhone
2  = Remove WireGuard/wg_manager                    8  = Peer management [ "list" | "category" | "new" ] | [ {Peer | category} [ del | show | add [{"auto="[y|n|p]}] ]
                                    9  = Create Key-pair for Peer {Device} e.g. Nokia6310i (creates Nokia6310i.conf etc.)
3  = List ACTIVE Peers Summary [Peer...] [full]                10 = IPSet management [ "list" ] | [ "upd" { ipset [ "fwmark" {fwmark} ] | [ "enable" {"y"|"n"}] | [ "dstsrc"] ] } ]
4  = Start   [ [Peer [nopolicy]...] | category ] e.g. start clients                                
5  = Stop    [ [Peer... ] | category ] e.g. stop clients                                  
6  = Restart [ [Peer... ] | category ] e.g. restart servers                                  

?  = About Configuration                  
v  = View ('/jffs/addons/wireguard/WireguardVPN.conf')      

e  = Exit Script [?]

E:Option ==> 4 3310

    Requesting WireGuard VPN Peer start (3310)

    wireguard-client3310: Initialising Wireguard VPN 'client' Peer (3310) to publicip:51820 (# Unidentified) DNS=
    wireguard-client3310: Initialisation complete.


     WireGuard ACTIVE Peer Status: Clients 1, Servers 0

FYI, wgm works best when Peer names follow the naming convention

wg1x        e.g. wg11 for a 'client' Peer

vs.

wg2x        e.g. wg21 for a 'server' Peer

rather than using '3310.conf'

P.S. If you use the diag command, this hopefully should elaborate on the terse aka useless/unhelpful summary 'the connection isn't working'

 

[noah]

When you say the "default" settings, did you at least define the mode and server/client settings? You do need to set those specific to your setup. I forget what the default "as-installed" settings were, but I did have to change it to my use. i.e. I use WireGuard as a server, so the top portion of of my S50wireguard file looks like this;

#!/bin/sh

PATH=/opt/sbin:/opt/bin:/opt/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

Mode=server   #server or client

#server
export Subnet=10.100.10.1/24   #e.g.)10.50.50.1/24
export wgport=51006

#client
export LocalIP=   #e.g.)10.50.50.2
Route=default   #default or policy
export wgdns=
export Nipset=wgvpn

As I am using WireGuard as a Server, the client directives means nothing.

For my wg1.conf file (wg1.conf is for a server), I have;

## Set Up WireGuard VPN ##

[Interface]
#Address directive is commented out here as not supported.  It is defined instead in S50wiregurad
#Address = 10.100.10.1/24
ListenPort = 51006
PrivateKey = <Key>

## Setup NAT to the rest of the net ##
## Commented out as not supported in this version

#PostUp = /etc/wireguard/wg-up.sh %i
#PostDown = /etc/wireguard/wg-down.sh %i

[Peer]
## Peer 1 ##
PublicKey = <Key>
PresharedKey = <Key>
AllowedIPs = 10.100.10.10/32

Hi Jeffrey, I already have wg server. I need to set client mode.
I will try to reconfigure the client mode in this S50wiregurad file.
If you have a better suggestion, please tell me. Thanks.

 

[Jeffrey Young]

Hi Jeffrey, I already have wg server. I need to set client mode.
I will try to reconfigure the client mode in this S50wiregurad file.
If you have a better suggestion, please tell me. Thanks.

If I understand correctly, you already have a wireguard server running on the router and now want to add a client along side the server ... right?

I believe there was a post further up this thread that discussed doing just that.

Leave the S50wireguard file that you have set up for the server alone (assuming it is working). Make a copy of S50wireguard file and call it S51wireguard. Edit S51wiregaurd accordingly to make the client. The *.conf file for the client is wg0.conf.

Personally, I would rename the Sxx files S50wgserver and S51wgclient just so I know which is which 6 months from now when I have to look at them (the important part of the entware init.d filenames is S and two numbers. The S, I am guessing means Service, and the xx is the order number the service gets called.

 

[noah]

如果我理解正確，您已經在路由器上運行了Wireguard服務器，現在想在服務器旁邊添加一個客戶端...對嗎？

我相信在該主題的後面還有一篇文章討論瞭如何做到這一點。

單獨保留為服務器設置的S50wireguard文件（假設它正在運行）。製作S50wireguard文件的副本，並將其命名為S51wireguard。相應地編輯S51wiregaurd以創建客戶端。客戶端的* .conf文件為wg0.conf。

就個人而言，我只是將Sxx文件重命名為S50wgserver和S51wgclient，所以我知道從現在起六個月後才需要查看它們（entit init.d文件名的重要部分是S和兩個數字。我猜是說服務，而xx是服務被調用的訂單號。

Ťhank you~. I fix my problem.

 

[PiCkLeS76]

First, a big thank you to the creator of this thread and the many good replies that made it a breeze to get a wireguard client up and running on my router. My few devices on my LAN is now all going through the client tunnel to my upstream VPN provider.

Now a question.

If I take down the tunnel (/op/etc/init.d/S50wireguard stop) all and any port forwarding rules I set in the Asus/Merlin web-gui works fine, ie I have one for ssh on port 44500 to the internal machine 10.0.1.100.

When I activate the tunnel so its UP, those forwarders dies. I can add as many port forwarders I want, none of them will work until I bring the tunnel back down. Im a terrible and too old of a person to understand exactly what to search for when it comes to iptables and other things I might need to understand to fix this.

So if anyone have a pointer or two on where to start to approach this I will be eternally grateful.

//P

 

[ZebMcKayhan]

First, a big thank you to the creator of this thread and the many good replies that made it a breeze to get a wireguard client up and running on my router. My few devices on my LAN is now all going through the client tunnel to my upstream VPN provider.

Now a question.

If I take down the tunnel (/op/etc/init.d/S50wireguard stop) all and any port forwarding rules I set in the Asus/Merlin web-gui works fine, ie I have one for ssh on port 44500 to the internal machine 10.0.1.100.

When I activate the tunnel so its UP, those forwarders dies. I can add as many port forwarders I want, none of them will work until I bring the tunnel back down. Im a terrible and too old of a person to understand exactly what to search for when it comes to iptables and other things I might need to understand to fix this.

So if anyone have a pointer or two on where to start to approach this I will be eternally grateful.

//P

port forwarding usually means an entry in NAT table to change the destination address (DNAT) of incoming package, usually on a specific port. you should be able to identify the entry that does this by listing the table:

iptables -t nat -L -v

while sending data over the forwarded port from external you should be able to see movement in the pkts/bytes number on this specific entry.

there should also be an entry in the filter table to allow packages on this specific port to be forwarded to the internal network:

iptables -t filter -L -v

also here you can track data by the pkts/byte numbers to verify data is identified and forwarded.

there is nothing that the wireguard config does that really changes any of this. the entries shall still be there.
however, if you have routed all outgoing data through VPN (default) than you might want to think about how a reply from your local client is supposed to find its way back to WAN when all outgoing data is routed through VPN?

//Zeb

 

[PiCkLeS76]

port forwarding usually means an entry in NAT table to change the destination address (DNAT) of incoming package, usually on a specific port. you should be able to identify the entry that does this by listing the table:

iptables -t nat -L -v

while sending data over the forwarded port from external you should be able to see movement in the pkts/bytes number on this specific entry.

there should also be an entry in the filter table to allow packages on this specific port to be forwarded to the internal network:

iptables -t filter -L -v

also here you can track data by the pkts/byte numbers to verify data is identified and forwarded.

there is nothing that the wireguard config does that really changes any of this. the entries shall still be there.
however, if you have routed all outgoing data through VPN (default) than you might want to think about how a reply from your local client is supposed to find its way back to WAN when all outgoing data is routed through VPN?

//Zeb

Thank you very much for your reply, very much appreciated. Im sorry of this reply will be much more chaotic then yours.

So. When the tunnel is up and I havent added any rules of my own yet, this is what exists in PREROUTING:

86 4776 GAME_VSERVER all -- * * 0.0.0.0/0 <my.public.ip.on.eth0>
86 4776 VSERVER all -- * * 0.0.0.0/0 <my.public.ip.on.eth0>

But if I make a connection from the outside, of course the bytes is ticking up from other traffic, so its hard to check specifically without say tcpdump or something. But I assumed that its of course is incoming there.

So I added another one, still in PREROUTING, that breaks out port 44500 specifically:

5 260 VSERVER tcp -- eth0 * 0.0.0.0/0 0.0.0.0/0 tcp dpt:44500

Now when I try to connect on that port from the outside, I see the bytes ticking in, in that specific rule. So far so good. Now its not bundled togheter with all the other traffic that I see in the general catch-all rule VSERVER. Again, just for testing purposes.

Now. Since its VSERVER, I took at look at that. The only rule defined in VSERVER is this (2 references due to me making a specific PREROUTING one for my port):

Chain VSERVER (2 references)
122 6583 VUPNP all -- * * 0.0.0.0/0 0.0.0.0/0

And sure enough, when I try to connect from the outside, with my specific port, I see those bytes ticking up a bit faster. But I want to see if its "my" traffic..

To see my specific traffic on port 44500, I added another rule to the VSERVER chain:

10 520 VUPNP tcp -- eth0 * 0.0.0.0/0 0.0.0.0/0 tcp dpt:44500

And again, if I try to connect from the outside, I see that new rule ticking up with only "my" bytes everytime I try on that port.

So the "chain of events" so far is:

* Traffic comes in on port 44500 on the if eth0
* A rule in PREROUTING shovs it further to the chain VSERVER
* At VSERVER I can see the traffic at 10 520 VUPNP tcp -- eth0 * 0.0.0.0/0 0.0.0.0/0 tcp dpt:44500
* The end. The router doesnt know what to do next with the traffic..

So .. how do I add a rule/rules that from VSERVER (or directly from PREROUTING) makes sure that all traffic incoming (tcp, port 44500) gets to my internal device called 10.0.1.100? Because right now, it stops in the chain VSERVER -> VUPNP.

As I wrote in my initial question, I can get port forwarding to with if the vpn tunnel is down:

If I add these rules:

iptables -t nat -I PREROUTING -i eth0 -p tcp --dport 44500 -j DNAT --to-destination 10.0.1.100
iptables -t nat -I PREROUTING -i eth0 -p udp --dport 44500 -j DNAT --to-destination 10.0.1.100
iptables -t nat -I POSTROUTING -o eth0 -p tcp --sport 44500 -j MASQUERADE
iptables -t nat -I POSTROUTING -o eth0 -p udp --sport 44500 -j MASQUERADE

I can connect as per my wishes to port 44500 (ssh) on my internal device (10.0.1.100) IF and only IF the wg client is stopped so that the tunnel is brought down.

I wish I was a bit smarter to understand this..

Oh, by the way, I've seen posts like this, but I cant edit /etc/iproute2/rt_tables to try that specific howto, but it seems logical to do. But no matter what I try or do, rt_tables are locked for me to do anything to:

Howto Bypass Wireguard VPN for specific port

Again, a huge thanks for the answer.

//P

 

[ZebMcKayhan]

sounds like you got a grip on the most part...
first of all you might need to disable the reverse path lookup filter, which will block incoming packages that does not appear to being replied to out the same interface. routing data through different tables, interfaces will/may cause this to fail:

echo 0 > /proc/sys/net/ipv4/conf/eth0/rp_filter

but the problem persists, even if the packages make it from WAN to 10.0.1.100 any reply will likely not make it back, since it will be routed out on VPN according to your main routing table.

I am not familiar if/how you can force packages through different interfaces without making a new routing table. a good start would be to read up on "ip rule". hopefully someone with better knowledge than me might now and answer this.

the most straight forward path would be to switch to policy based routing where you add rules for everything that should be routed through VPN.

alternatively; I have a similar setup as you were I have used default VPN routing but manually made a second routing table for WAN and I can make rules (as certain source ip's and ports) to use my other table were default routing is WAN.

anyway, it is a bit of a hazzle but I dont know any other way.

I could probably kit something together later if you would need it.

//Zeb

 

[ZebMcKayhan]

Ok, so this is basically how I did it.

Create a bash file with:

#!/bin/sh
#

#################################
# Create ip table 117 without VPN
#################################
ip route flush table 117 2>/dev/null # Clear table 117
ip route show table main | while read ROUTE # Copy all routes from main table to table 117 except wg0 routes
do
    {
        if ! echo "$ROUTE" | grep 'wg0' ; then
                ip route add table 117 $ROUTE
        fi
    } 1> /dev/null
done
###############################
echo 2 > /proc/sys/net/ipv4/conf/eth0/rp_filter

Now you have created a new routing table with id 117 that should be identical to your main routing table before wireguard was started. Now you need to add rules to tell the system on when to use this table instead of the main table (which means vpn)

So in your case adding a simple

ip rule add from 10.0.1.100 table 117 prio 9997

Would make all packets from this ip to go out through wan and I guess your ssh session would work?

You also might need

ip route flush cache

At the end

Now if you still want other packages from this ip to go through vpn but ssh to go on wan it gets more complicated since ip rule does not support "sport" atleast on my router. You will need to mark packages:

ip rule add fwmark 0x7000 table 117 prio 9997
iptables -t mangle -I PREROUTING -s 10.0.1.100 -p tcp --sport 44500 -j MARK --set-mark 0x7000

Where ip rule makes all marked packages use our wan route table.
The iptables mark the packages before they are routed if source adress and port matches.
I hope I spelled everything right, I have no possibilities to test.

Most of these should be deleted in wg-down but you better check.

Finalize the script according to your needs, make it executable and you could add it to execute at the end of wg-up to make it execute as wireguard is started.

//Zeb

 

[PiCkLeS76]

Ok, so this is basically how I did it.

Create a bash file with:

[snip]
Okay this blows me away in two ways.

Firstly because it works. I've been fiddling around so many hours now so I just flushed all that first and did a reboot -> made sure the tunnel was up -> added my four iptables rules for the port forwarding -> ran your script (and included the flush in the script + added also the rule).

Secondly, because you didnt just take the time to help me with an answer, you made a script and also in a very logical and clear way described what you did. This and the last post from you is why I still have faith in Internet as a whole.

Thank you so very much!

//P

EDIT: The disclaimer below does not apply (anymore) if you choose to use Zeb's version and mark the traffic. Then its only that traffic that goes outside of the tunnel.

DISCLAIMER: Just so we both are clear for anyone coming in here afterwards its very important to understand that the specific device that my address belongs to (10.0.1.100) is a machine Im happy with not going through the VPN tunnel.

What this does is not only make it accessible with the ports you define with iptables from the outside, it also makes this specifice device do ANY traffic from the inside of that device go out through the PUBLIC interface/ip address, ie via your upstream ISP. It wont be behind and inside the VPN tunnel.

So if you also do this, make sure its a static ip on your lan so it doesnt happen to be a device you want to protect at all times and it gets this specific ip via DHCP.

 

[Martineau]

.... since ip rule does not support "sport" atleast on my router.

FYI, I did recently ask @RMerlin to investigate the possibility of enabling the ip sport/dport Kernel support on v4.xx kernel-based routers, but not sure if it is a case of simply enabling a compile time option to include the necessary kernel module or that he has identified it being far more complicated and ultimately deemed it may not be worth the effort to replace the (potentially fragile) fwmark method.

 

[PiCkLeS76]

ip rule add from 10.0.1.100 table 117 prio 9997

ip rule add fwmark 0x7000 table 117 prio 9997
iptables -t mangle -I PREROUTING -s 10.0.1.100 -p tcp --sport 44500 -j MARK --set-mark 0x7000

//Zeb

(EDIT! I wrote a huge post on that this new updated version you made didnt work - but I was adding the general rule for ALL traffic so of course it didnt work - now it does! Great, now its only the ports/traffic that goes on the public interface, the rest is travelling through the vpn tunnel!

I added both these:

ip rule add from 10.0.1.2 table 117 prio 9997
ip rule add fwmark 0x7000 table 117 prio 9997

..which was stupid since the first one there will still make all traffic go beside the tunnel :-D

You sir, are fantastic - thank you!

//P - a very happy guy

 

[ZebMcKayhan]

I added both these:

ip rule add from 10.0.1.2 table 117 prio 9997
ip rule add fwmark 0x7000 table 117 prio 9997

Glad you made it work!

yea, I choose prio 9997 partly because this prio is removed in wg-down and partly because I was hoping if someone used both they would overwrite each other. so I guess they didnt overwrite, or the later was not added since the priority was already taken?

you can list all rule's in the system in the terminal by:

ip rule

but as you figured out, in this case only one rule should be used since they are intended for the same traffic.

other good to know ways is to use rules for input interface:

ip rule add iif br1 table 117

would send entire guest network (2gHz) to WAN instead of VPN

//Zeb