Is wireguard available on RT-AC86U ?

[Jeffrey Young]

That is strange.

Don't worry about the insmod warning. It is just telling you that it can't insert the kernel module because it is already inserted (inserted the first time you ran the script after a reboot).

You don't need the sh in front of the script call.

YazFi does a lot of tearing down and rebuilding the network. I have a suspicion that YazFi is not done processing a startup before your wireguard script is called. Maybe try putting a sleep statement between YazFi and the the wg script. Say sleep for 10 seconds (sleep 10

I am tucked in bed now for the night, but I can modify my script tomorrow morning so that it writes its output to a log file for analysis.

 

[Jherb]

That is strange.

Don't worry about the insmod warning. It is just telling you that it can't insert the kernel module because it is already inserted (inserted the first time you ran the script after a reboot).

You don't need the sh in front of the script call.

YazFi does a lot of tearing down and rebuilding the network. I have a suspicion that YazFi is not done processing a startup before your wireguard script is called. Maybe try putting a sleep statement between YazFi and the the wg script. Say sleep for 10 seconds (sleep 10

I am tucked in bed now for the night, but I can modify my script tomorrow morning so that it writes its output to a log file for analysis.

Great, thank you. I've remove the whole line for YazFi as I don't use it. The only line in the services-start is now the wg1_start. It didn't make a difference, but just wanted to let you know.

Also, I rebooted the router, logged in via putty, and used the wg command and saw the below (asterisks placed by me).

interface: wg1​

public key: ******=​

private key: (hidden)​

listening port: 51006​

​

peer: ***=​

allowed ips: 10.100.10.2/32​

​

peer: ***=​

allowed ips: 10.100.10.3/32​

​

peer: *****​

allowed ips: 10.100.10.4/32​

On the client side, it said its connected, but When I run the "wg" command again, I don't see a client working and they can't access the internet, etc.

When I manually run the start_wg1, immediately the clients work and I can see a peer connected when running the "wg" command again.

peer: **=
endpoint: 172.54.704.49:27183
allowed ips: 10.100.10.4/32
latest handshake: 13 seconds ago. (sec:13)
transfer: 156.75 KiB received, 2.28 MiB sent


I'm not sure if I'm interpreting correctly, but does that mean the the wireguard stuff started but there is something else going on?



Thank you very much for your help.

 

[Jeffrey Young]

Great, thank you. I've remove the whole line for YazFi as I don't use it. The only line in the services-start is now the wg1_start. It didn't make a difference, but just wanted to let you know.
Thank you very much for your help.

Yeah, I just checked my services-start script and I don't have a sleep statement between YazFi and wg start.

You can put a logger statement in before and after the wg call, then check the system log to see if it ran...

#!/bin/sh

sh /jffs/addons/cake-qos/cake-qos mountui # CakeQOS-Merlin
/jffs/scripts/YazFi startup & # YazFi

logger "starting wireguard server (wg1)"
/jffs/addons/wireguard/start_wg1.sh
logger "Wireguard server (wg1) has been started"

You can also place a logger statement inside the wg1 script.

 

[Jeffrey Young]

Ok, this this one is bugging me, so I got up and altered my server script a bit. The script now writes it output to a text file at /tmp/wg1_startup_log.txt.

Reboot your router and have a look to see if the debug text file exists and see what it says.

The new script is below. Again, you will have to edit for your system

#!/bin/sh

set -x # enable debugging.  Comment out when debugging is not needed

{
date

KERNEL=$(uname -r)
WGaddress=10.100.10.1/24
WGport=51006

modprobe xt_set
insmod /lib/modules/${KERNEL}/kernel/net/wireguard/wireguard.ko

ip link del dev wg1 2>/dev/null
ip link add dev wg1 type wireguard
wg setconf wg1 /jffs/addons/wireguard/wg1.conf
ip address add dev wg1 $WGaddress
ip link set up dev wg1
#ifconfig wg1 mtu 1380 # origional set by setup script
ifconfig wg1 mtu 1380
ifconfig wg1 txqueuelen 1000

iptables -t mangle -D PREROUTING -i wg1 -j MARK --set-xmark 0x01/0x7 2>/dev/null
iptables -t mangle -D FORWARD -o wg1 -j MARK --set-xmark 0x01/0x7 2>/dev/null
iptables -t mangle -D FORWARD -i wg1 -p tcp -m tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu 2>/dev/null
iptables -t mangle -D FORWARD -o wg1 -p tcp -m tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu 2>/dev/null

iptables -D INPUT -p udp --dport $WGport -j ACCEPT 2>/dev/null
iptables -D INPUT -i wg1 -j ACCEPT 2>/dev/null
iptables -D FORWARD -i wg1 -j ACCEPT 2>/dev/null
iptables -D FORWARD -o wg1 -j ACCEPT 2>/dev/null
iptables -D OUTPUT -o wg1 -j ACCEPT 2>/dev/null
iptables -t nat -D PREROUTING -p udp --dport $WGport -j ACCEPT 2>/dev/null

iptables -t mangle -I FORWARD -o wg1 -p tcp -m tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu
iptables -t mangle -I FORWARD -i wg1 -p tcp -m tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu
iptables -t mangle -I FORWARD -o wg1 -j MARK --set-xmark 0x01/0x7
iptables -t mangle -I PREROUTING -i wg1 -j MARK --set-xmark 0x01/0x7

iptables -I INPUT -p udp --dport $WGport -j ACCEPT
iptables -I INPUT -i wg1 -j ACCEPT
iptables -I FORWARD -i wg1 -j ACCEPT
iptables -I FORWARD -o wg1 -j ACCEPT
iptables -I OUTPUT -o wg1 -j ACCEPT
iptables -t nat -I PREROUTING -p udp --dport $WGport -j ACCEPT
} > /tmp/wg1_startup_log.txt 2>&1

 

[Jeffrey Young]

One other thing, and forgive me asking, but your start_wg1.sh script is actually located at /jffs/addons/wireguard?

Just checking.....

 

[Jherb]

One other thing, and forgive me asking, but your start_wg1.sh script is actually located at /jffs/addons/wireguard?

Just checking.....

Good question, and yes it is. When I manually execure services-start, everything works. I'm going to try your modified script above.

Sorry to keep you up

thanks

 

[Jherb]

Ok, there was a wg1_startup_log.txt. The contents are below.
I did try to connect a client again, and still they don't have access to anything.


+ date
Sat May 5 01:05:13 DST 2018
+ uname -r
+ KERNEL=4.1.27
+ WGaddress=10.100.10.1/24
+ WGport=51006
+ modprobe xt_set
+ insmod /lib/modules/4.1.27/kernel/net/wireguard/wireguard.ko
+ ip link del dev wg1
+ ip link add dev wg1 type wireguard
+ wg setconf wg1 /jffs/addons/wireguard/wg1.conf
+ ip address add dev wg1 10.100.10.1/24
+ ip link set up dev wg1
+ ifconfig wg1 mtu 1380
+ ifconfig wg1 txqueuelen 1000
+ iptables -t mangle -D PREROUTING -i wg1 -j MARK --set-xmark 0x01/0x7
+ iptables -t mangle -D FORWARD -o wg1 -j MARK --set-xmark 0x01/0x7
+ iptables -t mangle -D FORWARD -i wg1 -p tcp -m tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu
+ iptables -t mangle -D FORWARD -o wg1 -p tcp -m tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu
+ iptables -D INPUT -p udp --dport 51006 -j ACCEPT
+ iptables -D INPUT -i wg1 -j ACCEPT
+ iptables -D FORWARD -i wg1 -j ACCEPT
+ iptables -D FORWARD -o wg1 -j ACCEPT
+ iptables -D OUTPUT -o wg1 -j ACCEPT
+ iptables -t nat -D PREROUTING -p udp --dport 51006 -j ACCEPT
+ iptables -t mangle -I FORWARD -o wg1 -p tcp -m tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu
+ iptables -t mangle -I FORWARD -i wg1 -p tcp -m tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu
+ iptables -t mangle -I FORWARD -o wg1 -j MARK --set-xmark 0x01/0x7
+ iptables -t mangle -I PREROUTING -i wg1 -j MARK --set-xmark 0x01/0x7
+ iptables -I INPUT -p udp --dport 51006 -j ACCEPT
+ iptables -I INPUT -i wg1 -j ACCEPT
+ iptables -I FORWARD -i wg1 -j ACCEPT
+ iptables -I FORWARD -o wg1 -j ACCEPT
+ iptables -I OUTPUT -o wg1 -j ACCEPT
+ iptables -t nat -I PREROUTING -p udp --dport 51006 -j ACCEPT

 

[Jeffrey Young]

I'm not sure if I'm interpreting correctly, but does that mean the the wireguard stuff started but there is something else going on?

I missed this part. Yes, I agree. The services start is running, but something else is amist.

 

[Jherb]

I missed this part. Yes, I agree. The services start is running, but something else is amist.

I just tried with your new script. I do get the log as above, but nothing works until I manually type "/jffs/addons/wireguard/start_wg1.sh" in putty. Then everything works.

 

[Jeffrey Young]

+ date
Sat May 5 01:05:13 DST 2018
+ uname -r

This tells me something! Your WAN is not up yet. My WAN is set up for a static address from my ISP, so the WAN is likely up before my script is called. If you are not using a static WAN address, your WAN is not likely up in time.

I would take the wg script call out of the services-start script and place the following in the /jffs/scripts/wan-event script;

EDIT1: added sleep statement as per discussion in post 51
EDIT2: modified script

#!/bin/sh

# variables passed to wan-event
# $1 > Wan Adaptor Number
# $2 > Wan state
#        init
#        connecting
#        connected
#        disconnected
#        stopped
#        disabled
#        stopping

if [ "$2" = "connected" ];then
    ( sleep 10 ; /jffs/addons/wireguard/start_wg1.sh ) &
fi

This will ensure the vpn server is brought up after the WAN is started and is connected.

 

[Jherb]

I'm trying to learn here. I assume because the date/time was off, you deduced that the WAN wasn't up? Very clever.

I used your wan-event but it still didn't work. Then I read

"The old wan-start script would occur on a "connected" event. Note that after a connected event occurs, the Internet connection might not yet be fully functional. You should probably add a slight pause before you try to initiate anything requiring Internet access, or write your own code to wait until the connection becomes functional."​

So I added a "sleep 10" before the start_wg1.sh and now it seems to work. I rebooted a few times, and it worked.

You rock. I really appreciate your help, and I learned a lot. I'm new to all this merlin scripting.

 

[Jeffrey Young]

I'm trying to learn here. I assume because the date/time was off, you deduced that the WAN wasn't up? Very clever.

I used your wan-event but it still didn't work. Then I read

"The old wan-start script would occur on a "connected" event. Note that after a connected event occurs, the Internet connection might not yet be fully functional. You should probably add a slight pause before you try to initiate anything requiring Internet access, or write your own code to wait until the connection becomes functional."​

So I added a "sleep 10" before the start_wg1.sh and now it seems to work. I rebooted a few times, and it worked.

You rock. I really appreciate your help, and I learned a lot. I'm new to all this merlin scripting.

Excellent. Glad it all worked out. I need to alter my own wan-event script now with that knowledge.

The date told me that the ntp service had not synced the system time yet, which led me to believe that the WAN was not up yet.

Should not have made a difference as far as wireguard goes, but the WAN connecting likely restarted the firewall, which would have zapped the firewall rules we added in the wg script. You observed that when you saw that the wiregaurd interface was up (when you issued the wg command). But the firewall rules that let wireguard traffic pass got zapped when the WAN connected.

Cheers.

 

[Jherb]

Got it. Thanks alot.
I am trying to understand those firewall rules, but I'm not there yet. Maybe after you sleep, you can tell me the high level concept.

Thanks again.

 

[Martinski]

Contents of service-start script are below

#!/bin/sh

sh /jffs/addons/cake-qos/cake-qos mountui # CakeQOS-Merli
/jffs/scripts/YazFi startup & # YazFi
sh /jffs/addons/wireguard/start_wg1.sh

YazFi does a lot of tearing down and rebuilding the network. I have a suspicion that YazFi is not done processing a startup before your wireguard script is called. Maybe try putting a sleep statement between YazFi and the the wg script. Say sleep for 10 seconds (sleep 10

The call to YazFi /jffs/scripts/YazFi startup & # YazFi has an explicitly hard-coded, 12-second sleep delay before the startup actions begin.

I would take the wg script call out of the services-start script and place the following in the /jffs/wan-event script;

/jffs/scripts/wan-event

Just FYI.

 

[Jherb]

Thanks alot. Appreciate all the help. Its working now, so that's a lot of progress

 

[Martinski]

#!/bin/sh
...
if [ "$2" = "connected" ];then
    sleep 10
    /jffs/addons/wireguard/start_wg1.sh
fi

BTW, I'd recommend putting any sleep delay inside the custom script being called, or placing the calls from the event script in the background like so:

if [ "$2" = "connected" ];then
    ( sleep 10 ; /jffs/addons/wireguard/start_wg1.sh ) &
fi

Some of the built-in event scripts are blocking scripts and some are not, but as a general rule and best practice technique, it's better to avoid delaying unnecessarily the execution of any event scripts to prevent unintended side effects.

My 2 cents.

 

[Jeffrey Young]

The call to YazFi /jffs/scripts/YazFi startup & # YazFi has an explicitly hard-coded, 12-second sleep delay before the startup actions begin.


/jffs/scripts/wan-event

Just FYI.

Good catch, edited post 50 to fix. Thanks for the info on YazFi

 

[Jeffrey Young]

@Jherb , can you check your nat-start script.... Something still bothers me. Even if the firewall did restart, which almost always calls nat-start as well, the nat-start script would have restarted the wg script. The /jffs/scripts/nat-start should looks something like this (and also have execute permission). You will have to edit to use what ever interface names you used in your script.

#!/bin/sh

WVPNROUTE=`ip route show | grep -i -a "dev wg1"`
logger -s -t "($(basename $0))" $$ "Checking if WireGuard Server is UP...."$WVPNROUTE
if [ "$WVPNROUTE" != "" ];then
           logger -s -t "($(basename $0))" $$ "**Warning WireGuard Server is UP.... restarting WireGuard to refresh firewall rules"
        /jffs/addons/wireguard/start_wg1.sh
fi

@Martinski , thanks for the suggestion. wan-event is not a blocking script - still your advice is well taken. I, for one appreciate the advise.

 

[Jeffrey Young]

Got it. Thanks alot.
I am trying to understand those firewall rules, but I'm not there yet. Maybe after you sleep, you can tell me the high level concept.

Thanks again.

Well, I am by no means an expert at iptables. I learn as I go and I use Mr. Google a lot.

I use this link quite a bit. The pictorial representation in the link that shows the flow of network traffic through the router and how each of the tables and chains are work is my main goto when I have to figure out where to start.

An In-Depth Guide to iptables, the Linux Firewall

The Linux kernel comes with a packet filtering framework named netfilter. It allows you to allow, drop and modify traffic leaving in and out of a system. A tool, iptables builds upon this functionality to

www.booleanworld.com

The start_wg1.sh script should probably be renamed restart_wg1.sh as the the script actually tears down the wireguard interface first, if it exists, then rebuilds the interface. The script then deletes all the related firewall rules, if they exist (the -D option in the iptables commands), then re-adds the firewall rules (the -I option).

Since we do not know when nat-start gets called and if the router wipes the iptables rules with each call, we need to delete the rules first. If we did not, and the router did not rebuild its firewall rules, we would end up duplicate rule entries. Over the course of many months between router restarts, the list of rules would grow massive and tie the router up in knots working each packet through the list of rules.

The various iptables rules are there to allow the router to accept wireguard traffic and to allow the wireguard traffic to be forwarded so that you can talk to your lan. That is the INPUT, FORWARD, and OUTPUT chains of the "filter" table. Note, if you don't specify a table in the iptables command (-t option), then the "filter table is used.

The mangle table rules is where you can "alter" the packets. Here we are clamping the packets so that their size does not exceed our MTU (badly explained here, hope you get the jist) and also to mark the packets so that they bypass the router's hardware nat-acceleration as wireguard does not play well with nat-acceleration. Lots of forum discussion as to rather or not the marking is required or not now adays. Back in 2018 when the first wireguard kernel module came to the AC86U via a form member, this was required. Now, with later firmware versions, it was been reported that it is not required by some routers (such as the AC86U) and some say that the marks don't work anymore at all. If marking the inbound packets don't work and you find your wireguard throughput is in the dumpster, you are left with having to disable nat-acceleration altogether , but that creates other issues if your ISP speed is above 300mbits/s. You really just have to experiment with leaving these lines in or out. Some router functions, when set, disables nat-acceleration anyway. Also, the latest alpha and beta firmware have ASUS upsteam fixes that does the bypassing for you. In deed, you should soon be able to remove these marking rules altogether.

The last rule is to add a rule to the PREROUTING chain of the NAT table to allow inbound packets to reach the FORWARD chain of the filer table.

For clarity, here is the flow picture from the link I posted above;

Hope this helps.

 

[Jherb]

Hi,
I used the exact code for nat-start as you posted above, and made sure it had execute permissions. I didn't change anything since I also used wg1. Do I need to change anything else?

I did the following. I removed the "sleep" command from wan-event for testing.

1. I rebooted and it was like it was before where things weren't working. 2. I then put a sleep 10 command at the start of the Nat-start script, and rebooted. Still couldn't connect.
3. I put a sleep command at the top of the start_wg1 script so it waits wherever its called from. --this worked and clients are able to connect.

I'm not sure how nat-start is called so I'm not sure what the issue is.