Wireguard Session Manager - Discussion (3rd) thread

[Martineau]

should web ui work? For me clicking anything apart from import config does not do anything, also version etc is not even filled
this is how it looks for me

View attachment 43125

The WebUI is still experimental, however the basics should work...

Can you try the following command

e  = Exit Script [?]

E:Option ==> www refreshX

    WebUI page 'user?.asp' ('wg_manager.asp') unmounted
    WebUI page ('wg_manager.asp') mounted as 'user?.asp'
    [?] Restarted service_httpd for WebUI

 

[evlo]

E:Option ==> www refreshX

        WebUI page 'wg_manager.asp' not mounted!

        ***ERROR: WebUI TAB ('wg_manager.asp') already mounted!
        [✔] Restarted service_httpd for WebUI


        WireGuard® ACTIVE Peer Status: Clients 0, Servers 1

E:Option ==> ?

        Router RT-AX68U Firmware (v386.7_2)

        [✔] Entware Architecture arch=aarch64


        v4.18 WireGuard® Session Manager (Change Log: https://github.com/MartineauUK/wireguard/commits/main/wg_manager.sh)
        MD5=1036fe1ca4df245cb50db0e069550da4 /jffs/addons/wireguard/wg_manager.sh

                v4.17.9 (wg_client)
                v4.17.1 (wg_server)

        [✔] WireGuard® Kernel module/User Space Tools included in Firmware (1.0.20210124)


        [✔] WebUI Addon Enabled

        [✔] DNSmasq is listening on ALL WireGuard® interfaces 'wg*'

        [✔] firewall-start is monitoring WireGuard® Firewall rules

        [✖] WAN KILL-Switch is DISABLED (use 'vx' command for info)
        [✖] UDP monitor is DISABLED

        [✔] Flow Cache is ENABLED

        [✖] IPv6 Service is DISABLED
        [ℹ ] IPv4,46.*,v1.1,,,See http://ip6.me/docs/ for api documentation

        [✔] Reverse Path Filtering ENABLED

        [✖] Use 3rd-party Entware/Userspace Tools modules is DENIED

        [✔] Use of 'Pg-Up' Key for command retrieval is ENABLED

        [✔] Statistics gathering is ENABLED

        [ℹ ] Speedtest link https://fast.com/en/gb/

        [ℹ ] IPv6 Test link https://ipv6-test.com/

        [ℹ ] WireGuard© Official Site https://www.wireguard.com/

        [ℹ ] @ZebMcKayhan's Hint's and Tips Guide https://github.com/ZebMcKayhan/WireguardManager/blob/main/README.md#table-of-content


        WireGuard® ACTIVE Peer Status: Clients 0, Servers 1

site2site is what i want to try so i will still try to get this working and see how fast want is if i'm able to to get it to work.

maybe this
[✖] Use 3rd-party Entware/Userspace Tools modules is DENIED
is an issue?


also
? = About Configuration (WebUI http://://192.168.50.1:/user6.asp)
does not seem exactly right


I tried reisntalling, only trough amtm, but it is the same

 

[Martineau]

E:Option ==> www refreshX

        WebUI page 'wg_manager.asp' not mounted!

        ***ERROR: WebUI TAB ('wg_manager.asp') already mounted!
        [✔] Restarted service_httpd for WebUI

Strange, if the WebUI wasn't there to be be unmounted, not sure why the WebUI apparently appears to be already mounted?

Can you issue these two commands

df

ls -lah /tmp/var/wwwext | grep -TE "user[1-9]+.*";grep -TH . /tmp/var/wwwext/*.title;grep -THE "user[1-9]\." /tmp/menuTree.js | sort

and post the results

[✔] WireGuard® Kernel module/User Space Tools included in Firmware (1.0.20210124)
maybe this
[✖] Use 3rd-party Entware/Userspace Tools modules is DENIED
is an issue?

This simply means you are using the firmware's WireGuard modules (compiled 2021/01/24), and do not want to override the firmware modules by 3rd-party compiled modules such as @ZebMcKayhan's compiled on 2022/06/27

GitHub - ZebMcKayhan/Wireguard: Wireguard kernel module and userspace-tools for asus routers running Entware 3.10

Wireguard kernel module and userspace-tools for asus routers running Entware 3.10 - ZebMcKayhan/Wireguard

github.com

also
? = About Configuration (WebUI http://://192.168.50.1:/user6.asp)
does not seem exactly right

Indeed a bug...

Update wg_manager.sh · MartineauUK/wireguard@9d5b742 · GitHub

Please upgrade wg_manager to Beta v4.19b2

e  = Exit Script [?]

E:Option ==> uf dev

and please test (with a REBOOT) to see if the WebUI is still broken.

 

[ZebMcKayhan]

site2site is what i want to try so i will still try to get this working and see how fast want is if i'm able to to get it to work

While you are struggling with the gui:
https://github.com/ZebMcKayhan/WireguardManager/blob/main/README.md#site-2-site

setting up site-2-site from wgm command line is as easy as:

E:Option ==> site2site Home Cabin lan=192.168.111.0/24

That is:

E:Option ==> site2site <SiteAName> <SiteBName> lan=<SiteBLanIp>

Good luck!

Ooh, and by the way: please report back wheither you could keep flowcache enabled.

 

[evlo]

@Martineau

admin@asusAX68U:/tmp/home/root# ls -lah /tmp/var/wwwext | grep -TE "user[1-9]+.*";grep -TH . /tmp/var/wwwext/*.title;grep -THE "user[1-9]\." /tmp/menuTree.js | sort
-rw-rw-rw-    1 admin    root       40.4K Jul 27 11:44 user1.asp
-rw-rw-rw-    1 admin    root           9 Jul 27 11:44 user1.title
-rw-rw-rw-    1 admin    root        4.8K Jul 27 11:44 user2.asp
-rw-rw-rw-    1 admin    root       54.2K Jul 27 11:44 user3.asp
-rw-rw-rw-    1 admin    root          10 Jul 27 11:44 user3.title
-rw-rw-rw-    1 admin    root      132.1K Jul 27 11:44 user4.asp
-rw-rw-rw-    1 admin    root           8 Jul 27 11:44 user4.title
-rw-rw-rw-    1 admin    root       82.0K Jul 27 11:44 user5.asp
-rw-rw-rw-    1 admin    root          10 Jul 27 11:44 user5.title
-rw-rw-rw-    1 admin    root       60.3K Jul 27 21:12 user6.asp
/tmp/var/wwwext/user1.title:    scMerlin
/tmp/var/wwwext/user3.title:    dn-vnstat
/tmp/var/wwwext/user4.title:    connmon
/tmp/var/wwwext/user5.title:    spdMerlin
/tmp/menuTree.js:       {url: "user1.asp", tabName: "scMerlin"},
/tmp/menuTree.js:       {url: "user2.asp", tabName: "Sitemap"},
/tmp/menuTree.js:       {url: "user3.asp", tabName: "dn-vnstat"},
/tmp/menuTree.js:       {url: "user4.asp", tabName: "connmon"},
/tmp/menuTree.js:       {url: "user5.asp", tabName: "spdMerlin"},
/tmp/menuTree.js:       {url: "user6.asp", tabName: "WireGuard® Manager"},

@ZebMcKayhan thank you, honestly i'm still very beginner with writeguard, starting to understand what even is peer and what ip to assign the interface etc.
Until now it seemed to me, that WG does not consider one peer server and other client, but after I run command you postat it ended with

[✔] Config Home import as wg22 Site-to-Site (FORCED as 'server') success

, but actually in this case it should be client ...

@Martineau no change after

uf dev

, and reboot
then I did run

E:Option ==> www refreshX

        WebUI page 'user1.asp' ('wg_manager.asp') unmounted
        WebUI page ('wg_manager.asp') mounted as 'user1.asp'
        [✔] Restarted service_httpd for WebUI

no change in web ui, but at least no self contradicting errors

uf dev output:

I did try firefox, as I think it is only remaining non chromium browser, but same, this is in console

I guess it should be plus empty string or no plus at all?

 

[ZebMcKayhan]

@ZebMcKayhan thank you, honestly i'm still very beginner with writeguard, starting to understand what even is peer and what ip to assign the interface etc.

No worries! Not too long ago I was there too. Since Wireguard is niether client nor server but actually capable of both a peer is simply what to call it instead.
Usually there is no need of controlling the ips of the peer itself and thats specially true for site2site setup as in the end it will be lan2lan and you never need to care about the peer ip's. The only important thing is that your 2 lan ip must not overlap. I.e both cannot be 192.168.50.x.

Whenever you feel ready, try to enter the command in wgm and see what happens. You won't break anything and whatever peers that are created are easaly removed if they did not turn out correctly.

I will be here helping you out all the way if needed.

 

[evlo]

hmm, I did copied the config that was generated by wire guard manager on asus router to the "server" (server = other side) I did run

wg setconf wg0 Home.conf

but I got

Line unrecognized: `Address=10.9.8.1/32'
Configuration parsing error

Regarding IPs, one side is 192.168.50., other 192.168.51. and third one is 192.168.52.x .... anyway third one does not come into play now, i meant 10.9.8.1/32 - IP of the wireguard interface, but I guess i do not need to care about it on asus side and on my side, I can go with 192.168.200.1

... oh, there is something called wg-quick, let's see ... I copied all 6 files that were generated by WGM on asus into /etc/wireguard/ on the other peer and then run

wg-quick up Home

result

...
[#] ip -4 route add 192.168.51.0/24 dev Home
RTNETLINK answers: File exists
[#] ip link delete dev Home

I guess it should be ip -4 route add 192.168.50.0/24 dev Home as 51 is the other side lan range

this is how i dir run site2site command

site2site Home Cabin lan=192.168.51.0/24

, reading https://github.com/ZebMcKayhan/WireguardManager/blob/main/README.md#site-2-site looks like it is correct, maybe when WGM is saying
Import Home.conf on remote site using 'import Home type=device' it is true if remote is another wgm, but I sould actually do

wg-quick up Cabin

, again, it is explained in your article
If SiteB does not consist of an Asus HND router the Cabin.conf should be suitable for running with wg-quick but you might need to make device specific modifications to it to make it work.

[#] iptables -I INPUT -p udp --dport %p -j ACCEPT; iptables -I INPUT -i OCI -j ACCEPT; iptables -I FORWARD -i OCI -j ACCEPT
iptables v1.8.4 (nf_tables): Port "%p" does not resolve to anything.

Maybe it makes more sense to start (again) on the non-Asus side.

# WireGuard (%p - ListenPort; %wan - WAN interface; %lan - LAN subnet; %net - IPv4 Tunnel subnet ONLY recognised by Martineau's WireGuard Manager/wg-quick2)
so maybe i should i should look into wg-quick2? ... https://www.snbforums.com/threads/session-manager-discussion-2nd-thread.75129/page-34#post-744506

 

[ZebMcKayhan]

Regarding IPs, one side is 192.168.50., other 192.168.51. and third one is 192.168.52.x ....

Ooh, 3 sites.... dunno if wgm is capable of this (@Martineau?).

But ok, good to start with 2 sites maybee.

Soo your other 2 sites are not running asus routers? What are you trying to run it on?

If SiteB does not consist of an Asus HND router the Cabin.conf should be suitable for running with wg-quick but you might need to make device specific modifications to it to make it work.

I would assume you should execute:

wg-quick up Cabin

on the remote site to import the peer. But dont know if wg-quick sets up firewall as needed, but you will find out.

 

[evlo]

For now I had deleted --dport part from PostUp and PostDown lines in the conf file and it at least created the interface on the "Cabin".

Actually siteC is asus router, until now I was running it as openVPN, and we do not need to care for that site for now, siteB "Cabin" is linux machine.

E:Option ==> 8

        Peers (Auto start: Auto=P - Policy, Auto=S - Site-to-Site)
Server  Auto  Subnet        Port   Annotate
wg21    Y     10.50.1.1/24  51820  # RT-AX68U Server #1
wg22    S     10.9.8.1/32   61820  # Home - 192.168.50.0/24


        Peers (Auto=X - External i.e. Cell/Mobile/Site)
Device  Auto  IP           DNS  Allowed IPs                   Annotate
Cabin     X     10.9.8.2/32       10.9.8.1/32, 192.168.50.0/24  # Cabin Site-to-Site LAN 192.168.51.0/24

WireGuard® ACTIVE Peer Status: Clients 0, Servers 2

 

[ZebMcKayhan]

[#] iptables -I INPUT -p udp --dport %p -j ACCEPT; iptables -I INPUT -i OCI -j ACCEPT; iptables -I FORWARD -i OCI -j ACCEPT

%p is not valid for wg-quick. I suggest you manually edit Cabin.conf and replace %p with the Cabin incooming wireguard port number (should be stated in Cabin.conf ListenPort?)

 

[evlo]

Current status:

I can ping the gateway on the other side from asus router
I can NOT ping devices behind other side gateway from asus router

I can ping the gateway on the other side from device behind asus router
I can NOT ping devices behind other side gateway from device behind asus router

I can NOT ping devices behind asus router from device behind other side gateway
I can NOT ping asus router from device behind other side gateway

I can ping devices behind asus router from other side gateway
I can ping asus router from other side gateway


Do you think no adding dport to iptables can have this effect?
I think i'm missing some kind of routing/forwarding on the linux machine side
I had the same problem with openVpn, but usually it magically fixed itself or i fixed it messing around not know what i'm doing




Regarding your Flow Cache question:

[✔] Flow Cache is ENABLED

Speedtest result for WAN

Bandwidth - Download: 912.48 Mbps (data used: 1.1 GB ) - Upload: 789.12 Mbps (data used: 980.2 MB )
Quality - Latency: 2.18 ms (0.12 ms jitter) - Packet Loss: 0.0%

I tried two other servers and it is pretty much the same, hard to say if it is wire guard issue.

maybe i bit worse on the upload then usual, but livable with

 

[ZebMcKayhan]

Do you think no adding dport to iptables can have this effect?

No... as long as the peers are connected you should be fine. Altough without it Home can probably not initiate the connection but it is enough one side can but ofcourse redundancy if both sides are capable.

I think i'm missing some kind of routing/forwarding on the linux machine side

Sounds like it.

On the Cabin side, what's the output of

ip rule
ip route
iptables -nvL FORWARD -t filter

and mask any public ip before postings.

 

[evlo]

# ip rule
0:      from all lookup local
32766:  from all lookup main
32767:  from all lookup default




# ip route
default via 192.168.51.1 dev enp0s3
default via 192.168.51.1 dev enp0s3 proto dhcp src 192.168.51.2 metric 100
10.9.8.1 dev Cabin scope link
169.254.0.0/16 dev enp0s3 scope link
169.254.0.0/16 dev enp0s3 proto dhcp scope link src 192.168.51.2 metric 100
192.168.50.0/24 dev Cabin scope link
192.168.51.0/24 dev enp0s3 proto kernel scope link src 192.168.51.2
192.168.51.0/24 dev enp0s3 proto kernel scope link src 192.168.51.2 metric 100





# iptables -nvL FORWARD -t filter
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     all  --  Cabin    *       0.0.0.0/0            0.0.0.0/0

 

[ZebMcKayhan]

# ip rule
0:      from all lookup local
32766:  from all lookup main
32767:  from all lookup default




# ip route
default via 192.168.51.1 dev enp0s3
default via 192.168.51.1 dev enp0s3 proto dhcp src 192.168.51.2 metric 100
10.9.8.1 dev Cabin scope link
169.254.0.0/16 dev enp0s3 scope link
169.254.0.0/16 dev enp0s3 proto dhcp scope link src 192.168.51.2 metric 100
192.168.50.0/24 dev Cabin scope link
192.168.51.0/24 dev enp0s3 proto kernel scope link src 192.168.51.2
192.168.51.0/24 dev enp0s3 proto kernel scope link src 192.168.51.2 metric 100





# iptables -nvL FORWARD -t filter
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     all  --  Cabin    *       0.0.0.0/0            0.0.0.0/0

Hmm... this is no router right? This is a device on a network?

Probably IP forwarding is not enabled on the device... try to google too find how to enable it... too tired now, continue tomorrow...

 

[evlo]

yes, it is oracle linux

you are right

cat /proc/sys/net/ipv4/ip_forward

, but 100% i did set it to 1

echo 1 > /proc/sys/net/ipv4/ip_forward
cat /proc/sys/net/ipv4/ip_forward

I can ping asus router from device behind other side gateway
I can ping devices behind asus router from device behind other side gateway

I can ping devices behind other side gateway from device behind asus router


So weird, that:
1. with open openVPN lately i was never able to ping devices behind other side gateway (openVPN server)
2. ipv4 forwarding got disabled - well mainly this is confusing to me, I'm still new to linux
3. it works with flow cache enabled and even wan speeds are not bad

Also I noticed that with wireguard first 2 "pings" to the device take 100ms and then normal 15ms, while with openVPN it is 15ms all the time.


Thanks for your help.

 

[ZebMcKayhan]

I can ping devices behind asus router from device behind other side gateway

I can ping devices behind other side gateway from device behind asus router

Great!

Altough I must say I dont really understand why this is working on Cabin side devices, as this is not a router. It would not be normal for packages to appear to this ip for unknown (Home) destinations. Unknown destinations would appear in the router forward zone and since this network is unknown for the router it would follow default route, which means out WAN. Altough I'm not that into all mysteries of ipv4 routing, perhaps some kind of auto-discovery takes place.

Also I noticed that with wireguard first 2 "pings" to the device take 100ms and then normal 15ms, while with openVPN it is 15ms all the time.

This could be a consequence of this auto-discovery(?). Don't know why OpenVPN would be any different unless it uses MAQUARADE on Cabin side so the devices thinks all remote packages comes from Cabin local ip. But that solution should be avoided in my opinion, but might prove nessisary in the end for full access to the clients behind the peer as some devices (like Windows) could be reluctant to accept packages from other networks.

A better solution is if you can access your Cabin router (192.168.51.1?) and tell it to route packages to 192.168.50.0/24 via 192.168.51.2

That would hopefully take care of your ping times.

Speedtest result for WAN Bandwidth - Download: 912.48 Mbps (data used: 1.1 GB ) - Upload: 789.12 Mbps (data used: 980.2 MB ) Quality - Latency: 2.18 ms (0.12 ms jitter) - Packet Loss: 0.0%

What would be more interesting is your speed performance over the Wireguard tunnel. See if you could setup Iperf3 on each side and run a speed test.

Also, check your syslog for kernel error messages (blog mCast...)

Maybee site2site gets away with keeping fc enabled since there are no NAT taking place for packages going over the Wireguard tunnel. Fingers crossed!

 

[evlo]

there are things like

Jul 31 09:27:13 kernel: blog_link: 18 callbacks suppressed
Jul 31 09:27:13 kernel: blog_link:overwriting ct_p=ffffffc0103b9e20, new_ct=ffffffc01029de20 idx=0
Jul 31 09:27:13 kernel:         NFCT: ct<0xffffffc0103b9e20>, master<0x          (null)>
Jul 31 09:27:13 kernel:                 F_NAT<ffffffc0114ad5f8> keys[0x00000000 0x00000000] dir<DIR_ORIG>
Jul 31 09:27:13 kernel:                 help<0x          (null)> helper<NONE> status=8000018e refcnt=3 zone=0
Jul 31 09:27:13 kernel: tuple ffffffc0103b9eb8: 17 IP:61820 -> IP:61821
Jul 31 09:27:13 kernel: tuple ffffffc0103b9ef0: 17 IP:61821 -> IP:61820
Jul 31 09:27:13 kernel:                 STATUS[ IPS_SEEN_REPLY_BIT IPS_ASSURED_BIT IPS_CONFIRMED_BIT IPS_SRC_NAT_DONE_BIT IPS_DST_NAT_DONE_BIT IPS_BLOG_BIT ]

in the syslog

other then that I still can't ping stuff behind other side gateway from asus router itself, but site2site all good from both sides

A better solution is if you can access your Cabin router (192.168.51.1?) and tell it to route packages to 192.168.50.0/24 via 192.168.51.2

I forgot to worte that, that is already setup as route

Iperf I may try later today, no promises (i don't have any fast enough wired linux or even windows device at home )

I was still not able to get avahi-reflector or whatever other way to get mdns working. I now consider mDNS over site2site VPN nice to have as I was not able to get it working with openVPN also.

Asus

nano /tmp/avahi/avahi-daemon.conf

[Server]
host-name=asusAX68U
aliases=RT-AX68U
aliases_llmnr=RT-AX68U
use-ipv4=yes
use-ipv6=no
ratelimit-interval-usec=1000000
ratelimit-burst=1000

[publish]
publish-a-on-ipv6=no
publish-aaaa-on-ipv4=no
enable-dbus=yes

[reflector]
enable-reflector=yes

[wide-area]
enable-wide-area=yes

Cabin linux

[server]
use-ipv4=yes
use-ipv6=no
enable-dbus=yes
allow-point-to-point=yes
ratelimit-interval-usec=1000000
ratelimit-burst=1000

[wide-area]
enable-wide-area=yes

[publish]
publish-addresses=yes

[reflector]
enable-reflector=yes

pretty much everything in log on cabin server is this (nothing about 192.168.50.x)

Jul 31 07:41:29 openvpn avahi-daemon[1510741]: Joining mDNS multicast group on interface enp0s3.IPv4 with address 192.168.51.2.
Jul 31 07:41:29 openvpn avahi-daemon[1510741]: New relevant interface enp0s3.IPv4 for mDNS.
Jul 31 07:41:29 openvpn avahi-daemon[1510741]: Network interface enumeration completed.
Jul 31 07:41:29 openvpn avahi-daemon[1510741]: Registering new address record for 192.168.51.2 on enp0s3.IPv4.

 

[ZebMcKayhan]

in the syslog

Same as reported here:
https://www.snbforums.com/threads/syslog-filled-with-errors-386-7_2.80083/
could be fc interfering. If it seems to affect things you might need to disable fc. If not you could filter them out using I.e Scribe addon.

other then that I still can't ping stuff behind other side gateway from asus router itself, but site2site all good from both sides

Probably because router is using wg21 ip as source. If you ping using br0 ip as source it would probably work. You could also add on Cabin router route 10.9.8.0/24 via 192.168.51.2 (site2site peer ips, which could be a good idea anyway)

I was still not able to get avahi-reflector or whatever other way to get mdns working. I now consider mDNS over site2site VPN nice to have as I was not able to get it working with openVPN also.

I cannot help much with this as Im not (have never) used any of this, but rock-on! And please report back your findings!

 

[evlo]

10 streams

Probably CPU limit on the other peer, this is 4 streams

10 streams 30 seconds
I think CPU on other is really the limit here (for WG not for iPerf stream generating as it is on some routers)

 

[ZebMcKayhan]

View attachment 43233
10 streams

View attachment 43237
View attachment 43238
Probably CPU limit on the other peer, this is 4 streams
View attachment 43240
10 streams 30 seconds
I think CPU on other is really the limit here (for WG not for iPerf stream generating as it is on some routers)
View attachment 43242

Well, doesnt look like there are issues with flow cache. How are the syslog messages? Once now and then or flooding it?

Let me know when you want to add the 3rd peer, or try it yourself and report back your findings!