Unable to connect client Win 10 laptop to Asus RT-AC86U OpenVPN server

[ItsMark]

I am a newly registered forum member who is trying to utilize the Asus RT-AC86U wireless
router, running Asuswrt-Merlin firmware, to set up a OpenVPN server so that I may remotely
connect to one of my LAN PCs through a laptop client while traveling.

To begin, here are some details of my home network:

(1) DSL internet comes to my home through a Westell 6100 modem (very old, circa 2004).
Modem is NAT'd before going to Asus RT-AC86U router (gateway). The Westell 6100 modem is
connected to the Asus router's WAN port.

(2) The Asus RT-AC86U router is running Asuswrt-Merlin firmware, Ver. 384.8_2.

(3) I have registered with No-IP for DDNS.

(4) My router is running custom script "ddns-start" to circumvent double-NAT. The script
taken from the following link:
https://github.com/Meliox/Utils/blob/master/ddns-start/ddns-start


I am trying to establish a VPN tunnel between the OpenVPN server running on the router and
a client device. The client device is a Windows 10 Home laptop which is running OpenVPN
GUI. The laptop is internet connected via wi-fi, using my cellular provider's LTE for data
access (T-Mobile Hotspot).

Right now, I am unable to get the client to connect. When I attempt to initiate a VPN
connection to the Asus router from the client, the OpenVPN GUI status basically tries to
connect, then shows a timeout, then attempts to reconnect. Then it tries again, times out
again, and tries again... and so on. I am hoping that maybe someone here can review some
logs and supporting data to help me find out what needs to be done to establish the VPN
connection.


Here is all the relevant data I could think to assemble here for review.

[EDIT] Unable to paste logs, due to Cloudflare Ray ID: 498b88a47d0d20c6. All I could upload are my Asus RT-AC86U OpenVPN server settings (attached image). How may I paste log data to this forum?

 

[ColinTaylor]

(1) DSL internet comes to my home through a Westell 6100 modem (very old, circa 2004).
Modem is NAT'd before going to Asus RT-AC86U router (gateway). The Westell 6100 modem is
connected to the Asus router's WAN port.

I'm not familiar with this device. Can you clarify whether this is a modem or modem/router. Is it in bridge mode? If not how are you port forwarding to the Asus.

How may I paste log data to this forum?

You'll have to upload it to somewhere like pastebin and post a link to it.

 

[martinr]

Just to rule one thing out, you could temporarily change the setting “Username/password auth only” from No to Yes (and Apply). So if there was a problem with the public key infrastructure (keys and certs) you would then be able to connect.

But as Colin is inferring, the complexity of your setup is such that the problem almost certainly lies elsewhere.

The log you’re trying to upload, is that the OpenVPN log, which might just explain where the problem is?

Is this the first time you’ve set up OpenVPN or are you familiar with it?

 

[elorimer]

If you can, you might start by setting the modem to bridge mode: http://www.dslreports.com/faq/13600
With the 86U I wouldn't think you need the routing function of the Westell, and then everything will be simpler. You have several discrete pieces to work out but what you are trying to do is routinely done.

 

[ItsMark]

Wow, you guys are awesome! I followed the directions in the dslreports link (thanks elorimer) and switched the Westell device (which turns out was a modem/router) to bridge mode, rebooted everything, and then the OpenVPN client was able to connect to the OpenVPN server on the Asus router. Thanks so much for the assistance!

@martinr... this is my very first time working with OpenVPN and I probably have encountered every possible roadblock. But hopefully it's apparent that I did do my homework prior to posting here. Thanks again for helping me set up my first VPN!

Quick question... since the modem is in bridge mode, does that mean that its NAT has been disabled? And, if so, would the "ddns-start" script no longer be necessary since the network is no longer running double-NAT?

 

[elorimer]

Quick question... since the modem is in bridge mode, does that mean that its NAT has been disabled? And, if so, would the "ddns-start" script no longer be necessary since the network is no longer running double-NAT?

That might depend. With the modem in bridge mode, your gateway is seeing on its WAN side the same IP address your ISP is giving the modem. Is that a routable address? When I had a DSL connection Verizon was handing out a non routable address.

Now that you have it working, there are some other things you can do.

1. Get rid of compression in your server setup (you will have to re-export the config). If you are RDPing into the LAN PC it is compressed anyway.

2. For a further simplification, you can use the Asus ddns service built in to the router. Works fine for me, doing the same thing you want to do.

3. Another step you'll want is to run a script that wakes up your LAN PC when you make the OpenVPN connection, rather than having to go first to the router page to do WOL.

4. Consider setting up the second OpenVPN server on port 443 and exporting that configuration as well. I have found some hotspots where that connection would work, and one on 1194 wouldn't. Also, if you mess up one, you can still connect with the other and fix it when you are off-site.

 

[ItsMark]

That might depend. With the modem in bridge mode, your gateway is seeing on its WAN side the same IP address your ISP is giving the modem. Is that a routable address? When I had a DSL connection Verizon was handing out a non routable address.

Now that you have it working, there are some other things you can do.

1. Get rid of compression in your server setup (you will have to re-export the config). If you are RDPing into the LAN PC it is compressed anyway.

2. For a further simplification, you can use the Asus ddns service built in to the router. Works fine for me, doing the same thing you want to do.

3. Another step you'll want is to run a script that wakes up your LAN PC when you make the OpenVPN connection, rather than having to go first to the router page to do WOL.

4. Consider setting up the second OpenVPN server on port 443 and exporting that configuration as well. I have found some hotspots where that connection would work, and one on 1194 wouldn't. Also, if you mess up one, you can still connect with the other and fix it when you are off-site.

Thanks so much for the tips. I really appreciate it. I did actually set the VPN up for connecting via RDP to one of my home PCs from the laptop client. This was accomplished, but the endeavor has me now thinking that I should probably upgrade my internet service from the current ancient DSL (upload/download speeds are 128Kbps/1Mbps) to something better.

One last question... When I first request the RDP to my home PC, I receive the message shown in the screenshot (picture attached). Is this because I'm connecting to a Win7 Pro machine from a client running Win10?

Thanks again to everyone who took the time to help!

 

[elorimer]

I get that message first time for every combo. Check the box and click through.

On the speed, go into the Win10 client options and reduce the color depth, turn off the animations, etc. Keep OpenVPN as a TUN and not a TAP to avoid network traffic.

I don't recommend it, but you can also forward port 3389 on the router and connect directly to the PC. You then have an encrypted connection directly between the client and the home pc. Over OpenVPN you are encrypting twice on the Win10 client. once on the router, and once on the home pc. You can look at your utilization on the Win10 client and see if it is straining, but I think it is unlikely. I used to do your setup with a laptop with a little dual core atom and 2GB of memory and it would bog down a bit.

I don't recommend the direct option because you can't wake the PC that way, Since my home pc is a beast leaving it on would cost me bunches.

 

[Grisu]

Merlin added double NAT support for DDNS in his last firmware releases, there is no need to do anything else.
But now you dont even need this as there is nomore double/multi-NAT in your config (modem in bridge mode).