What's new

Setting up Merlin VPN and OpenVPN Server and Windows Client

  • SNBForums Code of Conduct

    SNBForums is a community for everyone, no matter what their level of experience.

    Please be tolerant and patient of others, especially newcomers. We are all here to share and learn!

    The rules are simple: Be patient, be nice, be helpful or be gone!

gbguy71

Occasional Visitor
Setting up Merlin VPN and OpenVPN 2.2.2 Server and Windows Client

[NOTE: there are unresolved issues with the Windows installation of OpenVPN 2.3.3 due to the separation of Easy-RSA from the OpenVPN install package. As of this date the issues are being worked. You may be able to find answers here https://forums.openvpn.net/server-administration-f4.html

[Some additional "learning opportunities" from when I updated to a newer Merlin release and reinstalled OpenVPN]

I encountered some issues/”learning opportunities” when setting up my ASUS RT-N66U as a VPN server. I noticed that others had similar experiences, so this post may be helpful to future VPN users. I’m a noob at this, so experts feel free to correct me. I’ll edit this as needed. I’m not an Apple guy, so I’ll concentrate on Windows clients and you’ll need to go elsewhere for setting up the router as an OpenVPN client.

My approach is to reference other posts/articles, but with additional notes so that some issues can be avoided.

First, the Merlin firmware offers two VPN implementations: the basic ASUS PPTP (Point to Point Tunnel Protocol) and OpenVPN. PPTP is simple to set up but not totally secure. OpenVPN is more complex to set up, but much more secure. Look at their Wikipedia entries for more information.

First, before you do anything else, determine if your router has a public or an internal IP address. You need to have a public one for any of this work. This post on the ASUS forum tells you how using www.whatismyip.com. My ISP’s DSL modem was configured as a router. They changed it to a bridge for a one-time charge (you do NOT need to get a static IP address assigned. We’ll talk later about assigning a hostname that will handle a router’s possibly changing IP address).

Setting Up the ASUS PPTP Server

Even if you know you want to exclusively use OpenVPN you might want to go ahead and try the PPTP server just to make sure all your connections work before you jump into setting up OpenVPN. ASUS provides a good guide here. Windows has PPTP client support built into it, so once the router is set up you can get going in a heartbeat. There are a number of good articles on how to set up a VPN client on the web. This one is pretty good. If you decide you don’t want to continue using the PPTP server be sure and disable it in the router. No sense wasting router resources.

NOTE: When you look at the Windows Network and Sharing Center you’ll see the VPN connection.

Setting up the OpenVPN Server

The key article is this one about configuring a Tomato router and Windows clients (skip down to Configuring OpenVPN). You obviously don’t need to add the OpenVPN software to the router, but the setup instructions for the router and the windows client are good. Here are some additional points:

  1. The article assumes you’ll use the OpenVPN GUI that is part of the OpenVPN distribution. From what I’ve heard it isn’t the most current. Instead, install this version from SourceForge (openvpn-gui.exe). [My guess at the installation instructions:] You need to download it and copy it into the OpenVPN\bin directory once OpenVPN, including its OpenVPN GUI, has been installed. I renamed the original openvpn-gui, but kept it around.
  2. When you edit the files (e.g., vars.bat) run Notepad++ as administrator.
  3. Use this OpenVPN document (Setting up your own Certificate Authority (CA) and generating certificates and keys for an OpenVPN server and multiple clients) to guide you with setting up the keys, etc. One thing this article does is emphasize that your "common names" need to be different for each client, which caused me problems.
  4. New info for Windows:. I made these changes to the vars.bat file to resolve some issues with easy-rsa 2.2.2 (Easy-Rsa 3.x uses a different installation method that I have not used):

    1. added "path=%ProgramFiles%\OpenVPN\bin" so that openssl.exe could be found.

    2. changed the set KEY_CONFIG=openssl-1.0.0.cnf which has an illegal filename to set KEY_CONFIG=openssl.cnf
    (I first copied the openssl-1.0.0.cnf file (via renaming) to openssl.cnf)
  5. Be sure and include the -----BEGIN CERTIFICATE-----/ -----END CERTIFICATE----- , -----BEGIN PRIVATE KEY-----/ -----END PRIVATE KEY-----, and -----BEGIN DH PARAMETERS----- , -----END DH PARAMETERS----- lines of text in the router parameters.
  6. The newer Merlin releases have the VPN keys, certificates in a "VPN Details" tab. Click on "Content modification of Keys and Certificates" to access/update them.
  7. Disabled LAN > Switch Control's "NAT Acceleration" so DNS would work. (I don't recall having done that with my earlier Merlin release).
  8. The OpenVPN download is for everything, server and client. I don’t know if there is a minimal download for a strict client, one in which the keys are generated on a different machine.
  9. Don’t ask me about their recommended VPN server parameters (Push LAN to clients, etc.) I don’t know what they mean :)

NOTE: The VPN connection will NOT be visible in the Windows Network and Sharing Center. You’ll be able check the status from the Open GUI hidden icon.

This article covers much of the same stuff, though in less detail. Its main advantage is that it is from the OpenVPN group, so that if something changes it should be up to date.



Miscellaneous Goodies

If your router provides it, use the ASUS DDNS service to provide a hostname for your router. It will automatically be updated if your router’s IP address changes. Many of the tutorial articles suggest using DynDNS. However, in their free version your hostname registration will expire if your IP address doesn’t change for 30 days [I can’t find a concise reference for this].

How to set up your router so that it can be managed outside your local network.

If you want to be able to ping your router (good for testing) go the Firewall section and enable “Respond Ping Request from WAN”

If you are interested in a Static Key OpenVPN set up, this mini-article explains how.

General How To topics on “all things OpenVPN”.

Hope this will make things a little easier for you.
 
Last edited:
Revoke client

Hi,
I'm using RT-N66U with Merlin build 3.0.0.4.372.30_3.
I have successfully set up OpenVPN server on the router.
May I know how I can revoke client cert?
Thanks a lot.
 
I am trying to set this up and the screens on Tomato shown in the HowToGeek tutorial are somewhat different than the 37.40 Merlin pages. In particular I don't see any place in the Merlin pages where you import the Keys generated on your PC. Am I missing something? Where to you input the keys in Merlin?
 
I am trying to set this up and the screens on Tomato shown in the HowToGeek tutorial are somewhat different than the 37.40 Merlin pages. In particular I don't see any place in the Merlin pages where you import the Keys generated on your PC. Am I missing something? Where to you input the keys in Merlin?

There's a link that says " Content modification of Keys & Certificates." on that page - just click on it.
 
I'm trying to setup an OpenVPN server on my RT-AC68U running Merlin. I realize this thread is a number of years old now and somewhat out of date, but I'm running into trouble and not really sure where to start. I've been following the guide from here, which makes it seem like it is a bit more straightforward than when this post was written. Everything seems good up until step 8. It goes to connecting and posts the following pretty quickly

Thu Jan 03 15:15:30 2019 OpenVPN 2.4.6 x86_64-w64-mingw32 [SSL (OpenSSL)] [LZO] [LZ4] [PKCS11] [AEAD] built on Apr 26 2018
Thu Jan 03 15:15:30 2019 Windows version 6.2 (Windows 8 or greater) 64bit
Thu Jan 03 15:15:30 2019 library versions: OpenSSL 1.1.0h 27 Mar 2018, LZO 2.10
Thu Jan 03 15:15:30 2019 WARNING: --ns-cert-type is DEPRECATED. Use --remote-cert-tls instead.
Thu Jan 03 15:15:31 2019 TCP/UDP: Preserving recently used remote address: [AF_INET]xx.xxx.xxx.xxx:1194
Thu Jan 03 15:15:31 2019 UDP link local: (not bound)
Thu Jan 03 15:15:31 2019 UDP link remote: [AF_INET]xx.xxx.xxx.xxx:1194

But then times out. I never get prompted to enter a username and password. I can't stress enough how much of a noob I am at this kind of thing. Trying to learn by doing, but not even really sure where to start. So far the only troubleshooting step I've done is to make sure I've got an exception in Windows Firewall for OpenVPN. The other troubleshooting steps I've come across are either conflicting with one another, or it isn't clear that they are current and applicable to setting up on a router.

Would greatly appreciate if anyone could offer a suggestion, or point me in the right direction for background research.
 

Latest threads

Support SNBForums w/ Amazon

If you'd like to support SNBForums, just use this link and buy anything on Amazon. Thanks!

Sign Up For SNBForums Daily Digest

Get an update of what's new every day delivered to your mailbox. Sign up here!
Top