x3mRouting x3mRouting ~ Selective Routing for Asuswrt-Merlin Firmware (1-Nov-2020)

[Brainstorm]

Look in the System Log for clues as to why the VPN connection is having issues. Most providers allow you to create and import an ovpn config file which helps eliminates errors. Policy Rules (Strict) setting needs to be enabled in the OpenVPN client screen for x3mRouting to work.

System log says "ovpn client Missing or incorrect parameter" Use --help for more information. Where to use --help? In the ssh shell? Can't get that to work. I'm using expressvpn's imported config file. It works fine before I load the x3Rmrouting files, it fails after I load them. Every time. Am I the only one with this issue?

 

[Brainstorm]

Well, no replies or help here......No matter what I do, loading x3MRouting files on my Asus AC86U causes express vpn configs to fail, and I don't know where to begin looking to diagnose the issue.

 

[fsb]

Well, no replies or help here......No matter what I do, loading x3MRouting files on my Asus AC86U causes express vpn configs to fail, and I don't know where to begin looking to diagnose the issue.

The Asus log file should point you to the correct place in the .ovpn file where the problem lies.

I have the same router and Expressvpn. You have to open the Expressvpn .opvn file in a text editor and delete the line "verify-x509-name Server name-prefix".

For me, it wasn't the x3MRouting files that caused the problem, it was that line in the .ovpn file.

Do that, save the file using the same name, then load that into the router like normal. I can't speak for what this does, exactly, but the Asus log file gave me that line name as causing the error. Once I did that, it worked perfectly.

 

[Brainstorm]

fsb. Thanks for that. I'll give it a try in the morning. I can see that line in the .ovpn file. I'll comment it out and try again. Just for my info, did you have a problem with express vpn connections BEFORE you loaded the x3mRouting files? I ask because express works perfectly for me, until I load the files. Once I've loaded x3mRouting, NONE of my express vpn's will connect. And did you load ALL the x3m files (amtm selections 1,2,3 and 4)?
Also, how is your router log file set up? I have mine logging everything it can, but all I see when the vpn connection fails is "ovpn client missing or incorrect parameter" closely followed by a "sigterm" killing the process. In the openvpn page, next to the ON/OFF button, it says "Error-check configuration", and to recover, I have to reload the .ovpn files and username/passwords before it will work again.

 

[Jack Yaz]

fsb. Thanks for that. I'll give it a try in the morning. I can see that line in the .ovpn file. I'll comment it out and try again. Just for my info, did you have a problem with express vpn connections BEFORE you loaded the x3mRouting files? I ask because express works perfectly for me, until I load the files. Once I've loaded x3mRouting, NONE of my express vpn's will connect. And did you load ALL the x3m files (amtm selections 1,2,3 and 4)?
Also, how is your router log file set up? I have mine logging everything it can, but all I see when the vpn connection fails is "ovpn client missing or incorrect parameter" closely followed by a "sigterm" killing the process. In the openvpn page, next to the ON/OFF button, it says "Error-check configuration", and to recover, I have to reload the .ovpn files and username/passwords before it will work again.

i'd check the custom config box on the ovpn client screen, sounds like something might be amiss there.
and double check that the rest of the fields are filled in

 

[Brainstorm]

The custom config box stays the same, line for line, all other fields are exactly the same, whether the x3 scripts are installed or not.
I've taken a look at the Openvpn MAN page for info about "verify-x509-name Server name-prefix", but, to be honest, not sure what it all means. Could it be that this command is deprecated? Is it dependent on which OpenVpn version is being used?
It LOOKS like you can use tls authorisation and/or verify-x509 name Server name-prefix.
I have found if I disable "Verify Server Certificate" in the OpenVPN clients page, Express vpn works whether x3 files are loaded or not. I'm sure there's some vpn gurus on here that can explain the difference, and advise if doing this is dangerous from a security perspective. I note NORD vpn has this option disabled, but it's not in their .ovpn files, either.
Where's Colin when you need him?
Also, could someone PLEASE explain how to make the syslog files more verbose? Not sure what I'm doing wrong, but I definitely don't see the detail fsb described in his earlier post.

 

[Brainstorm]

Aaaargh. Think I've found the issue. When I load amtm option 2, OpenVpn client GUI, the option "Verify Server Certificate" in "Client Settings" gets set to a NULL value ie: neither YES or NO is set. I tried to set it to YES and put in the name "Server", but it won't accept itand doesn't work. If I set it to NO, the Expressvpn works fine. Not sure why setting this option to YES doesn't work. Am I putting the wrong name in the option box?

 

[fsb]

fsb. Thanks for that. I'll give it a try in the morning. I can see that line in the .ovpn file. I'll comment it out and try again. Just for my info, did you have a problem with express vpn connections BEFORE you loaded the x3mRouting files? I ask because express works perfectly for me, until I load the files. Once I've loaded x3mRouting, NONE of my express vpn's will connect. And did you load ALL the x3m files (amtm selections 1,2,3 and 4)?

I didn't try to load the .ovpn file before I installed x3MRouting, sorry. I installed Merlin first, then did x3MRouting, then loaded different EVPN .ovpn files. I have 4 different files for different cities around me and I had to delete the same line from all of them to get them to work.

Also, how is your router log file set up? I have mine logging everything it can, but all I see when the vpn connection fails is "ovpn client missing or incorrect parameter" closely followed by a "sigterm" killing the process. In the openvpn page, next to the ON/OFF button, it says "Error-check configuration", and to recover, I have to reload the .ovpn files and username/passwords before it will work again.

My log had these entries:

ovpn-client2[19828]: Options error: Unrecognized option or missing or extra parameter(s) in config.ovpn:18: verify-x509-name (2.5.2)

That's how I knew which line was causing the problem. Once I deleted that line and reloaded the file, using the username and password provided by ExpressVPN, I wouldn't get the error (the same one you saw).

My 'Verify Server Certificate' setting is set to No.

 

[fsb]

Also, could someone PLEASE explain how to make the syslog files more verbose? Not sure what I'm doing wrong, but I definitely don't see the detail fsb described in his earlier post.

My log is:

• Default message log level: warning
• Log only messages more urgent than: warning

I'm running Merlin v386.2_4, if that helps.

 

[Background]

Where is the VPN > OpenVPN Clients tab? Can't find "Policy Rules" I only want one device to use a VPN but cant work out how AC88U 386.2_beta3.

 

[Xentrk]

Where is the VPN > OpenVPN Clients tab? Can't find "Policy Rules" I only want one device to use a VPN but cant work out how AC88U 386.2_beta3.

I posted OpenVPN Client instructions that may help on my blog site. I need to update it but it will give you an overview.

 

[Xentrk]

My log is:

• Default message log level: warning
• Log only messages more urgent than: warning

I'm running Merlin v386.2_4, if that helps.

 

[Brainstorm]

My log is:

• Default message log level: warning
• Log only messages more urgent than: warning

I'm running Merlin v386.2_4, if that helps.

I've installed "OpenVPN Event & x3mRouting.sh Script". I've installed "x3mRouting Utility Scripts".
I have 5 vpn's installed, 3 Express, 2 Nord vpn. I've created static ip's for 5 clients under DHPC server. I've removed them from all the vpn configurations. Policy rules are STRICT, DNS is EXCLUSIVE
I've enabled dnsmasq logging.
I haven't installed or run LAN CLIENT ROUTING.
I've run getdomainnames, I can see there's content there when I control C, I can see the files created ie: NOWTV, when I use "liststats", although to begin with sometimes they appear empty. They also increment, with no input from me.
I then run, for example "x3mRouting ALL 4 NOWTV autoscan=now" I can see scripts running, it looks like it's been successful. But nothing works. Now TV says not available in my country (I'm abroad at the moment).
I can see IPSET appears in the /jffs/configs/dnsmasq.conf.add file
VPN 4 is nordvpn. It's connected, if I put a DHCP static ip device in the VPN client screen "Rules for routing client traffic through the tunnel", it works. I can access UK NOW TV. This behaviour is the same for anything I configure using x3mRouting.
Now I've installed and run LAN client routing. I've left the file un-edited.
Still nothing works.
I'm obviously still doing something fundamentally wrong. Can anyone help please?

 

[Brainstorm]

Sorry. Thanks for the added info, fsb. Muchos Gracias.......

 

[Xentrk]

I've installed "OpenVPN Event & x3mRouting.sh Script". I've installed "x3mRouting Utility Scripts".
I have 5 vpn's installed, 3 Express, 2 Nord vpn. I've created static ip's for 5 clients under DHPC server. I've removed them from all the vpn configurations. Policy rules are STRICT, DNS is EXCLUSIVE
I've enabled dnsmasq logging.
I haven't installed or run LAN CLIENT ROUTING.
I've run getdomainnames, I can see there's content there when I control C, I can see the files created ie: NOWTV, when I use "liststats", although to begin with sometimes they appear empty. They also increment, with no input from me.
I then run, for example "x3mRouting ALL 4 NOWTV autoscan=now" I can see scripts running, it looks like it's been successful. But nothing works. Now TV says not available in my country (I'm abroad at the moment).
I can see IPSET appears in the /jffs/configs/dnsmasq.conf.add file
VPN 4 is nordvpn. It's connected, if I put a DHCP static ip device in the VPN client screen "Rules for routing client traffic through the tunnel", it works. I can access UK NOW TV. This behaviour is the same for anything I configure using x3mRouting.
Now I've installed and run LAN client routing. I've left the file un-edited.
Still nothing works.
I'm obviously still doing something fundamentally wrong. Can anyone help please?

Additional analysis is required to determine the domain names being used. x3mRouting has the scripts/tools to help.

First, you need to configure your VPN client to force ALL network traffic or the device you are streaming NOW TV from to use the VPN Client required with NOW TV. Then, run getdomainnames.sh (see complete instructions at link) script to get a bigger picture of the domains being used for NOW TV. The laptop or computer you are using to SSH into the router will also generate some domain traffic from background processes running. Best to close all browsers too the analysis to minimize noise. Then, you need to go to the app or browser and select menu options and stream to generate traffic. After selecting all of the options, you can press ctrl-c to exit and produce the report.

Then use the autoscan.sh script to narrow down the domains from the terms you think belong to NOWTV and exclude the background domains. (e.g. autoscan=term1,term2,term3).

As part of the analysis, you can also view the website page source code for .com or .net for clues as to what domains are being used by right clicking on the webpage and selecting the view source code option.

You can also try ASN method. However, I still recommend doing the analysis as many media companies hosting on AWS or using content delivery networks.

# asn nowtv.com

----------------------------
| ASN lookup for nowtv.com |
----------------------------

- Resolving "nowtv.com"... 1 IP address found:

 90.216.151.68 +PTR -
               +ASN 5607 (BSKYB-BROADBAND-AS, GB)
               +ORG BSKYB-BROADBAND-AS
               +NET 90.192.0.0/11 (BSKYB-BROADBAND)
               +ABU abuse@sky.uk
               +GEO London, England (GB)


Tracing path to 90.216.151.68 (press CTRL-C to cancel)...^C
Interrupted
<snip>

 

[Brainstorm]

Additional analysis is required to determine the domain names being used. x3mRouting has the scripts/tools to help.

First, you need to configure your VPN client to force ALL network traffic or the device you are streaming NOW TV from to use the VPN Client required with NOW TV. Then, run getdomainnames.sh (see complete instructions at link) script to get a bigger picture of the domains being used for NOW TV. The laptop or computer you are using to SSH into the router will also generate some domain traffic from background processes running. Best to close all browsers too the analysis to minimize noise. Then, you need to go to the app or browser and select menu options and stream to generate traffic. After selecting all of the options, you can press ctrl-c to exit and produce the report.

Then use the autoscan.sh script to narrow down the domains from the terms you think belong to NOWTV and exclude the background domains. (e.g. autoscan=term1,term2,term3).

As part of the analysis, you can also view the website page source code for .com or .net for clues as to what domains are being used by right clicking on the webpage and selecting the view source code option.

You can also try ASN method. However, I still recommend doing the analysis as many media companies hosting on AWS or using content delivery networks.

# asn nowtv.com

----------------------------
| ASN lookup for nowtv.com |
----------------------------

- Resolving "nowtv.com"... 1 IP address found:

90.216.151.68 +PTR -
               +ASN 5607 (BSKYB-BROADBAND-AS, GB)
               +ORG BSKYB-BROADBAND-AS
               +NET 90.192.0.0/11 (BSKYB-BROADBAND)
               +ABU abuse@sky.uk
               +GEO London, England (GB)


Tracing path to 90.216.151.68 (press CTRL-C to cancel)...^C
Interrupted
<snip>

Hi Xentrk. Thanks for your time and patience.
I tried what you suggested. It looks logical and makes sense that maybe I'd not done enough analysis. However.....
I have 2 devices I want to use for different TV apps. A 2018 LG OLED, and a 4k Roku stick.
Here's what I found when I tried your suggestions, just using NOW TV for test purposes.
I put the TV in VPN 4, a nordvpn connection. NOW TV works fine.
I run getdomainnames, file name NOWTV. ip address of the TV, control C, file is empty. If I take the TV OUT of the vpn, NOW TV fails because of a geolocation error, but I DO see output to the file, plenty of it. I autoscan for sky, now, lightyear, amazon, which all appear in the getdomainnames output. I can see in the IPSET file "NOWTV" lots of entries for all of those.
Then, with the TV still in the VPN, I try Now TV and it works. However, obviously, ALL the TV traffic is now going down the VPN, which is not what I want. So I take the TV out of the vpn, NOW TV fails with a geolocation error.
Should the TV be IN the VPN, or OUT?
What exactly does the command "x3mRouting ALL 4 NOWTV autoscan=sky, now, amazon, lightyear" do? "ALL traffic to VPN 4 matching IPSET list NOWTV", yes? But it autoscanning the NOWTV IPSET file?
How do the entry numbers of the IPSET file increment on their own? What's happening there?
I'm sure I'm being a complete numpty and not understanding what everyone else seems to, but I'd really like to get this working, understand what it's doing, how it's doing it and how to use it to it's fullest potential.
Same scenario, but using te ROKU stick, I don't get ANY output from getdomainnames whether the Roku is IN the VPN or OUT.
I'll fully understand if I'm asking for too much support. I'm sure it already takes up a huge amount of your time...

 

[Brainstorm]

I run "x3mRouting ALL 5 ITV autoscan=itv" It looks like it's completed it ok. Then I run "x3mRouting 5 0 ITV", and I see "ERROR! The save/restore file opt/tmp/ITV does not exist". What does that mean? I see the file "ITV" if I do "liststats". I see "ERROR! The save/restore file opt/tmp/xxx does not exist" a lot.
I'm close to giving up because after hours of trying I still have no idea what's going on....am I really the only one here who can't get this to work? I DID get one service working as I wanted, but I found that it was also causing another service to go with it down it's chosen vpn, which stopped the other service working.

 

[Xentrk]

Hi Xentrk. Thanks for your time and patience.
I tried what you suggested. It looks logical and makes sense that maybe I'd not done enough analysis. However.....
I have 2 devices I want to use for different TV apps. A 2018 LG OLED, and a 4k Roku stick.
Here's what I found when I tried your suggestions, just using NOW TV for test purposes.
I put the TV in VPN 4, a nordvpn connection. NOW TV works fine.
I run getdomainnames, file name NOWTV. ip address of the TV, control C, file is empty. If I take the TV OUT of the vpn, NOW TV fails because of a geolocation error, but I DO see output to the file, plenty of it. I autoscan for sky, now, lightyear, amazon, which all appear in the getdomainnames output. I can see in the IPSET file "NOWTV" lots of entries for all of those.
Then, with the TV still in the VPN, I try Now TV and it works. However, obviously, ALL the TV traffic is now going down the VPN, which is not what I want. So I take the TV out of the vpn, NOW TV fails with a geolocation error.
Should the TV be IN the VPN, or OUT?
What exactly does the command "x3mRouting ALL 4 NOWTV autoscan=sky, now, amazon, lightyear" do? "ALL traffic to VPN 4 matching IPSET list NOWTV", yes? But it autoscanning the NOWTV IPSET file?
How do the entry numbers of the IPSET file increment on their own? What's happening there?
I'm sure I'm being a complete numpty and not understanding what everyone else seems to, but I'd really like to get this working, understand what it's doing, how it's doing it and how to use it to it's fullest potential.
Same scenario, but using te ROKU stick, I don't get ANY output from getdomainnames whether the Roku is IN the VPN or OUT.
I'll fully understand if I'm asking for too much support. I'm sure it already takes up a huge amount of your time...

If no records showed up in the report, then you have Policy Rules set to Strict and Accept DNS Configuration = Exclusive. When you have Accept DNS Configuration set to Exlusive + Policy Rules enabled, the vpn tunnel exclusively uses the DNS of the VPN provider, and dnsmasq is bypassed which prevents the dnsmasq method from working, and domains are not being logged to dnsmasq.log file.

Please see the work around solutions on the wiki Using dnsmasq with Policy Rules

dnsmasq will query the domains (query record) and a lookup is perfomed. Reply records are returned with domain name and IPv4 address. dnsmasq will load these IPv4 addresses into the ipset list. Man Page of IPSET.

 

[Xentrk]

I run "x3mRouting ALL 5 ITV autoscan=itv" It looks like it's completed it ok. Then I run "x3mRouting 5 0 ITV", and I see "ERROR! The save/restore file opt/tmp/ITV does not exist". What does that mean? I see the file "ITV" if I do "liststats". I see "ERROR! The save/restore file opt/tmp/xxx does not exist" a lot.
I'm close to giving up because after hours of trying I still have no idea what's going on....am I really the only one here who can't get this to work? I DID get one service working as I wanted, but I found that it was also causing another service to go with it down it's chosen vpn, which stopped the other service working.

You are using the "Manual" method with the command. x3mRouting is looking for the file containing the IPv4 addresses in /opt/tmp/ITV (unless you specified another directory location) that was manually created and it does not exist. So it throws the error.

The manual method is used to create IPSET lists from a file in the backup/restore directory containing the IPv4 addresses and/or IPv4 CIDR format that you created manually, either using an editor, script or other method to populate the file with IPv4 addresses

If you are going to use the manual method, you have to add IPv4 addresses to the file first.

 

[Brainstorm]

If no records showed up in the report, then you have Policy Rules set to Strict and Accept DNS Configuration = Exclusive. When you have Accept DNS Configuration set to Exlusive + Policy Rules enabled, the vpn tunnel exclusively uses the DNS of the VPN provider, and dnsmasq is bypassed which prevents the dnsmasq method from working, and domains are not being logged to dnsmasq.log file.

Please see the work around solutions on the wiki Using dnsmasq with Policy Rules

dnsmasq will query the domains (query record) and a lookup is perfomed. Reply records are returned with domain name and IPv4 address. dnsmasq will load these IPv4 addresses into the ipset list. Man Page of IPSET.

Once again, thanks for your time....
So the records I acquired by NOT going through the tunnel are valid, but won't work without DNS filtering, is that correct?
And it's not obvious which DNS server to configure in the filtering section. Can I put anything I like in there and it will work? The same DNS that I use for my wan connections? Google DNS?
How does using filtering stop the leaks?