x3mRouting x3mRouting ~ Selective Routing for Asuswrt-Merlin Firmware

[Xentrk]

Hi Xentrk,
I came from the first version of your script and updated it to the actual version a long time ago. It works more or less, because some routing didn´t work for my streaming service (ZDF).
If I am directly connected with the VPN via mobile app, it is working, but not with the router only. So some routing didn´t worked.
I wanted to solve the problem with you getdomainname script, but there came the problem.
I added as you described the logging lines to the file:
log-async
log-queries
log-facility=/opt/var/log/dnsmasq.log

and restarted dnsmasq.
After that time, no internet traffice for all my devices was possible.
I restarted the router no effect. I deleted the lines and the internet everything was fine.
What did I wrong ?

Thanks a lot for your support.

Hugo.


ug 9 11:18:44 watchdog: start ddns.
Aug 9 11:18:44 rc_service: watchdog 1317:notify_rc start_ddns
Aug 9 11:18:44 custom_script: Running /jffs/scripts/service-event (args: start ddns)
Aug 9 11:18:44 start_ddns: update CUSTOM , wan_unit 0
Aug 9 11:18:44 rc_service: watchdog 1317:notify_rc start_dnsmasq
Aug 9 11:18:44 custom_script: Running /jffs/scripts/service-event (args: start dnsmasq)
Aug 9 11:18:44 custom_config: Appending content of /jffs/configs/dnsmasq.conf.add.
Aug 9 11:18:44 custom_script: Running /jffs/scripts/dnsmasq.postconf (args: /etc/dnsmasq.conf)
Aug 9 11:18:44 Diversion: restarted Dnsmasq to apply settings
Aug 9 11:18:44 stubby[7797]: Read config from file /etc/stubby/stubby.yml
Aug 9 11:18:44 dnsmasq[7800]: illegal repeated keyword at line 60 of /etc/dnsmasq.conf
Aug 9 11:18:44 dnsmasq[7800]: FAILED to start up
Aug 9 11:18:44 wlceventd: WLCEVENTD wlceventd_proc_event(481): eth5: Disassoc 56:2D:B4:AC:56:FE, status: 0, reason: Disassociated because sending station is leaving (or has left) BSS (8)
Aug 9 11:19:09 rc_service: httpd 1309:notify_rc restart_rstats;restart_conntrack;restart_dnsmasq
Aug 9 11:19:09 custom_script: Running /jffs/scripts/service-event (args: restart rstats)
Aug 9 11:19:09 custom_script: Running /jffs/scripts/service-event (args: restart conntrack)
Aug 9 11:19:09 modprobe: module nf_conntrack_proto_gre not found in modules.dep
Aug 9 11:19:09 modprobe: module nf_nat_proto_gre not found in modules.dep
Aug 9 11:19:09 modprobe: module nf_conntrack_pptp not found in modules.dep
Aug 9 11:19:09 modprobe: module nf_nat_pptp not found in modules.dep
Aug 9 11:19:09 custom_script: Running /jffs/scripts/service-event (args: restart dnsmasq)
Aug 9 11:19:09 custom_config: Appending content of /jffs/configs/dnsmasq.conf.add.
Aug 9 11:19:09 custom_script: Running /jffs/scripts/dnsmasq.postconf (args: /etc/dnsmasq.conf)
Aug 9 11:19:10 Diversion: restarted Dnsmasq to apply settings
Aug 9 11:19:10 stubby[8069]: Read config from file /etc/stubby/stubby.yml
Aug 9 11:19:10 dnsmasq[8071]: illegal repeated keyword at line 60 of /etc/dnsmasq.conf
Aug 9 11:19:10 dnsmasq[8071]: FAILED to start up
Aug 9 11:19:14 watchdog: start ddns.
Aug 9 11:19:14 rc_service: watchdog 1317:notify_rc start_ddns
Aug 9 11:19:14 custom_script: Running /jffs/scripts/service-event (args: start ddns)
Aug 9 11:19:14 start_ddns: update CUSTOM , wan_unit 0
Aug 9 11:19:14 rc_service: watchdog 1317:notify_rc start_dnsmasq
Aug 9 11:19:14 custom_script: Running /jffs/scripts/service-event (args: start dnsmasq)
Aug 9 11:19:14 custom_config: Appending content of /jffs/configs/dnsmasq.conf.add.
Aug 9 11:19:14 custom_script: Running /jffs/scripts/dnsmasq.postconf (args: /etc/dnsmasq.conf)
Aug 9 11:19:14 Diversion: restarted Dnsmasq to apply settings

I suspect you have duplicate lines for the dnsmasq log parameters "illegal repeated keyword at line 60 of /etc/dnsmasq.conf". Diversion creates the logging from the /jffs/scripts/dnsmasq.postconf after applying any settings in dnsmasq.conf.add. So that is how the duplicates got there.

cat /etc/dnsmasq.conf

Delete the lines you added to dnsmasq.conf.add and restart dnsmasq. With Diversion, you also have the follow the log file option available for viewing dnsmasq entries.

 

[mister]

Hi Xentrk,
thanks a lot for your support. Diversion already added the logging in the dnsmasq.conf . So everything should be ok. But unfortunately my problems are still there. To be concrete

1. I want see a streamingvideo in the german mediathek. So my traffic should be routed via my VPN 1, which is located in germany. The link of the streamingadress begins with http://tvdlzdf-a.akamaidh.net/de/zdf........
So I added the lines
sh /jffs/scripts/x3mRouting/x3mRouting.sh ALL 1 Akamai4 autoscan=akamaihd,akamai
and
sh /jffs/scripts/x3mRouting/x3mRouting.sh ALL 1 Mediatheken dnsmasq=zdf.de,zdfmediathek.de,ard.de,wdr.de,kika.de,phoenix.de,swr.de,swrmediathek.de,br.de,ardmediathek.de,wdrmediathek.de,phoenix.de,akamaihd.net,hr.de,akamaistream.net,dw.de,sr-online.de,ndr.de,rbb-online.de,apa.at,tagesschau.de,heute.de,akamai.com

to the nat-start file.
But it doesn´t work. I got the error message, that the content is geoblocked.

If I am using the App of my VPN Provider und connect diretly with my smartphone to the server. It is working. (by the way: I tested the same VPN-Server - IP Adress)
Listats says:

Mediatheken - 2593
Akamai4 - 1

So, what do I wrong ? How to test, that the link (http://tvdlzdf-a.akamaidh.net/de/zdf....) is routed correctly?

PS:
I tested the logging - it should work, or ?

su@RT-AC86U-7298:/tmp/home/root# ps | grep "dnsmasq --log-async"
12158 su 5568 S grep dnsmasq --log-async
30323 nobody 9660 S dnsmasq --log-async
30325 su 5012 S dnsmasq --log-async

su@RT-AC86U-7298:/tmp/home/root# nvram get vpn_client1_adns
0



2. As I wrote before, I am using the VPNfailover script to switch between VPN1 and VPN2, to stay connected. In the "old" Version I had the VPN1client-down file to delete the routing rules with the del parameter . Do I have to add something new in another file, if I modify the routing rules in the nat-start file?

 

[obs0lete]

Everything checks out in the info you provided. What I don't see are any "query" records in dnsmasq.log. Go to a website and see if it logs the query record.. Also note the "ipset add" entries for IPSET lists in my log file snip below.

Aug  9 07:39:15 dnsmasq[32460]: ipset add MOVETV 67.26.57.252 cbd46b77.cdn.cms.movetv.com.c.footprint.net
Aug  9 07:39:15 dnsmasq[32460]: reply cbd46b77.cdn.cms.movetv.com.c.footprint.net is 67.26.57.252
Aug  9 07:39:16 dnsmasq[32460]: query[A] secure-dcr.imrworldwide.com from 192.168.22.165
Aug  9 07:39:16 dnsmasq[32460]: /opt/share/diversion/list/blockinglist secure-dcr.imrworldwide.com is 192.168.22.2
Aug  9 07:39:17 dnsmasq[32460]: query[A] ichnaea.netflix.com from 192.168.22.165
Aug  9 07:39:17 dnsmasq[32460]: cached ichnaea.netflix.com is <CNAME>
Aug  9 07:39:17 dnsmasq[32460]: cached ichnaea.geo.netflix.com is <CNAME>
Aug  9 07:39:17 dnsmasq[32460]: forwarded ichnaea.netflix.com to 1.1.1.1
Aug  9 07:39:17 dnsmasq[32460]: reply ichnaea.netflix.com is <CNAME>
Aug  9 07:39:17 dnsmasq[32460]: reply ichnaea.geo.netflix.com is <CNAME>
Aug  9 07:39:17 dnsmasq[32460]: ipset add NETFLIX-DNS 52.34.255.169 ichnaea.us-west-2.prodaa.netflix.com
Aug  9 07:39:17 dnsmasq[32460]: reply ichnaea.us-west-2.prodaa.netflix.com is 52.34.255.169
Aug  9 07:39:17 dnsmasq[32460]: ipset add NETFLIX-DNS 54.148.229.18 ichnaea.us-west-2.prodaa.netflix.com

tail -f /opt/var/log/dnsmasq.log | grep query

What router model and firmware release are you on?

Are you able to perform an nslookup on a domain? e.g. nslookup github.com
Do you see a query entry in dnsmasq.log? Try toggling the setting on/off Tools->Other Settings Wan: Use local caching DNS server as system resolver (default: No)[/URL] to see if it matters. Some of us experienced an issue with this.

On the LAN->DHCP Server tab, make sure DNS Server 1 and 2 are empty.

Are you using unbound?

If the above doesn't work, I can ask @thelonelycoder , the author of Diversion and amtm for his thoughts as to why the query records are not getting logged.

What router model and firmware release are you on?

I am using an ASUS RT-AC5000, Merlin FW 384.18.

Are you able to perform an nslookup on a domain? e.g. nslookup github.com

Yes, I can:

Server:         127.0.0.1
Address:        127.0.0.1#53

Non-authoritative answer:
Name:   github.com
Address: 140.82.113.4

Do you see a query entry in dnsmasq.log?

I am using Pi-hole with unbound, so my resolver logs are actually generated by unbound. Below is a quick sample of my logs:

[1596995098] unbound[593:0] info: 127.0.0.1 netflix.com. A IN
[1596995098] unbound[593:0] info: 127.0.0.1 netflix.com. A IN
[1596995098] unbound[593:0] info: 127.0.0.1 netflix.com. DS IN
[1596995098] unbound[593:0] info: 127.0.0.1 www.netflix.com. A IN
[1596995098] unbound[593:0] info: 127.0.0.1 www.netflix.com. A IN
[1596995101] unbound[593:0] info: 127.0.0.1 push.prod.netflix.com. A IN
[1596995101] unbound[593:0] info: 127.0.0.1 ae.nflximg.net. A IN
[1596995101] unbound[593:0] info: 127.0.0.1 push.prod.netflix.com. A IN
[1596995101] unbound[593:0] info: 127.0.0.1 ae.nflximg.net. A IN
[1596995102] unbound[593:0] info: 127.0.0.1 nflximg.net. DS IN

Try toggling the setting on/off Tools->Other Settings Wan: Use local caching DNS server as system resolver (default: No)

This change did nothing on my end.

On the LAN->DHCP Server tab, make sure DNS Server 1 and 2 are empty.

I have DNS 1 Set to my Pi-hole's IP so that the devices on my network would use it for adblocking.
I did try disabling Pi-hole, but still not working.

 

[Xentrk]

I am using an ASUS RT-AC5000, Merlin FW 384.18.



Yes, I can:

Server:         127.0.0.1
Address:        127.0.0.1#53

Non-authoritative answer:
Name:   github.com
Address: 140.82.113.4

I am using Pi-hole with unbound, so my resolver logs are actually generated by unbound. Below is a quick sample of my logs:

[1596995098] unbound[593:0] info: 127.0.0.1 netflix.com. A IN
[1596995098] unbound[593:0] info: 127.0.0.1 netflix.com. A IN
[1596995098] unbound[593:0] info: 127.0.0.1 netflix.com. DS IN
[1596995098] unbound[593:0] info: 127.0.0.1 www.netflix.com. A IN
[1596995098] unbound[593:0] info: 127.0.0.1 www.netflix.com. A IN
[1596995101] unbound[593:0] info: 127.0.0.1 push.prod.netflix.com. A IN
[1596995101] unbound[593:0] info: 127.0.0.1 ae.nflximg.net. A IN
[1596995101] unbound[593:0] info: 127.0.0.1 push.prod.netflix.com. A IN
[1596995101] unbound[593:0] info: 127.0.0.1 ae.nflximg.net. A IN
[1596995102] unbound[593:0] info: 127.0.0.1 nflximg.net. DS IN

This change did nothing on my end.


I have DNS 1 Set to my Pi-hole's IP so that the devices on my network would use it for adblocking.
I did try disabling Pi-hole, but still not working.

If you are using unbound available on asuswrt-merlin, there is an option to continue to use local dnsmasq that I have tested with that works. But I did not specifically look at the dnsmasq method working though. I know that my rules all worked. I'll have to check it out.

But for your set-up, it looks like pi-hole is doing all of the lookups rather than local dnsmasq on the router, which is why there are no entries in the dnsmasq log. When you removed the link to pi-hole, did you substitute DNS1 and DNS2 on the WAN page with a DNS such as Cloudflare or Google? Removing the link to pi-hole should fixed it.

 

[Xentrk]

Hi Xentrk,
thanks a lot for your support. Diversion already added the logging in the dnsmasq.conf . So everything should be ok. But unfortunately my problems are still there. To be concrete

1. I want see a streamingvideo in the german mediathek. So my traffic should be routed via my VPN 1, which is located in germany. The link of the streamingadress begins with http://tvdlzdf-a.akamaidh.net/de/zdf........
So I added the lines
sh /jffs/scripts/x3mRouting/x3mRouting.sh ALL 1 Akamai4 autoscan=akamaihd,akamai
and
sh /jffs/scripts/x3mRouting/x3mRouting.sh ALL 1 Mediatheken dnsmasq=zdf.de,zdfmediathek.de,ard.de,wdr.de,kika.de,phoenix.de,swr.de,swrmediathek.de,br.de,ardmediathek.de,wdrmediathek.de,phoenix.de,akamaihd.net,hr.de,akamaistream.net,dw.de,sr-online.de,ndr.de,rbb-online.de,apa.at,tagesschau.de,heute.de,akamai.com

to the nat-start file.
But it doesn´t work. I got the error message, that the content is geoblocked.

If I am using the App of my VPN Provider und connect diretly with my smartphone to the server. It is working. (by the way: I tested the same VPN-Server - IP Adress)
Listats says:

Mediatheken - 2593
Akamai4 - 1

So, what do I wrong ? How to test, that the link (http://tvdlzdf-a.akamaidh.net/de/zdf....) is routed correctly?

PS:
I tested the logging - it should work, or ?

su@RT-AC86U-7298:/tmp/home/root# ps | grep "dnsmasq --log-async"
12158 su 5568 S grep dnsmasq --log-async
30323 nobody 9660 S dnsmasq --log-async
30325 su 5012 S dnsmasq --log-async

su@RT-AC86U-7298:/tmp/home/root# nvram get vpn_client1_adns
0

2. As I wrote before, I am using the VPNfailover script to switch between VPN1 and VPN2, to stay connected. In the "old" Version I had the VPN1client-down file to delete the routing rules with the del parameter . Do I have to add something new in another file, if I modify the routing rules in the nat-start file?

Let's focus on getting it working for one VPN Client for now.

More analysis will be required. Download the utility scripts in option 4. On the router, configure a client to use the VPN that works with the streaming service and route all traffic to it. Run the getdomainnames.sh script. A separate log file will get created to see what domain are being referenced. Then, go to the site and generate traffic by watching some videos and selecting different options. When done, go back to the SSH session and press ctrl-c to exit out of getdomainnames.sh script. You will be present with a full list of domains accessed during this time. See usage instructions on the readme.

Before doing all that, you may want to try AS43354 based on the nslookup and whob I did on one of the domain names.

# nslookup zdfmediathek.de
Server:    127.0.0.1
Address 1: 127.0.0.1 localhost.localdomain

Name:      zdfmediathek.de
Address 1: 91.197.29.49 shapira-shapira.de

# whob 91.197.29.49
IP: 91.197.29.49
Origin-AS: 43354
Prefix: 91.197.29.0/24
AS-Path: 34224 6453 55002 43354
AS-Org-Name: ZDF-AS
Org-Name: ZDF-Net
Net-Name: ZDF-Net
Cache-Date: 1596958184
Latitude: 49.984190
Longitude: 8.279100
City: Mainz
Region: Rheinland-Pfalz
Country: Germany
Country-Code: DE

 

[obs0lete]

If you are using unbound available on asuswrt-merlin, there is an option to continue to use local dnsmasq that I have tested with that works. But I did not specifically look at the dnsmasq method working though. I know that my rules all worked. I'll have to check it out.

But for your set-up, it looks like pi-hole is doing all of the lookups rather than local dnsmasq on the router, which is why there are no entries in the dnsmasq log. When you removed the link to pi-hole, did you substitute DNS1 and DNS2 on the WAN page with a DNS such as Cloudflare or Google? Removing the link to pi-hole should fixed it.

I tired removing the Pi-hole's IP from the LAN section, rebooted router and then re-entered the commands for NetflixDNS, ASNUM and Amazon Global region...and still nothing. When I do listsstats, NetflixDNS shows as 0.

I remember this working with the 1.9 releases of x3mrouting. Is therea way I can manually find the list of IP's and populate them myself somwehere or somehow have x3mrouting forward the request to the DNS resolver defined in the router?

 

[Xentrk]

I tired removing the Pi-hole's IP from the LAN section, rebooted router and then re-entered the commands for NetflixDNS, ASNUM and Amazon Global region...and still nothing. When I do listsstats, NetflixDNS shows as 0.

I remember this working with the 1.9 releases of x3mrouting. Is therea way I can manually find the list of IP's and populate them myself somwehere or somehow have x3mrouting forward the request to the DNS resolver defined in the router?

I'm not sure why dnsmasq is not logging the query records. Since the LAN DNS1 and DNS2 entries are empty, and you have a valid DNS1 and/or DNS2 in the WAN tab along with dnsmasq logging enabled, it should work now. You may want to try to install Diversion adblocker as it will handle the dnsmasq logging and see if that works. But the dnsmasq log file parms you use you are the same one's used by Diversion.

I have 384.19 beta 1 installed and am using the x3mRouting 384.19 test branch. Here is the versions of ipset and dnsmasq I'm using

# ipset -V
ipset v6.32, protocol version: 6

# dnsmasq -v
Dnsmasq version 2.82-34-gb309cca  Copyright (c) 2000-2020 Simon Kelley
Compile time options: IPv6 GNU-getopt no-RTC no-DBus no-UBus no-i18n no-IDN DHCP DHCPv6 no-Lua TFTP no-conntrack ipset no-auth DNSSEC no-ID loop-detect no-inotify no-dumpfile

This software comes with ABSOLUTELY NO WARRANTY.
Dnsmasq is free software, and you are welcome to redistribute it
under the terms of the GNU General Public License, version 2 or 3.

I uploaded the list NETFLIX-DNS to GitHub I've collected when streaming from a US location. Use the command below to download to /opt/tmp. You may want to change the IPSET name. If so, open up the file in an editor and do a search and replace as well as change the file name.

/usr/sbin/curl --retry 3 "https://raw.githubusercontent.com/Xentrk/Asuswrt-Merlin-Linux-Shell-Scripts/master/NETFLIX-DNS" -o "/opt/tmp/NETFLIX-DNS"

 

[obs0lete]

Thanks for the info! I'll need to try the file later on.

There's one thing I noticed however, I cleared all my existing rulesets and expited the files from the /opt/tmp/ folder.
When I re-ran each command, I noticed that when exeucting the following command:

x3mRouting 1 0 NETFLIX-DNS dnsmasq=netflix.com,nflxext.com,nflximg.com,nflximg.net,nflxso.net,nflxvideo.net,amazonaws.com

The NETFLIX-DNS file was not generated - could this be what is causing the issue? I did make sure that Accept DNS Configuration was disabled first, and then tried to access Netflix on a non-VPN device.

Here is the output of that command:

ntamm@router:/tmp/home/root# x3mRouting 1 0 NETFLIX-DNS dnsmasq=netflix.com,nflxext.com,nflximg.com,nflximg.net,nflxso.net,nflxvideo.net,amazonaws.com
(x3mRouting): 25136 Starting Script Execution 1 0 NETFLIX-DNS dnsmasq=netflix.com,nflxext.com,nflximg.com,nflximg.net,nflxso.net,nflxvideo.net,amazonaws.com

Done.
(x3mRouting): 25136 IPSET created: NETFLIX-DNS hash:net family inet hashsize 1024 maxelem 65536
(x3mRouting): 25136 CRON schedule created: #NETFLIX-DNS# '0 2 * * * ipset save NETFLIX-DNS'
(x3mRouting): 25136 Selective Routing Rule via WAN created for NETFLIX-DNS fwmark 0x8000/0x8000
(x3mRouting): 25136 iptables -t mangle -D PREROUTING -i br0 -m set --match-set NETFLIX-DNS dst -j MARK --set-mark 0x8000/0x8000 2>/dev/null added to /jffs/scripts/x3mRouting/vpnclienp
(x3mRouting): 25136 iptables -t mangle -A PREROUTING -i br0 -m set --match-set NETFLIX-DNS dst -j MARK --set-mark 0x8000/0x8000 added to /jffs/scripts/x3mRouting/vpnclient1-route-up
(x3mRouting): 25136 iptables -t mangle -D PREROUTING -i br0 -m set --match-set NETFLIX-DNS dst -j MARK --set-mark 0x8000/0x8000 2>/dev/null added to /jffs/scripts/x3mRouting/vpnclienn
(x3mRouting): 25136 sh /jffs/scripts/x3mRouting/x3mRouting.sh 1 0 NETFLIX-DNS dnsmasq=netflix.com,nflxext.com,nflximg.com,nflximg.net,nflxso.net,nflxvideo.net,amazonaws.com added to t
(x3mRouting): 25136 Completed Script Execution

Executing the following two commands did correctly generate the files:

x3mRouting 1 0 AMAZON-GLOBAL aws_region=GLOBAL
x3mRouting 1 0 NETFLIX-AS2906 asnum=AS2906

 

[Xentrk]

Thanks for the info! I'll need to try the file later on.

There's one thing I noticed however, I cleared all my existing rulesets and expited the files from the /opt/tmp/ folder.
When I re-ran each command, I noticed that when exeucting the following command:

x3mRouting 1 0 NETFLIX-DNS dnsmasq=netflix.com,nflxext.com,nflximg.com,nflximg.net,nflxso.net,nflxvideo.net,amazonaws.com

The NETFLIX-DNS file was not generated - could this be what is causing the issue? I did make sure that Accept DNS Configuration was disabled first, and then tried to access Netflix on a non-VPN device.

Here is the output of that command:

ntamm@router:/tmp/home/root# x3mRouting 1 0 NETFLIX-DNS dnsmasq=netflix.com,nflxext.com,nflximg.com,nflximg.net,nflxso.net,nflxvideo.net,amazonaws.com
(x3mRouting): 25136 Starting Script Execution 1 0 NETFLIX-DNS dnsmasq=netflix.com,nflxext.com,nflximg.com,nflximg.net,nflxso.net,nflxvideo.net,amazonaws.com

Done.
(x3mRouting): 25136 IPSET created: NETFLIX-DNS hash:net family inet hashsize 1024 maxelem 65536
(x3mRouting): 25136 CRON schedule created: #NETFLIX-DNS# '0 2 * * * ipset save NETFLIX-DNS'
(x3mRouting): 25136 Selective Routing Rule via WAN created for NETFLIX-DNS fwmark 0x8000/0x8000
(x3mRouting): 25136 iptables -t mangle -D PREROUTING -i br0 -m set --match-set NETFLIX-DNS dst -j MARK --set-mark 0x8000/0x8000 2>/dev/null added to /jffs/scripts/x3mRouting/vpnclienp
(x3mRouting): 25136 iptables -t mangle -A PREROUTING -i br0 -m set --match-set NETFLIX-DNS dst -j MARK --set-mark 0x8000/0x8000 added to /jffs/scripts/x3mRouting/vpnclient1-route-up
(x3mRouting): 25136 iptables -t mangle -D PREROUTING -i br0 -m set --match-set NETFLIX-DNS dst -j MARK --set-mark 0x8000/0x8000 2>/dev/null added to /jffs/scripts/x3mRouting/vpnclienn
(x3mRouting): 25136 sh /jffs/scripts/x3mRouting/x3mRouting.sh 1 0 NETFLIX-DNS dnsmasq=netflix.com,nflxext.com,nflximg.com,nflximg.net,nflxso.net,nflxvideo.net,amazonaws.com added to t
(x3mRouting): 25136 Completed Script Execution

Executing the following two commands did correctly generate the files:

x3mRouting 1 0 AMAZON-GLOBAL aws_region=GLOBAL
x3mRouting 1 0 NETFLIX-AS2906 asnum=AS2906

With the other methods, the save/restore file gets created first and the ipset is then loaded from the file. With the dnsmasq method, the IP addresses are collected and loaded into the ipset list dynamically as the domains are queried. The backup file does not get generated until 2 AM. A cron job dumps the contents of the IPSET list to the file in /opt/tmp using the ipset-save command.

 

[Luizlp10]

I'm seeing a "min upd" on x3mRouting but no option 7 to update from. Currently on 384.19 Beta2 and with option 3 installed.

 

[Xentrk]

I'm seeing a "min upd" on x3mRouting but no option 7 to update from. Currently on 384.19 Beta2 and with option 3 installed.

For 384.19 test branch users, please ignore the "min upd" message. amtm isn't designed to work with test branches. amtm compares the version of the master branch on github with the local installed version. amtm considers the master branch to be the latest version available.

 

[Olivier L]

I want to force a bunch of websites (no streaming) to go through vpn client 1.
What is the best method to do so ? dnsmasq=site1.xyz, site2.xyz, site3.xyz ?
Will it be ok if the sites are protected through cloudflare ?

 

[Xentrk]

I want to force a bunch of websites (no streaming) to go through vpn client 1.
What is the best method to do so ? dnsmasq=site1.xyz, site2.xyz, site3.xyz ?
Will it be ok if the sites are protected through cloudflare ?

That is where I would start at. The url may not be enough though. There may be other top level domain names that the site uses. If manageable, you may want to fist test each site one by one to find out if other domains are involved. I sometimes use the dnsmasq method for whatismyipaddress.com when testing the VPN Bypass feature of x3mRouting.

 

[mr8]

That is where I would start at. The url may not be enough though. There may be other top level domain names that the site uses. If manageable, you may want to fist test each site one by one to find out if other domains are involved. I sometimes use the dnsmasq method for whatismyipaddress.com when testing the VPN Bypass feature of x3mRouting.

I'm in a similar situation. I've been running my own spaghetti script to do all the ipsets and vpn routing, but interested in switching over to x3mRouting since it's actively maintained.

Would it be possible to point the dnsmasq parameter directly to a file with a list of domains for easier management? e.g.:

x3mRouting ipset_name=MYSET1 dnsmasq=/path/to/file

 

[Xentrk]

I'm in a similar situation. I've been running my own spaghetti script to do all the ipsets and vpn routing, but interested in switching over to x3mRouting since it's actively maintained.

Would it be possible to point the dnsmasq parameter directly to a file with a list of domains for easier management? e.g.:

x3mRouting ipset_name=MYSET1 dnsmasq=/path/to/file

Yes, I can do that. I proposed this solution here but didn't get any interest. The only difference is the parm 'dnsmasq_file='. Using the 'dnsmasq=' parm should work too.

 

[mister]

Let's focus on getting it working for one VPN Client for now.

More analysis will be required. Download the utility scripts in option 4. On the router, configure a client to use the VPN that works with the streaming service and route all traffic to it. Run the getdomainnames.sh script. A separate log file will get created to see what domain are being referenced. Then, go to the site and generate traffic by watching some videos and selecting different options. When done, go back to the SSH session and press ctrl-c to exit out of getdomainnames.sh script. You will be present with a full list of domains accessed during this time. See usage instructions on the readme.

Before doing all that, you may want to try AS43354 based on the nslookup and whob I did on one of the domain names.

# nslookup zdfmediathek.de
Server:    127.0.0.1
Address 1: 127.0.0.1 localhost.localdomain

Name:      zdfmediathek.de
Address 1: 91.197.29.49 shapira-shapira.de

# whob 91.197.29.49
IP: 91.197.29.49
Origin-AS: 43354
Prefix: 91.197.29.0/24
AS-Path: 34224 6453 55002 43354
AS-Org-Name: ZDF-AS
Org-Name: ZDF-Net
Net-Name: ZDF-Net
Cache-Date: 1596958184
Latitude: 49.984190
Longitude: 8.279100
City: Mainz
Region: Rheinland-Pfalz
Country: Germany
Country-Code: DE

Hi Xentrk,
I already made the addition of the AS number without any success.
Furthermore I tested the getdomainnames.sh and the autoscan.sh scripts without finding any additional domainnames.
Could it be, that the VPN Provider (Cyberghost) has a different setup in their native apps and if you use OpenVPN via router?
As I described I used the IP Adress of the VPNserver I got from IPLEAK.net on my mobile (where it was working) and used it in my router , but I was geoblocked.
So I tested using the DNS server of VPN Provider (normally it is disabled) in Merlin, but even with this setting , I didn´t removed the geoblocking.
So it seems to me, that there is a difference between the using the APP and the router, but I don´t know what it could be.

I used for testing a trial phase of another VPN Providers and interestingly I got different results:

1. At one VPN Provider it was working and I was able to resume the streaming (if paused) without problems
2. At another VPN Provider I was able to start the streaming one time, but if I paused the stream (e.g. for a minute) and wanted to resume, I got the geoblocked error. Restarting streaming from the beginning didn´t work as well. The IP Adress of the VPN Server was still the same ! Very confusing....

For me as a newbee it is not clear , what is running in the backgrounds of the Openvpn script - I can live with the option using the app and streaming then, but more comfortable it would be, if I don´t have to use the app.

Maybe my testing results could be interesting for you and if you have an idea, what I could test, please let me know.

Many thanks again for your replies and your support.

Hugo.

 

[Xentrk]

Hi Xentrk,
I already made the addition of the AS number without any success.
Furthermore I tested the getdomainnames.sh and the autoscan.sh scripts without finding any additional domainnames.
Could it be, that the VPN Provider (Cyberghost) has a different setup in their native apps and if you use OpenVPN via router?
As I described I used the IP Adress of the VPNserver I got from IPLEAK.net on my mobile (where it was working) and used it in my router , but I was geoblocked.
So I tested using the DNS server of VPN Provider (normally it is disabled) in Merlin, but even with this setting , I didn´t removed the geoblocking.
So it seems to me, that there is a difference between the using the APP and the router, but I don´t know what it could be.

I used for testing a trial phase of another VPN Providers and interestingly I got different results:

1. At one VPN Provider it was working and I was able to resume the streaming (if paused) without problems
2. At another VPN Provider I was able to start the streaming one time, but if I paused the stream (e.g. for a minute) and wanted to resume, I got the geoblocked error. Restarting streaming from the beginning didn´t work as well. The IP Adress of the VPN Server was still the same ! Very confusing....

For me as a newbee it is not clear , what is running in the backgrounds of the Openvpn script - I can live with the option using the app and streaming then, but more comfortable it would be, if I don´t have to use the app.

Maybe my testing results could be interesting for you and if you have an idea, what I could test, please let me know.

Many thanks again for your replies and your support.

Hugo.

Some providers use dns proxy to circumvent blocks put in place by streaming media services. NordVPN and Express are two that I am aware of. In those cases, the dnsmasq method won’t work if Accept DNS Configuration is set to Exclusive as dnsmasq is bypassed. My service provides private IP addresses that are not shared.

Finding the domains to route is the most difficult part of selective routing.
I went thru something similar with BBC but eventually got there with a lot of trial and error. I first tried the ASN but that didn’t work. I mined close to 80 domain names. But in the end, I only needed three top level domains I used from the BBC website but had to combine it with the two ASNs that belong to BBC. You may need to do something similar.

Make sure location tracking is turned off on the device you are testing with.

 

[Xentrk]

384.19 Asuswrt-Merlin release is now available. The version of x3mRouting on the master branch is not compatible with 384.19.

My Plan is to move the 384.19 test branch to the master branch when I get home tomorrow afternoon. Before doing so, I will create a 384.18 branch for those who may need to revert back to a prior firmware release.

 

[Xentrk]

x3mRouting Version 2.1.0 Available (Not compatible with 384.18!) 15 August, 2020
x3mRouting Version 2.1.0 has been updated for asuswrt-merlin 384.19 firmware release.

384.19 asuswrt-merlin firmware impacted the following x3mRouting options:

• Option 1 - LAN Client Routing
• Option 2 - x3mRouting + Modified OpenVPN Client Screen

Additional features added include:

• Reinstatement of routing rules after a firewall restart for those who route IPSET lists via the modified OpenVPN Client Screen.
• Implemented Accept DNS Configuration = Exclusive rules if selected on the OpenVPN Client Screen for those using the LAN Client Routing option and reinstatement of rules after a firewall restart.

Please use the following instructions based on your current installation.

x3mRouting v2.0.0 + 384.18 firmware currently installed
Since x3mRouting v2.1.0 is not compatible with 384.18 firmware, only update to V 2.1.0 if you intend to immediately update to 384.19 immediately afterwards.

• Download the 384.19 firmware and extract to local directory on your computer.
• Take a backup of the jffs partition and system config in case you need to revert back.
• Type x3mMenu from the command line or access via amtm. Select option [7] Update x3mRouting Menu
• Run [5] Check for updates to existing x3mRouting installation option to perform cleanup and finalize the update.
• Unmount the USB
• Upload the 384.19 firmware

x3mRouting 384.19 test branch is installed
You will need to issue the command below to download the new menu and point your local installation back to the master x3mRouting branch on GitHub.

sh -c "$(curl -sL https://raw.githubusercontent.com/Xentrk/x3mRouting/master/Install_x3mRouting.sh)"

• Run [5] Check for updates to existing x3mRouting installation option to perform cleanup and finalize the update.

x3mRouting v2.0.0 + 384.19 firmware installed

• Type x3mMenu from the command line or access via amtm. Select option [7] Update x3mRouting Menu
• Run [5] Check for updates to existing x3mRouting installation option to perform cleanup and finalize the update.

Reverting to 384.18
If you use options 1 and 2 and need to revert to 384.18, please visit https://github.com/Xentrk/x3mRouting/tree/x3mRouting-384.18 and copy/paste the following

sh -c "$(curl -sL https://raw.githubusercontent.com/Xentrk/x3mRouting/x3mRouting-384.18/Install_x3mRouting.sh)"

to download and install version 2.0.0. You can't revert using amtm.

 

[Kingp1n]

Anyone have HBOMax working? I have apply these rules with no luck:
x3mRouting 1 0 AMAZON asnum=AS16509
x3mRouting 1 0 AMAZON asnum=AS14618
x3mRouting 1 0 HBOMAX dnsmasq=hbomax.com,play.hbomax.com

I tried these rules below and nothing things to work. It was working fine with the 384.19 beta2 fw.

service restart_firewall
service restart_dnsmasq
service restart_vpnclient1