Selective Routing with Asuswrt-Merlin

[Xentrk]

I'm using the AC68U with Firmware 384.3 dnsmasq and jffs are both enabled. I have 3 active vpn clients running usually. Sure, sent me the script and I'll give it a go. Detailed installation instructions are always good too. Thanks for your help!

Okay. I will send you a PM with the code. Some AC68U users have reported issues with selective routing in this thread. Hopefully, it will work for you.

 

[mr8]

Hello, thought I'd share my domain-based script:

#!/bin/sh

#Domains to route through VPN
cleint1_domains="domain1.com domain2.xyz"

#Persistent VPN IP rules
#e.g. cleint1_clientlist="<rule.name>0.0.0.0>127.0.0.1>VPN" 
cleint1_clientlist=""

#Persistent DNSmasq rules
#e.g. dnsmasq="address=/test.com/127.0.0.1" 
dnsmasq=""

#DNS server for lookup
dns_server=8.8.8.8

for domain in $cleint1_domains
do
    dnsmasq=$dnsmasq"server=/$domain/$dns_server\n"
    ip=`nslookup $domain $dns_server|tail +4|grep 'Address'|awk -F ":" '{print $2}'|awk '{print $1;}'`
    count=1
    for i in $ip
    do
        if expr "$i" : '[0-9][0-9]*\.[0-9][0-9]*\.[0-9][0-9]*\.[0-9][0-9]*$' >/dev/null; then
            cleint1_clientlist=$cleint1_clientlist'<'${domain:0:12}$count'>0.0.0.0>'$i'>VPN'
            let "count++"
        fi
    done
done

echo -e "$dnsmasq" > /jffs/configs/dnsmasq.conf.add
echo "restarting dnsmasq"
service restart_dnsmasq

nvram unset vpn_client_clientlist
nvram unset vpn_client1_clientlist
nvram set vpn_client1_clientlist=$cleint1_clientlist
nvram commit
echo "restarting VPN 1"
service restart_vpnclient1

It's currently set to route specific domains over the VPN, but can easily modified to do the reverse.

 

[Martineau]

Hello, thought I'd share my domain-based script:

#!/bin/sh

#Domains to route through VPN
cleint1_domains="domain1.com domain2.xyz"

#Persistent VPN IP rules
#e.g. cleint1_clientlist="<rule.name>0.0.0.0>127.0.0.1>VPN"
cleint1_clientlist=""

#Persistent DNSmasq rules
#e.g. dnsmasq="address=/test.com/127.0.0.1"
dnsmasq=""

#DNS server for lookup
dns_server=8.8.8.8

for domain in $cleint1_domains
do
    dnsmasq=$dnsmasq"server=/$domain/$dns_server\n"
    ip=`nslookup $domain $dns_server|tail +4|grep 'Address'|awk -F ":" '{print $2}'|awk '{print $1;}'`
    count=1
    for i in $ip
    do
        if expr "$i" : '[0-9][0-9]*\.[0-9][0-9]*\.[0-9][0-9]*\.[0-9][0-9]*$' >/dev/null; then
            cleint1_clientlist=$cleint1_clientlist'<'${domain:0:12}$count'>0.0.0.0>'$i'>VPN'
            let "count++"
        fi
    done
done

echo -e "$dnsmasq" > /jffs/configs/dnsmasq.conf.add
echo "restarting dnsmasq"
service restart_dnsmasq

nvram unset vpn_client_clientlist
nvram unset vpn_client1_clientlist
nvram set vpn_client1_clientlist=$cleint1_clientlist
nvram commit
echo "restarting VPN 1"
service restart_vpnclient1

It's currently set to route specific domains over the VPN, but can easily modified to do the reverse.

Hmm the following code statement

echo -e "$dnsmasq" > /jffs/configs/dnsmasq.conf.add

will totally trash my existing 200 line '/jffs/configs/dnsmasq.conf.add'
So rather than be inundated by irate users, I highly recommend you immediately change this dangerous statement to

echo -e "$dnsmasq" >> /jffs/configs/dnsmasq.conf.add

or preferably check the new statements do not already exist before inserting the dnsmasq directives.
NOTE: Not sure of the benefit of using the geo-aware Google DNS servers...why not simply use the routers DNS configuration to resolve the IP/Subnets?

P.S. You should really merge the 'domain' entries (eliminating duplicates) with the existing NVRAM variable that may already contain Selective 'WAN' entries and LAN/Guest WiFiVPN entries etc., and for the v384.xx firmware be aware there are NVRAM length limits imposed that may cause unexpected truncation/corruption etc.

For domains, it is now recommended to use IPSETs as sadly your script does not comprehensively identify all IP subnets/address for a given domain e.g. Netflix.com (see @Xentrk's method)

 

[mr8]

Hmm the following code statement

echo -e "$dnsmasq" > /jffs/configs/dnsmasq.conf.add

will totally trash my existing 200 line '/jffs/configs/dnsmasq.conf.add'
So rather than be inundated by irate users, I highly recommend you immediately change this dangerous statement to

echo -e "$dnsmasq" >> /jffs/configs/dnsmasq.conf.add

or preferably check the new statements do not already exist before inserting the dnsmasq directives.

Thanks for the feedback. My actual script is a little more complicated for easier editing via my browser-based gui, but I simplified it to share on here.

I did consider using `>>` but opted for the `>` for easier editing via my gui. I did include the option for persistent dnsmasq rules at the top of the script `$dnsmasq` that should survive any changes.

NOTE: Not sure of the benefit of using the geo-aware Google DNS servers...why not simply use the routers DNS configuration to resolve the IP/Subnets?

Again, this is configureable, and I did originally use the router's DNS server but noticed that some of the CDNs were geo blocked even when routed through the VPN. Using google's DNS server seemed to bypass those geo-blocked CDNs while allowing other traffic to route through closer CDNs.

P.S. You should really merge the 'domain' entries (eliminating duplicates) with the existing NVRAM variable that may already contain Selective 'WAN' entries and LAN/Guest WiFiVPN entries etc., and for the v384.xx firmware be aware there are NVRAM length limits imposed that may cause unexpected truncation/corruption etc.
For domains, it is now recommended to use IPSETs as sadly your script does not comprehensively identify all IP subnets/address for a given domain e.g. Netflix.com (see @Xentrk's method)
#Persistent DNSmasq rules

This is something I would like to look into. I was thinking of moving the the DNS lookups to a separate device and passing a simple json or txt output back into my script.

I may also look into IPSETs, but I'm only routing a handful of subdomains through two VPNs different right now and iptables sacare me!
For now, it actually works pretty and beats looking up IPs. I was previously manually looking up CIDRs and had to constantly update my rules to keep up with changing IPs and non-geo content getting routed through my VPN servers.

 

[mr8]

For domains, it is now recommended to use IPSETs as sadly your script does not comprehensively identify all IP subnets/address for a given domain e.g. Netflix.com (see @Xentrk's method)

So I think I want to switch over to implementing IPSET into my script. Any suggestions or examples to help get me started would be greatly appreciate.

 

[Martineau]

So I think I want to switch over to implementing IPSET into my script. Any suggestions or examples to help get me started would be greatly appreciate.

An ongoing collaboration.... Selective Routing for Netflix

So actually iptables is not scary at all...simply 1 rule per IPSET as per @Xentrk's implementation (makes it easier for visual debugging).

I have posted how to combine any number of Selective Routing IPSETs ipset for dynamic routing with openvpn client under a single static iptables rule, and also how to use dnsmasq to automatically keep the contents of the Selective Routing (domain) IPSETs 'current' although they can retain obsolete entries, so a manual flushing of the IPSETs is recommended from time to time.

 

[mr8]

An ongoing collaboration.... Selective Routing for Netflix

So actually iptables is not scary at all...simply 1 rule per IPSET as per @Xentrk's implementation (makes it easier for visual debugging).

I have posted how to combine any number of Selective Routing IPSETs ipset for dynamic routing with openvpn client under a single static iptables rule, and also how to use dnsmasq to automatically keep the contents of the Selective Routing (domain) IPSETs 'current' although they can retain obsolete entries, so a manual flushing of the IPSETs is recommended from time to time.

Thanks again for all the info!

I'm still learning about IPSET, and have a few questions.
So basically I create an ipset with `ipset create vpn1 hash:ip` and then add my domains to my dnsmasq with `ipset=/domain.com/other.com/vpn1` and then add the ipset to my iptables with something like this
`iptables -t mangle -A PREROUTING -i tun11 -p tcp -m set --match-set vpn1 dst,dst -j MARK --set-mark 0x7000/0x7000`

Did I miss anything? Is really that simple? Also, do I need to restart anything other than dnsmasq when adding additional domains?

Thanks!

 

[Martineau]

So basically I create an ipset with

ipset create vpn1 hash:ip

Correct

and then add my domains to my dnsmasq with

ipset=/domain.com/other.com/vpn1

Correct

then add the ipset to my iptables with something like this

iptables -t mangle -A PREROUTING -i tun11 -p tcp -m set --match-set vpn1 dst,dst -j MARK --set-mark 0x7000/0x7000

Incorrect should be (for traffic outbound via VPN Client 1)

iptables -t mangle -A PREROUTING -i br0 -p tcp -m set --match-set vpn1 dst,dst -j MARK --set-mark 0x1000/0x1000

Did I miss anything?

Yes, you also need the RPDB rules

ip rule add from 0/0 fwmark 0x7000/0x7000 table main prio 9990
ip rule add from 0/0 fwmark 0x1000/0x1000 table 111  prio 9991
etc.
ip route flush cache

So now the matching IPSET destination hosts are tagged with the appropriate fwmark (0x7000 is WAN, 0x1000 is VPN1, 0x2000 is VPN2 etc.) and the RPDB rules then pass the tagged packet outbound to the appropriate interface WAN,VPN Client 1, VPN Client2 etc.

Is really that simple?

Yes.

Do I need to restart anything other than dnsmasq when adding additional domains?

No.

However it may be prudent to create more-human friendly memorable descriptive names for your IPSETs, then group them under say the WAN0,VPN1,VPN2 etc. IPSETs
(Much easier to debug)

EDIT: Also IPSETs are not persistent, so you should schedule an IPSET backup to USB storage, then you can quickly restore a populated IPSET, rather than wait until dnsmasq slowly (re)populates the empty IPSET(s)

 

[mr8]

Thanks again @Martineau
Got it working! Was really easy and much simpler!

Thanks for the tip, I’ll script each domain into its own IPSET for easier debugging and cleanup.

Btw, is there any reason to backup IPSETs since the IPs change? Won’t they automatically get re-added to the IPSET (via dnsmasq) each time? I don’t want to run into the same problem I had previously where I had a bunch of IPs that were no longer used by certain domains. Perhaps a cron job could clear the ipset every night and repopulate IPSET with another cron nslookups every hour..?

 

[Martineau]

Thanks again @Martineau
Got it working! Was really easy and much simpler!

Indeed but it's a pity the RPDB rules and static fwmark tagging rules are still not included in the firmware but you can always tweak '/usr/sbin/vpnrouting.sh' if you feel brave!

Thanks for the tip, I’ll script each domain into its own IPSET for easier debugging and cleanup.

Btw, is there any reason to backup IPSETs since the IPs change? Won’t they automatically get re-added to the IPSET (via dnsmasq) each time? I don’t want to run into the same problem I had previously where I had a bunch of IPs that were no longer used by certain domains. Perhaps a cron job could clear the ipset every night and repopulate IPSET with another cron nslookups every hour..?

Having stale entries in the IPSETs shouldn't be a problem when there are only a few IP subnets/addresses to be maintained and they are relatively static.

In theory, using dnsmasq to populate the IPSET with the target Selective Routing domain should always work even if the IPSET is actually empty at the time of the first request.

NOTE: I did find that it was useful to have a .txt file of the IPSET collated IP subnets/address just in case for debugging etc.

So it really is up to you to decide how frequently you perform the necessary housekeeping to remove obsolete entries from the IPSETs but I suspect once every hour would be a bit excessive!

 

[mr8]

Hello again @Martineau
I've been looking into your WiFiVPN script, and had a quick question.
I was looking into my ip route tables and noticed that I already have two tables that seem to correspond to my ovpn clients: ovpnc1 and ovpnc2.

Are these two tables standard on Asus merlin, and would it be safe to use them for my selective routing script? Or should I create another table specifically for the selective routing..?

 

[Martineau]

I was looking into my ip route tables and noticed that I already have two tables that seem to correspond to my ovpn clients: ovpnc1 and ovpnc2.

Are these two tables standard on Asus merlin, and would it be safe to use them for my selective routing script? Or should I create another table specifically for the selective routing..?

The default Selective Routing/Dual-WAN interfaces ovpnc1,ovpnc2 etc. are defined in

/etc/iproute2/rt_tables

and the entries are used by '/usr/sbin/vpnrouting.sh' to create the necessary Selective Routing environment (apart from fwmark based Selective Routing for ports,MACs,IPSETs etc.)

I prefer to customise the entries so I have renamed 'ovpnc1' to 'NewYork' and 'ovpnc5' to 'UK' etc. to make it visually more human friendly when tinkering with the configs, and to reduce the possibility of typo errors when I start to tinker with Selective Routing.
i.e. it is far less likely I would confuse NewYorK with UK whereas ovpnc1/ovpnc2 etc. could easily be incorrectly used by mistake because of a stupid 1 char typo etc.

ip rule

0:  from all lookup local
10000:  from all fwmark 0x7000/0x7000 lookup main
10100:  from all fwmark 0x1000/0x1000 lookup NewYork
10101:  from 172.16.1.1 lookup NewYork
10901:  from 172.16.5.5 lookup UK
20100:  from 10.88.101.0/24 lookup NewYork
32766:  from all lookup main
32767:  from all lookup default

ip route show table NewYork

10.16.0.0/24 dev tun22  scope link
10.88.101.0/24 dev br1  scope link
10.8.0.0/24 dev tun21  scope link
10.88.8.0/24 dev br0  proto kernel  scope link  src 10.88.8.1
100.120.242.0/23 dev tun11  proto kernel  scope link  src 100.120.243.15
default via 100.120.242.1 dev tun11

rather than

ip rule

0:  from all lookup local
10000:  from all fwmark 0x7000/0x7000 lookup main
10100:  from all fwmark 0x1000/0x1000 lookup ovpnc1
10101:  from 172.16.1.1 lookup ovpnc1
10901:  from 172.16.5.5 lookup ovpnc5
20100:  from 10.88.101.0/24 lookup ovpnc1
32766:  from all lookup main
32767:  from all lookup default

ip route show table ovpnc1

10.16.0.0/24 dev tun22  scope link
10.88.101.0/24 dev br1  scope link
10.8.0.0/24 dev tun21  scope link
10.88.8.0/24 dev br0  proto kernel  scope link  src 10.88.8.1
100.120.242.0/23 dev tun11  proto kernel  scope link  src 100.120.243.15
default via 100.120.242.1 dev tun11

However, unless you have a very advanced need, then I would recommend you stick with the default ovpncX tables i.e. why (re)write scripts if all the work has already been done for you? - especially if you want quicker support from the forum.

 

[mr8]

...
However, unless you have a very advanced need, then I would recommend you stick with the default ovpncX tables i.e. why (re)write scripts if all the work has already been done for you? - especially if you want quicker support from the forum.

I get what you mean with human friendly names - I use airport codes to ID my servers (e.g. SFO, DEN, LAX), but I'll keep it simple and stick with the default ovpncX.

Not sure if this deserves its own topic, but would it be possible to route one of my openvpn servers to use the same ipset selective routing that's already set up on my router?

 

[Martineau]

Not sure if this deserves its own topic, but would it be possible to route one of my openvpn servers to use the same ipset selective routing that's already set up on my router?

Not 100% sure what you mean...do you mean a 'pass-thru' scenario where a remote client connects inbound to either of the OpenVPN servers running on your router and is Selectively Routed outbound via the VPN Client?

 

[mr8]

Not 100% sure what you mean...do you mean a 'pass-thru' scenario where a remote client connects inbound to either of the OpenVPN servers running on your router and is Selectively Routed outbound via the VPN Client?

Yes, I currently run one OpenVPN server on the router to use while I’m on public hotspots and would like to keep that one untouched, but I would also like to take advantage of the selective routing on the second OpenVPN server. If that makes sense.
The remote clients on the second server don’t need to be on the same subnet as the local clients, I just want the remote clients on the second server to have the same browsing experience as the local clients with the selective routing.

Could I just add another iptables entry for ipset traffic on tun22..?

 

[mr8]

Hi @Martineau
I've been trying to get selective routing working on my tun22 interface, but can't get it to work.

Any suggestions or tips?

iptables -t mangle -A PREROUTING -i tun22 -p tcp -m set --match-set sfovpn dst,dst -j MARK --set-mark 0x1100/0x1100

Thanks!

 

[Martineau]

Hi @Martineau
I've been trying to get selective routing working on my tun22 interface, but can't get it to work.

Any suggestions or tips?

iptables -t mangle -A PREROUTING -i tun22 -p tcp -m set --match-set sfovpn dst,dst -j MARK --set-mark 0x1100/0x1100

Thanks!

Typo?

iptables -t mangle -A PREROUTING -i tun22 -p tcp -m set --match-set sfovpn src,dst -j MARK --set-xmark 0x1000/0x1000

assuming you have the appropriate MASQUERADE rule for the OpenVPN Server IP pool.

 

[DanGaron]

For anyone that doesn't realize it segmenting the VPN and WAN and forcing certain IPs through VPN while excluding all the others is now built in as part of the GUI... I've been racking my brain for the past two days trying to figure out the proper script to run for 2 of my devices to be forced on VPN while not having any of the other devices use it.

I'm sure this is noted somewhere, but with 39 pages of thread it took a long time to realize it because of the wording in the Merlin GUI has. If you need to force specific IPs through VPN go to VPN > VPN Client and after you have your openvpn config installed use the Redirect Internet Traffic section to enable policy rules and from there can you force specific IPs to use the VPN.

This Youtube video finally made me realize that is what the setting was for.

 

[madfusker]

Yeah, this is a great feature for any VPN user that uses video streaming services and it's been working well the last couple years for me. So glad it's part of the merlin firmware!

 

[Xentrk]

For anyone that doesn't realize it segmenting the VPN and WAN and forcing certain IPs through VPN while excluding all the others is now built in as part of the GUI... I've been racking my brain for the past two days trying to figure out the proper script to run for 2 of my devices to be forced on VPN while not having any of the other devices use it.

I'm sure this is noted somewhere, but with 39 pages of thread it took a long time to realize it because of the wording in the Merlin GUI has. If you need to force specific IPs through VPN go to VPN > VPN Client and after you have your openvpn config installed use the Redirect Internet Traffic section to enable policy rules and from there can you force specific IPs to use the VPN.

This Youtube video finally made me realize that is what the setting was for.

Selective Routing is my main interest in networking. I recall reading this thread from beginning to end awhile back trying to understand how it all worked. It was interesting to see how the technique evolved over time. I could not have done it without the great support from @Martineau. I recently launched a blog site and plan to write up some posts on how to implement Selective Routing in AsusWRT-Merlin and pfSense. I wrote some scripts that mimic the Web GUI functionality that I plan to post on GitHub as well. Always a great moment when the light bulb goes off and one learns how to accomplish their goal and things work like you want them too. This forum has been a great resource in my learning process. I am very GRATEFUL.

Here is a OpenVPN Setup Guide I wrote for Asuswrt-Merlin using TorGuard that explains some of the configuration options. I will starting working on the updates to reflect the changes in the 384.5 release.