route certain domains through client vpn

[waeking]

Is there a way to route certain domains through a client vpn?

I have been using Asus Merlin's build for a few months now. I moved from TomatoUSB to here. There is no going back. The only thing missing is to route domain names. With the use of cloudflare and others you can't rely and ip addresses any more for routing certain places through a vpn anymore.

On Tomato this was handled by the gui. I am ok with scripts and such. Just haven't been able to find a thread started on here.

TIA

 

[waeking]

I've been doing so research and usually don't like reinventing the "wheel". I have found this link.
https://github.com/jamesmacwhite/ip...rt/wiki/Using-ipset-with-dnsmasq-and-iptables

any thoughts?

 

[waeking]

I have the ipset working well....

added /jffs/configs/dnsmasq.conf.add

ipset=/whatismyip.org/VPNTUN12_V4

but when I use

iptables -I PREROUTING -t mangle -m set --match-set VPNTUN12_V4 dst -j MARK --set-mark 10012
iptables v1.4.15: Set VPNTUN12_V4 doesn't exist.

 

[Martineau]

I have the ipset working well....

added /jffs/configs/dnsmasq.conf.add

ipset=/whatismyip.org/VPNTUN12_V4

but when I use

iptables -I PREROUTING -t mangle -m set --match-set VPNTUN12_V4 dst -j MARK --set-mark 10012
iptables v1.4.15: Set VPNTUN12_V4 doesn't exist.

You need to manually create the IPSET before dnsmasq can auto-populate it

ipset create VPNTUN12_V4 hash:net family inet hashsize 1024 maxelem 65536

then get dnsmasq to populate it
e.g.

cat /etc/dnsmasq.conf | grep ipset

nslookup whatismyip.org

ipset list VPNTUN12_V4

 

[waeking]

Thanks so much I got it working. I wish I knew more about scripting to automate some of it.

I added this to dnsmasq.conf.add, it will allow *.conf to be added to the dnsmasq.conf

conf-dir=/tmp/mnt/Merlin/dnsmasq.d/,*.conf

I use this config for address.conf, server.conf, and interface.conf....
then I created a file ipset.conf

ipset=/privateinternetaccess.com/whatismyip.org/VPNTUN12_V4

I would love to create a script that will do something like this. Sadly I don't know this part. Then you could call this from the up script in openvpn client custom configuration.

#!/bin/sh
for LINE in `cat /tmp/mnt/Merlin/dnsmasq.d/ipset.conf`
do
  nslookup $LINE |grep|awk|sed...
done

Would be nice if I could find a tutorial and an online editor to test with. But I am not sure what to even search for.

 

[waeking]

perhaps and easier way is to create a txt file.

ie: ipset_tun12.txt

privateinternetaccess.com
whatismyip.org

then run an openvpn up script that does a few things.

#!/bin/sh
ipset create $dev"_V4" hash:net family inet hashsize 1024 maxelem 65536
ipset create $dev"_V6" hash:net family inet hashsize 1024 maxelem 65536

#create ipset.conf to look like
#ipset=/privateinternetaccess.com/whatismyip.org/tun12_V4,tun12_V6
touch "ipset_"$dev.conf
echo "ipset=/"> "ipset_"$dev.conf
for DOMAIN in `cat ipset_$dev.txt`
do
   echo -n $DOMAIN  >> "ipset_"$dev.conf
   echo -n "/" >> "ipset_"$dev.conf
done
echo -n $dev"_V4",$dev"_V6" >> "ipset_"$dev.conf

#restart dnsmasq
service restart_dnsmasq

# Do nslookup for all lines in ipset_tun12.txt
for LINE in `cat "ipset_"$dev.txt`
do
      nslookup $LINE
done

iptables -I PREROUTING -t mangle -m set --match-set $dev"_V4" dst -j MARK --set-mark 10012
ip6tables -I PREROUTING -t mangle -m set --match-set $dev"_V6" dst -j MARK --set-mark 10012
ip rule add prio 100 fwmark 10012 lookup 100
ip route add table 100 default dev $dev

 

[Martineau]

perhaps and easier way

Even easier..use something that is proven to work.

I strongly suggest you review this. Whilst it would be convenient if the script was tweaked (by @Xentrk ???!!! ) to allow a custom IPSET to be used (rather than specifically written for Netflix), with a few simple changes, the script can be modified to use your custom Domains.

then run an openvpn up script that does a few things.

Why use the openvpn-event UP trigger to needlessly bounce dnsmasq and perform the nslookups every time the VPN starts?

P.S. The use of 'nslookup' is not necessary, it is simply a way of manually testing if your 'ipset=' directive is correctly telling dnsmasq to automatically resolve the domains, and is actually adding their current IP to the IPSET.

 

[waeking]

well that changes a few things..... That script looks awsome! would require a lot more what than what I am hoping for. However knowing that nslookup is not needed is a great help, thanks!

so I just needed the following
wan-start:

ipset create VPNtun11_V4 hash:net family inet hashsize 1024 maxelem 65536
ipset create VPNtun11_V6 hash:net family inet hashsize 1024 maxelem 65536
ipset create VPNtun12_V4 hash:net family inet hashsize 1024 maxelem 65536
ipset create VPNtun12_V6 hash:net family inet hashsize 1024 maxelem 65536

dnsmasq.conf.add:

ipset=/WEBURLS/whatismyip.com/VPNtun11_V4,VPNtun11_V6
ipset=/WEBURLS/whatismyip.org/VPNtun12_V4,VPNtun12_V6

Make sure to add security-script 2 to the custom config for the openvpn client.
up.sh:

#!/bin/sh
iptables -I PREROUTING -t mangle -m set --match-set "VPN"$dev"_V4" dst -j MARK --set-mark ${dev/tun/100}
ip6tables -I PREROUTING -t mangle -m set --match-set "VPN"$dev"_V6" dst -j MARK --set-mark ${dev/tun/100}
ip rule add prio ${dev/tun/99} fwmark ${dev/tun/100} lookup ${dev/tun/100}
ip route add table ${dev/tun/100} default dev $dev

down.sh:

#!/bin/sh
iptables -D PREROUTING -t mangle -m set --match-set "VPN"$dev"_V4" dst -j MARK --set-mark ${dev/tun/100}
ip6tables -D PREROUTING -t mangle -m set --match-set "VPN"$dev"_V6" dst -j MARK --set-mark ${dev/tun/100}
ip rule del prio ${dev/tun/99} fwmark ${dev/tun/100} lookup ${dev/tun/100}
ip route del table ${dev/tun/100} default dev $dev

EDIT: CHANGED to work for all vpnclients

 

[waeking]

Just a few more questions. Will the ipset list repopulate as the ip changes for a domain? if so could we add:

ipset save "VPN"$dev"_V4" > "VPN"$dev"_V4".list
ipset restore "VPN"$dev"_V4" < "VPN"$dev"_V4".list

Is this the proper restore technique? Would this speedup anything?

 

[Martineau]

I strongly suggest you review this. Whilst it would be convenient if the script was tweaked (by @Xentrk ???!!! ) to allow a custom IPSET to be used (rather than specifically written for Netflix), with a few simple changes, the script can be modified to use your custom Domains.

@Xentrk ,

I know you are currently extremely busy with your Stubby project but I have taken the liberty to hack your script here

Apologies if this causes offense, if so then I will take it down immediately.

Basically the modified script should make it easy for novice users to implement the dnsmasq harvesting method to collate IPs from domains and route them explicitly either via the WAN or a VPN Client. (NOTE: IPv6 has not been added)

Spoiler: Script Usgae help

#
# Usage:     IPSET_Domains.sh   {[0|1|2|3|4|5]  ipset_name  domains[,...]}  [del]
#                        
# Usage:     IPSET_Domains.sh   2   BBC   bbc.co.uk
#                               Create IPSET BBC via VPN Client 2 and autopopulate IPs for domain 'bbc.co.uk'
# Usage:     IPSET_Domains.sh   2   BBC   bbc.co.uk   del
#                               Delete IPSET BBC and remove from VPN Client 2
# Usage:     IPSET_Domains.sh   0   MyIPSET   privateinternetaccess.com,whatismyip.org
#                               Create ipset MyIPSET via WAN and autopopulate IPs for two domains 'privateinternetaccess.com' & 'whatismyip.org'

e.g. To implement automated IPSET tracking of the IPs associated with domain 'bbc.co.uk' into IPSET 'BBC' and route them via VPN Client 2

./IPSET_Domains.sh   2   BBC   bbc.co.uk

and to delete the IPSET, and remove the selective routing via VPN Client 2 etc.

IPSET_Domains.sh   2   BBC   bbc.co.uk   del

I hope it still replicates your original usage...​

./IPSET_Domains.sh  0   x3mRouting_NETFLIX_DNSMASQ   amazonaws.com,netflix.com,nflxext.com,nflximg.net,nflxso.net,nflxvideo.net

(IPSET_Domains.sh): 14500 Starting Script Execution
(IPSET_Domains.sh): 14500 IPSET created: x3mRouting_NETFLIX_DNSMASQ hash:net family inet hashsize 1024 maxelem 65536
(IPSET_Domains.sh): 14500 CRON schedule created: #x3mRouting_NETFLIX_DNSMASQ# '0 2 * * * ipset save x3mRouting_NETFLIX_DNSMASQ'
(IPSET_Domains.sh): 14500 Selective Routing Rule via WAN created (TAG fwmark 0x8000/0x8000)
(IPSET_Domains.sh): 14500 Completed Script Execution

cat /etc/dnsmasq.conf | grep x3m

ipset=/amazonaws.com/netflix.com/nflxext.com/nflximg.net/nflxso.net/nflxvideo.net/x3mRouting_NETFLIX_DNSMASQ

etc.

​

 

[Martineau]

That script looks awsome! would require a lot more what than what I am hoping for.

Apart from IPv6 support the script does precisely everything you need (including ipset save/restore)....correctly.

NOTE: I've never used IPv6 so not even sure if you can selectively route IPv6 via the current VPN Client implementation in the @RMerlin firmware?

Clearly there are technical differences between the base DD-WRT and the advanced @RMerlin firmware etc., presumably that is why you suddenly abandoned your home-grown idea of using the dnsmasq 'conf-dir' directive etc.?

So comments/questions such as 'Make sure to add security-script 2 to the custom config for the openvpn client' and 'Are there different routing tables per VPN client' are redundant, since @RMerlin's firmware already includes the necessary options for you, and unless you absolutely know what you are doing, I would stay away from modifying the 'up.sh/down.sh' scripts but instead read RMerlin's Wiki and learn how to use user/post-conf scripts correctly e.g. openvpn-event triggers such as route-up/route-down

What was your quote again about 'reinventing the wheel'?

P.S. If you re-read the R7000 DD-WRT IPSET usage you will hopefully spot why dnsmasq will currently be unable to automatically collate the IPv6 addresses for the domains..

 

[Xentrk]

@Xentrk ,

I know you are currently extremely busy with your Stubby project but I have taken the liberty to hack your script here

Apologies if this causes offense, if so then I will take it down immediately.

Actually, I am very Grateful for the feedback. Much of my knowledge on the Selective Routing topic can be directly attributed to your help and support. The whole reason for placing it on GitHub was so others can collaborate on the project and make it even better.

You are correct in that the Stubby project has consumed more of my time than I anticipated. The team of testers and I are still discussing how DNSSEC is working on Stubby and we appear to have had a breakthrough in the last 12 hours.

I have time tomorrow to analyze the changes in more depth.

Also, I had to make some changes to the Chk_Entware function for the Stubby project that I wanted to let you know about. I'll send you a PM on that topic soon.

 

[Martineau]

Also, I had to make some changes to the Chk_Entware function for the Stubby project that I wanted to let you know about.

Oh dear yet another one of my embarrassingly shoddy scripts?

P.S. I have just examined your Stubby install script, and I notice that you appear to be using my Chk_Entware function differently from your other scripts published back in June?

Chk_Entware 'jq' || { echo -e "\a***ERROR*** Entware" $ENTWARE_UTILITY "not available";exit 99; }

e.g. install_stubby.sh

###################### Main ################

Set_Color_Parms

Chk_Entware
    if [ "$READY" -eq "0" ]; then
        opkg update && printf "entware successfully updated\n" || printf "An error occurred updating entware\n" || exit 1
    else
        printf "You must first install Entware before proceeding\n"
        printf "Exiting %s\n" "$(basename "$0")"
        exit 1
    fi

Chk_Entware stubby
    if [ "$READY" -eq "0" ]; then
        printf "existing stubby package found\n"
        opkg update stubby && printf "stubby successfully updated\n" || printf "An error occurred updating stubby\n" || exit 1
    else
        opkg install stubby && printf "stubby successfully installed\n" || printf "An error occurred installing stubby\n" || exit 1
    fi

My function returns a value to global variable '$?'...i.e. it does not pass the local variable '$READY'....

Consequently your script produces the following error:

 opkg list-installed stubby
+ [ ! -z stubby - 0.2.3-1 ]
+ READY=0
+ break
+ return 0
+ [  -eq 0 ]
[: bad number

I suggest you amend your code as follows:

###################### Main ################

Set_Color_Parms

Chk_Entware
    if [ $? -eq 0 ]; then
        opkg update && printf "entware successfully updated\n" || printf "An error occurred updating entware\n" || exit 1
    else
        printf "You must first install Entware before proceeding\n"
        printf "Exiting %s\n" "$(basename "$0")"
        exit 1
    fi

Chk_Entware stubby
    if [ $? -eq 0 ]; then
        printf "existing stubby package found\n"
        opkg update stubby && printf "stubby successfully updated\n" || printf "An error occurred updating stubby\n" || exit 1
    else
        opkg install stubby && printf "stubby successfully installed\n" || printf "An error occurred installing stubby\n" || exit 1
    fi

Also, you appear to have included the function twice in the script resulting in script bloat?

Apologies to @waeking for off-topic post.

 

[waeking]

@Martineau I really appreciate your help! You have answered all of my questions as they come. You have taken a script that I barely understand and changed it to fit my needs. Thankyou! Does this script only need to be ran one time to add or delete ipsets? Will the final outcome survive reboots? Does the script need entware to run the commands within. I realize that it checks for entware. It would be nice to have a script that does require a USB stick for entware as this is not possible for everyone.

I don't understand the concept of openvpn-event. I understand when it is ran. The wiki simply states "Uses the same syntax/parameters as the "up" and "down" scripts in OpenVPN." If I add something in openvpn-event it will be ran every time someone connects to an openvpn server that is also on the router, when it would only need to be ran when tun12 is up or down. Also how do you add iptables when say tun12 is created and delete ip tables when tun12 goes down, within one script? Again my limited knowledge here. I should have a whole thread just on openvpn-event.

I still use the dnsmasq 'conf-dir' directive because it allows for me to have an easy way to sort things out. Is this practice is considered bad? I didn't realize there are technical differences in dnsmasq between differing firmwares. I simply stated to add the ipset to dnsmasq.conf.add so that others in the future would be able to use this as an example.

 

[Martineau]

I don't understand the concept of openvpn-event. I understand when it is ran. The wiki simply states "Uses the same syntax/parameters as the "up" and "down" scripts in OpenVPN."

The full list of trigger events (--up cmd,--down cmd etc.) are listed in OpenVPN 2.4 man page

Simply copy '/jffs/scripts/openvpn-event' from @john9527's openvpn-event script template as-is, so you can explicitly choose which openvpn-event trigger code is only to used by each individual Server/Client.

Now you can create unique trigger event custom scripts
e.g. Create script 'vpnclient1-route-up' to add your iptables rules only for VPN Client 1 and delete them using 'vpnclient1-down'

 

[Martineau]

Does this script only need to be ran one time to add or delete ipsets? Will the final outcome survive reboots? Does the script need entware to run the commands within. I realize that it checks for entware. It would be nice to have a script that does require a USB stick for entware as this is not possible for everyone.

@Xentrk 's scripts do sometimes use actual Entware utilities (but not in this script) however the IPSET backup file needs to be on a USB/HDD attached to the router so that it can be quickly restored during the boot process.

Entware is usually mounted on '/opt' (hence the script will work without having to search for the actual USB/HDD mount point) but the IPSET_Domain.sh script can be modified to use '/tmp/mnt/sdaX' etc. and '/jffs/scripts/post-mount' can be used to execute IPSET_Domains.sh to restore/create the IPSET during the boot.

EDIT: Updated script see post #10 to allow use of the 'dir=' directive to override default 'opt/tmp' directory for IPSET save/restore.

 

[Xentrk]

@Xentrk 's scripts do sometimes use actual Entware utilities (but not in this script) however the IPSET backup file needs to be on a USB/HDD attached to the router so that it can be quickly restored during the boot process.

That is correct. The utility was not used to check for a specific entware package. The back-up IPSET lists are stored on the USB. The Chk_Entware utility was used to validate the USB was mounted (e.g. /opt directory) before trying to restore the IPSET lists from the back-up location on the USB.

 

[andresmorago]

@Martineau
thanks for the latest version 1.3 of the script. I wanted to consult with you why its not working in my case
here is what i ran in order to redirect 2 specific domain to my client #1

andresmorago@RT-AC68U-5358:/jffs/scripts# ./IPSET_Domains.sh 1 principal pandora.com
(IPSET_Domains.sh): 24637 Starting Script Execution
(IPSET_Domains.sh): 24637 IPSET created: principal hash:net family inet hashsize 1024 maxelem 65536
(IPSET_Domains.sh): 24637 CRON schedule created: #principal# '0 2 * * * ipset save principal'
(IPSET_Domains.sh): 24637 Selective Routing Rule via VPN Client 1 created (TAG fwmark 0x1000/0x1000)
(IPSET_Domains.sh): 24637 Completed Script Execution

andresmorago@RT-AC68U-5358:/jffs/scripts# ./IPSET_Domains.sh 1 principal ifconfig.io
(IPSET_Domains.sh): 25058 Starting Script Execution
0 2 * * * ipset save principal > /opt/principal #principal#
(IPSET_Domains.sh): 25058 Selective Routing Rule via VPN Client 1 created (TAG fwmark 0x1000/0x1000)
(IPSET_Domains.sh): 25058 Completed Script Execution

dnsmasq.conf.add shows them correctly

strict-order

dhcp-option=lan,42,10.0.0.1 # ntpMerlin
server=/pool.ntp.org/1.1.1.1

ipset=/pandora.com/principal
ipset=/ifconfig.io/principal

but traffic to these 2 domains is just not going to my vpn but to wan

andresmorago@RT-AC68U-5358:/jffs/scripts# iptables -nvL PREROUTING -t mangle --line
Chain PREROUTING (policy ACCEPT 642 packets, 116K bytes)
num   pkts bytes target     prot opt in     out     source               destination
1        0     0 MARK       all  --  br0    *       0.0.0.0/0            0.0.0.0/0            match-set principal dst MARK or 0x1000
andresmorago@RT-AC68U-5358:/jffs/scripts#

andresmorago@RT-AC68U-5358:/jffs/scripts# ip rule
0:      from all lookup local
9990:   from all fwmark 0x8000/0x8000 lookup main
9991:   from all fwmark 0x3000/0x3000 lookup ovpnc5
9992:   from all fwmark 0x7000/0x7000 lookup ovpnc4
9993:   from all fwmark 0x4000/0x4000 lookup ovpnc3
9994:   from all fwmark 0x2000/0x2000 lookup ovpnc2
9995:   from all fwmark 0x1000/0x1000 lookup ovpnc1
32766:  from all lookup main
32767:  from all lookup default

Accept DNS Configuration is STRICT

i also see that the cron job disappears after a reboot

0 2 * * * ipset save principal > /opt/principal #principal#

am i missing something?

thanks

 

[Martineau]

@Martineau
thanks for the latest version 1.3 of the script. I wanted to consult with you why its not working in my case
here is what i ran in order to redirect 2 specific domain to my client #1

andresmorago@RT-AC68U-5358:/jffs/scripts# ./IPSET_Domains.sh 1 principal pandora.com
(IPSET_Domains.sh): 24637 Starting Script Execution
(IPSET_Domains.sh): 24637 IPSET created: principal hash:net family inet hashsize 1024 maxelem 65536
(IPSET_Domains.sh): 24637 CRON schedule created: #principal# '0 2 * * * ipset save principal'
(IPSET_Domains.sh): 24637 Selective Routing Rule via VPN Client 1 created (TAG fwmark 0x1000/0x1000)
(IPSET_Domains.sh): 24637 Completed Script Execution

andresmorago@RT-AC68U-5358:/jffs/scripts# ./IPSET_Domains.sh 1 principal ifconfig.io
(IPSET_Domains.sh): 25058 Starting Script Execution
0 2 * * * ipset save principal > /opt/principal #principal#
(IPSET_Domains.sh): 25058 Selective Routing Rule via VPN Client 1 created (TAG fwmark 0x1000/0x1000)
(IPSET_Domains.sh): 25058 Completed Script Execution

dnsmasq.conf.add shows them correctly

strict-order

dhcp-option=lan,42,10.0.0.1 # ntpMerlin
server=/pool.ntp.org/1.1.1.1

ipset=/pandora.com/principal
ipset=/ifconfig.io/principal

but traffic to these 2 domains is just not going to my vpn but to wan

andresmorago@RT-AC68U-5358:/jffs/scripts# iptables -nvL PREROUTING -t mangle --line
Chain PREROUTING (policy ACCEPT 642 packets, 116K bytes)
num   pkts bytes target     prot opt in     out     source               destination
1        0     0 MARK       all  --  br0    *       0.0.0.0/0            0.0.0.0/0            match-set principal dst MARK or 0x1000
andresmorago@RT-AC68U-5358:/jffs/scripts#

andresmorago@RT-AC68U-5358:/jffs/scripts# ip rule
0:      from all lookup local
9990:   from all fwmark 0x8000/0x8000 lookup main
9991:   from all fwmark 0x3000/0x3000 lookup ovpnc5
9992:   from all fwmark 0x7000/0x7000 lookup ovpnc4
9993:   from all fwmark 0x4000/0x4000 lookup ovpnc3
9994:   from all fwmark 0x2000/0x2000 lookup ovpnc2
9995:   from all fwmark 0x1000/0x1000 lookup ovpnc1
32766:  from all lookup main
32767:  from all lookup default

Accept DNS Configuration is STRICT

i also see that the cron job disappears after a reboot

0 2 * * * ipset save principal > /opt/principal #principal#

am i missing something?

thanks

My original script in post #10 was a proof of concept for a 'pull' request for @Xentrk's project.

So ,I suggest you try @Xentrk's ongoing project script load_dnsmasq_ipset_iface.sh and also refer to Helpful and troubleshooting section

So basic troubleshooting would be

Ensure dnsmasq is able to populate the IPSET

service restart_dnsmasq

and check if the IPSET 'principal' is being populated

nslookup pandora.com

nslookup ifconfig.io

ipset list principal

and ensure Selective Routing is active

ip route show table main | grep -E "^0\.|^128.|^default"

ip route show table ovpnc1

i also see that the cron job disappears after a reboot

0 2 * * * ipset save principal > /opt/principal #principal#[/COLOR][/FONT][/LEFT]
[FONT=Georgia][COLOR=rgb(20, 20, 20)]
[LEFT]

​

am i missing something?​

No, the 'pull' request code, was not complete, as init-start/services-start would need to be modified to reinstate the cru (cron) job during the boot process.

NOTE: Since there is no inbuilt housekeeping function for the contents of the IPSET, no restore of an existing domain IPSET is performed.

​

 

[andresmorago]

thank you both
ill do some tests tonight and come back if i require more help