Selective Routing for Netflix

[EduardS]

Don't know if it's of any help, but I've attached a connection log while watching netflix on my laptop (192.168.1.205).

 

[Xentrk]

Using version 3.3 of the script, I tried to duplicate your configuration and did not have any issues with Netflix bypassing the VPN and routing thru the WAN interface. Also, please insert any code or command line output using the insert code function so it is easier to read.

# ip rule
0:      from all lookup local
9990:   from all fwmark 0x7000/0x7000 lookup main
10001:  from 192.168.22.1 lookup main
10101:  from 192.168.22.0/24 lookup ovpnc1
32766:  from all lookup main
32767:  from all lookup default

# iptables -nvL PREROUTING -t mangle --line
Chain PREROUTING (policy ACCEPT 4102 packets, 1974K bytes)
num   pkts bytes target     prot opt in     out     source               destination
1      635  709K MARK       all  --  tun11  *       0.0.0.0/0            0.0.0.0/0            MARK xset 0x1/0x7
2        1    60 MARK       all  --  tun12  *       0.0.0.0/0            0.0.0.0/0            MARK xset 0x1/0x7
3        1    60 MARK       all  --  tun13  *       0.0.0.0/0            0.0.0.0/0            MARK xset 0x1/0x7
4        0     0 MARK       all  --  tun21  *       0.0.0.0/0            0.0.0.0/0            MARK xset 0x1/0x7
5     9612 8025K BWDPI_FILTER  udp  --  ppp0   *       0.0.0.0/0            0.0.0.0/0
6        0     0 MARK       tcp  --  br0    *       0.0.0.0/0            0.0.0.0/0            match-set NETFLIX dst,dst MARK or 0x7000
7      231 55363 MARK       tcp  --  br0    *       0.0.0.0/0            0.0.0.0/0            match-set AMAZONAWS dst,dst MARK or 0x7000

I have three VPN tunnels active.

# ip route show table main |  grep -E "^0\.|^128\.|^default|tun1"
10.22.0.13 dev tun11  proto kernel  scope link  src 10.22.0.14
10.24.0.9 dev tun13  proto kernel  scope link  src 10.24.0.10
10.9.0.9 dev tun12  proto kernel  scope link  src 10.9.0.10

These IP addresses are pushed by my provider..

When comparing the ip route show table command with your output, I notice you have a different output. Not sure if this is due to your router or VPN provider.

default via 84.81.218.1 dev vlan34

I looked at the logs. The only issue that stands out is seeing the script starting up two instances concurrently:

Apr 18 09:36:22 (IPSET_Netflix): 666 Starting IPSET_Netflix.sh... /jffs/scripts/IPSET_Netflix.
Apr 18 09:36:23 nat: apply nat rules (/tmp/nat_rules_vlan34_vlan34)
Apr 18 09:36:23 custom_script: Running /jffs/scripts/nat-start
Apr 18 09:36:23 (IPSET_Netflix): 677 Starting IPSET_Netflix.sh... /jffs/scripts/IPSET_Netflix.

There is code in the script to prevent this from occurring. Please confirm the script version you are using. I may need to go back and revisit. I only see one instance in the log where this occurred. I may need to add additional logging to capture any issues.

To rule out an issue with nat-start running concurrently, I recommend that you rename nat-start to nat-start.bk. Power cycle the router to clear any routing. Then, run the script from the command line and test.

 

[EduardS]

I used version 3.3 of your script. I guess my geo location is the root of my problem.
Will try your suggestions later, I'm on my way to work right now.

 

[Xentrk]

Here are some additional debugging tips.

Add a rule to route 108.160.151.3 (whatismypublicip.com) thru the wan in the OpenVPN Client GUI.

# nslookup whatismypublicip.com
Server:    127.0.0.1
Address 1: 127.0.0.1 localhost.localdomain

Name:      whatismypublicip.com
Address 1: 108.160.151.39 whatismypublicip.com

Go to whatismypublicip.com. The ip address it reports should be how Netflix sees you, which is your WAN IP address. You can also confirm your WAN address on the web gui.

Go to whatismyipaddress.com. It should report the IP address of your VPN location.

Are you able to access Netflix successfully when you turn off the VPN? Do you have the Block internet traffic if the tunnel goes down checked or unchecked?

 

[EduardS]

I have posted a connection log while watching netflix on my laptop (post #101), so yes I can watch netflix when I turn off the VPN. I do not block routed clients if tunnel goes down.

Regarding my geo location, checking netflix ip addresses on https://www.peeringdb.com/asn/2906 I see for instance two AMS-IX ip addresses netflix uses (80.249.211.250 and 80.249.210.250) which are not in the NETFLIX ipset your script creates.

 

[EduardS]

When comparing the ip route show table command with your output, I notice you have a different output. Not sure if this is due to your router or VPN provider.

default via 84.81.218.1 dev vlan34

This is due to my internet provider (not my VPN provider which is IPVanish), internet is tagged on vlan 34. My router is an ASUS RT-AC5300.

 

[EduardS]

Here are some additional debugging tips.

Add a rule to route 108.160.151.3 (whatismypublicip.com) thru the wan in the OpenVPN Client GUI.

# nslookup whatismypublicip.com
Server:    127.0.0.1
Address 1: 127.0.0.1 localhost.localdomain

Name:      whatismypublicip.com
Address 1: 108.160.151.39 whatismypublicip.com

Go to whatismypublicip.com. The ip address it reports should be how Netflix sees you, which is your WAN IP address. You can also confirm your WAN address on the web gui.

My ISP WAN IP address is shown.

Go to whatismyipaddress.com. It should report the IP address of your VPN location.

My VPN IP address is shown.

 

[Xentrk]

My ISP WAN IP address is shown.


My VPN IP address is shown.

So it appears the Netflix and/or Amazon ASN may not contain all of the IP addresses required to work from your geolocation. Before I learned about using ASN, I would capture the domain names logged in dnsmasq.log file. I use ASN for most of my streaming media services. But for some, I have to use domain names. When I returned from my recent trip, one of the ASN numbers stopped working for SlingTV. I got it to work again by adding the domain names I captured. So, let us try the domain name method rather than ASN numbers.

I use AB-Solution which handles dnsmasq. Navigate to the location of the dnsmasq.log file directory:

cd /tmp/mnt/absolution/adblocking/logs

Send a copy of the log to a file called netflix:
tail -f dnsmasq.log > netflix

Browse around Netflix and select as many menu options and watch a few shows for a few minutes to generate traffic.

press ctrl-c to exit

Edit this script and replace the IP address with the IP address your device used to watch Netflix:

#!/bin/sh
# script name: getdomainnames.sh
#set -xo
# This script will format the output of tail -f dnsmasq.log > logfile
# where logfile is the output of tail -f dnsmasq.log
#  1. extract records whose contents contain the word "query"
#  2. output only the domain name
#  3. sort file for unique contents to elimnate duplicates
#  4. save to $1_output
#
# Parameters Passed
# $1 = provide the name of the source file when running the script
#     e.g. ./getdomainnames.sh logfile
#
source_file=/tmp/mnt/absolution/adblocking/logs/$1
output_file=$source_file"_output"
cat $source_file | grep query | grep 192.168.22.157 | awk '{ print $6 }' | sort -u > $output_file

Run the script and provide the name of the file you used in the tail command. e.g.
./getdomainnames.sh netflix

This will create a file called netflix_output listing the domain names that were called when watching netflix.

You have to do a sanity check on the list of domain names and strip out any domain names that are not related. For example, if you used your PC or laptop to watch Netflix, you may also see calls to Microsoft or your anti-virus software cloud service. When doing this on my Roku, I also see Roku domain names that send telemetry data to Roku on usage, etc. You can remove those entries.

Copy the file contents of /tmp/mnt/absolution/adblocking/logs/netflix_output to /jffs/scripts/NETFLIX

This script will use the domain names rather than IPSET to route Netflix traffic.

#!/bin/sh
####################################################################################################
# Written By: Xentrk
# Script Name: netflix_bypass.sh
# Version 1.0
#
# Description:
#   Selective Routing Script for WAN and OpenVPN Clients
#
# Grateful:
#   Thank you to @Martineau on snbforums.com for sharing his Selective Routing expertise
#   and on-going support!
####################################################################################################
logger -t "($(basename $0))" $$ Starting..." $0${*:+ $*}."
# Uncomment the line below for debugging
set -xo

# Prevent script from running twice at boot up
#exec 400>/tmp/vpnroutingcheck.lck
#flock -x 400 || exit 0
#sleep 120
PROGNAME=$(basename "$0")
LOCKFILE_DIR=/tmp
LOCK_FD=200

lock() {
    local prefix=$1
    local fd=${2:-$LOCK_FD}
    local lock_file=$LOCKFILE_DIR/$prefix.lock

    # create lock file
    eval "exec $fd>$lock_file"

    # acquier the lock
    flock -n $fd \
        && return 0 \
        || return 1
}

eexit() {
    local error_str="$@"
    echo $error_str
    exit 1
}

main() {
    lock $PROGNAME \
        || eexit "Only one instance of $PROGNAME can run at one time."

ipset create LAN_GW hash:net family inet hashsize 1024 maxelem 65536

# WAN
ip rule del fwmark 0x7000
ip rule add fwmark 0x7000 table 254 prio 9990

#VPN Client 1
#ip rule del fwmark 0x1000
#ip rule add fwmark 0x1000 table 111 prio 9991

ip route flush cache

# extract LAN ip addresses
ipset add LAN_GW $(nvram get lan_ipaddr)

iptables -t mangle -D PREROUTING -i br0 -p tcp -m set --match-set LAN_GW src,dst -j MARK --set-mark 0x7000/0x7000
iptables -t mangle -A PREROUTING -i br0 -p tcp -m set --match-set LAN_GW src,dst -j MARK --set-mark 0x7000/0x7000

#Pull Netflix Domain Names and route to WAN
for DNS in $(awk '{ print $1 }' /jffs/scripts/NETFLIX)
    do
      iptables -t mangle -D PREROUTING -i br0 -d $DNS -j MARK --set-mark 0x7000/0x7000
      iptables -t mangle -A PREROUTING -i br0 -d $DNS -j MARK --set-mark 0x7000/0x7000
    done
ip route flush cache

logger -t "($(basename $0))" $$ Ending..." $0${*:+ $*}."
}
main

If you need a VPN provider that can get around VPN blocks put in place by Netflix and other streaming media and financial services companies, I can recommend my provider. I posted my experience in my new blog site: https://x3mtek.com/why-i-use-torguard-as-my-vpn-provider/

 

[Xentrk]

I have posted a connection log while watching netflix on my laptop (post #101), so yes I can watch netflix when I turn off the VPN. I do not block routed clients if tunnel goes down.

Regarding my geo location, checking netflix ip addresses on https://www.peeringdb.com/asn/2906 I see for instance two AMS-IX ip addresses netflix uses (80.249.211.250 and 80.249.210.250) which are not in the NETFLIX ipset your script creates.

I will look into the differences and analyze what changes may be required to the script to change the ASN source from ipinfo.io to peeringdb.com. There are some other sources as well. It would be interesting to compare and determine if there are any differences among the providers of ASN information.

Update
I noticed another ASN for Netflix Streaming Services - AS40027. You can try adding the following code to see if it fixes your issue.

#Pull all IPv4s listed for Netflix Streaming Services, Inc - AS40027
netsv4=`curl https://ipinfo.io/AS40027 2>/dev/null | grep -E "a href.*40027\/" | grep -v ":" |sed 's/^.*\">//; s/<.*//; /^\s*$/d'`
for net in $netsv4
do
  ipset add NETFLIX $net
done
unset netsv4

 

[EduardS]

That last one didn't do the trick either, but I found out where the culprit lies. It lies in the Netflix Open Connect network;

https://www.theregister.co.uk/2016/06/22/boffins_map_netflixs_open_connect_cdn/

https://arxiv.org/abs/1606.05519v3

https://arxiv.org/pdf/1606.05519v3.pdf

https://openconnect.netflix.com/en/

http://oc.nflxvideo.net/docs/OpenConnect-Deployment-Guide.pdf

https://media.netflix.com/en/compan...e-globe-to-deliver-a-great-viewing-experience

Netflix provides video streaming appliances to local ISP's to provide high speed video streaming experience for the ISP customers. These appliances have local ISP network IP addresses. Now on my laptop I started a Netflix session and with [Shift][Ctrl][Alt][D] I retrieved the dns names of the Netflix Open Connect streaming appliances of my ISP (KPN), these were c016.rtm.001.kpn.isp.nflxvideo.net and c023.rtm001.kpn.isp.nflxvideo.net, with nslookup I resolved these to IP4 addresses 195.121.126.230 and 195.121.126.236, I added these to the NETFLIX ipset and voila, netflix was finally playing with VPN tunnel enabled. Of course next time playing another ISP Netflix Open Connect streaming appliance was used (c019.ams001.kpn.isp.nflxvideo.net) which was also a 195.121.126.x ip address so I added 195.121.126.0/24 to the NETFLIX ipset and now netflix is playing from all my devices without problems so far.

Conclusion, the script is fine but one has to add the local ip addresses of the ISP Netflix Open Connect streaming appliances. I now can finally protect my privacy with router based VPN and reroute netflix outside the VPN tunnel.

 

[Xentrk]

That last one didn't do the trick either, but I found out where the culprit lies. It lies in the Netflix Open Connect network;

https://www.theregister.co.uk/2016/06/22/boffins_map_netflixs_open_connect_cdn/

https://arxiv.org/abs/1606.05519v3

https://arxiv.org/pdf/1606.05519v3.pdf

https://openconnect.netflix.com/en/

http://oc.nflxvideo.net/docs/OpenConnect-Deployment-Guide.pdf

https://media.netflix.com/en/compan...e-globe-to-deliver-a-great-viewing-experience

Netflix provides video streaming appliances to local ISP's to provide high speed video streaming experience for the ISP customers. These appliances have local ISP network IP addresses. Now on my laptop I started a Netflix session and with [Shift][Ctrl][Alt][D] I retrieved the dns names of the Netflix Open Connect streaming appliances of my ISP (KPN), these were c016.rtm.001.kpn.isp.nflxvideo.net and c023.rtm001.kpn.isp.nflxvideo.net, with nslookup I resolved these to IP4 addresses 195.121.126.230 and 195.121.126.236, I added these to the NETFLIX ipset and voila, netflix was finally playing with VPN tunnel enabled. Of course next time playing another ISP Netflix Open Connect streaming appliance was used (c019.ams001.kpn.isp.nflxvideo.net) which was also a 195.121.126.x ip address so I added 195.121.126.0/24 to the NETFLIX ipset and now netflix is playing from all my devices without problems so far.

Conclusion, the script is fine but one has to add the local ip addresses of the ISP Netflix Open Connect streaming appliances. I now can finally protect my privacy with router based VPN and reroute netflix outside the VPN tunnel.

Thanks @EduardS, excellent feedback! I have seen the nflxvideo.net domains posted on other forums. Sounds like they will differ by ISP provider.

You should be able to harvest the domain names over time. Once you collect them, place the domain names in a file use the script in post #108 to add to ipset list.

Following is an alternative method to add the IPv4 addresses to the NETFLIX ipset list. But I prefer the post #108 method. It does return an error message if the nslookup returns an IPv6 addresses. But does not cause the collection of IPv4 addresses to fail. Some additional coding would be required to capture the IPv6 error and send the message to /dev/null.

# list domains for selective routing
for domain in \
"netflix.com" \
"ichnaea.netflix.com" \
"movies.netflix.com" \
"www.netflix.com" \
"nflxext.com" \
"cdn1.nflxext.com" \
"nflximg.com" \
"nflxvideo.net" \
"ipv4_1.cxl0.c145.sjc002.ix.nflxvideo.net" \
"amazonaws.com" \
"whatsmyip.org"
do
  # extract ip addresses
    for ip in $(nslookup $domain | awk '/^Name:/,0{if (/^Addr/)print $3}'); do
    ipset add NETFLIX $ip
done

 

[EduardS]

Thanks @EduardS, excellent feedback! I have seen the nflxvideo.net domains posted on other forums. Sounds like they will differ by ISP provider.

Correct, every ISP uses his own Netflix Open Connect appliances. Of course some ISP's are just fronts for other ISP's, like the Dutch company Telfort is just a front for the Dutch ISP KPN, thus Telfort uses the Netflix appliances of KPN. Once the ip addresses of these ISP streaming appliances are know, one can effectively reroute Netflix apart from the VPN thanks to your script.

You should be able to harvest the domain names over time. Once you collect them, place the domain names in a file use the script in post #108 to add to ipset list.

Following is an alternative method to add the IPv4 addresses to the NETFLIX ipset list. But I prefer the post #108 method. It does return an error message if the nslookup returns an IPv6 addresses. But does not cause the collection of IPv4 addresses to fail. Some additional coding would be required to capture the IPv6 error and send the message to /dev/null.

# list domains for selective routing
for domain in \
"netflix.com" \
"ichnaea.netflix.com" \
"movies.netflix.com" \
"www.netflix.com" \
"nflxext.com" \
"cdn1.nflxext.com" \
"nflximg.com" \
"nflxvideo.net" \
"ipv4_1.cxl0.c145.sjc002.ix.nflxvideo.net" \
"amazonaws.com" \
"whatsmyip.org"
do
  # extract ip addresses
    for ip in $(nslookup $domain | awk '/^Name:/,0{if (/^Addr/)print $3}'); do
    ipset add NETFLIX $ip
done

Again a very helpful script addition. Thanks for all your continuing help and support!

 

[loki_berra]

I used v3.3 but there was an issue with the sed portion. I had to change it to sed 's/^.*<a href="\/AS2906\///; s/" >//' in order to get the IPs from ipinfo.io
Any suggestions on why the original sed would not work for me?

The IPs got populated in the IPSET but netflix still would not work. I then tried harvesting the domain names (I used PIHole) but I still had no luck when adding them using the code in post #111.

 

[Xentrk]

I used v3.3 but there was an issue with the sed portion. I had to change it to sed 's/^.*<a href="\/AS2906\///; s/" >//' in order to get the IPs from ipinfo.io
Any suggestions on why the original sed would not work for me?

The IPs got populated in the IPSET but netflix still would not work. I then tried harvesting the domain names (I used PIHole) but I still had no luck when adding them using the code in post #111.

I will look at the sed/ipinfo.io syntax again in case something changed.

First thing to check is if the routing is working and you are able to bypass the VPN. Following the tips in post 104. You can do it by manually issuing the command after running the script:

ipset add 108.10.151.39 NETFLIX

Now, go to whatismypublicip.com, it should now report WAN IP address. If you go to whatismyipaddress.com, it should report VPN IP address. If this works, then I suspect you may have the issue @EduardS describes and will need to spend time harvesting the domain names.

In regards to post #111, this is a better method. You place all of the domain names you collected in a file. Then, add this code to loop through the file and route as necessary. In this example, I could never get the ASN to work for CBS. So, I had to use domain names that were collected as described in post #108.

#Pull CBS Domain Names and route to OVPNC2
for DNS in $(awk '{ print $1 }' /jffs/scripts/CBS_Domains)
    do
      iptables -t mangle -D PREROUTING -i br0 -d $DNS -j MARK --set-mark 0x2000/0x2000
      iptables -t mangle -A PREROUTING -i br0 -d $DNS -j MARK --set-mark 0x2000/0x2000
    done
ip route flush cache

 

[Adamm]

@Xentrk

So I decided to dabble into selective routing but in the opposite direction then posted about here. I want only my Netflix traffic to flow through my Astrill VPN not vise versa (I'm surprised how limited the information was compared to having Netflix WAN only). I'm not exactly familiar with the ip binary or packet marking but from the posts I read on the topic I was able to put something together (pointing out any mistakes would be appreciated!)

#!/bin/sh
# VPNFlix By Adamm - 5/5/18
# Route Netflix Traffic Thorugh VPN Client1

Check_Lock () {
        if [ -f "/tmp/vpnflix.lock" ] && [ -d "/proc/$(sed -n '2p' /tmp/vpnflix.lock)" ] && [ "$(sed -n '2p' /tmp/vpnflix.lock)" != "$$" ] ; then
            logger -st Netflix "[INFO] Lock File Detected ($(sed -n '1p' /tmp/vpnflix.lock)) (pid=$(sed -n '2p' /tmp/vpnflix.lock)) - Exiting (cpid=$$)"
            echo
            exit 1
        else
            echo "$@" > /tmp/vpnflix.lock
            echo "$$" >> /tmp/vpnflix.lock
        fi
}

case "$1" in
   
    start)
        Check_Lock "$@"
        if [ -d "/opt/bin" ] && [ ! -f "/opt/bin/vpnflix" ]; then
            ln -s /jffs/scripts/vpnflix.sh /opt/bin/vpnflix
        fi
        if [ -f "/jffs/scripts/netflix.ipset" ]; then ipset restore -! -f "/jffs/scripts/netflix.ipset"; else ipset create NETFLIX hash:net; fi   
        ip rule del fwmark 0x7000/0x7000
        ip rule add fwmark 0x7000/0x7000 table 254 prio 9990
        ip rule del fwmark 0x1000/0x7000
        ip rule add fwmark 0x1000/0x7000 table 111 prio 9991
        iptables -D PREROUTING -t mangle -m set --match-set NETFLIX dst -j MARK --set-mark 0x1000/0x1000   
        iptables -A PREROUTING -t mangle -m set --match-set NETFLIX dst -j MARK --set-mark 0x1000/0x1000
        sed -i '\~#VPNFlix~d' /jffs/configs/dnsmasq.conf.add
        echo "ipset=/netflix.com/nflxvideo.net/nflxso.net/nflxext.com/nflximg.net/NETFLIX #VPNFlix" >> /jffs/configs/dnsmasq.conf.add
        chmod +x /jffs/configs/dnsmasq.conf.add
        cru d Netflix_save
        cru a Netflix_save "30 * * * * sh /jffs/scripts/vpnflix.sh save"
    ;;
    save)
        Check_Lock "$@"
        echo "Saving Netflix Server List..."
        if ipset -L -n NETFLIX >/dev/null 2>&1; then ipset save NETFLIX > "/jffs/scripts/netflix.ipset"; fi
        echo "Complete! - $(wc -l < /jffs/scripts/netflix.ipset) Entries Total"
    ;;
    disable)
        Check_Lock "$@"
        echo "Disabing Netflix Policy Routing..."
        ip rule del fwmark 0x7000/0x7000
        ip rule del fwmark 0x1000/0x7000
        iptables -D PREROUTING -t mangle -m set --match-set NETFLIX dst -j MARK --set-mark 0x1000/0x1000
        if ipset -L -n NETFLIX >/dev/null 2>&1; then ipset save NETFLIX > "/jffs/scripts/netflix.ipset"; fi
        ipset destroy NETFLIX
        echo "Complete!"
    ;;
    *)
        echo "Command Not Recognized, Please Try Again"
        echo "Accepted Commands Are; (sh $0 [start|save|disable])"
        echo; exit 2
    ;;

esac

if [ -f "/tmp/vpnflix.lock" ] && [ "$$" = "$(sed -n '2p' /tmp/vpnflix.lock)" ]; then rm -rf "/tmp/vpnflix.lock"; fi

I notice for starters in this script you cover every AmazonAWS IP along with Netflix's ASN. I'm not sure this is exactly necessary, including all of Amazons IP space is a-little excessive (nor did it work in my case, some IPs are owned by Akami/Time Warner/TPG I assume for local caching). Anyway..

I feel like the solution here is a learning approach. Netflix seem to use a bunch of servers but common domains which can be filtered.

With a few minutes of browsing I was able to capture the following queries;

aam2beujngdc6o5kpyc6q.r.nflxso.net
aczzqee67micyn37j5htq.r.nflxso.net
ad4pa67hlk7n4ou3jttje.r.nflxso.net
anycast.ftl.netflix.com
assets.nflxext.com
codex.nflxext.com
future.prod.ftl.netflix.com
ipv4-c001-bne001-waia-isp.1.oca.nflxvideo.net
ipv4-c004-was001-ix.1.oca.nflxvideo.net
ipv4-c015-was001-ix.1.oca.nflxvideo.net
ipv4-c016-syd002-ix.1.oca.nflxvideo.net
ipv4-c016-was001-ix.1.oca.nflxvideo.net
ipv4-c092-was001-ix.1.oca.nflxvideo.net
ipv4-c104-was001-ix.1.oca.nflxvideo.net
oca-api.netflix.com
occ-0-2088-2567.1.nflxso.net
occ-0-2430-2433.1.nflxso.net
push.prod.netflix.com
www.netflix.com

Perfect.. Breaking down this data it gives us the following domains to work with;

netflix.com
nflxso.net
nflxext.com
nflxvideo.net
nflximg.net

Then its as simple as using dnsmasq to-do the heavily lifting and populate our new IPSet on the fly.

ipset=/netflix.com/nflxvideo.net/nflxso.net/nflxext.com/nflximg.net/NETFLIX

So yeah, I'll probably improve on this solution in the coming days for personal use if you don't intend to have a "reverse" method of your current script, but just thought I'd post my findings due to how few examples I could find.


One problem I did come across was the following default rule causing everything to fail. Not sure exactly what it does, but an explanation would be appreciated (and how to fix my setup without just deleting it assuming its important).

-A PREROUTING -i tun11 -j MARK --set-xmark 0x1/0x7

 

[Xentrk]

@Xentrk

So I decided to dabble into selective routing but in the opposite direction then posted about here. I want only my Netflix traffic to flow through my Astrill VPN not vise versa (I'm surprised how limited the information was compared to having Netflix WAN only). I'm not exactly familiar with the ip binary or packet marking but from the posts I read on the topic I was able to put something together (pointing out any mistakes would be appreciated!)..

Hi @Adamm, Welcome to the world of Selective Routing. Like you, I prefer to route my Netflix traffic thru one of my OpenVPN tunnels rather than the WAN iface. User testing of the script to route Netflix traffic thru the WAN using the ASN for Amazon AWS and Netflix exposed some information. An unexpected outcome is Amazon and Amazon Prime traffic will also get routed to the WAN. In my use case, I route both to the the same VPN tunnel. @EduardS discovered the Netflix Open Connect domains which using the ASN method does not address. So doing the old fashion method of mining the log file for domains appears to be the best option. Or using the ipset method to collect the domains as you are using. I have experimented with that method previously and wrote some test scripts to help in my learning. They are posted somewhere in this thread.

There are some comments in the thread from @Martineau explaining why the Selective Routing code in the Wiki, which your code appears to use, is flawed. See this post for more information of the reason. This code uses the recommended method.

What I am going to share with you is the next iteration of VPN_Routing.sh script I last worked on. I still have some work to do. It can replace the selective routing functionality of the Web GUI. Or, the two can coexist. I am using Prior numbers that do not conflict with AsusWrt-Merlin. The first part of the script assigns the LAN clients to the WAN or appropriate VPN Interface. You may not required this part of the code. There is another script I will need to send you to automate creation of rules for LAN clients if you want to use this. The later portion sets up the fwmarks for each OpenVPN client that are used by the separate scripts I created for each streaming media. Easier to manage that way. So, nat-start looks like

#!/bin/sh
sh /jffs/scripts/VPN_Routing.sh
sh /jffs/scripts/netflix.sh
sh /jffs/scripts/cbs.sh
sh /jffs/scripts/bbc.sh

#!/bin/sh
####################################################################################################
# Script Name: VPN_Routing.sh
# Written By: Xentrk
# Version 4.0
#
# Grateful:
#   Thank you to @Martineau on snbforums.com for sharing his Selective Routing expertise
#   and on-going support!
#
# Description:
#   Selective Routing Script for LAN Clients using Asus-MerlinWRT firmware.
#   Specify the LAN Clients that will use each OpenVPN Client interface in a singe file rather
#   than the Web GUI page for each OpenVPN Client.  See (2) Pre-Staging below for instructions.
#
# Instructions:
#   (1) place the line:
#         sh /jffs/scripts/VPN_Routing.sh
#       inside of /jffs/scripts/nat-start so the script will run automatically at boot.
#       Place other selective routing scripts in nat-start after the entry above.
#
#   (2) Pre-Staging;
#         (a) You must define static-ip address to your lan clients on the DHCP Server page.
#             Run the script get_lan_clients.sh to create the initial /jffs/configs/lan_clients configuration
#             file.  The next step is to assign the iface to each lan client by editing the first column in the
#             /jffs/configs/lan_clients file as follows: 0=WAN, 1=OVPNC1, 2=OVPNC2, 3=OVPNC3, 4=OVPNC4, 5=OVPNC5.
#             Optionally, you can delete any DHCP clients defined to use the WAN as these will be ignored by the script.
#             DHCP
#             clients not defined to use a VPN Client will autoatically default to the WAN.
#         (b) get_lan_clients.sh will create a backup of the /jffs/configs/lan_clients file if one exits. This provides the
#             ability to restore a previous version if neccessary. If you do create a new version
#             of a /jffs/configs/lan_clients file, run /jffs/scripts/purge_lan_clients_sr.sh to purge the old
#             routing rules before running VPN_Routing.sh or you may get unexpected results!  You have been warned.
#
####################################################################################################
logger -t "($(basename $0))" $$ Starting..." $0${*:+ $*}."
# Uncomment the line below for debugging
set -x

# Prevent script from running concurrently when called from nat-start

PROGNAME=$(basename "$0")
LOCKFILE_DIR=/tmp
LOCK_FD=200

lock() {
    local prefix=$1
    local fd=${2:-$LOCK_FD}
    local lock_file=$LOCKFILE_DIR/$prefix.lock

    # create lock file
    eval "exec $fd>$lock_file"

    # acquier the lock
    flock -n $fd \
        && return 0 \
        || return 1
}

eexit() {
    local error_str="$@"
    echo $error_str
    exit 1
}

main() {
    lock $PROGNAME \
        || eexit "Only one instance of $PROGNAME can run at one time."


# check if /jffs/congis/lan_clients file exist. Exit script if true.
if [ ! -f /jffs/configs/lan_clients ]
  then
    logger -t "($(basename $0))" $$ Warning: required file /jffs/configs/lan_clients does not exist. Exiting script..." $0${*:+ $*}."
    exit
fi


# WAN
ip rule del fwmark 0x7000/0x7000
ip rule add fwmark 0x7000/0x7000 table 254 prio 4000

# Any special overrides for WAN traffic go here. Start with prio 4001 and increment by 1
#ip rule del from 192.168.1.152 table 254
#ip rule add from 192.168.1.152 table 254 prio 4001

# Any special overrides for OVPNC1 traffic go here. Start with prio 4101 and increment by 1
#ip rule del from 192.168.1.152 table 100
#ip rule add from 192.168.1.152 table 100 prior 4101

# Any special overrides for OVPNC2 traffic go here. Start with prio 4201 and increment by 1
# Route whatismypublicip.com to OVPNC2 no matter the iface connected to
ip rule del to 108.160.151.39 table 112
ip rule add to 108.160.151.39 table 112 prio 4201

# Any special overrides for OVPNC3 traffic go here. Start with prio 4301 and increment by 1

# Any special overrides for OVPNC4 traffic go here. Start with prio 4401 and increment by 1

# Any special overrides for OVPNC5 traffic go here. Start with prio 4501 and increment by 1

# Perform routing for lan clients specified in /jffs/configs/lan_clients

old_IFS="$IFS"
IFS=" "

file="/jffs/configs/lan_clients"

while IFS=" " read -r f1 f2
do
case "$f1" in
 1) echo "$f2" >> /tmp/ovpnc1.$$ ;;
 2) echo "$f2" >> /tmp/ovpnc2.$$ ;;
 3) echo "$f2" >> /tmp/ovpnc3.$$ ;;
 4) echo "$f2" >> /tmp/ovpnc4.$$ ;;
 5) echo "$f2" >> /tmp/ovpnc5/$$ ;;
 *) echo "Unexpected value ignored" ;;
esac
# Uncomment the lines below for debugging
#printf 'iface: %s ip: %s desc: %s\n' "$f1" "$f2" "$f3"
#echo "$line"
done < "$file"
IFS=$old_IFS

# route OVPNC1 clients
if [ -f /tmp/ovpnc1.$$ ]
then
count=5100
    for ip in $(awk '{ print $1 }' /tmp/ovpnc1.$$); do
       ip rule del from $ip table 111 2> /dev/null > /dev/null
       ip rule add from $ip table 111 prior $count
       count=`expr $count + 1`
    done
unset ip
fi

# route OVPNC2 clients
if [ -f /tmp/ovpnc1.$$ ]
then
count=5200
for ip in $(awk '{ print $1 }' /tmp/ovpnc2.$$); do
        ip rule del from $ip table 112 2> /dev/null > /dev/null
        ip rule add from $ip table 112 prior $count
        count=`expr $count + 1`
    done
unset ip
fi

# route OVPNC3 clients
if [ -f /tmp/ovpnc3.$$ ]
then
count=5300
for ip in $(awk '{ print $1 }' /tmp/ovpnc3.$$)
    do
        ip rule del from $ip table 113 2> /dev/null > /dev/null
        ip rule add from $ip table 113 prior $count
        count=`expr $count + 1`
    done
unset ip
fi

# route OVPNC4 clients
if [ -f /tmp/ovpnc4.$$ ]
then
count=5400
for ip in $(awk '{ print $1 }' /tmp/ovpnc4.$$)
    do
        ip rule del from $ip table 114 2> /dev/null > /dev/null
        ip rule add from $ip table 114 prior $count
        count=`expr $count + 1`
    done
unset ip
fi

# route OVPNC5 clients
if [ -f /tmp/ovpnc5.$$ ]
then
count=5500
for ip in $(awk '{ print $1 }' /tmp/ovpnc5.$$)
    do
        ip rule del from $ip table 115 2> /dev/null > /dev/null
        ip rule add from $ip table 115 prior $count
        count=`expr $count + 1`
    done
unset ip
fi

# Remove temporary files
rm /tmp/ovpnc*.*

################################################################################################
# Optional Section
# Enable fwmarks for OpenVPN Clients to support selective routing by other scripts
################################################################################################

#VPN Client 1
ip rule del fwmark 0x1000/0x1000
ip rule add fwmark 0x1000/0x1000 table 111 prio 5001

#VPN Client 2
ip rule del fwmark 0x2000/0x2000
ip rule add fwmark 0x2000/0x2000 table 112 prio 5002

#VPN Client 3
ip rule del fwmark 0x3000/0x3000
ip rule add fwmark 0x3000/0x3000 table 113 prio 5003

#VPN Client 4
#ip rule del fwmark 0x4000/0x4000
#ip rule add fwmark 0x4000/0x4000 table 114 prio 5004

#VPN Client 5
#ip rule del fwmark 0x5000/0x5000
#ip rule add fwmark 0x5000/0x5000 table 115 prio 5005

ip route flush cache

# Default non VPN clients to the WAN iface
ipset create LAN_GW hash:net family inet hashsize 1024 maxelem 65536
ipset add LAN_GW $(nvram get lan_ipaddr)

iptables -t mangle -D PREROUTING -i br0 -p tcp -m set --match-set LAN_GW src,dst -j MARK --set-mark 0x7000/0x7000
iptables -t mangle -A PREROUTING -i br0 -p tcp -m set --match-set LAN_GW src,dst -j MARK --set-mark 0x7000/0x7000

logger -t "($(basename $0))" $$ Ending..." $0${*:+ $*}."
}
main

 

[Xentrk]

I then run separate scripts for the streaming media selective routing to make things easier to maintain.

Netflix is being routed to OVPNC1

#!/bin/sh
####################################################################################################
# Written By: Xentrk
# Version 2.0
#
# Description:
#   Selective Routing Script Netflix
#
# Grateful:
#   Thank you to @Martineau on snbforums.com for sharing his Selective Routing expertise
#   and on-going support!
# 108.160.151.39
#
####################################################################################################

logger -t "($(basename $0))" $$ Starting netflix.sh..." $0${*:+ $*}."

# Uncomment the line below for debugging
set -xo

# Create IPSET lists
ipset create NETFLIX hash:net family inet hashsize 1024 maxelem 65536

#Pull all IPv4s listed for Netflix USA - AS2906
netsv4=`curl http://ipinfo.io/AS2906 2>/dev/null | grep -E "a href.*2906\/" | grep -v ":" |sed 's/^.*\">//; s/<.*//; /^\s*$/d'`
for net in $netsv4
do
  ipset add NETFLIX $net
done
unset netsv4

#Pull all IPv4s listed for Amazon AWS - AS16509
netsv4=`curl http://ipinfo.io/AS16509 2>/dev/null | grep -E "a href.*16509\/" | grep -v ":" | sed 's/^.*\">//; s/<.*//; /^\s*$/d'`
for net in $netsv4
do
  ipset add NETFLIX $net
done
unset netsv4

# VPN Client 1
iptables -t mangle -D PREROUTING -i br0 -p tcp -m set --match-set NETFLIX dst,dst -j MARK --set-mark 0x1000/0x1000
iptables -t mangle -A PREROUTING -i br0 -p tcp -m set --match-set NETFLIX dst,dst -j MARK --set-mark 0x1000/0x1000

logger -t "($(basename $0))" $$ Ending netflix.sh..." $0${*:+ $*}."

CBS is being routed to OVPNC2 using domain names harvested from dnsmasq

#!/bin/sh
####################################################################################################
# Written By: Xentrk
# Version 2.0
#
# Description:
#   Selective Routing Script for CBS
#
# Grateful:
#   Thank you to @Martineau on snbforums.com for sharing his Selective Routing expertise
#   and on-going support!
####################################################################################################
logger -t "($(basename $0))" $$ Starting cbs.sh..." $0${*:+ $*}."
# Uncomment the line below for debugging
set -xo

#Pull CBS Domain Names and route to OVPNC2
for DNS in $(awk '{ print $1 }' /jffs/scripts/CBS_Domains)
    do
      iptables -t mangle -D PREROUTING -i br0 -d $DNS -j MARK --set-mark 0x2000/0x2000
      iptables -t mangle -A PREROUTING -i br0 -d $DNS -j MARK --set-mark 0x2000/0x2000
    done
ip route flush cache

logger -t "($(basename $0))" $$ Ending cbs.sh..." $0${*:+ $*}."

Spoiler: CBS_Domains

cbsdai-ads.akamaized.net
cbsdai-vh.akamaihd.net
cbskwch-i.akamaihd.net
cbsplaylistserver.aws.syncbak.com
cbsservice.aws.syncbak.com
cws.conviva.com
dai.google.com
dw.cbsi.com
dyywvl37rgjgd.cloudfront.net
feeds-cbsn.cbsnews.com
imasdk.googleapis.com
om.cbsi.com
syncbakblackout.akamaized.net
www.cbs.com
wwwimage.cbsstatic.com

BBC is being routed to OVPNC3 using a combination of ASN and domain names harvested from dnsmasq.

#!/bin/sh
####################################################################################################
# Written By: Xentrk
# Version 2.0
#
# Description:
#   Selective Routing Script for BBC
#
# Grateful:
#   Thank you to @Martineau on snbforums.com for sharing his Selective Routing expertise
#   and on-going support!
####################################################################################################

logger -t "($(basename $0))" $$ Starting bbc.sh..." $0${*:+ $*}."
# Uncomment the line below for debugging
set -xo

# Create IPSET lists
ipset create IPLAYER hash:net family inet hashsize 1024 maxelem 65536

#Pull all IPv4s listed for BBC - AS2818
netsv4=`curl https://ipinfo.io/AS2818 2>/dev/null | grep -E "a href.*2818\/" | grep -v ":" |sed 's/^.*\">//; s/<.*//; /^\s*$/d'`
for net in $netsv4
do
  ipset add IPLAYER $net
done
unset netsv4

#Pull all IPv4s listed for BBC - AS31459
netsv4=`curl https://ipinfo.io/AS31459 2>/dev/null | grep -E "a href.*31459\/" | grep -v ":" |sed 's/^.*\">//; s/<.*//; /^\s*$/d'`
for net in $netsv4
do
  ipset add IPLAYER $net
done
unset netsv4

#Pull all IPv4s listed for BBC - AS54113
#netsv4=`curl https://ipinfo.io/AS54113 2>/dev/null | grep -E "a href.*54113\/" | grep -v ":" |sed 's/^.*\">//; s/<.*//; /^\s*$/d'`
#for net in $netsv4
#do
#  ipset add IPLAYER $net
#done
#unset netsv4
#========================================================================================
# Route BBC Player to VPN Client 3
for DNS in $(awk '{ print $1 }' /jffs/scripts/BBCpfsense); do
      iptables -t mangle -D PREROUTING -i br0 -d $DNS -j MARK --set-mark 0x3000/0x3000
      iptables -t mangle -A PREROUTING -i br0 -d $DNS -j MARK --set-mark 0x3000/0x3000
    done
#========================================================================================
# VPN Client 3
iptables -t mangle -D PREROUTING -m set --match-set IPLAYER dst,dst -j MARK --set-mark 0x3000/0x3000
iptables -t mangle -A PREROUTING -m set --match-set IPLAYER dst,dst -j MARK --set-mark 0x3000/0x3000

logger -t "($(basename $0))" $$ Ending bbc.sh..." $0${*:+ $*}."

Spoiler: BBC Domains

a1089.d.akamai.net
a1104.w10.akamai.net
a2.w10.akamai.net
account-origin-live.bbc.net.uk
account.bbc.com b1rbsov.bidi.live.bbc.co.uk
bbcdotcom.2cnt.net
bbciplayer.metafaq.com
bn1305.storage.live.com
bootstrapcdn.jdorfman.netdna-cdn.com
cd-megavolt.90fe2324ce3eb149.xhst.bbci
component.iplayer.api.bbc.co.uk
e3891.dscf.akamaiedge.net
e3891.f.akamaiedge.net
e8218.dscb1.akamaiedge.net
emp.bbc.co.uk emp.bbci.co.uk
fig.bbc.co.uk fig.bbc.net.uk
gn.symcd.com ibl.api.bbc.co.uk
ichef.bbc.co.uk
ichef.bbci.co.uk
iplayer-web.files.bbci.co.uk
iplayerhelp.external.bbc.co.uk
ipv4only.arpa
live-ibl-componen-3y285w56k7w5-887784694.eu-west-1.elb.amazonaws.com
live-matc-componen-14ucw7bt4o3x5-61844696.eu-west-1.elb.amazonaws.com
live-noti-componen-9nj5c6fwh1nl-1633728249.eu-west-1.elb.amazonaws.com
live-tvip-componen-poadok30hype-1266449070.eu-west-1.elb.amazonaws.com
login.live.com
maxcdn.bootstrapcdn.com
mm.bidi.bbc.co.uk
mobile.pipe.aria.microsoft.com
music.files.bbci.co.uk mvt.api.bbc.com
mybbc-analytics.files.bbci.co.uk
mybbc.files.bbci.co.uk nav.files.bbci.co.uk
navpromo.90fe2324ce3eb149.xhst.bbci.co.uk
navpromo.api.bbci.co.uk ocsp.comodoca.com
ocsp.usertrust.com open-live.bbc.net.uk
open.live.bbc.co.uk polling.bbc.co.uk
preferences.notifications.api.bbc.co.uk
r.bbci.co.ukr.bbci.co.uk
s.w.org sa-live.com
sa.bbc.co.uk
search.bbc.co.uk
search.bbc.net.uk
search.files.bbci.co.uk
session-origin-live.bbc.net.uk
session.bbc.co.uk session.bbc.com
ssl.bbc.co.uk ssl.bbc.net.uk
uf2f.com vod-dash-uk-live.akamaized.net
vod-dash-uk-live.bbcfmt.hs.llnwd.net
vod-thumb-uk-live.akamaized.net
vod-thumb-uk-live.bbcfmt.hs.llnwd.net
www-bbc-com.bbc.net.uk
www.bbc.co.uk
www.bbc.com
www.bbc.net.uk
a.files.bbci.co.uk
ve-hls-uk-live.akamaized.net
vs-hls-uk-live.akamaized.net
bbc01.sitestat.com
static.bbc.co.uk
static.bbci.co.uk
a753.w10.akamai.net
a936.w16.akamai.net
b3rbsov.bidi.live.bbc.co.uk
b3thdo.bidi.live.bbc.co.uk
b5rbsov.bidi.live.bbc.co.uk
dub-events-lb-1-1837859054.eu-west-1.elb.amazonaws.com
ess.api.bbci.co.uk
ichef-bbci.bbc.net.uk r.bbci.co.uk
static-bbci.bbc.net.uk
stats.bbc.net.uk
vod-hls-uk-live.akamaized.net
vod-sub-uk-live.bbcfmt.hs.llnwd.net
a1670.w10.akamai.net
b2thdo.bidi.live.bbc.co.uk

 

[Adamm]

Hi @Adamm, Welcome to the world of Selective Routing. Like you, I prefer to route my Netflix traffic thru one of my OpenVPN tunnels rather than the WAN iface. User testing of the script to route Netflix traffic thru the WAN using the ASN for Amazon AWS and Netflix exposed some information. An unexpected outcome is Amazon and Amazon Prime traffic will also get routed to the WAN. In my use case, I route both to the the same VPN tunnel. @EduardS discovered the Netflix Open Connect domains which using the ASN method does not address. So doing the old fashion method of mining the log file for domains appears to be the best option. Or using the ipset method to collect the domains as you are using. I have experimented with that method previously and wrote some test scripts to help in my learning. They are posted somewhere in this thread.

There are some comments in the thread from @Martineau explaining why the Selective Routing code in the Wiki, which your code appears to use, is flawed. See this post for more information of the reason. This code uses the recommended method.

What I am going to share with you is the next iteration of VPN_Routing.sh script I last worked on. I still have some work to do. It can replace the selective routing functionality of the Web GUI. Or, the two can coexist. I am using Prior numbers that do not conflict with AsusWrt-Merlin. The first part of the script assigns the LAN clients to the WAN or appropriate VPN Interface. You may not required this part of the code. There is another script I will need to send you to automate creation of rules for LAN clients if you want to use this. The later portion sets up the fwmarks for each OpenVPN client that are used by the separate scripts I created for each streaming media. Easier to manage that way. So, nat-start looks like

Thanks for pointing me in the right direction, so going by the linked post my rules should look something like this instead?

ip rule add fwmark 0x7000 table 254 prio 9990
ip rule add fwmark 0x1000 table 111 prio 9991
iptables -A PREROUTING -t mangle -m set --match-set NETFLIX dst -j MARK --set-mark 0x1000/0x1000

I think I'm slowly starting to wrap my head around it . I assume any packets marked 0x1000 are send to client 1 and 0x7000 wan correct? Still no clue what the table and prio values are for

In any case, thanks for the scripts to reference, made my life a bunch easier. May I suggest a few improvements;

For pulling ASN's use the following domain instead, they are plaintext and require no extra processing (besides converting spaces to newlines) - http://asn.blawk.net/2906

"ipset add" in a for loop is slow and inefficient, I suggest using a similar method as I do in Skynet via the restore console. You can add/remove hundreds of thousands of entries in a fraction of a second using a single ipset process.

Print all your IP's to a list (or stdin), add the correct format via awk;

awk '{print "add NETFLIX " $1}'

Then pipe the output to ipset restore -!

 

[Xentrk]

Thanks for pointing me in the right direction, so going by the linked post my rules should look something like this instead?

ip rule add fwmark 0x7000 table 254 prio 9990
ip rule add fwmark 0x1000 table 111 prio 9991
iptables -A PREROUTING -t mangle -m set --match-set NETFLIX dst -j MARK --set-mark 0x1000/0x1000

I think I'm slowly starting to wrap my head around it . I assume any packets marked 0x1000 are send to client 1 and 0x7000 wan correct? Still no clue what the table and prio values are for

Correct. But, you should also apply a mask when you set the ip rules:

ip rule add fwmark 0x7000/0x7000 table 254 prio 9970

See this post for more info.

The table numbers from the file rt_tables:

cat /etc/iproute2/rt_tables
100 wan0
111 ovpnc1
112 ovpnc2
113 ovpnc3
114 ovpnc4
115 ovpnc5
200 wan1

Priority Numbers
The lower the priority number, the higher the priority order e.g. WAN priority of 9990 gets a higher priority than OVPNC1 priority of 9991.

In any case, thanks for the scripts to reference, made my life a bunch easier. May I suggest a few improvements;

For pulling ASN's use the following domain instead, they are plaintext and require no extra processing (besides converting spaces to newlines) - http://asn.blawk.net/2906

"ipset add" in a for loop is slow and inefficient, I suggest using a similar method as I do in Skynet via the restore console. You can add/remove hundreds of thousands of entries in a fraction of a second using a single ipset process.

Print all your IP's to a list (or stdin), add the correct format via awk;

awk '{print "add NETFLIX " $1}'

Then pipe the output to ipset restore -!

Love the suggestions for improvement. I'll incorporate those changes in the next version. Thanks for the tips.

 

[Xentrk]

Helpful Commands for debugging:

route -n
ip rule
ip route show
ip route show table ovpnc1
ip route show table 111
ip route | grep tun
iptables -nvL PREROUTING -t mangle --line
iptables --line -nvL OVPN