Selective Routing for Netflix

[Xentrk]

Hello Xentrk,

When I run the bypass script, my device shows my vpn IP and Netflix streams without any problem. Unfortunately Kodi is bypassing the VPN too [emoji45] and Plex Media Server also.

Kr.,
Patrick


Verzonden vanaf mijn iPhone met Tapatalk

Sorry for the delayed reply as I have been ill for the past several days

Can you tell me how you have your routing rules defined in the OpenVPN client? Do you list each LAN client individually or by issuing a blanket 192.168.1.0/24 rule?

The command ip rule will show the priority order

iptables -nvL PREROUTING -t mangle --line will show the mangle chain for routing rules


A more robust solution is in the works and I may want to recruit you as a beta tester.

 

[Patje]

Sorry for the delayed reply as I have been ill for the past several days

Can you tell me how you have your routing rules defined in the OpenVPN client? Do you list each LAN client individually or by issuing a blanket 192.168.1.0/24 rule?

The command ip rule will show the priority order

iptables -nvL PREROUTING -t mangle --line will show the mangle chain for routing rules


A more robust solution is in the works and I may want to recruit you as a beta tester.

Hello Xentrk,

Doesn't matter, I hope you're feeling better now.

First I've manually assigned static ip addresses in the DHCP Server list for some devices.
Each device is listed individually in the vpn routing-rules as showing in the pic below (only one client visible on the picture):

When I run the command:

ip rule:
0: from all lookup local
9990: from all fwmark 0x7000/0x7000 lookup main
10101: from 10.54.1.98 lookup ovpnc1
10102: from 10.54.1.210 lookup ovpnc1
10103: from 10.54.1.213 lookup ovpnc1
10301: from 10.54.1.99 lookup ovpnc2
10302: from 10.54.1.209 lookup ovpnc2
10303: from 10.54.1.201 lookup ovpnc2
10304: from 10.54.1.200 lookup ovpnc2
32766: from all lookup main
32767: from all lookup default

iptables -nvL PREROUTING -t mangle --line
Chain PREROUTING (policy ACCEPT 2674 packets, 560K bytes)
num pkts bytes target prot opt in out source destination
1 71116 81M MARK all -- tun12 * 0.0.0.0/0 0.0.0.0/0 MARK xset 0x1/0x7
2 3405K 4450M MARK all -- tun11 * 0.0.0.0/0 0.0.0.0/0 MARK xset 0x1/0x7
3 0 0 MARK all -- tun21 * 0.0.0.0/0 0.0.0.0/0 MARK xset 0x1/0x7
4 3496K 4819M BWDPI_FILTER udp -- eth0 * 0.0.0.0/0 0.0.0.0/0
5 0 0 MARK tcp -- br0 * 0.0.0.0/0 0.0.0.0/0 match-set NETFLIX dst,dst MARK or 0x7000
6 131 33266 MARK tcp -- br0 * 0.0.0.0/0 0.0.0.0/0 match-set AMAZONAWS dst,dst MARK or 0x7000

And of course I'll help you as a beta tester.


kr.,
Patrick

 

[Xentrk]

Hello Xentrk,

Doesn't matter, I hope you're feeling better now.

First I've manually assigned static ip addresses in the DHCP Server list for some devices.
Each device is listed individually in the vpn routing-rules as showing in the pic below (only one client visible on the picture):
View attachment 13960

When I run the command:

ip rule:
0: from all lookup local
9990: from all fwmark 0x7000/0x7000 lookup main
10101: from 10.54.1.98 lookup ovpnc1
10102: from 10.54.1.210 lookup ovpnc1
10103: from 10.54.1.213 lookup ovpnc1
10301: from 10.54.1.99 lookup ovpnc2
10302: from 10.54.1.209 lookup ovpnc2
10303: from 10.54.1.201 lookup ovpnc2
10304: from 10.54.1.200 lookup ovpnc2
32766: from all lookup main
32767: from all lookup default

iptables -nvL PREROUTING -t mangle --line
Chain PREROUTING (policy ACCEPT 2674 packets, 560K bytes)
num pkts bytes target prot opt in out source destination
1 71116 81M MARK all -- tun12 * 0.0.0.0/0 0.0.0.0/0 MARK xset 0x1/0x7
2 3405K 4450M MARK all -- tun11 * 0.0.0.0/0 0.0.0.0/0 MARK xset 0x1/0x7
3 0 0 MARK all -- tun21 * 0.0.0.0/0 0.0.0.0/0 MARK xset 0x1/0x7
4 3496K 4819M BWDPI_FILTER udp -- eth0 * 0.0.0.0/0 0.0.0.0/0
5 0 0 MARK tcp -- br0 * 0.0.0.0/0 0.0.0.0/0 match-set NETFLIX dst,dst MARK or 0x7000
6 131 33266 MARK tcp -- br0 * 0.0.0.0/0 0.0.0.0/0 match-set AMAZONAWS dst,dst MARK or 0x7000

And of course I'll help you as a beta tester.


kr.,
Patrick

Is it working now? It currently shows zero pkts and bytes for the NETFLIX iptables chain. Check that the ipset list NETFLIX is populated (ipset -L NETFLIX | grep entries).

Also, I see you use AB-Solution. What to you have Accept DNS Configuration set to? Just wondered if you have encountered the issue in the spoiler

Spoiler: DNSMASQ and DNS when using OpenVPN Client

AB-Solution is the ad blocking solution for Asus routers using Asuswrt-Merin firmware. AB-Solution requires DNSmasq to work properly. With Asuswrt-Merlin firmware, OpenVPN clients use the VPN tunnel’s DNS. As a result, AB-Solution will not work for LAN clients connected to the VPN tunnel when using Policy Rules since DNSmasq is by-passed. AB-Solution will still work for devices connected to the WAN though.

John9547 LTS fork has implemented DNS differently than Asuswrt-Merlin. The DNS rules are reversed. With Accept DNS Configuration set to Exclusive, the VPN clients will use DNSmasq and AB-Solution will work. There is also a check box on how you want to handle the WAN clients. If you leave it unchecked, the WAN clients will also use the VPN DNS servers (but not the tunnel) and they can use AB-Solution. If you check the box, the WAN client requests are sent directly to the WAN DNS servers and AB-Solution will not be available.

To resolve the DNS and routing issues when using Policy Rules with Asuswrt-Merlin, set Accept DNS Configuration to “Strict” and specify the DNS server for the VPN tunnel to use by adding the dhcp-option DNS command in the Custom Configuration section. Without the dhcp-option command, AB-Solution updates will fail, the AB-Solution email function will no longer work and the wget command will not able to resolve the domain name. The downside with these settings is that DNS will leak. Having my DNS leak has not caused me any issues for my use case.

 

[Patje]

Is it working now? It currently shows zero pkts and bytes for the NETFLIX iptables chain. Check that the ipset list NETFLIX is populated (ipset -L NETFLIX | grep entries).

Also, I see you use AB-Solution. What to you have Accept DNS Configuration set to? Just wondered if you have encountered the issue in the spoiler

Spoiler: DNSMASQ and DNS when using OpenVPN Client

AB-Solution is the ad blocking solution for Asus routers using Asuswrt-Merin firmware. AB-Solution requires DNSmasq to work properly. With Asuswrt-Merlin firmware, OpenVPN clients use the VPN tunnel’s DNS. As a result, AB-Solution will not work for LAN clients connected to the VPN tunnel when using Policy Rules since DNSmasq is by-passed. AB-Solution will still work for devices connected to the WAN though.

John9547 LTS fork has implemented DNS differently than Asuswrt-Merlin. The DNS rules are reversed. With Accept DNS Configuration set to Exclusive, the VPN clients will use DNSmasq and AB-Solution will work. There is also a check box on how you want to handle the WAN clients. If you leave it unchecked, the WAN clients will also use the VPN DNS servers (but not the tunnel) and they can use AB-Solution. If you check the box, the WAN client requests are sent directly to the WAN DNS servers and AB-Solution will not be available.

To resolve the DNS and routing issues when using Policy Rules with Asuswrt-Merlin, set Accept DNS Configuration to “Strict” and specify the DNS server for the VPN tunnel to use by adding the dhcp-option DNS command in the Custom Configuration section. Without the dhcp-option command, AB-Solution updates will fail, the AB-Solution email function will no longer work and the wget command will not able to resolve the domain name. The downside with these settings is that DNS will leak. Having my DNS leak has not caused me any issues for my use case.

At the moment of the commands, Kodi was bypassing my VPN.

I'm at work now, so I can't check anything at the moment.
But I know for sure that Accept DNS Configuration is set to Exclusive.


Verzonden vanaf mijn iPhone met Tapatalk

 

[Patje]

Is it working now? It currently shows zero pkts and bytes for the NETFLIX iptables chain. Check that the ipset list NETFLIX is populated (ipset -L NETFLIX | grep entries).

Also, I see you use AB-Solution. What to you have Accept DNS Configuration set to? Just wondered if you have encountered the issue in the spoiler

Spoiler: DNSMASQ and DNS when using OpenVPN Client

AB-Solution is the ad blocking solution for Asus routers using Asuswrt-Merin firmware. AB-Solution requires DNSmasq to work properly. With Asuswrt-Merlin firmware, OpenVPN clients use the VPN tunnel’s DNS. As a result, AB-Solution will not work for LAN clients connected to the VPN tunnel when using Policy Rules since DNSmasq is by-passed. AB-Solution will still work for devices connected to the WAN though.

John9547 LTS fork has implemented DNS differently than Asuswrt-Merlin. The DNS rules are reversed. With Accept DNS Configuration set to Exclusive, the VPN clients will use DNSmasq and AB-Solution will work. There is also a check box on how you want to handle the WAN clients. If you leave it unchecked, the WAN clients will also use the VPN DNS servers (but not the tunnel) and they can use AB-Solution. If you check the box, the WAN client requests are sent directly to the WAN DNS servers and AB-Solution will not be available.

To resolve the DNS and routing issues when using Policy Rules with Asuswrt-Merlin, set Accept DNS Configuration to “Strict” and specify the DNS server for the VPN tunnel to use by adding the dhcp-option DNS command in the Custom Configuration section. Without the dhcp-option command, AB-Solution updates will fail, the AB-Solution email function will no longer work and the wget command will not able to resolve the domain name. The downside with these settings is that DNS will leak. Having my DNS leak has not caused me any issues for my use case.

Now, my devices bypassing my vpn after running the script.

Spoiler

ipset -L NETFLIX | grep entries
Number of entries: 108

ip rule
0: from all lookup local
9990: from all fwmark 0x7000/0x7000 lookup main
10101: from 10.54.1.98 lookup ovpnc1
10102: from 10.54.1.210 lookup ovpnc1
10103: from 10.54.1.213 lookup ovpnc1
10301: from 10.54.1.99 lookup ovpnc2
10302: from 10.54.1.209 lookup ovpnc2
10303: from 10.54.1.201 lookup ovpnc2
10304: from 10.54.1.200 lookup ovpnc2
32766: from all lookup main
32767: from all lookup default

iptables -nvL PREROUTING -t mangle --line

Chain PREROUTING (policy ACCEPT 9906 packets, 2409K bytes)
num pkts bytes target prot opt in out source destination
1 416K 553M MARK all -- tun12 * 0.0.0.0/0 0.0.0.0/0 MARK xset 0x1/0x7
2 3852K 5050M MARK all -- tun11 * 0.0.0.0/0 0.0.0.0/0 MARK xset 0x1/0x7
3 0 0 MARK all -- tun21 * 0.0.0.0/0 0.0.0.0/0 MARK xset 0x1/0x7
4 4296K 5953M BWDPI_FILTER udp -- eth0 * 0.0.0.0/0 0.0.0.0/0
5 0 0 MARK tcp -- br0 * 0.0.0.0/0 0.0.0.0/0 match-set NETFLIX dst,dst MARK or 0x7000
6 527 175K MARK tcp -- br0 * 0.0.0.0/0 0.0.0.0/0 match-set AMAZONAWS dst,dst MARK or 0x7000

 

[Xentrk]

Now, my devices bypassing my vpn after running the script.

Spoiler

ipset -L NETFLIX | grep entries
Number of entries: 108

ip rule
0: from all lookup local
9990: from all fwmark 0x7000/0x7000 lookup main
10101: from 10.54.1.98 lookup ovpnc1
10102: from 10.54.1.210 lookup ovpnc1
10103: from 10.54.1.213 lookup ovpnc1
10301: from 10.54.1.99 lookup ovpnc2
10302: from 10.54.1.209 lookup ovpnc2
10303: from 10.54.1.201 lookup ovpnc2
10304: from 10.54.1.200 lookup ovpnc2
32766: from all lookup main
32767: from all lookup default

iptables -nvL PREROUTING -t mangle --line

Chain PREROUTING (policy ACCEPT 9906 packets, 2409K bytes)
num pkts bytes target prot opt in out source destination
1 416K 553M MARK all -- tun12 * 0.0.0.0/0 0.0.0.0/0 MARK xset 0x1/0x7
2 3852K 5050M MARK all -- tun11 * 0.0.0.0/0 0.0.0.0/0 MARK xset 0x1/0x7
3 0 0 MARK all -- tun21 * 0.0.0.0/0 0.0.0.0/0 MARK xset 0x1/0x7
4 4296K 5953M BWDPI_FILTER udp -- eth0 * 0.0.0.0/0 0.0.0.0/0
5 0 0 MARK tcp -- br0 * 0.0.0.0/0 0.0.0.0/0 match-set NETFLIX dst,dst MARK or 0x7000
6 527 175K MARK tcp -- br0 * 0.0.0.0/0 0.0.0.0/0 match-set AMAZONAWS dst,dst MARK or 0x7000

In my Fire TV, there is a app called my.I.P. I use to verify if the device is routing to the VPN or WAN. Are you using something similar?

I don't have the BWDPI_FILTER entry when I issue the iptables -nvL PREROUTING -t mangle --line command. I did a search and see a post from the forum Selective Routing expert @Martineau here:

BWDPI_FILTER Adaptive Qos->Qos ('DPI Engine' when Qos Type=Adaptive Qos)

Try turning off Adaptive Qos to see if it changes behavior.

 

[Patje]

In my Fire TV, there is a app called my.I.P. I use to verify if the device is routing to the VPN or WAN. Are you using something similar?

I don't have the BWDPI_FILTER entry when I issue the iptables -nvL PREROUTING -t mangle --line command. I did a search and see a post from the forum Selective Routing expert @Martineau here:

BWDPI_FILTER Adaptive Qos->Qos ('DPI Engine' when Qos Type=Adaptive Qos)

Try turning off Adaptive Qos to see if it changes behavior.

I've different devices:

1st- my laptop with Kodi installed (for this test)
When I check my IP in a webbrowser (https://ipleak.net) everything goes trough my vpn. When I check my IP in Kodi (with Indigo) it shows my ISP IP adress (after rebooting the router it shows my VPN address, after running the script it shows my ISP address).
2nd- a RPi with Kodi
After rebooting the router it shows my VPN address in Kodi, after running the script it shows my ISP address).
3rd- a small OMV nas with a PMS in a docker
My Public IP is changing from my VPN IP to my ISP IP when running the script.

Spoiler: PMS

This is a screenshot I took from internet!!

4th- Nvidia Shield tv
when I check my address within Puffin webbrowser I see my VPN IP, within Kodi it shows my ISP IP

Changing QOS from Adaptive to Original doesn't change anything, completely disable QOS and rebooting the router doesn't also change anything.

 

[Xentrk]

I've different devices:

1st- my laptop with Kodi installed (for this test)
When I check my IP in a webbrowser (https://ipleak.net) everything goes trough my vpn. When I check my IP in Kodi (with Indigo) it shows my ISP IP adress (after rebooting the router it shows my VPN address, after running the script it shows my ISP address).
2nd- a RPi with Kodi
After rebooting the router it shows my VPN address in Kodi, after running the script it shows my ISP address).
3rd- a small OMV nas with a PMS in a docker
My Public IP is changing from my VPN IP to my ISP IP when running the script.

Spoiler: PMS

This is a screenshot I took from internet!!

4th- Nvidia Shield tv
when I check my address within Puffin webbrowser I see my VPN IP, within Kodi it shows my ISP IP

Changing QOS from Adaptive to Original doesn't change anything, completely disable QOS and rebooting the router doesn't also change anything.

Since you turned off QoS and there was no change, that tells me there is no conflict with the fwmark and bitmask combo used by the script with those used by QoS.

I suspect the routing of the Amazon AWS IP addresses to the WAN may be contributing to the issues with kodi.

After running the script, type ipset flush AMAZONAWS at the command line to flush the ipset list. This way, Amazon AWS traffic will no longer go thru WAN. It will not go thru the VPN. Then see if you still have problems with Kodi or Plex.

 

[Patje]

Since you turned off QoS and there was no change, that tells me there is no conflict with the fwmark and bitmask combo used by the script with those used by QoS.

I suspect the routing of the Amazon AWS IP addresses to the WAN may be contributing to the issues with kodi.

After running the script, type ipset flush AMAZONAWS at the command line to flush the ipset list. This way, Amazon AWS traffic will no longer go thru WAN. It will not go thru the VPN. Then see if you still have problems with Kodi or Plex.

Okay,

Try it when I'm at home tonight, I'll let you know the results.

 

[Xentrk]

Okay,

Try it when I'm at home tonight, I'll let you know the results.

If you have good luck with the test above, lets' try this.

I have found that many streaming services use Content Deliver Network (CDN) to reduce buffering. In that case, I have had good look gathering the ip addresses used by CDN using the ipset method in dnsmasq. And if we are lucky, we will only get the AmazonAWS addresses Netflix requires rather than the entire enterprise.

You can copy and paste the code below into a command line and run it. Go to the browser and generate traffic on the netflix website. IP addresses used by netflix and CDN will be placed in the NETFLIX_WEB ipset list and direct that traffic to the WAN interface.

printf "ipset=/netflix.com/www.netflix.com/NETFLIX_WEB\\n" >> /jffs/configs/dnsmasq.conf.add
service restart_dnsmasq
ipset create NETFLIX_WEB hash:net family inet hashsize 1024 maxelem 65536
iptables -t mangle -D PREROUTING -i br0 -p tcp -m set --match-set NETFLIX_WEB dst,dst -j MARK --set-mark 0x7000/0x7000 > /dev/null 2>&1
iptables -t mangle -A PREROUTING -i br0 -p tcp -m set --match-set NETFLIX_WEB dst,dst -j MARK --set-mark 0x7000/0x7000

Then, try on your devices. In this test, make sure the AMAZONAWS ipset list is empty.

 

[Patje]

If you have good luck with the test above, lets' try this.

I have found that many streaming services use Content Deliver Network (CDN) to reduce buffering. In that case, I have had good look gathering the ip addresses used by CDN using the ipset method in dnsmasq. And if we are lucky, we will only get the AmazonAWS addresses Netflix requires rather than the entire enterprise.

You can copy and paste the code below into a command line and run it. Go to the browser and generate traffic on the netflix website. IP addresses used by netflix and CDN will be placed in the NETFLIX_WEB ipset list and direct that traffic to the WAN interface.

printf "ipset=/netflix.com/www.netflix.com/NETFLIX_WEB\\n" >> /jffs/configs/dnsmasq.conf.add
service restart_dnsmasq
ipset create NETFLIX_WEB hash:net family inet hashsize 1024 maxelem 65536
iptables -t mangle -D PREROUTING -i br0 -p tcp -m set --match-set NETFLIX_WEB dst,dst -j MARK --set-mark 0x7000/0x7000 > /dev/null 2>&1
iptables -t mangle -A PREROUTING -i br0 -p tcp -m set --match-set NETFLIX_WEB dst,dst -j MARK --set-mark 0x7000/0x7000

Then, try on your devices. In this test, make sure the AMAZONAWS ipset list is empty.

When I run the script and flush AMAZONAWS (ipset flush AMAZONAWS), I get my VPN IP's in Kodi and Plex but Netflix isn't working anymore. When I run te code afterwards, nothing changes.

 

[Xentrk]

When I run the script and flush AMAZONAWS (ipset flush AMAZONAWS), I get my VPN IP's in Kodi and Plex but Netflix isn't working anymore. When I run te code afterwards, nothing changes.

Thanks for the feedback @Patje. I will have to fire up Kodi and Plex on my end and to see what domains they are querying and perform analaysis to understand the source of the conflict. It appears the sending AmazonAWS IPs to the WAN iface is causing the problem. I'll get back to you after I have completed the analysis.

 

[Xentrk]

@Patje

I created a test-branch on github. In addition to a lot of code tuning, the main update is restricting the Amazon AWS ip addresses to the US Region. All of the other Amazon AWS regions will now go thru the VPN. Netflix is still bypassing the VPN tunnel in my testing.

Copy and Paste on the command line

/usr/sbin/curl --retry 3  "https://raw.githubusercontent.com/Xentrk/netflix-vpn-bypass/test-branch/IPSET_Netflix.sh" -o "/jffs/scripts/IPSET_Netflix.sh" && chmod 755 "/jffs/scripts/IPSET_Netflix.sh"

Bonus:
Copy and paste this code inside the /jffs/configs/profile.add file.

Spoiler: liststats

liststats () {
GREEN='\033[0;32m'
RED='\033[0;31m'
NC='\033[0m' # No Color
true > /tmp/ipsetlist
for SETLIST in $(ipset -L | grep "Name:" | sed 's/Name: //')
do
ENTRIES=$(ipset -L "$SETLIST" | grep "entries:" | sed 's/Number of entries: //')
printf '%s %b%s%b\n' "$SETLIST" "$GREEN" "$ENTRIES" "$NC" >> /tmp/ipsetlist
done
cat /tmp/ipsetlist | sort -u
rm /tmp/ipsetlist
}

Then, open up a new SSH session and type the command liststats

Skynet-Blacklist 109200
Skynet-BlockedRanges 1701
Skynet-Master 2
Skynet-Whitelist 1080
x3mRouting_AMAZONAWS_US 282
x3mRouting_NETFLIX 106

If you still have issues after testing, I will need to analyze the domains kodi and plex are calling which may take some time.

If it works, I'll promote the version to the master branch.

 

[Patje]

@Patje

I created a test-branch on github. In addition to a lot of code tuning, the main update is restricting the Amazon AWS ip addresses to the US Region. All of the other Amazon AWS regions will now go thru the VPN. Netflix is still bypassing the VPN tunnel in my testing.

Copy and Paste on the command line

/usr/sbin/curl --retry 3  "https://raw.githubusercontent.com/Xentrk/netflix-vpn-bypass/test-branch/IPSET_Netflix.sh" -o "/jffs/scripts/IPSET_Netflix.sh" && chmod 755 "/jffs/scripts/IPSET_Netflix.sh"

Bonus:
Copy and paste this code inside the /jffs/configs/profile.add file.

Spoiler: liststats

liststats () {
GREEN='\033[0;32m'
RED='\033[0;31m'
NC='\033[0m' # No Color
true > /tmp/ipsetlist
for SETLIST in $(ipset -L | grep "Name:" | sed 's/Name: //')
do
ENTRIES=$(ipset -L "$SETLIST" | grep "entries:" | sed 's/Number of entries: //')
printf '%s %b%s%b\n' "$SETLIST" "$GREEN" "$ENTRIES" "$NC" >> /tmp/ipsetlist
done
cat /tmp/ipsetlist | sort -u
rm /tmp/ipsetlist
}

Then, open up a new SSH session and type the command liststats

Skynet-Blacklist 109200
Skynet-BlockedRanges 1701
Skynet-Master 2
Skynet-Whitelist 1080
x3mRouting_AMAZONAWS_US 282
x3mRouting_NETFLIX 106

If you still have issues after testing, I will need to analyze the domains kodi and plex are calling which may take some time.

If it works, I'll promote the version to the master branch.

I tested the script from the test branche, but Kodi and Plex are showing my ISP SP an Netflix isn't working anymore.

/jffs/configs/profile.add This file doesn't exist on my router. I created the file and copied the tekst in it.

liststats gives me:
x3mRouting_AMAZONAWS_US 282
x3mRouting_NETFLIX 106

btw I don't use Netflix US region

 

[Xentrk]

I tested the script from the test branche, but Kodi and Plex are showing my ISP SP an Netflix isn't working anymore.

/jffs/configs/profile.add This file doesn't exist on my router. I created the file and copied the tekst in it.

liststats gives me:
x3mRouting_AMAZONAWS_US 282
x3mRouting_NETFLIX 106

btw I don't use Netflix US region

Thanks for the feedback. Looks like I'll need to research the Kodi and Plex relationship with Amazon AWS.

 

[Xentrk]

@Patje,
I started looking into the issue yesterday. Plex is using Amazon AWS ip addresses.

# nslookup plex.tv
Server:    127.0.0.1
Address 1: 127.0.0.1 localhost.localdomain

Name:      plex.tv
Address 1: 34.246.147.206 ec2-34-246-147-206.eu-west-1.compute.amazonaws.com
Address 2: 52.211.233.249 ec2-52-211-233-249.eu-west-1.compute.amazonaws.com
Address 3: 34.249.151.238 ec2-34-249-151-238.eu-west-1.compute.amazonaws.com
Address 4: 54.77.129.25 ec2-54-77-129-25.eu-west-1.compute.amazonaws.com
Address 5: 34.248.194.188 ec2-34-248-194-188.eu-west-1.compute.amazonaws.com
Address 6: 52.210.134.93 ec2-52-210-134-93.eu-west-1.compute.amazonaws.com
Address 7: 54.77.87.2 ec2-54-77-87-2.eu-west-1.compute.amazonaws.com
Address 8: 52.17.130.55 ec2-52-17-130-55.eu-west-1.compute.amazonaws.com

################################################################
/jffs/scripts# drill plex.tv
;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 58624
;; flags: qr rd ra ; QUERY: 1, ANSWER: 8, AUTHORITY: 4, ADDITIONAL: 8
;; QUESTION SECTION:
;; plex.tv.     IN      A

;; ANSWER SECTION:
plex.tv.        60      IN      A       52.211.233.249
plex.tv.        60      IN      A       52.17.130.55
plex.tv.        60      IN      A       54.77.129.25
plex.tv.        60      IN      A       54.77.87.2
plex.tv.        60      IN      A       34.249.151.238
plex.tv.        60      IN      A       34.246.147.206
plex.tv.        60      IN      A       52.210.134.93
plex.tv.        60      IN      A       52.208.108.2

;; AUTHORITY SECTION:
plex.tv.        142187  IN      NS      ns-1566.awsdns-03.co.uk.
plex.tv.        142187  IN      NS      ns-282.awsdns-35.com.
plex.tv.        142187  IN      NS      ns-1342.awsdns-39.org.
plex.tv.        142187  IN      NS      ns-1005.awsdns-61.net.

;; ADDITIONAL SECTION:
ns-1005.awsdns-61.net.  122687  IN      A       205.251.195.237
ns-1005.awsdns-61.net.  122687  IN      AAAA    2600:9000:5303:ed00::1
ns-1342.awsdns-39.org.  133401  IN      A       205.251.197.62
ns-1342.awsdns-39.org.  133401  IN      AAAA    2600:9000:5305:3e00::1
ns-1566.awsdns-03.co.uk.        148222  IN      A       205.251.198.30
ns-1566.awsdns-03.co.uk.        148222  IN      AAAA    2600:9000:5306:1e00::1
ns-282.awsdns-35.com.   17790   IN      A       205.251.193.26
ns-282.awsdns-35.com.   17790   IN      AAAA    2600:9000:5301:1a00::1

;; Query time: 265 msec
;; SERVER: 127.0.0.1
;; WHEN: Wed Aug 15 02:41:34 2018
;; MSG SIZE  rcvd: 470

Looking at the nslookup results, Plex is using the Amazon AWS in EU, which is located in Ireland. I would have expected the code in the github test-branch to work for you since it only pulled the Amazon AWS located in the US.

Using the ASN method for Netflix and AmazonAWS appears to be casting to wide of a net. I was able to capture domains that work using the ipset method in dnsmasq:

ipset=/amazonaws.com/netflix.net/nflxvideo.net/netfliximg.net/nflxext.com/nffxso.net/netflix.com/netflix.com/x3mRouting_NETFLIX_WEB

I tried to limit my surfing to Netflix why mining the domains in an attempt to focus on just the AmazonAWS domains used by Netflix.

I checked against the ipset list this created and do not see the plex.tv IP addresses listed. List is here https://pastebin.com/yZyWbFhQ

I need to ponder the approach. If we use the ipset list I generated, it may not work for those in EU. I have a theory that if a user is in the EU, Netflix may use the AmazonAWS data center in Ireland rather than directing traffic across the pond to Amazon AWS in Oregon. I hope some of the EU users on the thread who use Netflix can look at their dnsmasq.log file and confirm my theory. CDN come into play as well, as noted in an earlier post in this post.

I may just have to teach users how to fish. In other words, how to create their own ipset list for Netflix as the list may vary by geo-location.

I wrote a program to lookup the information on the domains I mined using whob. I pasted the results here if you want to take a peek.

https://pastebin.com/q5BADU60

I can work to pare the list to just the prefixes e.g. Prefix: 184.73.0.0/16. I am so amazed at how fast ipset is.

Netflix sent me some emails noticing that I had logged on from Thailand as I usually stream using my VPN Provider TorGuard. https://x3mtek.com/why-i-use-torguard-as-my-vpn-provider/ One thing I noticed is that it captured the outgoing gateway in Thailand. I removed it from the https://pastebin.com/q5BADU60 pastebin post.

 

[Xentrk]

@Patje
You can try the ipset method again using the additional domains:

Turn off the VPN so your traffic goes thru the WAN.

Copy and past the commands below in an SSH:

printf "ipset=/amazonaws.com/netflix.com/netflix.net/nflxvideo.net/nflximg.net/nflxext.com/nflxso.net/x3mRouting_NETFLIX_WEB\n" >> /jffs/configs/dnsmasq.conf.add
service restart_dnsmasq
ipset create NETFLIX_WEB hash:net family inet hashsize 1024 maxelem 65536

You can do nslookups on the Netflix domains but not the amazonaws domain. e.g.

 nslookup netflix.com

Then, startup Netflix in a browser or on TV and navigate around.

Turn the VPN back on. Make sure the streaming media device is assigned to use the VPN tunnel with Policy Rules or Policy Rules (Strict).

Run these commands to route the Netflix traffic to WAN.

iptables -t mangle -D PREROUTING -i br0 -p tcp -m set --match-set x3mRouting_NETFLIX_WEB dst,dst -j MARK --set-mark 0x7000/0x7000 > /dev/null 2>&1
iptables -t mangle -A PREROUTING -i br0 -p tcp -m set --match-set x3mRouting_NETFLIX_WEB dst,dst -j MARK --set-mark 0x7000/0x7000

Let me know if Netflix is now bypassing the VPN tunnel.

If it does, save the ipset list: ipset save x3mRouting_NETFLIX_WEB > /jffs/scripts/netflix_web.

Comment out the entry created in dnsmasq.conf.add to prevent amazonaws addresses not associated with Netflix from being added to the ipset list and restart dnsmasq.

I can provide additional instructions once you let me know the result.

I would also like you to try the ipset list I posted on pastebin. Rather than creating the list dynamacilly from the dnsmasq.conf.entry, create the list from the file. Take the pastebin file and download to /jffs/scripts/netflix_web. Enter the command to create and load the list:

ipset restore < /jffs/scripts/netflix_web

Then, run the iptables commands and test. If this works, then I may change the code to use this list rather than the ASN method.

 

[Xentrk]

Sigh. I converted the ipset list I generated yesterday to use prefixes. When I went to test it did not work. I then reverted to the ipset list and it also no longer worked. I re-enabled dnsmasq to capture the ipv4 addresses and dynamically add to the ipset list. it started working again.

I found a list that is very recent.

https://github.com/black-cerberon/Route-OpenVPN/blob/master/media/netflix-openvpn.conf

I will test it tomorrow.

 

[Patje]

Sigh. I converted the ipset list I generated yesterday to use prefixes. When I went to test it did not work. I then reverted to the ipset list and it also no longer worked. I re-enabled dnsmasq to capture the ipv4 addresses and dynamically add to the ipset list. it started working again.

I found a list that is very recent.

https://github.com/black-cerberon/Route-OpenVPN/blob/master/media/netflix-openvpn.conf

I will test it tomorrow.

Hello Xentrk,

Do you still want me to test your previous post?


Verzonden vanaf mijn iPhone met Tapatalk

 

[Xentrk]

Hello Xentrk,

Do you still want me to test your previous post?


Verzonden vanaf mijn iPhone met Tapatalk

If you have time, please test the method in post 217.. It will help with the analysis. The ipset method in dnsmasq.conf.add works when active as it builds the list dynamically.