Selective Routing for Netflix

[Xentrk]

@Patje,
Here is an update on the domains

/jffs/scripts/Patje# grep "Origin-AS:" /jffs/scripts/Patje/NETFLIX_DOMAIN_DETAILS.txt | sort -u

Origin-AS: 14618
Origin-AS: 16509
Origin-AS: 2906
Origin-AS: 40027
Origin-AS: 45758

What is interesting is how Hurricane Electric reports Akamai AS45758 as my local ISP . It has to do with how CDN works.

Origin-AS: 14618 https://bgp.he.net/AS14618#_prefixes AS14618 Amazon.com, Inc.
Origin-AS: 16509 https://bgp.he.net/AS16509 AS16509 Amazon.com, Inc.
Origin-AS: 2906 https://bgp.he.net/AS2906 AS2906 Netflix Streaming Services Inc.
Origin-AS: 40027 https://bgp.he.net/AS40027#_prefixesAS40027 Netflix Streaming Services Inc.
Origin-AS: 45758 https://bgp.he.net/AS45758 Akamai

With this information, I could revised the original script to pull the IPv4 addresses for the additional ASN numbers. Let's see what happens after you create a new Netflix IPSET list from the code above using the domain names you mined first.

 

[trubio]

Thanks for your work. I don't have the compatible equipment but I was able to use some of your code to make my ip list to run in windows. It works great thanks

 

[Xentrk]

Thanks for your work. I don't have the compatible equipment but I was able to use some of your code to make my ip list to run in windows. It works great thanks

Thank you for the encouragement! @Martineau has been kind enough to share his knowledge with me and others on the forum how Selective Routing works on Asuswrt-Merlin. Just trying to pass on what I have learned to help others. A more robust and flexible solution is in progress. Glad this has been of some help and that you got it working!

 

[Xentrk]

The project has been updated with new code and documentation. All project documentation and install instructions are available on github.com.

There are two selective routing scripts used in this project. Each one uses a different method to collect the IPv4 addresses required for selective routing. Both scripts use the features of IPSET to collect IPv4 addresses in IPSET lists and match against the IPSET lists. Users can select the script that works best for their environment.

IPSET_Netflix.sh collects the IPv4 addresses used by Netflix from https://ipinfo.io using the Autonomous System Number (ASN) assigned to Netflix. Amazon AWS supplies the list of IPv4 addresses in the json file at https://ip-ranges.amazonaws.com/ip-ranges.json

Only the Amazon AWS US Regions are extracted from ip-ranges.json. As a result, the script will also route all Amazon AWS traffic bound for the US, including Amazon Prime traffic, to the WAN interface.

IPSET_Netflix_Domains.sh uses the IPSET feature built into dnsmasq to dynamically generate the IPv4 address used by Netflix and Amazon AWS dynamically. This approach can be useful when your ISP is using the Netflix Open Connect Network.

 

[Patje]

The project has been updated with new code and documentation. All project documentation and install instructions are available on github.com.

There are two selective routing scripts used in this project. Each one uses a different method to collect the IPv4 addresses required for selective routing. Both scripts use the features of IPSET to collect IPv4 addresses in IPSET lists and match against the IPSET lists. Users can select the script that works best for their environment.

IPSET_Netflix.sh collects the IPv4 addresses used by Netflix from https://ipinfo.io using the Autonomous System Number (ASN) assigned to Netflix. Amazon AWS supplies the list of IPv4 addresses in the json file at https://ip-ranges.amazonaws.com/ip-ranges.json

Only the Amazon AWS US Regions are extracted from ip-ranges.json. As a result, the script will also route all Amazon AWS traffic bound for the US, including Amazon Prime traffic, to the WAN interface.

IPSET_Netflix_Domains.sh uses the IPSET feature built into dnsmasq to dynamically generate the IPv4 address used by Netflix and Amazon AWS dynamically. This approach can be useful when your ISP is using the Netflix Open Connect Network.

Hi Xentrk,

It are some very busy weeks, I’ll try things ASAP.

Kr.,
Patje


Verzonden vanaf mijn iPhone met Tapatalk

 

[Xentrk]

Hi Xentrk,

It are some very busy weeks, I’ll try things ASAP.

Kr.,
Patje


Verzonden vanaf mijn iPhone met Tapatalk

Thank you. I appreciate it. Hoping one of the methods will solve your issues. I would like feedback on the project documentation (README.md) too. Let me know if any of the documentation lacks clarity or if you spot any spelling or grammar issues.

 

[Patje]

Thank you. I appreciate it. Hoping one of the methods will solve your issues. I would like feedback on the project documentation (README.md) too. Let me know if any of the documentation lacks clarity or if you spot any spelling or grammar issues.

Hi Xentrk,

With the first option I haven't succes.
The second option is the option that works for me, finally

Thanks for all your work

kr.,
Patrick

 

[Xentrk]

Hi Xentrk,

With the first option I haven't succes.
The second option is the option that works for me, finally

Thanks for all your work

kr.,
Patrick

Great news! I was hoping the ipset feature of dnsmasq would work for you. @Martineau was helping me recently and we uncovered an issue with the fwmark/bitmask values and priorities. This version has the fix for the issue. I suspect that may have been why the previous version I posted in the test-branch did not work for you.

For some streaming media sites, I use a combination of the ASN and ipset= inside of dnsmasq method. On some sites, I can get by just using the ASN method if I am watching on my media player (e.g. Roku). But once I try to watch from a web browser, I have to add the ipset=/ inside of dnsmasq. I will be posting a more comprehensive selective routing project next that will include other streaming services (BBC, Hulu, CBS and SlingTV)

 

[tejesh83]

@Xentrk

Thank you for posting these!

It worked perfectly for me. Like you, I had to use both the ASN and dnsmasq methods to get my FireTV device and browsers to work with Amazon Prime Video. I'm looking forward to your more comprehensive project.

 

[Xentrk]

@Xentrk

Thank you for posting these!

It worked perfectly for me. Like you, I had to use both the ASN and dnsmasq methods to get my FireTV device and browsers to work with Amazon Prime Video. I'm looking forward to your more comprehensive project.

Thank you for the feedback! I found in my testing that the ASN method worked fine for streaming media device like Fire TV or Roku. However, when I tested streaming from a browser for some streaming media services, I had to also include the IPv4 addresses dynamically generated using the ipset feature built into dnsmasq to capture the additional domains.

The project has taken some twist and turns in the approach. I put it on temporary hold to work on getting Stubby to work on the firwmare. I plan to pick it up next week.

 

[Matt87]

This works perfectly!

Any chance we could see something for Amazon Prime Video?

 

[Xentrk]

This works perfectly!

Any chance we could see something for Amazon Prime Video?

Thank you for letting me know you had success with the code.

The script is already routing Amazon Prime Video to the WAN interface. Because Netflix hosts on Amazon AWS servers, the script also routes all Amazon AWS traffic bound for the US, including Amazon Prime Video traffic, to the WAN interface. This may or not be an issue for some people. If I don't include the Amazon AWS servers in the script, the Netflix bypass does not work.

However, if you are in another region like the EU, perhaps Amazon routes you to their data center in Ireland? In that case, I would need to specify EU region.

One way to verify is look in dnsmasq.log file and look at some of the domain names invoked when you watch Amazon. Then, do an nslookup on the domain name to get the IPv4 address. Then, use the entware package whob to lookup information on the IPv4 address, which will disclose the location.

#nslookup atv-ext.amazon.com
Server:    127.0.0.1
Address 1: 127.0.0.1 localhost.localdomain

Name:      atv-ext.amazon.com
Address 1: 52.94.232.249

# whob 52.94.232.249
IP: 52.94.232.249
Origin-AS: 16509
Prefix: 52.94.224.0/20
AS-Path: 20912 174 16509
AS-Org-Name: Amazon.com, Inc.
Org-Name: Amazon Technologies Inc.
Net-Name: AT-88-Z
Cache-Date: 1539505496
Latitude: 47.627500
Longitude: -122.346200
City: Seattle
Region: Washington
Country: United States
Country-Code: US

 

[Matt87]

I'm in the US, Maryland, specifically.

I sounded like a total noob there, huh. Sorta true...

Amazon Prime Video is still giving me the run around --> "Buy or rent now for $xx.xx" & "This title is not available in your location"

I know AWS servers are included in the first script. I'll have to play around some more with the second to see what's going on. Your tutorial set me on a quest to reconfigure a few things, so. Will let ya know when I figure it out.

Again, thanks!!! I've learned a lot from you!

 

[Xentrk]

I'm in the US, Maryland, specifically.

I sounded like a total noob there, huh. Sorta true...

Amazon Prime Video is still giving me the run around --> "Buy or rent now for $xx.xx" & "This title is not available in your location"

I know AWS servers are included in the first script. I'll have to play around some more with the second to see what's going on. Your tutorial set me on a quest to reconfigure a few things, so. Will let ya know when I figure it out.

Again, thanks!!! I've learned a lot from you!

Do you still have the issue when turning off the VPN client? Could be some CDN domains also need to be included. Could be that Amazon Prime is starting to clamp down on shared VPN servers like Netflix and Hulu. I can test on my end and let you know what I see.

 

[Matt87]

Do you still have the issue when turning off the VPN client? Could be some CDN domains also need to be included.

No I do not have this issue when the VPN client is off.

EDIT: That's what I was thinking, too, but unsure where to go from here.

New question, though!

Have VPN server running on the router.
Have firewall-start setup to allow pass-thru of the connected server clients to my VPN service provider (Martineau's response on "openvpn server and client question" thread).
Have policy rules setup to redirect all of the subnet traffic for server 1 (10.8.0.0/24) thru the VPN client.

Web surfing on the server client device is now covered by my router's VPN client and working great, but netflix on the device is not sending the requests over WAN and is giving me the proxy error. Netflix is working on WIFI connected devices though.

Not sure what I should look into next to resolve this. Pointers would be much appreciated, sir!

Thanks

 

[Xentrk]

I did a quick test on three shared vpn servers and get the geo restriction message from Amazon Prime. I have not noticed because I use a private VPN IP address. See https://x3mtek.com/why-i-use-torguard-as-my-vpn-provider/ for the details. We may need to use the technique on the GitHub repo to mine dnasmasq to see what other domains are being called when you use Amazon Prime. Appears there may be more domain names involved that are not accounted for in the US region for Amazon AWS. Just a suspicion for now.

 

[Xentrk]

No I do not have this issue when the VPN client is off.

EDIT: That's what I was thinking, too, but unsure where to go from here.

New question, though!

Have VPN server running on the router.
Have firewall-start setup to allow pass-thru of the connected server clients to my VPN service provider (Martineau's response on "openvpn server and client question" thread).
Have policy rules setup to redirect all of the subnet traffic for server 1 (10.8.0.0/24) thru the VPN client.

Web surfing on the server client device is now covered by my router's VPN client and working great, but netflix on the device is not sending the requests over WAN and is giving me the proxy error. Netflix is working on WIFI connected devices though.

Not sure what I should look into next to resolve this. Pointers would be much appreciated, sir!

Thanks

I can look into it tomorrow. Yes, @Martineau knowledge on the topic is superior. He has been a big help to me and others on the forum. Very Grateful that he is willing to help and share his knowledge.

 

[Matt87]

See https://x3mtek.com/why-i-use-torguard-as-my-vpn-provider/ for the details.

I got too cheap of a deal with PIA, and didn't do enough research into torguard. Wife would kill me if I popped for another VPN service provider. :-/

I can look into it tomorrow.

Much appreciated, sir!

 

[Matt87]

I can look into it tomorrow. Yes, @Martineau knowledge on the topic is superior. He has been a big help to me and others on the forum. Very Grateful that he is willing to help and share his knowledge.

Reconfigured my VPN server and server clients. Tested it on a public network today and everything worked on both clients. No clue man.

Still unable to access my NAS, but can ping it. Can access router and RDP into my desktop though. Will go to another thread for these issues.

Ty for your help! Will get back to you about the Prime Video Geo lock thing.

Thanks!

 

[Xentrk]

Reconfigured my VPN server and server clients. Tested it on a public network today and everything worked on both clients. No clue man.

Still unable to access my NAS, but can ping it. Can access router and RDP into my desktop though. Will go to another thread for these issues.

Ty for your help! Will get back to you about the Prime Video Geo lock thing.

Thanks!

Thanks for the update. I recall a recent thread on the forums about not being able to access the NAS when accessing home network using the VPN server connection.