x3mRouting x3mRouting ~ Selective Routing for Asuswrt-Merlin Firmware

[Xentrk]

First option give me this:
admin@RT-AC86U-8020:/tmp/home/root# sh /jffs/scripts/x3mRouting/load_ASN_ipset_i
face.sh 1 NETFLIX AS2906
(load_ASN_ipset_iface.sh): 19795 Starting Script Execution
(load_ASN_ipset_iface.sh): 19795 Selective Routing Rule via VPN Client 1 created for NETFLIX (TAG fwmark 0x1000/0x1000)
(load_ASN_ipset_iface.sh): 19795 Ending Script Execution
It looks like everything is going thru vpn. For now my best option is disable openvpn after i finish
watch netflix (its blocking my other streaming services hbogo and amazon)

Policy Rules needs to be set to "Strict". Amazon blocks known VPN servers too. See the README for how to configure for Amazon. Not sure if HBO blocks known VPN servers though.

 

[dojrude]

@Xentrk
I'm going to follow the same procedure for BBC, except route this over VPN Client 2.

I've been trying to do the same for the last week or so, but am struggling with the config.

I mined IP addresses for iPlayer and tried those and they work fine, but it seems that some websites then stop working - I assume I've included too much in the rules or the CDN providers are affecting traffic other than iPlayer.

The mined IPs were from a range of domains;

2x IPs - AKAMAI-AS, US
2x IPs - AKAMAI-ASN1, US
10x IPs - AMAZON-02, US
2x IPs - AMAZON-AES, US
2x IPs - BANDWIDTH-AS, GB
3x IPs - BBC BBC Internet Services, UK, GB
5x IPs - CLOUDFLARENET, US
2x IPs - GOOGLE, US
1x IP - HIGHWINDS3, US
1x IP - LLNW, US
10x IPs - MICROSOFT-CORP-MSN-AS-BLOCK, US
1x IP - SOFTLAYER, US
2x IPs - TEFINCOMSA-AS-AP TEFINCOM S.A., PA

Then I tried using the DNSMASQ scripts instead with domains I found on this and another page, but that doesn't appear to work at all and gives me the standard "you are not in the UK" error;

sh /jffs/scripts/x3mRouting/load_DNSMASQ_ipset.sh UKTV bbc.net.uk,bbc.co.uk,bbc.com,bbc.gscontxt.net,bbci.co.uk,ssl-bbcsmarttv.2cnt.net,itv.com,channel4.com,channel5.com,llnwd.net,edgefcs.net

Ideally I'd like to get iPlayer, ITV, Channel 4 and Chanel 5 all working via ipset tables if possible, as I don't want to force an entire device down the VPN and break local streaming.

Has anyone had any luck doing this recently or know of up to date ipset lists for these services as I notice the posts are from a couple of years ago that mention it.

Thanks.

 

[Xentrk]

I've been trying to do the same for the last week or so, but am struggling with the config.

I mined IP addresses for iPlayer and tried those and they work fine, but it seems that some websites then stop working - I assume I've included too much in the rules or the CDN providers are affecting traffic other than iPlayer.

The mined IPs were from a range of domains;

2x IPs - AKAMAI-AS, US
2x IPs - AKAMAI-ASN1, US
10x IPs - AMAZON-02, US
2x IPs - AMAZON-AES, US
2x IPs - BANDWIDTH-AS, GB
3x IPs - BBC BBC Internet Services, UK, GB
5x IPs - CLOUDFLARENET, US
2x IPs - GOOGLE, US
1x IP - HIGHWINDS3, US
1x IP - LLNW, US
10x IPs - MICROSOFT-CORP-MSN-AS-BLOCK, US
1x IP - SOFTLAYER, US
2x IPs - TEFINCOMSA-AS-AP TEFINCOM S.A., PA

Then I tried using the DNSMASQ scripts instead with domains I found on this and another page, but that doesn't appear to work at all and gives me the standard "you are not in the UK" error;

sh /jffs/scripts/x3mRouting/load_DNSMASQ_ipset.sh UKTV bbc.net.uk,bbc.co.uk,bbc.com,bbc.gscontxt.net,bbci.co.uk,ssl-bbcsmarttv.2cnt.net,itv.com,channel4.com,channel5.com,llnwd.net,edgefcs.net

Ideally I'd like to get iPlayer, ITV, Channel 4 and Chanel 5 all working via ipset tables if possible, as I don't want to force an entire device down the VPN and break local streaming.

Has anyone had any luck doing this recently or know of up to date ipset lists for these services as I notice the posts are from a couple of years ago that mention it.

Thanks.

This should be all you need for iPlayer.

sh /jffs/scripts/x3mRouting/load_DNSMASQ_ipset_iface.sh 1 BBC bbc.co.uk,bbc.com,bbc.gscontxt.net,bbci.co.uk,bbctvapps.co.uk,ssl-bbcsmarttv.2cnt.net,llnwd.net

Substitute the interface 1 with the appropriate interface.

BBC blocks known VPN servers. I use a private/dedicated VPN IP address to get around the blocks.

 

[dojrude]

This should be all you need for iPlayer.

sh /jffs/scripts/x3mRouting/load_DNSMASQ_ipset_iface.sh 1 BBC bbc.co.uk,bbc.com,bbc.gscontxt.net,bbci.co.uk,bbctvapps.co.uk,ssl-bbcsmarttv.2cnt.net,llnwd.net

Substitute the interface 1 with the appropriate interface.

BBC blocks known VPN servers. I use a private/dedicated VPN IP address to get around the blocks.

Thanks for replying.

The only difference from my list and yours is that I don't have bbctvapps.co.uk, but I excluded that as I couldn't resolve or ping that domain name.

Is there any difference from using the DNSMASQ_ipset_iface script and specifying the interface or running the other DNSMASQ script to create the ipset and then setting the ipset and 0.0.0.0 in the policy rules section of the VPN gui with the other dummy address on the bogan range as per the docs?

I don't think it's the VPN that's the issue as if I set a local IP of my streaming device to route to 0.0.0.0 in the same VPN config, iPlayer works fine, but then locally Oz streaming breaks - which the kids can't do without.

Thanks.

 

[Xentrk]

Thanks for replying.

The only difference from my list and yours is that I don't have bbctvapps.co.uk, but I excluded that as I couldn't resolve or ping that domain name.

Is there any difference from using the DNSMASQ_ipset_iface script and specifying the interface or running the other DNSMASQ script to create the ipset and then setting the ipset and 0.0.0.0 in the policy rules section of the VPN gui with the other dummy address on the bogan range as per the docs?

I don't think it's the VPN that's the issue as if I set a local IP of my streaming device to route to 0.0.0.0 in the same VPN config, iPlayer works fine, but then locally Oz streaming breaks - which the kids can't do without.

Thanks.

Add bbctvapps.co.uk back to the list. This is the entry in dnsmasq.log:

www.live.bbctvapps.co.uk

You will need to use it.

The OpenVPN Client Screen modifications does not support routing of IPSET lists to the WAN interface. The screen modifications was done as a proof of concept. There wasn't room to specify the WAN iface.

The screen only supports routing of IPSET lists thru the VPN interface. Looks like I failed to mention this on the README page. Sorry about that. I'll add it to my follow-up list.

You will have to use the script that has the word "_iface.sh" at the end of the name if you want to route to the WAN.

There was one more entry as the www.bbc.co.uk returns another ip address compared to bbc.co.uk without the www.

#nslookup www.bbc.co.uk
Server:    127.0.0.1
Address 1: 127.0.0.1 localhost.localdomain

Name:      www.bbc.co.uk
Address 1: 212.58.244.66 bbc-vip111.telhc.bbc.co.uk
Address 2: 212.58.249.208 bbc-vip146.lbh.bbc.co.uk

# nslookup bbc.co.uk
Server:    127.0.0.1
Address 1: 127.0.0.1 localhost.localdomain

Name:      bbc.co.uk
Address 1: 2a04:4e42::81
Address 2: 151.101.64.81
Address 3: 151.101.128.81
Address 4: 151.101.192.81
Address 5: 151.101.0.81

Route BBC to iface WAN0 using the DNSMASQ method

sh /jffs/scripts/x3mRouting/load_DNSMASQ_ipset_iface.sh 0 BBC www.bbc.co.uk,bbc.co.uk,bbc.com,bbc.gscontxt.net,bbci.co.uk,bbctvapps.co.uk,ssl-bbcsmarttv.2cnt.net,llnwd.net

 

[Janusz Fornal]

Policy Rules needs to be set to "Strict". Amazon blocks known VPN servers too. See the README for how to configure for Amazon. Not sure if HBO blocks known VPN servers though.

If i change Force Internet traffic through tunnel to Policy rules (strict) vpn not working with netflix and everything else.
BTW my goal is to setup expressvpn for everything except hbogo.pl and amazon (eu). Easiest way should be just setup netflix only for vpn (on smart tv) but its not (for me) .Hbogo.pl blocks vpn servers i found hideipvpn is working with hbogo.pl but its slow...

 

[Skeptical.me]

For sometime now I haven't been able to use selective routing, or Unbound, because of DNS leaks when connected to ExpressVPN. And when my DNS leaks I get proxy warning when trying to stream video from Netflix, Amazon Prime Video, and HULU. Also, Diversion stops working.

Is there any way I can watch Netflix, Hulu, Amazon Prime video, and have Diversion & Unbound working, plus be able to use policy routing? Is this possible? (I can use another VPN service like Nord if need be)

 

[Xentrk]

For sometime now I haven't been able to use selective routing, or Unbound, because of DNS leaks when connected to ExpressVPN. And when my DNS leaks I get proxy warning when trying to stream video from Netflix, Amazon Prime Video, and HULU. Also, Diversion stops working.

Is there any way I can watch Netflix, Hulu, Amazon Prime video, and have Diversion & Unbound working, plus be able to use policy routing? Is this possible? (I can use another VPN service like Nord if need be)

TorGuard has the private IP and you can use any DNS you want. I have tested all Accept DNS Configuration settings without issues.
https://x3mtek.com/why-i-use-torguard-as-my-vpn-provider/

 

[Xentrk]

If i change Force Internet traffic through tunnel to Policy rules (strict) vpn not working with netflix and everything else.
BTW my goal is to setup expressvpn for everything except hbogo.pl and amazon (eu). Easiest way should be just setup netflix only for vpn (on smart tv) but its not (for me) .Hbogo.pl blocks vpn servers i found hideipvpn is working with hbogo.pl but its slow...

With NordVPN and ExpressVPN, you have to configure Accept DNS Configuration to Exclusive as it is using their DNS as a proxy to get around the VPN blocks.

 

[Skeptical.me]

TorGuard has the private IP and you can use any DNS you want. I have tested all Accept DNS Configuration settings without issues.
https://x3mtek.com/why-i-use-torguard-as-my-vpn-provider/

So, just to clarify Diversion will work with a private ip with torguard?

Thanks for the reply.

 

[Xentrk]

So, just to clarify Diversion will work with a private ip with torguard?

Thanks for the reply.

Yes, as long as you don't set Accept DNS Configuration to Exclusive when using Policy Rules, Diversion will work.

 

[dojrude]

Add bbctvapps.co.uk back to the list. This is the entry in dnsmasq.log:

www.live.bbctvapps.co.uk

You will need to use it.

The OpenVPN Client Screen modifications does not support routing of IPSET lists to the WAN interface. The screen modifications was done as a proof of concept. There wasn't room to specify the WAN iface.

The screen only supports routing of IPSET lists thru the VPN interface. Looks like I failed to mention this on the README page. Sorry about that. I'll add it to my follow-up list.

You will have to use the script that has the word "_iface.sh" at the end of the name if you want to route to the WAN.

There was one more entry as the www.bbc.co.uk returns another ip address compared to bbc.co.uk without the www.

#nslookup www.bbc.co.uk
Server:    127.0.0.1
Address 1: 127.0.0.1 localhost.localdomain

Name:      www.bbc.co.uk
Address 1: 212.58.244.66 bbc-vip111.telhc.bbc.co.uk
Address 2: 212.58.249.208 bbc-vip146.lbh.bbc.co.uk

# nslookup bbc.co.uk
Server:    127.0.0.1
Address 1: 127.0.0.1 localhost.localdomain

Name:      bbc.co.uk
Address 1: 2a04:4e42::81
Address 2: 151.101.64.81
Address 3: 151.101.128.81
Address 4: 151.101.192.81
Address 5: 151.101.0.81

Route BBC to iface WAN0 using the DNSMASQ method

sh /jffs/scripts/x3mRouting/load_DNSMASQ_ipset_iface.sh 0 BBC www.bbc.co.uk,bbc.co.uk,bbc.com,bbc.gscontxt.net,bbci.co.uk,bbctvapps.co.uk,ssl-bbcsmarttv.2cnt.net,llnwd.net

Thanks for the info, but tried exactly what you've quoted and still doesn't work.
Also appeared to be routing some traffic that was destined for VPN client 1 connection, which broke access to some of my download clients.
Seems some of my downloads are directing to llnw.net, which must be getting picked up by the llnwd.net reference in the rule.

I think I'm going to give up and accept defeat as I've wasted far too much time on this and everyone else in the house is moaning that things keep breaking while I'm experimenting.

 

[Skeptical.me]

Yes, as long as you don't set Accept DNS Configuration to Exclusive when using Policy Rules, Diversion will work.

It's working.

After all of this time I could have just read your posts and had this working. ExpressVPN, while great, seems to complicate things for me.

I'll make sure to read all of your posts to get a good understanding for how I can take advantage of selective routing.

Thanks again.

 

[Xentrk]

It's working.

After all of this time I could have just read your posts and had this working. ExpressVPN, while great, seems to complicate things for me.

I'll make sure to read all of your posts to get a good understanding for how I can take advantage of selective routing.

Thanks again.

If you still want to use Express, there may be compromise solution depending on your use case.

Set up OpenVPN Client 1 with Policy Rules + Accept DNS Configuration = Exclusive. Diversion won't work on this interface.

Similarly, set up OpenVPN Client 2 with Policy Rules + Accept DNS Configuration = Disabled. It will default to WAN specified on DNS page. You can setup DoT to encrypt DNS. Cloudflare and Quad9 are both good choices. Diversion will work with this setup. Repeat this step if you have other servers you want to connect to.

Now, assign the route for each LAN client to OpenVPN Client 2 or 3, etc. You can use the Policy Routing section of the GUI or use Method 1 of x3mRouting.

At this point, you have two options.

You can assign your streaming devices to client 2 or 3. If you do this, you will need to use the x3mRouting scripts to create the IPSET lists for Netflix, Amazon, Hulu, etc and assign the interface to OpenVPN Client 1.

Alternatively, you could just assign your Streaming devices to use OpenVPN Client 1 and you shouldn't have to use the x3mRouting scripts.

 

[Skeptical.me]

If you still want to use Express, there may be compromise solution depending on your use case.

Set up OpenVPN Client 1 with Policy Rules + Accept DNS Configuration = Exclusive. Diversion won't work on this interface.

Similarly, set up OpenVPN Client 2 with Policy Rules + Accept DNS Configuration = Disabled. It will default to WAN specified on DNS page. You can setup DoT to encrypt DNS. Cloudflare and Quad9 are both good choices. Diversion will work with this setup. Repeat this step if you have other servers you want to connect to.

Now, assign the route for each LAN client to OpenVPN Client 2 or 3, etc. You can use the Policy Routing section of the GUI or use Method 1 of x3mRouting.

At this point, you have two options.

You can assign your streaming devices to client 2 or 3. If you do this, you will need to use the x3mRouting scripts to create the IPSET lists for Netflix, Amazon, Hulu, etc and assign the interface to OpenVPN Client 1.

Alternatively, you could just assign your Streaming devices to use OpenVPN Client 1 and you shouldn't have to use the x3mRouting scripts.

I think the last option would be the best considering the Telstra TV Box (only Aussie content) is routed to the WAN, and Apple TV (only US Content) is routed through the VPN.

I like your idea, I'll set up the OpenVPN clients like that.

If a device is not included in Policy Routing, what does it default too, the WAN?

 

[Xentrk]

Thanks for the info, but tried exactly what you've quoted and still doesn't work.
Also appeared to be routing some traffic that was destined for VPN client 1 connection, which broke access to some of my download clients.
Seems some of my downloads are directing to llnw.net, which must be getting picked up by the llnwd.net reference in the rule.

I think I'm going to give up and accept defeat as I've wasted far too much time on this and everyone else in the house is moaning that things keep breaking while I'm experimenting.

Hopefully you can find a time when the family is offline to keep at it.

If you don't have any clients devices listed in Policy Routing section, make sure you have the DummyVPN entry.

I'll test it without the llnwd.net this weekend to see if it can be removed. It's always a little bit of trial and error. Content Delivery Networks come into play with streaming services. As a result, you may need to mine dnsmasq. A lot of effort can go into mining dnsmasq to collect the domains names. Some services are easier than others. The getdomainnames.sh utility script helps with the effort. See the link for the usage and download instructions.

 

[Xentrk]

I think the last option would be the best considering the Telstra TV Box (only Aussie content) is routed to the WAN, and Apple TV (only US Content) is routed through the VPN.

I like your idea, I'll set up the OpenVPN clients like that.

If a device is not included in Policy Routing, what does it default too, the WAN?

That is correct. The device will default to the WAN if not in the GUI. The ip rule command allows you to see the routing policy database rules in priority order. The lower the number, the higher the priority. Devices set to use the WAN won't appear in the list.

 

[Skeptical.me]

That is correct. The device will default to the WAN if not in the GUI. The ip rule command allows you to see the routing policy database rules in priority order. The lower the number, the higher the priority. Devices set to use the WAN won't appear in the list.

Everything is working great. OpenVPN client 1 "Accept DNS Configuration" set to "Exclusive" and the Apple TV is routed through the VPN. OpenVPN Client 2 "Accept DNS Configuration" set to "Disabled" and various devices are either routed through the VPN or WAN. Diversion is working, NextDNS is working, and Skynet etc all working well. I wonder now if Unbound would work with this set up.

Thanks for your help, appreciated.

 

[L&LD]

@Skeptical.me, that's the spirit! Everything is good. Now... let's try to break it again!

Keep pushing for better and keep us informed of your trials too.

Btw, v2.06 of unbound_manager is working great here.

If you do get unbound working in your more complicated setup? You can always try IPv6 again too.

 

[Kingp1n]

@Skeptical.me, that's the spirit! Everything is good. Now... let's try to break it again!

Keep pushing for better and keep us informed of your trials too.

Btw, v2.06 of unbound_manager is working great here.

If you do get unbound working in your more complicated setup? You can always try IPv6 again too.

L&LD do you have IVP6 enabled and are you using the DNS64 with unbound?