x3mRouting x3mRouting ~ Selective Routing for Asuswrt-Merlin Firmware (1-Nov-2020)

[abracadabra11]

Running into some problems streaming NETFLIX using x3mRouting on my RT-AX3000 (Merlin 384.18 and compatible x3m branch).

I've only ever used the ASN method on my RT-AC68U and I suspect that this might be causing some issues. Should I attempt using dnsmasq method?

x3mRouting ipset_name=NETFLIX dnsmasq=netflix.com,nflxext.com,nflximg.net,nflxso.net,nflxvideo.net

Or is there a preferred method?

 

[Xentrk]

Installed this, only first option gets installed the other 3 throws out an error, as in expected package size is different

I think entware is the issue as option 2 thru 4 require entware. Try running these commands:

/opt/bin/opkg update
/opt/bin/opkg upgrade

Then, try installing the option again via x3mMenu

May want to run a chkdsk on the USB drive. amtm has a scan utility and the main page on the firmware has a disk scan utility too.

 

[Xentrk]

Running into some problems streaming NETFLIX using x3mRouting on my RT-AX3000 (Merlin 384.18 and compatible x3m branch).

I've only ever used the ASN method on my RT-AC68U and I suspect that this might be causing some issues. Should I attempt using dnsmasq method?

x3mRouting ipset_name=NETFLIX dnsmasq=netflix.com,nflxext.com,nflximg.net,nflxso.net,nflxvideo.net

Or is there a preferred method?

ASN method will cast a wide net as many streaming services use the same ASN as Netflix. With the above syntax, you need to use the modified OpenVPN screen to create the rule since you are not specifying the interface.

With the Syntax above, you have to use the custom x3mRouting OpenVPN Client screen to create the routing rule in the policy section. Whereas the syntax below will bypass VPN Client 1 and route Netflix traffic to the WAN... aka**VPN Bypass** and you don't need to enter the IPSET list in the screen.

x3mRouting 1 0 NETFLIX dnsmasq=netflix.com,nflxext.com,nflximg.net,nflxso.net,nflxvideo.net

 

[Xentrk]

Dear Xentrk,
for a few days, I have problems with populating my amazon prime video rules. In my natstart the following entries are there:
sh /jffs/scripts/x3mRouting/x3mRouting.sh 1 0 AMAZON-EU aws_region=EU
sh /jffs/scripts/x3mRouting/x3mRouting.sh 1 0 AMAZON-US aws_region=US
sh /jffs/scripts/x3mRouting/x3mRouting.sh 1 0 AMAZON-CA aws_region=CA
sh /jffs/scripts/x3mRouting/x3mRouting.sh 1 0 AMAZON-AP aws_region=AP
sh /jffs/scripts/x3mRouting/x3mRouting.sh 1 0 AMAZON-CN aws_region=CN
sh /jffs/scripts/x3mRouting/x3mRouting.sh 1 0 AMAZON-GLOBAL aws_region=GLOBAL
sh /jffs/scripts/x3mRouting/x3mRouting.sh 1 0 AMAZON-SA aws_region=SA

My router reboots everyday as a cron job. As I recognized , that prime video has sometimes problems, I checked the liststats.

I found out, that sometimes the lists are populating very differently. Sometimes only the EU List is populated (532 entries), sometimes Global (108) and EU and sometimes other regions are populated as well. I rebootet the router a lot of times and I am getting different population. Do you have an idea, why that is occuring ? Could my DNS Server sometimes filter, so that not all lists are populated ?
Forward local domain queries to upstream DNS --> No , using quad9 as dns servers.
Netflix is working normally.

Is there a command for forcing a new population of the iprules (so that I don´t have to reboot everytime the router for testing)? Is it possible to add a cron job that the router is trying to repopulate after e.g. 5 Minutes of rebooting again and add the missing entries ?


Thanks a lot for your support.

Hugo


PS: The best "population" I recieved after reboot was (and that was working perfectly):
AMAZON-AP - 579
AMAZON-CA - 61
AMAZON-CN - 93
AMAZON-EU - 532
AMAZON-GLOBAL - 108
AMAZON-SA - 75
AMAZON-US - 963

I suspect AMAZON-GLOBAL covers all of the AWS last time I looked at it. I need to go back to their website and validate though. I'll get back to you after I have had time to look into it.

Look at the system log for clues. The code was recently updated so only one instance of x3mRouting can run at a time. So no concurrent processing issues. I suspect there is a limit on how many times you can download in a day. Some sites I used to collect IPv4 addresses do have a limit and I recently changed to another source.

 

[abracadabra11]

ASN method will cast a wide net as many streaming services use the same ASN as Netflix. With the above syntax, you need to use the modified OpenVPN screen to create the rule since you are not specifying the interface.

Should have specified that I did have an active OpenVPN rule. Is there a preferred method for routing?

 

[mister]

I suspect AMAZON-GLOBAL covers all of the AWS last time I looked at it. I need to go back to their website and validate though. I'll get back to you after I have had time to look into it.

Look at the system log for clues. The code was recently updated so only one instance of x3mRouting can run at a time. So no concurrent processing issues. I suspect there is a limit on how many times you can download in a day. Some sites I used to collect IPv4 addresses do have a limit and I recently changed to another source.

Dear Xentrk,
thanks a lot for your support. The behavior is really strange. Because after the reboot I always get different populations of the liststats, whereby the outgoing IP address does not change. I've tried a little more and I suspect that it could have something to do with either the VPN connections or the DNS servers. For example, if I If the DNS server in the WAN section changes, the population changes too, but not always in the same way.
As already mentioned, sometimes AMAZON EU lists are populated, sometimes only globally, sometimes only in other regions. Often some are simply missing. Before, I only had "one" entry for AWS Region for all, separated by ",". I noticed there that the number of entries had changed. But I only noticed this after I updated AMTM because at some point Amazon stopped working.
I don't know if that could be related (just for info).

Or maybe it could be about the time when the lists are received. Possibly the router has not yet established the Internet connection, so that the lists are not completely retrieved ?? So internet is missing, the RT86 AC asks about Amazon EU, gets no answer - list is not populated, etc. Sometimes the router already has an internet connection and then everything runs smoothly??
It's just a wild guess now, but unfortunately my log was no longer available to check. And I just don't dare to reboot because it just works .

If you have any tips or ideas about what else I could try, please let me know. Thank you again for your effort.

Is x3mrouting actually compatible with the current beta firmware or should I wait with the update?

Thanks a lot

Hugo

 

[Xentrk]

Should have specified that I did have an active OpenVPN rule. Is there a preferred method for routing?

I just depends on ones use case. I am an expat living in the Land of Smiles and using ASN method for Netflix works good for me as there are other services I subscribe too that are also hosted on AS2906. So it works fine for me that way. However, using ASN was causing an issue for another x3mRouting user in the thread with other streaming services and switching to the dnsmasq method solved their issue. The dnsmasq method was what I first used several years ago when I first started learning about selective routing. It was later on that I learned about ASN from another forum site. I am currently using dnsmasq method for Netflix on my Asus router I develop on. But on my pfSense appliance, I use the AS2906 for Netflix, which also includes Disney+ and a few others I subscribe to.

 

[Xentrk]

Dear Xentrk,
thanks a lot for your support. The behavior is really strange. Because after the reboot I always get different populations of the liststats, whereby the outgoing IP address does not change. I've tried a little more and I suspect that it could have something to do with either the VPN connections or the DNS servers. For example, if I If the DNS server in the WAN section changes, the population changes too, but not always in the same way.
As already mentioned, sometimes AMAZON EU lists are populated, sometimes only globally, sometimes only in other regions. Often some are simply missing. Before, I only had "one" entry for AWS Region for all, separated by ",". I noticed there that the number of entries had changed. But I only noticed this after I updated AMTM because at some point Amazon stopped working.
I don't know if that could be related (just for info).

Or maybe it could be about the time when the lists are received. Possibly the router has not yet established the Internet connection, so that the lists are not completely retrieved ?? So internet is missing, the RT86 AC asks about Amazon EU, gets no answer - list is not populated, etc. Sometimes the router already has an internet connection and then everything runs smoothly??
It's just a wild guess now, but unfortunately my log was no longer available to check. And I just don't dare to reboot because it just works .

If you have any tips or ideas about what else I could try, please let me know. Thank you again for your effort.

Is x3mrouting actually compatible with the current beta firmware or should I wait with the update?

Thanks a lot

Hugo

Glad to be of help.


Scribe and uiScribe are a big help when trying to analyze the system log. Next time it happens, search for x3mRouting. See if it is running as expected. You will see 4 to 8 lines each time x3mRouting runs in the system log. Just depends on the method being used. There is a unique PID number for each program. There should be a start and end message and other output in between the start and end messages. You also want to scroll up and see if you can find the messages that start when the router is rebooted. Then, scroll down and see if there are any error messages about services starting on the router. Make sure you uncheck the box to scroll when the system log is updated or it will drive you nuts when trying to view the log.

Do you have the little box checked to block access to the internet if VPN is down? If so, uncheck it. I always recommend policy routing users have an entry in the VPN Client 1 screen to route the router IP address to the WAN. This way, services like NTP can still access the internet if the VPN has not started.

Also, can you try just using the Global region rather than all of them?

x3mRouting was recently updated using the same method that Skynet uses for populating IPSET lists. Before the change, I would first download the IPv4 addresses to a file. Then, load the ipset list from the file in /opt/tmp directory. In addition, It would only download a new json file if the file was more than 7 days old. x3mRouting now loads the IPv4 addresses directly into memory rather than first downloading the data to a file in /opt/tmp.

I've been traveling the past few days and we have finally arrived at our final destination this afternoon. I should be able to spare some time to check the AWS website to see if they limit the number of download attempts. Also, if you are having issues during the reboot and the list is not being populated, run nat-start and see if the lists are populated as expected.

One other idea is to go to the AWS website and click on the link to download the json file and make sure it is not being blocked.

https://docs.aws.amazon.com/general/latest/gr/aws-ip-ranges.html

As an alternative, you can use the ASN method for AWS such as AS16509 and AS14618 until we get find out the root cuase of the issue. I have also used these in the past with success before I learned about the json file Amazon produces. Start with the two ASN listed above. If you still have issues, add some of the other Amazon ASN shown on Hurricane Electric: https://bgp.he.net/search?search[search]=amazon+aws&commit=Search

You can specify more than one ASN separated by a comma. For example:

x3mRouting 1 0 AWS asnum=AS16409,AS14618

 

[Kingp1n]

@Xentrk,
I'm running latest asusmerlin fw (386.1 beta). I setup my router again and currently installed your script (option 3). When i used my back-up nat-start file and upload in the /jffs/scripts/ folder. I can no longer just run the command thru ssh:

/jffs/scripts/nat-start

I keep getting a "not found" error. Any ideas what I can be doing wroing? Thanks!

I'm not sure what happen by all my rules are gone after I manually set them last night:

admin@RT-AX88U-0D80:/tmp/home/root# iptables -nvL PREROUTING -t mangle --line
Chain PREROUTING (policy ACCEPT 32157 packets, 34M bytes)
num   pkts bytes target     prot opt in     out     source               destination

UPDATE: I removed cake and x3mRouting script. I went back and re-install cake. The commands seems to be working so I'll re-install your script again. Wish me luck haha

 

[mister]

Glad to be of help.


Scribe and uiScribe are a big help when trying to analyze the system log. Next time it happens, search for x3mRouting. See if it is running as expected. You will see 4 to 8 lines each time x3mRouting runs in the system log. Just depends on the method being used. There is a unique PID number for each program. There should be a start and end message and other output in between the start and end messages. You also want to scroll up and see if you can find the messages that start when the router is rebooted. Then, scroll down and see if there are any error messages about services starting on the router. Make sure you uncheck the box to scroll when the system log is updated or it will drive you nuts when trying to view the log.

Do you have the little box checked to block access to the internet if VPN is down? If so, uncheck it. I always recommend policy routing users have an entry in the VPN Client 1 screen to route the router IP address to the WAN. This way, services like NTP can still access the internet if the VPN has not started.

Also, can you try just using the Global region rather than all of them?

x3mRouting was recently updated using the same method that Skynet uses for populating IPSET lists. Before the change, I would first download the IPv4 addresses to a file. Then, load the ipset list from the file in /opt/tmp directory. In addition, It would only download a new json file if the file was more than 7 days old. x3mRouting now loads the IPv4 addresses directly into memory rather than first downloading the data to a file in /opt/tmp.

I've been traveling the past few days and we have finally arrived at our final destination this afternoon. I should be able to spare some time to check the AWS website to see if they limit the number of download attempts. Also, if you are having issues during the reboot and the list is not being populated, run nat-start and see if the lists are populated as expected.

One other idea is to go to the AWS website and click on the link to download the json file and make sure it is not being blocked.

https://docs.aws.amazon.com/general/latest/gr/aws-ip-ranges.html

View attachment 28337
As an alternative, you can use the ASN method for AWS such as AS16509 and AS14618 until we get find out the root cuase of the issue. I have also used these in the past with success before I learned about the json file Amazon produces. Start with the two ASN listed above. If you still have issues, add some of the other Amazon ASN shown on Hurricane Electric: https://bgp.he.net/search?search[search]=amazon+aws&commit=Search

You can specify more than one ASN separated by a comma. For example:

x3mRouting 1 0 AWS asnum=AS16409,AS14618

Dear Xentrk,
thanks again for your reply.
I rebooted my router and found in the description a potential problem, which occurs at a lot of my entries, and is maybe responsible for my problem:

May 5 07:05:44 (x3mRouting.sh): 4971 Starting Script Execution 1 0 AMAZON-EU aws_region=EU
May 5 07:05:44 (x3mRouting.sh): [*] Lock File Detected (1 0 NETFLIX-2906 asnum=AS2906) (pid=4645) - Exiting (cpid=4971)

I rebooted again and it worked. In the syslog the following entry occurs:
May 5 07:05:43 (x3mRouting.sh): 4739 Starting Script Execution 1 0 AMAZON-EU aws_region=EU
May 5 07:05:43 (x3mRouting.sh): 4739 IPSET created: AMAZON-EU

So it seems to be a problem mit a lock file, wheras I don´t know why it occured...
Any ideas ?

 

[mister]

Dear Xentrk,
I tested a little bit more and found out, that the position of the entry in my natstart has an influence of the population.
I moved the Amazon entries at the beginning of the natstart file and rebooted.

All Lists were populated as expected, but suddenly I had problems with my other Entry "mediatheken" which I didn´t had before. In the syslog the following entry occured:

5117 Starting Script Execution ALL 1 Mediatheken dnsmasq=zdf.de,zdfmediathek.de,ard.de,wdr.de,kika.de,phoenix.de,swr.de,swrmediathek.de,br.de,ardmediathek.de,wdrmediathek.de,phoenix.de,akamaihd.net,hr.de,akamaistream.net,dw.de,sr-online.de,ndr.de,rbb-online.de,apa.at,tagesschau.de,heute.de,akamai.com,daserste.de,ardaudiothek.de,deutschlandfunk.de,3sat.de,sr-mediathek.de,swrmediathek.de,mdr.de,arte.de,ndr.de,tvdlzdf-a.akamaihd.net/de,akamaiedge.net,nrodlzdf-a.akamaihd.ne
May 5 07:05:46 (x3mRouting.sh): [*] Lock File Detected (1 0 NETFLIX-812 asnum=AS812) (pid=4515) - Exiting (cpid=5117)

So it seemed to my , that the ASN Entry of Netflix blocks the population.

What can I do ?
Thanks a lot.

Hugo


PS: Even the Systemlog of the router after the reboot looks differently every time. Sometimes the first entries are from the x3mscript, sometimes kernel messages etc

PPS: I had skynet installed for a while, but even with skynet I had sometimes the failure that a file was locked

PPPS: I checked in the liststats the entries: All ASN Entries didn´t populate , and there might be the reason for my problems because they are producing lock files. I will "#" all ASN entries in my natstart and see, what is happening. Nevertheless, I am hoping that you might find a solution for my problem:
NETFLIX-14618 - 0
NETFLIX-2906 - 0
NETFLIX-394406 - 0
NETFLIX-812 - 0

 

[mister]

Dear Xentrk,
I tested a little bit more yesterday: What I found:

1. If you vary the order of the entries in the nat-start file, I got different results (sometimes Netflix Lists didn´t populate , sometimes my mediathek lists, sometimes Amazon).
2. It seems to me, that different "reboot" variation (over ssh, over webUI or over power off) gives different messages in the log files of the system (e.g. Kernel).
3. If I manually repeat the execution of nat-start over SSH after a while after the router has rebooted it seems to me, that then everything seems to run smoothly (whereas I have to say, I tested that only twice , so it could be that it was coincidence) .


What I would like to test:
Inserting a cron job that runs the nat-start again 2 minutes after the router has booted. Unfortunately I do not know the exact command or where I have to create the corresponding entry. Can someone help me there ?

 

[Sean Rhodes]

I recommend doing some analysis using the scripts available in option 4. I would start by routing all traffic to the VPN client you use for BBC and make sure iPlayer works okay that way. Then, use the autoscan.sh script and search for "uk" or "bbc" terms. The getdomainnames.sh script will give you a big picture view of what is being queried.

Hi Xentrk,

I need to pick your brain again regarding my latest test results. I set my VPN to strict and routed all internet traffic through ovpn3. I ran iPlayer on my AppleTV and found most programs connected without a hitch. I tried a few searches and all seemed well. However for some shows I get the message "something went wrong please try again later", a few repeated tries would get some shows to play, but others, no matter what I did they would not play. I ran autoscan.sh and getdomainnames.sh and below are my outputs compared to my ipset BBC_WEB4:

sh autoscan.sh autoscan=bbc

bbc.com
co.uk
net.uk

sh getdomainnames.sh

Enter a descriptive name of the output file ==> bbc_iplayer_results

Enter the IP address ==> 10.0.1.60

Press Ctrl-C to stop logging
^C
Done capturing domains from dnsmasq.log
Sorting file.
File contents are:

a1.api.bbc.co.uk
a1051.b.akamai.net
appletv.iplayer.api.bbc.co.uk
b2rbsov.bidi.live.bbc.co.uk
b3rbsov.bidi.live.bbc.co.uk
b3thdo.bidi.live.bbc.co.uk
b5rbsov.bidi.live.bbc.co.uk
dh53v7vqnorkn.cloudfront.net
e3891.dscf.akamaiedge.net
edibl.f7f1036195026b0a.xhst.bbci.co.uk
graph.ibl.api.bbc.co.uk
guzzoni-apple-com.v.aaplimg.com
ibl-live-alb-ibl-edibl-196200881.eu-west-1.elb.amazonaws.com
ibl.api.bbci.co.uk
ichef-bbci.bbc.net.uk
ichef.bbci.co.uk.edgekey.net
init-p01md.apple.com
live-compo-qkdrft3n54j1-1326475104.eu-west-1.elb.amazonaws.com
mm.bidi.bbc.co.uk
open.live.bbc.co.uk
time.apple.com
vod-hls-uk-live.bbcfmt.s.llnwi.net

File location is: /opt/var/log/bbc_iplayer_results

BBC_WEB4 dnsmasq=2cnt.net,bbc.com,bbcverticals.com,co.uk,llnwi.net,net.uk

As you can see, all the top domains from autoscan.sh are captured, along with llnwi.net. Is there anything else I can try? I'm using NordVPN, so I was thinking of trying to use their DNS as opposed to setting the DNS to router. I will keep playing, but wanted to know if anything obvious stands out to you i.e. edgekey.net?

Thanks

Update:
OK, I changed my DNS to the NordVPN instead of the router, 103.86.96.100, and the non-working shows now work. However getdomainnames.sh reveals an empty dnsmasq.log, presumably due to being bypassed and using the Nord DNS.

I closed iPlayer and restarted again trying to access the same show I just watched and I got the "this show can only watched in the UK", so it appears the Nord DNS must be leaking my IP, or it's a known DNS and therefore blocked.

I reset to router and the show now works, so its a little confusing since my setup is just the same

Update2:
I played a little more, adding edgekey.net broke my ipset, so I had to remove it. Also adding my NordVPN DNS IP under OVPN2 custom config (re your post to Wolvenstein

https://www.snbforums.com/threads/x3mrouting-selective-routing-for-asuswrt-merlin-firmware-1-nov-2020.67388/post-635106

didn't work either. So far the best config I can get is as follows:
1. Under LAN --> DNS Filter, set DNS Filter to router
2. Under OVPN3, set accept DNS config to exclusive
3. Under OVPN3, force internet traffic through tunnel set to policy rules (strict)
4. Setting my ipset list to BBC_WEB4

This gives me about 95%

 

[Xentrk]

@Xentrk,
I'm running latest asusmerlin fw (386.1 beta). I setup my router again and currently installed your script (option 3). When i used my back-up nat-start file and upload in the /jffs/scripts/ folder. I can no longer just run the command thru ssh:

/jffs/scripts/nat-start

I keep getting a "not found" error. Any ideas what I can be doing wroing? Thanks!

I'm not sure what happen by all my rules are gone after I manually set them last night:

admin@RT-AX88U-0D80:/tmp/home/root# iptables -nvL PREROUTING -t mangle --line
Chain PREROUTING (policy ACCEPT 32157 packets, 34M bytes)
num   pkts bytes target     prot opt in     out     source               destination

UPDATE: I removed cake and x3mRouting script. I went back and re-install cake. The commands seems to be working so I'll re-install your script again. Wish me luck haha

I've been offline taking a much needed break. Very sorry for delay in reply. Looks like you got it fixed. But don't forget you have to do the "sh" thingy.

fully qualified path

sh /jffs/scripts/nat-start

If you are already in the /jffs/scripts directory.

sh nat-start

 

[Xentrk]

Dear Xentrk,
thanks again for your reply.
I rebooted my router and found in the description a potential problem, which occurs at a lot of my entries, and is maybe responsible for my problem:

May 5 07:05:44 (x3mRouting.sh): 4971 Starting Script Execution 1 0 AMAZON-EU aws_region=EU
May 5 07:05:44 (x3mRouting.sh): [*] Lock File Detected (1 0 NETFLIX-2906 asnum=AS2906) (pid=4645) - Exiting (cpid=4971)

I rebooted again and it worked. In the syslog the following entry occurs:
May 5 07:05:43 (x3mRouting.sh): 4739 Starting Script Execution 1 0 AMAZON-EU aws_region=EU
May 5 07:05:43 (x3mRouting.sh): 4739 IPSET created: AMAZON-EU

So it seems to be a problem mit a lock file, wheras I don´t know why it occured...
Any ideas ?

That is strange as the pids are different. One is 4971 and the other 4645. I cloned the code skynet uses in one of the recent updates so only one instance of x3mRouting can run at a time. It makes it much easier to debug issues when looking at the log file. If x3mRouting didn't finish, the lock file would not have gotten removed.

Look in the /tmp directory for the lock file and manually remove it using the "rm" command.

cd /tmp
ls -al | grep lock
rm *.lock

x3mRouting will load the ipset lists and create the rules even if the OpenVPN clients have not started. As a result, if the box is checked to block internet access if the internet is down, the IPSET lists won't be able to access the internet to pull the IPv4 addresses.

 

[Xentrk]

Hi Xentrk,

I need to pick your brain again regarding my latest test results. I set my VPN to strict and routed all internet traffic through ovpn3. I ran iPlayer on my AppleTV and found most programs connected without a hitch. I tried a few searches and all seemed well. However for some shows I get the message "something went wrong please try again later", a few repeated tries would get some shows to play, but others, no matter what I did they would not play. I ran autoscan.sh and getdomainnames.sh and below are my outputs compared to my ipset BBC_WEB4:

sh autoscan.sh autoscan=bbc

bbc.com
co.uk
net.uk

sh getdomainnames.sh

Enter a descriptive name of the output file ==> bbc_iplayer_results

Enter the IP address ==> 10.0.1.60

Press Ctrl-C to stop logging
^C
Done capturing domains from dnsmasq.log
Sorting file.
File contents are:

a1.api.bbc.co.uk
a1051.b.akamai.net
appletv.iplayer.api.bbc.co.uk
b2rbsov.bidi.live.bbc.co.uk
b3rbsov.bidi.live.bbc.co.uk
b3thdo.bidi.live.bbc.co.uk
b5rbsov.bidi.live.bbc.co.uk
dh53v7vqnorkn.cloudfront.net
e3891.dscf.akamaiedge.net
edibl.f7f1036195026b0a.xhst.bbci.co.uk
graph.ibl.api.bbc.co.uk
guzzoni-apple-com.v.aaplimg.com
ibl-live-alb-ibl-edibl-196200881.eu-west-1.elb.amazonaws.com
ibl.api.bbci.co.uk
ichef-bbci.bbc.net.uk
ichef.bbci.co.uk.edgekey.net
init-p01md.apple.com
live-compo-qkdrft3n54j1-1326475104.eu-west-1.elb.amazonaws.com
mm.bidi.bbc.co.uk
open.live.bbc.co.uk
time.apple.com
vod-hls-uk-live.bbcfmt.s.llnwi.net

File location is: /opt/var/log/bbc_iplayer_results

BBC_WEB4 dnsmasq=2cnt.net,bbc.com,bbcverticals.com,co.uk,llnwi.net,net.uk

As you can see, all the top domains from autoscan.sh are captured, along with llnwi.net. Is there anything else I can try? I'm using NordVPN, so I was thinking of trying to use their DNS as opposed to setting the DNS to router. I will keep playing, but wanted to know if anything obvious stands out to you i.e. edgekey.net?

Thanks

Update:
OK, I changed my DNS to the NordVPN instead of the router, 103.86.96.100, and the non-working shows now work. However getdomainnames.sh reveals an empty dnsmasq.log, presumably due to being bypassed and using the Nord DNS.

I closed iPlayer and restarted again trying to access the same show I just watched and I got the "this show can only watched in the UK", so it appears the Nord DNS must be leaking my IP, or it's a known DNS and therefore blocked.

I reset to router and the show now works, so its a little confusing since my setup is just the same

Update2:
I played a little more, adding edgekey.net broke my ipset, so I had to remove it. Also adding my NordVPN DNS IP under OVPN2 custom config (re your post to Wolvenstein

https://www.snbforums.com/threads/x3mrouting-selective-routing-for-asuswrt-merlin-firmware-1-nov-2020.67388/post-635106

didn't work either. So far the best config I can get is as follows:
1. Under LAN --> DNS Filter, set DNS Filter to router
2. Under OVPN3, set accept DNS config to exclusive
3. Under OVPN3, force internet traffic through tunnel set to policy rules (strict)
4. Setting my ipset list to BBC_WEB4

This gives me about 95%

I think the issue is NordVPN (and ExpressVPN) requires you to use their DNS to get around blocks. If you have Accept DNS Configuration = Exclusive, then dnsmasq is bypassed and the dnsmasq method won't collect the IPv4 addresses. I powered off my network before I left for my holiday so I can't remote into it. When I return (eta 5 to 7 days), I will post my BBC ipset list on GitHub and you can download it.

 

[Sean Rhodes]

I think the issue is NordVPN (and ExpressVPN) requires you to use their DNS to get around blocks. If you have Accept DNS Configuration = Exclusive, then dnsmasq is bypassed and the dnsmasq method won't collect the IPv4 addresses. I powered off my network before I left for my holiday so I can't remote into it. When I return (eta 5 to 7 days), I will post my BBC ipset list on GitHub and you can download it.

Thanks

 

[mister]

That is strange as the pids are different. One is 4971 and the other 4645. I cloned the code skynet uses in one of the recent updates so only one instance of x3mRouting can run at a time. It makes it much easier to debug issues when looking at the log file. If x3mRouting didn't finish, the lock file would not have gotten removed.

Look in the /tmp directory for the lock file and manually remove it using the "rm" command.

cd /tmp
ls -al | grep lock
rm *.lock

x3mRouting will load the ipset lists and create the rules even if the OpenVPN clients have not started. As a result, if the box is checked to block internet access if the internet is down, the IPSET lists won't be able to access the internet to pull the IPv4 addresses.

Dear Xentrk,
thanks a lot for your hints and your suggestions.
Meanwhile I have a workaround for the problem, because my problem is not easy to reproduce:

1. I paused "#" all ASN number entries
2. I add the following command into the services-start file: sleep 60 && sh ./jffs/scripts/nat-start

So the nat-start is running twice. With that approach all lists are populated correctly without any lock and the routing is working.
I will test your approach as well.

Hugo.

 

[OBENZ]

I'm kind of lost to be honest, it's overwhelming to do this for a newbie. I want to route traffic from specific devices to a vpn specifically the Xbox live ips

 

[L&LD]

Search for the original thread. This toothless dragon ('lost cluster' of a thread) has little to bite into.