Xentrk
Part of the Furniture
The ip rule command will show the RPDB rules for LAN clients.
Check the TV settings for a region setting. I've seen that mess things up before.
x3mRouting x3mRouting ~ Selective Routing for Asuswrt-Merlin Firmware (1-Nov-2020)
- Thread starter Xentrk
- Start date
Check the TV settings for a region setting. I've seen that mess things up before.
Ooh not sure how that happened. So 0x8000 is to WAN 0x1000 is to vpn client 1
I have them pointing at the right thing now but the dnsmasq rules still didn't seem to be getting used at all. But I just spotted your reply to obenz about the vpn dns configuration, mine was set to exclusive which I expect is what was stopping the dnsmasq method from working all along. I set it to strict and restarted the vpn and things seem to be working again. I left off the amazon rules but added back the netflix asn, netflix web, bbc asn, and bbc web rules and they seem to be working again now, and binance is no longer getting through. It looks like it was the amazon rules letting too much through so I have left those off as I don't really use amazon video anyway now. I will remove the asn rules and see if everything is still ok just using the dns masq method as I would prefer to use specific ip's instead of opening up half the internet by using asn's.
Sean Rhodes
Regular Contributor
Hi Xentrk,
Are you able to tell me what the item in the middle of my dnsmasq.log is referring to?
Code:
rhodess@RT-AC3200-4200:/tmp/home/root# grep ipset /opt/var/log/dnsmasq.log
Nov 24 06:59:08 dnsmasq[25526]: ipset add BBC_WEB3 151.101.126.133 d.sni.global.fastly.net
Nov 24 07:11:06 dnsmasq[25526]: ipset add BBC_WEB3 151.101.126.110 f4.shared.global.fastly.net
Nov 24 08:11:32 dnsmasq[25526]: ipset add BBC_WEB3 151.101.126.133 d.sni.global.fastly.net
Nov 24 08:11:33 dnsmasq[25526]: ipset add BBC_WEB3 216.239.36.54 us-central1-adaptive-growth.cloudfunctions.net
Nov 24 08:11:34 dnsmasq[25526]: ipset add BBC_WEB3 151.101.126.110 f4.shared.global.fastly.net
Nov 24 20:22:20 dnsmasq[25526]: ipset add BBC_WEB3 151.101.126.133 na-eu.atlanticmedia.map.fastly.net
Nov 24 20:22:21 dnsmasq[25526]: ipset add BBC_WEB3 151.101.125.111 dualstack.guardian.map.fastly.net
Nov 24 20:22:21 dnsmasq[25526]: ipset add BBC_WEB3 151.101.126.62 domains.gannett.map.fastly.net
Nov 24 20:22:21 dnsmasq[25526]: ipset add BBC_WEB3 151.101.125.164 nytimes.map.fastly.net
Nov 24 20:22:21 dnsmasq[25526]: ipset add BBC_WEB3 151.101.124.239 condenast.map.fastly.net
Nov 24 20:22:42 dnsmasq[25526]: ipset add BBC_WEB3 151.101.125.52 vox-chorus.map.fastly.net
Nov 24 20:25:48 dnsmasq[25526]: ipset add BBC_WEB3 151.101.125.164 nytimes.map.fastly.net
Nov 24 20:25:48 dnsmasq[25526]: ipset add BBC_WEB3 151.101.126.62 domains.gannett.map.fastly.net
Nov 24 20:25:48 dnsmasq[25526]: ipset add BBC_WEB3 151.101.124.239 condenast.map.fastly.net
Nov 24 20:25:48 dnsmasq[25526]: ipset add BBC_WEB3 151.101.126.133 na-eu.atlanticmedia.map.fastly.net
Nov 24 20:25:48 dnsmasq[25526]: ipset add BBC_WEB3 151.101.125.52 vox-chorus.map.fastly.net
Nov 24 20:25:48 dnsmasq[25526]: ipset add BBC_WEB3 151.101.125.111 dualstack.guardian.map.fastly.net
Nov 24 20:27:28 dnsmasq[25526]: ipset add BBC_WEB3 151.101.126.62 domains.gannett.map.fastly.net
Nov 24 20:27:28 dnsmasq[25526]: ipset add BBC_WEB3 151.101.126.62 domains.gannett.map.fastly.net
Nov 24 20:27:28 dnsmasq[25526]: ipset add BBC_WEB3 151.101.125.164 nytimes.map.fastly.net
Nov 24 20:27:28 dnsmasq[25526]: ipset add BBC_WEB3 151.101.125.164 nytimes.map.fastly.net
Nov 24 20:27:28 dnsmasq[25526]: ipset add BBC_WEB3 151.101.124.239 condenast.map.fastly.net
Nov 24 20:27:28 dnsmasq[25526]: ipset add BBC_WEB3 151.101.124.239 condenast.map.fastly.net
Nov 24 20:27:28 dnsmasq[25526]: ipset add BBC_WEB3 151.101.126.133 na-eu.atlanticmedia.map.fastly.net
Nov 24 20:27:37 dnsmasq[25526]: ipset add BBC_WEB3 151.101.125.52 vox-chorus.map.fastly.net
Nov 24 20:27:37 dnsmasq[25526]: ipset add BBC_WEB3 151.101.125.111 dualstack.guardian.map.fastly.net
Nov 24 20:27:37 dnsmasq[25526]: ipset add BBC_WEB3 151.101.125.111 dualstack.guardian.map.fastly.net
Nov 24 20:27:43 dnsmasq[25526]: ipset add BBC_WEB3 151.101.126.217 c3.shared.global.fastly.net
Nov 24 20:38:56 dnsmasq[25526]: ipset add BBC_WEB3 151.101.126.62 domains.gannett.map.fastly.net
Nov 24 20:38:56 dnsmasq[25526]: ipset add BBC_WEB3 151.101.125.164 nytimes.map.fastly.net
Nov 24 20:38:56 dnsmasq[25526]: ipset add BBC_WEB3 151.101.124.239 condenast.map.fastly.net
Nov 24 20:38:56 dnsmasq[25526]: ipset add BBC_WEB3 151.101.126.133 na-eu.atlanticmedia.map.fastly.net
Nov 24 20:38:56 dnsmasq[25526]: ipset add BBC_WEB3 151.101.125.52 vox-chorus.map.fastly.net
Nov 24 20:38:56 dnsmasq[25526]: ipset add BBC_WEB3 151.101.125.111 dualstack.guardian.map.fastly.net
Nov 24 22:37:40 dnsmasq[9806]: compile time options: IPv6 GNU-getopt no-RTC no-DBus no-UBus no-i18n no-IDN DHCP DHCPv6 no-Lua TFTP no-conntrack ipset no-auth DNSSEC no-ID loop-detect no-inotify no-dumpfile
Nov 24 23:28:45 dnsmasq[9806]: ipset add BBC_WEB3 151.101.126.62 domains.gannett.map.fastly.net
Nov 24 23:28:48 dnsmasq[9806]: ipset add BBC_WEB3 151.101.126.133 na-eu.atlanticmedia.map.fastly.net
Nov 24 23:28:50 dnsmasq[9806]: ipset add BBC_WEB3 151.101.125.111 dualstack.guardian.map.fastly.net
Nov 24 23:34:22 dnsmasq[9806]: ipset add BBC_WEB3 151.101.125.194 confiant-integrations.global.ssl.fastly.netSean Rhodes
Regular Contributor
Can you post the other list also so I can compare? It works most of the time, occasionally however I have to use scmerlin and restart ovpn3 and the dhcp to restart the dnsmasq
Xentrk
Part of the Furniture
This line?
Code:
no-IDN DHCP DHCPv6 no-Lua TFTP no-conntrack ipset no-auth DNSSEC no-ID loop-detect no-inotify no-dumpfile
Code:
Nov 25 11:57:49 dnsmasq[27859]: exiting on receipt of SIGTERM
Nov 25 11:57:50 dnsmasq[20569]: started, version 2.82-34-gb309cca cache disabled
Nov 25 11:57:50 dnsmasq[20569]: compile time options: IPv6 GNU-getopt no-RTC no-DBus no-UBus no-i18n no-IDN DHCP DHCPv6 no-Lua TFTP no-conntrack ipset no-auth DNSSEC no-ID loop-detect no-inotify no-dumpfile
Nov 25 11:57:50 dnsmasq[20569]: warning: interface pptp* does not currently exist
<snip>How is BBC routing working so far?
Sean Rhodes
Regular Contributor
So far all is working well, occasionally it still blocks, hence my question about the above message in the dnsmasq, showing that I had to restart the dns server and ovpn3. It seems that all works fine until a show ends, at that point it tries to autoload the next episode, which usually works. occasionally though I get the black screen with the prohibited icon on the play timeline. Exiting to the main iplayer screen and re-selecting the episode usually works, but on some occassions I have to fully restart. Next time I will try just restarting the ovpn to see if it is just related to the vpn, then try restarting the dns server and not the vpn.
Xentrk
Part of the Furniture
I'll make another pass to see if there are more domains. I only selected a few shows to make sure they played before I stopped mining domains.
Can you also include the ASN for BBC to see if that helps. Looks like I also have it in my nat-start file.
Code:
sh /jffs/scripts/x3mRouting/x3mRouting.sh ipset_name=BBC asnum=AS2818,AS31459The issue with fastly.net is it may also cast too wide of a net since it is a CDN (content delivery network). I want to experiment to see if I can get it to only capture the domain names that are used by bbc and not other sites.
Xentrk
Part of the Furniture
@Sean Rhodes
I stripped it down slightly and removed the fastly.net reference. Tested on Fire TV, Firefox Win 10 and iOS
backup /jffs/scripts/nat-start
Remove the prior entry but retain physical backup file
Create version 4 of BBC_WEB
I stripped it down slightly and removed the fastly.net reference. Tested on Fire TV, Firefox Win 10 and iOS
backup /jffs/scripts/nat-start
Code:
cp /jffs/scripts/nat-start /jffs/scripts/nat-start.bkup_26novRemove the prior entry but retain physical backup file
Code:
x3mRouting ipset_name=BBC_WEB3 del
(x3mRouting): 2946 Starting Script Execution ipset_name=BBC_WEB3 del
Done.
(x3mRouting): 2946 IPSET BBC_WEB3 deleted!
DANGER ZONE!
Delete the backup file in /opt/tmp/BBC_WEB3
[1] --> Yes
[2] --> No
[1-2]: 2
(x3mRouting): 2946 Completed Script ExecutionCreate version 4 of BBC_WEB
Code:
sh /jffs/scripts/x3mRouting/x3mRouting.sh ALL 3 BBC_WEB4 dnsmasq=2cnt.net,bbc.com,bbcverticals.com,co.uk,llnwi.net,net.uk
Last edited:
Sean Rhodes
Regular Contributor
Mine is slightly different than yours, maybe due to running the 384.18 firmware version, but here is my output:
Code:
rhodess@RT-AC3200-4200:/tmp/home/root# cp /jffs/scripts/nat-start /jffs/scripts/nat-start.bkup_25nov
rhodess@RT-AC3200-4200:/tmp/home/root# x3mRouting ipset_name=BBC_WEB3 del
(x3mRouting): 25462 Starting Script Execution ipset_name=BBC_WEB3 del
(x3mRouting): 25462 IPSET BBC_WEB3 deleted from /jffs/configs/dnsmasq.conf.add
Done.
(x3mRouting): 25462 Script entry for BBC_WEB3 deleted from /jffs/scripts/nat-start
/jffs/scripts/nat-start has 1 shebang entry and 0 empty lines.
Would you like to remove ?
[1] --> Yes
[2] --> No
[1-2]: 2
(x3mRouting): 25462 ipset BBC_WEB3 entry deleted from /jffs/scripts/x3mRouting/vpnclient3-route-up
/jffs/scripts/x3mRouting/vpnclient3-route-up has 1 shebang entry and 0 empty lines.
Would you like to remove ?
[1] --> Yes
[2] --> No
[1-2]: 2
(x3mRouting): 25462 ipset BBC_WEB3 entry deleted from /jffs/scripts/x3mRouting/vpnclient3-route-pre-down
/jffs/scripts/x3mRouting/vpnclient3-route-pre-down has 1 shebang entry and 0 empty lines.
Would you like to remove ?
[1] --> Yes
[2] --> No
[1-2]: n
[*] n Isn't An Option!
0 2 * * * ipset save BBC_WEB3 > /opt/tmp/BBC_WEB3 #BBC_WEB3#
(x3mRouting): 25462 CRON schedule deleted: #BBC_WEB3# '0 2 * * * ipset save BBC_WEB3'
(x3mRouting): 25462 IPSET BBC_WEB3 deleted!
(x3mRouting): 25462 Completed Script Execution
rhodess@RT-AC3200-4200:/tmp/home/root# sh /jffs/scripts/x3mRouting/x3mRouting.sh ALL 3 BBC_WEB4 dnsmasq=2cnt.net,bbc.com,b
bcverticals.com,co.uk,llnwi.net,net.uk
(x3mRouting.sh): 30770 Starting Script Execution ALL 3 BBC_WEB4 dnsmasq=2cnt.net,bbc.com,bbcverticals.com,co.uk,llnwi.net,net.uk
(x3mRouting.sh): 30770 ipset=/2cnt.net/bbc.com/bbcverticals.com/co.uk/llnwi.net/net.uk/BBC_WEB4 added to /jffs/configs/dnsmasq.conf.add
Done.
(x3mRouting.sh): 30770 IPSET created: BBC_WEB4 hash:net family inet hashsize 1024 maxelem 65536
(x3mRouting.sh): 30770 CRON schedule created: #BBC_WEB4# '0 2 * * * ipset save BBC_WEB4'
(x3mRouting.sh): 30770 Selective Routing Rule via VPN Client 3 created for BBC_WEB4 fwmark 0x4000/0x4000
(x3mRouting.sh): 30770 iptables -t mangle -D PREROUTING -i br0 -m set --match-set BBC_WEB4 dst -j MARK --set-mark 0x4000/0x4000 2>/dev/null added to /jffs/scripts/x3mRouting/vpnclient3-route-up
(x3mRouting.sh): 30770 iptables -t mangle -A PREROUTING -i br0 -m set --match-set BBC_WEB4 dst -j MARK --set-mark 0x4000/0x4000 added to /jffs/scripts/x3mRouting/vpnclient3-route-up
(x3mRouting.sh): 30770 iptables -t mangle -D PREROUTING -i br0 -m set --match-set BBC_WEB4 dst -j MARK --set-mark 0x4000/0x4000 2>/dev/null added to /jffs/scripts/x3mRouting/vpnclient3-route-pre-down
(x3mRouting.sh): 30770 sh /jffs/scripts/x3mRouting/x3mRouting.sh ALL 3 BBC_WEB4 dnsmasq=2cnt.net,bbc.com,bbcverticals.com,co.uk,llnwi.net,net.uk added to /jffs/scripts/nat-start
(x3mRouting.sh): 30770 Completed Script ExecutionThanks
Xentrk
Part of the Furniture
Excellent! I did some more surfing of iPlayer on Fire TV and now have up to 125 IPv4 addresses collected so far. It is working without using the ASN for BBC.
Sean Rhodes
Regular Contributor
That's good news
Sean Rhodes
Regular Contributor
So far I haven't seen what's disrupting mine, it seems no matter what I do, every 3rd show I have to restart my dnsmasq and ovpn. Usually, that works, but lately, I had to exit and re-enter iPlayer to get it to play
Xentrk
Part of the Furniture
I recommend doing some analysis using the scripts available in option 4. I would start by routing all traffic to the VPN client you use for BBC and make sure iPlayer works okay that way. Then, use the autoscan.sh script and search for "uk" or "bbc" terms. The getdomainnames.sh script will give you a big picture view of what is being queried.
Sean Rhodes
Regular Contributor
I will do. Ideally I would like to shutdown everything except the one machine I’m using and on that have nothing except iplayer running and then log the files. It will have to be an early morning chore though otherwise I get too many wife and son complaints when I shut things down. Ideally I should grab another router and setup a different network and test on that I think.
abracadabra11
Regular Contributor
How does one install an older version of x3m? Can I just use the curl command and replace the master with desired version?
So for version 2.3, replace this:
With this?
Thanks!
Want to stick with Merlin 384.18 for now since this configuration was working great.
So for version 2.3, replace this:
Code:
sh -c "$(curl -sL https://raw.githubusercontent.com/Xentrk/x3mRouting/master/Install_x3mRouting.sh)"With this?
Code:
sh -c "$(curl -sL https://raw.githubusercontent.com/Xentrk/x3mRouting/x3mRouting-2.3.0/Install_x3mRouting.sh)"Thanks!
Want to stick with Merlin 384.18 for now since this configuration was working great.
Sean Rhodes
Regular Contributor
abracadabra11
Regular Contributor
What's thread 14 and fit page?
Xentrk
Part of the Furniture
I saved the compatible version in a branch:
GitHub - Xentrk/x3mRouting at x3mRouting-384.18
Selectively route LAN clients, website or streaming media traffic over the WAN or OpenVPN client interfaces on Asuswrt-Merlin firmware - GitHub - Xentrk/x3mRouting at x3mRouting-384.18
github.com
Installation
Code:
sh -c "$(curl -sL https://raw.githubusercontent.com/Xentrk/x3mRouting/x3mRouting-384.18/Install_x3mRouting.sh)"Will work with 384.13 to 384.18 firmware versions.
Dear Xentrk,
for a few days, I have problems with populating my amazon prime video rules. In my natstart the following entries are there:
sh /jffs/scripts/x3mRouting/x3mRouting.sh 1 0 AMAZON-EU aws_region=EU
sh /jffs/scripts/x3mRouting/x3mRouting.sh 1 0 AMAZON-US aws_region=US
sh /jffs/scripts/x3mRouting/x3mRouting.sh 1 0 AMAZON-CA aws_region=CA
sh /jffs/scripts/x3mRouting/x3mRouting.sh 1 0 AMAZON-AP aws_region=AP
sh /jffs/scripts/x3mRouting/x3mRouting.sh 1 0 AMAZON-CN aws_region=CN
sh /jffs/scripts/x3mRouting/x3mRouting.sh 1 0 AMAZON-GLOBAL aws_region=GLOBAL
sh /jffs/scripts/x3mRouting/x3mRouting.sh 1 0 AMAZON-SA aws_region=SA
My router reboots everyday as a cron job. As I recognized , that prime video has sometimes problems, I checked the liststats.
I found out, that sometimes the lists are populating very differently. Sometimes only the EU List is populated (532 entries), sometimes Global (108) and EU and sometimes other regions are populated as well. I rebootet the router a lot of times and I am getting different population. Do you have an idea, why that is occuring ? Could my DNS Server sometimes filter, so that not all lists are populated ?
Forward local domain queries to upstream DNS --> No , using quad9 as dns servers.
Netflix is working normally.
Is there a command for forcing a new population of the iprules (so that I don´t have to reboot everytime the router for testing)? Is it possible to add a cron job that the router is trying to repopulate after e.g. 5 Minutes of rebooting again and add the missing entries ?
Thanks a lot for your support.
Hugo
PS: The best "population" I recieved after reboot was (and that was working perfectly):
AMAZON-AP - 579
AMAZON-CA - 61
AMAZON-CN - 93
AMAZON-EU - 532
AMAZON-GLOBAL - 108
AMAZON-SA - 75
AMAZON-US - 963
for a few days, I have problems with populating my amazon prime video rules. In my natstart the following entries are there:
sh /jffs/scripts/x3mRouting/x3mRouting.sh 1 0 AMAZON-EU aws_region=EU
sh /jffs/scripts/x3mRouting/x3mRouting.sh 1 0 AMAZON-US aws_region=US
sh /jffs/scripts/x3mRouting/x3mRouting.sh 1 0 AMAZON-CA aws_region=CA
sh /jffs/scripts/x3mRouting/x3mRouting.sh 1 0 AMAZON-AP aws_region=AP
sh /jffs/scripts/x3mRouting/x3mRouting.sh 1 0 AMAZON-CN aws_region=CN
sh /jffs/scripts/x3mRouting/x3mRouting.sh 1 0 AMAZON-GLOBAL aws_region=GLOBAL
sh /jffs/scripts/x3mRouting/x3mRouting.sh 1 0 AMAZON-SA aws_region=SA
My router reboots everyday as a cron job. As I recognized , that prime video has sometimes problems, I checked the liststats.
I found out, that sometimes the lists are populating very differently. Sometimes only the EU List is populated (532 entries), sometimes Global (108) and EU and sometimes other regions are populated as well. I rebootet the router a lot of times and I am getting different population. Do you have an idea, why that is occuring ? Could my DNS Server sometimes filter, so that not all lists are populated ?
Forward local domain queries to upstream DNS --> No , using quad9 as dns servers.
Netflix is working normally.
Is there a command for forcing a new population of the iprules (so that I don´t have to reboot everytime the router for testing)? Is it possible to add a cron job that the router is trying to repopulate after e.g. 5 Minutes of rebooting again and add the missing entries ?
Thanks a lot for your support.
Hugo
PS: The best "population" I recieved after reboot was (and that was working perfectly):
AMAZON-AP - 579
AMAZON-CA - 61
AMAZON-CN - 93
AMAZON-EU - 532
AMAZON-GLOBAL - 108
AMAZON-SA - 75
AMAZON-US - 963
Last edited:
Installed this, only first option gets installed the other 3 throws out an error, as in expected package size is differentI saved the compatible version in a branch:
GitHub - Xentrk/x3mRouting at x3mRouting-384.18
Selectively route LAN clients, website or streaming media traffic over the WAN or OpenVPN client interfaces on Asuswrt-Merlin firmware - GitHub - Xentrk/x3mRouting at x3mRouting-384.18
Installation
Code:sh -c "$(curl -sL https://raw.githubusercontent.com/Xentrk/x3mRouting/x3mRouting-384.18/Install_x3mRouting.sh)"
Will work with 384.13 to 384.18 firmware versions.
Similar threads
Similar threads
Similar threads
-
How do I get x3mrouting working with AdGuard Home ?
- Started by ComputerSteve
- Replies: 6
-
x3mRouting x3mRouting vs Domain based VPN routing
- Started by Armandooooo
- Replies: 29
-
DomainVPNRouting Domain VPN Routing v3.0.3 ***Release***
- Started by Ranger802004
- Replies: 45
-
Question about DomainBased Routing & Re-connection
- Started by ComputerSteve
- Replies: 0
-
-
DomainVPNRouting Domain VPN Routing Question
- Started by Armandooooo
- Replies: 32
-
DomainVPNRouting Domain-based VPN routing script
- Started by johndoe85
- Replies: 2
-