Selective Routing for Netflix
- Thread starter Xentrk
- Start date
I'm getting file not found when trying to tail the log
Tried navigating to /var/opt/log - same message.
Tried the troubleshoot commands, but can't really make sense what I'm looking for. I get IP list when running
ipset -L x3mRouting_NETFLIX.
Any clues?
Overall trying to add BBC and Sky Go IPs if anyone knows of any JSON lists to add.
Edit: added the file manually using the touch command, still unable to tail.
/tmp/home/root# tail -f dnsmasq.log
tail: can't open 'dnsmasq.log': No such file or directory
tail: no files
tail: can't open 'dnsmasq.log': No such file or directory
tail: no files
Tried navigating to /var/opt/log - same message.
Tried the troubleshoot commands, but can't really make sense what I'm looking for. I get IP list when running
ipset -L x3mRouting_NETFLIX.
Any clues?
Overall trying to add BBC and Sky Go IPs if anyone knows of any JSON lists to add.
Edit: added the file manually using the touch command, still unable to tail.
tail -f dnsmasq.log > Netflix
-sh: can't create Netflix: Read-only file system
-sh: can't create Netflix: Read-only file system
Last edited:
Installed this Netflix-VPN-Bypass today (nice job Xentrk). Works well except that once I reboot my Asus RT-AC86U router Netflix gets blocked again. I have to issue command (sh /jffs/scripts/IPSET_Netflix.sh) on SSH terminal to make this Netflix-VPN-Bypass work again. How to fix this problem?
Btw, Diversion, DNSCrypt & Skynet are installed in my router. WAN DNS is set manually using 1.1.1.1 & 1.0.0.1. OpenVPN client with IPVanish & DNS config is set to Disabled & redirect internet traffic to Policy Rules (Strict).
nano /jffs/scripts/nat-start - opens the file in text editor
Enter this:
Code:
#!/bin/sh
sh /jffs/scripts/IPSET_Netflix.shctrl + x -exits text editor
yes -saves the file
chmod 755 /jffs/scripts/nat-start -makes the file executable
reboot
wait 5 mins for script to load and retry
That fix it man. Thanks a lot.
No problem. Hopefully someone can help with my post now...
Martineau
Part of the Furniture
Do you actually have logging directives in dnsmasq?
/etc/dnsmasq.conf
Code:
log-async
log-queries
log-facility=/opt/var/log/dnsmasq.loge.g. /jffs/scripts/post-mount
Code:
if [ ! -f /jffs/scripts/post-mount.div ];then # Let Diversion handle Entware
echo -e
logger -st "($(basename $0))" $$ "Mounting Entware '"$1"/entware' on '/tmp/opt'....."
ln -nsf $1/entware /tmp/opt
fiYes.
That was missing, so I've now added it to the last 3 lines of the file. - rebooted, and still fails.
I'm not sure with this... i've ran /jffs/scripts/post-mount and nothing happens.
I can no longer cd ...
tmp/home/root# cd /var/opt/log
-sh: cd: can't cd to /var/opt/log[/QUOTE]
Martineau
Part of the Furniture
Did you correctly add the three lines to '/jffs/configs/dnsmasq.conf.add' then restart dnsmasq?
Code:
service restart_dnsmasqSo you have installed Entware ...Yes?
If you haven't installed Entware, then simply change 'log-facility=/a_directory_that_exists on_your_mounted_USB/dnsmasq.log' then tail/browse that file.
I was a little confused by your post (I'm a n00b), but think I got it working thanks. I can now tail dnsmasq.log
I did install Entware originally (as per OP).
This is what I done... Hopefully correct!
Code:
nano /jffs/configs/dnsmasq.conf.add
*insert*
log-async
log-queries
log-facility=/opt/var/log/dnsmasq.log
ctrl + x
yes
Enter
service restart_dnsmasqNow to tail and find Sky Go and BBC
So I've tried logging Skygo and all i'm seeing is Nest and Microsoft DNS queries... any ideas?
Code:
Dec 10 20:08:39 dnsmasq[5168]: query[A] logsink.devices.nest.com from 192.168.1.118
Dec 10 20:08:39 dnsmasq[5168]: forwarded logsink.devices.nest.com to 1.1.1.1
Dec 10 20:08:39 dnsmasq[5168]: validation result is INSECURE
Dec 10 20:08:39 dnsmasq[5168]: reply logsink.devices.nest.com is 35.190.54.210
Dec 10 20:08:51 dnsmasq[5168]: query[A] dns.msftncsi.com from 127.0.0.1
Dec 10 20:08:51 dnsmasq[5168]: cached dns.msftncsi.com is 131.107.255.255
Dec 10 20:09:06 dnsmasq[5168]: query[A] dns.msftncsi.com from 127.0.0.1
Dec 10 20:09:06 dnsmasq[5168]: forwarded dns.msftncsi.com to 1.1.1.1
Dec 10 20:09:06 dnsmasq[5168]: validation result is INSECURE
Dec 10 20:09:06 dnsmasq[5168]: reply dns.msftncsi.com is 131.107.255.255
Dec 10 20:09:21 dnsmasq[5168]: query[A] dns.msftncsi.com from 127.0.0.1
Dec 10 20:09:21 dnsmasq[5168]: cached dns.msftncsi.com is 131.107.255.255
Dec 10 20:09:36 dnsmasq[5168]: query[A] dns.msftncsi.com from 127.0.0.1
Dec 10 20:09:36 dnsmasq[5168]: forwarded dns.msftncsi.com to 1.1.1.1
Dec 10 20:09:36 dnsmasq[5168]: validation result is INSECURE
Dec 10 20:09:36 dnsmasq[5168]: reply dns.msftncsi.com is 131.107.255.255Martineau
Part of the Furniture
Works for me..
e.g. Fired up the SkyGo app on my laptop 10.88.8.111
Code:
grep -F "21:45" /opt/var/log/dnsmasq.log | grep sky >sky.log
Code:
Dec 10 21:45:15 dnsmasq[3225]: query[A] init.sky.com from 10.88.8.111
Dec 10 21:45:15 dnsmasq[3225]: forwarded init.sky.com to 79.79.79.78
Dec 10 21:45:15 dnsmasq[3225]: reply init.sky.com is <CNAME>
Dec 10 21:45:15 dnsmasq[3225]: reply init.sky.com.edgekey.net is <CNAME>
Dec 10 21:45:17 dnsmasq[3225]: query[A] desktop.client.ott.sky.com from 10.88.8.111
Dec 10 21:45:17 dnsmasq[3225]: forwarded desktop.client.ott.sky.com to 79.79.79.78
Dec 10 21:45:17 dnsmasq[3225]: reply desktop.client.ott.sky.com is <CNAME>
Dec 10 21:45:17 dnsmasq[3225]: reply desktop.client.ott.sky.com.edgekey.net is <CNAME>
Dec 10 21:45:22 dnsmasq[3225]: query[A] config.ott.sky.com from 10.88.8.111
Dec 10 21:45:22 dnsmasq[3225]: forwarded config.ott.sky.com to 79.79.79.78
Dec 10 21:45:22 dnsmasq[3225]: reply config.ott.sky.com is <CNAME>
Dec 10 21:45:22 dnsmasq[3225]: reply config.ott.sky.com.edgekey.net is <CNAME>
Dec 10 21:45:24 dnsmasq[3225]: query[A] awk.epgsky.com from 10.88.8.111
Dec 10 21:45:24 dnsmasq[3225]: forwarded awk.epgsky.com to 79.79.79.78
Dec 10 21:45:24 dnsmasq[3225]: reply awk.epgsky.com is <CNAME>
Dec 10 21:45:24 dnsmasq[3225]: reply awk.epgsky.com.edgekey.net is <CNAME>
Dec 10 21:45:24 dnsmasq[3225]: query[A] init.sky.com from 10.88.8.111
Dec 10 21:45:24 dnsmasq[3225]: cached init.sky.com is <CNAME>
Dec 10 21:45:24 dnsmasq[3225]: cached init.sky.com.edgekey.net is <CNAME>
Dec 10 21:45:24 dnsmasq[3225]: query[A] p.sky.com from 10.88.8.111
Dec 10 21:45:24 dnsmasq[3225]: forwarded p.sky.com to 79.79.79.78
Dec 10 21:45:24 dnsmasq[3225]: reply p.sky.com is <CNAME>
Dec 10 21:45:24 dnsmasq[3225]: reply p.sky.com.edgekey.net is <CNAME>
Dec 10 21:45:26 dnsmasq[3225]: query[A] config.skyq-b.interactive.sky.com from 10.88.8.111
Dec 10 21:45:26 dnsmasq[3225]: forwarded config.skyq-b.interactive.sky.com to 79.79.79.78
Dec 10 21:45:26 dnsmasq[3225]: reply config.skyq-b.interactive.sky.com is <CNAME>
Dec 10 21:45:26 dnsmasq[3225]: reply config.skyq-b.interactive.sky.com.edgekey.net is <CNAME>
Dec 10 21:45:27 dnsmasq[3225]: query[A] config.ott.sky.com from 10.88.8.111
Dec 10 21:45:27 dnsmasq[3225]: cached config.ott.sky.com is <CNAME>
Dec 10 21:45:27 dnsmasq[3225]: cached config.ott.sky.com.edgekey.net is <CNAME>
Dec 10 21:45:27 dnsmasq[3225]: forwarded config.ott.sky.com to 79.79.79.78
Dec 10 21:45:27 dnsmasq[3225]: forwarded config.ott.sky.com to 79.79.79.77
Dec 10 21:45:27 dnsmasq[3225]: forwarded config.ott.sky.com to 127.0.0.1
Dec 10 21:45:27 dnsmasq[3225]: reply config.ott.sky.com is <CNAME>
Dec 10 21:45:27 dnsmasq[3225]: reply config.ott.sky.com.edgekey.net is <CNAME>
Dec 10 21:45:27 dnsmasq[3225]: query[A] entity.search.sky.com from 10.88.8.111
Dec 10 21:45:27 dnsmasq[3225]: forwarded entity.search.sky.com to 79.79.79.78
Dec 10 21:45:27 dnsmasq[3225]: reply entity.search.sky.com is <CNAME>
Dec 10 21:45:27 dnsmasq[3225]: reply freeflow2.entity.search.sky.com.edgesuite.net is <CNAME>
Dec 10 21:45:28 dnsmasq[3225]: query[A] config.skyq-b.interactive.sky.com from 10.88.8.111
Dec 10 21:45:28 dnsmasq[3225]: cached config.skyq-b.interactive.sky.com is <CNAME>
Dec 10 21:45:28 dnsmasq[3225]: cached config.skyq-b.interactive.sky.com.edgekey.net is <CNAME>
Dec 10 21:45:28 dnsmasq[3225]: forwarded config.skyq-b.interactive.sky.com to 79.79.79.78
Dec 10 21:45:28 dnsmasq[3225]: reply config.skyq-b.interactive.sky.com is <CNAME>
Dec 10 21:45:28 dnsmasq[3225]: reply config.skyq-b.interactive.sky.com.edgekey.net is <CNAME>
Dec 10 21:45:28 dnsmasq[3225]: query[A] images.metadata.sky.com from 10.88.8.111
Dec 10 21:45:28 dnsmasq[3225]: forwarded images.metadata.sky.com to 79.79.79.78
Dec 10 21:45:28 dnsmasq[3225]: reply images.metadata.sky.com is <CNAME>
Dec 10 21:45:28 dnsmasq[3225]: reply prdpdimages.opg.sky.com is <CNAME>
Dec 10 21:45:28 dnsmasq[3225]: reply images.metadata.sky.com.edgekey.net is <CNAME>
Dec 10 21:45:28 dnsmasq[3225]: query[A] analytics.faw.sky.com from 10.88.8.111
Dec 10 21:45:28 dnsmasq[3225]: /opt/share/diversion/list/blockinglist analytics.faw.sky.com is 10.88.8.3
Dec 10 21:45:29 dnsmasq[3225]: query[A] static.skyq-b.interactive.sky.com from 10.88.8.111
Dec 10 21:45:29 dnsmasq[3225]: forwarded static.skyq-b.interactive.sky.com to 79.79.79.78
Dec 10 21:45:29 dnsmasq[3225]: reply static.skyq-b.interactive.sky.com is <CNAME>
Dec 10 21:45:29 dnsmasq[3225]: reply static.skyq-b.interactive.sky.com.edgekey.net is <CNAME>
Dec 10 21:45:29 dnsmasq[3225]: query[A] metrics.sky.com from 10.88.8.111
Dec 10 21:45:29 dnsmasq[3225]: /opt/share/diversion/list/blockinglist metrics.sky.com is 10.88.8.3
Dec 10 21:45:30 dnsmasq[3225]: query[A] uk.imageservice.sky.com from 10.88.8.111
Dec 10 21:45:30 dnsmasq[3225]: forwarded uk.imageservice.sky.com to 79.79.79.78
Dec 10 21:45:30 dnsmasq[3225]: reply uk.imageservice.sky.com is <CNAME>
Dec 10 21:45:30 dnsmasq[3225]: reply uk.imageservice.sky.com.edgekey.net is <CNAME>
Dec 10 21:45:33 dnsmasq[3225]: query[A] qgo-desktop-sgw.skyanywhere.com from 10.88.8.111
Dec 10 21:45:33 dnsmasq[3225]: forwarded qgo-desktop-sgw.skyanywhere.com to 79.79.79.78
Dec 10 21:45:33 dnsmasq[3225]: reply qgo-desktop-sgw.skyanywhere.com is <CNAME>
Dec 10 21:45:33 dnsmasq[3225]: reply ott-sgw.skyanywhere.com is 90.211.176.31Are you sure the device you are using the SkyGo app on is actually using your LAN and is not bypassing the router's dnsmasq?
Thanks again...
Yeah using SkyGo app on my Macbook connected to router via WiFi. I disable VPN and then remove the device from the VPN policy rules (to bypass the kill switch).
I wonder if some sort of IP rule is preventing it being logged? But then I do see some services with my devices IP, so strange.
I see nothing ref sky in the log.
What is the "validation result is INSECURE" referencing?
Cheers again
Martineau
Part of the Furniture
Usually it is best to not use the VPN DNS while attempting to collate the domains, unless that is what is desired.
I doubt it...dnsmasq simply responds to a request, and with the three logging directives ACTIVE, simply writes the request/results to the file so how why/would a firewall rule block the write to disk?
Search the forum? but basically don't use Cloudflare DNS.....
e.g. Cloudflare is quirky for some users but not all....
Cloudflare DNSSEC INSECURE?
Never used a Mac, so try a proper
computer/tablet to see if the SkyGo app works on those and only uses the router for its DNS requests.Hint: Perhaps explicitly configure the DNSFILTER option to force the Mac to use the router DNS or even Google DNS for testing?
Xentrk
Part of the Furniture
Sorry for not helping sooner. My laptop was in the shop for an extended amount of time for repair.Now to tail and find Sky Go and BBC
I have harvested the IPv4 addresses for BBC. It was the hardest site I've mined. I had to use a combination of techniques. I will post them on GitHub and update this post with the link.
UPDATE
There are three IPv4 files for BBC on the site https://github.com/Xentrk/Asuswrt-Merlin-Selective-Routing/tree/master.
They are:
You need all three for BBC to work. The first list is from the AS number for BBC. The BBC_IPLAYER list was created by collecting the hostnames from dnsmasq on the router and doing a lookup to get the IPv4 address. The BBC_WEB list was generate using the feature of IPSET to dynamically generate and collect the IPv4 addresses when using a web browser and iPad client. I have found that using a browser or iPad App to watch some streaming media services generates additional domains or IPv4 addresses when compared to a streaming media device.
Last edited:
My DNS was Cloudflare - I don't use the VPN's DNS
Thanks, I did search DDG (not directly on SNB and didn't find anything (but also never thought to link it Cloudflare for some reason)
Ah thanks! I think I missed a trick here - I had this OFF. I've now switched to Router Mode. Hopefully that'll help. I had Static routes setup to avoid 8.8.8.8 on my Chromecast, but never looked at the DNSFILTER.
Last edited:
I appreciate this thanks! I am a little stuck on how to add this to the Script though... Could you point me in the right direction how to pull these IP's from GitHub
I also notice (when trying to figure out the above) there is US region settings in the IPSET_Netflix.sh for pulling AWS JSON IP's... Should users in the EU etc be changing this setting?
Is worth adding SkyGo to you GitHub as listed here...?
Xentrk
Part of the Furniture
@CMc
Sky Go IPv4 address can be found here:
https://bgp.he.net/AS5607
However, additional mining of using features of dnsmasq may be required if things don't work at first.
Sky Go IPv4 address can be found here:
https://bgp.he.net/AS5607
However, additional mining of using features of dnsmasq may be required if things don't work at first.
Xentrk
Part of the Furniture
You can use curl command to download them manually for now. e.g.
Code:
/usr/sbin/curl --retry 3 "https://raw.githubusercontent.com/Xentrk/Asuswrt-Merlin-Selective-Routing/master/x3mRouting_BBC" -o "/opt/tmp/x3Routing_BBC"The IPv4 list is static. So no need to hammer GitHub with frequent downloads. I originally setup the Repo for my more comprehensive selective routing project. The project took a few detours and I had to place it on hold. I want to pick it back up after the holidays are over.
You can disable the filter for US if you have issues. I would be curious to know if you do. Using the US region worked for one of the forum members from an EU country that used the script to bypass the VPN. I get the impression that NF is hosted in the AWS data center in US and CDN is used for world wide delivery.
EDIT: Updated code to include more efficient loading of IPSET list
Here is a code snip from earlier version where I was not filtering for US region.
Code:
Chk_Entware 'jq' || { echo -e "\a***ERROR*** Entware" $ENTWARE_UTILITY "not available";exit 99; }
# Download Amazon AWS json file
wget https://ip-ranges.amazonaws.com/ip-ranges.json -O /opt/tmp/ip-ranges.json
# Create IPSET lists
ipset create AMAZONAWS hash:net family inet hashsize 1024 maxelem 65536
#Pull all IPv4s listed for Amazon AWS
jq -r '.prefixes | .[].ip_prefix' < /opt/tmp/ip-ranges.json > /opt/tmp/AmazonAWS
ipset flush AMAZONAWS
awk '{print "add AMAZONAWS " $1}' /opt/tmp/AmazonAWS | ipset restore -!I can add the Sky Go if you confirm it works. I don't want to pay for a subscription so I have no way to test on my end.
Xentrk
Part of the Furniture
You can run https://github.com/Xentrk/netflix-vpn-bypass/blob/master/getdomainnames.sh to format the file into a list of domain names that are sorted with dupes removed. I'm back home for a few days before traveling again and have some time to do it tomorrow. Just wanted you to be aware of the script to clean up the ouput dnsmasq produces. See the README.md for more info.
Xentrk
Part of the Furniture
@CMc
Regarding the download of the files from github. Since they are static files, you can do a one time download to a location on the router. Once you download the files, you are all set. No need to download on a regular frequency like I do on the Netflix and AmazonAWS json files. I recommend an location on the /opt directory. Optional location is /jffs. /opt has several sub-directories. Right now, I am storing the Netflix file in /opt/tmp in the current script. Hint: pending changes to the directory location are coming soon!
Code overview changes update to use BBC.
1. Starting with cloning/copying the code functions
2. update the function
3. Below these lines...
Regarding the download of the files from github. Since they are static files, you can do a one time download to a location on the router. Once you download the files, you are all set. No need to download on a regular frequency like I do on the Netflix and AmazonAWS json files. I recommend an location on the /opt directory. Optional location is /jffs. /opt has several sub-directories. Right now, I am storing the Netflix file in /opt/tmp in the current script. Hint: pending changes to the directory location are coming soon!
Code overview changes update to use BBC.
1. Starting with cloning/copying the code functions
- check_netflix_ipset_list_exist ()
- check_netflix_ipset_list_values ()
2. update the function
- create_routing_rules
Code:
iptables -t mangle -D PREROUTING -i br0 -m set --match-set x3mRouting_NETFLIX dst -j MARK --set-mark "$FWMARK_WAN" > /dev/null 2>&1
iptables -t mangle -A PREROUTING -i br0 -m set --match-set x3mRouting_NETFLIX dst -j MARK --set-mark "$FWMARK_WAN"3. Below these lines...
- check_amazonaws_ipset_list_values
- check_amazonaws_ipset_list_exist
Similar threads
DomainVPNRouting
Domain VPN Routing v3.0.1 ***Release***
Similar threads
| Thread starter | Title | Forum | Replies | Date |
|---|---|---|---|---|
| H | Routing wireguard | VPN | 0 |
Similar threads
Latest threads
-
ROG 98 Pro: Fire Stick 4K Max won’t connect to WiFi 6E???
- Started by Poseidon
- Replies: 2
-
-
Help with RT-ac88u network switch issues - causing reboot
- Started by rkalinka
- Replies: 0
-
TP-Link routers are being taken over by botnet
- Started by TheLostSwede
- Replies: 3
-
RT-AX86U Pro - Merlin: can't get p2p cameras available to wan.
- Started by mightyoakbob
- Replies: 3
Support SNBForums w/ Amazon
If you'd like to support SNBForums, just use this link and buy anything on Amazon. Thanks!
Sign Up For SNBForums Daily Digest
Get an update of what's new every day delivered to your mailbox. Sign up here!