robahearts
Regular Contributor
I tested it and it works but once I reboot the router the rules are not staying. Should I be calling the script from nat-start?
Selective Routing for Netflix
- Thread starter Xentrk
- Start date
I tested it and it works but once I reboot the router the rules are not staying. Should I be calling the script from nat-start?
Xentrk
Part of the Furniture
Great. I wonder what the issue was before.
Call the script from /jffs/scripts/nat-start:
Code:
#!/bin/sh
sh /jffs/scripts/IPSET_Netflix.shIt will then handle reboots, openvpn changes, etc... remember to make nat-start executable
Last edited:
robahearts
Regular Contributor
This is weird. I added it to ne-start and after rebooting I can see everything is added but I can't use Netflix or Amazon. If I run the script manually then I can access Amazon and Netflix.
Log is here https://pastebin.com/yZDHV8HC
Last edited:
robahearts
Regular Contributor
So decided to start from the begining. Formarted /jffs/scripts/ then
- Setup VPN
- Setup AB-Solution
- Setup Skynet
- Setup Entware and package "jq"
- Added the steps to call script from /jffs/scripts/nat-start
Rebooted and no Netflix/Amazon
Amazon
Netflix
IP Rule
iptables -nvL PREROUTING -t mangle --line
I manually run the script ./IPSET_Netflix3.sh and the Amazon address get added and I can play videos from Amazon and Netflix.
I then reboot. Netflix address are there but I can’t play anything. Amazon address are missing and I can’t play anything.
I run ./nat-start manually and everything works until the next reboot.
- Setup VPN
- Setup AB-Solution
- Setup Skynet
- Setup Entware and package "jq"
- Added the steps to call script from /jffs/scripts/nat-start
Rebooted and no Netflix/Amazon
Amazon
Netflix
IP Rule
iptables -nvL PREROUTING -t mangle --line
I manually run the script ./IPSET_Netflix3.sh and the Amazon address get added and I can play videos from Amazon and Netflix.
I then reboot. Netflix address are there but I can’t play anything. Amazon address are missing and I can’t play anything.
I run ./nat-start manually and everything works until the next reboot.
Xentrk
Part of the Furniture
Need to verify that nat-start is being run at boot or if you bounce the WAN iface.
If you can run nat-start from the command line, then it must be executable. Execute the "chmod 755 nat-start" command just to make sure. That is how I have my permissions set. Make sure you are calling the correct script in nat-start. In the post above I used sh /jffs/scripts/IPSET_Netflix.sh as an example. But the revised script I posted is called IPSET_Netflix3.sh.
Go to the WAN page and select the Apply button on the bottom of the page. Then, check the system log to see if nat-start was called. Navigate to the System Log tab. In the web browser, do a Ctrl-F to open up a search on the page and search for nat-start. Here is a snip from mine.
Code:
Feb 26 01:25:22 start_nat_rules: apply the nat_rules(/tmp/nat_rules_ppp0_eth0)!
Feb 26 01:25:22 custom_script: Running /jffs/scripts/nat-start
Feb 26 01:25:23 (VPN_Routing2.sh): 27371 Starting... /jffs/scripts/VPN_Routing2.sh.
Last edited:
robahearts
Regular Contributor
Is calling the script when I bounce it but I can't play Netflix or Amazon
It doesn't add the address for Amazon
I can see address for NETFLIX but I can't play anything. If I manually call ./nat-start then everything works until the next reboot.
Mar 2 00:39:49 (IPSET_Netflix.sh): 4767 Starting IPSET_Netflix.sh... /jffs/scripts/IPSET_Netflix.sh.
Mar 2 00:39:52 (IPSET_Netflix.sh): 4767 Ending IPSET_Netflix.sh... /jffs/scripts/IPSET_Netflix.sh.
Xentrk
Part of the Furniture
In the OpenVPN client near the bottom of the page, there is the option to drop traffic if the tunnel goes down. Change it to No and test. Let me know what happens.
robahearts
Regular Contributor
Same issue. After rebooting ipset list AMAZONAWS doesnt show any IPs
Merlin@RT-AC86U-CE18:/tmp/home/root# ipset list AMAZONAWS
Name: AMAZONAWS
Type: hash:net
Revision: 6
Header: family inet hashsize 1024 maxelem 65536
Size in memory: 352
References: 1
Number of entries: 0
Members:
I have to manually run the script to get it to work
System logs shows the script running after rebooting.
Maybe the script is running too early?
Xentrk
Part of the Furniture
Thanks for the help @Martineau. That makes sense. I will add a check for entware and post and update. I won’t be able to work on it until about 12 hrs from now. @robahearts will need to add the sleep 10 for now as a temporary work around. If believe @thelonelycoder has a check in his code for entware being mounted that I can incorporate into the script.
thelonelycoder
Part of the Furniture
Just test for opkg and wait for it to be true, something like this:
Code:
i=10
until [ "$(which opkg)" ]; do
i=$(($i-1))
if [ "$i" -lt "1" ];then
logger "Unable to start whatever for the reason ...."
exit
fi
sleep 1
done
Martineau
Part of the Furniture
But what if the required Entware utility i.e. in this case 'jq' isn't installed?
e.g.
Code:
Chk_Entware || { echo -e "\a***ERROR*** Entware not available";exit 99; }
or
Chk_Entware 'jq' || { echo -e "\a***ERROR*** Entware" $ENTWARE_UTILITY "not available";exit 99; }
Code:
Chk_Entware () {
# ARGS [wait attempts] [specific_entware_utility]
local READY=1 # Assume Entware Utilities are NOT available
local ENTWARE="opkg"
ENTWARE_UTILITY= # Specific Entware utility to search for
local MAX_TRIES=30
if [ ! -z "$2" ] && [ ! -z "$(echo $2 | grep -E '^[0-9]+$')" ];then
local MAX_TRIES=$2
fi
if [ ! -z "$1" ] && [ -z "$(echo $1 | grep -E '^[0-9]+$')" ];then
ENTWARE_UTILITY=$1
else
if [ -z "$2" ] && [ ! -z "$(echo $1 | grep -E '^[0-9]+$')" ];then
MAX_TRIES=$1
fi
fi
# Wait up to (default) 30 seconds to see if Entware utilities available.....
local TRIES=0
while [ $TRIES -lt $MAX_TRIES ];do
if [ ! -z "$(which $ENTWARE)" ] && [ "$($ENTWARE -v | grep -o "version")" == "version" ];then
if [ ! -z "$ENTWARE_UTILITY" ];then # Specific Entware utility installed?
if [ ! -z "$($ENTWARE list-installed $ENTWARE_UTILITY)" ];then
READY=0 # Specific Entware utility found
else
# Not all Entware utilities exists as a stand-alone package e.g. 'find' is in package 'findutils'
if [ -d /opt ] && [ ! -z "$(find /opt/ -name $ENTWARE_UTILITY)" ];then
READY=0 # Specific Entware utility found
fi
fi
else
READY=0 # Entware utilities ready
fi
break
fi
sleep 1
logger -st "($(basename $0))" $$ "Entware" $ENTWARE_UTILITY "not available - wait time" $((MAX_TRIES - TRIES-1))" secs left"
local TRIES=$((TRIES + 1))
done
return $READY
}
Last edited:
Martineau
Part of the Furniture
Also you should fix the use of 'jq'
Code:
for IPv4 in `jq -r '.prefixes | .[].ip_prefix' < ip-ranges.json`
Code:
for IPv4 in `jq -r '.prefixes | .[].ip_prefix' < /jffs/scripts/ip-ranges.json`Xentrk
Part of the Furniture
@robahearts. I hope the entware check will fix your issues.
Script updates include:
1. Fixed the use of jq by applying full directory path of ip-ranges.json file as noted by
@Martineau
2. Included check for entware mount. I need more time to take a look at the code @Martineau posted. Looks very thorough. I tested with the entware check code @thelonelycoder uses in servies-start before I saw the posts. Will look things over in more detail tomorrow.
3. Removed output messages for the code that checks if a shared whitelist file exists.
4. Warning messages that were generated when the script was executed the first time are now sent to /dev/null.
Thanks for the input everyone.
Script updates include:
1. Fixed the use of jq by applying full directory path of ip-ranges.json file as noted by
@Martineau
2. Included check for entware mount. I need more time to take a look at the code @Martineau posted. Looks very thorough. I tested with the entware check code @thelonelycoder uses in servies-start before I saw the posts. Will look things over in more detail tomorrow.
3. Removed output messages for the code that checks if a shared whitelist file exists.
4. Warning messages that were generated when the script was executed the first time are now sent to /dev/null.
Thanks for the input everyone.
Code:
#!/bin/sh
####################################################################################################
# Script: IPSET_Netflix3.sh
# Author: Xentrk
# 3-Mar-2018 Version 3.1
#
# Thank you to @Martineau on snbforums.com for educating myself and others on Selective
# Routing using Asuswrt-Merlin firmware.
#
#####################################################################################################
# Script Description:
#
# The purpose of this script is for selective routing of Netflix traffic using
# Autonomous System Numbers (ASNs). ASNs are assigned to entities such as Internet
# Service Providers and other large organizations that control blocks of IP addresses.
#
# Netflix and other services that use Amazon AWS servers are blocking VPN's.
#
# This script will
# 1. Create shared whitelist entry for ipinfo.io in /jffs/shared-SelectiveRouting-whitelist for use by AB-Solution and Skynet.
# Otherwise, ipinfo.io may be blocked and the script will not work.
# 2. Obtain the IPv4 addresses used by Netflix and Amazon AWS USA from ipinfo.io.
# IPv6 addresses are excluded in this version.
# 3. Create the IPSET list NETFLIX
# 4. Add the IPv4 address to the IPSET list NETFLIX
# 5. Route IPv4 addresses in IPSET list NETFLIX to WAN interface.
#
# Note 1: IPSET syntax differs between version 6 and 4.5
# Syntax for ipset v6
# ipset create WAN0 list:set
# ipset add WAN0 setlist (e.g. SPEEDTEST)
# for routers running ipset v4.5 (ipset -V)
# ipset -N WAN0 setlist (e.g. SPEEDTEST)
#
# Note 2: In the event one needs to use IPv6 in the future, the syntax is: ipset -N NETFLIX-v6 hash:net family ipv6
#
# Note 3: Troubleshooting
#
# You can use these sites for AS validation and troubleshooting to lookup ASNs:
#
# https://bgp.he.net/AS16509 (Click on the prefixes tab to view IP addresses)
# http://ipinfo.io/AS2906
#
# Note 4: Required OpenVPN Client Settings
#
# - Redirect Internet Traffic = Policy Rules or Policy Rules (Strict)
# - Others?
#
#######################################################################
logger -t "($(basename $0))" $$ Starting IPSET_Netflix.sh..." $0${*:+ $*}."
# Uncomment for debugging
set -x
# Prevent script from running concurrently when called from nat-start
PROGNAME=$(basename "$0")
LOCKFILE_DIR=/tmp
LOCK_FD=200
lock() {
local prefix=$1
local fd=${2:-$LOCK_FD}
local lock_file=$LOCKFILE_DIR/$prefix.lock
# create lock file
eval "exec $fd>$lock_file"
# acquier the lock
flock -n $fd \
&& return 0 \
|| return 1
}
eexit() {
local error_str="$@"
echo $error_str
exit 1
}
main() {
lock $PROGNAME \
|| eexit "Only one instance of $PROGNAME can run at one time."
# check shared-SelectiveRouting-whitelist so ipinfo.io is not blocked
# by AB-Solution and Skynet
# checking if shared-SelectiveRouting-whitelist exists
if [ -s "/jffs/shared-SelectiveRouting-whitelist" ];then
# file found, no further checks
else
# create shared white list for ABS and Skynet"
echo "ipinfo.io" > /jffs/shared-SelectiveRouting-whitelist
fi
ipset create NETFLIX hash:net family inet hashsize 1024 maxelem 65536
#Pull all IPv4s listed for Netflix USA - AS2906
netsv4=`curl http://ipinfo.io/AS2906 2>/dev/null | grep -E "a href.*2906\/" | grep -v ":" |sed 's/^.*\">//; s/<.*//; /^\s*$/d'`
for net in $netsv4
do
ipset add NETFLIX $net
done
unset netsv4
# Prevent entware funcion jq from executing until entware has mounted
RC='/opt/etc/init.d/rc.unslung'
i=30
until [ -x "$RC" ] ; do
i=$(($i-1))
if [ "$i" -lt 1 ] ; then
logger "entware in error state which prevents script from running"
exit
fi
sleep 1
done
# Download Amazon AWS json file
wget https://ip-ranges.amazonaws.com/ip-ranges.json -O /jffs/scripts/ip-ranges.json
# Create IPSET lists
ipset create AMAZONAWS hash:net family inet hashsize 1024 maxelem 65536
#Pull all IPv4s listed for Amazon AWS
for IPv4 in `jq -r '.prefixes | .[].ip_prefix' < /jffs/scripts/ip-ranges.json`
do
ipset add AMAZONAWS $IPv4
done
unset IPv4
###########################################################
#Create table to contain items added automatically by wan #
###########################################################
ip rule del prio 9990 > /dev/null 2>&1
ip rule add from 0/0 fwmark 0x7000/0x7000 table main prio 9990
iptables -t mangle -D PREROUTING -i br0 -p tcp -m set --match-set NETFLIX dst,dst -j MARK --set-mark 0x7000/0x7000 > /dev/null 2>&1
iptables -t mangle -A PREROUTING -i br0 -p tcp -m set --match-set NETFLIX dst,dst -j MARK --set-mark 0x7000/0x7000
iptables -t mangle -D PREROUTING -i br0 -p tcp -m set --match-set AMAZONAWS dst,dst -j MARK --set-mark 0x7000/0x7000 > /dev/null 2>&1
iptables -t mangle -A PREROUTING -i br0 -p tcp -m set --match-set AMAZONAWS dst,dst -j MARK --set-mark 0x7000/0x7000
logger -t "($(basename $0))" $$ Ending IPSET_Netflix.sh..." $0${*:+ $*}."
}
main
Last edited:
thelonelycoder
Part of the Furniture
I was assuming that this is checked during install.But what if the required Entware utility i.e. in this case 'jq' isn't installed?
My code therefore is only delaying the start of some other code until entware is mounted and with it the already installed packages.
Martineau
Part of the Furniture
Now your script may potentially wait up to 30 secs for Entware, it may be prudent to change the way your script is called from nat-start, as there is no reason why nat-start execution needs to be stalled unnecessarily.
Use
Code:
sh /jffs/scripts/IPSET_Netflix.sh &robahearts
Regular Contributor
Xentrk
Part of the Furniture
Embarrassed about the error
But good news that it works . I'll fix the error you received on the check for the existence of the whitelist file and update the code that checks for entware as well. We should then have a version that I can post on github rather than keeping the source code on the forum. I will need your help to test the updated script.
Also, please make the change the script is called per @Martineau recommendation.
Xentrk
Part of the Furniture
I did some searching on the forum to see what issues others have had with entware. For example, what happens if someone removes the USB containing entware? I came across the wiki article on this topic (see Gotchas, Extended Boot Time section). Sounds like there have been some lively discussions on the delay loop in services start method vs moving the entware rc.unslung call to post-mount that I was not aware of.
Last edited:
thelonelycoder
Part of the Furniture
AB4 will only use dnsmasq.postconf and post-mount. I have moved all code to linked files in these two.
I detect and act accordingly if entware is not found during boot or when a device is plugged in.
My new code works 100% reliable for all situations I played through. There are no delay routines at all in my scripts.
Either the Entware folder is present when post-mount runs or not. There is no if's and but's in my code for that.
Similar threads
DomainVPNRouting
Domain VPN Routing v3.0.3 ***Release***
Similar threads
| Thread starter | Title | Forum | Replies | Date |
|---|---|---|---|---|
| H | Routing wireguard | VPN | 0 |
Similar threads
Latest threads
-
-
Release ASUS ZenWiFi BQ16 Firmware version 3.0.0.6.102_36801 (26/11/2024)
- Started by Stepalex
- Replies: 4
-
-
Wireguard VPN Director for VPN - WireGuard® Manager© v1.04 by Martineau- Started by unclebuk
- Replies: 0
-
Support SNBForums w/ Amazon
If you'd like to support SNBForums, just use this link and buy anything on Amazon. Thanks!
Sign Up For SNBForums Daily Digest
Get an update of what's new every day delivered to your mailbox. Sign up here!