Selective Routing with Asuswrt-Merlin

[jinx10000]

Thank you!

I'm going to take a little time to absorb that code and may have a few questions afterwards. Until starting this I have never written a single line of code in bash or for a router.

I have noticed that the script stops working on occasion (usually when I wake up in the morning). I assume that is what this is. Probably has a small part in the timing issue I previously mentioned too. I set a timer for the router to reboot at 5 AM so that will take care of most instances I think except when I'm on vacation and staying up late trying to learn the basics of a new programming language as well as routing basics. That is if it only happens at 2 AM...

One more thing. I don't think you use Netflix if I remember right, but the hard coded DNS issue raised it's head again. I'm thinking about running a script that pulls in all known Netflix IPs from radb.net along with the other ipset-netmasq script to see if that catches everything. Any thoughts on this or better ideas?

Thanks again for your assistance. I have been reading for more than two weeks before I stumbled on this thread. You're a godsend.

 

[Martineau]

Until starting this I have never written a single line of code in bash or for a router.

Actually neither had I

I don't think you use Netflix if I remember right, but the hard coded DNS issue raised it's head again. I'm thinking about running a script that pulls in all known Netflix IPs from radb.net along with the other ipset-netmasq script to see if that catches everything. Any thoughts on this or better ideas?

I do use Netflix, and used (for throughput performance) a smart DNS to change my region!

Using a script (my script uses IRR database ipinfo.io) to manually grab the published IPv4 and IPv6 IPs for Netflix/Facebook etc. is useful for say Selective domain routing, but if Netflix has the I/Ps on their known region 'blacklist' then it wont work.

P.S. I believe a couple of VPN providers have apparently managed to circumvent the Netflix geo-restriction but at the moment I haven't investigated further.

 

[jinx10000]

I actually don't know how much more I'm going to poke around this. I find it to be puzzling and interesting so I will probably at least figure out the nat-start script that Martineau posted. I decided to buy the newer fire TV sticks that have built in VPN support. With that, I can bypass per app from the device and probably won't need this any more. This process should allow you to Bypass the VPN and watch US Netflix through WAN on Android phone/fire stick/ and PC from US (likely other devices as well). I'm using RT-AC68U with Firmware:380.63_2. Don't make too much fun I know I don't what I'm doing... lol.

This seems to be working great. It's currently under wan-start which needs to be corrected. It's cobbled together from a ton of sites and is messy but as I said it works. Maybe it will help someone else who is struggling with this issue. It's also long so I broke it down into sections. Not sure if I did it right.

Spoiler: GUI Setup

Under Wireless:
Left everything default except SSID and wireless security settings.

Under LAN:
DHCP Server tab
Left DNS Server 1 and DNS Server 2 blank.
Unchecked all options if they were checked.

• Advertise router's IP in addition to user-specified DNS
• Forward local domain queries to upstream DNS
• Enable DNSSEC support

Everything else is default except I've got some manual assignments for DHCP.
Route tab
Enable static routes to yes
8.8.4.4/255.255.255.255/192.168.1.1/Metric =2/LAN
8.8.8.8/255.255.255.255/192.168.1.1/Metric =2/LAN

Under WAN:
Unchecked Connect to DNS Server automatically
Entered my VPN DNSs for DNS Server 1 and DNS Server 2
Everything else was left as default

Under IPV6:
Default as disabled

Under VPN:
Open VPN clients tab
Followed instructions provided by my VPN host.
Important: Change redirect internet traffic to Policy rules.
Change block routed clients if tunnel goes down to yes.
Rules for routing client traffic through the tunnel (Max Limit : 100)

• Add devices you want to go through VPN. Must add at least one is my understanding.
• Ex. Nexus6P\192.168.1.105\0.0.0.0\VPN

Under Firewall:
No changes

Under Administration:
Enable JFFS custom scripts and configs
Enable SSH to at least LAN only
Enable Telnet

Under System Log:
You will see this about 3 minutes after the device reboots:

• Netflix IPSET Script complete
• Until you see this it will likely not work. You can test by going to whatsmyip.org and seeing WAN IP and going to IPchicken.com and seeing the VPN IP

Other changes:
I made some other changes like port forwarding and USB support but I don't believe this will have affected the process in anyway.

Messy but working wan-start script:

Spoiler: wan-start

#!/bin/sh
ipset -N NETFLIX hash:ip
sleep 100
touch /tmp/000wanstarted
# new ipset - first run of clean router
ipsetv=6
lsmod | grep "xt_set" > /dev/null 2>&1 || for module in ip_set ip_set_hash_net ip_set_hash_ip xt_set
do
insmod $module
done
sleep 10
insmod xt_set.ko
ipset -N NETFLIX hash:ip
sleep 1
#Adds info to DNSMASQ Not required except on first clena run of router.
#echo "ipset=/whatsmyip.org/netflix.com/nflxext.com/nflximg.com/nflxvideo.net/amazonaws.com/NETFLIX" >>/jffs/configs/dnsmasq.conf.add
service restart_dnsmasq
sleep 1
ipset -N NETFLIXCIDR hash:net
sleep 1
#Pull all IPs listed for Netflix on radb.net
opkg install whob
netsv4=`whob -h whois.radb.net -- '-i origin AS2906' | grep -Eo "([0-9.]+){4}/[0-9]+"`
for net in $netsv4
do
ipset -A NETFLIXCIDR $net
done
netsv4=`whob -h whois.radb.net -- '-i origin AS55095' | grep -Eo "([0-9.]+){4}/[0-9]+"`
for net in $netsv4
do
ipset -A NETFLIXCIDR $net
done
netsv4=`whob -h whois.radb.net -- '-i origin AS40027' | grep -Eo "([0-9.]+){4}/[0-9]+"`
for net in $netsv4
do
ipset -A NETFLIXCIDR $net
done
#Create table to contain items added automatically by opening site/netflix
TAG_MARK=0x7000
ip rule del fwmark $TAG_MARK 2> /dev/null > /dev/null
ip rule add fwmark $TAG_MARK table main prio 9990
#Create table to contain items added by querying radb.net
TAG_MARK2=0x7001
ip rule del fwmark $TAG_MARK2 2> /dev/null > /dev/null
ip rule add fwmark $TAG_MARK2 table main prio 9991
ip route flush cache
#Set mark for automatic IPs added by opening site/netflix
iptables -A PREROUTING -t mangle -m set --match-set NETFLIX dst -j MARK --set-mark $TAG_MARK/$TAG_MARK
#Set mark for IPs added by radb.net
iptables -A PREROUTING -t mangle -m set --match-set NETFLIXCIDR dst -j MARK --set-mark $TAG_MARK2/$TAG_MARK2

cru a ScheduledReboot "0 15 * * * /sbin/reboot"
cru a ScheduledReboot2 "0 7 * * * /sbin/reboot"
chmod a+rx /jffs/scripts/*
touch /tmp/001wanstarted
logger "Netflix IPSET Script complete"

Spoiler: Additional Instructions

Additional instructions:
Save this script to \jffs\scripts\wan-start using winscp. Search for instruction very easy.
Telnet into the device and run the following:

• dos2unix /jffs/scripts/wan-start
  • #seems to help remove unseen carriage returns. I'm sure someone will prove me wrong.
• sed -i 's/\r//g' /jffs/scripts/wan-start
  • #recommended in this thread. Not sure what it does but it didn't hurt.
• chmod a+rx /jffs/scripts/wan-start
  • Makes script executable.
• Place a device in the USB port with a EXT partition. I chose EXT2 format. I formatted using these instructions from the router. http://www.algissalys.com/how-to/format-and-partition-usb-asuswrt-routers
• Install Entware for required modules. I used the instructions on this page. Basically just run the telnet command entware-setup.sh and follow instructions : https://github.com/RMerl/asuswrt-merlin/wiki/Entware

Spoiler: Notes

Notes:

• You will have to uncomment this line after first run. I found if you kept it uncommented it would write it in dnsmasq.conf multiple times. This may be because I was making changes to it but just commented it out to be sure.
  • #echo "ipset=/whatsmyip.org/netflix.com/nflxext.com/nflximg.com/nflxvideo.net/amazonaws.com/NETFLIX" >>/jffs/configs/dnsmasq.conf.add
• Some of these steps may not be necessary. I just tried to provide a complete list as possible for the setup for what I did.
• These lines are used to pull in known Netflix servers that may not be pulled in from the automated process. I believe this is required due to the hardcoded dns in Android Netflix. I have read that you can get around this using Netflix APK 3.7.1 but it seemed to be a little flaky. Had to open a separate section in the android app before the shows would populate. I have not verified that these are all actually Netflix servers as stated by radb.net and may open you up to unexpected websites with an unprotected VPN status.
  • netsv4=`whob -h whois.radb.net -- '-i origin AS2906' | grep -Eo "([0-9.]+){4}/[0-9]+"`
  • for net in $netsv4
  • do
  • ipset -A NETFLIXCIDR $net
  • done
• These sections are not needed if you are using your PC and may leave you slightly more protected to exclude them:
  • ipset -N NETFLIXCIDR hash:net
  • From #Create table to contain items added by querying radb.net to line before #Set mark for automatic IPs added by opening site/Netflix
  • #Set mark for IPs added by radb.net
    iptables -A PREROUTING -t mangle -m set --match-set NETFLIXCIDR dst -j MARK --set-mark $TAG_MARK2/$TAG_MARK2

• This code/instructions etc... are almost completely not mine but compiled from hundreds of sites. Thanks to everyone I "borrowed this from"
• I am falling asleep as I'm typing this. Very late here. Hope this helps someone who has been having the same problems and even better if someone can come in and improve on it.

 

[jinx10000]

Actually neither had I



I do use Netflix, and used (for throughput performance) a smart DNS to change my region!

Using a script (my script uses IRR database ipinfo.io) to manually grab the published IPv4 and IPv6 IPs for Netflix/Facebook etc. is useful for say Selective domain routing, but if Netflix has the I/Ps on their known region 'blacklist' then it wont work.

P.S. I believe a couple of VPN providers have apparently managed to circumvent the Netflix geo-restriction but at the moment I haven't investigated further.

Thanks Martineau. I'm not trying to circumvent anything. I'm trying to use Netflix in the US from the US. They just don't like that I'm using a VPN because apparently that could mean I'm trying to circumvent their geo-restrictive licensing. Best method for me is to just route all Netflix traffic through WAN. I added the list as stated. I believe radb.net pulls from ipinfo.io possibly. I also have it auto pulling IPs for anthing that isn't hardcoded using the ipset-dnsmasq marking. Crossing my fingers that the combo will work. I did read somewhere that ipinfo.io and radb.net overstate their IPs. Using /16 for example when the actual range is /32 or something. I looked through a list of some of the items that were posted for sites that would be allowed outside Netflix and didn't feel in any particular danger.

 

[Martineau]

......and even better if someone can come in and improve on it.

Sorry couldnt resist!

Rules for routing client traffic through the tunnel (Max Limit : 100)

•Add devices you want to go through VPN. Must add at least one is my understanding
•Ex. Nexus6P\192.168.1.105\0.0.0.0\VPN

Doesn't have to be a real address on your LAN

e.g. Dummy 172.16.0.1 0.0.0.0 VPN

but as you want to only force Netflix via the WAN you should use a single LAN rule 192.168.1.0/24 0.0.0.0 VPN to force everything via the VPN? Also why not set 'Accept DNS Configuration=Exclusive' in the VPN GUI rather than manually setting the VPN DNS in the WAN GUI page!? i.e. Suppose your VPN provider uses a different DNS next time the VPN starts?...mine does 10.xxx.xxx.xxx

Also you don't need individual fwmarks (...although it does significantly aid debugging!) i.e. you have used one for each IPSET

i.e. Change

iptables -A PREROUTING -t mangle -m set --match-set NETFLIX dst -j MARK --set-mark $TAG_MARK/$TAG_MARK

to

iptables -A PREROUTING -t mangle -m set --match-set WAN0 dst -j MARK --set-mark $TAG_MARK/$TAG_MARK

then create an IPSET list called WAN0

ipset create WAN0 list:set

or for routers running ipset v4.5 (ipset -V)

ipset -N WAN0 setlist

then add the individual IPSETs to the 'group/catalog' list

ipset -A WAN0 NETFLIX
ipset -A WAN0 NETFLIXCIDR

and the single RPDB fwmark 0x7000/PREROUTING rule pair will route both IPSETs via the WAN.

Furthermore, suppose in the future you need to selectively route say Netflix IPV6 subnets?

Simply create an appropriate new IPv6 ipset..

ipset -N NETFLIXCIDR-v6 hash:net family ipv6

...populate it with the appropriate subnets (as per the IPV4 subnets script)

then simply add this new IPSET to the already created WAN list

ipset -A NETFLIXCIDR-v6 WAN0

As you are rebooting every day then having all of your commands in wan-start will work, but there are probably better places to issue your VPN dependent commands such as nat-start/openvpn-event which may save you the hassle of a physical reboot - you could simple bounce the VPN client.

Also you may want to consider saving the IPSETs by a schedule , so that you will have a 'working' set of identified Netflix subnets if dsnmasq/your script accidentally wipes/corrupts them!

However, as you have stated, all this is moot now given your decision to use custom hardware to solve your issue!

EDIT: Include ipset v4.5 syntax for ipset commands and fix typo

 

[xman0980]

RTFM ?

Example for '*netflix.com' tracking to IPSET 'Netflix'

Commands:

ipset -N Netflix iphash

echo "ipset=/netflix.com/Netflix" >>/jffs/configs/dnsmasq.conf.add
service restart_dnsmasq

nslookup netflix.com

ipset -L Netflix

Explanation.....

Step 1. Create the IPSET...NOTE: ipset v4 syntax (as shown) should still work for ipset v6 which is now available on ARM routers using firmware 380.63

admin@RT-AC68U:/tmp/home/root# ipset -N Netflix iphash

....check if it is created

admin@RT-AC68U:/tmp/home/root# ipset -L Netflix
Name: Netflix
Type: hash:ip
Revision: 0
Header: family inet hashsize 1024 maxelem 65536
Size in memory: 8248
References: 0
Members:

Step 2. Configure dnsmasq (/jffs/configs/dnsmasq.conf.add) with 'ipset=' directives (use editor of choice nano/vi/WinSCP/notepad++ etc. ) or for a quick'n'dirty addition via the command line...

admin@RT-AC68U:/tmp/home/root# echo "ipset=/netflix.com/Netflix" >>/jffs/configs/dnsmasq.conf.add

...instruct dnsmasq to use the new config

admin@RT-AC68U:/tmp/home/root# service restart_dnsmasq

Step 3. Perform any action on a device to cause a DNS reference i.e. open target web page, ping etc.

admin@RT-AC68U:/tmp/home/root# nslookup netflix.com
Server:    127.0.0.1
Address 1: 127.0.0.1 localhost.localdomain

Name:      netflix.com
Address 1: 38.130.79.37

Step 4. Check if the IPSET is populated with an IP address

admin@RT-AC68U:/tmp/home/root# ipset -L Netflix
Name: Netflix
Type: hash:ip
Revision: 0
Header: family inet hashsize 1024 maxelem 65536
Size in memory: 8264
References: 0
Members:
38.130.79.37

If you now periodically check the IPSET, hopefully it will be populated with additional IP adresses depending on the related activity.

By default, dnsmasq will use the DNS as configured on the router to resolve the IP addresses.

However, you can add a matching 'server=' (for the 'ipset=') directive to /jffs/configs/dnsmasq.conf.add

server=/netflix.com/xxx.xxx.xxx.xxx

where xxx.xxx.xxx.xxx is either a SmartDNS or preferably (if this is a concern, to prevent DNS leaks ) a DNS server exclusive to the VPN Client.

Unfortunately, my VPN supplied DNS are non-static 10.xxx.xxx.xxx addresses which change each time the VPN Client is started...

Chain DNSVPN1 (2 references)
num   pkts bytes target     prot opt in     out     source               destination
1        0     0 DNAT       all  --  *      *       172.0.0.1            0.0.0.0/0            to:10.200.199.1
2        0     0 DNAT       all  --  *      *       168.0.0.1            0.0.0.0/0            to:10.200.199.1

Chain DNSVPN2 (6 references)
num   pkts bytes target     prot opt in     out     source               destination
1        0     0 DNAT       all  --  *      *       172.0.0.2            0.0.0.0/0            to:10.200.192.1

so I need to use a script to dynamically modify the IPSET related 'server=' directives in /jffs/configs/dnsmasq.conf.add every time the VPN Client is started.

Finally, you should save the IPSETs at regular intervals (using a cru aka cron schedule), then you can restore the populated IPSETs from say firewall-start when the router is rebooted.

Regards,

So following these 4 steps after setting up a VPN, will allow netflix to still be watched? Code wise its way over my head, but damn can i copy and paste

 

[jinx10000]

Edit: Martineau. Started making some of these changes. This part didn't work.

ipset create WAN0 list:set
ipset add NETFLIX WAN0
ipset add NETFLIXCIDR WAN0

Received this error:
ipset v6.29: Syntax error: cannot parse WAN0: resolving to IPv4 address failed

Edit 2:
I tracked down my "temporary" problem.
These lines are not executing from script or when I copy them in more than one line at a time in Telnet. Basically Telnet just hangs like it's waiting for something. Any ideas? I ran dos2unix and the other command I mentioned on the file.

netsv4=`whob -h whois.radb.net -- '-i origin AS2906' | grep -Eo "([0-9.]+){4}/[0-9]+"`
for net in $netsv4
do
ipset -A NETFLIXCIDR $net
done

Sorry couldnt resist!
Doesn't have to be a real address on your LAN
e.g. Dummy 172.16.0.1 0.0.0.0 VPN
but as you want to only force Netflix via the WAN you should use a single LAN rule 192.168.1.0/24 0.0.0.0 VPN to force everything via the VPN?

I think I may have misspoken on my intentions. I want to pass everything through the VPN (except Netflix) on certain devices. Not all devices. When I started, I wasn't sure if that was possible so didn't mention it. Devices going through VPN are one PC and two first generation Fire TV sticks.

• I think this would pass nothing through the VPN, correct? Dummy 172.16.0.1 0.0.0.0 VPN
• This would pass all devices through the VPN? 192.168.1.0/24 0.0.0.0 VPN
• Did what I mention correctly target individual devices?

Also why not set 'Accept DNS Configuration=Exclusive' in the VPN GUI rather than manually setting the VPN DNS in the WAN GUI page!? i.e. Suppose your VPN provider uses a different DNS next time the VPN starts?...mine does 10.xxx.xxx.xxx

I'm not sure if mine does. I did a search for my VPNs DNS and I used the two DNS IPs I found in their forums. They said in their setup instruction to set 'Accept DNS Configuration=Strict' and I wasn't really sure what it did.

I'm guessing this won't help with the hardcoded DNS in Android?

Also you don't need individual fwmarks (...although it does significantly aid debugging!) i.e. you have used one for each IPSET

i.e. Change

iptables -A PREROUTING -t mangle -m set --match-set NETFLIX dst -j MARK --set-mark $TAG_MARK/$TAG_MARK

to

iptables -A PREROUTING -t mangle -m set --match-set WAN0 dst -j MARK --set-mark $TAG_MARK/$TAG_MARK

then create an IPSET list called WAN0

ipset create WAN0 list:set

then add the individual IPSETs to the 'group/catalog' list

ipset add NETFLIX WAN0
ipset add NETFLIXCIDR WAN0

and the single RPDB fwmark 0x7000/PREROUTING rule pair will route both IPSETs via the WAN.

I will make this change later today and post back the code afterwards.

Furthermore, suppose in the future you need to selectively route say Netflix IPV6 subnets?

Simply create an appropriate new IPv6 ipset..

ipset -I NETFLIXCIDR-v6 hash:net family ipv6

...populate it with the appropriate subnets (as per the IPV4 subnets script)

then simply add this new IPSET to the already created WAN list

ipset add NETFLIXCIDR-v6 WAN0

I have no intention of using IPV6 for this or anything else but it is worth noting in case I have some other project in the future. I don't really know what the point of IPV6 is. I may have to look into this. I read somewhere there was likely to be more IP leaks if VPN is used but didn't look into it since my VPN says to disable it in their setup instructions.

As you are rebooting every day then having all of your commands in wan-start will work, but there are probably better places to issue your VPN dependent commands such as nat-start/openvpn-event which may save you the hassle of a physical reboot - you could simple bounce the VPN client.

I will probably still look into this. I assume basic idea is that when nat-start event occurs then I would have to clean out all marks etc. Wait for something to start and then set everything back up. Just need to take some time to look at your script.

It seems like some of the code was in external scripts in the brief look I had at it. Not sure if that was needed or not.

Also you may want to consider saving the IPSETs by a schedule , so that you will have a 'working' set of identified Netflix subnets if dsnmasq/your script accidentally wipes/corrupts them!

However, as you have stated, all this is moot now given your decision to use custom hardware to solve your issue!

I find it to be a puzzle. I go back to the grind next week so hopefully I will find some time to play with it in the near future.

 

[jinx10000]

So following these 4 steps after setting up a VPN, will allow netflix to still be watched? Code wise its way over my head, but damn can i copy and paste

I am very new to this and learning. A lot of it depends on your setup.
If you are in the US trying to watch Netflix but are getting VPN blocked then I posted some instructions a couple posts back that are working for me (for the most part). This works perfectly on my PC. Unfortunately, I don't sit in front of my PC to watch Netflix. I'm still having some intermittent issues with Android.

Android is a bit more difficult because they hard code the DNS and it doesn't appear to use the normal port 53 so I can force it to pass through the router as a DNS request that can be examined.

Directly under my instruction Martineau provided a lot of really good feedback on those instructions that may help.

On your original question:
I think the code that he provided in the first few lines is basically just populating a list of IPs that can be rerouted. It is not rerouting it. He is later talking about how to pass Netflix.com to a smartDNS or other DNS. I'm not sure if that is related or completely separate from the previous ipset code. I don't see how it's related but I am a total novice.

I can say that you will need to add more than just Netflix.com in the ipset rule to catch all possible IPs.

 

[Martineau]

ipset create WAN0 list:set
ipset add NETFLIX WAN0
ipset add NETFLIXCIDR WAN0

Received this error:
ipset v6.29: Syntax error: cannot parse WAN0: resolving to IPv4 address failed

Apologies I inadvertently reversed the ipset args ....and used ipset v6 syntax, so I have now edited the commands for ipset v4.5 users.

Corrected syntax is:

ipset create WAN0 list:set
ipset add WAN0 NETFLIX
ipset add WAN0 NETFLIXCIDR

or ipset v4.5

ipset -N WAN0 setlist
ipset -A WAN0 NETFLIX
ipset -A WAN0 NETFLIXCIDR

Updated original post.

I tracked down my "temporary" problem.

netsv4=`whob -h whois.radb.net -- '-i origin AS2906' | grep -Eo "([0-9.]+){4}/[0-9]+"`
for net in $netsv4
do
ipset -A NETFLIXCIDR $net
done

These lines are not executing from script or when I copy them in more than one line at a time in Telnet. Basically Telnet just hangs like it's waiting for something. Any ideas? I ran dos2unix and the other command I mentioned on the file.

Your code works for me? (once I installed 'whob' from entware)

If you copy the lines into the command line, and it pauses (issues the '>' prompt) then usually it indicates that a terminating code clause/quote character/command is missing.

admin@RT-AC68U:/tmp/home/root# netsv4="Martineau"
admin@RT-AC68U:/tmp/home/root# for net in $netsv4
> do
> echo $net
> done
Martineau
admin@RT-AC68U:/tmp/home/root# unset netsv4
admin@RT-AC68U:/tmp/home/root#

If you want to have a 'one-line' program to be entered on the command line then this works...

 netsv4=`whob -h whois.radb.net -- '-i origin AS2906' | grep -Eo "([0-9.]+){4}/[0-9]+"`;for net in $netsv4;do echo $net;done;unset netsv4

 

[jinx10000]

Apologies I inadvertently reversed the ipset args ....and used ipset v6 syntax, so I have now edited the commands for ipset v4.5 users.

Corrected syntax is:

ipset create WAN0 list:set
ipset add WAN0 NETFLIX
ipset add WAN0 NETFLIXCIDR

or ipset v4.5

ipset -N WAN0 setlist
ipset -A WAN0 NETFLIX
ipset -A WAN0 NETFLIXCIDR

Updated original post.

This part now works with no issues.

 netsv4=`whob -h whois.radb.net -- '-i origin AS2906' | grep -Eo "([0-9.]+){4}/[0-9]+"`;for net in $netsv4;do echo $net;done;unset netsv4

This part isn't working even putting on one line. I copied my output. It isn't giving the > like it's waiting for something. It just goes to a blank cursor. It is also dropping some of the next line when I copy multiple lines and paste at the same time... Not sure what's going on here.

It still works when I copy one line at a time even though it's now doing all of the work on one line...

jinx10000@RT-AC68U-5610:/tmp/home/root# netsv4=`whob -h whois.radb.net -- '-i origin AS2906' | grep -Eo "([0-9.]+){4}/[0-9]+"`;for net in $netsv4;do ipset -A NETFLIXCIDR $net;done;unset netsv4
LIXCIDR $net;done;unset netsv4
netsv4=`whob -h whois.radb.net -- '-i origin AS40027' | grep -Eo "([0-9.]+){4}/[0-9]+"`;for net in $netsv4;do ipset -A NETFLIXCIDR $net;done;unset netsv4

 

[Martineau]

This part isn't working even putting on one line. I copied my output. It isn't giving the > like it's waiting for something. It just goes to a blank cursor. It is also dropping some of the next line when I copy multiple lines and paste at the same time... Not sure what's going on here.

No idea why you appear to have issues. I have tried 3 different clients (WinSCP,PuTTY and highly recommended/preferred Xshell5) to open a SSH connection, and all successfully (using cut'n'paste) executed the one-line program on the command line.

To be honest, whilst it may be frustrating (by being a hindrance to your script development), testing of lengthy (one-line) script snippets on the command line isn't something I personally need to do on a regular basis, so there is probably little benefit devoting time to debug this weird issue....my point being that your goal is to ensure the command sequence works as expected in your script!

 

[jinx10000]

my point being that your goal is to ensure the command sequence works as expected in your script!

Agreed. I would be happy if it would work in the script. However for me, it isn't working in the script or in telnet. It only works if I copy it one line at a time and press enter.

I added some lines of code that I found that allow me to log the results.

set -x # uncomment/comment to enable/disable debug mode
(
opkg install whob
sleep 5
netsv4=`whob -h whois.radb.net -- '-i origin AS40027'|grep -Eo "([0-9.]+){4}/[0-9]+"`;for net in $netsv4;do ipset -A NETFLIXCIDR $net;done;unset netsv4
sleep 3
netsv4=`whob -h whois.radb.net -- '-i origin AS55095'|grep -Eo "([0-9.]+){4}/[0-9]+"`;for net in $netsv4;do ipset -A NETFLIXCIDR $net;done;unset netsv4
sleep 3
netsv4=`whob -h whois.radb.net -- '-i origin AS2906'|grep -Eo "([0-9.]+){4}/[0-9]+"`;for net in $netsv4;do ipset -A NETFLIXCIDR $net;done;unset netsv4
sleep 3
ipset -L NETFLIXCIDR
) 2>&1 | logger -t "ovpn_split[$$]"

Results from script on reboot.

Spoiler

Nov 27 16:31:13 ovpn_split[776]: + opkg install whob
Nov 27 16:31:13 ovpn_split[776]: Package whob (3.76-1) installed in root is up to date.
Nov 27 16:31:13 ovpn_split[776]: + sleep 5
Nov 27 16:31:18 ovpn_split[776]: + whob -h whois.radb.net -- -i origin AS40027
Nov 27 16:31:18 ovpn_split[776]: + grep -Eo ([0-9.]+){4}/[0-9]+
Nov 27 16:31:24 ovpn_split[776]: + netsv4=
Nov 27 16:31:24 ovpn_split[776]: + unset netsv4
Nov 27 16:31:24 ovpn_split[776]: + sleep 3
Nov 27 16:31:27 ovpn_split[776]: + whob -h whois.radb.net -- -i origin AS55095
Nov 27 16:31:27 ovpn_split[776]: + grep -Eo ([0-9.]+){4}/[0-9]+
Nov 27 16:31:27 ovpn_split[776]: + netsv4=
Nov 27 16:31:27 ovpn_split[776]: + unset netsv4
Nov 27 16:31:27 ovpn_split[776]: + sleep 3
Nov 27 16:31:30 ovpn_split[776]: + whob -h whois.radb.net -- -i origin AS2906
Nov 27 16:31:30 ovpn_split[776]: + grep -Eo ([0-9.]+){4}/[0-9]+
Nov 27 16:31:30 ovpn_split[776]: + netsv4=
Nov 27 16:31:30 ovpn_split[776]: + unset netsv4
Nov 27 16:31:30 ovpn_split[776]: + sleep 3
Nov 27 16:31:33 ovpn_split[776]: + ipset -L NETFLIXCIDR
Nov 27 16:31:33 ovpn_split[776]: Name: NETFLIXCIDR
Nov 27 16:31:33 ovpn_split[776]: Type: hash:net
Nov 27 16:31:33 ovpn_split[776]: Revision: 0
Nov 27 16:31:33 ovpn_split[776]: Header: family inet hashsize 1024 maxelem 65536
Nov 27 16:31:33 ovpn_split[776]: Size in memory: 8500
Nov 27 16:31:33 ovpn_split[776]: References: 0
Nov 27 16:31:33 ovpn_split[776]: Members:

As you can see nothing is being added.

I put the results of running it from Telnet in the next post. Space issues.

 

[jinx10000]

These are the results from running from TELNET. It's a step up because I can run more than one line at once.

Deleted some of the lines where they were similar to make room for the 10,000 character limit. I think it still shows the point that it is finding and adding IPs...

Spoiler

Nov 27 16:32:39 ovpn_split[1677]: + opkg install whob
Nov 27 16:32:40 ovpn_split[1677]: Package whob (3.76-1) installed in root is up to date.
Nov 27 16:32:40 ovpn_split[1677]: + sleep 5
Nov 27 16:32:45 ovpn_split[1677]: + whob -h whois.radb.net -- -i origin AS40027
Nov 27 16:32:45 ovpn_split[1677]: + grep -Eo ([0-9.]+){4}/[0-9]+
Nov 27 16:32:45 ovpn_split[1677]: + netsv4=69.53.250.0/24
Nov 27 16:32:45 ovpn_split[1677]: 69.53.254.0/24
Nov 27 16:32:45 ovpn_split[1677]: 69.53.229.0/24
Nov 27 16:32:45 ovpn_split[1677]: 69.53.228.0/24
Nov 27 16:32:45 ovpn_split[1677]: 69.53.227.0/24
Nov 27 16:32:45 ovpn_split[1677]: 69.53.253.0/24
Nov 27 16:32:45 ovpn_split[1677]: + ipset -A NETFLIXCIDR 69.53.250.0/24
Nov 27 16:32:45 ovpn_split[1677]: + ipset -A NETFLIXCIDR 69.53.254.0/24
Nov 27 16:32:45 ovpn_split[1677]: + ipset -A NETFLIXCIDR 69.53.229.0/24
Nov 27 16:32:45 ovpn_split[1677]: + unset netsv4
Nov 27 16:32:45 ovpn_split[1677]: + sleep 3
Nov 27 16:32:48 ovpn_split[1677]: + whob -h whois.radb.net -- -i origin AS55095
Nov 27 16:32:48 ovpn_split[1677]: + grep -Eo ([0-9.]+){4}/[0-9]+
Nov 27 16:32:48 ovpn_split[1677]: + netsv4=208.75.76.0/22
Nov 27 16:32:48 ovpn_split[1677]: 69.53.224.0/19
Nov 27 16:32:48 ovpn_split[1677]: 38.72.126.0/24
Nov 27 16:32:48 ovpn_split[1677]: 69.53.242.0/24
Nov 27 16:32:48 ovpn_split[1677]: + ipset -A NETFLIXCIDR 208.75.76.0/22
Nov 27 16:32:48 ovpn_split[1677]: + ipset -A NETFLIXCIDR 69.53.224.0/19
Nov 27 16:32:48 ovpn_split[1677]: + unset netsv4
Nov 27 16:32:48 ovpn_split[1677]: + sleep 3
Nov 27 16:32:51 ovpn_split[1677]: + whob -h whois.radb.net -- -i origin AS2906
Nov 27 16:32:51 ovpn_split[1677]: + grep -Eo ([0-9.]+){4}/[0-9]+
Nov 27 16:32:52 ovpn_split[1677]: + netsv4=69.53.224.0/19
Nov 27 16:32:52 ovpn_split[1677]: 208.75.76.0/22
Nov 27 16:32:52 ovpn_split[1677]: 37.77.184.0/21
Nov 27 16:32:52 ovpn_split[1677]: 208.75.76.0/24
Nov 27 16:32:52 ovpn_split[1677]: 208.75.77.0/24
Nov 27 16:32:52 ovpn_split[1677]: 208.75.78.0/24
Nov 27 16:32:52 ovpn_split[1677]: 208.75.79.0/24
Nov 27 16:32:52 ovpn_split[1677]: 108.175.32.0/20
Nov 27 16:32:52 ovpn_split[1677]: 198.38.96.0/19
Nov 27 16:32:52 ovpn_split[1677]: 198.45.48.0/20
Nov 27 16:32:52 ovpn_split[1677]: 185.2.220.0/22
Nov 27 16:32:52 ovpn_split[1677]: 185.2.220.0/22
Nov 27 16:32:52 ovpn_split[1677]: 185.2.220.0/24
Nov 27 16:32:52 ovpn_split[1677]: 185.2.221.0/24
Nov 27 16:32:52 ovpn_split[1677]: 192.173.64.0/18
Nov 27 16:32:52 ovpn_split[1677]: 23.246.0.0/18
Nov 27 16:32:52 ovpn_split[1677]: 45.57.0.0/17
Nov 27 16:32:52 ovpn_split[1677]: 64.120.128.0/17
Nov 27 16:32:52 ovpn_split[1677]: 64.120.128.0/17
Nov 27 16:32:52 ovpn_split[1677]: 66.197.128.0/17
Nov 27 16:32:52 ovpn_split[1677]: 66.197.128.0/17
Nov 27 16:32:52 ovpn_split[1677]: 23.246.20.0/24
Nov 27 16:32:52 ovpn_split[1677]: 23.246.30.0/24
Nov 27 16:32:52 ovpn_split[1677]: 23.246.31.0/24
Nov 27 16:32:52 ovpn_split[1677]: 69.53.242.0/24
Nov 27 16:32:52 ovpn_split[1677]: 37.77.188.0/23
Nov 27 16:32:52 ovpn_split[1677]: 37.77.188.0/23
Nov 27 16:32:52 ovpn_split[1677]: 23.246.15.0/24
Nov 27 16:32:52 ovpn_split[1677]: 108.175.47.0/24
Nov 27 16:32:52 ovpn_split[1677]: 23.246.29.0/24
Nov 27 16:32:52 ovpn_split[1677]: 23.246.28.0/24
Nov 27 16:32:52 ovpn_split[1677]: + ipset -A NETFLIXCIDR 69.53.224.0/19
Nov 27 16:32:52 ovpn_split[1677]: + ipset -A NETFLIXCIDR 23.246.20.0/24
Nov 27 16:32:52 ovpn_split[1677]: + ipset -A NETFLIXCIDR 23.246.30.0/24
Nov 27 16:32:52 ovpn_split[1677]: + ipset -A NETFLIXCIDR 23.246.31.0/24
Nov 27 16:32:52 ovpn_split[1677]: + ipset -A NETFLIXCIDR 69.53.242.0/24
Nov 27 16:32:52 ovpn_split[1677]: + ipset -A NETFLIXCIDR 37.77.184.0/23
Nov 27 16:32:52 ovpn_split[1677]: + ipset -A NETFLIXCIDR 37.77.184.0/23
Nov 27 16:32:52 ovpn_split[1677]: + unset netsv4
Nov 27 16:32:52 ovpn_split[1677]: + sleep 3
Nov 27 16:32:55 ovpn_split[1677]: + ipset -L NETFLIXCIDR
Nov 27 16:32:55 ovpn_split[1677]: Name: NETFLIXCIDR
Nov 27 16:32:55 ovpn_split[1677]: Type: hash:net
Nov 27 16:32:55 ovpn_split[1677]: Revision: 0
Nov 27 16:32:55 ovpn_split[1677]: Header: family inet hashsize 1024 maxelem 65536
Nov 27 16:32:55 ovpn_split[1677]: Size in memory: 9844
Nov 27 16:32:55 ovpn_split[1677]: References: 1
Nov 27 16:32:55 ovpn_split[1677]: Members:
Nov 27 16:32:55 ovpn_split[1677]: 198.38.119.0/24
Nov 27 16:32:55 ovpn_split[1677]: 23.246.31.0/24
Nov 27 16:32:55 ovpn_split[1677]: 45.57.0.0/17
Nov 27 16:32:55 ovpn_split[1677]: 198.38.117.0/24
Nov 27 16:32:55 ovpn_split[1677]: 23.246.0.0/18
Nov 27 16:32:55 ovpn_split[1677]: 185.2.221.0/24
Nov 27 16:32:55 ovpn_split[1677]: 198.38.118.0/24
Nov 27 16:32:55 ovpn_split[1677]: 23.246.28.0/24
Nov 27 16:32:55 ovpn_split[1677]: 198.45.48.0/20
Nov 27 16:32:55 ovpn_split[1677]: 37.77.186.0/23
Nov 27 16:32:55 ovpn_split[1677]: 208.75.79.0/24
Nov 27 16:32:55 ovpn_split[1677]: 208.75.76.0/22
Nov 27 16:32:55 ovpn_split[1677]: 37.77.188.0/23
Nov 27 16:32:55 ovpn_split[1677]: 64.120.128.0/17
Nov 27 16:32:55 ovpn_split[1677]: 198.38.120.0/24
Nov 27 16:32:55 ovpn_split[1677]: 185.2.220.0/24
Nov 27 16:32:55 ovpn_split[1677]: 69.53.227.0/24
Nov 27 16:32:55 ovpn_split[1677]: 208.75.77.0/24
Nov 27 16:32:55 ovpn_split[1677]: 69.53.242.0/24
Nov 27 16:32:55 ovpn_split[1677]: 23.246.15.0/24
Nov 27 16:32:55 ovpn_split[1677]: 198.38.96.0/19
Nov 27 16:32:55 ovpn_split[1677]: 69.53.253.0/24
Nov 27 16:32:55 ovpn_split[1677]: 192.173.64.0/18
Nov 27 16:32:55 ovpn_split[1677]: 66.197.128.0/17
Nov 27 16:32:55 ovpn_split[1677]: 37.77.184.0/23
Nov 27 16:32:55 ovpn_split[1677]: 69.53.229.0/24

I thought maybe there was a small chance that it was a timing issue. So I added an additional 30 second delay and added this portion to the end of the script. Same results from running from script on reboot.

 

[Martineau]

I would be happy if it would work in the script. However for me, it isn't working in the script

Cut'n'paste this script

#!/bin/sh

set -x # uncomment/comment to enable/disable debug mode

opkg install whob

netsv4=`whob -h whois.radb.net -- '-i origin AS40027'|grep -Eo "([0-9.]+){4}/[0-9]+"`;for net in $netsv4;do ipset -A NETFLIXCIDR $net;done;unset netsv4

netsv4=`whob -h whois.radb.net -- '-i origin AS55095'|grep -Eo "([0-9.]+){4}/[0-9]+"`;for net in $netsv4;do ipset -A NETFLIXCIDR $net;done;unset netsv4

netsv4=`whob -h whois.radb.net -- '-i origin AS2906'|grep -Eo "([0-9.]+){4}/[0-9]+"`;for net in $netsv4;do ipset -A NETFLIXCIDR $net;done;unset netsv4

ipset -L NETFLIXCIDR

 

[jinx10000]

Cut'n'paste this script

Still nothing. Copied and pasted directly into my current script through WINSCP minus the #!/bin/sh line.
Should I post it in a new script? If so, how do I call the new script?

Is it possible that running these two commands have caused issues with the formatting of the file? I found these in previous posts.
dos2unix /jffs/scripts/wan-start
sed -i 's/\r//g' /jffs/scripts/wan-start

Any specific way I should format the script?

I know I'm probably wearing you out with questions. I really do appreciate your help.

 

[Martineau]

Still nothing. Copied and pasted directly into my current script through WINSCP minus the #!/bin/sh line.
Should I post it in a new script? If so, how do I call the new script?

Is it possible that running these two commands have caused issues with the formatting of the file? I found these in previous posts.
dos2unix /jffs/scripts/wan-start
sed -i 's/\r//g' /jffs/scripts/wan-start

Any specific way I should format the script?

I know I'm probably wearing you out with questions. I really do appreciate your help.

When I said cut'n'paste I did not say remove MANDATORY lines and make up your own "!£$"%" code !!!!!!

I suggest you read/digest/comprehend the Wiki https://github.com/RMerl/asuswrt-merlin/wiki ....especially the section named 'User Scripts' etc.

When U cut'n'paste the lines, and they are in a script file (preferably with the standard '.sh' suffix to differentiate your custom scripts from the inbuilt scripts) and located in /jffs/scripts ...and you have ensured that it has the execute attributes (octal 777 or 755) as shown when you hit PF9 in WinSCP,...and although I recall that the internal WinSCP editor does correctly use Unix LF chars, in your case you should ensure that the script is in Unix format by executing

dos2unix  /jffs/scripts/name_of_your_script

then execute the script by name using this syntax

cd /jffs/scripts

./name_of_your_script

then if the code works, then you can call it from any of the system scripts such as wan-start/nat-start etc. by including the line

/jffs/scripts/name_of_your_script

 

[jinx10000]

When I said cut'n'paste I did not say remove MANDATORY lines and make up your own "!£$"%" code !!!!!!

Lol. I really do understand your frustration. Really made me laugh.

I understand the line I removed is mandatory but it is already in my script. That's why I was asking if you wanted me to make a new script.

I didn't make up the "!£$"%" code but I did run that previously on my main wan-start script and didn't know if it was causing an issue. I think it was something you posted about 10 pages back. Like I said I've been reading for weeks.

Since the time I posted my last post and the time that you responded I went back and looked at your nat-start post and figured out how to call a script.

I did this. Created a new text file. Called it RADB-Script.sh.
Copied your text EXACTLY into it.
Moved it into /jffs/scripts
From telnet tried to run it by typing /jffs/scripts/RADB-Script.sh. Of course it said I don't have permissions.
So I ran chmod a+rx /jffs/scripts/*
Then I got a complaint that it didn't exist.
So I ran: dos2unix /jffs/scripts/RADB-Script.sh
From telnet I ran /jffs/scripts/RADB-Script.sh
Everything worked perfectly!

Then I added this line to my main wan-start script.
/jffs/scripts/RADB-Script.sh

Still doesn't work. I don't know how to put your great faces in here but .

 

[Martineau]

Then I added this line to my main wan-start script.
/jffs/scripts/RADB-Script.sh

Still doesn't work. I don't know how to put your great faces in here but .

So RADB-Script.sh does work?....if so then try calling it from nat-start

or if that doesn't work then from init-start with an appropriate delay

#!/bin/sh

# Ensures only called once per boot after the router is definitely UP?
sleep 180

/jffs/scripts/RADB-Script.sh

 

[jinx10000]

So RADB-Script.sh does work?....if so then try calling it from nat-start

or if that doesn't work then from init-start with an appropriate delay

#!/bin/sh

# Ensures only called once per boot after the router is definitely UP?
sleep 180

/jffs/scripts/RADB-Script.sh

I will give that a try but I put it at the end of my wan-start script that already has a 100 second delay. Put this at the end of the script.

sleep 120
/jffs/scripts/RADB-Script.sh

So that is a 220 second delay not including the other pauses that I have and it is still not working.

I put the logging info back in it:

From script I get this:

Spoiler

Nov 27 18:58:43 ovpn_split[1524]: + opkg install whob
Nov 27 18:58:43 ovpn_split[1524]: Package whob (3.76-1) installed in root is up to date.
Nov 27 18:58:43 ovpn_split[1524]: + whob -h whois.radb.net -- -i origin AS40027
Nov 27 18:58:43 ovpn_split[1524]: + grep -Eo ([0-9.]+){4}/[0-9]+
Nov 27 18:58:44 ovpn_split[1524]: + netsv4=
Nov 27 18:58:44 ovpn_split[1524]: + unset netsv4
Nov 27 18:58:44 ovpn_split[1524]: + whob -h whois.radb.net -- -i origin AS55095
Nov 27 18:58:44 ovpn_split[1524]: + grep -Eo ([0-9.]+){4}/[0-9]+
Nov 27 18:58:44 ovpn_split[1524]: + netsv4=
Nov 27 18:58:44 ovpn_split[1524]: + unset netsv4
Nov 27 18:58:44 ovpn_split[1524]: + whob -h whois.radb.net -- -i origin AS2906
Nov 27 18:58:44 ovpn_split[1524]: + grep -Eo ([0-9.]+){4}/[0-9]+
Nov 27 18:58:44 ovpn_split[1524]: + netsv4=
Nov 27 18:58:44 ovpn_split[1524]: + unset netsv4
Nov 27 18:58:44 ovpn_split[1524]: + ipset -L NETFLIXCIDR

When running from telnet it shows all of the IPs being added still.

Edit:
Tried it from nat-start. Waited 20 seconds and same results.

 

[john9527]

Try adding the full path to the 'whob' command in the script....