x3mRouting x3mRouting ~ Selective Routing for Asuswrt-Merlin Firmware

[Xentrk]

hello again @Xentrk

im still curious about if i can have several routing rules running to the same vpn client?
for example:

x3mRouting ALL 1 aws1 dnsmasq_file=/jffs/scripts/x3mRouting/aws_domains_01
x3mRouting ALL 1 uic asnum=AS6200

if this is possible, where are the AS6200 ip addresses stored?

Yes, you can have many rules for the same VPN Client.

For the IPSET list "uic", x3mRouting will download the IPv4 addresses for AS6200 to the file /opt/tmp/uic. Once the download has been completed, x3mRouting will then load the IPSET list with the IPv4 addresses from the file /opt/tmp/uic. You can view the ipset list contents using the command ipset -L uic.

 

[Xentrk]

I'm currently running the 384.19 version of selective routing set of scripts, method 3 (scripts only.)
Over the past few days I made some changes to IPsets, interfaces they're routed through etc. It all works very well, as it always did
However, I've seen on more than a couple of occasions that the cron jobs get wiped out with the exception of Diversion and the last IPset I edited (which show the 2:00 am update correctly.) Everything else is gone. For all the scripts I'm running under normal conditions there are ~30 jobs in the list at any given time.

Has anyone else seen this issue?

Are all cron jobs getting wiped or just the ones for x3mRouting? I'm not aware of a command that will wipe all the jobs unless one rebooted or perhaps you wiped the contents of /tmp/var/spool/cron/crontabs?

Here are the messages x3mRouting creates when it creates and deletes and ipset list.

(x3mRouting.sh): 14273 CRON schedule created: #QQTEST# '0 2 * * * ipset save QQTEST'
(x3mRouting.sh): 11914 CRON schedule deleted: #QQTEST# '0 2 * * * ipset save QQTEST'

If the jobs are not getting created after a reboot, check the system log and see if your USB is taking longer than usual to mount properly. Try disabling the disk check utility if you have it enabled in amtm. You may want to try moving USB to from the 2.0 to the 3.0 port.

 

[andresmorago]

hi. me again

so after adding 3 different routing rules to my openvpn client 1, im only seeing 2 backup files instead of 3. i used to see a aws1 backup file under /opt/tmp and i dont to anymore.

x3mRouting ALL 1 aws1 dnsmasq_file=/jffs/scripts/x3mRouting/aws_domains_01
x3mRouting ALL 1 uic1 asnum=AS6200
x3mRouting ALL 1 uic2 asnum=AS698

at /opt/tmp im able to see uic1 and uic2 respectively but aws1 isnt being created at all.

 

[Xentrk]

hi. me again

so after adding 3 different routing rules to my openvpn client 1, im only seeing 2 backup files instead of 3. i used to see a aws1 backup file under /opt/tmp and i dont to anymore.

x3mRouting ALL 1 aws1 dnsmasq_file=/jffs/scripts/x3mRouting/aws_domains_01
x3mRouting ALL 1 uic1 asnum=AS6200
x3mRouting ALL 1 uic2 asnum=AS698

at /opt/tmp im able to see uic1 and uic2 respectively but aws1 isnt being created at all.

With the dnsmasq method, the IPv4 addresses are generated dynamically using the ipset feature built into dnsmasq using the top level domain names. They addresses are stored in memory as they are generated. A cron job runs at 2:00 AM which saves the contents of the ipset list stored in memory to /opt/tmp or the directory specified.

For the other methods, the IPv4 addresses are first downloaded or manually input into the file before being loaded into the ipset list's memory.

 

[Kingp1n]

@Xentrk
Can we use your script with unbound? It seems when I enable unbound with the internal VPN feature, HBOMax, Hule and Prime stop working. Just curious if anyone has had success using unbound with this script.

 

[Xentrk]

@Xentrk
Can we use your script with unbound? It seems when I enable unbound with the internal VPN feature, HBOMax, Hule and Prime stop working. Just curious if anyone has had success using unbound with this script.

It worked for me when I tested it. But for the dnsmasq method to work, you need to have dnsmasq enabled when using Unbound.

 

[andresmorago]

hello @Xentrk

so my very loved isp is blocking access to rstudio.com. none of my devices on my lan can access while i can from my mobile data connection.

tracetcp rstudio.com -t 200

Tracing route to 104.198.14.52 [52.14.198.104.bc.googleusercontent.com] on port 80
Over a maximum of 30 hops.
1       *       *       *       Request timed out.
2       *       *       *       Request timed out.
3       *       *       *       Request timed out.
4       *       *       *       Request timed out.
5       *       *       *       Request timed out.
6
Terminate Event Occurred.

tracetcp rstudio.com:443 -t 200

Tracing route to 104.198.14.52 [52.14.198.104.bc.googleusercontent.com] on port 443
Over a maximum of 30 hops.
1       *       *       *       Request timed out.
2       *       *       *       Request timed out.
3       *       *       *       Request timed out.
4       *       *       *       Request timed out.
5       *       *       *       Request timed out.
6       *
Terminate Event Occurred.

i have added ip address 104.198.14.52 to my aws1 ipset on the router

ipset add aws1 104.198.14.52

still, im not able to bypass that blockage and have this ip address routed through my openvpn client 1. the rest of the addresses on aws1 ipset do work ok (pandora.com, for example)

can you please give me some additional ideas?

thanks


EDIT:
Nevermind. Skynet was blocking it.

[$] /jffs/scripts/firewall stats search ip 104.198.14.52 10
=============================================================================================================
104.198.14.52 is NOT in set Skynet-Whitelist.
104.198.14.52 is in set Skynet-Blacklist.
104.198.14.52 is NOT in set Skynet-BlockedRanges.

Blacklist Reason;
 "BanMalware: firehol_level3.netset"


Associated Domain(s);
rstudio.com
storj.io

 

[Torson]

Is there any way to retrieve the local private IP address associated with the VPN provider's public IP from the command line (like shown on the VPN Status tab in the GUI)?
I see that the public VPN IP is set to the nvram, but the private local allocated IP is not - shows 10.8.0.2 for all 5 configured and working VPN clients.

 

[octopus]

Is there any way to retrieve the local private IP address associated with the VPN provider's public IP from the command line (like shown on the VPN Status tab in the GUI)?
I see that the public VPN IP is set to the nvram, but the private local allocated IP is not - shows 10.8.0.2 for all 5 configured and working VPN clients.

You can use this environment variables: "echo "${ifconfig local}"" display Local ip when vpn-client starts.

 

[Xentrk]

x3mRouting Version 2.3.0 (1 September, 2020)

Summary
-Modified how the creation and removal of RPDB rules for fwmarks are performed.
-Enhanced the 'del' parameter to also remove any VPN Server to IPSET routing rules that may exist for an IPSET list.
-Removed the tightly coupled integration with asuswrt-merlin to reduce the impact of future firmware changes to x3mRouting.

Details
x3mRouting.sh

-The fwmark RPDB rule assigned to a VPN client will now be deleted if no other rules exist for the fwmark RPDB rule when specifying both the 'ipset_name=' + 'del' parameters'.
-Rules and configurations for VPN Server to IPset LIST will also be removed, if they exist, when specifying both the 'ipset_name=' + 'del' parameters'. Prior to the change, one had to specify both the 'server=' and 'ipset_name=' parm

x3mRouting OpenVPN Screen users

-The fwmark RPDB rule for the VPN Client will only get created if an IPSET routing rule exists.
-The fwmark RPDB rule assigned to a VPN client or WAN iface will be deleted if no other rules exist for the fwmark RPDB rule when deleteing the IPSET from the modified OpenVPN Client Screen.

Change integration with asuswrt-merlin firmware

-Modify openvpn-event to execute updown-dns.sh and x3mvpnrouting.sh if installed rather than executing from the openvpn-event up/down files for each VPN client.
-Replaced vpnrouting.sh with x3mvpnrouting.sh script. Tuned code and eliminated the mount of the vpnrouting.sh file

 

[Xentrk]

2 September, 2020

Just pushed a minor update to the x3mRouting Menu to fix the code performing the cleanup of deprecated updown-dns.sh references in the vpnclientX-route-up and vpnclientX-route-pre-down files. No version change. Run option 5 after downloading the updated x3mRouting Menu.

 

[Xentrk]

Deleting the backup files
Currently, when you use the 'del' parm in x3mRouting.sh, it will delete all the IPSET references and routing rules except for the backup file (default location is "/opt/tmp"). The main reason for not deleting the file is it would have a negative impact on those who use the manual method, and perhaps the dnsmasq method, depending on your perspective.

The Amazon and AWS methods refresh the backup file from the cloud each time they are run. So deleting the backup file won't cause harm.

Whereas the manual method is a list that has been manually created/edited. If you specify the 'del' parm to remove a routing rule for the list, then run x3mRouting to create a new rule, the ipset list you created manually is now gone!

With dnsmasq method, the list is generated dynamically, a backup is saved at 2 AM. The backup is used to reload the IPSET list at system boot. If the list got deleted, dnsmasq would continue to generate the entries dynamically in real-time. Still, why delete the backup file and all of the entries that have been collected if one just wants to reuse the list for another rule?

My idea is to prompt if you want to remove the backup file. I also added more verbose output so you know what items the script is doing to clean-up the references and rules.

Preview

x3mRouting.sh ALL 1 MYLIST asnum=AS2906 del
(x3mRouting.sh): 20073 Starting Script Execution ALL 1 MYLIST asnum=AS2906 del
(x3mRouting.sh): 20073 Checking /jffs/configs/dnsmasq.conf.add...
(x3mRouting.sh): 20073 Checking /jffs/scripts/nat-start...
(x3mRouting.sh): 20073 Script entry for MYLIST deleted from /jffs/scripts/nat-start
(x3mRouting.sh): 20073 Checking /jffs/scripts/x3mRouting/vpnclient1-route-up...
(x3mRouting.sh): 20073 ipset MYLIST entry deleted from /jffs/scripts/x3mRouting/vpnclient1-route-up
(x3mRouting.sh): 20073 ipset MYLIST entry deleted from /jffs/scripts/x3mRouting/vpnclient1-route-pre-down
(x3mRouting.sh): 20073 Checking /jffs/scripts/x3mRouting/vpnclient3-route-up...
(x3mRouting.sh): 20073 Checking crontab...
(x3mRouting.sh): 20073 Checking PREROUTING iptables rules...
(x3mRouting.sh): 20073 Checking POSTROUTNG iptables rules...
(x3mRouting.sh): 20073 Checking if IPSET list MYLIST exists...
(x3mRouting.sh): 20073 IPSET MYLIST deleted!
(x3mRouting.sh): 20073 Checking if IPSET backup file exists...

DANGER ZONE!

Delete the backup file in /opt/tmp/MYLIST
[1]  --> Yes
[2]  --> No

[1-2]: 1

/opt/tmp/MYLIST file deleted.

(x3mRouting.sh): 20073 Completed Script Execution

Please let me know your thoughts!

 

[Kingp1n]

Anyone having issues with Disneyplus? It seems Disney started blacklising PIA as I can no longer get it work. This is what I have while using option 3 of the script:

x3mRouting.sh 1 0 DISNEYPLUS dnsmasq=disneyplus.com,disney-plus.net,thewaltdisneycompany.com,demdex.net,disney.com,disney.io,footprint.net,go.com
x3mRouting.sh 1 0 AMAZON asnum=AS16509

Any other ideas I can try?

 

[Xentrk]

Anyone having issues with Disneyplus? It seems Disney started blacklising PIA as I can no longer get it work.

This is what I have plus using Amazon GLOBAL region to route to my private IP.

sh /jffs/scripts/x3mRouting/x3mRouting.sh ALL 1 DISNEY dnsmasq=demdex.net,disney-plus.net,disneyplus.co,disneyplus.com,dssott.com,go.com

disneyplus.co is a new addition after I ran a new search for disney references:

sh autoscan.sh autoscan=disney

demdex.net
disney-plus.net
disneyplus.co
disneyplus.com
dssott.com
go.com

grep disney /opt/var/log/dnsmasq.log | grep query | awk '{print $6}' | sort -u

appconfigs.disney-plus.net
cdn.registerdisney.go.com
disney.demdex.net
disneyplus.co
disneyplus.com
preview.disneyplus.com
prod-ripcut-delivery.disney-plus.net
prod-static.disney-plus.net
sanalytics.disneyplus.com
search-api-disney.svcs.dssott.com
www.disneyplus.com

Currently, the autoscan.sh script only returns the top level domains as one would enter them in dnsmasq. I like seeing the additional detail though for analysis. Is it helpful?

 

[Kingp1n]

This is what I have plus using Amazon GLOBAL region to route to my private IP.

sh /jffs/scripts/x3mRouting/x3mRouting.sh ALL 1 DISNEY dnsmasq=demdex.net,disney-plus.net,disneyplus.co,disneyplus.com,dssott.com,go.com

disneyplus.co is a new addition after I ran a new search for disney references:

sh autoscan.sh autoscan=disney

demdex.net
disney-plus.net
disneyplus.co
disneyplus.com
dssott.com
go.com

grep disney /opt/var/log/dnsmasq.log | grep query | awk '{print $6}' | sort -u

appconfigs.disney-plus.net
cdn.registerdisney.go.com
disney.demdex.net
disneyplus.co
disneyplus.com
preview.disneyplus.com
prod-ripcut-delivery.disney-plus.net
prod-static.disney-plus.net
sanalytics.disneyplus.com
search-api-disney.svcs.dssott.com
www.disneyplus.com

Currently, the autoscan.sh script only returns the top level domains as one would enter them in dnsmasq. I like seeing the additional detail though for analysis. Is it helpful?

I added those additional ones you provided and same thing. However, when I added the amazon global ( x3mRouting 1 0 AMAZON-GLOBAL aws_region=GLOBAL), it started working again!!!

Thanks ALOT!

BTW....should I remove the aws_region=US or leave it along with using GLOBAL as well? Can you I use both?

 

[Xentrk]

x3mRouting.sh Script Update (3 September, 2020)

+ Update 'del' x3mRouting script option to include additional output messages, add -w option to grep for exact match of IPSET name in files, prompt for removal of back-up file.
+ Fix incorrect variable name in shebang check function

Spoiler: 'del' parm

x3mRouting.sh ALL 1 MYLIST asnum=AS2906 del
(x3mRouting.sh): 20073 Starting Script Execution ALL 1 MYLIST asnum=AS2906 del
(x3mRouting.sh): 20073 Checking /jffs/configs/dnsmasq.conf.add...
(x3mRouting.sh): 20073 Checking /jffs/scripts/nat-start...
(x3mRouting.sh): 20073 Script entry for MYLIST deleted from /jffs/scripts/nat-start
(x3mRouting.sh): 20073 Checking /jffs/scripts/x3mRouting/vpnclient1-route-up...
(x3mRouting.sh): 20073 ipset MYLIST entry deleted from /jffs/scripts/x3mRouting/vpnclient1-route-up
(x3mRouting.sh): 20073 ipset MYLIST entry deleted from /jffs/scripts/x3mRouting/vpnclient1-route-pre-down
(x3mRouting.sh): 20073 Checking /jffs/scripts/x3mRouting/vpnclient3-route-up...
(x3mRouting.sh): 20073 Checking crontab...
(x3mRouting.sh): 20073 Checking PREROUTING iptables rules...
(x3mRouting.sh): 20073 Checking POSTROUTNG iptables rules...
(x3mRouting.sh): 20073 Checking if IPSET list MYLIST exists...
(x3mRouting.sh): 20073 IPSET MYLIST deleted!
(x3mRouting.sh): 20073 Checking if IPSET backup file exists...

DANGER ZONE!

Delete the backup file in /opt/tmp/MYLIST
[1] --> Yes
[2] --> No

[1-2]: 1

/opt/tmp/MYLIST file deleted.

(x3mRouting.sh): 20073 Completed Script Execution

 

[Xentrk]

I added those additional ones you provided and same thing. However, when I added the amazon global ( x3mRouting 1 0 AMAZON-GLOBAL aws_region=GLOBAL), it started working again!!!

Thanks ALOT!

BTW....should I remove the aws_region=US or leave it along with using GLOBAL as well? Can you I use both?

I ended up dropping the US region a month or so ago when I was having an issue and went GLOBAL. It's been working fine since then. Let me know your experience. I'll change the references on the readme to use GLOBAL going forward once confirmed.

 

[mister]

Dear Xentrk,
Since the last update, my access to my VPN Server doesn´t work anymore.
I didn´t route the traffic via a specific vpn client, but over the WAN port by using Routing rules in the VPN client section. (e.g. Routing 192.168.5.0/24 (VPN Server net) to 192.168.111.0/24 (Intranet): WAN and the other way round).

Unfortunately I don´t get an response from my VPN Server anymore, if I try to access. The IP adress for the server is resolved correctly. He is permanently waiting for server response so I think it is a routing problem.

Is it possible to modify your commands ?

VPN Server to VPN Client Routing

Route from VPN Server 1,2 or both to VPN Client 1,2,3,4 or 5.

x3mRouting server=1 client=1
x3mRouting server=2 client=1
x3mRouting server=both client=1

to:
VPN Server to VPN Client Routing

Route from VPN Server 1,2 or both to WAN

x3mRouting server=both client=0 ??


What could be my problem ?

Thanks a lot again

Hugo.

 

[mister]

Sorry , my mistake. I had at one VPN "force all traffic over VPN" activated instead of using policies. :-( .

Sorry!

 

[OKLY]

I'm not sure why the IP address list downloaded via ASN number is always 2KB and not complete. I'm trying to exclude all Akamai CDN servers and the list downloaded from ipinfo never seem to be a complete list.