Menu
What's new
By:
Filters Better Search

Privacy Filter (Another IPSET Script)

  • SNBForums Code of Conduct

    SNBForums is a community for everyone, no matter what their level of experience.

    Please be tolerant and patient of others, especially newcomers. We are all here to share and learn!

    The rules are simple: Be patient, be nice, be helpful or be gone!

my idea was to check if there is a .add file check if that add file has a entry for privacy filter then if it hasnt add it, in addition i wanted the script if it has a .add file check if the conf has loaded the privacy filter if not add it

think this is a secure way of doing it.
 
swetoast said:
my idea was to check if there is a .add file check if that add file has a entry for privacy filter then if it hasnt add it, in addition i wanted the script if it has a .add file check if the conf has loaded the privacy filter if not add it

think this is a secure way of doing it.
Click to expand...

Your script in its present version checks to see if the .add file is there.... and if it isn't then check to see if it has an entry for privacy-filter?... if the .add file doesn't exist then no point trying to grep it.
swetoast said:
if [ ! -f $dnsmasq_cfg ]; then if grep -Fxq "#privacy-filter" $dnsmasq_cfg then logger -s -t privacy-filter is present in $dnsmasq_cfg
Click to expand...
 
So a new revision is up

Changes:
  • Added Requirement for entware since it now needs the package hostip to look up ip from dns
  • Rewrite for blocking domains.
  • A new blocklist for privacy
Note: if you get this [name does not exist] when running privacy block then its perfectly fine its just hostip messages if the dns is not working.
 
ok from the weeds and bushes, is there a step by step on how to do this to a router. where to place files how to execute etc? totally new to this but familiar with bat files and such. I looked at the other scripts you had also and just cannot grasp where to place as my ac56r under merlin does not have the dirs mentioned in the scripts and if I make them and place the .sh set to execute it never goes and even if I telnet in and launch it does nothing. is there a kindergarten class somewhere for this?
thanks
 
well yeah

https://github.com/RMerl/asuswrt-merlin/wiki/Entware

you need to install entware in order to use this particular script so begin with that.

And if you need support on that start a new thread after you installed entware you just place that script at /jffs/script/firewall-start and another script at where ever you want preferable /opt/var/cache/privacy-filter/privacy-filter.list and then make /jffs/script/firewall-start executable

this all takes basic linux knowledge and the wiki is always a good place for info along with this forum.
 
thanks for the info.hate to admit I use to maintain aix boxes so some what have a feel for it.
will go for it and thanks for the info.


swetoast said:
well yeah

https://github.com/RMerl/asuswrt-merlin/wiki/Entware

you need to install entware in order to use this particular script so begin with that.

And if you need support on that start a new thread after you installed entware you just place that script at /jffs/script/firewall-start and another script at where ever you want preferable /opt/var/cache/privacy-filter/privacy-filter.list and then make /jffs/script/firewall-start executable

this all takes basic linux knowledge and the wiki is always a good place for info along with this forum.
Click to expand...
 
I updated to the latest version as well as the new blocklist. Unfortunately, one or more of the google.com hosts in the blocklist result in Gmail no longer being able to authenticate, occurring on multiple Android devices:

Code: 
googleadservices.com
www.google-analytics.com
google-analytics.com
ssl.google-analytics.com

I temporarily removed all four hosts, and Gmail works again.

Has anyone else experienced this?
 
nope, but leave em out if they cause an issue for you

note: removed em so i dont have to give support if they can cause a problem, its good enough blocking telemetry and shodan.io and chinese collection servers.

plus its easy enough for adding more dnses to block if anyone wants to block more.
 
@swetoast. Hmm the requirement for this is Entware? As you probably know from my other thread, I did this without Entware or installing any thirdparty program.
 
im working on making it without entware but this revision uses the package hostip to resolve dns to ip, im looking for a way to not use that package unless entware is already install.. so to summarize im working on it :)
 
Oh Okay. Well I'm not using the latest revision, I'm using the one before that, so I guess Entware doesn't apply to me then :)
 
exactly and again the goal is to have it working without entware but its a challange cause i want to resolve the ips from the dns before the block since some of the domain doesnt have static ips
 
well i made something that kinda works but there is an issue

Code: 
for i in `cat $path/privacy-filter.list`; do nslookup $i | tail -3 | grep -oE "$regexp" >> $path/privacy_block.pre; done

the issue is that it lets LAN IP into the mix and thats a deal breaker for me, so if anyone got some sort of solution beside scrubbing the blocklist for LAN IP then please suggest something.

Code: 
for i in `cat $path/privacy-filter.list`; do traceroute $i | head -1 | grep -oE "$regexp" >> $path/privacy_block.pre; done

same issue with this line, so far hostip package is the most secure way of doing this.

bumped it to rev6 with initial way to do it without entware when we find a secure way to do it
 
Last edited:
I have rt-ac3200 running 380.65 which i hard reset
installed entware in putty using entware-setup.sh installs no errors
using winscp to copy the two files from OP to jffs/scripts and set them both to 0755

my firewall-start
Code: 
#!/bin/sh
# Author: Toast
# Contributers: Tomsk
# Revision 5

path=/jffs/scripts/privacy-filter.list                  # Set your path here
regexp=`echo "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b"`         # Dont change this value

if [ -z "$(which opkg)" ]; then logger -s -t system "no package manager found"; exit 0; else
if [ -z "$(opkg list-installed | grep hostip)" ]; then opkg install hostip; fi fi

        if [ -f $path/privacy_block.list ]; then rm $path/privacy_block.list; fi
        for i in `cat $path/privacy-filter.list`; do hostip $i >>$path/privacy_block.pre; done
        sort -u $path/privacy_block.pre > $path/privacy_block.list
        if [ -f $path/privacy_block.pre ]; then rm $path/privacy_block.pre; fi

case $(ipset -v | grep -oE "ipset v[0-9]") in
*v6) # Value for ARM Routers

    MATCH_SET='--match-set'
    HASH='hash:ip'
    SYNTAX='add'
    SWAPPED='swap'
    DESTROYED='destroy'

     ipsetv=6
     lsmod | grep "xt_set" > /dev/null 2>&1 || \
     for module in ip_set ip_set_hash_net ip_set_hash_ip xt_set
     do
          insmod $module
     done
;;

*v4) # Value for Mips Routers

    MATCH_SET='--set'
    HASH='iphash'
    SYNTAX='-q -A'
    SWAPPED='-W'
    DESTROYED='--destroy'

     ipsetv=4
     lsmod | grep "ipt_set" > /dev/null 2>&1 || \
     for module in ip_set ip_set_nethash ip_set_iphash ipt_set
     do
          insmod $module
     done
;;
esac


run_ipset () {

ipset -L privacy-filter >/dev/null 2>&1
if [ $? -ne 0 ]; then
    if [ "$(ipset --swap privacy-filter privacy-filter 2>&1 | grep -E 'Unknown set|The set with the given name does not exist')" != "" ]; then
    nice ipset -N privacy-filter $HASH
    for i in `cat $path/privacy_block.list`; do nice -n 2 ipset $SYNTAX privacy-filter $i ; done
fi
else
    nice -n 2 ipset -N privacy-update $HASH
    for i in `cat $path/privacy_block.list`; do nice -n 2 ipset $SYNTAX privacy-update $i ; done
    nice -n 2 ipset $SWAPPED privacy-update privacy-filter
    nice -n 2 ipset $DESTROYED privacy-update
fi

iptables -L | grep privacy-filter > /dev/null 2>&1
if [ $? -ne 0 ]; then
    nice -n 2 iptables -I FORWARD -m set $MATCH_SET privacy-filter src,dst -j REJECT
else
    nice -n 2 iptables -D FORWARD -m set $MATCH_SET privacy-filter src,dst -j REJECT
    nice -n 2 iptables -I FORWARD -m set $MATCH_SET privacy-filter src,dst -j REJECT
fi
}

run_ipset
exit $?
}

run_ipset
exit $?

my privacy-filter.list

Code: 
bing.com
a.ads1.msn.com
a.ads2.msads.net
a.ads2.msn.com
a.rad.msn.com
a-0001.a-msedge.net
a-0002.a-msedge.net
a-0003.a-msedge.net
a-0004.a-msedge.net
a-0005.a-msedge.net
a-0006.a-msedge.net
a-0007.a-msedge.net
a-0008.a-msedge.net
a-0009.a-msedge.net
ac3.msn.com
ad.doubleclick.net
adnexus.net
adnxs.com
ads.msn.com
ads1.msads.net
ads1.msn.com
aidps.atdmt.com
aka-cdn-ns.adtech.de
a-msedge.net
apps.skype.com
az361816.vo.msecnd.net
az512334.vo.msecnd.net
b.ads1.msn.com
b.ads2.msads.net
b.rad.msn.com
bs.serving-sys.com
c.atdmt.com
c.msn.com
cdn.atdmt.com
cds26.ams9.msecn.net
choice.microsoft.com
choice.microsoft.com.nsatc.net
compatexchange.cloudapp.net
corp.sts.microsoft.com
corpext.msitadfs.glbdns2.microsoft.com
cs1.wpc.v0cdn.net
db3aqu.atdmt.com
df.telemetry.microsoft.com
diagnostics.support.microsoft.com
ec.atdmt.com
fe2.update.microsoft.com.akadns.net
feedback.microsoft-hohm.com
feedback.search.microsoft.com
feedback.windows.com
flex.msn.com
g.msn.com
h1.msn.com
i1.services.social.microsoft.com
i1.services.social.microsoft.com.nsatc.net
lb1.www.ms.akadns.net
live.rads.msn.com
m.adnxs.com
m.hotmail.com
msedge.net
msftncsi.com
msnbot-65-55-108-23.search.msn.com
msntest.serving-sys.com
oca.telemetry.microsoft.com
oca.telemetry.microsoft.com.nsatc.net
pre.footprintpredict.com
preview.msn.com
pricelist.skype.com
rad.live.com
rad.msn.com
redir.metaservices.microsoft.com
reports.wes.df.telemetry.microsoft.com
s.gateway.messenger.live.com
s0.2mdn.net
schemas.microsoft.akadns.net
secure.adnxs.com
secure.flashtalking.com
services.wes.df.telemetry.microsoft.com
settings-sandbox.data.microsoft.com
settings-win.data.microsoft.com
sls.update.microsoft.com.akadns.net
sqm.df.telemetry.microsoft.com
sqm.telemetry.microsoft.com
sqm.telemetry.microsoft.com.nsatc.net
static.2mdn.net
statsfe1.ws.microsoft.com
statsfe2.update.microsoft.com.akadns.net
statsfe2.ws.microsoft.com
survey.watson.microsoft.com
telecommand.telemetry.microsoft.com
telecommand.telemetry.microsoft.com.nsatc.net
telemetry.appex.bing.net
telemetry.microsoft.com
telemetry.urs.microsoft.com
view.atdmt.com
vortex.data.microsoft.com
vortex-bn2.metron.live.com.nsatc.net
vortex-cy2.metron.live.com.nsatc.net
vortex-sandbox.data.microsoft.com
vortex-win.data.microsoft.com
watson.live.com
watson.microsoft.com
watson.ppe.telemetry.microsoft.com
watson.telemetry.microsoft.com
watson.telemetry.microsoft.com.nsatc.net
wes.df.telemetry.microsoft.com
www.msftncsi.com
www.msftncsi.org
nametests.com
oyag.lhzbdvm.com
oyag.prugskh.net
oyag.prugskh.com
census1.shodan.io
census2.shodan.io
census3.shodan.io
census4.shodan.io
census5.shodan.io
census6.shodan.io
census7.shodan.io
census8.shodan.io
census9.shodan.io
census10.shodan.io
census11.shodan.io
census12.shodan.io
atlantic.census.shodan.io
pacific.census.shodan.io
rim.census.shodan.io
pirate.census.shodan.io
ninja.census.shodan.io
border.census.shodan.io
burger.census.shodan.io
atlantic.dns.shodan.io
hello.data.shodan.io

I have rebooted but nothing on privacy list is blocked i can still ping all of them. I apologise in advance if i'm being thick, my only experience is following guide for disable win 10 tracking, which used to work and now doesn't.
 
you have a broken script at the end there is a double command

Code: 
run_ipset
exit $?
}

run_ipset
exit $?
should just be

Code: 
run_ipset
exit $?

ohh and rebooting resets your firewall until the script runs again.
 
too check if its working or not just type

Code: 
iptables -vnL FORWARD

should read out something like

Code: 
 pkts bytes target     prot opt in     out     source               destination         
  425 20088 REJECT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            match-set privacy-filter src,dst reject-with icmp-port-unreachable
 
Thanks for pointing out my inability to copy/paste :p

the result of
iptables -vnL FORWARD
Code: 
Chain FORWARD (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 REJECT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            match-set privacy-filter src,dst reject-with icmp-port-unreachable
   87 40309 ACCEPT     all  --  tun11  *       0.0.0.0/0            0.0.0.0/0   
  534 44761 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            state RELATED,ESTABLISHED
    0     0 DROP       all  --  !br0   eth0    0.0.0.0/0            0.0.0.0/0   
    0     0 DROP       all  --  eth0   *       0.0.0.0/0            0.0.0.0/0            state INVALID
    0     0 ACCEPT     all  --  br0    br0     0.0.0.0/0            0.0.0.0/0   
  141 23435 NSFW       all  --  *      *       0.0.0.0/0            0.0.0.0/0   
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate DNAT
  141 23435 ACCEPT     all  --  br0    *       0.0.0.0/0            0.0.0.0/0

still isnt working btw, I am sorry for being such a pia.
 
give it time mine didnt start immediately either

keyword here is patience

Code: 
watch iptables -vnL FORWARD | grep privacy
 1513 71592 REJECT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            match-set privacy-filter src,dst reject-with icmp-port-unreachable
 1516 71736 REJECT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            match-set privacy-filter src,dst reject-with icmp-port-unreachable
 1517 71776 REJECT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            match-set privacy-filter src,dst reject-with icmp-port-unreachable
 1520 71928 REJECT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            match-set privacy-filter src,dst reject-with icmp-port-unreachable
 1523 72072 REJECT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            match-set privacy-filter src,dst reject-with icmp-port-unreachable
 1526 72212 REJECT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            match-set privacy-filter src,dst reject-with icmp-port-unreachable

Windows 10 doesnt talk all the time to its telemetry servers not do you get hit by shodan scanners
 
You must log in or register to reply here.

Similar threads

Xentrk
ipset example script on GitHub..Edits made for arm/mips syntax
Replies
0
Views
2K
Xentrk
Xentrk
Xentrk
Selective Routing over OpenVPN tunnel gone awry
Replies
11
Views
3K
Xentrk
Xentrk
Similar threads
Thread starter Title Forum Replies Date
M RT-AX86U and DNS Privacy over USB WAN Asuswrt-Merlin 0
T Merlin closed source parts and privacy questions Asuswrt-Merlin 15
TheLyppardMan MAC Filter Options Asuswrt-Merlin 29
M Wireless Mac filter defaulting to whitelist. Asuswrt-Merlin 5
R Wireless MAC (Band filter filtering out Guest network) Asuswrt-Merlin 2
G Feature request: firewall url filter log Asuswrt-Merlin 0
TanyaC cannot add domain to router URL filter Asuswrt-Merlin 15
C How to filter out foreign countries Asuswrt-Merlin 3
P Using Avahi to cast to a PyChromeCast on another subnet Asuswrt-Merlin 4
S Transfer files to another device (raspi) in the same network Asuswrt-Merlin 3

Similar threads

Latest threads

Support SNBForums w/ Amazon

If you'd like to support SNBForums, just use this link and buy anything on Amazon. Thanks!

Sign Up For SNBForums Daily Digest

Get an update of what's new every day delivered to your mailbox. Sign up here!

Members online

Total: 680 (members: 31, guests: 649)
Top