Privacy Filter (Another IPSET Script)

[swetoast]

well it actually is within / / if you look at the regexp

and it still doesnt explain why it works for the rest of us when we are running the same software

system: Privacy Filter (ipv4) loaded 119 unique ip addresses that will be rejected from contacting your router.

 

[Martineau]

well it actually is within / / if you look at the regexp

and it still doesnt explain why it works for the rest of us when we are running the same software

system: Privacy Filter (ipv4) loaded 119 unique ip addresses that will be rejected from contacting your router.

Explicitly enclose the awk 'program' with single quotes

awk '$local_v4' .............

or provide the OP with a diagnostic version of the script that adds the debug directives around the statement that is causing the syntax error.

 

[swetoast]

done and done

 

[eclp]

OK
Thank you very much! The syntax error is history ... but why are no IPv6 addresses loaded?

ASUSWRT-Merlin RT-AC87U 380.65-4 Wed Mar 29 04:40:59 UTC 2017
...@RT-AC87U:/tmp/home/root# /jffs/scripts/privacy-filter
system: Privacy Filter (ipv4) loaded 81 unique ip addresses that will be rejected from contacting your router.
system: Privacy Filter (ipv6) loaded 0 unique ip addresses that will be rejected  from contacting your router.

 

[swetoast]

is IPv6 enabled in your router ? and is the firewall for ipv6 enabled ?

 

[swetoast]

could try and install hostip via entware

opkg install hostip

if ipv6 loads addresses then i got a bug present

 

[mrchow]

OK
Thank you very much! The syntax error is history ... but why are no IPv6 addresses loaded?

ASUSWRT-Merlin RT-AC87U 380.65-4 Wed Mar 29 04:40:59 UTC 2017
...@RT-AC87U:/tmp/home/root# /jffs/scripts/privacy-filter
system: Privacy Filter (ipv4) loaded 81 unique ip addresses that will be rejected from contacting your router.
system: Privacy Filter (ipv6) loaded 0 unique ip addresses that will be rejected  from contacting your router.

It only loads the specific lists if you use the protocols in question.


Sent from my iPhone using Tapatalk

 

[eclp]

is IPv6 enabled in your router ? and is the firewall for ipv6 enabled ?

yes and yes

ASUSWRT-Merlin RT-AC87U 380.65-4 Wed Mar 29 04:40:59 UTC 2017
...@RT-AC87U:/tmp/home/root# /jffs/scripts/privacy-filter
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
[no records in the reply]
system: Privacy Filter (ipv4) loaded 152 unique ip addresses that will be rejected from contacting your router.
system: Privacy Filter (ipv6) loaded 12 unique ip addresses that will be rejected from contacting your router.

 

[swetoast]

kewl tnx.. will take care of it in due time, thx for confirming.

 

[swetoast]

cat /jffs/privacy-filter.list | xargs -n 5 -I {} sh -c "nslookup {}" | grep -i "Address" | awk '{print $3}'

this might be simpler to maintain less crap to deal with

 

[jayten]

@jayten there is a command for showing how much is blocked in the thread

but here it is again

iptables -L -v | grep "privacy-filter_ipv"

the numbers in the row indicate how much was blocked

 pkts bytes target     prot opt in     out     source               destination      
  158  8232 REJECT     all  --  any    any     anywhere             anywhere             match-set privacy-filter_ipv4 dst reject-with icmp-port-unreachable

Dont remember if windows 7 updates was able to remove telemetry rarely run older OS anywhere

Yes, for the given URLs, it definitely works I'm thinking a bit more broadly, i.e. tracking if Microsoft is implementing updates that are changing URLs and IP addresses on a regular basis which would require making changes to the filter list constantly. It's not something specific to the privacy-filter script.

 

[swetoast]

so i found a reliable way to get ip addresses and sort em accordingly, also gonna implement CIDR ranges and whitelisting for some services incase the CIDR blocks too much.

https://gitlab.com/swe_toast/privacy-filter/issues/3
https://gitlab.com/swe_toast/privacy-filter/issues/4

will update in due time cause im working alot this week so i acknowledge that there is a bug in IPv6 blocking and that it doesnt block ipv6 addresses without hostip installed.

But i got a roadmap and a solution for these issues and a new version is in the works just need to find some time to implement em.

 

[swetoast]

test version is up

NOTE: test only if it doesnt work for you report it here then revert to rev 19.

Rev 20

• Changes to how lists are sorted and ip addresses are addded.
• Hostip is not needed anymore.

https://gitlab.com/swe_toast/privacy-filter/raw/master/WIP/privacy-filter

 

[swetoast]

wow no takers ? or is there nothing to report on this version if so ill bump it officially

 

[eclp]

Rev 20:

ASUSWRT-Merlin RT-AC87U 380.65-4 Wed Mar 29 04:40:59 UTC 2017
...@RT-AC87U:/tmp/home/root# /jffs/scripts/privacy-filter
nslookup: can't resolve 'oyag.prugskh.net'
nslookup: can't resolve 'oyag.prugskh.com'
ipset v6.29: Element cannot be added to the set: it's already added
ipset v6.29: Element cannot be added to the set: it's already added
ipset v6.29: Element cannot be added to the set: it's already added
ipset v6.29: Element cannot be added to the set: it's already added
ipset v6.29: Element cannot be added to the set: it's already added
ipset v6.29: Element cannot be added to the set: it's already added
ipset v6.29: Element cannot be added to the set: it's already added
system: Privacy Filter (ipv4) loaded 76 unique ip addresses that will be rejected from contacting your router.
system: Privacy Filter (ipv6) loaded 6 unique ip addresses that will be rejected from contacting your router.

 

[swetoast]

seems that you updated twice there considering the already added message, and for the no resolve there thats nothing to worry about.

btw are you running AB-Solution also ?

 

[Xentrk]

wow no takers ? or is there nothing to report on this version if so ill bump it officially

Something not right from my testing...

If I run the new version of the privacy-filter script. Since it is already active, I get errors:

nslookup: can't resolve 's.gateway.messenger.live.com'
ipset v6.29: Element cannot be added to the set: it's already added
ipset v6.29: Element cannot be added to the set: it's already added
ipset v6.29: Element cannot be added to the set: it's already added...

So, I rebooted to get a clean run and get the following in System Log:

system: Privacy Filter (ipv4) loaded 0 unique ip addresses that will be rejected from contacting your router.

We had a power outage last night. When I powered the router back on this morning, the prior version in the log reports:

system: Privacy Filter (ipv4) loaded 57 unique ip addresses that will be rejected from contacting your router.

I then ran the prior version from the command line:

nslookup: can't resolve 's.gateway.messenger.live.com'

system: Privacy Filter (ipv4) loaded 49 unique ip addresses that will be rejected from contacting your router.

I suspect if I reboot, it will load 57 instead of the 49.

 

[swetoast]

i updated the test build again added sort -u to the script to sort out duplicate the low numbers are due to ab-solutions large blocklists mine is getting around 117-120 blocks

 

[Xentrk]

i updated the test build again added sort -u to the script to sort out duplicate the low numbers are due to ab-solutions large blocklists mine is getting around 117-120 blocks

That is good to know! That makes sense. I ran the updated version and it appears to work.

nslookup: can't resolve 's.gateway.messenger.live.com'

system: Privacy Filter (ipv4) loaded 49 unique ip addresses that will be rejected from contacting your router.

I suspect if I reboot and run it will load more.

Is s.gateway.messenger.live.com being blocked because privacy-filer or malware-filter was already active when I ran the script?

admin@RT-AC88U:/tmp/home/root# nslookup s.gateway.messenger.live.com
Server:    127.0.0.1
Address 1: 127.0.0.1 localhost.localdomain

 

[swetoast]

still that should only result in the entry being removed by

https://gitlab.com/swe_toast/privacy-filter/blob/master/WIP/privacy-filter#L57