Privacy Filter (Another IPSET Script)

[swetoast]

Did you even check the wiki ? there is no s in AB-Solution there so there is nothing to change might have done a typo here in the thread but that doesnt matter.

 

[thelonelycoder]

Did you even check the wiki ? there is no s in AB-Solution there so there is nothing to change might have done a typo here in the thread but that doesnt matter.

I merely pointed out a spelling error of my app name, your name is swetoast, not swetoasts, isn't it?
All I ask is to correct it in the quoted text in post #1 in this thread.
No need to be confrontational about this.

 

[swetoast]

not confrontational at all just pointing out the fact that we all spell wrong at time to time but as long as the documentation is in order and that there are no spelling errors there then its all good and English is not my native language.

 

[swetoast]

but post 1 edited and i removed the entry about and AB-Solution & ublockr all together and left that on the wiki, hope that settles it.

sorry for the confusion

 

[FalconB]

AB-Solution does not block incoming traffic.
If it were so, you would not need the privacy filter as I could simply use the same filter lists and put @swetoast's great work out of business...

AB simply tells dnsmasq to direct queries for blocked domains FROM clients to an internal IP address (0.0.0.0 or the pixelserv-tls IP) instead of the real domain's IP address.
So, if any of the domains in the privacy filter also happen to be in the blocking file or blacklist of AB, then these domains are already blocked from reaching the real server.
Preventing them from sending telemetry data to their greedy servers.
The Privacy filter and AB complement each other, some overlap is expected but it has no effect on the overall outcome.
In cases where they do overlap, your devices are still prevented from sending telemetry data and you will not see ads.

OK, thanks for the explanation. But from what I understand, beeing a novice at these kinds of things, AB is using the domain name (ie www.somesite.net) to redirect the DNS-queries to the internal IP address/pixelsrv while PF is using the domain name for populating its PF list and then using only the IP address to block outgoing traffic. Wouldn't that still cause an issue as I described before, or have I messed things up in my poor lite mind ? I am sorry if this is common knowledge to everyone but me, just trying to understand it all...

@thelonelycoder & @swetoast: A suggestion to maybe limit the number of quesitions regarding how your respectively scripts work (like mine ), would be to create a topic in your FAQ called "How does the script work" (or similar) where you explain in text not only what the script does but how it does it. Or maybe that's way to complicated to formulate in text?

Anyways, thanks for great scripts and taking the time to answer and explain!

 

[swetoast]

Bumped officially to rev 20, tnx to everyone who tested.

• Minor tweaks and fixes to the script
• New sorting system
• No dependacy on entware anymore

https://gitlab.com/swe_toast/privacy-filter/raw/master/privacy-filter

 

[Xentrk]

All three ASUS Routers running Merlin FW and one router running DD-WRT have been updated with version 20. Everything working. Many thanks!

 

[swetoast]

Awesome next version of this will support CIDR ranges and then we will block MS even more

 

[FalconB]

Bumped officially to rev 20, tnx to everyone who tested.

• Minor tweaks and fixes to the script
• New sorting system
• No dependacy on entware anywmore

https://gitlab.com/swe_toast/privacy-filter/raw/master/privacy-filter

Hi again!

Tried to update but got an error. I found it to be in the funciton below:

run_ipv4_block () {
if [ -f /tmp/privacy-filter_ipv4_sorted.part ]; then rm /tmp/privacy-filter_ipv4_sorted.part; fi
cat /tmp/privacy-filter_raw.part | \
awk '!/(^127\.)|(^10\.)|(^172\.1[6-9]\.)|(^172\.2[0-9]\.)|(^172\.3[0-1]\.)|(^192\.168\.)/' privacy-filter_raw.part | \
grep -oE "$regexp_v4" | sort -u > /tmp/privacy-filter_ipv4_sorted.part
}

In the "awk"-commandline the path is missing, ie "/tmp/" ahead of "privacy-filter_raw.part". So instead of

awk '!/(^127\.)|(^10\.)|(^172\.1[6-9]\.)|(^172\.2[0-9]\.)|(^172\.3[0-1]\.)|(^192\.168\.)/' privacy-filter_raw.part | \

it should be

awk '!/(^127\.)|(^10\.)|(^172\.1[6-9]\.)|(^172\.2[0-9]\.)|(^172\.3[0-1]\.)|(^192\.168\.)/' /tmp/privacy-filter_raw.part | \

That fixed the problem for me...

 

[swetoast]

will fix

 

[swetoast]

Rev 21

• ipv6 fixes
• path fixes (tnx FalconB)

https://gitlab.com/swe_toast/privacy-filter/raw/master/privacy-filter

 

[FalconB]

will fix

Great! Another thing In your wiki you have for all of your scripts the following:

Finally call this at the end of your existing /jffs/firewall-start:

Should be:

Finally call this at the end of your existing /jffs/scripts/firewall-start:

 

[swetoast]

done and done

 

[FalconB]

done and done

Well done!

 

[skeal]

done and done

On the github site in your installation instructions it says that if you are running ab-solution to disable it to update privacy filter.
Can you tell me how to temporarily disable ab-solution as the process is not clear to me please?

 

[sfx2000]

Awesome next version of this will support CIDR ranges and then we will block MS even more

CIDR ranges are important.... as we're seeing cloud based scanners these days...

See below - without CIDR range blocks...

 --------------------- fail2ban-messages Begin ------------------------
 Banned services with Fail2Ban:                             Bans:Unbans
   sshd:                                                   [1123:1123]
      116.31.116.15                                          18:18
      31.207.47.36                                            9:9
      91.197.232.103                                          8:8
      103.79.143.132                                          8:8
      111.40.166.130                                          8:8
      166.130.8.104 (mobile-166-130-8-104.mycingular.net)     8:8
      76.75.17.130                                            7:7
      201.177.31.187 (201-177-31-187.speedy.com.ar)           6:6
      39.155.136.34                                           6:6
      46.237.127.58 (static.bulsat.com)                       6:6
      61.177.172.14                                           6:6
      62.61.163.20                                            6:6
      69.81.51.209 (user-12l2cuh.cable.mindspring.com)        6:6
      103.207.37.169                                          6:6
      111.202.133.66                                          6:6
      113.73.119.0                                            6:6

blah blah blah - lot's of single entries below this line...

And for what it's worth - over the same timeframe...

sshd:
Authentication Failures:
root (116.31.116.15): 394 Time(s)

It's China...

whois 116.31.116.15
% [whois.apnic.net]
% Whois data copyright terms    http://www.apnic.net/db/dbcopyright.html

% Information related to '116.16.0.0 - 116.31.255.255'

inetnum:        116.16.0.0 - 116.31.255.255
netname:        CHINANET-GD
descr:          CHINANET Guangdong province network
descr:          China Telecom
descr:          No.31,jingrong street
descr:          Beijing 100032
country:        CN
admin-c:        CH93-AP
tech-c:         IC83-AP
mnt-by:         APNIC-HM
mnt-lower:      MAINT-CHINANET-GD
mnt-routes:     MAINT-CHINANET-GD
status:         ALLOCATED PORTABLE
remarks:        --------------------------------------------------------
remarks:        To report network abuse, please contact mnt-irt
remarks:        For troubleshooting, please contact tech-c and admin-c
remarks:        Report invalid contact via www.apnic.net/invalidcontact
remarks:        --------------------------------------------------------
source:         APNIC
mnt-irt:        IRT-CHINANET-CN
changed:        hm-changed@apnic.net 20070307

 

[swetoast]

@skeal ask on the AB-Solution thread for support on that script

@sfx2000 yeah my other filter for malware has CIDR ranges support for just that purpose this filter is more about privacy rights for snooping services but Malware-filter has just that purpose your on about there

 

[thelonelycoder]

@swetoast, how exactly do you make sure uBlockr does not block the update of the filter list?
I'd like to make sure AB users don't need to disable ad-blocking for the update to get the full list.

 

[swetoast]

well it fairly easy, dont use every list out there, but if you have a simple deactive and active command i could include a check for your script and deactive and re activate again after update.

else i have to figure out some other way to get my results bringing back hostip or something.

 

[thelonelycoder]

well it fairly easy, dont use every list out there, but if you have a simple deactive and active command i could include a check for your script and deactive and re activate again after update.

else i have to figure out some other way to get my results bringing back hostip or something.

AB is not built with command line in mind, too many things are intertwined for all the services to work.
So, a simple command to disable/enable it is not possible atm.
Are there specific domains that need to be whitelisted so that it works?