iblocklist.com generic ipset loader for ipset v6 and v4

[Csection]

I am having trouble with yahoo mail after I run this script. I have put yahoo into the "Whitelist" file, but it still disrupts the yahoo mail interface and makes it run very slowly. Any ideas why this script is playing havoc with yahoo?
I am only using list #'s : 13, 14, 15, 27 and 34.

 

[redhat27]

yahoo mail interface and makes it run very slowly

Couple of things to try:

• If you know which IPs are being blocked (nslookup/hostip the slow domain), you can use the shell function to determine which ipset is blocking it.
• You can change the IPTABLES_BLOCK_TARGET to REJECT for immediate return on failure (will not be slow)
• Is you are using an older version on pixelserv in conjunction with ad-blocking, make use to use the latest version of pixelserv-tls.
  

 

[Jack Yaz]

Couple of things to try:

• If you know which IPs are being blocked (nslookup/hostip the slow domain), you can use the shell function to determine which ipset is blocking it.
• You can change the IPTABLES_BLOCK_TARGET to REJECT for immediate return on failure (will not be slow)
• Is you are using an older version on pixelserv in conjunction with ad-blocking, make use to use the latest version of pixelserv-tls.

The firewall-start reload script 't doesn't work on boot. I end up with the below. It seems firewall start is firing before service-start has finished.

admin@RT-AC87U:/tmp/home/root# blockstats
 pkts bytes target     prot opt in     out     source               destination
    0     0 DROP       all  --  any    any     anywhere             anywhere             match-set t1IP src,dst
    0     0 DROP       all  --  any    any     anywhere             anywhere             match-set tCIDR src,dst
    2    80 DROP       all  --  any    any     anywhere             anywhere             match-set YAMalwareBlockCIDR src
    0     0 DROP       all  --  any    any     anywhere             anywhere             match-set YAMalwareBlock2IP src
    2    84 DROP       all  --  any    any     anywhere             anywhere             match-set YAMalwareBlock1IP src

My services-start, for reference:

#!/bin/sh

RC='/opt/etc/init.d/rc.unslung'

i=90
until [ -x "$RC" ] ; do
  i=$(($i-1))
  if [ "$i" -lt 1 ] ; then
    logger "Could not start Entware"
    exit
  fi
  sleep 1
done
$RC start

# Load TOR, Country and Microsoft Telemetry blocking rules
# Load malicious IP blocking, and malware IP blocking rules
sh /jffs/scripts/ya-malware-block.sh && sh /jffs/scripts/iblocklist-loader.sh

 

[redhat27]

The firewall-start reload script 't doesn't work on boot. I end up with the below. It seems firewall start is firing before service-start has finished.

Okay, I agree that keeping that iptables rule reinstatement snippet before the service-start finished would create iptables entries for the temporary sets. Not good. Since the firewall-start may get called multiple times when your router has just rebooted and is busy init-ing at lot of stuff, I'd recommend either introducing a long sleep (I'd say around 600 seconds) before the call to both your scripts (last line)
OR
Remove the services-start call altogether and rely on cru to run those
OR
I put some sort of a flag on that snippet to check if these scripts are running before trying to reinstate the iptables rules. The last option is a bit messy (need to create a /tmp file marker or query ps) Any of the first two should work without issues.

 

[Jack Yaz]

Would a sleep in services-start not cause a block in the boot sequence?

 

[redhat27]

Would a sleep in services-start not cause a block in the boot sequence?

No, since all the other services have already started at $RC start you should be good. services-start is non-blocking afaik (no other process waits for service-start to complete)

 

[Jack Yaz]

@RMerlin are you able to comment please? I don't want to potentially put a block in my boot, the wife would have my guts for garters - the rebooting tonight to test is already pushing it!

 

[redhat27]

the wife would have my guts for garters

 

[Jack Yaz]

Interestingly, even without the firewall reload, iBlocklist doesnt come up on boot, but malware does. And they're both called in a now separate script (so services-start references one file), and the file is the 2 command chained together with &&

Running it manually is fine, so I wonder what iblocklist is doing that boot time doesn't like

 

[Adamm]

Interestingly, even without the firewall reload, iBlocklist doesnt come up on boot, but malware does. And they're both called in a now separate script (so services-start references one file), and the file is the 2 command chained together with &&

Running it manually is fine, so I wonder what iblocklist is doing that boot time doesn't like

The system is picky with what it runs, I was having issues in my script on boot because of how I was blocking stderr and stdout on my iptables commands, could be something similar.

 

[redhat27]

It should work.

edit: @Adamm, these scripts do no such thing

You can also call them in succession (in a separate line or with ; instead of &&)
&& will not run the following command if for some reason the first one exits with a non-zero exit code

 

[Jack Yaz]

The system is picky with what it runs, I was having issues in my script on boot because of how I was blocking stderr and stdout on my iptables commands, could be something similar.

I notice some &>/dev/null, which is stdout, and others with 2>/dev/null which is stderr, could those be the issue you saw @Adamm ?

 

[Adamm]

I notice some &>/dev/null, which is stdout, and others with 2>/dev/null which is stderr, could those be the issue you saw @Adamm ?

I haven't looked closely enough at his code to give an educated guess, but just something to keep in mind as it took me awhile to figure out with tunnel vision. I'd start by looking at all the code that isn't being executed, then see what they have in common, could be something entirely different though.

 

[Jack Yaz]

It should work.

edit: @Adamm, these scripts do no such thing

You can also call them in succession (in a separate line or with ; instead of &&)
&& will not run the following command if for some reason the first one exits with a non-zero exit code

The separate lines seem to have done the trick. Wonder why malware isnt a non-zero, surely it just exits after the logger call and would be 0?

 

[Adamm]

Sorry I glanced over and misread, I thought you were saying his script wasn't being executed properly from firewall-start in its default state.

 

[redhat27]

it just exits after the logger call and would be 0

My would say that [ -t 1 ] is returning a false. It will only return true if called in an interactive shell, not without an attached terminal (like the case with run from init)

 

[redhat27]

I notice some &>/dev/null, which is stdout, and others with 2>/dev/null which is stderr, could those be the issue

These would never block or fail... I do not see your point

 

[RMerlin]

@RMerlin are you able to comment please? I don't want to potentially put a block in my boot, the wife would have my guts for garters - the rebooting tonight to test is already pushing it!

None of the -start scripts are blocking, they all run in parallel to the rest of the firmware.

The postconf scripts are the only ones that can block the router, and the mount ones (if I recall, I probably documented it on the Wiki).

 

[Jack Yaz]

These would never block or fail... I do not see your point

I was following @Adamm's musings, that was all. Posted before you replied saying that behaviour wasn't present in your scripts. Apologies for any confusion

None of the -start scripts are blocking, they all run in parallel to the rest of the firmware.

The postconf scripts are the only ones that can block the router, and the mount ones (if I recall, I probably documented it on the Wiki).

Thanks for confirming, I think I saw on the Wiki that something was blocking, and forgot to check it again before mis-assuming. In any case, i threw a sleep 300 in, and works nicely now. Thanks all.

 

[Xentrk]

One of the teachers is a bing search engine user. But it is being blocked by multiple sources:

MatchIP 204.79.197.200
204.79.197.200 not found in WhitelistDomains
204.79.197.200 found in BlacklistDomains
204.79.197.200 not found in BluetackProxyCIDR
204.79.197.200 not found in BluetackProxySingle
204.79.197.200 not found in BluetackWebexploitCIDR
204.79.197.200 not found in BluetackWebexploitSingle
204.79.197.200 not found in BluetackDshieldCIDR
204.79.197.200 not found in BluetackDshieldSingle
204.79.197.200 not found in BluetackSpiderCIDR
204.79.197.200 not found in BluetackSpiderSingle
204.79.197.200 found in YAMalwareBlock1IP
204.79.197.200 not found in YAMalwareBlock2IP
204.79.197.200 not found in YAMalwareBlockCIDR
204.79.197.200 not found in BlockedCountries
204.79.197.200 not found in TorNodes
204.79.197.200 not found in BruteForceLogins
204.79.197.200 not found in Whitelist
204.79.197.200 not found in Blacklist
204.79.197.200 found in YAMalwareBlock1IP
204.79.197.200 not found in YAMalwareBlock2IP
204.79.197.200 not found in YAMalwareBlockCIDR
204.79.197.200 not found in Blacklist
204.79.197.200 found in MicrosoftSpyServers

As a result, how do you recommend I proceed in getting it to work?