Need a script that auto reboot if internet is down

[xScoutx]

FYI, I changed the script to check every 2 minutes to make it re-initiate the WAN connection in a reasonable amount of time. When the WAN goes down, by the time the script checks each IP address listed and then retries them the specified number of times in the script, it can be many minutes. I think mine with 3 IP addresses and 3 tries and then restarting the WAN only (i.e. not rebooting the router), it can take from 5 to 15 minutes...can't remember for sure.

Just saying to keep an eye on it if/when it happens...first see how long it takes to restart the connection and then adjust the parameters as needed. I'd say you don't want to ping too frequently.

Thanks. For my purposes I'm thinking about skipping the Is_Private_IPv4() method entirely. I think I will drop it to two minutes too and probably remove some of the guts to create a "CheckLan.sh" for this node. For now it's appears it is working though. Thanks @Martineau, great script.

 

[stefan21]

I have a question of understanding. Here's my setup:

Default message log level --> notice
Log only messages more urgent than --> info

/jffs/scripts# cat services-start
#!/bin/sh
cru a root_servers "12 4 * * * curl -o \/opt\/var\/lib\/unbound\/root\.hints https://www.internic.net/domain/named.cache" # unbound_manager
cru a Restart_WAN "*/2 * * * * /jffs/scripts/ChkWAN.sh wan quiet force nowait" # check every 2 minutes internet connection if fail restore WAN
cru a Reboot_WAN "*/3 * * * * /jffs/scripts/ChkWAN.sh reboot quiet force nowait" # check every 3 minutes s.o. if fail reboot

#cru l
12 4 * * * curl -o \/opt\/var\/lib\/unbound\/root\.hints https://www.internic.net/domain/named.cache #root_servers#
*/2 * * * * /jffs/scripts/ChkWAN.sh wan quiet force nowait #Restart_WAN#
*/3 * * * * /jffs/scripts/ChkWAN.sh reboot quiet force nowait #Reboot_WAN#
59 * * * * /jffs/addons/unbound/unbound_stats.sh generate #Unbound_Stats.sh#
57 * * * * /jffs/addons/unbound/unbound_log.sh #Unbound_Log.sh#
*/10 * * * * /jffs/scripts/ntpmerlin generate #ntpMerlin#

Everything is working.

In the admin-panel under syslog:
...
Oct 16 00:06:01 (ChkWAN.sh): 17580 v1.17 Monitoring WAN connection using 5 target PING hosts (192.168.x.1 1.1.1.1 149.112.112.112 9.9.9.9 1.1.1.1) (Tries=3)
Oct 16 00:06:01 (ChkWAN.sh): 17580 Private LAN 192.168.x.1 will be skipped for WAN PING check!
Oct 16 00:06:02 (ChkWAN.sh): 17580 cURL 10MByte transfer took: 00:00.64 secs @16284790 B/sec
Oct 16 00:08:01 (ChkWAN.sh): 18214 v1.17 Monitoring WAN connection using 5 target PING hosts (192.168.x.1 1.1.1.1 149.112.112.112 9.9.9.9 1.1.1.1) (Tries=3)
Oct 16 00:08:01 (ChkWAN.sh): 18214 Private LAN 192.168.x.1 will be skipped for WAN PING check!
Oct 16 00:08:03 (ChkWAN.sh): 18214 cURL 10MByte transfer took: 00:00.68 secs @15434239 B/sec
Oct 16 00:09:01 (ChkWAN.sh): 18562 v1.17 Monitoring WAN connection using 5 target PING hosts (192.168.x.1 1.1.1.1 149.112.112.112 9.9.9.9 1.1.1.1) (Tries=3)
Oct 16 00:09:01 (ChkWAN.sh): 18562 Private LAN 192.168.x.1 will be skipped for WAN PING check!
Oct 16 00:09:02 (ChkWAN.sh): 18562 cURL 10MByte transfer took: 00:00.62 secs @16832102 B/sec
Oct 16 00:10:01 (ChkWAN.sh): 18912 v1.17 Monitoring WAN connection using 5 target PING hosts (192.168.x.1 1.1.1.1 149.112.112.112 9.9.9.9 1.1.1.1) (Tries=3)
Oct 16 00:10:02 (ChkWAN.sh): 18912 Private LAN 192.168.x.1 will be skipped for WAN PING check!
Oct 16 00:10:02 (ChkWAN.sh): 18912 cURL 10MByte transfer took: 00:00.61 secs @17258692 B/sec
Oct 16 00:12:00 (ChkWAN.sh): 19807 v1.17 Check WAN monitor ALREADY running...ABORTing
Oct 16 00:12:01 (ChkWAN.sh): 19806 v1.17 Monitoring WAN connection using 5 target PING hosts (192.168.x.1 1.1.1.1 149.112.112.112 9.9.9.9 1.1.1.1) (Tries=3)
Oct 16 00:12:02 (ChkWAN.sh): 19806 Private LAN 192.168.x.1 will be skipped for WAN PING check!
Oct 16 00:12:02 (ChkWAN.sh): 19806 cURL 10MByte transfer took: 00:00.64 secs @16364132 B/sec
Oct 16 00:14:01 (ChkWAN.sh): 20437 v1.17 Monitoring WAN connection using 5 target PING hosts (192.168.x.1 1.1.1.1 149.112.112.112 9.9.9.9 1.1.1.1) (Tries=3)
Oct 16 00:14:01 (ChkWAN.sh): 20437 Private LAN 192.168.x.1 will be skipped for WAN PING check!
Oct 16 00:14:03 (ChkWAN.sh): 20437 cURL 10MByte transfer took: 00:00.62 secs @16953066 B/sec
Oct 16 00:15:01 (ChkWAN.sh): 20785 v1.17 Monitoring WAN connection using 5 target PING hosts (192.168.x.1 1.1.1.1 149.112.112.112 9.9.9.9 1.1.1.1) (Tries=3)
Oct 16 00:15:01 (ChkWAN.sh): 20785 Private LAN 192.168.x.1 will be skipped for WAN PING check!
Oct 16 00:15:03 (ChkWAN.sh): 20785 cURL 10MByte transfer took: 00:01.14 secs @9222142 B/sec
...

What exactly does "quiet" mean? I would have expected that logging s.o. is quiet?

and:

."..Check WAN monitor ALREADY running...ABORTing"

Does this mean, the interval between the two jobs is too short?

Thank's for clarification.
stefan

 

[ColinTaylor]

What exactly does "quiet" mean?

It means there are fewer messages in the log.

."..Check WAN monitor ALREADY running...ABORTing"

Does this mean, the interval between the two jobs is too short?

Your two cru statements have conflicting time intervals. Every 6 minutes the two processes will run at the same time.

 

[stefan21]

It means there are fewer messages in the log.


Your two cru statements have conflicting time intervals. Every 6 minutes the two processes will run at the same time.

O.k. got it.

Your two cru statements have conflicting time intervals. Every 6 minutes the two processes will run at the same time.

Douhh! Stupid me. Of course. While doing a little math and thinking over, I could have find out for myself. Sorry for bothering.

Thank you for your help.

 

[V_2]

Could you please confirm the correct syntax to restart the secondary WAN (LAN4) connection at say 3AM?

It seems my Dual WAN setup stops using the secondary WAN connection every other day around 3AM but toggling it off and back on in the WebUI fixes that. Dont need periodic monitoring and not sure it would even work since the secondary WAN connection is technically still up.

 

[ColinTaylor]

Could you please confirm the correct syntax to restart the secondary WAN (LAN4) connection at say 3AM?

It seems my Dual WAN setup stops using the secondary WAN connection every other day around 3AM but toggling it off and back on in the WebUI fixes that. Dont need periodic monitoring and not sure it would even work since the secondary WAN connection is technically still up.

cru a RestartWAN1 "00 03 * * * service 'restart_wan_if 1'"

 

[V_2]

cru a RestartWAN1 "00 03 * * * service 'restart_wan_if 1'"

Thank you for replying so quickly!

Stopped being lazy myself and started looking at the script's code, saw that it uses "service restart_wan_if" then reviewed the services code to figure out that I should reference "1". Posting what I did for someone else searching.

To get persistence, I added the line below in /jffs/scripts/wan-event (remember reading a while ago CRU schedules does not save on reboot). Seemed that script is the correct place, found a reference saying wan-start is deprecated?. /jffs/scripts/services-start

There was no file with that name so i created it with nano, added the CRU command then made the script executable

Create script file

nano /jffs/scripts/wan-event

Add this line to script file

cru a Restart_WAN2 "15 3 * * * services \"restart_wan_if 1\""

Make script executable

chmod 755 /jffs/scripts/wan-event

Will know if this worked in a few days. Hope I am not misguiding anyone with the above .

thanks for the corrections from ColinTaylor, adjusted the above

 

[ColinTaylor]

@V_2 Don't use wan-event, use services-start instead. It only needs to setup the cron job once when the router starts up.

If the script doesn't already exist the first line needs to be:

#!/bin/sh

 

[V_2]

@V_2 Don't use wan-event, use services-start instead. It only needs to setup the cron job once when the router starts up.

If the script doesn't already exist the first line needs to be:

#!/bin/sh

One more thank you!

Still a lot to learn/remember on *nix systems. Here's Wikipedia article on what this extra line does. Not very user friendly but I presume this is legacy stuff that just carried over.

 

[V_2]

One more thank you!

Still a lot to learn/remember on *nix systems. Here's Wikipedia article on what this extra line does. Not very user friendly but I presume this is legacy stuff that just carried over.

Now this got my interest and instead of hard-coding the secondary WAN restart at a specific time, was wondering how to correctly create the CRU jobs to run the ChkWAN.sh script so that the script can check and restart either of the two connections. I do not use "failover" but rather "load balancing" mode for Dual WAN and have specified ranges of internal network IPs that would exit via each WAN (using the routing rules).

cru a Check_Wan1 "* * * * * ./ChkWAN.sh ping=1.1.1.1,8.8.8.8,4.2.2.2,9.9.9.9 tries=1 nowait verbose wan once i=wan0"
cru a Check_Wan2 "* * * * * ./ChkWAN.sh ping=1.1.1.1,8.8.8.8,4.2.2.2,9.9.9.9 tries=1 nowait verbose wan once i=wan1"

Would this conflict since it runs every minute at the same time?

EDIT: I thought I posted a reply but seems i did not. To avoid the conflict that ColinTaylor pointed out, I setup the check for each interface to run on odd and even minutes like so:

cru a Check_WAN1 "*/2 * * * * /jffs/scripts/ChkWAN.sh ping=1.1.1.1,8.8.8.8,4.2.2.2,9.9.9.9 tries=1 nowait quiet wan once i=wan0"
cru a Check_WAN2 "1-59/2 * * * * /jffs/scripts/ChkWAN.sh ping=1.1.1.1,8.8.8.8,4.2.2.2,9.9.9.9 tries=1 nowait quiet wan once i=wan1"

 

[ColinTaylor]

Would this conflict since it runs every minute at the same time?

Yes. See post #103.

 

[mdover]

This script has been working great for me until the last few days. Unfortunately, my internet has gone down every day or two for the last week and I see disconnected on the router main page but the router is reporting a successful ping to 9.9.9.9 so it doesn’t initiate a reboot through the script. I’m not very good at this - I reviewed the log and can’t figure it out.

The latest occurrence happened sometime around 00:16 on Nov 21, as I received an email alert from synology that my NAS went offline around that time. So at least that gives me a place to start as I try to figure out what’s happening.

I’ve attached the log. Can anyone whose familiar with this stuff take a quick peek at that time frame and see if it’s anything obvious?

Here's a link to a bit of the log from when it went offline. I tried pasting it below but the forum isn't allowing. I've also attached the full log, but I think the answer is probably embedded in the part that I've posted on pastebin and provided the link for. Maybe it’s an obvious message that I just don’t understand. I guess because it still gets a successful ping to 9.9.9.9 it doesn't reboot - but the internet is effectively down and reports as diconnected.

Nov 21 00:16:06 (ChkWAN.sh): 2230 Monitoring connection OK.....(Successful ping - Pastebin.com

Pastebin.com is the number one paste tool since 2002. Pastebin is a website where you can store text online for a set period of time.

pastebin.com

Thank you for any assistance!

https://pastebin.com/8b23psA3

 

[mdover]

Went offline again today. This has me stumped.

 

[Martineau]

This script has been working great for me until the last few days. Unfortunately, my internet has gone down every day or two for the last week and I see disconnected on the router main page but the router is reporting a successful ping to 9.9.9.9 so it doesn’t initiate a reboot through the script. I’m not very good at this - I reviewed the log and can’t figure it out.

The latest occurrence happened sometime around 00:16 on Nov 21, as I received an email alert from synology that my NAS went offline around that time. So at least that gives me a place to start as I try to figure out what’s happening.

I’ve attached the log. Can anyone whose familiar with this stuff take a quick peek at that time frame and see if it’s anything obvious?

Here's a link to a bit of the log from when it went offline. I tried pasting it below but the forum isn't allowing. I've also attached the full log, but I think the answer is probably embedded in the part that I've posted on pastebin and provided the link for. Maybe it’s an obvious message that I just don’t understand. I guess because it still gets a successful ping to 9.9.9.9 it doesn't reboot - but the internet is effectively down and reports as diconnected.

Nov 21 00:16:06 (ChkWAN.sh): 2230 Monitoring connection OK.....(Successful ping - Pastebin.com

Pastebin.com is the number one paste tool since 2002. Pastebin is a website where you can store text online for a set period of time.

pastebin.com

Thank you for any assistance!

https://pastebin.com/8b23psA3

If there is a false positive for the ping 9.9.9.9, then perhaps you could temporarily override the ping and force a curl data transfer?

./ChkWAN.sh nowait reboot curl

 

[Jedis]

Hey there. Without fully understanding what this script is doing for every parameter - I am wanting it to test the internet connection, and if it's down, reboot the device. I am calling it via cron with the following entry, to check every 5 minutes. Would appreciate some guidance to confirm it will actually work in the manner I want it, with the following cron entry:

*/5 * * * * /etc/init.d/checkwan.sh reboot force nowait #Reboot_WAN#

Is this all that I need? The script itself mentions a "/jffs/scripts/ChkWAN_Reset_CRON.sh" script, but I don't see that in the GitHub repo. Is that old cruft that was never cleaned out? Some of the log messages are confusing. For example, with that running, it shows it will ping targets with 3 attempts, but then actually does what it should, performing a large file download.

On another note, I forked the script on GitHub and made a few small changes to get this to run on a Netgear Nighthawk hotspot (MR variants). They didn't have cru on them, so with a bit of searching, I found the script and added it to supplement some of the functionality for devices missing it. They also do not have curl, but have wget. I ended up downloading an armv7 binary of curl to get it working in a hurry, but did notice wget/curl is mentioned interchangeably, with some of the variables named accordingly.

So far, the script seems to work, at least for testing a good connection. I have not been able to simulate an outage to see if the device actually reboots as it should just yet.

 

[gelsbern]

Loving this script. It is amazing and works well. Any chance for an enhancement maybe where it sends an email when it has to reboot?

 

[L&LD]

amtm supports a built-in email script. But how will the email get sent if the WAN is down?

 

[gelsbern]

Perhaps it could write something to the file system to flag that it needs to send an email when lan is restored?

 

[Nesalex]

Loving this script. It is amazing and works well. Any chance for an enhancement maybe where it sends an email when it has to reboot?

wicens script

 

[Martineau]

Perhaps it could write something to the file system to flag that it needs to send an email when lan is restored?

Do you mean when the WAN is retored?

See https://github.com/RMerl/asuswrt-merlin.ng/wiki/User-scripts#wan-event

So you would create a script that fires on the 'WAN0-connected' event and calls your email script.