BACKUPMON BACKUPMON v1.5.10 -Mar 1, 2024- Backup/Restore your Router: JFFS + NVRAM + External USB Drive! (**Thread closed due to age**)

[Tech9]

My GT-AX6000 also has very little unused RAM — currently, 92% used

No, your GT-AX6000 router boots with >450MB free RAM. RT-AX86S has <100MB after boot.

Your RAM is used by buffers and memory management is possible. Not possible when you have no RAM available. I don't even know how @CaptainSTX fit all the scripts there. I had one RT-AX86S for tests and the 512MB RAM it has is just enough to run stock Asuswrt with Trend Micro. This model is not good for Asuswrt-Merlin with scripts on top. What most likely happens is OOM condition and active swapping. It brings down the router to a crawl speed and no wonder some processes take hours to complete.

 

[visortgw]

You might be backing up your backups to your backups, and they just keep multiplying in size!

Which is why I added *.tar and *.gz to my recommended exclusions to handle "orphans" of aborted backups...

 

[Viktor Jaep]

No, your GT-AX6000 router boots with about 450MB free RAM. RT-AX86S has <100Mb after boot.

Your RAM is used by buffers and memory management is possible. Not possible when you have no RAM available. I don't even know how @CaptainSTX fit all the scripts there. I had one RT-AX86S for tests and the 512MB RAM it has is just enough to run stock Asuswrt with Trend Micro. This model is not good for Asuswrt-Merlin with scripts on top. What most likely happens is OOM condition and active swapping. It brings down the router to a crawl speed and no wonder some processes take hours to complete.

Let's hope it's something easier to fix!

 

[Ripshod]

You might be backing up your backups to your backups

And possibly attempting to backup the backup you're currently backing up(?).
Don't think about that too much - my head!
We should stop right there and wait for @CaptainSTX to clear this up.

 

[Viktor Jaep]

And possibly attempting to backup the backup you're currently backing up(?).
Don't think about that too much - my head!

Yes, this circular logic is completely logical in this case!

 

[Tech9]

I'm just worrying about the cat.

 

[Viktor Jaep]

I'm just worrying about the cat.

 

[Tech9]

Sorry, but my Cat6 is way faster. Some say it can do 180K miles per second or higher.

 

[JGrana]

Sorry, but my Cat6 is way faster. Some say it can do 180K miles per second or higher.

Then I would be very careful with that Cat6. According to Einstein, it’s mass will be approaching infinite.

Just warning you ;-)

 

[Viktor Jaep]

@CaptainSTX ... please give this version a shot, and see if it's still inserting a bunch of garbage into your inbound emails, OK?

Between b2 -> b3... Just some small edits to the look/feel of the emails... the main fix might have been on the AMTM library script which inserted a <CR> after the email header, so we shall see!

curl --retry 3 "https://raw.githubusercontent.com/ViktorJp/BACKUPMON/master/backupmon-1.5.3b3.sh" -o "/jffs/scripts/backupmon.sh" && chmod 755 "/jffs/scripts/backupmon.sh"

 

[Martinski]

... the main fix might have been on the AMTM library script which inserted a <CR> after the email header, so we shall see!

Not a carriage return (CR), but a linefeed (LF) is now inserted between the header and the body of the email.

 

[Ripshod]

@CaptainSTX ... please give this version a shot, and see if it's still inserting a bunch of garbage into your inbound emails, OK?

Between b2 -> b3... Just some small edits to the look/feel of the emails... the main fix might have been on the AMTM library script which inserted a <CR> after the email header, so we shall see!

curl --retry 3 "https://raw.githubusercontent.com/ViktorJp/BACKUPMON/master/backupmon-1.5.3b3.sh" -o "/jffs/scripts/backupmon.sh" && chmod 755 "/jffs/scripts/backupmon.sh"

That's a fix for me

This is a TEST to check & verify if sending email notifications is working well from BACKUPMON.

Sent by the "backupmon.sh" script.
From the "AX88U" router.

2024-Feb-14, 02:01:04 PM GMT (Wed)

Date/Time: Feb 14 2024 14:04:09
Asus Router Model: RT-AX88U
Firmware/Build Number: 3004.388.6_0
EXT USB Drive Label Name: asus

SUCCESS: BACKUPMON completed a successful primary backup to destination: Network.

Sent by the "backupmon.sh" script.
From the "AX88U" router.

2024-Feb-14, 02:04:09 PM GMT (Wed)

Brilliant work. Strange how GMX is the only email provider (so far) to pick up the <CR> and act in this way.
@CaptainSTX your turn!

 

[Viktor Jaep]

That's a fix for me

Brilliant work. Strange how GMX is the only email provider (so far) to pick up the <CR> and act in this way.
@CaptainSTX your turn!

Thanks for the validation, @Ripshod ... that was all @Martinski's brilliant idea! Glad to see things working as they should now!

 

[CaptainSTX]

Yes, this circular logic is completely logical in this case!

OK Based on everyone's comments I am now backing up the SSD to another location which solves the long backup time.

Hadn't really noticed the issue previously as backup ran late at night.

Emails still have verbose header as I'm still running 1.5.2.b1.

 

[Viktor Jaep]

OK Based on everyone's comments I am now backing up the SSD to another location which solves the long backup time.

You could still continue backing up to your USB drive, but you would just need to be sure to add the necessary exclusions so that your backups don't back up your existing backups, and start taking longer and longer exponentially.

Emails still have verbose header as I'm still running 1.5.2.b1.

Please upgrade to 1.5.3b3 here and see if you get the same results as @Ripshod:

curl --retry 3 "https://raw.githubusercontent.com/ViktorJp/BACKUPMON/master/backupmon-1.5.3b3.sh" -o "/jffs/scripts/backupmon.sh" && chmod 755 "/jffs/scripts/backupmon.sh"

 

[CaptainSTX]

You could still continue backing up to your USB drive, but you would just need to be sure to add the necessary exclusions so that your backups don't back up your existing backups, and start taking longer and longer exponentially.


Please upgrade to 1.5.3b3 here and see if you get the same results as @Ripshod:

curl --retry 3 "https://raw.githubusercontent.com/ViktorJp/BACKUPMON/master/backupmon-1.5.3b3.sh" -o "/jffs/scripts/backupmon.sh" && chmod 755 "/jffs/scripts/backupmon.sh"

That fixed the problem. Thank you and to everyone that contributed to the solution.

I tried several variations of the exclusion file and nothing seemed to scale down the file size. Just worked better when I moved the backup to another USB. Didn't setup with NAS as I only use NAS for backups of PCs and the NAS powers itself off in the evening after backups completed.

Thanks again.

 

[Viktor Jaep]

Happy Valentine's Day! Forget flowers... chocolates... candy... date night... your sweetie really wants a new BACKUPMON update! Download it for her - she will be forever grateful for keeping her router safe! LOL Well, I'm very happy with how the AMTM email notifications implementation has gone, with many thanks to @Martinski for making this a cakewalk using his awesome AMTM shared email library! Enjoy!

What's new?
v1.5.3 - (February 14, 2024)
- MINOR: Added new functionality to BACKUPMON to give you the ability to receive backup SUCCESS and FAILURE email notifications. This capability is provided through the AMTM Email functionality (AMTM->em) and made possible through a wonderful common library graciously made available by @Martinski! When you enable the AMTM email notification functionality in the config menu (option #14), the script will download a library file into /jffs/addons/shared-libs. Libraries and functions like these can be shared between many other scripts from one common location. In BACKUPMON, you can enable notifcations for either SUCCESS or FAILURE events, or both. If using primary and secondary backups, you will get notifications for both, whether they succeed or fail. PLEASE NOTE: AMTM Email (AMTM->em) must be set up, configured and working before enabling this functionality in BACKUPMON.
- PATCH: Changed the versioning logic to align with the general accepted way of versioning, using the notation: major.minor.patch ... finally, right? After seeing @thelonelycoder changing his ways, I figured it was probably time for me as well. All my scripts moving forward will go this route. Change log wording is now changed to conform to the major/minor/patch standards. So previously, FIXED now conforms to PATCH, ADDED conforms to MINOR, and MAJOR stays the same!
- PATCH: Due to an overlap of common variable names when integrating with AMTM, some mitigations had to be made and will change the username and password variable names within the backupmon.cfg upon start. This will run 1x only once variable names have been corrected.
- PATCH: Added AMTM Email testing as a menu item off the main setup menu. This allows you to give the AMTM email capabilities a quick test, providing verbose on-screen feedback during the process.

Download link (or update directly within AMTM/BACKUPMON itself):

curl --retry 3 "https://raw.githubusercontent.com/ViktorJp/BACKUPMON/master/backupmon-1.5.3.sh" -o "/jffs/scripts/backupmon.sh" && chmod 755 "/jffs/scripts/backupmon.sh"

Significant Screenshots!

Items #14 in the config menu lets you enable AMTM email notifications on Success or Failure

A more detailed look at what you get with the AMTM Email Notifications option.

During a regular backup, email notifications will get sent immediately if something fails, or at the very end of the job on success

And just added, from the setup menu, you can now easily test your AMTM email notifications, and will shoot off a test email to your account, giving you verbose feedback if something doesn't work right.

 

[Quoc Huynh]

Thank you for your new version @Viktor Jaep. Updating and sending emails is flawlessly for me However, I received the error "bad decrypt" as in the screenshots below despite of the smooth email notification. By chance do you know which may cause that issue? Thank you.

 

[Viktor Jaep]

Thank you for your new version @Viktor Jaep. Updating and sending emails is flawlessly for me However, I received the error "bad decrypt" as in the screenshots below despite of the smooth email notification. By chance do you know which may cause that issue? Thank you.

Glad it seems to be working even with that error! That error is being returned from either the AMTM email settings or from @Martinski's email library, and wondering perhaps if it's some kind of return error code that's coming back from curl/openssl?

Does it happen every time? If you try doing a test email from AMTM itself, using the verbose mode (AMTM->em->11->2), do you see any mention of that error there?

 

[Quoc Huynh]

Glad it seems to be working even with that error! That error is being returned from either the AMTM email settings or from @Martinski's email library, and wondering perhaps if it's some kind of return error code that's coming back from curl/openssl?

Does it happen every time? If you try doing a test email from AMTM itself, using the verbose mode, do you see any mention of that error there?

I am happy that it works, too. Yes, it happens every time I choose to send a test mail. Moreover, you are actually correct. I also receive the "bad decrypt" error with a test email in AMTM.

bad decrypt
4147056656:error:0606506D:lib(6):func(101):reason(109):NA:0:
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 142.251.175.108:465...
* Connected to smtp.gmail.com (142.251.175.108) port 465
} [5 bytes data]
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
* TLSv1.3 (IN), TLS handshake, Server hello (2):
{ [122 bytes data]
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
{ [6 bytes data]
* TLSv1.3 (IN), TLS handshake, Certificate (11):
{ [4003 bytes data]
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
{ [79 bytes data]
* TLSv1.3 (IN), TLS handshake, Finished (20):
{ [52 bytes data]
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
} [1 bytes data]
* TLSv1.3 (OUT), TLS handshake, Finished (20):
} [52 bytes data]
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* Server certificate:
*  subject: CN=smtp.gmail.com
*  start date: Jan 29 08:19:39 2024 GMT
*  expire date: Apr 22 08:19:38 2024 GMT
*  issuer: C=US; O=Google Trust Services LLC; CN=GTS CA 1C3
*  SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
{ [5 bytes data]
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
{ [261 bytes data]
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
{ [261 bytes data]
* old SSL session ID is stale, removing
{ [5 bytes data]
< 220 smtp.gmail.com ESMTP g68-20020a636b47000000b005dcbb699abfsm2506593pgc.34 - gsmtp
} [5 bytes data]
> EHLO amtm-mail-body
  0     0    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0{ [5 bytes data]
< 250-smtp.gmail.com at your service, [125.168.159.43]
< 250-SIZE 35882577
< 250-8BITMIME
< 250-AUTH LOGIN PLAIN XOAUTH2 PLAIN-CLIENTTOKEN OAUTHBEARER XOAUTH
< 250-ENHANCEDSTATUSCODES
< 250-PIPELINING
< 250-CHUNKING
< 250 SMTPUTF8
} [5 bytes data]
> AUTH PLAIN
{ [5 bytes data]
< 334
} [5 bytes data]
> AHF1b2NkdW5nM0BnbWFpbC5jb20AYWxjZW9zb2pzdGtlbmh0bw==
{ [5 bytes data]
< 235 2.7.0 Accepted
  0     0    0     0    0     0      0      0 --:--:--  0:00:02 --:--:--     0} [5 bytes data]
> MAIL FROM:<quocdung3@gmail.com> SIZE=265
{ [5 bytes data]
< 250 2.1.0 OK g68-20020a636b47000000b005dcbb699abfsm2506593pgc.34 - gsmtp
} [5 bytes data]
> RCPT TO:<quocdung3@gmail.com>
{ [5 bytes data]
< 250 2.1.5 OK g68-20020a636b47000000b005dcbb699abfsm2506593pgc.34 - gsmtp
} [5 bytes data]
> DATA
{ [5 bytes data]
< 354  Go ahead g68-20020a636b47000000b005dcbb699abfsm2506593pgc.34 - gsmtp
  0   265    0     0    0     0      0      0 --:--:--  0:00:03 --:--:--     0} [5 bytes data]
104   265    0     0  104   277      0     69  0:00:03  0:00:03 --:--:--    70< 250 2.0.0 OK  1708087493 g68-20020a636b47000000b005dcbb699abfsm2506593pgc.34 - gsmtp
104   265    0     0  104   277      0     69  0:00:03  0:00:03 --:--:--    70
* Connection #0 to host smtp.gmail.com left intact

Here is my email configuration in AMTM:

Common SMTP Server settings
 Provider    Server                 Port Protocol
 ------------------------------------------------
 Gmail       smtp.gmail.com         465  smtps
 mail.com    smtp.mail.com          587  smtp
 Yahoo!      smtp.mail.yahoo.com    465  smtps
 outlook.com smtp-mail.outlook.com  587  smtp

  1. Edit From address:   quocxxx@gmail.com
  2. Edit To name:        Quoc Huynh
  3. Edit To address:     quocxxx@gmail.com
  4. Edit Router name:    xxx
  5. Edit User name:      quocxxx@gmail.com
  6. Edit Password:       select Edit to view
  7. Edit SMTP Server:    smtp.gmail.com
  8. Edit Server port:    465
  9. Edit Protocol:       smtps
 10. Edit SSL flag:       --insecure
 11. Send testmail to confirm settings