[Beta 3] Vnstat on Merlin - UI, CLI and email - data use monitoring - now with data limit monitoring

[maghuro]

V0.9.5
Steps to reproduce:
- set TB unit for bandwidth allowance
- set 1 (TB) bandwidth allowance
- check 'b' bandwidth and you'll get: "You have used 2.00% (0.02TB) of your 1.00TB monthly allowance"
- now change GB unit for bandwidth allowance.
- don't set a new bandwidth (it should be converted to 1000/1024 GB), but it's not.
- Check 'b' bandwidth and you'll get: "You have used 1623.00% (16.23GB) of your 1.00GB monthly allowance"

Note: The same applied when you set 1024 Gb and then change to TB (data allowance goes to 1024TB

 

[Jack Yaz]

V0.9.5
Steps to reproduce:
- set TB unit for bandwidth allowance
- set 1 (TB) bandwidth allowance
- check 'b' bandwidth and you'll get: "You have used 2.00% (0.02TB) of your 1.00TB monthly allowance"
- now change GB unit for bandwidth allowance.
- don't set a new bandwidth (it should be converted to 1000/1024 GB), but it's not.
- Check 'b' bandwidth and you'll get: "You have used 1623.00% (16.23GB) of your 1.00GB monthly allowance"

Note: The same applied when you set 1024 Gb and then change to TB (data allowance goes to 1024TB

I told you the conversion was needed @dev_null

 

[aru]

See posts 74, 76 & 84. Note, these didn't work for me and I had to go nuclear and uninstall & reinstall.

Okay, I will try to uninstall and reinstall again, thanks!

 

[dev_null]

found a lot of "TB" statistics? how to correct it?

I can see your router and firmware version in the screenshots. I have a few more questions.

Can you describe your connection type and speed, please. Also, if hardware acceleration was enabled (it's already been reported that with HA enabled, the vnstat statics are inaccurate).

What sort of internet work were you doing during this time period? Were you installing other add-ons (NTP merlin, or others)? Were you rebooting the router?

For diagnostic purposes, can you run vnstat --exportdb > vnstat-export-db-aru.txt (from the CLI; this file will end up in your /root/ directory and will be deleted at reboot) and DM me (don't post them here) the contents (paste them into a code box).

 

[dev_null]

All, I'm pleased to announce version 0.9.5 was just merged. In the dn-vnstat menu, choosing u should update (some may require a force update: uf). A force update is no different than a regular update for this version.

I've been fortunate to continue my collaboration with @Jack Yaz who deserves a huge round of applause (and more)!

What's new?​

• If this were in release, it would be an entirely new version!
• This is considered Beta 3; for all intents and for all intents and purposes it is a RC
• Includes many, many under the hood optimizations to improve reliability, accuracy and functionality
• For those with data caps, we've added user-specified data limits: check vnstat-reported data use against a user-set limit
  • Looking at you Comcast, Cox, others
• This data limit monitoring includes:
  • Optional email notification when you reach 75%, 90% and 100% of data limit
  • Data use warning message on UI page (75% and higher)
  • Data limit calculations - in both the UI and CLI menu (% data limit used)
    • CLI menu- and UI-updatable; selectable GB or TB limits
• There is an expanded number of UI settings options (the CLI menu has a few more)
• There is now a self-contained emailer, which still leverages Diversion credentials, but no longer calls a separate script
• During the initial install, the script automatically backs-up existing vnstat databases (if found)
• This version deprecate the "CLI daily total" graphic in favor of textual view
  • Obviates the need for imagmagick
  • Still retains the daily summary emailed at 23:59 local (if enabled)

The changelog is here: https://github.com/de-vnull/vnstat-on-merlin/blob/main/changelog.md

Known issues:​

• The day of month selection: changing this from the 1st of the month may not result in a change in calculations of data used. This is likely an issue with the current Entware version of vnstat (1.18). We're trying to determine if there is a work-around
• Changing data use units: the calculations if you switch from GB to TB and vice versa are not fully functional, particularly from the UI
  • Recommendation: change the UNIT, then save, then go back and change the number, save, and check to make sure the expected data limit is set. Or use the menu version, which should be calculating correctly.
• CLI vnstat flags do not work properly in the current Entware version of vnstat (1.18). For example, the --reset flag does not reset. A full removal of dn-vnstat, including removing the database files, is required, followed by a full install (via amtm).
• If hardware acceleration is enabled, the numbers reported by vnstat are likely inaccurate. This script *does not* disable HwA because it is up to the user to determine if data reporting or HwA is the more critical activity.

As usual, if you have any findings to report, please include your router model, firmware version, other add-ons you may be using, and description of the event, including actions or observations proximal to the event.​

Please note: the information that is reported by vnstat (and therefore Vnstat on Merlin) regarding data use should be considered a guide - an approximation of actual use. As noted above, and on the github readme.md, vnstat reported totals may or may not match that reported by your provider, or may start and stop on a different date, or be affected by different variables such as hardware acceleration or scheduled reboots. You must conduct proper due diligence to determine if the usage reported by vnstat aligns with your provider. You assume all responsibility for the use of the information provided by vnstat and Vnstat on Merlin.

 

[maghuro]

All, I'm pleased to announce version 0.9.5 was just merged. In the dn-vnstat menu, choosing u should update (some may require a force update: uf). A force update is no different than a regular update for this version.

I've been fortunate to continue my collaboration with @Jack Yaz who deserves a huge round of applause (and more)!

What's new?​

• If this were in release, it would be an entirely new version!
• This is considered Beta 3; for all intents and for all intents and purposes it is a RC
• Includes many, many under the hood optimizations to improve reliability, accuracy and functionality
• For those with data caps, we've added user-specified data limits: check vnstat-reported data use against a user-set limit
  • Looking at you Comcast, Cox, others
• This data limit monitoring includes:
  • Optional email notification when you reach 75%, 90% and 100% of data limit
  • Data use warning message on UI page (75% and higher)
  • Data limit calculations - in both the UI and CLI menu (% data limit used)
    • CLI menu- and UI-updatable; selectable GB or TB limits
• There is an expanded number of UI settings options (the CLI menu has a few more)
• There is now a self-contained emailer, which still leverages Diversion credentials, but no longer calls a separate script
• During the initial install, the script automatically backs-up existing vnstat databases (if found)
• This version deprecate the "CLI daily total" graphic in favor of textual view
  • Obviates the need for imagmagick
  • Still retains the daily summary emailed at 23:59 local (if enabled)

The changelog is here: https://github.com/de-vnull/vnstat-on-merlin/blob/main/changelog.md

Known issues:​

• The day of month selection: changing this from the 1st of the month may not result in a change in calculations of data used. This is likely an issue with the current Entware version of vnstat (1.18). We're trying to determine if there is a work-around
• Changing data use units: the calculations if you switch from GB to TB and vice versa are not fully functional, particularly from the UI
  • Recommendation: change the UNIT, then save, then go back and change the number, save, and check to make sure the expected data limit is set. Or use the menu version, which should be calculating correctly.
• CLI vnstat flags do not work properly in the current Entware version of vnstat (1.18). For example, the --reset flag does not reset. A full removal of dn-vnstat, including removing the database files, is required, followed by a full install (via amtm).
• If hardware acceleration is enabled, the numbers reported by vnstat are likely inaccurate. This script *does not* disable HwA because it is up to the user to determine if data reporting or HwA is the more critical activity.

As usual, if you have any findings to report, please include your router model, firmware version, other add-ons you may be using, and description of the event, including actions or observations proximal to the event.​

Good job! Thanks both of you!
Just a question, can I uninstall diversion right now? Because I only have diversion installed so I can use the email option on vnstat

 

[dev_null]

Good job! Thanks both of you!
Just a question, can I uninstall diversion right now? Because I only have diversion installed so I can use the email option on vnstat

The script still uses the Diversion email attributes.

I haven't tried it, but in theory if you save the /opt/share/diversion/.conf (hidden) folder, remove Diversion and then put this folder back, it should work. I believe the only two files used are email.conf and emailpw.enc.

 

[JohnD5000]

The script still uses the Diversion email attributes.

I haven't tried it, but in theory if you save the /opt/share/diversion/.conf (hidden) folder, remove Diversion and then put this folder back, it should work. I believe the only two files used are email.conf and emailpw.enc.

Are there future plans to create a Diversion-free email capability? I had Diversion at one time, but found it blocked some stuff I did not want blocked. I prefer to do that at the browser.

 

[JGrana]

Can Imagemagick be removed? I noticed recently my router is suffering opkg bloat from too much "experimenting". I am attempting to keep things cleaned up ;-)

 

[Jack Yaz]

Can Imagemagick be removed? I noticed recently my router is suffering opkg bloat from too much "experimenting". I am attempting to keep things cleaned up ;-)

with 0.9.5, yes. we didn't want to explicitly removed it in case anyone had installed it themselves for other uses

 

[dev_null]

Are there future plans to create a Diversion-free email capability? I had Diversion at one time, but found it blocked some stuff I did not want blocked. I prefer to do that at the browser.

As noted in the github readme.md, Diversion is only needed for the encrypted email credentials. It does not need to be running, just installed, email configured, and then it can be disabled.

Otherwise you have 3 potential approaches:

• Follow the instructions above to save the needed files, uninstall and then replace the folder and files: https://www.snbforums.com/goto/post?id=673732. This is untested.
• Remove Diversion and lose the email capabilities (be sure to disable the email options in Vnstat on Merlin before uninstalling to prevent any conflicts/error messages) - this loss of functionality includes the daily usage reports AND notifications related to data limits.
• Set up the daily update email manually using a cronjob, by using the send-vnstat approach documented on the github readme.md. This approach stores email credentials in a plain text file, and will NOT work for data limit notifications, only daily emails, and would need to be configured to select the file /tmp/vnstat.txt. While send-vnstat was tested on earlier versions of Vnstat on Merlin, it has not been tested on the current version (it is considered deprecated).

 

[visortgw]

As noted in the github readme.md, Diversion is only needed for the encrypted email credentials. It does not need to be running, just installed, email configured, and then it can be disabled.

Otherwise you have 3 potential approaches:

• Follow the instructions above to save the needed files, uninstall and then replace the folder and files: https://www.snbforums.com/goto/post?id=673732. This is untested.
• Remove Diversion and lose the email capabilities (be sure to disable the email options in Vnstat on Merlin before uninstalling to prevent any conflicts/error messages) - this loss of functionality includes the daily usage reports AND notifications related to data limits.
• Set up the daily update email manually using a cronjob, by using the send-vnstat approach documented on the github readme.md. This approach stores email credentials in a plain text file, and will NOT work for data limit notifications, only daily emails, and would need to be configured to select the file /tmp/vnstat.txt. While send-vnstat was tested on earlier versions of Vnstat on Merlin, it has not been tested on the current version (it is considered deprecated).

Would it be possible to leave Diversion installed with e-mail configuration in place, but disable it via the script? Would e-mail configuration still work? (I use Diversion -- this is me thinking out loud.)

 

[dev_null]

Would it be possible to leave Diversion installed with e-mail configuration in place, but disable it via the script? Would e-mail configuration still work? (I use Diversion -- this is me thinking out loud.)

To disable Diversion without uninstalling: amtm > 1 (enter Diversion) > d (Diversion) > 1 (disable Diversion) > e (exit).

We leverage Diversion's proven storage and encryption of your email credentials. It is up to the user to decide whether the benefits accorded (secure email communication from Vnstat on Merlin) outweigh the perceived downside (configuration via Diversion).

 

[lbtboy]

Router ASUS RT-AX88U Firmware Version:386.1_2 Merlin Vnstat v0.9.5

Error

admin@RT-AX88U-0BE5:/tmp/home/root# /jffs/scripts/dn-vnstat summary
vnstati updating stats for UI

vnstati: error while loading shared libraries: libjpeg.so.62: cannot open shared object file: No such file or directory
vnstati: error while loading shared libraries: libjpeg.so.62: cannot open shared object file: No such file or directory
vnstati: error while loading shared libraries: libjpeg.so.62: cannot open shared object file: No such file or directory
vnstati: error while loading shared libraries: libjpeg.so.62: cannot open shared object file: No such file or directory
vnstati: error while loading shared libraries: libjpeg.so.62: cannot open shared object file: No such file or directory
vnstati: error while loading shared libraries: libjpeg.so.62: cannot open shared object file: No such file or directory
vnstats as of:
Wed Mar 17 18:43:37 GMT 2021
eth0 / monthly

month rx | tx | total
-------------------------+-------------+------------
21-03 861.11 MB | 1.26 GB | 2.10 GB
-------------------------+-------------+------------
estimated 1.57 GB | 2.35 GB | 3.93 GB

eth0 / weekly

rx | tx | total
---------------------------+-------------+------------
last 7 days 861.11 MB | 1.26 GB | 2.10 GB
current week 861.11 MB | 1.26 GB | 2.10 GB
---------------------------+-------------+------------
estimated 2.14 GB | 3.20 GB | 5.34 GB

eth0 / daily

day rx | tx | total
-------------------------+-------------+------------
2021-03-17 861.11 MB | 1.26 GB | 2.10 GB
-------------------------+-------------+------------
estimated 1.08 GB | 1.61 GB | 2.69 GB

vnstat_totals summary generated

Attempting to send summary statistic email

/jffs/scripts/dn-vnstat: line 16: can't open /jffs/addons/dn-vnstat.d/images/vnstat_s.png: no such file
/jffs/scripts/dn-vnstat: line 16: can't open /jffs/addons/dn-vnstat.d/images/vnstat_h.png: no such file
/jffs/scripts/dn-vnstat: line 16: can't open /jffs/addons/dn-vnstat.d/images/vnstat_d.png: no such file
/jffs/scripts/dn-vnstat: line 16: can't open /jffs/addons/dn-vnstat.d/images/vnstat_t.png: no such file
/jffs/scripts/dn-vnstat: line 16: can't open /jffs/addons/dn-vnstat.d/images/vnstat_m.png: no such file
/jffs/scripts/dn-vnstat: line 16: can't open /jffs/addons/dn-vnstat.d/images/vnstat_hs.png: no such file
Email sent successfully
____________________________________________________________________________________________________

admin@RT-AX88U-0BE5:/tmp/home/root# /jffs/scripts/dn-vnstat generate
vnstati updating stats for UI

vnstati: error while loading shared libraries: libjpeg.so.62: cannot open shared object file: No such file or directory
vnstati: error while loading shared libraries: libjpeg.so.62: cannot open shared object file: No such file or directory
vnstati: error while loading shared libraries: libjpeg.so.62: cannot open shared object file: No such file or directory
vnstati: error while loading shared libraries: libjpeg.so.62: cannot open shared object file: No such file or directory
vnstati: error while loading shared libraries: libjpeg.so.62: cannot open shared object file: No such file or directory
vnstati: error while loading shared libraries: libjpeg.so.62: cannot open shared object file: No such file or directory
vnstats as of:
Wed Mar 17 18:43:41 GMT 2021
eth0 / monthly

month rx | tx | total
-------------------------+-------------+------------
21-03 861.15 MB | 1.26 GB | 2.10 GB
-------------------------+-------------+------------
estimated 1.57 GB | 2.35 GB | 3.93 GB

eth0 / weekly

rx | tx | total
---------------------------+-------------+------------
last 7 days 861.15 MB | 1.26 GB | 2.10 GB
current week 861.15 MB | 1.26 GB | 2.10 GB
---------------------------+-------------+------------
estimated 2.14 GB | 3.20 GB | 5.34 GB

eth0 / daily

day rx | tx | total
-------------------------+-------------+------------
2021-03-17 861.15 MB | 1.26 GB | 2.10 GB
-------------------------+-------------+------------
estimated 1.08 GB | 1.61 GB | 2.69 GB

vnstat_totals summary generated

You have used 2.10GB of data this month


Decision

opkg install libjpeg-turbo

 

[Jack Yaz]

Router ASUS RT-AX88U Firmware Version:386.1_2 Merlin Vnstat v0.9.5

Error

admin@RT-AX88U-0BE5:/tmp/home/root# /jffs/scripts/dn-vnstat summary
vnstati updating stats for UI

vnstati: error while loading shared libraries: libjpeg.so.62: cannot open shared object file: No such file or directory
vnstati: error while loading shared libraries: libjpeg.so.62: cannot open shared object file: No such file or directory
vnstati: error while loading shared libraries: libjpeg.so.62: cannot open shared object file: No such file or directory
vnstati: error while loading shared libraries: libjpeg.so.62: cannot open shared object file: No such file or directory
vnstati: error while loading shared libraries: libjpeg.so.62: cannot open shared object file: No such file or directory
vnstati: error while loading shared libraries: libjpeg.so.62: cannot open shared object file: No such file or directory
vnstats as of:
Wed Mar 17 18:43:37 GMT 2021
eth0 / monthly

month rx | tx | total
-------------------------+-------------+------------
21-03 861.11 MB | 1.26 GB | 2.10 GB
-------------------------+-------------+------------
estimated 1.57 GB | 2.35 GB | 3.93 GB

eth0 / weekly

rx | tx | total
---------------------------+-------------+------------
last 7 days 861.11 MB | 1.26 GB | 2.10 GB
current week 861.11 MB | 1.26 GB | 2.10 GB
---------------------------+-------------+------------
estimated 2.14 GB | 3.20 GB | 5.34 GB

eth0 / daily

day rx | tx | total
-------------------------+-------------+------------
2021-03-17 861.11 MB | 1.26 GB | 2.10 GB
-------------------------+-------------+------------
estimated 1.08 GB | 1.61 GB | 2.69 GB

vnstat_totals summary generated

Attempting to send summary statistic email

/jffs/scripts/dn-vnstat: line 16: can't open /jffs/addons/dn-vnstat.d/images/vnstat_s.png: no such file
/jffs/scripts/dn-vnstat: line 16: can't open /jffs/addons/dn-vnstat.d/images/vnstat_h.png: no such file
/jffs/scripts/dn-vnstat: line 16: can't open /jffs/addons/dn-vnstat.d/images/vnstat_d.png: no such file
/jffs/scripts/dn-vnstat: line 16: can't open /jffs/addons/dn-vnstat.d/images/vnstat_t.png: no such file
/jffs/scripts/dn-vnstat: line 16: can't open /jffs/addons/dn-vnstat.d/images/vnstat_m.png: no such file
/jffs/scripts/dn-vnstat: line 16: can't open /jffs/addons/dn-vnstat.d/images/vnstat_hs.png: no such file
Email sent successfully
____________________________________________________________________________________________________

admin@RT-AX88U-0BE5:/tmp/home/root# /jffs/scripts/dn-vnstat generate
vnstati updating stats for UI

vnstati: error while loading shared libraries: libjpeg.so.62: cannot open shared object file: No such file or directory
vnstati: error while loading shared libraries: libjpeg.so.62: cannot open shared object file: No such file or directory
vnstati: error while loading shared libraries: libjpeg.so.62: cannot open shared object file: No such file or directory
vnstati: error while loading shared libraries: libjpeg.so.62: cannot open shared object file: No such file or directory
vnstati: error while loading shared libraries: libjpeg.so.62: cannot open shared object file: No such file or directory
vnstati: error while loading shared libraries: libjpeg.so.62: cannot open shared object file: No such file or directory
vnstats as of:
Wed Mar 17 18:43:41 GMT 2021
eth0 / monthly

month rx | tx | total
-------------------------+-------------+------------
21-03 861.15 MB | 1.26 GB | 2.10 GB
-------------------------+-------------+------------
estimated 1.57 GB | 2.35 GB | 3.93 GB

eth0 / weekly

rx | tx | total
---------------------------+-------------+------------
last 7 days 861.15 MB | 1.26 GB | 2.10 GB
current week 861.15 MB | 1.26 GB | 2.10 GB
---------------------------+-------------+------------
estimated 2.14 GB | 3.20 GB | 5.34 GB

eth0 / daily

day rx | tx | total
-------------------------+-------------+------------
2021-03-17 861.15 MB | 1.26 GB | 2.10 GB
-------------------------+-------------+------------
estimated 1.08 GB | 1.61 GB | 2.69 GB

vnstat_totals summary generated

You have used 2.10GB of data this month


Decision

opkg install libjpeg-turbo

it's weird that vnstati didn't install it as a dependency. probably due to the ancient vnstat we have in Entware, please log this as an issue on the Entware repo and we'll update dn-vnstat to explicitly install that package

 

[lbtboy]

it's weird that vnstati didn't install it as a dependency. probably due to the ancient vnstat we have in Entware, please log this as an issue on the Entware repo and we'll update dn-vnstat to explicitly install that package

Sorry I would register it, but I do not know how to register it in the Entware repository. Maybe you can do it yourself.

 

[visortgw]

it's weird that vnstati didn't install it as a dependency. probably due to the ancient vnstat we have in Entware, please log this as an issue on the Entware repo and we'll update dn-vnstat to explicitly install that package

I do not see a dependency error. Could it be that the OP had installed previously installed vnstat before there was a dependency?

 

[dev_null]

Router ASUS RT-AX88U Firmware Version:386.1_2 Merlin Vnstat v0.9.5

Error

admin@RT-AX88U-0BE5:/tmp/home/root# /jffs/scripts/dn-vnstat summary
vnstati updating stats for UI

vnstati: error while loading shared libraries: libjpeg.so.62: cannot open shared object file: No such file or directory
vnstati: error while loading shared libraries: libjpeg.so.62: cannot open shared object file: No such file or directory
vnstati: error while loading shared libraries: libjpeg.so.62: cannot open shared object file: No such file or directory
vnstati: error while loading shared libraries: libjpeg.so.62: cannot open shared object file: No such file or directory
vnstati: error while loading shared libraries: libjpeg.so.62: cannot open shared object file: No such file or directory
vnstati: error while loading shared libraries: libjpeg.so.62: cannot open shared object file: No such file or directory
vnstats as of:


Decision

opkg install libjpeg-turbo

Did you respond to my earlier questions when you posted this the first time: https://www.snbforums.com/goto/post?id=670805 ? I see your router model and firmware, but need to know what other add-ons, what the connection type is, when was the last reset, etc.

So far this is the only report we've seen on this issue - I suspect it's something particular to your router/setup. Please provide the requested information, either in this thread or to me directly.

 

[lbtboy]

Did you respond to my earlier questions when you posted this the first time: https://www.snbforums.com/goto/post?id=670805 ? I see your router model and firmware, but need to know what other add-ons, what the connection type is, when was the last reset, etc.

So far this is the only report we've seen on this issue - I suspect it's something particular to your router/setup. Please provide the requested information, either in this thread or to me directly.

The problem was on two models of routers RT-AX88U and RT-AC86U Firmware Version:386.1_2. There were no resets for a year, it is configured and works. The first installation of vnstat was from version 0.9.4 via amtm. After updating to vnstat 0.9.5, the problem was not resolved. Checked the installation of the libjpeg package was installed "Package libjpeg (9c-2) installed in root is up to date.", decided to additionally install the libjpeg-turbo package, installed and immediately the problem was fixed. I do not understand why you need a connection, but if you need it, it is 100 mbit/sec from the provider via WAN FTTb. About the add-ons, too, I do not quite understand, if you are about what I thought, then they are in the attached screenshots from both RT-AX88U and RT-AC86U routers.

 

[JohnD5000]

To disable Diversion without uninstalling: amtm > 1 (enter Diversion) > d (Diversion) > 1 (disable Diversion) > e (exit).

We leverage Diversion's proven storage and encryption of your email credentials. It is up to the user to decide whether the benefits accorded (secure email communication from Vnstat on Merlin) outweigh the perceived downside (configuration via Diversion).

So, if Diversion is installed and then disabled, can vnstat email then be setup? Or, do need to install diversion, setup vnstat email, and then disable diversion?