[Beta 3] Vnstat on Merlin - UI, CLI and email - data use monitoring - now with data limit monitoring

[dev_null]

vnStat-on-Merlin - beta 3​

I'm pleased to announce the availability of beta 3 (RC1) of vnStat-on-Merlin. Details can be found in this post: https://www.snbforums.com/threads/b...-with-full-install-and-menu.70727/post-673711

This newest version includes a huge list of changes, so be sure to read the post.

---

This script the successor to the manual/alpha/beta version previously posted.

This version includes not only the UI, CLI and daily email summaries from the previos verion, but now features a fully scripted install, menu, updated UI and version checking, courtesy of a collaboration with @Jack Yaz.

The main vnStat on Merlin github page is here: https://github.com/de-vnull/vnstat-on-merlin

Background​

• This is an implementation of vnstat/vnstati for use on AsusWRT-Merlin routers. This effort was started to enable accurate measurement of data use in a local database, to supplement the internal monitoring tool, Traffic Analyzer > Traffic Monitor, which will peridically record false “17GB” usage bursts. This only occurs on some routers on some firmware (e.g., RT-AC68U and RT-AC66U_B1 on 386.1).
• This became a particular concern when Xfinity began implementing 1.2TB caps nationwide in January 2021 (note: postponed in the Northeast until 2022).
• This was created for personal use, but is being published for the benefit to the community of users. Improvement and enhancement suggestions are welcomed (I already have a few in mind).
  • The totals are consistent with the firmware "Traffic Monitor/Traffic Analyzer". The totals reported by vnStat are slightly higher than those currently being reported by Comcast (about 10-15%).
  • Feedback and comments are welcomed. PM to @dev_null @ snbforums
  • Any errors or omissions will be corrected upon notice, but the user assumes all risk.

Installation notes​

• Prior versions (alpha, beta 1, and manual) required a series of steps to be taken for full install. This beta 2 implementation is nearly completely automated, with several additional options available through the dn-vnstat command menu, post install.
• A manual reboot may be required to purge duplicate vnStat-UI pages.

Minimum requirements​

• AsusWRT Merlin version 384.19 or later. Tested on 386.1 beta 1-5 and 386.1 release versions.
  • Earlier versions (384.x only) may likewise function; kindly report any further experiences (include router model and firmware version).
  • Initial testing on John's fork appears to demonstrate expected functionality, except that the "CLI report" image isn't working. Kindly report any further experiences (include router model and firmware version).
  • Hardware tested RT-AC66U_B1, RT-AC68U, RT-AC86U, RT-AX86U and RT-N66U.
• Diversion and it's corresponding install of Entware. Diversion does not need to be running, as long as Entware is installed.
• Properly setup email (Diversion "communications" option) to use the encrypted username/password to email vnStat reports.
  • Please run an Entware update to ensure the most current repository lists are available.
  • Please test Diversion email (amtm > 1 - Diversion > c - Communication > 5 - Edit email settings, send testmail > follow steps to set up and test) before enabling the vnStat on Merlin usage email.
• Database configuration
  • Unlike in the alpha/beta1/manual installations, there is no separate database configuration required for the UI-installed version. This is now part of the automation. Existing database files will not be deleted.
  • Note: if you have a custom location for your database files, you will need to either update vnstat.conf with those locations, re-initialize the database in the standard location (losing history), or terminate vnstatd (killall vnstatd) and copy your database files to the standard location. See the github repository for detailed steps on backing up and restoring your database files.

Install script - UI version - beta 2​

• From the CLI, issue the following command (triple click to select all):

/usr/sbin/curl --retry 3 "https://raw.githubusercontent.com/de-vnull/vnstat-on-merlin/main/dn-vnstat.sh" -o "/jffs/scripts/dn-vnstat" && chmod 0755 /jffs/scripts/dn-vnstat && /jffs/scripts/dn-vnstat install

• Follow the prompts; at the end of the install the dn-vnstat menu will be presented. Press 'e' to exit back to the command shell.

The AddOns tab showing the UI (with the "rate" column removed via vnstat.conf setting), and the "daily report" collapsed; Asus-theme:



The dn-vnstat menu (CLI via SSH):



A sample of the email message output - plain text; option for (beta) html with graphics can be found in the dn-vnstat menu:

​

Updating from manual/alpha/beta 1​

• This beta 2 version is re-written from the ground up, and therefore scripts from previous installations (manual or otherwise) need to be removed.
• The beta 2 install script will detect any previously installed scripts and will inform you that these will be removed. Database files will be intact on the device.
• If you don't want to migrate to the new version, you can abort the install.
• A manual reboot may be required to purge duplicate vnStat-UI pages after upgrading.

CLI version​

• For non-UI users, the CLI (non-UI) install instructions can be found on the vnStat-on-Merlin github landing page.
• Instructions on how to install email notifications using the non-UI version are in the readme.md on the landing page.

Feedback welcomed! Please list your router model and firmware version in your posts.​

Please keep all vnStat-on-Merlin issues contained to this thread. ​

If it expires, another will be opened to replace it.

Please note: the information that is reported by vnstat (and therefore vnStat-on-Merlin) regarding data use should be considered a guide - an approximation of actual use. As noted above, and on the github readme.md, vnStat-reported totals may or may not match that reported by your provider, or may start and stop on a different date, or be affected by different variables such as hardware acceleration or scheduled reboots. You must conduct proper due diligence to determine if the usage reported by vnStat aligns with your provider. You assume all responsibility for the use of the information provided by vnStat and vnStat-on-Merlin.

 

[dev_null]

Notes, versions and errata.

Versions

• See changelog here: https://github.com/de-vnull/vnstat-on-merlin/blob/main/changelog.md

 

[dev_null]

Reserved for FAQ, wishlist, and mea-culpas.

FAQ:

• Please review the github readme.md for additional detail.

Wishlist/to-do:

• Monitor data use and send an email warning at 80% (notice!), 90% (warning!), 100% (reached!) and 105% (over!). One warning of each type. - DONE (0.9.5)
• Replace the "daily email report" graphic in the UI with text field (similar to Diversion report). - DONE (0.9.5)
• Add "% data used" to daily report - Done 0.9.6

 

[thelonelycoder]

Now we’re getting there. Good job @dev_null and @Jack Yaz, it’s going to be real good after a rocky start.

 

[elorimer]

Nice.

In the OP, I think the xfinity cap is 1.2TB, not 1.2GB? Whew--I'm around 350GB.

 

[JohnD5000]

Now we’re getting there. Good job @dev_null and @Jack Yaz, it’s going to be real good after a rocky start.

Hmmm AMTM?

 

[JohnD5000]

Looks like old version not removed. How do I remove it manually?

 

[dev_null]

Looks like old version not removed. How do I remove it manually?

View attachment 31353

Did this persist after a reboot? If you haven't, do a manual reboot when convenient.

 

[JohnD5000]

Did this persist after a reboot? If you haven't, do a manual reboot when convenient.

I can't do a reboot now, I'll let you know when network is free and I can reboot it.

 

[JohnD5000]

Did this persist after a reboot? If you haven't, do a manual reboot when convenient.

I was able to do a manual boot and it fixed it. You might want to add in your "Updating from manual/alpha/beta 1" section that a reboot may be needed.

​

 

[Jack Yaz]

I was able to do a manual boot and it fixed it. You might want to add in your "Updating from manual/alpha/beta 1" section that a reboot may be needed.

​

It's already there?

Installation notes
Prior versions (alpha, beta 1, and manual) required a series of steps to be taken for full install. This beta 2 implementation is nearly completely automated, with several additional options available through the dn-vnstat command menu, post install.
A manual reboot may be required to purge duplicate vnstat-UI page s.

 

[New2This]

Thanks for this, running it on a AX88U

 

[HardCat]

Just installed on RT-AX88u, all went well during the installation. Thanks...

 

[JohnD5000]

It's already there?

I meant in the section on updating from a previous version, not installing from scratch.

 

[JR Godwin]

Great work! Congrats and hooray for collaboration.

 

[dev_null]

Tip: if you update to 0.9.4 or later and you want to have a look more consistent with the Asus theme, head back into the menu dn-vnstat and edit the bottom of the vnstat.conf:

EDIT: use the colors but leave the transparency setting at '0' if you're using HTML email option (otherwise it might be white text on white background).

# transparent background (1 = enabled, 0 = disabled)
TransparentBg 0

# image colors
CBackground     "475A5F"
CEdge           "475A5F"
CHeader         "475A5F"
CHeaderTitle    "FFFFFF"
CHeaderDate     "EAE303"
CText           "FFFFFF"
CLine           "B0B0B0"
CLineL          "EAE303"
CRx             "C5C5CE"
CTx             "0EC009"
CRxD            "-"
CTxD            "-"

And if you're wondering, in my setup, I've disabled the "rate" settings in vnstat.conf too, for a cleaner look.

# output style
# 0 = minimal & narrow, 1 = bar column visible
# 2 = same as 1 except rate in summary and weekly
# 3 = rate column visible
OutputStyle 0

# show hours with rate (1 = enabled, 0 = disabled)
HourlyRate 0

# show rate in summary (1 = enabled, 0 = disabled)
SummaryRate 0

We'll see what we can do about making these the defaults .

Edit 2: new installs will get this theme-based view without the rate columns. Current users need to make the changes above. I'll add some detail to the github page if you want to revert.

 

[JR Godwin]

In the default config, the bandwidth detection, max bandwidth, is disabled....What does enabling the detection do in practical terms, and if interface specific limits are given, is this enforcing that limit or used for labeling?

 

[dev_null]

In the default config, the bandwidth detection, max bandwidth, is disabled....What does enabling the detection do in practical terms, and if interface specific limits are given, is this enforcing that limit or used for labeling?

In my experience, vnstat was not good about detecting BW and kept putting errors in the syslog. Disabling BW detection has no discernible impact, and it's not clear what vnstat is doing with that anyway (in this setting, anyway). For Max BW, this was set at 1000, and with the advent of faster ports (hello AX86U!), it seemed prudent to just disable.

But please play with the settings and report back.

 

[Morac]

Does this work with hardware acceleration enabled or will installing it disable HW acceleration?

 

[dev_null]

Neither the install nor vnstat modify HwA settings. However, in the alpha, @ColinTaylor reported that vnstat "appears to be using the regular interface stats and therefore suffers from the same issues regarding hardware acceleration (interface vlan2), namely no traffic is recorded for wired Ethernet clients". So with HwA enabled, vnstat-collected statistics may not be accurate or complete.