[Beta 3] Vnstat on Merlin - UI, CLI and email - data use monitoring - now with data limit monitoring

[Khadanja]

Will this work with my settlings? nvram get wan0_ifname shows vlan10

 

[dev_null]

Will this work with my settlings? nvram get wan0_ifname shows vlan10

IPTV is a total unknown because I don't have any experience with it. I might suggest (and ask for other's thoughts) to try Vnstat on vlan10 and see what happens (honestly I'm not sure whether vnstat handles vlans very well or even at all). Try it and let us know.

I'm curious about one other thing - I'm going to PM you because it could be a wild goose chase.

 

[JR Godwin]

I did a full router reset on the 1st, and turned on the Traffic Analyzer shortly after I installed dn-vnstat so the time frame would be almost the same. Here's the output over the past few days. Have to say, it seems pretty much in lockstep. I can't give you what my ISP is saying the usage is because it's not available to me in my customer portal.

 

[JohnD5000]

I rebooted my router and the spdMerlin and vnStat Tabs are no longer displayed on the Addons Tab in Merlin GUI

But, if I go into XShell, I can see that both are still running.

Here is spdMerlin

And, Here is vnstat

How do I get the pages back? The only change I made recently was adding VNSTAT, so I am assuming this is a vnstat issue and not a spdMerlin issue. In any case, Jack Yaz posts here, too.

What should I do to solve this?

Thanks

 

[JohnD5000]

So, as an update:

I decided to remove vnstat and reinstall.

I

1. Delete the scripts associated with vnstat in the /jffs/scripts and /jffs/add-ons directories.
2. Remove the cron jobs (cru d job-name)

Rebooted.
spdMerlin was tab was there and running OK

Ran vnStat install and rebooted

Everything seems to be back running OK (but I lost the vnstat history).

 

[dev_null]

Everything seems to be back running OK (but I lost the vnstat history).

I did not see this conflict in my testing, so this may be an edge case.

For others that want to purge and reset, you could consider exporting your current usage data and importing it. I describe those steps on the Vnstat on Merlin github site.

Spoiler: Here are the steps

Export/import:

a. Export with vnstat --exportdb > /path/to/vnstat-export-db.txt which will export your current database
Re-import the data by running
b. /opt/etc/init.d/S33vnstat kill, followed by
c. vnstat --importdb /path/to/vnstat-export-db.txt -i eth0 --force (enter the correct interface for your setup), followed by
d. /opt/etc/init.d/S33vnstat start

I did notice on one occasion that the import cause a "spike" in daily use (on the date of import). I could not repeat it (that did not occur on subsequent imports) but I put it out there for the sake of complete disclosure.

 

[JohnD5000]

I did not see this conflict in my testing, so this may be an edge case.

For others that want to purge and reset, you could consider exporting your current usage data and importing it. I describe those steps on the Vnstat on Merlin github site.

Spoiler: Here are the steps

Export/import:

a. Export with vnstat --exportdb > /path/to/vnstat-export-db.txt which will export your current database
Re-import the data by running
b. /opt/etc/init.d/S33vnstat kill, followed by
c. vnstat --importdb /path/to/vnstat-export-db.txt -i eth0 --force (enter the correct interface for your setup), followed by
d. /opt/etc/init.d/S33vnstat start

I did notice on one occasion that the import cause a "spike" in daily use (on the date of import). I could not repeat it (that did not occur on subsequent imports) but I put it out there for the sake of complete disclosure.

Thanks, good to know. I just realized that there is an Uninstall option in the vnstat menu when run at the terminal. I should have used that (was not paying attention when I ran it to see if it was still running and didn't look at all the options --DOH!).

Does the uninstall option remove the history? Does it offer a way to save the history if so?

Also, I understand this is still in beta, so I expect glitches. But why would a glitch in vnStat cause spdMerlin tab not to show up in GUI?

 

[Jack Yaz]

I rebooted my router and the spdMerlin and vnStat Tabs are no longer displayed on the Addons Tab in Merlin GUI

View attachment 31548

But, if I go into XShell, I can see that both are still running.

Here is spdMerlin
View attachment 31549

And, Here is vnstat
View attachment 31550

How do I get the pages back? The only change I made recently was adding VNSTAT, so I am assuming this is a vnstat issue and not a spdMerlin issue. In any case, Jack Yaz posts here, too.

What should I do to solve this?

Thanks

Do you have a system log from this reboot?

 

[JohnD5000]

Dev_Null

I have a suggestion for vnstat. Is there any way to add a graph that shows rx & tx over time. I'm thinking something similar to the way spdMerlin shows dl & ul bandwidth over various time

Your time frames could be 1 week, 1 month, 1 year or something similar.

Thanks

 

[Jack Yaz]

Dev_Null

I have a suggestion for vnstat. Is there any way to add a graph that shows rx & tx over time. I'm thinking something similar to the way spdMerlin shows dl & ul bandwidth over various time
View attachment 31577
Your time frames could be 1 week, 1 month, 1 year or something similar.

Thanks

If the Entware team honour my package request and update vnstat to v2.x, yes, as that version moves from a c struct database to Sqlite. Sqlite is used by my chart pages, so it should be trivial to hook up

 

[JohnD5000]

Do you have a system log from this reboot?

I tried to PM you but I could not figure out how to attach a file to a PM

 

[Jack Yaz]

I tried to PM you but I could not figure out how to attach a file to a PM

Please upload it to pastebin as unlisted and pm me the link

 

[dev_null]

Dev_Null

I have a suggestion for vnstat. Is there any way to add a graph that shows rx & tx over time. I'm thinking something similar to the way spdMerlin shows dl & ul bandwidth over various time

Your time frames could be 1 week, 1 month, 1 year or something similar.

Thanks

As Jack commented, he's opened a ticket to hopefully get Entware/vnstat updated. I'd be interested in seeing what magic can be done with sqlite.

For me the beauty of the current view - and the reason I pursued vnstat (vnstati) as opposed to other Entware tools - is you can see "at a glance" the current usage: for all time periods, in a condensed manner with no further drop-downs or other manipulation. I'm a very big fan of "glances" or "dashboards".

(Telling example: years ago someone posted a workaround that allowed dd-wrt users to see all settings on a single, scrollable page. It was about 30 screens long, but everything was there, on one page. Swoon!)

 

[Jack Yaz]

I rebooted my router and the spdMerlin and vnStat Tabs are no longer displayed on the Addons Tab in Merlin GUI

View attachment 31548

But, if I go into XShell, I can see that both are still running.

Here is spdMerlin
View attachment 31549

And, Here is vnstat
View attachment 31550

How do I get the pages back? The only change I made recently was adding VNSTAT, so I am assuming this is a vnstat issue and not a spdMerlin issue. In any case, Jack Yaz posts here, too.

What should I do to solve this?

Thanks

looks like the ol' NTP delay

May  5 01:12:25 RT-AC86U-07F8 uiScribe: NTP failed to sync after 5 minutes. Please resolve!
May  5 01:12:25 RT-AC86U-07F8 spdMerlin: NTP failed to sync after 5 minutes. Please resolve!

I'm not sure why dn-vnstat didn't print a similar log entry. I'm going to take a look at improving this a bit, I think.
Maybe if this gets integrated we'll have a fully reliable way on 386 https://github.com/RMerl/asuswrt-merlin.ng/pull/707
Considering moving to the diskmon event that @dave14305 suggested, but it feels wrong to use that event!

Currently I deliberately design my scripts so the tab mounts don't happen if NTP doesn't sync (for scripts that need time accuracy such as dn-vnstat, spdMerlin etc.) as a clear indicator something isn't right. Perhaps I should still mount the tab but with a message warning re. NTP...I'll experiment

 

[JohnD5000]

looks like the ol' NTP delay

May  5 01:12:25 RT-AC86U-07F8 uiScribe: NTP failed to sync after 5 minutes. Please resolve!
May  5 01:12:25 RT-AC86U-07F8 spdMerlin: NTP failed to sync after 5 minutes. Please resolve!

I'm not sure why dn-vnstat didn't print a similar log entry. I'm going to take a look at improving this a bit, I think.
Maybe if this gets integrated we'll have a fully reliable way on 386 https://github.com/RMerl/asuswrt-merlin.ng/pull/707
Considering moving to the diskmon event that @dave14305 suggested, but it feels wrong to use that event!

Currently I deliberately design my scripts so the tab mounts don't happen if NTP doesn't sync (for scripts that need time accuracy such as dn-vnstat, spdMerlin etc.) as a clear indicator something isn't right. Perhaps I should still mount the tab but with a message warning re. NTP...I'll experiment

Currently my NTP setup is:

Should I add something to Secondary NTP? Should I enable Local NTP? Should I change NTP Server? Thanks.

 

[JohnD5000]

As Jack commented, he's opened a ticket to hopefully get Entware/vnstat updated. I'd be interested in seeing what magic can be done with sqlite.

For me the beauty of the current view - and the reason I pursued vnstat (vnstati) as opposed to other Entware tools - is you can see "at a glance" the current usage: for all time periods, in a condensed manner with no further drop-downs or other manipulation. I'm a very big fan of "glances" or "dashboards".

(Telling example: years ago someone posted a workaround that allowed dd-wrt users to see all settings on a single, scrollable page. It was about 30 screens long, but everything was there, on one page. Swoon!)

Is there a way to combine the two once sqlite is added? I.e. keep your dashboard view at top (like it is now) and at bottom, add a dropdown for more information, if desired?

 

[JohnD5000]

If the Entware team honour my package request and update vnstat to v2.x, yes, as that version moves from a c struct database to Sqlite. Sqlite is used by my chart pages, so it should be trivial to hook up

So, I looked on github and saw this. Does the added "New Package" comment mean that the Entware team has honored your request and added the package and it will be available in next release?

 

[ColDen]

Currently my NTP setup is:

View attachment 31632

Should I add something to Secondary NTP? Should I enable Local NTP? Should I change NTP Server? Thanks.

Hi @JohnD5000,

I have also a RT-AC86U and have noticed that your seem to be missing the "Intercept NTP client requests" option and that your "DST time zone changes ends" have been set to the second Sunday of October, which should be the first Sunday of November:

Therefore it is possible that your reported issues might be related of corruptions in your router configuration.

 

[thelonelycoder]

@dev_null or @Jack Yaz please change line 679 here to not include quotes for the $SSL_FLAG variable, like so:

--user "$USERNAME:$PASSWORD" $SSL_FLAG

Or else curl throws this error:

* Closing connection -1
curl: (3) URL using bad/illegal format or missing URL
dn-vnstat: Summary statistic email failed to send

 

[elorimer]

I'm not seeing that error with gmail's smtp, but clearly dn-vstat is using its formulation sending the stats. My other things emailing use the line as @thelonelycoder has it and don't throw an error either.

I'm still thinking it is better to have one script the router uses for all emails, passing the subject line, text and any attachment to it, and returning a result. When diversion changes to add the encryption flag we'll have to change a bunch of places.