Restore Traffic Stats

[sbsnb]

For the last few times I've upgraded my firmware it nukes the traffic stats except for the previous 30 days. I have them stored every 6 hours in /mnt/sda1/stats/, but it seems like that's not allowing previous months' history to be retained after firmware updates. Is there a way to manually restore from backups I have?

 

[Maverickcdn]

Do you factory reset on updates or no?

First steps would be to see if the file matches what the router is looking for

What is the setting on Tools-Other Settings


What is the filename on USB in that location?

Is anything populated in the MAC Clone block under Admin

If nothing in the box above does your router MAC address match the filename on USB?
Through SSH you can use to check, otherwise it should be on the router sticker.

nvram get '0:macaddr'

 

[sbsnb]

No factory resets. The file is correct as you can see the last modified time gets updated every six hours. No clone MAC. Filename matches router's MAC.

I tried manually restoring a backup from before I updated and it still doesn't have anything beyond 30 days ago. I suspect that when it's saving data it's not saving all of the data. If so, that would mean the restore is happening but the file doesn't have all the data.

 

[sbsnb]

If I'm not misunderstanding rstats.c, it has an upper limit of 62 days. I'm curious why mine is half of that, but I'd prefer there was no upper bound.

 

[RMerlin]

If I'm not misunderstanding rstats.c, it has an upper limit of 62 days. I'm curious why mine is half of that, but I'd prefer there was no upper bound.

No, it goes longer than that. I have over two years of historical data myself:

 

[sbsnb]

I have several backups taken at various times over the last year. I'm curious how I can open them to see what data is in there. I tried sqllite, but it reported: Error: file is not a database

There are only 845 bytes in the stats files not counting the null bytes. I can't see how it's possible to store years worth of traffic statistics in 845 bytes, so I'm guessing it's not being stored for whatever reason.

 

[RMerlin]

The data file is in a custom format, it's not human-readable.

Note that it might not be keeping daily data for the whole history, it's possible that it only retains monthly data for older periods. It's been years since I've looked at the rstats code, so I don't remember.

 

[sbsnb]

In my case there's no daily, weekly, or monthly data before March 1. I noted this same effect when I updated my firmware the last time in August, but I thought it was a fluke. I don't know if it's related to the firmware update or just rebooting since I generally only reboot when I update.

 

[Keenan]

I noticed the same thing, everything before Mar 1 was deleted or missing.

 

[RMerlin]

My daily data goes all the way back to January 30th, so that would fit with the 62 days mentioned earlier.

 

[Jeffrey Young]

I wrote an Excel script to import my usage into another database. I shared it here

MS Excel Script to read Tomato RStat Files

Hello everyone, Updated April 3, 2022 Re-deigned to read historical data in file if the file rolled over. Thanks @sbsnb After trying a number of python scripts that I found elsewhere that claims to read the Tomato Rstat files that Merlin creates to no avail (did not find any...

www.snbforums.com

 

[sbsnb]

I'm still trying to figure out how to debug why I'm losing my traffic stats every upgrade or reboot even though the backups are being written. I would like to be able to verify that what is being written does contain all of the data independently of the router UI, but it seems that there's no way to do that. My next thought is to manually stop rstats and copy the tomato_rstats_xxxxxxxxxxxx.gz file to /var/lib/misc/rstats-history.gz and restart rstats.

Right now I'm taking a swag that there's some issue reading the USB device when rstats starts and the backup is not restored.

 

[sbsnb]

Thinking before coffee has its consequences. There has to be some sort of restore happening or I wouldn't have data back to March 1. The question is why is the rest truncated? The last reboot prior to yesterday was August 21, 2021. The rstats file on the USB device has a last-modified time only two hours before upgrading the firmware.

 

[Jeffrey Young]

I might not have been clear in my intention. You can use my script to Independently look at your rstat file to see what data is there or not there.

 

[sbsnb]

Thanks for that. This just deepens the mystery. Assuming all the data is read correctly, there is something very wrong.

Where is all the daily data? The daily data from March 1 is visible in the UI. The monthly data is in the spreadsheet but does not appear in the router UI.

 

[sbsnb]

This is reading in the data that's directly pulled from /var/lib/misc/rstats-history.gz. It matches what's being stored in /mnt/sda1/stats, which is where the router's UI is configured to save backups.

 

[Jeffrey Young]

I have had data mysteriously go awal before as well. I tracked my issue down to my USB drive unmounting for no apparent reason. I moved the rstat storage location to the jffs partition and have no further issues. It also lead to a cron job to check the amount status of the USB drive every couple of hours and remount if required.

Perhaps it is time to give the VNSTAT addons some consideration. I believe it is available from AMTM now. That project came about partially from issues other folks were having with rstat

 

[sbsnb]

USB issues were my first thought. That's why I checked the data in /var/lib/misc/rstats-history.gz. That data has been written since I updated and it's not on the USB, yet it's faulty also. There is something wrong in how the data is read, written, or both. Right now it's hard to say if the issue is with rstats, the UI, or some combination.

 

[Jeffrey Young]

I am not familiar with the AX models, but is the var/lib directory a ram drive. In other words, it is over written on a reboot. Possible that the data is being replaced from that history file at boot? Just brain storming here. What tells the AX model where to put that history file (I have never checked to see if I have one on my AC86U)

 

[Jeffrey Young]

Did you reboot by any chance on March 28?