vnStat [Solved] vnstat issue

[Milan]

Hi,

Idetected that the vnstat stopped updating data so I checked the settings and it was using jffs to store data. So i tried to change the place of storage but i am getting this error always:

chmod: /jffs/scripts/dn-vnstat: No space left on device

when uninstalling then :

rm: can't remove '/jffs/addons/dn-vnstat.d/vnstat-ui.asp': No space left on device
No packages removed.
No packages removed.
rm: can't remove '/jffs/addons/dn-vnstat.d/.vnstatusage': No space left on device
rm: can't remove '/jffs/addons/dn-vnstat.d/images/vnstat_m.png': No space left on device
rm: can't remove '/jffs/addons/dn-vnstat.d/images/.vnstat_s.htm': No space left on device
rm: can't remove '/jffs/addons/dn-vnstat.d/images/.vnstat_d.htm': No space left on device
rm: can't remove '/jffs/addons/dn-vnstat.d/images/vnstat_hg.png': No space left on device
rm: can't remove '/jffs/addons/dn-vnstat.d/images/.vnstat_t.htm': No space left on device
rm: can't remove '/jffs/addons/dn-vnstat.d/images/vnstat_s.png': No space left on device
rm: can't remove '/jffs/addons/dn-vnstat.d/images/vnstat_d.png': No space left on device
rm: can't remove '/jffs/addons/dn-vnstat.d/images/.vnstat_hg.htm': No space left on device
rm: can't remove '/jffs/addons/dn-vnstat.d/images/vnstat_t.png': No space left on device
rm: can't remove '/jffs/addons/dn-vnstat.d/images/.vnstat_m.htm': No space left on device
rm: can't remove '/jffs/addons/dn-vnstat.d/images': Directory not empty
rm: can't remove '/jffs/addons/dn-vnstat.d/csv/DataUsage_day_monthly.htm': No space left on device
rm: can't remove '/jffs/addons/dn-vnstat.d/csv/DataUsage_fiveminute_monthly.htm': No space left on device
rm: can't remove '/jffs/addons/dn-vnstat.d/csv/DataUsage_hour_monthly.htm': No space left on device
rm: can't remove '/jffs/addons/dn-vnstat.d/csv/DataUsage_fiveminute_weekly.htm': No space left on device
rm: can't remove '/jffs/addons/dn-vnstat.d/csv/DataUsage_hour_daily.htm': No space left on device
rm: can't remove '/jffs/addons/dn-vnstat.d/csv/WeekSummary.htm': No space left on device
rm: can't remove '/jffs/addons/dn-vnstat.d/csv/dn-vnstatdata.zip': No space left on device
rm: can't remove '/jffs/addons/dn-vnstat.d/csv/DataUsage_fiveminute_daily.htm': No space left on device
rm: can't remove '/jffs/addons/dn-vnstat.d/csv/DataUsage_day_weekly.htm': No space left on device
rm: can't remove '/jffs/addons/dn-vnstat.d/csv/DataUsage_hour_weekly.htm': No space left on device
rm: can't remove '/jffs/addons/dn-vnstat.d/csv/WeekPrev.htm': No space left on device
rm: can't remove '/jffs/addons/dn-vnstat.d/csv/WeekThis.htm': No space left on device
rm: can't remove '/jffs/addons/dn-vnstat.d/csv/DataUsage_day_daily.htm': No space left on device
rm: can't remove '/jffs/addons/dn-vnstat.d/csv': Directory not empty
sed: can't create temp file '/jffs/addons/custom_settings.txtFG5vNB': No space left on device

sed: can't create temp file '/jffs/addons/custom_settings.txtooanMA': No space left on device

Jffs is not full by firmware:

How to solve this? it is happenning only for this one script ...

 

[ColinTaylor]

Reboot the router. If it still isn't working post the output of the following command:

mount | grep jffs

 

[Milan]

Reboot the router. If it still isn't working post the output of the following command:

mount | grep jffs

Reboot helped. Will see if the data is updated ...

EDIT: all is working fine.

 

[Zim]

I'm also having an issue with this script.

When I installed it on a new router, it worked - the stats were showing on the GUI etc. After a few reboots, the stats are no longer updating. The GUI only shows stats for about 3 to 5 hours after the installation, after that the stats are no longer updating.

I ran this command:

mount | grep jffs

Here is the output:
ubi:jffs2 on /jffs type ubifs (rw,relatime,assert=read-only,ubi=0,vol=13)
ubi:jffs2 on /www/Advanced_DHCP_Content.asp type ubifs (rw,relatime,assert=read-only,ubi=0,vol=13)
ubi:jffs2 on /www/Main_LogStatus_Content.asp type ubifs (rw,relatime,assert=read-only,ubi=0,vol=13)


I've tried uninstalling and reinstalling the script, but the issue persists. The script stops working after reboot.

Would love to get this working again. Appreciate the help!

 

[elorimer]

There are two (or three) parts to the puzzle. The first is the script itself, which I've never had issues with (either the installation or the chron part). The second is vnstat itself, and the third is S33vnstat module that entware uses to start etc., vnstat.

I've found on occasion that S33vnstat doesn't successfully start vnstat after a reboot. I think (or thought--it happened to me just yesterday) that this can be mitigated by deleting some of the code in the beginning of S33vnstat, particularly the lines that test for where storage is (that can be hardcoded), and the lines that test for and remove S32vnstat (the older version of dn-vnstat). (I think these lines should be part of the installation, and not part of the start sequence.)

So you might consider framing your question more exactly. Is vnstat stopping at some point after it has started ("3 to 5 hours after installation"), or is it not starting on reboot ("the script stops working after reboot"). An easy way to tell is that a daily stat email gives you a yellow line at midnight if vnstat isn't running.

 

[dev_null]

Here is the output:
ubi:jffs2 on /jffs type ubifs (rw,relatime,assert=read-only,ubi=0,vol=13)
ubi:jffs2 on /www/Advanced_DHCP_Content.asp type ubifs (rw,relatime,assert=read-only,ubi=0,vol=13)
ubi:jffs2 on /www/Main_LogStatus_Content.asp type ubifs (rw,relatime,assert=read-only,ubi=0,vol=13)


I've tried uninstalling and reinstalling the script, but the issue persists. The script stops working after reboot.

Would love to get this working again. Appreciate the help!

The way I read this your jffs is mounted read-only (message highlighted), and if so this is a router issue, not a script issue.

Something similar was described here: https://www.snbforums.com/threads/jffs-partition-is-read-only-and-cannot-format-restore.85631/

You should try a complete reset, check to see if the "read-only" message is gone, and manually rebuild your configuration.

 

[dev_null]

I've found on occasion that S33vnstat doesn't successfully start vnstat after a reboot. I think (or thought--it happened to me just yesterday) that this can be mitigated by deleting some of the code in the beginning of S33vnstat, particularly the lines that test for where storage is (that can be hardcoded), and the lines that test for and remove S32vnstat (the older version of dn-vnstat). (I think these lines should be part of the installation, and not part of the start sequence.)

@elorimer: As I indicated separately, I'm happy to work with you on this, but it appears to be something in your particular configuration, since this isn't a widespread (or even much beyond very infrequent). If you find a modification of the script that solves the issue, please let me know.

The script was designed to be self-contained and I'd be open to updating it to remove certain (legacy) actions if these can be conclusively proven to be causing the issue you're experiencing. But once dn-vnstat is installed it's mostly the (Linux app) S33vnstat daemon doing the work. After your initial report, I checked with the (Linux app) vnstat author (Teemu) who has no similar reports.

I know this "stall" of the daemon happens - I saw it once in my early testing. But I cannot reproduce it using modern hardware and I'm using dn-vnstat alongside multiple scripts.

 

[elorimer]

My particular issue is not a stall of the daemon but a failure to start, which I thought might be the last poster. I didn't mean to restart the conversation.

But while I have you, I'm curious as to why S33 is written to have all of

if [ -f /opt/etc/init.d/S32vnstat ]; then
    /opt/etc/init.d/S32vnstat stop
    rm -f /opt/etc/init.d/S32vnstat
fi

if [ -f /opt/etc/init.d/S32vnstat2 ]; then
    /opt/etc/init.d/S32vnstat2 stop
    rm -f /opt/etc/init.d/S32vnstat2
fi

if [ -f "/opt/share/dn-vnstat.d/config" ]; then
    SCRIPT_STORAGE_DIR="/opt/share/dn-vnstat.d"
else
    SCRIPT_STORAGE_DIR="/jffs/addons/dn-vnstat.d"
fi

instead of

SCRIPT_STORAGE_DIR="/jffs/addons/dn-vnstat.d"

and

if [ "$1" = "start" ] || [ "$1" = "restart" ]; then
    if [ -f "$SCRIPT_STORAGE_DIR/vnstat.conf" ]; then
        IFACE="$(grep "Interface " "$SCRIPT_STORAGE_DIR/vnstat.conf" | awk '{print $2}' | sed 's/"//g')"
        logger -st vnstat "Starting monitoring for interface $IFACE"
    else
        logger -st vnstat "Configuration file missing - $SCRIPT_STORAGE_DIR/vnstat.conf - please check"
        exit 1
    fi
   
    if [ "$(nvram get ntp_ready)" -eq 0 ]; then
        logger -st vnstat "NTP not synced, exiting"
        exit 1
    fi
   
    if ! vnstat --config "$SCRIPT_STORAGE_DIR/vnstat.conf" -i "$IFACE" >/dev/null 2>&1; then
        vnstatd $ARGS >/dev/null 2>&1
        vnstat --config "$SCRIPT_STORAGE_DIR/vnstat.conf" --add -i "$IFACE" >/dev/null 2>&1
        killall vnstatd >/dev/null 2>&1
    fi
fi

instead of something shorter, since some of this could be determined once at installation, instead of rerun each time?

 

[dev_null]

My particular issue is not a stall of the daemon but a failure to start, which I thought might be the last poster. I didn't mean to restart the conversation.

But while I have you, I'm curious as to why S33 is written to have all of [snip]

instead of something shorter, since some of this could be determined once at installation, instead of rerun each time?

The second part of your question is an install of (Linux) vnstat (or vnstat2) via Entware names its daemon S32vnstat, and so our script checks to see if the Entware version is installed and prunes it.

For the first part, these are all failsafe measures to ensure that all the configuration files exist, in case something was modified. As for NTP, dn-vnstat 2.x data is logged into a sqlite file, so having accurate capture and ordering is important.

If nothing changes on your machine, and the clock immediately syncs, then these checks are redundant, but I don't know if we can count on that across the user base.

I don't believe that any of these checks should impact the "regular" execution of the script, but I'm open to optimizations.

 

[Zim]

The way I read this your jffs is mounted read-only (message highlighted), and if so this is a router issue, not a script issue.

Something similar was described here: https://www.snbforums.com/threads/jffs-partition-is-read-only-and-cannot-format-restore.85631/

You should try a complete reset, check to see if the "read-only" message is gone, and manually rebuild your configuration.

@dev_null when I saw the output, I also thought the same that the JFFS is read only, but I'm running other scripts like spdMerlin and that is writing the results to JFFS without issue.

Oddly enough, dn-vnstat has been running well for me for over 2 years. I only started having problems with it since late last year (not sure what changed). At that time I was using AX11K and now I'm using AXE16K. Running all the same scripts on both setups and only this script has this issue.

I've done clean install a dozen times with various scripts in different order (not that it should make a difference), but this is the only script that stops working after a reboot. Like I mentioned in the other thread, dn-vnstat runs after reboot, but vnstat is "dead". S33vnstat doesn't start on its own. Running this: /opt/etc/init.d/S33vnstat start and things start rolling again.

 

[Martinski]

mount | grep jffs

Here is the output:
ubi:jffs2 on /jffs type ubifs (rw,relatime,assert=read-only,ubi=0,vol=13)
ubi:jffs2 on /www/Advanced_DHCP_Content.asp type ubifs (rw,relatime,assert=read-only,ubi=0,vol=13)

The way I read this your jffs is mounted read-only (message highlighted), and if so this is a router issue, not a script issue.
...

That's an incorrect interpretation of the mount cmd output.

ubi:jffs2 on /jffs type ubifs (rw,relatime,assert=read-only,ubi=0,vol=13)

Essentially, the above line is saying that the JFFS partition is mounted (on a UBI volume) with Read/Write permissions (rw); however, in the event that an assert is triggered by an unrecoverable error (as determined by the implementation), the R/W mounted partition will be switched to Read-Only mode (assert=read-only). This response to a failed assertion is usually done in an effort to protect the data already written to the filesystem and to prevent any possible data corruption.

IOW, at some point after being initially mounted R/W, it's certainly possible that the JFFS partition can become Read-Only due to a failed assert call in the code. In such cases, /proc/mounts should reflect the R/O mode change.

@dev_null when I saw the output, I also thought the same that the JFFS is read only, but I'm running other scripts like spdMerlin and that is writing the results to JFFS without issue.

This makes sense since the mounted JFFS partition has not been switched to Read-Only from its initial Read/Write mode.

HTH.

 

[dev_null]

@dev_null when I saw the output, I also thought the same that the JFFS is read only, but I'm running other scripts like spdMerlin and that is writing the results to JFFS without issue.

Oddly enough, dn-vnstat has been running well for me for over 2 years. I only started having problems with it since late last year (not sure what changed). At that time I was using AX11K and now I'm using AXE16K. Running all the same scripts on both setups and only this script has this issue.

I've done clean install a dozen times with various scripts in different order (not that it should make a difference), but this is the only script that stops working after a reboot. Like I mentioned in the other thread, dn-vnstat runs after reboot, but vnstat is "dead". S33vnstat doesn't start on its own. Running this: /opt/etc/init.d/S33vnstat start and things start rolling again.

I'm not sure either. As I indicated in the dedicated vnStat-on-Merlin thread, I contacted the owner of (Linux app) vnStat and there are no similar reports, though I'm not sure how frequently the (Linux) app is used on router hardware outside of dn-vnstat.

I could tweak the script to add an additional S33vnstat restart, but for the 99.999% of users this would be redundant and could potentially interfere with data collection. Another approach might be to S33vnstat check and if the result is "dead" then issue a S33vnstat restart, but that is potentially a feedback loop and still doesn't help us understand why, in a few instances, the (Linux) app dies.

Here's a quick and dirty script you can run at reboot (or shortly after) to check the status of the daemon; it's something like this I could build into dn-vnstat:

#!/bin/sh
sleep 120
MSG=$(/opt/etc/init.d/S33vnstat check)
echo $MSG | grep 'dead'
if [ $? -eq 0 ]
then
echo "S33vnstat is dead... restarting"
/opt/etc/init.d/S33vnstat restart && sleep 10 && /opt/etc/init.d/S33vnstat check
logger -s -t vnstat-check S33vnstat restart triggered
else
echo "S33vnstat is running"
exit
fi

I'm sure someone will point out the inefficiency of the above, but it works.

EDIT: inserted a line to add the restart to the system log.

EDIT 2: added an early sleep command in case the daemon crashes rather than doesn't start, so the script waits 2 minutes to check the daemon status. Credit @Zim for the idea (see below).

 

[Zim]

I'm not sure either. As I indicated in the dedicated vnStat-on-Merlin thread, I contacted the owner of (Linux app) vnStat and there are no similar reports, though I'm not sure how frequently the (Linux) app is used on router hardware outside of dn-vnstat.

I could tweak the script to add an additional S33vnstat restart, but for the 99.999% of users this would be redundant and could potentially interfere with data collection. Another approach might be to S33vnstat check and if the result is "dead" then issue a S33vnstat restart, but that is potentially a feedback loop and still doesn't help us understand why, in a few instances, the (Linux) app dies.

Here's a quick and dirty script you can run at reboot (or shortly after) to check the status of the daemon; it's something like this I could build into dn-vnstat:

#!/bin/sh
MSG=$(/opt/etc/init.d/S33vnstat check)
echo $MSG | grep 'dead'
if [ $? -eq 0 ]
then
echo "S33vnstat is dead... restarting"
/opt/etc/init.d/S33vnstat restart && sleep 10 && /opt/etc/init.d/S33vnstat check
logger -s -t vnstat-check S33vnstat restart triggered
else
echo "S33vnstat is running"
exit
fi

I'm sure someone will point out the inefficiency of the above, but it works.

EDIT: inserted a line to add the restart to the system log.

Thanks for looking into this and putting together a mini-script for this @dev_null. Much apprecaited!

For the uninitiated, where should this script go?


EDIT: I tried to be creative and put

sleep 120 & /opt/etc/init.d/S33vnstat start

at the end of the services-start file in /jffs/scripts but it didn't work. Why not?

 

[dev_null]

Thanks for looking into this and putting together a mini-script for this @dev_null. Much apprecaited!

For the uninitiated, where should this script go?


EDIT: I tried to be creative and put

sleep 120 & /opt/etc/init.d/S33vnstat start

at the end of the services-start file in /jffs/scripts but it didn't work. Why not?

Create the script as a standalone file (check-vnstat.sh) in the /jffs/scripts folder, make it executable (chmod +x check-vnstat.sh) and add the command to the end of services-start (add the command sh /jffs/scripts/check-vnstat.sh) towards the end of that file.

Adding a sleep command is probably a reasonable step, especially if the daemon times out rather than doesn't start. To run that on a single line you need to double ampersand I believe (&&) between commands, or add a sleep command (sleep 120) as the second line in the new check-vnstat.sh script, which allows your services-start to finish executing while the check-vnstat.sh does its thing separately.

The way you've written it might not "start" if S33vnstat already running but I'm pretty sure that "restart" will either "start" or "restart".

 

[Zim]

Create the script as a standalone file (check-vnstat.sh) in the /jffs/scripts folder, make it executable (chmod +x check-vnstat.sh) and add the command to the end of services-start (add the command sh /jffs/scripts/check-vnstat.sh) towards the end of that file.

Adding a sleep command is probably a reasonable step, especially if the daemon times out rather than doesn't start. To run that on a single line you need to double ampersand I believe (&&) between commands, or add a sleep command (sleep 120) as the second line in the new check-vnstat.sh script, which allows your services-start to finish executing while the check-vnstat.sh does its thing separately.

The way you've written it might not "start" if S33vnstat already running but I'm pretty sure that "restart" will either "start" or "restart".

@dev_null writing to report that your instructions and the additional scripts is working fine. The stats are updating and am not seeing any issues. Will report back if I run into any issues, but as of now all scripts and router is humming along nicely.

Thanks.

 

[elorimer]

I've been trying to run this to earth on my AX-86Pro. I forced a reinstall of dn-vnstat to start over. Then I edited S33 to put logger messages everywhere.

In 20 or so reboots from the webui or cli, S33vnstat did not execute anything below the path statement, and so vnstat never started. Logger messages from above the normal rc calls printed fine. At the same time, S33vnstat startnever failed to start vnstat and pushed out all the logger statements. At the same time, I'm not (so far as I know) experiencing issues with IoT devices.

Diversion, Scribe, Unbound, scmerlin, vpnservers/clients, are all coming up and performing as expected.

I wish I could provide some clue. Vnstatd is running as the same admin user as everything else. My AX88 is not experiencing anything like this.

 

[dev_null]

I cannot envisage how vnstat (the Linux app or dn-vnstat which calls it) would have any effect on IoT devices specifically, since all it's doing is monitoring the amount of data passing through an interface.

I only have 6-8 IoT devices and they are isolated on a separate vlan, and where possible I block access back to IoT servers except in certain instances.

I'd be more inclined to think it's another service interfering with connectivity to IoT servers, but since I don't know your complete setup I wouldn't know where to start.

 

[dev_null]

n 20 or so reboots from the webui or cli, S33vnstat did not execute anything below the path statement, and so vnstat never started. Logger messages from above the normal rc calls printed fine. At the same time, S33vnstat startnever failed to start vnstat and pushed out all the logger statements. At the same time, I'm not (so far as I know) experiencing issues with IoT devices.

@elorimer you're saying that the S33vnstat daemon is halting in the middle of the script (checking for the existing of configuration and NTP sync)? That would imply your setup is for some reason not processing the commands correctly.

Have you checked that your changes in the script - or other scripts that are running - have not modified the variables that are being checked?

Here is the code for S32vnstat if you want to substitute (make sure the paths reflect your set-up). The only difference is the verification of the configs, the NTP sync and ensuring the the interface is correctly identified inS33vnstat:

#!/bin/sh

# Point right interfaces to monitor. Remove -i ppp0 to monitor all of them.
[ -f "/opt/var/lib/vnstat/*" ] || vnstat -u -i eth0

ENABLED=yes
PROCS=vnstatd
ARGS="-d"
PREARGS=""
DESC=$PROCS
PATH=/opt/sbin:/opt/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

. /opt/etc/init.d/rc.func

I have no idea why in your instance these checks fail but executing the S33vnstat start command works.

 

[elorimer]

Here are the logging statements I've added to a fresh install of S33vnstat, which is a lot different than S32vnstat:

#!/bin/sh
# shellcheck disable=SC1091
# shellcheck disable=SC2034
# shellcheck disable=SC2086

if [ -f /opt/etc/init.d/S32vnstat ]; then
    /opt/etc/init.d/S32vnstat stop
    rm -f /opt/etc/init.d/S32vnstat
    logger -st vnstat "S33: Removed old v1 S32 from entware"
fi

if [ -f /opt/etc/init.d/S32vnstat2 ]; then
    /opt/etc/init.d/S32vnstat2 stop
    rm -f /opt/etc/init.d/S32vnstat2
    logger -st vnstat "S33: Removed old v2 S32 from entware"

fi

if [ -f "/opt/share/dn-vnstat.d/config" ]; then
    SCRIPT_STORAGE_DIR="/opt/share/dn-vnstat.d"
    logger -st vnstat "S33: Set script storage on /opt usb"

else
    SCRIPT_STORAGE_DIR="/jffs/addons/dn-vnstat.d"
    logger -st vnstat "S33: Set script storage on /jffs internal"
fi

TZ=$(cat /etc/TZ)
export TZ

ENABLED=yes
PROCS=vnstatd
ARGS="-d --noadd --config $SCRIPT_STORAGE_DIR/vnstat.conf"
PREARGS=""
DESC="$PROCS"
PATH=/opt/sbin:/opt/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

logger -st vnstat "S33: Starting VNstat"
if [ -f "$SCRIPT_STORAGE_DIR/vnstat.conf" ]; then
    logger -st vnstat "S33: Setting interface"
    IFACE="$(grep "Interface " "$SCRIPT_STORAGE_DIR/vnstat.conf" | awk '{print $2}' | sed 's/"//g')"
        logger -st vnstat "Starting monitoring for interface $IFACE"
    else
        logger -st vnstat "Configuration file missing - $SCRIPT_STORAGE_DIR/vnstat.conf - please check"
        exit 1
    fi
  
    if [ "$(nvram get ntp_ready)" -eq 0 ]; then
        logger -st vnstat "NTP not synced, exiting"
        exit 1
    fi
  
    if ! vnstat --config "$SCRIPT_STORAGE_DIR/vnstat.conf" -i "$IFACE" >/dev/null 2>&1; then
    vnstatd $ARGS >/dev/null 2>&1
    logger -st vnstat "S33: something with vnstatd"
    vnstat --config "$SCRIPT_STORAGE_DIR/vnstat.conf" --add -i "$IFACE" >/dev/null 2>&1
    logger -st vnstat "S33: doing something with vnstat"
    killall vnstatd >/dev/null 2>&1
    logger -st vnstat "S33: did killall of vnstatd"
    fi


. /opt/etc/init.d/rc.func

On a reboot, the log shows execution at least down to the TZ line. Nothing below the PATH line executes. Manually running S33vnstat, the logging statements below the PATH line appear in the log and vnstatd is running. S32vnstat doesn't have anything below the PATH statement. (I don't understand why the script ends with a killall of vnstatd, BTW, but in any case rc.func would then run vnstatd -d --noadd --config /jffs/addons/dn-vnstat.d/vnstat.conf). This doesn't seem like a vnstatd or vnstat issue.