Thanks again for taking the time to help and share this info.You could probably add the VACUUM command to the script to shrink the file sizes when you run it, but that's outside of my use case.
You'd add something like/usr/sbin/sqlite3 <database file> 'vacuum;'
after each of the DELETE commands to the script
array()
object, I just copied/pasted instead of dealing with the research for arrays.vacuum
command to it. Here is the latest iteration to the clean-dbs.sh script. This will output to the console and vacuum the dbs during its run:#!/bin/sh
DBDIR="/tmp/.diag"
dbs="wifi_detect.db stainfo.db eth_detect.db net_detect.db sys_detect.db"
echo ""
echo "********************************************************************************"
echo "***** Running Clean Dbs *****"
echo "********************************************************************************"
echo ""
### List all tmpfs mounts and their sizes
echo ""
echo " TMPFS Sizes (Before):"
df -h | grep -Ew 'Filesystem|tmpfs'
echo ""
echo " Free (Before):"
free
echo ""
echo "Cleaning DBs:"
echo ""
for db in ${dbs}
do
echo "${db} count before delete"
/usr/sbin/sqlite3 $DBDIR/$db 'select count() FROM DATA_INFO;'
/usr/sbin/sqlite3 $DBDIR/$db 'delete from DATA_INFO WHERE strftime("%s","now") - data_time > 300;vacuum;'
echo "${db} count after delete"
/usr/sbin/sqlite3 $DBDIR/$db 'select count() FROM DATA_INFO;'
done
echo ""
echo "DBs cleaned"
### List tmp/.diag files and their sizes
echo " File Sizes:"
ls -lh /tmp/.diag
### List all tmpfs mounts and their sizes
echo ""
echo " TMPFS Sizes (After):"
df -h | grep -Ew 'Filesystem|tmpfs'
echo ""
echo " Free (After):"
free
echo ""
echo "********************************************************************************"
echo "********************************************************************************"
echo ""
Where are these charts located in Merlin, or this a plugin?Odd thing is I had this same setup on an AC56U and it worked fine, and also on this AX68U on prior firmware, it never used the swap file before.
I enabled the swap file again just so it can keep working if it needs it, removed those two scripts you suggested.
Memory looks about the same with those two removed and rebooting but maybe not running the speed test will help. I did have the speed test only running once a day, I think the default was much more often. I will keep an eye on it and see if it starts swapping again.
View attachment 46457
Also, NVRAM and JFFS has always had plenty of free space, so I assume all the errors are related to RAM even though it is complaining about writing files.
View attachment 46458
Its an addon-script "scmerlin", seems to be pretty light weight and has some handy stuff on it. I just went through and removed all the add on scripts I dont need and this was one I kept.Where are these charts located in Merlin, or this a plugin?
ZenWiFi_XD6 armv7l
BusyBox v1.24.1
Firmware Version:3.0.0.4.388_21380
********************************************************************************
***** Running Clean Dbs *****
********************************************************************************
TMPFS Sizes (Before):
Filesystem Size Used Available Use% Mounted on
tmpfs 250.0M 284.0K 249.7M 0% /var
tmpfs 250.0M 64.3M 185.6M 26% /tmp/mnt
tmpfs 250.0M 64.3M 185.6M 26% /tmp/mnt
tmpfs 250.0M 64.3M 185.6M 26% /tmp
Free (Before):
total used free shared buffers cached
Mem: 511928 435500 76428 66372 18820 112564
-/+ buffers/cache: 304116 207812
Swap: 0 0 0
Cleaning DBs:
wifi_detect.db count before delete
65536
wifi_detect.db count after delete
20
stainfo.db count before delete
325056
Error: database or disk is full
stainfo.db count after delete
325056
eth_detect.db count before delete
65536
eth_detect.db count after delete
20
net_detect.db count before delete
32768
net_detect.db count after delete
10
sys_detect.db count before delete
32768
sys_detect.db count after delete
10
DBs cleaned
File Sizes:
-rw-rw-rw- 1 root root 20.0K Feb 5 03:51 channel_change.db
-rw-rw-rw- 1 root root 20.0K Feb 16 13:47 eth_detect.db
-rw-rw-rw- 1 root root 20.0K Feb 16 13:47 net_detect.db
-rw-rw-rw- 1 root root 20.0K Feb 16 04:59 port_status_change.db
-rw-rw-rw- 1 root root 40.5M Feb 16 13:47 stainfo.db
-rw-rw-rw- 1 root root 20.0K Feb 15 01:09 stainfo_stable.db
-rw-rw-rw- 1 root root 20.0K Feb 16 13:47 sys_detect.db
-rw-rw-rw- 1 root root 20.0K Feb 5 03:51 sys_setting.db
-rw-rw-rw- 1 root root 20.0K Feb 16 13:47 wifi_detect.db
-rw-rw-rw- 1 root root 64.0K Feb 16 07:56 wifi_dfs.db
-rw-rw-rw- 1 root root 20.0K Feb 5 03:51 wifi_setting.db
TMPFS Sizes (After):
Filesystem Size Used Available Use% Mounted on
tmpfs 250.0M 284.0K 249.7M 0% /var
tmpfs 250.0M 43.3M 206.7M 17% /tmp/mnt
tmpfs 250.0M 43.3M 206.7M 17% /tmp/mnt
tmpfs 250.0M 43.3M 206.7M 17% /tmp
Free (After):
total used free shared buffers cached
Mem: 511928 415552 96376 44772 19200 91812
-/+ buffers/cache: 304540 207388
Swap: 0 0 0
stainfo.db count before delete
Error: no such table: DATA_INFO
Error: no such table: DATA_INFO
stainfo.db count after delete
Error: no such table: DATA_INFO
stainfo.db
file, @JimbobJay.#!/bin/sh
DBDIR="/tmp/.diag"
dbs="wifi_detect.db stainfo.db eth_detect.db net_detect.db sys_detect.db"
echo ""
echo "********************************************************************************"
echo "***** Running Clean Dbs *****"
echo "********************************************************************************"
echo ""
### List all tmpfs mounts and their sizes
echo ""
echo " TMPFS Sizes (Before):"
df -h | grep -Ew 'Filesystem|tmpfs'
echo ""
echo " Free (Before):"
free
echo ""
echo "Cleaning DBs:"
echo ""
for db in ${dbs}
do
if [[ -f "$DBDIR/$db" ]]; then
echo "${db} count before delete"
/usr/sbin/sqlite3 $DBDIR/$db 'select count() FROM DATA_INFO;'
/usr/sbin/sqlite3 $DBDIR/$db 'delete from DATA_INFO WHERE strftime("%s","now") - data_time > 300;vacuum;'
echo "${db} count after delete"
/usr/sbin/sqlite3 $DBDIR/$db 'select count() FROM DATA_INFO;'
fi
done
echo ""
echo "DBs cleaned"
### List tmp/.diag files and their sizes
echo ""
echo " File Sizes:"
ls -lh /tmp/.diag
### List all tmpfs mounts and their sizes
echo ""
echo " TMPFS Sizes (After):"
df -h | grep -Ew 'Filesystem|tmpfs'
echo ""
echo " Free (After):"
free
echo ""
echo "Process resources:"
ps -w | grep -E 'networkmap|PID|conn_diag|sysstate' | grep -v grep
echo ""
echo "********************************************************************************"
echo "********************************************************************************"
echo ""
Yeah, it seems that the set of *.db files is not going to be always the same in all routers, and apparently not all *.db files will exist on the same router all the time. I don't know what determines when or which specific *.db file gets created and under what circumstances.Looks like you are in the same boat as @mmacedo, in that for whatever reason, you don't havestainfo.db
, @JimbobJay. If you remove it from thedbs
array (~line 4), you won't get those errors.
To be clear, those errors shouldn't cause any problems, per se, they are just indicating that you don't have the SQLite file for stainfo.db.
If you want to remove it, the dbs variable should look like this:
Code:dbs="wifi_detect.db eth_detect.db net_detect.db sys_detect.db"
mkdir -m 755 -p /jffs/scripts
curl -kLSs --retry 3 --retry-delay 5 --retry-connrefused pastebin.com/raw/yfqjSMzp | tr -d '\r' > /jffs/scripts/CleanDBFiles.sh
chmod 755 /jffs/scripts/*.sh
Great work, @Martinski!Yeah, it seems that the set of *.db files is not going to be always the same in all routers, and apparently not all *.db files will exist on the same router all the time. I don't know what determines when or which specific *.db file gets created and under what circumstances.
For example, the following screenshot shows the current DB files on an RT-AX86S router:
View attachment 48041
Notice that it doesn't have the "stainfo.db" and "stainfo_stable.db" files that other users have shown, but it does have a "port_status_usb_change.db" file that others don't have.
BTW, since my cousin's RT-AX86S router was having the same problem with some *.db files getting too big, I took your original script (thank you for that) and made several changes so that it finds only the *.db files that actually exist in the "/tmp/.diag" folder, and then it checks for file size larger than 64KB; when found, it goes ahead and does the trimming of the DB file for data that's older than 30 minutes. This configuration is working well for my cousin's use case (he's running the script every hour as a cron job - no need to run it more often). The modified script also reports the file size just before and right after the trimming process; in addition, it creates a log file of all the results in the "/tmp/var/tmp" folder. If you manually run the script from its location (e.g. cd /jffs/scripts), it will also output the results to the terminal window.
Attached below is a sample log file (CleanDBFiles.log.txt) from an early run of the script.
If you're interested in taking a look at the modified version of the script, it's available in PasteBin:
For the customizable parameters, see the "CUSTOMIZABLE PARAMETERS SECTION" near the top of the script.Bash:mkdir -m 755 -p /jffs/scripts curl -kLSs --retry 3 --retry-delay 5 --retry-connrefused pastebin.com/raw/yfqjSMzp | tr -d '\r' > /jffs/scripts/CleanDBFiles.sh chmod 755 /jffs/scripts/*.sh
My 2 cents.
Hi there,
I've been keeping an eye on this thread as I was getting sporadic reboots on my AX86S running 388.1, I went back to 386_7.2 in the end.
I've recently loaded the latest beta - 388.2_beta1, The router runs with about 32MB free mem, it has a 2Gb swap file but I've never seen it used.
It's been running for a couple of days now without problems, but I've just noticed that the files in /tmp/.diag seem to have some kind of garbage collection going on.
I don't know if it's size-based or a regular job but stainfo.db, for example, seems to grow to around 1600000 before getting chopped down to around 800000.
Maybe ASUS stole your idea?
No, I have very few add-ons: YazDHCP, scMerlin, mosquitto, openssh-sftp-serverI have same/SIMILAR issue, do you run AGH?
I had exacly the same thing over here. I went back to 386_5.2 and treid the newer firmware. But this one was also not stable with my router. So I went back again...Hi there,
I've been keeping an eye on this thread as I was getting sporadic reboots on my AX86S running 388.1, I went back to 386_7.2 in the end.
I've recently loaded the latest beta - 388.2_beta1, The router runs with about 32MB free mem, it has a 2Gb swap file but I've never seen it used.
It's been running for a couple of days now without problems, but I've just noticed that the files in /tmp/.diag seem to have some kind of garbage collection going on.
I don't know if it's size-based or a regular job but stainfo.db, for example, seems to grow to around 1600000 before getting chopped down to around 800000.
Maybe ASUS stole your idea?
Welcome To SNBForums
SNBForums is a community for anyone who wants to learn about or discuss the latest in wireless routers, network storage and the ins and outs of building and maintaining a small network.
If you'd like to post a question, simply register and have at it!
While you're at it, please check out SmallNetBuilder for product reviews and our famous Router Charts, Ranker and plenty more!