Out of Memory Errors 388.1 / No space left on device [SOLUTIONS]

[mmacedo]

I suspect it might have something to do with AiMesh?
...
Top (I don't run entware/htop):

There's one record for each wireless device connected to your router and nodes. I counted 20 devices connected to the router, 8 connected to one node and 6 connected to the other node. I guess it adds to the table a snapshot of all devices every minute, so you'll have x new records every minute, being x the number of wireless devices connected at the moment. So, it'd grow quite fast if you weren't deleting older records. Weird thing is that I didn't have this stainfo.db on my RT-AX68U when conn_diag was running, just the other ones. What router do you have? Are you running Merlin 388.1?

As for top, try it again later and check if there's any process that increased memory usage significantly.

 

[lamentary]

. Weird thing is that I didn't have this stainfo.db on my RT-AX68U when conn_diag was running, just the other ones. What router do you have? Are you running Merlin 388.1?

I am also using an RT-AX68U running Merlin 388.1. I also have 2 RT-AC68U nodes, each running Merlin 386.9

 

[mmacedo]

I am also using an RT-AX68U running Merlin 388.1. I also have 2 RT-AC68U nodes, each running Merlin 386.9

Weird indeed. I don't why then.

 

[Donga120]

Your problem seems completely different than that discussed in this thread.

Appears to be that way, sort of, looking like in my case that AGH is eating mine up. I haven't actually seen any of my SWAP used, but I know it's normal for AGH to use a lot of RAM and I do wonder wther this causes complete lockup of the router, with pulling the power the only resort.

 

[jtp10181]

Appears to be that way, sort of, looking like in my case that AGH is eating mine up. I haven't actually seen any of my SWAP used, but I know it's normal for AGH to use a lot of RAM and I do wonder wther this causes complete lockup of the router, with pulling the power the only resort.

The errors you posted earlier about the lock files for networkmap would happen for anyone, its a bug in the firmware. Probably an insignificant bug at that. Your Adguard install is using significantly more memory than anything else. If you want to try my script to kill networkmap nightly go for it. If AdGuard is causing the issue, stop using it.

 

[jata]

Hello to all on this thread! Been away on holiday so have not been looking at my router or this thread. Back now and trying to catch-up. So do we think that the two issues we are seeing are related?

1. RAM slowly being consumed by a couple of processes (e.g. networkmap) and random reboot sometimes happens (OOM)
2. Storage in /tmp being filled by some .db files that are not being truncated/cleared resulting in 'no space on device' errors

It does feel like these are completely separate issues (unless /tmp is mounted into RAM - anyone know?). But...

While I was away, I had both of these issues occur at the same time... First I got the 'no space left' issue and this went on for around 8hrs - see below but note that this is not related to /tmp storage

Jan 28 09:08:44 dnsmasq-dhcp[1503]: failed to write /var/lib/misc/dnsmasq.leases: No space left on device (retry in 60 s)

Then I see that networkmap crashes (and I think restarted by watchdog). Router does not reboot.

Jan 28 09:09:54 kernel: CPU: 1 PID: 1567 Comm: networkmap Tainted: P           O    4.1.52 #2

At this point, router is totally stable and no more 'no space on device issues' but the router did not restart - I think networkmap crashing and restarting helped the router to recover - below are the file sizes from /tmp/.diag

-rw-rw-rw-    1  19742720 Jan 31 09:20 wifi_detect.db
-rw-rw-rw-    1   7122944 Jan 31 09:20 net_detect.db
-rw-rw-rw-    1   6823936 Jan 31 09:20 eth_detect.db
-rw-rw-rw-    1   4894720 Jan 31 09:20 sys_detect.db
-rw-rw-rw-    1     32768 Jan 31 09:09 port_status_change.db
-rw-rw-rw-    1     20480 Jan 17 10:05 channel_change.db
-rw-rw-rw-    1     20480 Jan 17 10:05 port_status_usb_change.db
-rw-rw-rw-    1     20480 Jan 17 11:59 stainfo.db
-rw-rw-rw-    1     20480 Jan 19 08:50 sys_setting.db
-rw-rw-rw-    1     20480 Jan 19 08:50 wifi_dfs.db
-rw-rw-rw-    1     20480 Jan 19 08:50 wifi_setting.db

Not sure if any of this is helpful

 

[RMerlin]

Asus confirmed that the conn_diag database issues were already fixed on their end, and should therefore be included in the next code release.

 

[jtp10181]

Cool. hopefully they will also fix the networkmap process from slowly eating all the ram until the router crashes.

That's really what this has turned into, how to manage this until Asus fixes it. I could downgrade but it seems to work fine for me besides the networkmap issue which is easily solved.

I only got the out of space issues once, so not sure what happened there, since I have no idea what /tmp looked like at that time.

 

[lamentary]

I have had the issue multiple times - in fact, it's what drove me to this thread. Whether these are multiple issues or a single one, I cannot say, but I'm looking at anywhere that might have a memory leak and attempting to mitigate. I'm glad to see that at least one front of this is being addressed by Asus. Thanks @RMerlin, for following up on it!

 

[jata]

I also hope both these issues are related and fixed with the new GPL that hopefully comes soon.

I don’t really see how the issues can be related as one is RAM and the other file space/storage.

We will see!

 

[mmacedo]

I don’t really see how the issues can be related as one is RAM and the other file space/storage.

TMPFS is on RAM: https://en.wikipedia.org/wiki/Tmpfs

 

[jata]

TMPFS is on RAM: https://en.wikipedia.org/wiki/Tmpfs

That is good news and will explain how the issues are related.

in my earlier post, I was getting the error while the system was trying to write to /var/lib/misc/dnsmasq.leases but maybe tmpfs is used somehow.

great to see a fix coming.

my system has the issue once every 1-2 weeks so I’m fine waiting for the next Merlin release.

 

[mmacedo]

in my earlier post, I was getting the error while the system was trying to write to /var/lib/misc/dnsmasq.leases but maybe tmpfs is used somehow.

Yes, that's because /var is on tmpfs as well. Run this command df -h | grep -Ew 'tmpfs|Filesystem' and you'll see something like this:

Filesystem                Size      Used Available Use% Mounted on
tmpfs                   206.0M    336.0K    205.7M   0% /var
tmpfs                   206.0M      2.1M    203.9M   1% /tmp/mnt
tmpfs                   206.0M      2.1M    203.9M   1% /tmp/mnt
tmpfs                   206.0M      2.1M    203.9M   1% /tmp

 

[jata]

Yes, that's because /var is on tmpfs as well. Run this command df -h | grep -Ew 'tmpfs|Filesystem' and you'll see something like this:

Filesystem                Size      Used Available Use% Mounted on
tmpfs                   206.0M    336.0K    205.7M   0% /var
tmpfs                   206.0M      2.1M    203.9M   1% /tmp/mnt
tmpfs                   206.0M      2.1M    203.9M   1% /tmp/mnt
tmpfs                   206.0M      2.1M    203.9M   1% /tmp

Thanks for this clarification.

All makes sense now and i'm now fairly confident that these issues are both related to those .db files filling up the tmpfs.

 

[lamentary]

Thanks for this clarification.

All makes sense now and i'm now fairly confident that these issues are both related to those .db files filling up the tmpfs.

They may still be unrelated, it's entirely possible that there are multiple memory leaks, and that fixing one may just delay the symptoms of the other from revealing themselves, but mitigating/fixing any (or, dare I say all?) of them will be helpful.

I have yet to add the networkmap and sysstate killers to my crons, but have been too obsessive over monitoring my RAMs state. As I've watched the usage of RAM increase I can't seem to just let them go without killing them from the shell. To be clear, this could just be normal, I haven't let it grow above ~80% for the last few days. I need to have a little more discipline and just wait for the router to crash out or not in order to figure out if the solution for the .db files is enough on my router to eliminate other possible issues.

 

[lamentary]

FWIW, here's the output of my scripts from ~ 9:00 AM PST this morning:

********************************************************************************
*****                      Running Clean Dbs                               *****
********************************************************************************
wifi_detect.db count before delete
36
wifi_detect.db count after delete
30
stainfo.db count before delete
219
stainfo.db count after delete
182
eth_detect.db count before delete
36
eth_detect.db count after delete
30
net_detect.db count before delete
6
net_detect.db count after delete
5
sys_detect.db count before delete
6
sys_detect.db count after delete

 File Sizes:
-rw-rw-rw-    1         20.0K Jan 31 11:28 channel_change.db
-rw-rw-rw-    1         20.0K Feb  1 09:00 eth_detect.db
-rw-rw-rw-    1         20.0K Feb  1 09:00 net_detect.db
-rw-rw-rw-    1         20.0K Jan 31 12:51 port_status_change.db
-rw-rw-rw-    1         20.0K Jan 31 11:28 port_status_usb_change.db
-rw-rw-rw-    1         56.0K Feb  1 09:00 stainfo.db
-rw-rw-rw-    1         20.0K Jan 31 16:28 stainfo_stable.db
-rw-rw-rw-    1         20.0K Feb  1 09:00 sys_detect.db
-rw-rw-rw-    1         20.0K Jan 31 11:28 sys_setting.db
-rw-rw-rw-    1         28.0K Feb  1 09:00 wifi_detect.db
-rw-rw-rw-    1         20.0K Jan 31 11:28 wifi_setting.db

 Free:
             total       used       free     shared    buffers     cached
Mem:        421864     284840     137024       2568      11612      31300
-/+ buffers/cache:     241928     179936
Swap:      2097148          0    2097148

********************************************************************************
********************************************************************************

And the most recent:

********************************************************************************
*****                      Running Clean Dbs                               *****
********************************************************************************
wifi_detect.db count before delete
36
wifi_detect.db count after delete
30
stainfo.db count before delete
210
stainfo.db count after delete
175
eth_detect.db count before delete
34
eth_detect.db count after delete
30
net_detect.db count before delete
6
net_detect.db count after delete
5
sys_detect.db count before delete
6
sys_detect.db count after delete
5
 File Sizes:
-rw-rw-rw-    1         20.0K Jan 31 11:28 channel_change.db
-rw-rw-rw-    1         20.0K Feb  1 18:31 eth_detect.db
-rw-rw-rw-    1         20.0K Feb  1 18:31 net_detect.db
-rw-rw-rw-    1         20.0K Feb  1 12:29 port_status_change.db
-rw-rw-rw-    1         20.0K Jan 31 11:28 port_status_usb_change.db
-rw-rw-rw-    1         56.0K Feb  1 18:31 stainfo.db
-rw-rw-rw-    1         20.0K Jan 31 16:28 stainfo_stable.db
-rw-rw-rw-    1         20.0K Feb  1 18:31 sys_detect.db
-rw-rw-rw-    1         20.0K Jan 31 11:28 sys_setting.db
-rw-rw-rw-    1         28.0K Feb  1 18:31 wifi_detect.db
-rw-rw-rw-    1         20.0K Jan 31 11:28 wifi_setting.db

 Free:
             total       used       free     shared    buffers     cached
Mem:        421864     294064     127800       2604      17988      33556
-/+ buffers/cache:     242520     179344
Swap:      2097148          0    2097148

********************************************************************************
********************************************************************************

 

[jata]

Thanks @lamentary - really helpful. Any chance you can repost your script and config instructions please?

I find that RAM often gets above 80% on the Asus routers I have used (AC68U, AX58U and AX86S) so I don't think it is a problem - the system manages memory fine.

What I think is different in this case/issue is that tmpfs is slowly using all the free RAM as storage that can't be released - and this causes the file storage and OOM errors.

I had a power cut today so everything is reset and router RAM at 70% (normal after reboot) so I have a couple of weeks without having to do anything

 

[lamentary]

I ended up going a slightly different route.

Without specifically knowing what conn_diag is used for, instead of killing it altogether, I wrote a shell script (in my rudimentary fashion) that deletes entries older than 5 mins from the files that mmacedo mentioned previously, and set a cron job to run it every minute (I wanted to keep them separate in case I wanted to run it manually).

I had noted that my various db tables are growing at different rates than mmacedo's were, particularly the stainfo.db, but the cron job/sql combos seem to have abated a lot of the nastiness of growth. I'll post more here after some time has passed.

...

I made both executable and have been monitoring my file sizes:


Thus far, my file sizes have been remaining steady. If this holds true for a few days, I will likely extend the time for records in their respective files prior to deleting them.

Sure, @jata, here is the original post. I am not strong with the Bourne shell, and there are probably a million ways to write this script better.

I've modified the clean-dbs.sh script slightly since then to generate the output, provide some additional metrics, and cleaned up the cron assignment in init-start. I'll provide a console-only output and a file output version of the clean-dbs script below so you can take whichever you like more.

I updated the /jffs/scripts/init-start script (you may have to create one if you don't already have it), and created a script in /jffs/scripts/clean-dbs.sh.

init-start

#!/bin/sh

### Cron job to delete conn_diag .db entries every minute
cru a cleanConnDiagDbs "* * * * * /bin/sh /jffs/scripts/clean-dbs.sh"

clean-dbs.sh (output to console only - output is only useful for manual runs of the script):

#!/bin/sh

echo ""
echo "********************************************************************************"
echo "*****                      Running Clean Dbs                               *****"
echo "********************************************************************************"

echo ""

### WIFI detect
echo "wifi_detect.db count before delete"
/usr/sbin/sqlite3 /tmp/.diag/wifi_detect.db 'select count() FROM DATA_INFO;'

/usr/sbin/sqlite3 /tmp/.diag/wifi_detect.db 'delete from DATA_INFO WHERE strftime("%s","now") - data_time > 300;'

echo "wifi_detect.db count after delete"
/usr/sbin/sqlite3 /tmp/.diag/wifi_detect.db 'select count() FROM DATA_INFO;'

### STA info
echo "stainfo.db count before delete"
/usr/sbin/sqlite3 /tmp/.diag/stainfo.db 'select count() FROM DATA_INFO;'

/usr/sbin/sqlite3 /tmp/.diag/stainfo.db 'delete from DATA_INFO WHERE strftime("%s","now") - data_time > 300;'

echo "stainfo.db count after delete"
/usr/sbin/sqlite3 /tmp/.diag/stainfo.db 'select count() FROM DATA_INFO;'

### ETH detect
echo "eth_detect.db count before delete"
/usr/sbin/sqlite3 /tmp/.diag/eth_detect.db 'select count() FROM DATA_INFO;'

/usr/sbin/sqlite3 /tmp/.diag/eth_detect.db 'delete from DATA_INFO WHERE strftime("%s","now") - data_time > 300;'

echo "eth_detect.db count after delete"
/usr/sbin/sqlite3 /tmp/.diag/eth_detect.db 'select count() FROM DATA_INFO;'

### NET detect
echo "net_detect.db count before delete"
/usr/sbin/sqlite3 /tmp/.diag/net_detect.db 'select count() FROM DATA_INFO;'

/usr/sbin/sqlite3 /tmp/.diag/net_detect.db 'delete from DATA_INFO WHERE strftime("%s","now") - data_time > 300;'

echo "net_detect.db count after delete"
/usr/sbin/sqlite3 /tmp/.diag/net_detect.db 'select count() FROM DATA_INFO;'

### SYS detect
echo "sys_detect.db count before delete"
/usr/sbin/sqlite3 /tmp/.diag/sys_detect.db 'select count() FROM DATA_INFO;'

/usr/sbin/sqlite3 /tmp/.diag/sys_detect.db 'delete from DATA_INFO WHERE strftime("%s","now") - data_time > 300;'

echo "sys_detect.db count after delete"
/usr/sbin/sqlite3 /tmp/.diag/sys_detect.db 'select count() FROM DATA_INFO;'

echo " File Sizes:"
ls -lh /tmp/.diag

echo ""
echo " Free:"
free
echo ""
echo "********************************************************************************"
echo "********************************************************************************"
echo ""

Or, you can run the clean-dbs.sh script with output to a file (say in your /jffs or /mnt directory) - but you'll want to monitor its size and respond appropriately; there's no sense in fixing a RAM issue only to cause a storage issue (replace the LOG_FILE assignment in the second line with the path your want to save the log to):

#!/bin/sh
LOG_FILE="<path-to-your-desired-output-file-location>"

echo "" >> $LOG_FILE
echo "********************************************************************************" >> $LOG_FILE
echo "***** Running Clean Dbs *****" >> $LOG_FILE
echo "********************************************************************************" >> $LOG_FILE

echo ""

### WIFI detect
echo "wifi_detect.db count before delete" >> $LOG_FILE
/usr/sbin/sqlite3 /tmp/.diag/wifi_detect.db 'select count() FROM DATA_INFO;' >> $LOG_FILE

/usr/sbin/sqlite3 /tmp/.diag/wifi_detect.db 'delete from DATA_INFO WHERE strftime("%s","now") - data_time > 300;'

echo "wifi_detect.db count after delete" >> $LOG_FILE
/usr/sbin/sqlite3 /tmp/.diag/wifi_detect.db 'select count() FROM DATA_INFO;' >> $LOG_FILE

### STA info
echo "stainfo.db count before delete" >> $LOG_FILE
/usr/sbin/sqlite3 /tmp/.diag/stainfo.db 'select count() FROM DATA_INFO;' >> $LOG_FILE

/usr/sbin/sqlite3 /tmp/.diag/stainfo.db 'delete from DATA_INFO WHERE strftime("%s","now") - data_time > 300;'

echo "stainfo.db count after delete" >> $LOG_FILE
/usr/sbin/sqlite3 /tmp/.diag/stainfo.db 'select count() FROM DATA_INFO;' >> $LOG_FILE

### ETH detect
echo "eth_detect.db count before delete" >> $LOG_FILE
/usr/sbin/sqlite3 /tmp/.diag/eth_detect.db 'select count() FROM DATA_INFO;' >> $LOG_FILE

/usr/sbin/sqlite3 /tmp/.diag/eth_detect.db 'delete from DATA_INFO WHERE strftime("%s","now") - data_time > 300;'

echo "eth_detect.db count after delete" >> $LOG_FILE
/usr/sbin/sqlite3 /tmp/.diag/eth_detect.db 'select count() FROM DATA_INFO;' >> $LOG_FILE

### NET detect
echo "net_detect.db count before delete" >> $LOG_FILE
/usr/sbin/sqlite3 /tmp/.diag/net_detect.db 'select count() FROM DATA_INFO;' >> $LOG_FILE

/usr/sbin/sqlite3 /tmp/.diag/net_detect.db 'delete from DATA_INFO WHERE strftime("%s","now") - data_time > 300;'

echo "net_detect.db count after delete" >> $LOG_FILE
/usr/sbin/sqlite3 /tmp/.diag/net_detect.db 'select count() FROM DATA_INFO;' >> $LOG_FILE

### SYS detect
echo "sys_detect.db count before delete" >> $LOG_FILE
/usr/sbin/sqlite3 /tmp/.diag/sys_detect.db 'select count() FROM DATA_INFO;' >> $LOG_FILE

/usr/sbin/sqlite3 /tmp/.diag/sys_detect.db 'delete from DATA_INFO WHERE strftime("%s","now") - data_time > 300;'

echo "sys_detect.db count after delete" >> $LOG_FILE
/usr/sbin/sqlite3 /tmp/.diag/sys_detect.db 'select count() FROM DATA_INFO;' >> $LOG_FILE

echo " File Sizes:" >> $LOG_FILE
ls -lh /tmp/.diag >> $LOG_FILE

echo "" >> $LOG_FILE
echo " Free:" >> $LOG_FILE
free >> $LOG_FILE
echo "" >> $LOG_FILE
echo "********************************************************************************" >> $LOG_FILE
echo "********************************************************************************" >> $LOG_FILE
echo "" >> $LOG_FILE

 

[jata]

Sure, @jata, here is the original post. I am not strong with the Bourne shell, and there are probably a million ways to write this script better.

I've modified the clean-dbs.sh script slightly since then to generate the output, provide some additional metrics, and cleaned up the cron assignment in init-start. I'll provide a console-only output and a file output version of the clean-dbs script below so you can take whichever you like more.

I updated the /jffs/scripts/init-start script (you may have to create one if you don't already have it), and created a script in /jffs/scripts/clean-dbs.sh.

init-start

#!/bin/sh

### Cron job to delete conn_diag .db entries every minute
cru a cleanConnDiagDbs "* * * * * /bin/sh /jffs/scripts/clean-dbs.sh"

clean-dbs.sh (output to console only - output is only useful for manual runs of the script):

#!/bin/sh

echo ""
echo "********************************************************************************"
echo "*****                      Running Clean Dbs                               *****"
echo "********************************************************************************"

echo ""

### WIFI detect
echo "wifi_detect.db count before delete"
/usr/sbin/sqlite3 /tmp/.diag/wifi_detect.db 'select count() FROM DATA_INFO;'

/usr/sbin/sqlite3 /tmp/.diag/wifi_detect.db 'delete from DATA_INFO WHERE strftime("%s","now") - data_time > 300;'

echo "wifi_detect.db count after delete"
/usr/sbin/sqlite3 /tmp/.diag/wifi_detect.db 'select count() FROM DATA_INFO;'

### STA info
echo "stainfo.db count before delete"
/usr/sbin/sqlite3 /tmp/.diag/stainfo.db 'select count() FROM DATA_INFO;'

/usr/sbin/sqlite3 /tmp/.diag/stainfo.db 'delete from DATA_INFO WHERE strftime("%s","now") - data_time > 300;'

echo "stainfo.db count after delete"
/usr/sbin/sqlite3 /tmp/.diag/stainfo.db 'select count() FROM DATA_INFO;'

### ETH detect
echo "eth_detect.db count before delete"
/usr/sbin/sqlite3 /tmp/.diag/eth_detect.db 'select count() FROM DATA_INFO;'

/usr/sbin/sqlite3 /tmp/.diag/eth_detect.db 'delete from DATA_INFO WHERE strftime("%s","now") - data_time > 300;'

echo "eth_detect.db count after delete"
/usr/sbin/sqlite3 /tmp/.diag/eth_detect.db 'select count() FROM DATA_INFO;'

### NET detect
echo "net_detect.db count before delete"
/usr/sbin/sqlite3 /tmp/.diag/net_detect.db 'select count() FROM DATA_INFO;'

/usr/sbin/sqlite3 /tmp/.diag/net_detect.db 'delete from DATA_INFO WHERE strftime("%s","now") - data_time > 300;'

echo "net_detect.db count after delete"
/usr/sbin/sqlite3 /tmp/.diag/net_detect.db 'select count() FROM DATA_INFO;'

### SYS detect
echo "sys_detect.db count before delete"
/usr/sbin/sqlite3 /tmp/.diag/sys_detect.db 'select count() FROM DATA_INFO;'

/usr/sbin/sqlite3 /tmp/.diag/sys_detect.db 'delete from DATA_INFO WHERE strftime("%s","now") - data_time > 300;'

echo "sys_detect.db count after delete"
/usr/sbin/sqlite3 /tmp/.diag/sys_detect.db 'select count() FROM DATA_INFO;'

echo " File Sizes:"
ls -lh /tmp/.diag

echo ""
echo " Free:"
free
echo ""
echo "********************************************************************************"
echo "********************************************************************************"
echo ""

Or, you can run the clean-dbs.sh script with output to a file (say in your /jffs or /mnt directory) - but you'll want to monitor its size and respond appropriately; there's no sense in fixing a RAM issue only to cause a storage issue (replace the LOG_FILE assignment in the second line with the path your want to save the log to):

#!/bin/sh
LOG_FILE="<path-to-your-desired-output-file-location>"

echo "" >> $LOG_FILE
echo "********************************************************************************" >> $LOG_FILE
echo "***** Running Clean Dbs *****" >> $LOG_FILE
echo "********************************************************************************" >> $LOG_FILE

echo ""

### WIFI detect
echo "wifi_detect.db count before delete" >> $LOG_FILE
/usr/sbin/sqlite3 /tmp/.diag/wifi_detect.db 'select count() FROM DATA_INFO;' >> $LOG_FILE

/usr/sbin/sqlite3 /tmp/.diag/wifi_detect.db 'delete from DATA_INFO WHERE strftime("%s","now") - data_time > 300;'

echo "wifi_detect.db count after delete" >> $LOG_FILE
/usr/sbin/sqlite3 /tmp/.diag/wifi_detect.db 'select count() FROM DATA_INFO;' >> $LOG_FILE

### STA info
echo "stainfo.db count before delete" >> $LOG_FILE
/usr/sbin/sqlite3 /tmp/.diag/stainfo.db 'select count() FROM DATA_INFO;' >> $LOG_FILE

/usr/sbin/sqlite3 /tmp/.diag/stainfo.db 'delete from DATA_INFO WHERE strftime("%s","now") - data_time > 300;'

echo "stainfo.db count after delete" >> $LOG_FILE
/usr/sbin/sqlite3 /tmp/.diag/stainfo.db 'select count() FROM DATA_INFO;' >> $LOG_FILE

### ETH detect
echo "eth_detect.db count before delete" >> $LOG_FILE
/usr/sbin/sqlite3 /tmp/.diag/eth_detect.db 'select count() FROM DATA_INFO;' >> $LOG_FILE

/usr/sbin/sqlite3 /tmp/.diag/eth_detect.db 'delete from DATA_INFO WHERE strftime("%s","now") - data_time > 300;'

echo "eth_detect.db count after delete" >> $LOG_FILE
/usr/sbin/sqlite3 /tmp/.diag/eth_detect.db 'select count() FROM DATA_INFO;' >> $LOG_FILE

### NET detect
echo "net_detect.db count before delete" >> $LOG_FILE
/usr/sbin/sqlite3 /tmp/.diag/net_detect.db 'select count() FROM DATA_INFO;' >> $LOG_FILE

/usr/sbin/sqlite3 /tmp/.diag/net_detect.db 'delete from DATA_INFO WHERE strftime("%s","now") - data_time > 300;'

echo "net_detect.db count after delete" >> $LOG_FILE
/usr/sbin/sqlite3 /tmp/.diag/net_detect.db 'select count() FROM DATA_INFO;' >> $LOG_FILE

### SYS detect
echo "sys_detect.db count before delete" >> $LOG_FILE
/usr/sbin/sqlite3 /tmp/.diag/sys_detect.db 'select count() FROM DATA_INFO;' >> $LOG_FILE

/usr/sbin/sqlite3 /tmp/.diag/sys_detect.db 'delete from DATA_INFO WHERE strftime("%s","now") - data_time > 300;'

echo "sys_detect.db count after delete" >> $LOG_FILE
/usr/sbin/sqlite3 /tmp/.diag/sys_detect.db 'select count() FROM DATA_INFO;' >> $LOG_FILE

echo " File Sizes:" >> $LOG_FILE
ls -lh /tmp/.diag >> $LOG_FILE

echo "" >> $LOG_FILE
echo " Free:" >> $LOG_FILE
free >> $LOG_FILE
echo "" >> $LOG_FILE
echo "********************************************************************************" >> $LOG_FILE
echo "********************************************************************************" >> $LOG_FILE
echo "" >> $LOG_FILE

Thanks. I will have a play around with this. Just run it from console when ram usage gets too high

 

[lamentary]

Thanks. I will have a play around with this. Just run it from console when ram usage gets too high

No problem. If you're just planning on running it on demand from the console, though, as opposed to scheduling it via cron job, it might be just as easy to kill the conn_diag process and delete the contents of the /tmp/.diag directory and then optionally restart the conn_diag process, unless you're particularly interested in the record count of the .db files before and after.

I have a USB mounted, so it's no big deal for me to output the results there and this allows me to hold on to the last 5 minutes worth of conn_diag data at all times, so I schedule it to run and am keeping track of its results.

ETA: Also worth noting: deleting records from SQL tables does not shrink the file size. This is actually a feature, because SQL assumes that if it needed a size of x bytes before, it will possibly need it again, and file growth operations are expensive. Running this on occasion only will not make the .db file usage (and therefore RAM usage) any smaller, so you will need to manage those .db files manually anyway.

FWIW, here's the latest output (I've since added the output of the tmpfs state to the script):

********************************************************************************
*****                      Running Clean Dbs                               *****
********************************************************************************
wifi_detect.db count before delete
36
wifi_detect.db count after delete
30
stainfo.db count before delete
216
stainfo.db count after delete
180
eth_detect.db count before delete
36
eth_detect.db count after delete
30
net_detect.db count before delete
6
net_detect.db count after delete
5
sys_detect.db count before delete
6
sys_detect.db count after delete
5
File Sizes:
-rw-rw-rw-    1        20.0K Jan 31 11:28 channel_change.db
-rw-rw-rw-    1        20.0K Feb  2 10:35 eth_detect.db
-rw-rw-rw-    1        20.0K Feb  2 10:35 net_detect.db
-rw-rw-rw-    1        20.0K Feb  2 04:01 port_status_change.db
-rw-rw-rw-    1        20.0K Jan 31 11:28 port_status_usb_change.db
-rw-rw-rw-    1        56.0K Feb  2 10:35 stainfo.db
-rw-rw-rw-    1        20.0K Feb  1 18:52 stainfo_stable.db
-rw-rw-rw-    1        20.0K Feb  2 10:35 sys_detect.db
-rw-rw-rw-    1        20.0K Jan 31 11:28 sys_setting.db
-rw-rw-rw-    1        28.0K Feb  2 10:35 wifi_detect.db
-rw-rw-rw-    1        20.0K Jan 31 11:28 wifi_setting.db

TMPFS Sizes:
Filesystem                Size      Used Available Use% Mounted on
tmpfs                   206.0M      1.0M    204.9M   1% /var
tmpfs                   206.0M      1.5M    204.5M   1% /tmp/mnt
tmpfs                   206.0M      1.5M    204.5M   1% /tmp/mnt
tmpfs                   206.0M      1.5M    204.5M   1% /tmp

Free:
             total       used       free     shared    buffers     cached
Mem:        421864     321972      99892       2768      29792      36612
-/+ buffers/cache:     255568     166296
Swap:      2097148          0    2097148

********************************************************************************
********************************************************************************