uiDivStats uiDivstats 4.0.5 - WebUI Stats for Diversion, December 15, 2024

[Aziron5]

Hi

Reporting that uiDivStats is heavily spamming syslog with messages about "No Entware Found" in that case, a wall of text in one day, hundreds of messages.
This kind of message should be the FW's responsibility IMO, goes in line with my past suggestion that FW should support a mode to enforce Entware for full operations, otherwise disallow WAN and partially restrict other functionality, and in that case keep warning in logs and GUI about missing USB disk / entware, but ofcourse with reasonable frequency.

Nov 22 00:00:50 uiDivStats: Entware not found, sleeping for 10s (attempt 6 of 10)
Nov 22 00:01:00 uiDivStats: Lock file found (age: 60 seconds) - statistic generation likely currently in progress
Nov 22 00:01:00 uiDivStats: Entware not found, sleeping for 10s (attempt 7 of 10)
Nov 22 00:01:10 uiDivStats: Entware not found, sleeping for 10s (attempt 8 of 10)
Nov 22 00:01:20 uiDivStats: Entware not found, sleeping for 10s (attempt 9 of 10)
Nov 22 00:01:30 uiDivStats: Entware not found, sleeping for 10s (attempt 10 of 10)
Nov 22 00:01:40 uiDivStats: Entware not found and is required for uiDivStats to run, please resolve
Nov 22 00:02:00 uiDivStats: Entware not found, sleeping for 10s (attempt 1 of 10)
Nov 22 00:02:10 uiDivStats: Entware not found, sleeping for 10s (attempt 2 of 10)
Nov 22 00:02:20 uiDivStats: Entware not found, sleeping for 10s (attempt 3 of 10)
Nov 22 00:02:30 uiDivStats: Entware not found, sleeping for 10s (attempt 4 of 10)
Nov 22 00:02:40 uiDivStats: Entware not found, sleeping for 10s (attempt 5 of 10)
Nov 22 00:02:50 uiDivStats: Entware not found, sleeping for 10s (attempt 6 of 10)
Nov 22 00:03:00 uiDivStats: Entware not found, sleeping for 10s (attempt 7 of 10)
Nov 22 00:03:10 uiDivStats: Entware not found, sleeping for 10s (attempt 8 of 10)
Nov 22 00:03:20 uiDivStats: Entware not found, sleeping for 10s (attempt 9 of 10)
Nov 22 00:03:30 uiDivStats: Entware not found, sleeping for 10s (attempt 10 of 10)
Nov 22 00:03:40 uiDivStats: Entware not found and is required for uiDivStats to run, please resolve
Nov 22 00:04:00 uiDivStats: Entware not found, sleeping for 10s (attempt 1 of 10)
Nov 22 00:04:00 uiDivStats: Entware not found, sleeping for 10s (attempt 1 of 10)
Nov 22 00:04:10 uiDivStats: Entware not found, sleeping for 10s (attempt 2 of 10)
Nov 22 00:04:10 uiDivStats: Entware not found, sleeping for 10s (attempt 2 of 10)
Nov 22 00:04:20 uiDivStats: Entware not found, sleeping for 10s (attempt 3 of 10)
Nov 22 00:04:20 uiDivStats: Entware not found, sleeping for 10s (attempt 3 of 10)
Nov 22 00:04:30 uiDivStats: Entware not found, sleeping for 10s (attempt 4 of 10)
Nov 22 00:04:30 uiDivStats: Entware not found, sleeping for 10s (attempt 4 of 10)
Nov 22 00:04:40 uiDivStats: Entware not found, sleeping for 10s (attempt 5 of 10)
Nov 22 00:04:40 uiDivStats: Entware not found, sleeping for 10s (attempt 5 of 10)
Nov 22 00:04:50 uiDivStats: Entware not found, sleeping for 10s (attempt 6 of 10)
Nov 22 00:04:50 uiDivStats: Entware not found, sleeping for 10s (attempt 6 of 10)
Nov 22 00:05:00 uiDivStats: Entware not found, sleeping for 10s (attempt 7 of 10)
Nov 22 00:05:00 uiDivStats: Entware not found, sleeping for 10s (attempt 7 of 10)
Nov 22 00:05:10 uiDivStats: Entware not found, sleeping for 10s (attempt 8 of 10)
Nov 22 00:05:10 uiDivStats: Entware not found, sleeping for 10s (attempt 8 of 10)
Nov 22 00:05:20 uiDivStats: Entware not found, sleeping for 10s (attempt 9 of 10)
Nov 22 00:05:20 uiDivStats: Entware not found, sleeping for 10s (attempt 9 of 10)
Nov 22 00:05:30 uiDivStats: Entware not found, sleeping for 10s (attempt 10 of 10)
Nov 22 00:05:30 uiDivStats: Entware not found, sleeping for 10s (attempt 10 of 10)
Nov 22 00:05:40 uiDivStats: Entware not found and is required for uiDivStats to run, please resolve
Nov 22 00:05:40 uiDivStats: Entware not found and is required for uiDivStats to run, please resolve
Nov 22 00:06:00 uiDivStats: Entware not found, sleeping for 10s (attempt 1 of 10)
Nov 22 00:06:10 uiDivStats: Entware not found, sleeping for 10s (attempt 2 of 10)
Nov 22 00:06:20 uiDivStats: Entware not found, sleeping for 10s (attempt 3 of 10)
Nov 22 00:06:30 uiDivStats: Entware not found, sleeping for 10s (attempt 4 of 10)
Nov 22 00:06:40 uiDivStats: Entware not found, sleeping for 10s (attempt 5 of 10)
Nov 22 00:06:50 uiDivStats: Entware not found, sleeping for 10s (attempt 6 of 10)
Nov 22 00:07:00 uiDivStats: Entware not found, sleeping for 10s (attempt 7 of 10)
Nov 22 00:07:10 uiDivStats: Entware not found, sleeping for 10s (attempt 8 of 10)
Nov 22 00:07:20 uiDivStats: Entware not found, sleeping for 10s (attempt 9 of 10)
Nov 22 00:07:30 uiDivStats: Entware not found, sleeping for 10s (attempt 10 of 10)
Nov 22 00:07:40 uiDivStats: Entware not found and is required for uiDivStats to run, please resolve
Nov 22 00:08:00 uiDivStats: Entware not found, sleeping for 10s (attempt 1 of 10)
Nov 22 00:08:10 uiDivStats: Entware not found, sleeping for 10s (attempt 2 of 10)
Nov 22 00:08:20 uiDivStats: Entware not found, sleeping for 10s (attempt 3 of 10)
Nov 22 00:08:30 uiDivStats: Entware not found, sleeping for 10s (attempt 4 of 10)
Nov 22 00:08:40 uiDivStats: Entware not found, sleeping for 10s (attempt 5 of 10)
Nov 22 00:08:50 uiDivStats: Entware not found, sleeping for 10s (attempt 6 of 10)
Nov 22 00:09:00 uiDivStats: Lock file found (age: 60 seconds) - statistic generation likely currently in progress
Nov 22 00:09:00 uiDivStats: Entware not found, sleeping for 10s (attempt 7 of 10)
Nov 22 00:09:10 uiDivStats: Entware not found, sleeping for 10s (attempt 8 of 10)
Nov 22 00:09:20 uiDivStats: Entware not found, sleeping for 10s (attempt 9 of 10)
Nov 22 00:09:30 uiDivStats: Entware not found, sleeping for 10s (attempt 10 of 10)
Nov 22 00:09:40 uiDivStats: Entware not found and is required for uiDivStats to run, please resolve
Nov 22 00:10:00 uiDivStats: Entware not found, sleeping for 10s (attempt 1 of 10)
Nov 22 00:10:10 uiDivStats: Entware not found, sleeping for 10s (attempt 2 of 10)
Nov 22 00:10:20 uiDivStats: Entware not found, sleeping for 10s (attempt 3 of 10)
Nov 22 00:10:30 uiDivStats: Entware not found, sleeping for 10s (attempt 4 of 10)
Nov 22 00:10:40 uiDivStats: Entware not found, sleeping for 10s (attempt 5 of 10)
Nov 22 00:10:50 uiDivStats: Entware not found, sleeping for 10s (attempt 6 of 10)
Nov 22 00:11:00 uiDivStats: Entware not found, sleeping for 10s (attempt 7 of 10)
Nov 22 00:11:10 uiDivStats: Entware not found, sleeping for 10s (attempt 8 of 10)
Nov 22 00:11:20 uiDivStats: Entware not found, sleeping for 10s (attempt 9 of 10)
Nov 22 00:11:30 uiDivStats: Entware not found, sleeping for 10s (attempt 10 of 10)
Nov 22 00:11:40 uiDivStats: Entware not found and is required for uiDivStats to run, please resolve
Nov 22 00:12:00 uiDivStats: Entware not found, sleeping for 10s (attempt 1 of 10)
Nov 22 00:12:10 uiDivStats: Entware not found, sleeping for 10s (attempt 2 of 10)
Nov 22 00:12:20 uiDivStats: Entware not found, sleeping for 10s (attempt 3 of 10)
Nov 22 00:12:30 uiDivStats: Entware not found, sleeping for 10s (attempt 4 of 10)
Nov 22 00:12:40 uiDivStats: Entware not found, sleeping for 10s (attempt 5 of 10)
Nov 22 00:12:50 uiDivStats: Entware not found, sleeping for 10s (attempt 6 of 10)
Nov 22 00:13:00 uiDivStats: Entware not found, sleeping for 10s (attempt 7 of 10)
Nov 22 00:13:10 uiDivStats: Entware not found, sleeping for 10s (attempt 8 of 10)
Nov 22 00:13:20 uiDivStats: Entware not found, sleeping for 10s (attempt 9 of 10)
Nov 22 00:13:30 uiDivStats: Entware not found, sleeping for 10s (attempt 10 of 10)
Nov 22 00:13:40 uiDivStats: Entware not found and is required for uiDivStats to run, please resolve
Nov 22 00:14:00 uiDivStats: Entware not found, sleeping for 10s (attempt 1 of 10)
Nov 22 00:14:00 uiDivStats: Entware not found, sleeping for 10s (attempt 1 of 10)
Nov 22 00:14:10 uiDivStats: Entware not found, sleeping for 10s (attempt 2 of 10)
Nov 22 00:14:10 uiDivStats: Entware not found, sleeping for 10s (attempt 2 of 10)
Nov 22 00:14:20 uiDivStats: Entware not found, sleeping for 10s (attempt 3 of 10)
Nov 22 00:14:20 uiDivStats: Entware not found, sleeping for 10s (attempt 3 of 10)
Nov 22 00:14:30 uiDivStats: Entware not found, sleeping for 10s (attempt 4 of 10)
Nov 22 00:14:30 uiDivStats: Entware not found, sleeping for 10s (attempt 4 of 10)
Nov 22 00:14:40 uiDivStats: Entware not found, sleeping for 10s (attempt 5 of 10)
Nov 22 00:14:40 uiDivStats: Entware not found, sleeping for 10s (attempt 5 of 10)
Nov 22 00:14:50 uiDivStats: Entware not found, sleeping for 10s (attempt 6 of 10)
Nov 22 00:14:50 uiDivStats: Entware not found, sleeping for 10s (attempt 6 of 10)
Nov 22 00:15:00 uiDivStats: Entware not found, sleeping for 10s (attempt 7 of 10)
Nov 22 00:15:00 uiDivStats: Entware not found, sleeping for 10s (attempt 7 of 10)
Nov 22 00:15:10 uiDivStats: Entware not found, sleeping for 10s (attempt 8 of 10)
Nov 22 00:15:10 uiDivStats: Entware not found, sleeping for 10s (attempt 8 of 10)
Nov 22 00:15:20 uiDivStats: Entware not found, sleeping for 10s (attempt 9 of 10)
Nov 22 00:15:20 uiDivStats: Entware not found, sleeping for 10s (attempt 9 of 10)

Also SNBF forum limitation of 10k characters per post where we have to post relatively large sections of code is kinda way too strict imo, also speaking for earlier post I made where I had to put it in an attachement.

 

[Ripshod]

Hi

Reporting that uiDivStats is heavily spamming syslog with messages about "No Entware Found" in that case, a wall of text in one day, hundreds of messages.
This kind of message should be the FW's responsibility IMO, goes in line with my past suggestion that FW should support a mode to enforce Entware for full operations, otherwise disallow WAN and partially restrict other functionality, and in that case keep warning in logs and GUI about missing USB disk / entware, but ofcourse with reasonable frequency.

Strange you should be seeing that. I'm also being spammed by usb related messages:

Nov 23 04:01:03 ripshod kernel: Write-error on swap-device (8:0:4072864)
Nov 23 04:01:03 ripshod kernel: Write-error on swap-device (8:0:4079864)
Nov 23 04:01:03 ripshod kernel: Write-error on swap-device (8:0:4082264)
Nov 23 04:01:03 ripshod kernel: Write-error on swap-device (8:0:4082272)
Nov 23 04:01:03 ripshod kernel: Write-error on swap-device (8:0:4082280)
Nov 23 04:01:03 ripshod kernel: Write-error on swap-device (8:0:4072872)
Nov 23 04:01:03 ripshod kernel: Write-error on swap-device (8:0:4079872)
Nov 23 04:01:03 ripshod kernel: Write-error on swap-device (8:0:4082288)
Nov 23 04:01:03 ripshod kernel: Write-error on swap-device (8:0:4082296)
Nov 23 04:01:03 ripshod kernel: Write-error on swap-device (8:0:4072880)
Nov 23 04:01:03 ripshod kernel: Write-error on swap-device (8:0:4079880)
Nov 23 04:01:03 ripshod kernel: Write-error on swap-device (8:0:4075064)

Different message but similar potential cause. Gotta reboot.

*edit* reboot fixed it and no errors detected by the disk check script

 

[Aziron5]

Strange you should be seeing that. I'm also being spammed by usb related messages:

In my case Entware and the whole USB disk really isn't present (unmounted/unplugged), so it's technically correct, it's just too noisy about it.

EDIT: I think it just happens if you eject the disk mid session, probably because entware / diversion don't even start loading and won't trigger it on a fresh boot without disk.

 

[Martinski]

...
I am providing databases that can generate charts normally and those that cannot. Please assist in testing and confirm if the issue can be reproduced. Thank you!

In addition to the file size and the number of records, what do you believe is the problem with your ~76MB database file that's causing the charts not to be generated and/or displayed on the webGUI page? Any clues?

I'm asking because this evening I quickly checked on 3 separate ASUS routers that have the latest uiDivStats 4.0.3 installed and the database sizes are all currently > 90MB - one being close to 140MB. In each case, the webGUI page shows no problems with generating & displaying all the charts.

Below are some screenshots from a router with a 94.6MB database (containing 212,409 records) running perfectly fine, which indicates that database sizes > ~75MB (with a large number of records) alone are not the root cause of the problem you're experiencing.

IMO, it appears that something else is going on with your current setup, maybe related to how you have uiDivStats configured, or perhaps related to the amount of RAM available at the time - I simply don’t know. But you have not provided enough context & details about the problem to figure out where to begin looking for clues. Without detailed context & data points, it would be like going on a "wild goose chase" due to the lack of a clear direction.

 

[aru]

Thank you for your input and for sharing the screenshots. I appreciate your insights. Based on your observation, it's good to know that a database size of ~75MB or larger with many records isn't necessarily the issue.


As for my setup, I’ve already tried resetting the router and reinstalling the uiDivStats package, but the issue persists. When the database size is large, the charts fail to generate and are stuck at the 'Refreshing charts...' stage.

I attempted to execute it via CMD but did not notice any obvious errors.

admins@GT-AX6000-0FC8:/tmp/mnt/usbkey/myaddon# /jffs/scripts/uiDivStats generate
Starting stat update


arp: 2001:b011:5c0b:162f:ac3f:add2:c126:699: Unknown host
arp: 2001:b011:5c0b:162f:1177:137f:43b0:dab8: Unknown host
arp: 2001:b011:5c0b:162f:7037:88f7:b7bc:bf3c: Unknown host
arp: 2001:b011:5c0b:162f:f1bf:bae9:5ca3:bc32: Unknown host
arp: 2001:b011:5c0b:162f:e244:4e45:3920:a8d0: Unknown host
arp: fe80::4829:b621:77b5:c585: Unknown host
arp: 2001:b011:5c0b:162f:c55f:9865:2333:7349: Unknown host
arp: 2001:b011:5c0b:162f:b428:3fc3:7565:c439: Unknown host
arp: fe80::f8b3:c86d:c78b:1c90: Unknown host
arp: 2001:b011:5c0b:162f:79b1:7d98:2f49:3ed: Unknown host
arp: 2001:b011:5c0b:162f:6be0:34f5:ca43:49fe: Unknown host
arp: fe80::723a:5c26:39b3:400d: Unknown host
arp: 2001:b011:5c0b:162f:f6c0:1d50:33dc:9c9d: Unknown host
arp: 2001:b011:5c0b:162f:84f2:6adb:6979:a156: Unknown host
arp: ::1: Unknown host
arp: fe80::22:57f5:9668:6e: Unknown host
arp: 2001:b011:5c0b:162f:37fb:ce29:1842:dcca: Unknown host
arp: 2001:b011:5c0b:162f:cb71:7b7b:356e:39f6: Unknown host
arp: 2001:b011:5c0b:162f:90d5:9f36:923d:ff84: Unknown host
arp: 2001:b011:5c0b:162f:c800:d904:548f:d1c7: Unknown host
arp: 2001:b011:5c0b:162f:33ee:3c0e:ec9e:43d6: Unknown host
arp: 2001:b011:5c0b:162f:8dc1:1629:b649:b6e6: Unknown host
arp: 2001:b011:5c0b:162f:e76b:1ef6:f354:adfa: Unknown host
arp: 2001:b011:5c0b:162f:a116:b7c9:1e64:a86d: Unknown host
arp: 2001:b011:5c0b:162f:947e:c4:ad34:6a82: Unknown host
arp: 2001:b011:5c0b:162f:ca66:930c:f702:4643: Unknown host
arp: 2001:b011:5c0b:162f:d5f8:202e:6fc1:56bb: Unknown host
arp: 2001:b011:5c0b:162f:f78f:1608:4226:f1cf: Unknown host
arp: 2001:b011:5c0b:162f:69c7:2996:f996:caf3: Unknown host
arp: 2001:b011:5c0b:162f:8df4:e243:52ae:22b6: Unknown host
arp: 2001:b011:5c0b:162f:489d:50e9:ab24:da89: Unknown host
Stats updated successfully


admins@GT-AX6000-0FC8:/tmp/mnt/usbkey/myaddon#

However, when I clear the database, the manual update works fine and the charts are generated without any problems.

I suspect the issue might be related to how uiDivStats is configured or potentially a resource limitation, such as RAM. I haven't yet been able to pinpoint the exact cause, but I’ll try to gather more details and context for further investigation. Any additional suggestions on how to narrow down the issue would be greatly appreciated!

 

[Martinski]

...
As for my setup, I’ve already tried resetting the router and reinstalling the uiDivStats package, but the issue persists. When the database size is large, the charts fail to generate and are stuck at the 'Refreshing charts...' stage.
...
However, when I clear the database, the manual update works fine and the charts are generated without any problems.
...

I noticed you have the "Cache Mode" option set to ENABLED in your uiDivStats configuration. I'm wondering if the available space on your router's "tmpfs" filesystem becomes "too low" when your database reaches some large size (e.g. > 75MB).

Try setting "Cache Mode" to DISABLED, click on "Save" and then test again.

To clarify, the above is just a hypothesis so it may or may not work, but it does look like the problem you're having is somehow related to the database file size & memory and/or disk space resources available on your router at the time.

To check the current space available in "tmpfs" type the following command:

df -hT | grep -E "(^Filesystem|^tmpfs .* /tmp$)"

I suggest you run this command BEFORE you click on "Update stats" and then AFTER when the problem occurs to see if there is any significant difference. Also, take BEFORE & AFTER screenshots of the top of the webGUI page and show the "Key Stats" data available right before the 1st chart.

HTH

 

[shama_001]

@Martinski , @aru

Just my 2 cents on something never noticed prior 4.0.3: I spotted 2 times on my setup where the second event below was missing
uiDivStats: Starting stat update
uiDivStats: Stats updated successfully

So the update started, but "Stats updated successfully" never pops out. All chart were empty with no text in it.
My setup relies on SSD (Swap etc) and all is fine. Instead of acting on anything I waited the next hour and both the 2 events & charts were fine again. It may have been the same in the past with previous version, hard to tell. But twice since 4.0.3 for sure, I'm that kind of guy checking things multiple times a day.

Maybe totally not part of the problem discussed, but worth the words

 

[agbommarito]

I have a lot of these messages in the system log:

Dec 1 13:10:00 uiDivStats: Starting stat update
Dec 1 13:10:09 uiDivStats: Stats updated successfully
Dec 1 14:10:00 uiDivStats: Starting stat update
Dec 1 14:10:09 uiDivStats: Stats updated successfully
Dec 1 15:10:00 uiDivStats: Starting stat update
Dec 1 15:10:09 uiDivStats: Stats updated successfully
Dec 1 16:10:00 uiDivStats: Starting stat update
Dec 1 16:10:09 uiDivStats: Stats updated successfully
Dec 1 17:10:00 uiDivStats: Starting stat update
Dec 1 17:10:09 uiDivStats: Stats updated successfully

They are tagged with a "warning" level. Wouldn't it be more appropriate for them to be tagged as "info" or "notice". Then they could be easily filtered out.

 

[Martinski]

@Martinski , @aru

Just my 2 cents on something never noticed prior 4.0.3: I spotted 2 times on my setup where the second event below was missing
uiDivStats: Starting stat update
uiDivStats: Stats updated successfully

So the update started, but "Stats updated successfully" never pops out. All chart were empty with no text in it.
My setup relies on SSD (Swap etc) and all is fine. Instead of acting on anything I waited the next hour and both the 2 events & charts were fine again. It may have been the same in the past with previous version, hard to tell. But twice since 4.0.3 for sure, I'm that kind of guy checking things multiple times a day.

Maybe totally not part of the problem discussed, but worth the words

Currently, except when doing database record trimming & cleanup, all the calls to SQLite3 do not handle or report errors or failures in the system log. AFAICT, this was done by design in the original code by @Jack Yaz. So it's certainly possible that, in some rare circumstances, the calls to process statistical data & generate records are failing silently and, therefore, the database would not be updated as expected. I can modify all the SQLite3 cmd calls to report errors in the system log, similar to what I did in the latest version for the record trimming & cleanup calls. This way, if the database is not updated as expected, the system log would show if any SQLite3 errors were reported.

 

[Martinski]

I have a lot of these messages in the system log:

Dec 1 13:10:00 uiDivStats: Starting stat update
Dec 1 13:10:09 uiDivStats: Stats updated successfully
Dec 1 14:10:00 uiDivStats: Starting stat update
Dec 1 14:10:09 uiDivStats: Stats updated successfully
Dec 1 15:10:00 uiDivStats: Starting stat update
Dec 1 15:10:09 uiDivStats: Stats updated successfully
Dec 1 16:10:00 uiDivStats: Starting stat update
Dec 1 16:10:09 uiDivStats: Stats updated successfully
Dec 1 17:10:00 uiDivStats: Starting stat update
Dec 1 17:10:09 uiDivStats: Stats updated successfully

They are tagged with a "warning" level. Wouldn't it be more appropriate for them to be tagged as "info" or "notice". Then they could be easily filtered out.

Well, I read your post twice and I really have no idea what you're actually talking about. What specifically are you referring to here? Where is this "tag" and what exactly is "being tagged"? Can you show these "warning level tags"? And, just as important, what or who is doing the "tagging"?

One suggestion: when posting about something that you see happened or is happening, and which you believe is somehow wrong, don’t just say it or describe it; in addition, attach a screenshot that clearly shows what you're talking about to help readers better understand what you’re trying to describe in your post.

"A picture is worth a thousand words" is an old & overused adage, but it works well when applied judiciously and under the right circumstances.

Just my 1.5 cents

 

[Martinski]

Hi

Reporting that uiDivStats is heavily spamming syslog with messages about "No Entware Found" in that case, a wall of text in one day, hundreds of messages.

In my case Entware and the whole USB disk really isn't present (unmounted/unplugged), so it's technically correct, it's just too noisy about it.

Keep in mind that when uiDivStats is installed, it automatically creates a daemon process and sets up 4 separate cron jobs, all of which run at different time intervals (some within minutes of each other). This was designed by @JackYaz to process Diversion statistics at regular intervals, generate database records, update CSV files with the daily, weekly & monthly statistical data, and perform a daily "cleanup" based on the number of days the user has chosen to keep the data (e.g. 30 days).

The key point here is that when uiDivStats is functioning as designed, separate instances of the script get launched at regular intervals to perform a specific & required task, so when you essentially "pull the rug from underneath" the working environment setup by removing the USB-attached drive where Entware & Diversion are installed, you are effectively disrupting & interfering with normal operations and with the processing of data that's expected to occur, so for each instance that gets executed, the script has no choice but to generate log msgs informing the user that "something wrong" is preventing it from doing its job.

Under the current design, the idea is that users want to be informed at every execution when uiDivStats is not able to perform its job so that whatever the root cause is can be resolved by the user as soon as possible.

Having said all that, I can also imagine some scenarios where, for some valid reason, users may need to *temporarily* stop all the background processing performed by the uiDivStats cron jobs. For such scenarios, I think it would be useful to have an option in the script CLI menu to "DISABLE" all the background processes which, in turn, will prevent the log messages from being generated.

Then, sometime later on when the USB-attached drive is again plugged in & functioning normally with Entware & Diversion, the script can be manually executed to reestablish the expected working environment setup & re-enable the required background processing of Diversion statistics.

Note that while the uiDivStats background processes are DISABLED, every time you execute the script manually, it would be operating in a heavily "constrained mode" if the USB drive with Entware is *not* found, but the idea is that this would be ONLY a *TEMPORARY* state until the USB drive, Entware & Diversion are put back in play.

Would the described menu option to essentially "DISABLE" the script, be a good solution to handle your specific situation where, for whatever reason, you have removed & unplugged the USB drive?

 

[shama_001]

Currently, except when doing database record trimming & cleanup, all the calls to SQLite3 do not handle or report errors or failures in the system log. AFAICT, this was done by design in the original code by @Jack Yaz. So it's certainly possible that, in some rare circumstances, the calls to process statistical data & generate records are failing silently and, therefore, the database would not be updated as expected. I can modify all the SQLite3 cmd calls to report errors in the system log, similar to what I did in the latest version for the record trimming & cleanup calls. This way, if the database is not updated as expected, the system log would show if any SQLite3 errors were reported.

This would be great indeed, to track issues and have a better view on such situation. So far I didn't noticed it again so this is not a regular situation I agree. Low priority if any .

 

[Rhialto]

Noticed this extra char on model line...

 

[Martinski]

Noticed this extra char on model line...

View attachment 62870

The shell script gets the router model directly from NVRAM variables, so it is what it is.

Example:

Tomayto... Tomahto. Potayto... Potahto. Same difference.

 

[Rhialto]

The shell script gets the router model directly from NVRAM variables, so it is what it is.

Exactly then I guess lenght of that string is known and removed from the total of spaces required after it.

 

[Martinski]

Exactly then I guess lenght of that string is known and removed from the total of spaces required after it.

Ah, you mean to line up the right border/column of the script header like this?

If so, it was a printf() call. Fixed for the next release. Thanks for reporting it.

 

[thelonelycoder]

Ah, you mean to line up the right border/column of the script header like this?

View attachment 62897

If so, it was a printf() call. Fixed for the next release. Thanks for reporting it.

Usability has improved by a factor of 2 with this fix!

 

[Rhialto]

Usability has improved by a factor of 2 with this fix!

Let's face it, you're also a perfectionnist when it's time to design the text menus, we can feel it so you would have fixed it too.

 

[Martinski]

Usability has improved by a factor of 2 with this fix!

Only a factor of 2?? LOL!!
I was thinking more like 2.5, but I'm obviously too optimistic & biased!!!

 

[agbommarito]

Well, I read your post twice and I really have no idea what you're actually talking about. What specifically are you referring to here? Where is this "tag" and what exactly is "being tagged"? Can you show these "warning level tags"? And, just as important, what or who is doing the "tagging"?

One suggestion: when posting about something that you see happened or is happening, and which you believe is somehow wrong, don’t just say it or describe it; in addition, attach a screenshot that clearly shows what you're talking about to help readers better understand what you’re trying to describe in your post.

"A picture is worth a thousand words" is an old & overused adage, but it works well when applied judiciously and under the right circumstances.

Just my 1.5 cents

When I said "tag" I was referring to the log level, i.e. the third parameter in the "Print_Output()" function that is used in the script.

Here's a screen shot of the router's system log:

Here's a screenshot of the log captured on a remote log server:

Note that the log entries are at a "warning" log level.

The router is set to only log entries more urgent than "notice". Since the log entries for stats updates are "tagged" (sorry for the term) with a level of "warning" they appear in the log. That is what prompted my suggestion that the log level be changed to "notice" or "info".

Maybe this behavior is somehow related to the use of a remote log server. I'll try turning that off and seeing if anything changes.