uiDivStats uiDivStats - high cpu usage

[abracadabra11]

Installed uiDivStats recently and have had high cpu usage since. Changing the query and cache method had negligible effect. Uninstalling returned low cpu usage. Earlier posts on the original thread seem to suggest USB drive is the limiting factor. I'm currently running mine at 2.0 speed.

Has anyone resolved the issue by changing to 3.0?

I'll certainly try this after reinstalling, but curious if there are any success stories with this simple change.

 

[abracadabra11]

Changed USB mode from 2.0 to 3.0 and it had no discernible effect on cpu usage with uDivStats installed.

 

[jsbeddow]

I don't recall the details, but this came up several months ago:IIRC, the solution was a better/faster USB flash drive (some of us are using repurposed SSDs in external adapters to solve that once and done). Might be worth trying, and/or moving the storage location for the stats onto either the USB drive or jffs, as needed in your case.

 

[DeltaKilo]

Haha, what are the odds? About 2 weeks ago I delved into the 'add-on' arena with Diversion, YazFi & uiDivStats. The system status CPU was showing a resting state of between 20%-30% even after a few hours.
I uninstalled & reinstalled uiDivStats with no change so decided to uninstall it.

About 30min before reading your post, I reinstalled it again to help hunt down a domain to whitelist. Lo and behold, CPU usage is normal and not idling high like before.

No other scripts have been added or changes made to router - just 2ish weeks running without uiDivStats. (queue X-Files music)

For me, it seems time helped heal

 

[alexholman]

I've noticed a similar behavior, I think because of uiDivStats. When I first installed, the system was working fine, but after a couple of months, it started getting dodgy, dropped packets, and frequently having to reboot the router. When I would check in the UI, at least one of the router's cores would be pinned at 100% usage. However, when I checked in top there would be nothing showing up above about 2% usage.
I just uninstalled uiDivStats and my CPU plot is hanging under 10%.

 

[grifo]

Just had this, the CPU was running crazy with IO, uninstalled uiDivStats and removed /opt/share/uiDivStats.d which had files as large as 240 MB, reinstalled uiDivstats and now it's working fine.

Edit: corrected the folder name to /opt/share/uiDivStats.d

 

[Jack Yaz]

tmpfs should help most people
Next version of uiDivStats takes steps to reduce CPU priority of its tasks by setting "nice" value of processes

 

[aa.]

I've been puzzling why after few days the router's web UI, AC86U, is not responding (though internet connection is still fine). I have to access the router by ssh and restarted it, often it have to be restarted manually by turning the router off/on.

from the top result, it shown that more than a hundred process of grep, awk, trim, nvram_get, tr etc. ignited by (supposedly) uiDivstat scripts

/opt/share/uiDivStats.d/dnsqueries.db size is 520MB

I'm not sure if this is normal

I just uninstalled uiDivStats, if it's indeed the culprit, I think I can live without it.

 

[jjones7791]

I'm running into this same issue on a RT-AX86U Pro as well running Diversion, Skynet and Uidivstats and Uidivstats seems to be the culprit as the issue ceased once I uninstalled it. Like others, showing huge spikes in IO on all cores about every 5 to 10 minutes or so pushing my top stats to over 5 in some cases, then back to normal closer to around 2 for a few minutes before it reverts. Before I uninstalled Uidivstats, I tried using a different USB on 3.0 instead of 2.0 and it seemed to have helped at first, then got worse over about two weeks, back to cpu spiking. Also tried turning the Uidivstats cache setting from tmp to none and seemed to actually make it worse. If anyone found a way to solve for this on other models, would love to hear what you did, as I love this app and the stats / graphs it's provides, but obviously not if it's maxing out my router cpu.

 

[bjark]

Same story here. I recently moved to a new house and brought my network setup, including an AC86U, with me. Relevant differences between WAN connections was going from IPv4 only to IPv4/6 and going from 250/100Mbps to 1/1Gbps. (It might be of interest here that besides WAN I have two more 1Gbps devices connected to the router, so it can suffer some traffic at times. I have used diversion, skynet and unbound since at least a year. The router does VPN server work for me, but has no VPN client role.) Had to fiddle with some things in the router setup though, due to the ISP not recognizing my router initially (somehow making only IPv6 WAN traffic work, IPv4 was diverted), and I didn't realize the root issue until after trying some other troubleshooting. Fixed that after registering router MAC with the ISP and also turned off IPv6 since it only gave me problems and I don't see any need for it that outweighs the issues.

However, there were still issues with the router "locking up" off and on. The interface would be unresponsive (by LAN) both both SSH and web. Traffic would halt momentarily and especially DNS lookups (using unbound on the router) had severe issues with timing out. Also CPU usage was notably high and temperature hovered around 95 and would sometimes pass over 100C leading to CPU throttling and even more issues.

After trying different other things to pin down the cause I noticed that uidivstats ran a lot of processes that used a lot of CPU time. After uninstalling uidivstats the issues seem gone, CPU average use is down to ~10% rather than ~80% and CPU temperature seem to be lowering.

(One test I used, that could reliably "hang" the router nine times out of ten, was the nameserver benchmark in the DNS Benchmark tool (a Windows executable) by GRC. After uninstalling uidivstats and with nothing else changed, I can run the benchmark repeatedly with no issues. The benchmark test is artificial of course but since the real issues were intermittent and I could not trigger them at will it was useful to have the benchmark for testing and it correlated with the real world issues even though that was not the specific purpose of it.)

 

[jjones7791]

@bjark you have a bit more complex setup than I do, as I'm only running Diversion, Skynet, and uiDivStats, however I can tell you 100% of my problems were related to the drive I was using to the host the scripts. I was using a typical usb flash drive, and once I moved to an SSD, these issues completely ceased. If you are using flash, even for a while, move to SSD. My flash drive ran fine for several months, until it didn't, and when I had uiDivStats running on the flash drive, it would push my load up to 4, sometimes even 5, due to high IO, and the flash drive was behind this, until it died completely. Since I took the great advice here with the SSD, my load barely hits 1.25 on average and no issues with any of the scripts above. Thread below discusses this topic if you decide to go this direction and want some SSD suggestions, likewise there is a specific brand of controller (ASMEDIA) in the SSD cases that work better as well based on some troubleshooting other folks have done.

https://www.snbforums.com/threads/usb-recommendations-for-diversion-skynet.84828/

 

[bjark]

@bjark you have a bit more complex setup than I do, as I'm only running Diversion, Skynet, and uiDivStats, however I can tell you 100% of my problems were related to the drive I was using to the host the scripts. I was using a typical usb flash drive, and once I moved to an SSD, these issues completely ceased. If you are using flash, even for a while, move to SSD. My flash drive ran fine for several months, until it didn't, and when I had uiDivStats running on the flash drive, it would push my load up to 4, sometimes even 5, due to high IO, and the flash drive was behind this, until it died completely. Since I took the great advice here with the SSD, my load barely hits 1.25 on average and no issues with any of the scripts above. Thread below discusses this topic if you decide to go this direction and want some SSD suggestions, likewise there is a specific brand of controller (ASMEDIA) in the SSD cases that work better as well based on some troubleshooting other folks have done.

https://www.snbforums.com/threads/usb-recommendations-for-diversion-skynet.84828/

I see what you are saying there. I have a spare 128GB SSD with USB connection. I will try that and see if things are smoother than they are already, and maybe if I can put uidivstats back on.

I think there is only the swap file on the USB stick at the moment. Guess a migrations procedure could be to disable the scripts that use the swap, turn off router, exchange stick for SSD, turn back on, recreate swap file on new device and turn scripts back on. Am I missing something important?

Edit: Not sure what I was thinking. The scripts and entware are all on the stick too I guess.

 

[jjones7791]

I think there is only the swap file on the USB stick at the moment. Guess a migrations procedure could be to disable the scripts that use the swap, turn off router, exchange stick for SSD, turn back on, recreate swap file on new device and turn scripts back on. Am I missing something important?

Edit: Not sure what I was thinking. The scripts and entware are all on the stick too I guess.

Your edit is spot on, trying to migrate them over is going to give you problems because Entware and your scripts all live on the swap. Not to say it's impossible, and there are smarter people here who know how to do this and can likely walk you through it. I feel like I've read a thread somewhere that eluded to this as well, but it's not ideal or easy from what I've seen. Personally, I prefer to burn everything down and start over so I know I'm not carrying over any issues the flash may have introduced. I keep a running document of my router setup so outside of reboot times, I can burn it down and have it back to 100% in a couple hours or so. Well that and in my case, my flash completely died, so I didn't really have an option, but I wouldn't have done that anyway.

 

[bjark]

I made the switch and it is a noticeable improvement. Application response times are much "snappier" now and I can have uidivstats running with no performance/stability issues. Temperatures are a tiny bit lower (with more things running). Thanks for your advice!

 

[jjones7791]

I made the switch and it is a noticeable improvement. Application response times are much "snappier" now and I can have uidivstats running with no performance/stability issues. Temperatures are a tiny bit lower (with more things running). Thanks for your advice!

Awesome news! Glad to hear and happy to help. But I will say I can take no credit for this. I went through this exercise just a few weeks back and thanks to @Tech9 , @L&LD , @gattaca , @Crimliar to name a few, they gave me great advice that I am happy to pass on.