uiDivStats Both pCPUS to 100% and many processes backed up

[gattaca]

Hi Everyone! Jack, just posting this here. It's the 2nd or 3rd time in a few months I've had to kick my AC-86U, (running all the current updates), over what appears to be something gone / going wrong inside uiDivStats. My router usually makes it about 45+ days before I have to reboot it. This is the sort of things I've found at least 2 or 3 x now. Just FYI and something to keep an eye out for. The quickest way to right this seems to be to perform a reboot either from the GUI (if it's working) or AMTM. I managed to capture more info this time before I just had to reboot it.

Symptoms are:

a) Internet traffic slows to a crawl - family starts complaining, ...
b) both pCPUs are locked and running at 100%.
c) htop shows many uiDivStats procs running..
d) Sometimes the GUI just stops working when it gets further along than this.

Stay safe, stay alive! Peace.

 

[Swinson]

Hi Everyone! Jack, just posting this here. It's the 2nd or 3rd time in a few months I've had to kick my AC-86U, (running all the current updates), over what appears to be something gone / going wrong inside uiDivStats. My router usually makes it about 45+ days before I have to reboot it. This is the sort of things I've found at least 2 or 3 x now. Just FYI and something to keep an eye out for. The quickest way to right this seems to be to perform a reboot either from the GUI (if it's working) or AMTM. I managed to capture more info this time before I just had to reboot it.

Symptoms are:

a) Internet traffic slows to a crawl - family starts complaining, ...
b) both pCPUs are locked and running at 100%.
c) htop shows many uiDivStats procs running..
d) Sometimes the GUI just stops working when it gets further along than this.

View attachment 28960
View attachment 28961
Stay safe, stay alive! Peace.

I occasionally have issues with the addons pages not loading data into graphs. Sometimes it will just hang and then load other times it won’t ever come up. I’ve always fixed this by just closing my chrome tab and starting a new one. I thought chrome was the culprit but now I’m wondering if this may be related to the issue you are describing.

Just wondering have you tried just restarting httpd rather than a full reboot?

 

[Jack Yaz]

Hi Everyone! Jack, just posting this here. It's the 2nd or 3rd time in a few months I've had to kick my AC-86U, (running all the current updates), over what appears to be something gone / going wrong inside uiDivStats. My router usually makes it about 45+ days before I have to reboot it. This is the sort of things I've found at least 2 or 3 x now. Just FYI and something to keep an eye out for. The quickest way to right this seems to be to perform a reboot either from the GUI (if it's working) or AMTM. I managed to capture more info this time before I just had to reboot it.

Symptoms are:

a) Internet traffic slows to a crawl - family starts complaining, ...
b) both pCPUs are locked and running at 100%.
c) htop shows many uiDivStats procs running..
d) Sometimes the GUI just stops working when it gets further along than this.

View attachment 28960
View attachment 28961
Stay safe, stay alive! Peace.

check if something is spamming DNS, you can check this either on the uiDivStats query log (if its working), or following the log file for all requests in Diversion

 

[gattaca]

^^^ No on restarting the GUI.. given all the uiDivStats processes showing backed up in the htop... the https is not likely the issue. A symptom when it advances more in this state is I usually cannot login to HTTPS (b/c the CPUs are overloaded). I can usually get in via SSH which is the biggest clue.

Jack I will look at this the next time it happens.... in 40+ days.. TY for the clue where to check.

 

[Swinson]

^^^ No on restarting the GUI.. given all the uiDivStats processes showing backed up in the htop... the https is not likely the issue. A symptom when it advances more in this state is I usually cannot login to HTTPS (b/c the CPUs are overloaded). I can usually get in via SSH which is the biggest clue.

Jack I will look at this the next time it happens.... in 40+ days.. TY for the clue where to check.

Yeah I understand that uiDivStats is what’s creating the processes that is overloading things my thought was that restarting httpd would reset divStats and clear it out but to be honest I’m not well versed on the inner workings of divStats. I know it has a few cron jobs if there is a worker process cleanup that runs you could try running that more aggressively but again I really don’t know the ins and outs of what it’s doing regularly. Jack’s suggestion is certainly the place to start but I never miss an opportunity to show off my ignorance lol.

 

[Jack Yaz]

i've been meaning to enhance my locking functions to check for dead processes and kill them. maybe I'll do that next after YazDHCP is done

 

[Swinson]

i've been meaning to enhance my locking functions to check for dead processes and kill them. maybe I'll do that next after YazDHCP is done

YazDHCP?! Best thing I’ve heard all day! I love yazfi but my echo dot is always misbehaving in one way or another. Lately it’s been fond of trying to stay on my main subnet after yazfi tries to bounce it.

 

[L&LD]

YazDHCP! Yes, I'm going to love this new addition too. Somebody needs to do the 'world of @Jack Yaz' thread to keep track of his prolific output.

 

[JaimeZX]

Meanwhile. Set your router to reboot weekly when everyone is asleep and you won't have to deal with stuff that crops up every 40 days. You'll also be more protected from VPNFilter.

 

[Dancing Lemur]

Did this ever get figured out? I was struggling with my RT-AC3100 running Merlin hitting 100% CPU usage (on both CPUs). Certain devices connected stopped being able to access the network. I wasn't sure if it was due to some wireless radio configuration or something else. I saw the suggestion to do a top/ps and noticed a huge number of uiDivStats processes -- 450+. They each were .3% of CPU but it adds up if you have that many.

I didn't look at uiDivStats that often, so for now I just disabled it. That didn't end the processes so I had to reboot my router. When I restarted, there was no runaway number of processes (as expected).

Anyway, did this ever get diagnosed? If there's a fix, I'd re-enable uiDivStats.

I'm hoping that the 100% CPU utilization is gone; I'll know if this goes a few days without that happening (it took a few days before the problem arose).

 

[Jack Yaz]

I did write a function to cull long running uiDivStats processes but I don't recall off the top of my head when/if it was released. I'll check tonight.

Do you happen to know which processes they were, like querylog?

 

[Dancing Lemur]

I closed the ssh session (or rather it closed when I rebooted), so I don't have that processes list anymore. They weren't really stale though. I had just re-enabled diversion after a few days of it being disabled -- this thread led me to believe that it was uiDivStats that was causing the 100% utilization and I confirmed it when the jobs blew up rather quickly.

If you want, I can try to isolate the specific process later? I'd have to reinstall it (I didn't see an easy way to disable rather than uninstall uiDivStats). It might take a while before the 100% situation arose. It seems random.