uiDivStats uiDivStats - WebUI for Diversion statistics

[Ola Malmstrom]

It takes about 40 seconds each hour on my RT-AC3200. If your CPU load continues there is probably something else going on. Try to switch your router off and then on again.

 

[terminator]

It takes about 40 seconds each hour on my RT-AC3200. If your CPU load continues there is probably something else going on. Try to switch your router off and then on again.

ok, thanks. The CPU was going 25% to 75% every few seconds on the UI. "top" command on ssh session was showing around 10% CPU usage for this. I restarted a few times and the CPU would start out at 2% (which is it's normal state) and then go back to 25% to 75% in a few minutes. I turned this off and the CPU went back to 2%. Not sure...

 

[bitoiu]

Hey folks, I've searched quite a bit for my issue but haven't found an answer.

I can't see the DivUIStats. I have a ax88U, I have the scripts up to date and running diversion for 2 weeks. I just went to amtm to generate the stats, but I go into Tools in the WebUI and I can only see the other plugin tools, like spdMerlin. Do I need to do anything to force the webUI update that allows me to check the stats?\

Thank you.

 

[QuikSilver]

Hey folks, I've searched quite a bit for my issue but haven't found an answer.

I can't see the DivUIStats. I have a ax88U, I have the scripts up to date and running diversion for 2 weeks. I just went to amtm to generate the stats, but I go into Tools in the WebUI and I can only see the other plugin tools, like spdMerlin. Do I need to do anything to force the webUI update that allows me to check the stats?\

Thank you.

What version firmware?

 

[bitoiu]

What version firmware?

Firmware Version:384.14
amtm 3.1.5
Diversion: v4.1.10
uiDivStats: v1.3.1
dnsmasq.log 9.2M

Thank you @quicksilver

 

[bitoiu]

Firmware Version:384.14
amtm 3.1.5
Diversion: v4.1.10
uiDivStats: v1.3.1
dnsmasq.log 9.2M

Thank you @quicksilver

I kid you not @quicksilver, just went to the WebUI again and the stats are there. The only thing I can think of is my update this afternoon to Diversion and other scripts.

¯\_(ツ)_/¯

 

[ankhazam]

Can someone help me understand how/what it does in the background? The problem is that my CPU is going haywire. It is constantly churning through the log files? Is there a way to do the processing in a nightly job?

Yup... edit the cron entry using the cru command. I changed my hourly into nightly at 4 AM... It was a hassle to loose Internet for few minutes each and every hour

 

[Jack Yaz]

what's this? dnsmasq logs but in sqlite?

 

[juched]

View attachment 21757
what's this? dnsmasq logs but in sqlite?

Great idea. Grab the records when the logs are small and put them into SQLite. Brilliant.

With how fast unbound is it makes me wonder how much faster dnsmasq would be without writing to disk constantly. No other way to get logs though. Wish one of them would give a different option. Miss my stats.

 

[Jack Yaz]

Great idea. Grab the records when the logs are small and put them into SQLite. Brilliant.

With how fast unbound is it makes me wonder how much faster dnsmasq would be without writing to disk constantly. No other way to get logs though. Wish one of them would give a different option. Miss my stats.

My realtime logging to sqlite should be just the ticket then! I use unbound but it sits behind dnsmasq/diversion.

 

[Jack Yaz]

Show of hands, who looks at the text file report that uiDivStats currently displays? I'm considering removing the current stats generation in its entirety from uiDivStats in favour of leveraging sqlite querying (i.e. no more long running greps on log files)

 

[Stevens243]

Show of hands, who looks at the text file report that uiDivStats currently displays? I'm considering removing the current stats generation in its entirety from uiDivStats in favour of leveraging sqlite querying (i.e. no more long running greps on log files)

I don't typically look at that.

 

[LimJK]

Show of hands, who looks at the text file report that uiDivStats currently displays? I'm considering removing the current stats generation in its entirety from uiDivStats in favour of leveraging sqlite querying (i.e. no more long running greps on log files)

Jack,
Now that you mention, I realised I have not click on it for a long time so I wouldn't miss it.
By the way, can you consider the shorten the Tab title? suggestion "Diversion" instead of "Diversion Stats" as most of the tabs are some form of stats, so that we have with a little more real estate for more tabs . Thank you.

 

[^Tripper^]

May be a minority here, but I look at the text stats a lot, more than the graphs actually.

Would hate to lose em.

Edit ; Clarification, does having the text stats affect diversion’s performance? If it does, happy to lose the text.

 

[juched]

My realtime logging to sqlite should be just the ticket then! I use unbound but it sits behind dnsmasq/diversion.

How do you get the results from Dnsmasq? Need to scrape the logs?

 

[HorseCalledHorse]

Show of hands, who looks at the text file report that uiDivStats currently displays? I'm considering removing the current stats generation in its entirety from uiDivStats in favour of leveraging sqlite querying (i.e. no more long running greps on log files)

You mean the Diversion Statistics Report up top? I use it.

 

[Jack Yaz]

You mean the Diversion Statistics Report up top? I use it.

Ok. I'll see if I can recreate it with my new approach. Otherwise, I'll make it optional with a warning that generating it can take a while/eat cpu cycles

 

[Jack Yaz]

How do you get the results from Dnsmasq? Need to scrape the logs?

Essentially I'm tailing the log files, and writing the relevant lines to sqlite. The tail is running through an awk script to filter/format the output

 

[HorseCalledHorse]

Ok. I'll see if I can recreate it with my new approach. Otherwise, I'll make it optional with a warning that generating it can take a while/eat cpu cycles

Sounds like a nice way to make everybody happy

 

[juched]

Essentially I'm tailing the log files, and writing the relevant lines to sqlite. The tail is running through an awk script to filter/format the output

Ok. Makes sense. The speed of unbound is what has made it hard for me to install diversion again as logging slows things down. Maybe it is just in my head. Is running diversion with unbound still feel speedy? Or faster without diversion due to writing to disk?

I loved UIDivstats but the increasing time to process the log file made my Wednesday’s having downtime once an hour. (Was setup for Thursday flush of logs).