uiDivStats uiDivStats - WebUI for Diversion statistics

[Jack Yaz]

@Jack Yaz Works great on ASUS RT-AC5300. Thank you!

The only thing I noticed was on the attached graph. The left side is all garbled. This is on the latest version of Safari on macOS Mojave 10.14.4 (in case it helps).

Same thing re. Scale.

You lot have some naughty devices on your network to generate that many blocks!

 

[#TY]

did that not appear? :/

I'm sorry, I don't recall at this point as everything is installed and working now.

 

[#TY]

You lot have some naughty devices on your network to generate that many blocks!

Kids

 

[Jack Yaz]

I'm sorry, I don't recall at this point as everything is installed and working now.

No worries!

 

[Goobi]

Got a few Roku still sticks that love to try and phone home back to scribe.logs.roku.com and are by far the biggest item blocked. Do you think I can block this is skynet so that it does not show up as a blocked domain in Diversion? That way I don’t have one huge bar and 9 small ones when viewing the stats in the GUI.

 

[L&LD]

Got a few Roku still sticks that love to try and phone home back to scribe.logs.roku.com and are by far the biggest item blocked. Do you think I can block this is skynet so that it does not show up as a blocked domain in Diversion? That way I don’t have one huge bar and 9 small ones when viewing the stats in the GUI.

Try it and see?

These are the IP's I got using 384.11 Beta2's Nslookup tool. Not sure if they will help, but here they are.

Name:      scribe.logs.roku.com
Address 1: 52.1.249.132 ec2-52-1-249-132.compute-1.amazonaws.com
Address 2: 52.203.113.231 ec2-52-203-113-231.compute-1.amazonaws.com
Address 3: 52.1.156.171 ec2-52-1-156-171.compute-1.amazonaws.com
Address 4: 52.204.87.181 ec2-52-204-87-181.compute-1.amazonaws.com
Address 5: 52.1.96.241 ec2-52-1-96-241.compute-1.amazonaws.com
Address 6: 52.20.9.181 ec2-52-20-9-181.compute-1.amazonaws.com
Address 7: 52.203.238.238 ec2-52-203-238-238.compute-1.amazonaws.com
Address 8: 52.1.166.147 ec2-52-1-166-147.compute-1.amazonaws.com

 

[Butterfly Bones]

I had one small glitch after updating from 384.11_beta 1 to beta 2 (no other of your scripts).
Checking router after upgrade, uiDivStats did not who in Tools tab. I ran the command in a window and choose to "Generate Stats", still nothing in webGUI, tried "check for update", nope still no tab. I then did a "uf - Force Update" and it appeared.

Maybe not an issue at all, but caused by my router version update method, very conservative, and much different than most. This goes back to my 25+ years using Linux.

• In Administration > System - change "Enable JFFS custom scripts and configs" to No and apply.
• Open SSH session that just closed and issue cli command "halt" - wait for router to shut down.
• Remove USB drive from router and power on
• Log into router, update firmware. Verify successful update, check Admin, syslog, WAN, LAN settings, etc.
• In Administration > System - change "Enable JFFS custom scripts and configs" to Yes and apply.
• Open SSH session again, issue cli command "halt" - wait for router to shut down.
• Insert USB stick into router and power on.
• Leave router alone for 15 minutes (don't cheat - set a phone timer)
• Log into router, verify settings applied, check Admin, syslog, WAN, LAN settings, etc.

Arms in air Victory gesture!

 

[martinr]

.....

• In Administration > System - change "Enable JFFS custom scripts and configs" to No and apply.
• Open SSH session that just closed and issue cli command "halt" - wait for router to shut down.
• Remove USB drive from router and power on
• Log into router, update firmware. Verify successful update, check Admin, syslog, WAN, LAN settings, etc.
• In Administration > System - change "Enable JFFS custom scripts and configs" to Yes and apply.
• Open SSH session again, issue cli command "halt" - wait for router to shut down.
• Insert USB stick into router and power on.
• Leave router alone for 15 minutes (don't cheat - set a phone timer)
• Log into router, verify settings applied, check Admin, syslog, WAN, LAN settings, etc.

Arms in air Victory gesture!

This could become the “dirty” equivalent of L&LD’s Manual and Minimal Guide.

 

[L&LD]

I had one small glitch after updating from 384.11_beta 1 to beta 2 (no other of your scripts).
Checking router after upgrade, uiDivStats did not who in Tools tab. I ran the command in a window and choose to "Generate Stats", still nothing in webGUI, tried "check for update", nope still no tab. I then did a "uf - Force Update" and it appeared.

Maybe not an issue at all, but caused by my router version update method, very conservative, and much different than most. This goes back to my 25+ years using Linux.

• In Administration > System - change "Enable JFFS custom scripts and configs" to No and apply.
• Open SSH session that just closed and issue cli command "halt" - wait for router to shut down.
• Remove USB drive from router and power on
• Log into router, update firmware. Verify successful update, check Admin, syslog, WAN, LAN settings, etc.
• In Administration > System - change "Enable JFFS custom scripts and configs" to Yes and apply.
• Open SSH session again, issue cli command "halt" - wait for router to shut down.
• Insert USB stick into router and power on.
• Leave router alone for 15 minutes (don't cheat - set a phone timer)
• Log into router, verify settings applied, check Admin, syslog, WAN, LAN settings, etc.

Arms in air Victory gesture!

Thanks for posting those steps! They may be very useful for someone that wants to do a 'dirty' upgrade, but for one reason or another can't complete it using the normal/simpler methods.

The other side of the above is a slightly more straightforward 'dirty-upgrade'.

I simply flashed the RT-AC86U from Beta 1 to Beta 2. When it rebooted, I waited 10 minutes and started checking that all scripts and services were running correctly.

Like you, only the uiDivStats script's GUI wasn't showing. I waited until the time passed the half-hour mark (9:35 PM) and checked again (this is when uiDivStats would have normally auto-updated the stats). Nothing.

I refreshed Edge (CTRL F5), but still nothing. I closed Edge. I then fired up PuTTY and ran uiDivStats, Generate Diversion Statistics now '1', let it finish and it then showed properly in the GUI.

After the router was up for just over an hour after the update (63 minutes), my final test was to reboot it and then check it again after 10 minutes leaving it to sit idle. No issues! Victory!

Now, the router has been working for the last 18+ hours without a hiccup.

 

[Butterfly Bones]

I had to do a complete wipe and restore of router, USB, etc. a few days back. Teach me to play with Alpha stuff!

Before that I only ever did a reset when RMerlin update states it is mandatory. I think this was the first since I got the AC86U in Dec. 2017. You and I have very different opinions on frequency of resets.

I go back with Linux many years when we used to compete for uptimes, trying to beat 1000 days or more. My Linux box only reboots when a new kernel is released due to fixing vulnerabilities.

$ uptime
11:06:26 up 54 days, 13:52,  1 user,  load average: 1.06, 1.22, 1.31

We have gotten way OT for this thread, sorry Jack! I thought my update method of disabling jffs might be the cause of uiDivStats not loading after update, but L&LD had the same. (shrug)

 

[Jack Yaz]

Race condition between this script and the other j scripts - they haven't been patched to be aware that this one exists

 

[Jack Yaz]

Any suggestions for additional graphs, by the way?

 

[L&LD]

Any suggestions for additional graphs, by the way?

I'm not sure what additional graphs are available?

 

[Jack Yaz]

I'm not sure what additional graphs are available?

pick what you like out of the text stats, and i can hopefully graph it

 

[EmeraldDeer]

Any suggestions for additional graphs, by the way?

• Percentage by query type
  • IPv4
  • IPv6
  • PTR
  • DNSSEC
• Percentage by reply type
  • cached
  • cached NXDOMAIN
  • forwarded
  • forwarded NXDOMAIN
  • hosts
  • config
  • diversion
  • SERVFAIL

• Not sure how it should be graphed or a table instead

• Perhaps SERVFAIL on its own because it means there is a problem to address above a reasonable rate

 

[truglodite]

Not a new graph suggestion, but a possible way to deal with the scaling. One way might be using an exponential scale. Another way may be to crop values above some nominal amount (say 1k, 2k, 5k, 10k...) and display those columns on the same scale, but with the cropped value preppended to the domain name... of course keep the columns sorted appropriately left to right. Not sure if I worded that last one clearly. :/

Alternatively, maybe a pie graph? ...like traffic stats has?

 

[Marin]

I had one small glitch after updating from 384.11_beta 1 to beta 2 (no other of your scripts).
Checking router after upgrade, uiDivStats did not who in Tools tab. I ran the command in a window and choose to "Generate Stats", still nothing in webGUI, tried "check for update", nope still no tab. I then did a "uf - Force Update" and it appeared.

Maybe not an issue at all, but caused by my router version update method, very conservative, and much different than most. This goes back to my 25+ years using Linux.

• In Administration > System - change "Enable JFFS custom scripts and configs" to No and apply.
• Open SSH session that just closed and issue cli command "halt" - wait for router to shut down.
• Remove USB drive from router and power on
• Log into router, update firmware. Verify successful update, check Admin, syslog, WAN, LAN settings, etc.
• In Administration > System - change "Enable JFFS custom scripts and configs" to Yes and apply.
• Open SSH session again, issue cli command "halt" - wait for router to shut down.
• Insert USB stick into router and power on.
• Leave router alone for 15 minutes (don't cheat - set a phone timer)
• Log into router, verify settings applied, check Admin, syslog, WAN, LAN settings, etc.

Arms in air Victory gesture!

Same here. For some reason, it doesn't seem to survive a reboot. Rebooted twice and used the install curl twice and the uiDivStats tab appeared again.

I like seeing the stats in GUI now-don't have to SSH in the router or wait for the weekly email from Diversion to have an idea what they are. Great work as always @Jack Yaz !

 

[Xentrk]

Not a new graph suggestion, but a possible way to deal with the scaling. One way might be using an exponential scale. Another way may be to crop values above some nominal amount (say 1k, 2k, 5k, 10k...) and display those columns on the same scale, but with the cropped value preppended to the domain name... of course keep the columns sorted appropriately left to right. Not sure if I worded that last one clearly. :/

Alternatively, maybe a pie graph? ...like traffic stats has?

The pie graph may be a limitation of the firmware. The pfBlockerNG package on pfSense has those features though. Following are two examples:

 

[Krvicch]

Getting this error when trying to run uiDivStats

gnuplot: error while loading shared libraries: libreadline.so.8: cannot open shared object file: No such file or directory


Any ideas?

 

[Marin]

try:

1. Uninstall and reinstall uiDivStats

or,

2. From the uiDivStats menu, do a force update (option uf)

see if that works.