What's new

uiDivStats uiDivStats - WebUI for Diversion statistics

  • SNBForums Code of Conduct

    SNBForums is a community for everyone, no matter what their level of experience.

    Please be tolerant and patient of others, especially newcomers. We are all here to share and learn!

    The rules are simple: Be patient, be nice, be helpful or be gone!

fresh new install:

cromo@RT-AX88U-8158:/tmp/home/root# /usr/sbin/curl --retry 3 "https://raw.githubusercontent.com/jackyaz/uiDivStats/maste
r/uiDivStats.sh" -o "/jffs/scripts/uiDivStats" && chmod 0755 /jffs/scripts/uiDivStats && /jffs/scripts/uiDivStats instal
l
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 43273 100 43273 0 0 355k 0 --:--:-- --:--:-- --:--:-- 449k
uiDivStats: Welcome to uiDivStats v2.0.0, a script by JackYaz

uiDivStats: Checking your router meets the requirements for uiDivStats

Downloading http://bin.entware.net/aarch64-k3.10/Packages.gz
Updated list of available packages in /opt/var/opkg-lists/entware
Package grep (3.4-1) installed in root is up to date.
Package sqlite3-cli (3310100-1) installed in root is up to date.
Package procps-ng-pkill (3.3.15-4) installed in root is up to date.
uiDivStats: New version of uidivstats_www.asp downloaded

uiDivStats: Mounting uiDivStats WebUI page as user3.asp

uiDivStats: New version of taildns.tar.gz downloaded

Downloading http://bin.entware.net/aarch64-k3.10/Packages.gz
Updated list of available packages in /opt/var/opkg-lists/entware
Package grep (3.4-1) installed in root is up to date.
Package sqlite3-cli (3310100-1) installed in root is up to date.
Package procps-ng-pkill (3.3.15-4) installed in root is up to date.
uiDivStats: Creating database table and enabling write-ahead logging...

uiDivStats: Creating database table indexes...

Error: near line 1: index idx_dns_domains already exists
Error: near line 1: index idx_dns_time already exists
Error: near line 1: index idx_dns_clients already exists
uiDivStats: Database ready, starting services...

uiDivStats: Starting first run of stat generation...

uiDivStats: Stats updated successfully



After this, the UI page is missing. even after a reboot


Fresh install did work!
i just missed that the page locaton moved!
stats generated.
all good. thanks!
 
@dave14305 it is probably sensible for me to hook into dnsmasq restarts, I'll work on that now
Or trust only the diversion.conf “loqQueriesExtra=on” presence. But still you’d want to check on dnsmasq restart if that setting changed.
 
Or trust only the diversion.conf “loqQueriesExtra=on” presence. But still you’d want to check on dnsmasq restart if that setting changed.
I've hooked into dnsmasq.postconf https://github.com/jackyaz/uiDivStats/commit/78b49993ee0bcf97e145f4b204dea7827eaea38d

I would use service-event but I have logic to discard non-uiDivStats service calls, and that would be a lot more complicated to fix. All taildns needs to do is check log-queries in dnsmasq.conf so this should work!
 
This is a great upgrade! A lot of complexity behind the scenes to make it look so cool. :cool:

Noticed this today that when dnssec is enabled there are extra lines in the log that probably want to be filtered out. I had dnssec disabled during beta testing. :oops:

View attachment 23936

Code:
Jun  6 12:08:13 dnsmasq[9727]: dnssec-query[DS] asuswrt-merlin.net to 9.9.9.11
Jun  6 12:08:13 dnsmasq[9727]: dnssec-query[DNSKEY] asuswrt-merlin.net to 9.9.9.11
I assume you get query[A] lines as normal? Can you provide the output of a full query please, from query to reply?
 
I assume you get query[A] lines as normal? Can you provide the output of a full query please, from query to reply?
Two examples before and after enabling extra.
Code:
Jun  6 12:08:13 dnsmasq[9727]: query[A] www.asuswrt-merlin.net from 192.168.1.86
Jun  6 12:08:13 dnsmasq[9727]: forwarded www.asuswrt-merlin.net to 9.9.9.11
Jun  6 12:08:13 dnsmasq[9727]: dnssec-query[DS] asuswrt-merlin.net to 9.9.9.11
Jun  6 12:08:13 dnsmasq[9727]: reply asuswrt-merlin.net is DS keytag 2371, algo 13, digest 2
Jun  6 12:08:13 dnsmasq[9727]: dnssec-query[DNSKEY] asuswrt-merlin.net to 9.9.9.11
Jun  6 12:08:13 dnsmasq[9727]: reply asuswrt-merlin.net is DNSKEY keytag 2371, algo 13
Jun  6 12:08:13 dnsmasq[9727]: reply asuswrt-merlin.net is DNSKEY keytag 34505, algo 13
Jun  6 12:08:13 dnsmasq[9727]: reply www.asuswrt-merlin.net is 104.18.41.167
Jun  6 12:08:13 dnsmasq[9727]: reply www.asuswrt-merlin.net is 104.18.40.167
Jun  6 12:08:13 dnsmasq[9727]: reply www.asuswrt-merlin.net is 172.67.183.16
Jun  6 12:53:02 dnsmasq[29185]: 100 192.168.1.86/54635 query[A] www.asuswrt-merlin.net from 192.168.1.86
Jun  6 12:53:02 dnsmasq[29185]: 100 192.168.1.86/54635 forwarded www.asuswrt-merlin.net to 9.9.9.11
Jun  6 12:53:02 dnsmasq[29185]: * 192.168.1.86/54635 dnssec-query[DS] asuswrt-merlin.net to 9.9.9.11
Jun  6 12:53:02 dnsmasq[29185]: * 192.168.1.86/54635 reply asuswrt-merlin.net is DS keytag 2371, algo 13, digest 2
Jun  6 12:53:02 dnsmasq[29185]: * 192.168.1.86/54635 dnssec-query[DNSKEY] asuswrt-merlin.net to 9.9.9.11
Jun  6 12:53:02 dnsmasq[29185]: * 192.168.1.86/54635 reply asuswrt-merlin.net is DNSKEY keytag 2371, algo 13
Jun  6 12:53:02 dnsmasq[29185]: * 192.168.1.86/54635 reply asuswrt-merlin.net is DNSKEY keytag 34505, algo 13
Jun  6 12:53:02 dnsmasq[29185]: 100 192.168.1.86/54635 reply www.asuswrt-merlin.net is 172.67.183.16
Jun  6 12:53:02 dnsmasq[29185]: 100 192.168.1.86/54635 reply www.asuswrt-merlin.net is 104.18.41.167
Jun  6 12:53:02 dnsmasq[29185]: 100 192.168.1.86/54635 reply www.asuswrt-merlin.net is 104.18.40.167
 
Two examples before and after enabling extra.
Code:
Jun  6 12:08:13 dnsmasq[9727]: query[A] www.asuswrt-merlin.net from 192.168.1.86
Jun  6 12:08:13 dnsmasq[9727]: forwarded www.asuswrt-merlin.net to 9.9.9.11
Jun  6 12:08:13 dnsmasq[9727]: dnssec-query[DS] asuswrt-merlin.net to 9.9.9.11
Jun  6 12:08:13 dnsmasq[9727]: reply asuswrt-merlin.net is DS keytag 2371, algo 13, digest 2
Jun  6 12:08:13 dnsmasq[9727]: dnssec-query[DNSKEY] asuswrt-merlin.net to 9.9.9.11
Jun  6 12:08:13 dnsmasq[9727]: reply asuswrt-merlin.net is DNSKEY keytag 2371, algo 13
Jun  6 12:08:13 dnsmasq[9727]: reply asuswrt-merlin.net is DNSKEY keytag 34505, algo 13
Jun  6 12:08:13 dnsmasq[9727]: reply www.asuswrt-merlin.net is 104.18.41.167
Jun  6 12:08:13 dnsmasq[9727]: reply www.asuswrt-merlin.net is 104.18.40.167
Jun  6 12:08:13 dnsmasq[9727]: reply www.asuswrt-merlin.net is 172.67.183.16
Jun  6 12:53:02 dnsmasq[29185]: 100 192.168.1.86/54635 query[A] www.asuswrt-merlin.net from 192.168.1.86
Jun  6 12:53:02 dnsmasq[29185]: 100 192.168.1.86/54635 forwarded www.asuswrt-merlin.net to 9.9.9.11
Jun  6 12:53:02 dnsmasq[29185]: * 192.168.1.86/54635 dnssec-query[DS] asuswrt-merlin.net to 9.9.9.11
Jun  6 12:53:02 dnsmasq[29185]: * 192.168.1.86/54635 reply asuswrt-merlin.net is DS keytag 2371, algo 13, digest 2
Jun  6 12:53:02 dnsmasq[29185]: * 192.168.1.86/54635 dnssec-query[DNSKEY] asuswrt-merlin.net to 9.9.9.11
Jun  6 12:53:02 dnsmasq[29185]: * 192.168.1.86/54635 reply asuswrt-merlin.net is DNSKEY keytag 2371, algo 13
Jun  6 12:53:02 dnsmasq[29185]: * 192.168.1.86/54635 reply asuswrt-merlin.net is DNSKEY keytag 34505, algo 13
Jun  6 12:53:02 dnsmasq[29185]: 100 192.168.1.86/54635 reply www.asuswrt-merlin.net is 172.67.183.16
Jun  6 12:53:02 dnsmasq[29185]: 100 192.168.1.86/54635 reply www.asuswrt-merlin.net is 104.18.41.167
Jun  6 12:53:02 dnsmasq[29185]: 100 192.168.1.86/54635 reply www.asuswrt-merlin.net is 104.18.40.167
OK so in theory, I think I can adjust the logic to only process a line if "query" is matched, and not "dnssec"
 
I assume you get query[A] lines as normal? Can you provide the output of a full query please, from query to reply?
Line 7 of dnsmasqtotalextra.awk needs incrementing to field 7.
OK so in theory, I think I can adjust the logic to only process a line if "query" is matched, and not "dnssec"
Or “ query” with a leading space?
 
@dave14305 new version of taildns uploaded, please update uiDivStats and retry with dnssec :)
Looking clean!
upload_2020-6-6_13-30-46.jpeg
 
I also had the topic that update to version 2.0.0 got stuck this morning. After solving this problem I was not able to see any stats on the Tab since hours. But since the update it works fine here (AC86U).
I Cant wait to look at the graph in 30 days...
Jack Yaz - Thank you for the effort and the possibility for detailed analysis of block/unblock domains!
 
Since I updated to v2, I have noticed that each of my 4 CPU cores hit 100%, but not all at the same time. Also, the general GUI ‘seems’ a bit sluggish.

When I uninstall, CPU cores just have the usual chatter. Reinstall v2, and the cores peek again.

Is this normal?
 
Since I updated to v2, I have noticed that each of my 4 CPU cores hit 100%, but not all at the same time. Also, the general GUI ‘seems’ a bit sluggish.

When I uninstall, CPU cores just have the usual chatter. Reinstall v2, and the cores peek again.

Is this normal?
Not normal at all. Check what is actually using cpu in top or htop
 
After updating to v2, After a Reboot, I lost my NtpMerlin tab, as well as the Diversion stats tab. The diversion tab in the LAN looks the same.
Anybody else? Before Reboot I still had the old diversion stats tab with nothing on it.
Reboot some more?
I forgot to mention I did a force update afterwards, same results.

Edit: 2nd Reboot was weird, didn't ever shut off, or the led's never went off, usually does it 3 times if it don't go to sleep (86U). But I've got the NTP daemon tab back, so it did something.

jts
RT-AC86U w/ 384.17, RT-AC68U Aimesh node w/ same, 250/10 cable w/ Netgear CM-1000, Diversion, UiDivstats, Skynet, AiProtection, DoT, Scribe, UiScribe, Connmon, SpdMerlin, ScMerlin, Nsrum, NtpMerlin
 
Last edited:
I've uninstalled Diversion and uiDivDtats, rebooted the router (twice), but their two tabs remain under LAN. Is there something else I need to do, or do they just stay there?
 
Love your work @Jack Yaz :D

Wondering if it would be possible for the "Client To Display" selection for Top Blocked and Top Requested to show friendly/host names rather than IP addresses, rather like what happens in Network Tools | Wake On LAN when you go to add an entry, shows active and even inactive clients. Don't know if you can access that data somehow to use in uiDivStats?

prF5N7p.jpg


Likewise, for the Query Log filtering, could the Client field of the filter similarly be a dropdown list of hostnames rather than having to enter an IP perhaps? Maybe keep a separate IP field as well so it goes Domain | IP | Client | Type | Result ?

What do others think?
 
Last edited:

Latest threads

Support SNBForums w/ Amazon

If you'd like to support SNBForums, just use this link and buy anything on Amazon. Thanks!

Sign Up For SNBForums Daily Digest

Get an update of what's new every day delivered to your mailbox. Sign up here!
Top