uiDivStats uiDivStats - WebUI for Diversion statistics

[Jack Yaz]

Choose an option:    2

awk: cmd. line:1: Division by zero
awk: cmd. line:1: Division by zero
awk: cmd. line:1: Division by zero
uiDivStats: Stats updated successfully

Press enter to continue...

I've just disabled extra query logging for dnsmasq and here is my current dnsmasq settings if it helps;

Dnsmasq settings:

 See the Dnsmasq man page what these options do:
 http://www.thekelleys.org.uk/dnsmasq/docs/dnsmasq-man.html

 1. Disable Dnsmasq settings
 2. log-async         20
 3. cache-size        2500
 4. bogus-priv        on
 5. domain-needed     on
 6. log-queries=extra

taildns seemed to be using the correct awk script for you. What do you get if you run:

sqlite3 /opt/share/uiDivStats.d/dnsqueries.db
.tables
select * from dnsqueries;

 

[Jack Yaz]

dana@RT-AC2900-72D8:/tmp/home/root# cat /etc/dnsmasq.conf | grep log
log-async
log-queries=extra
log-facility=/opt/var/log/dnsmasq.log
dana@RT-AC2900-72D8:/tmp/home/root#

thanks!

Can you try restarting taildns?

/opt/etc/init.d/S90taildns stop
sleep 5
/opt/etc/init.d/S90taildns start
ps | grep taildns

 

[pattiri]

Code:
sqlite /opt/share/uiDivStats.d/dnsqueries.db
.tables
select * from dnsqueries;

I guess I run them in wrong place?

admin@FaTiii:/tmp/home/root# sqlite /opt/share/uiDivStats.d/dnsqueries.db
-sh: sqlite: not found
admin@FaTiii:/tmp/home/root# sqlite /opt/share/uiDivStats.d/dnsqueries.db
-sh: sqlite: not found
admin@FaTiii:/tmp/home/root# .tables
-sh: .tables: not found
admin@FaTiii:/tmp/home/root# select * from dnsqueries;
-sh: select: not found
admin@FaTiii:/tmp/home/root#

 

[Jack Yaz]

I guess I run them in wrong place?

admin@FaTiii:/tmp/home/root# sqlite /opt/share/uiDivStats.d/dnsqueries.db
-sh: sqlite: not found
admin@FaTiii:/tmp/home/root# sqlite /opt/share/uiDivStats.d/dnsqueries.db
-sh: sqlite: not found
admin@FaTiii:/tmp/home/root# .tables
-sh: .tables: not found
admin@FaTiii:/tmp/home/root# select * from dnsqueries;
-sh: select: not found
admin@FaTiii:/tmp/home/root#

sorry, it should be sqlite3

 

[pattiri]

sorry, it should be sqlite3

No problem

admin@FaTiii:/tmp/home/root# sqlite3 /opt/share/uiDivStats.d/dnsqueries.db
SQLite version 3.31.1 2020-01-27 19:55:54
Enter ".help" for usage hints.
sqlite> .tables
dnsqueries         dnsqueriesmonthly  dnsqueriesweekly
sqlite> select * from dnsqueries;
sqlite>

 

[Dan Ljungberg]

OK so it looks like you have extra query logging for dnsmasq enabled, but taildns is using the standard awk script. Can you send the output of:

cat /etc/dnsmasq.conf | grep log

please?

dana@RT-AC2900-72D8:/tmp/home/root# cat /etc/dnsmasq.conf | grep log
log-async
log-queries=extra
log-facility=/opt/var/log/dnsmasq.log
dana@RT-AC2900-72D8:/tmp/home/root#

than

Can you try restarting taildns?

/opt/etc/init.d/S90taildns stop
sleep 5
/opt/etc/init.d/S90taildns start
ps | grep taildns

dana@RT-AC2900-72D8:/tmp/home/root# /opt/etc/init.d/S90taildns stop
 Checking taildns...              alive.
 Shutting down taildns...              done.
dana@RT-AC2900-72D8:/tmp/home/root# sleep 5
dana@RT-AC2900-72D8:/tmp/home/root# /opt/etc/init.d/S90taildns start
 Starting taildns...              done.
dana@RT-AC2900-72D8:/tmp/home/root# ps | grep taildns
21966 dana      3368 S    {taildns} /bin/sh /opt/bin/taildns
21969 dana      3368 S    {taildnstotal} /bin/sh /jffs/addons/uiDivStats.d/taildns.d/taildnstotal
21974 dana      3368 S    {dnsmasqtotalext} /usr/bin/awk -f /jffs/addons/uiDivStats.d/taildns.d/dnsmasqtotalextra.awk
21975 dana      3368 S    {taildnstotal} /bin/sh /jffs/addons/uiDivStats.d/taildns.d/taildnstotal
22029 dana      5432 S    grep taildns
dana@RT-AC2900-72D8:/tmp/home/root#

 

[Jack Yaz]

No problem

admin@FaTiii:/tmp/home/root# sqlite3 /opt/share/uiDivStats.d/dnsqueries.db
SQLite version 3.31.1 2020-01-27 19:55:54
Enter ".help" for usage hints.
sqlite> .tables
dnsqueries         dnsqueriesmonthly  dnsqueriesweekly
sqlite> select * from dnsqueries;
sqlite>

Well that confirms the data isn't making it into the database for some reason. Please try restarting taildns (using the steps I posted earlier) and then run:

tail -F /opt/var/log/dnsmasq.log /opt/var/log/dnsmasq.log3 | /jffs/addons/uiDivStats.d/taildns.d/dnsmasqtotal.awk

assuming you still have extra query logging turned off

 

[Jack Yaz]

dana@RT-AC2900-72D8:/tmp/home/root# cat /etc/dnsmasq.conf | grep log
log-async
log-queries=extra
log-facility=/opt/var/log/dnsmasq.log
dana@RT-AC2900-72D8:/tmp/home/root#

than

dana@RT-AC2900-72D8:/tmp/home/root# /opt/etc/init.d/S90taildns stop
 Checking taildns...              alive.
 Shutting down taildns...              done.
dana@RT-AC2900-72D8:/tmp/home/root# sleep 5
dana@RT-AC2900-72D8:/tmp/home/root# /opt/etc/init.d/S90taildns start
 Starting taildns...              done.
dana@RT-AC2900-72D8:/tmp/home/root# ps | grep taildns
21966 dana      3368 S    {taildns} /bin/sh /opt/bin/taildns
21969 dana      3368 S    {taildnstotal} /bin/sh /jffs/addons/uiDivStats.d/taildns.d/taildnstotal
21974 dana      3368 S    {dnsmasqtotalext} /usr/bin/awk -f /jffs/addons/uiDivStats.d/taildns.d/dnsmasqtotalextra.awk
21975 dana      3368 S    {taildnstotal} /bin/sh /jffs/addons/uiDivStats.d/taildns.d/taildnstotal
22029 dana      5432 S    grep taildns
dana@RT-AC2900-72D8:/tmp/home/root#

OK so taildns is using the right awk script now. Hopefully you'll see your database file growing/modified:

ls -lah /opt/share/uiDivStats.d/dnsqueries.db

 

[pattiri]

Well that confirms the data isn't making it into the database for some reason. Please try restarting taildns (using the steps I posted earlier) and then run:

tail -F /opt/var/log/dnsmasq.log /opt/var/log/dnsmasq.log3 | /jffs/addons/uiDivStats.d/taildns.d/dnsmasqtotal.awk

assuming you still have extra query logging turned off

here;

admin@FaTiii:/tmp/home/root# /opt/etc/init.d/S90taildns stop
 Checking taildns...              alive.
 Shutting down taildns...              done.
admin@FaTiii:/tmp/home/root# sleep 5
admin@FaTiii:/tmp/home/root# /opt/etc/init.d/S90taildns start
 Starting taildns...              done.
admin@FaTiii:/tmp/home/root# ps | grep taildns
28269 admin     1420 S    {taildns} /bin/sh /opt/bin/taildns
28272 admin     1420 S    {taildnstotal} /bin/sh /jffs/addons/uiDivStats.d/taildns.d/taildnstotal
28278 admin     1444 S    {dnsmasqtotal.aw} /usr/bin/awk -f /jffs/addons/uiDivStats.d/taildns.d/dnsmasqtotal.awk
28279 admin     1424 S    {taildnstotal} /bin/sh /jffs/addons/uiDivStats.d/taildns.d/taildnstotal
28335 admin     4656 S    grep taildns
admin@FaTiii:/tmp/home/root# tail -F /opt/var/log/dnsmasq.log /opt/var/log/dnsmasq.log3 | /jffs/addons/uiDivStats.d/taildns.d/dnsmasqtotal.awk
tail: can't open '/opt/var/log/dnsmasq.log3': No such file or directory
1591453572,172.24.5.2,spclient.wg.spotify.com,A,allowed
1591453618,172.24.5.8,chat.cdn.whatsapp.net,A,allowed
1591453648,172.24.5.15,mapi.speedtest.net,A,allowed
1591453648,127.0.1.1,speedtest.net,dnssec-DS,allowed
1591453649,172.24.5.2,mix.com,A,allowed
1591453649,172.24.5.2,mix.com,A,allowed
1591453649,127.0.1.1,mix.com,dnssec-DS,allowed
1591453649,172.24.5.2,mix.com,A,allowed
1591453649,172.24.5.2,mix.com,AAAA,allowed
1591453649,172.24.5.2,us.hola.com,A,allowed
1591453650,127.0.1.1,hola.com,dnssec-DS,allowed
1591453650,172.24.5.2,e13750.a.akamaiedge.net,A,allowed
1591453650,172.24.5.2,e13750.a.akamaiedge.net,AAAA,allowed
1591453650,172.24.5.2,tagger.opecloud.com,A,blocked (blocking list)
1591453650,172.24.5.2,amplifypixel.outbrain.com,A,blocked (blocking list)
1591453650,172.24.5.2,tagger.opecloud.com,A,blocked (blocking list)

I can see new logs as I browse internet. so I've chosen option 2 and here we have the happy result Thank you very much

 

[ugandy]

uninstalled existing version, and then installed new version.
install stuck at "Creating database table indexes..."

 

[Jack Yaz]

uninstalled existing version, and then installed new version.
install stuck at "Creating database table indexes..."

I've just pushed a hotfix which will hopefully show an error if the index creation won't work. Ctrl-C out then:

uiDivStats update

 

[Dan Ljungberg]

OK so taildns is using the right awk script now. Hopefully you'll see your database file growing/modified:

ls -lah /opt/share/uiDivStats.d/dnsqueries.db

OK so taildns is using the right awk script now. Hopefully you'll see your database file growing/modified:

ls -lah /opt/share/uiDivStats.d/dnsqueries.db

Still no growing database file and no stats
Is it easier if I uninstall and reinstall, or are you interested in troubleshooting more?

 

[ugandy]

fresh new install:

cromo@RT-AX88U-8158:/tmp/home/root# /usr/sbin/curl --retry 3 "https://raw.githubusercontent.com/jackyaz/uiDivStats/maste
r/uiDivStats.sh" -o "/jffs/scripts/uiDivStats" && chmod 0755 /jffs/scripts/uiDivStats && /jffs/scripts/uiDivStats instal
l
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 43273 100 43273 0 0 355k 0 --:--:-- --:--:-- --:--:-- 449k
uiDivStats: Welcome to uiDivStats v2.0.0, a script by JackYaz

uiDivStats: Checking your router meets the requirements for uiDivStats

Downloading http://bin.entware.net/aarch64-k3.10/Packages.gz
Updated list of available packages in /opt/var/opkg-lists/entware
Package grep (3.4-1) installed in root is up to date.
Package sqlite3-cli (3310100-1) installed in root is up to date.
Package procps-ng-pkill (3.3.15-4) installed in root is up to date.
uiDivStats: New version of uidivstats_www.asp downloaded

uiDivStats: Mounting uiDivStats WebUI page as user3.asp

uiDivStats: New version of taildns.tar.gz downloaded

Downloading http://bin.entware.net/aarch64-k3.10/Packages.gz
Updated list of available packages in /opt/var/opkg-lists/entware
Package grep (3.4-1) installed in root is up to date.
Package sqlite3-cli (3310100-1) installed in root is up to date.
Package procps-ng-pkill (3.3.15-4) installed in root is up to date.
uiDivStats: Creating database table and enabling write-ahead logging...

uiDivStats: Creating database table indexes...

Error: near line 1: index idx_dns_domains already exists
Error: near line 1: index idx_dns_time already exists
Error: near line 1: index idx_dns_clients already exists
uiDivStats: Database ready, starting services...

uiDivStats: Starting first run of stat generation...

uiDivStats: Stats updated successfully



After this, the UI page is missing. even after a reboot

 

[Jack Yaz]

Still no growing database file and no stats
Is it easier if I uninstall and reinstall, or are you interested in troubleshooting more?

worth a go - i'm stumped!

 

[Dan Ljungberg]

worth a go - i'm stumped!

did an

worth a go - i'm stumped!

Thanks for quick and good response with my troubleshooting.
I tested and uninstalled and deleted db from CLI, restarted the router and then reinstalled from CLI.
But still no stats

I'm also stumped

 

[Jack Yaz]

did an


Thanks for quick and good response with my troubleshooting.
I tested and uninstalled and deleted db from CLI, restarted the router and then reinstalled from CLI.
But still no stats

I'm also stumped

at this stage, would you be up for a screensharing session using Teamviewer or VNC or something?

 

[dave14305]

@Jack Yaz Might it be that taildns is starting before Diversion modifies /etc/dnsmasq.conf and therefore misses the log queries extra configuration?

 

[Jack Yaz]

@Jack Yaz Might it be that taildns is starting before Diversion modifies /etc/dnsmasq.conf and therefore misses the log queries extra configuration?

I thought Diversion loaded itself prior to launching Entware services? I could be wrong

 

[L&LD]

RT-AX88U on RT-AX88U_384.18_alpha1-g6368a955e8 firmware.

I had the previous uiDivStats version running before but uninstalled it for other troubleshooting issues (and waited for v2.00 to become available before I re-installed it again).

Just installed v2.00 with router currently 'up' 8 days and 9 hours. No issues, Tab shows in LAN, and everything running normally with no reboot needed (although I will probably do one in an hour or so, as per my 'normal' after any change in the GUI).

Thank you very much @Jack Yaz! Another solid release.

 

[dave14305]

This is a great upgrade! A lot of complexity behind the scenes to make it look so cool.

Noticed this today that when dnssec is enabled there are extra lines in the log that probably want to be filtered out. I had dnssec disabled during beta testing.

Jun  6 12:08:13 dnsmasq[9727]: dnssec-query[DS] asuswrt-merlin.net to 9.9.9.11
Jun  6 12:08:13 dnsmasq[9727]: dnssec-query[DNSKEY] asuswrt-merlin.net to 9.9.9.11