Menu
What's new
By:
Filters Better Search

Monitoring Web Usage with Asuswrt-Merlin

  • SNBForums Code of Conduct

    SNBForums is a community for everyone, no matter what their level of experience.

    Please be tolerant and patient of others, especially newcomers. We are all here to share and learn!

    The rules are simple: Be patient, be nice, be helpful or be gone!

bhall7

bhall7

Occasional Visitor
One thing I really miss from Tomato that I wish were available in Asuswrt-Merlin is the ability to show recently visited web sites and web searches (Under Status->Web Usage). It was a quick way to keep an eye on what the kids were searching for and the sites they were visiting.

Does such an option in Asuswrt-Merlin exist? I see the Traffic Manager feature which gives statistics about how much data is being sent/received per device, but I can't seem to drill down to find out what sites it is actually accessing, nor can I see recently entered web searches.
 
bhall7 said:
One thing I really miss from Tomato that I wish were available in Asuswrt-Merlin is the ability to show recently visited web sites and web searches (Under Status->Web Usage). It was a quick way to keep an eye on what the kids were searching for and the sites they were visiting.

Does such an option in Asuswrt-Merlin exist? I see the Traffic Manager feature which gives statistics about how much data is being sent/received per device, but I can't seem to drill down to find out what sites it is actually accessing, nor can I see recently entered web searches.
Click to expand...

I use OpenDNS for that. And with the new DNS filtering in latest Merlin fw it's a great way to monitor and control what the kids are up to and what they would have been up if I didn't check and control access! :)
 
I use OpenDNS (free offering for home users), and I currently filter for web content based on different categories. However, I've found that their reporting tends to be sporadic and not very robust.

Tell me more about how you are using the "DNS filtering" in Asuswrt-Merlin firmware.
 
Hi,

OpenDNS does not offer to track the timestamp of DNS requests, which would be important for a consistent monitoring.

Is it possible with Asuswrt-Merlin to log the information as Tomato does, i.e.:

Tue, 15 Jul 2014, 21:55:26 192.168.0.1 clients1.google.com
Tue, 15 Jul 2014, 21:55:19 192.168.0.1 crl.geotrust.com
Tue, 15 Jul 2014, 21:55:12 192.168.0.1 crl3.digicert.com
Tue, 15 Jul 2014, 21:55:12 192.168.0.1 ocsp.digicert.com
Tue, 15 Jul 2014, 21:54:06 192.168.0.1 ocsp.thawte.com
Tue, 15 Jul 2014, 21:38:08 192.168.0.181 ocsp.digicert.com

If yes, what are the necessary configuration steps? Logging onto an attached USB stick or harddisk would be the best option from my point of view.

Cheers,
Martin
 
You could tell dnsmasq to log all DNS requests but it will be very verbose!
Jul 15 19:34:21 dnsmasq[674]: query[A] d1af033869koo7.cloudfront.net from 192.168.1.193
Jul 15 19:34:21 dnsmasq[674]: forwarded d1af033869koo7.cloudfront.net to 208.67.222.222
Jul 15 19:34:21 dnsmasq[674]: reply d1af033869koo7.cloudfront.net is 54.230.3.159
Jul 15 19:34:21 dnsmasq[674]: reply d1af033869koo7.cloudfront.net is 54.230.3.29
Jul 15 19:34:21 dnsmasq[674]: reply d1af033869koo7.cloudfront.net is 54.230.0.209
Jul 15 19:34:21 dnsmasq[674]: reply d1af033869koo7.cloudfront.net is 54.230.1.221
Jul 15 19:34:21 dnsmasq[674]: reply d1af033869koo7.cloudfront.net is 54.230.3.158
Jul 15 19:34:21 dnsmasq[674]: reply d1af033869koo7.cloudfront.net is 54.230.0.236
Jul 15 19:34:21 dnsmasq[674]: reply d1af033869koo7.cloudfront.net is 54.230.1.26
Jul 15 19:34:21 dnsmasq[674]: reply d1af033869koo7.cloudfront.net is 54.230.0.182
Jul 15 19:34:21 dnsmasq[674]: query[A] t2.stormiq.com from 192.168.1.193
Jul 15 19:34:21 dnsmasq[674]: forwarded t2.stormiq.com to 208.67.222.222
Jul 15 19:34:21 dnsmasq[674]: reply t2.stormiq.com is 109.72.216.70
Jul 15 19:34:21 dnsmasq[674]: reply t2.stormiq.com is 109.73.68.219
Jul 15 19:34:21 dnsmasq[674]: reply t2.stormiq.com is 37.220.30.11
Jul 15 19:34:21 dnsmasq[674]: query[A] argospsp.px.247inc.net from 192.168.1.193
Jul 15 19:34:21 dnsmasq[674]: forwarded argospsp.px.247inc.net to 208.67.222.222
Jul 15 19:34:21 dnsmasq[674]: query[A] platform-chat.portal.gslb.247-inc.net from 192.168.1.193
Jul 15 19:34:21 dnsmasq[674]: forwarded platform-chat.portal.gslb.247-inc.net to 208.67.222.222
Jul 15 19:34:21 dnsmasq[674]: query[A] argos.tmvtp.com from 192.168.1.193
Jul 15 19:34:21 dnsmasq[674]: forwarded argos.tmvtp.com to 208.67.222.222
Jul 15 19:34:21 dnsmasq[674]: reply argospsp.px.247inc.net is 54.195.253.170
Jul 15 19:34:21 dnsmasq[674]: reply argos.tmvtp.com is 162.13.102.90
Jul 15 19:34:22 dnsmasq[674]: query[A] metrics.argos.co.uk from 192.168.1.193
Jul 15 19:34:22 dnsmasq[674]: forwarded metrics.argos.co.uk to 208.67.222.222
Jul 15 19:34:22 dnsmasq[674]: reply metrics.argos.co.uk is <CNAME>
Jul 15 19:34:22 dnsmasq[674]: reply argos.co.uk.112.2o7.net is 66.235.139.204
Jul 15 19:34:22 dnsmasq[674]: reply argos.co.uk.112.2o7.net is 66.235.139.206
Jul 15 19:34:22 dnsmasq[674]: reply argos.co.uk.112.2o7.net is 66.235.139.152
Jul 15 19:34:22 dnsmasq[674]: reply argos.co.uk.112.2o7.net is 66.235.139.110
Jul 15 19:34:22 dnsmasq[674]: reply argos.co.uk.112.2o7.net is 66.235.138.18
Jul 15 19:34:22 dnsmasq[674]: reply argos.co.uk.112.2o7.net is 66.235.138.19
Jul 15 19:34:22 dnsmasq[674]: reply argos.co.uk.112.2o7.net is 66.235.138.226
Jul 15 19:34:22 dnsmasq[674]: reply argos.co.uk.112.2o7.net is 66.235.138.224
Jul 15 19:34:22 dnsmasq[674]: reply argos.co.uk.112.2o7.net is 66.235.138.194
Jul 15 19:34:22 dnsmasq[674]: reply argos.co.uk.112.2o7.net is 66.235.138.192
Jul 15 19:34:22 dnsmasq[674]: reply argos.co.uk.112.2o7.net is 66.235.138.193
Jul 15 19:34:22 dnsmasq[674]: reply argos.co.uk.112.2o7.net is 66.235.138.195
Jul 15 19:34:22 dnsmasq[674]: reply argos.co.uk.112.2o7.net is 66.235.139.205
Jul 15 19:34:22 dnsmasq[674]: reply argos.co.uk.112.2o7.net is 66.235.139.207
Jul 15 19:34:22 dnsmasq[674]: reply platform-chat.portal.gslb.247-inc.net is 66.170.126.207
Jul 15 19:34:22 dnsmasq[674]: query[A] ib.adnxs.com from 192.168.1.193
Click to expand...
 
me too!

I'm looking for the exact same thing.

I played briefly with the log-queries option for dsnmasq, but it produced massive amouts of output, so that doesn't seem like a good option.

How do you think the Tomato variants did it? Modifying dnsmasq?

What about an entware option? A proxy maybe. Squid seems like overkill. Maybe TinyProxy?
 
stickmonster said:
I'm looking for the exact same thing.

I played briefly with the log-queries option for dsnmasq, but it produced massive amouts of output, so that doesn't seem like a good option.

How do you think the Tomato variants did it? Modifying dnsmasq?

What about an entware option? A proxy maybe. Squid seems like overkill. Maybe TinyProxy?
Click to expand...

They use the ipt_webmon kernel module.

The module is present in Asuswrt-Merlin, but I never implemented support for it. You'd have to do it manually.
 
Hey, Thanks for the reply Merlin!

The module seems to load fine with modprobe and can be seen with lsmod.

What would be next? Is there a daemon or something I need to bring in to interact with it? Or is loading the module supposed to generate additional logs? Or something else?

I'd really appreciate if you could point me in the right direction.

Thanks
 
Last edited:
stickmonster said:
Hey, Thanks for the reply Merlin!

The module seems to load fine with modprobe and can be seen with lsmod.

What would be next? Is there a daemon or something I need to bring in to interact with it? Or is loading the module supposed to generate additional logs? Or something else?

I'd really appreciate if you could point me in the right direction.

Thanks
Click to expand...

It's been months since I've looked at how this module works, so I can't tell for sure, sorry. I think it adds a new virtual device (probably under /proc) that you can read to view a list of visited websites.
 
If anyone gets this to run and can give a step-by-step instruction, this would be really cool - this is the only feature I am really missing in the Asus/Merlin versions.
 
I haven't been able to get this to work either.
Code: 
# modprobe ipt_webmon
# ls -l /proc/webmon_recent_*
-r--r--r--    1 admin    root             0 Sep 21 21:03 /proc/webmon_recent_domains
-r--r--r--    1 admin    root             0 Sep 21 21:03 /proc/webmon_recent_searches
# cat /proc/net/ip_tables_matches
webmon
account
u32
icmp
tcpmss
time
webstr
string
state
recent
multiport
multiport
mark
mac
limit
iprange
iprange
conntrack
connmark
connlimit
connbytes
udplite
udp
tcp
# iptables -m webmon --help
iptables v1.3.8: Couldn't load match `webmon':File not found

Try `iptables -h' or 'iptables --help' for more information.
#

UPDATE: I think the message is telling me that this file is missing (which it is):

/usr/lib/iptables/libipt_webmon.so
 
Last edited:
Yes, /proc/webmon_recent_domains and /proc/webmon_recent_searches are the files.

I've also observed that enabling the web usage on tomato modifies the iptables rules--they add a monitor chain which references WEBMON. I think this is important. I believe enabling it also does a "service logging restart" (or something close to that). Isn't there a shell script or plan text file where you can see the details of what happens on the service restarts? I thought I stumbled across something like that once, but now I can't find it.

Colin, it seems like you are on the same path, and I think I'm tracking what you are saying. I can see that /usr/lib/iptables/libipt_webmon.so is missing in MerlinWRT. Is there a way to add a file to that directory, or append to a "libpath" or something (entware's lib directory?), so that we can try adding that file ourselves?

Otherwise we'd be stuck, without Merlin's help. (I'm not interested in compiling my own firmware :)).
 
Last edited:
# cat /etc/ld.so.conf
/opt/lib
/opt/usr/lib
/lib
/usr/lib

so I copy to file from tomato to /opt/usr/lib, chmod +x, still no joy.

#export LD_LIBRARY_PATH=/tmp/mnt/opt/entware/usr/lib
#iptables -m webmon --help

no error message! there's no help for webmon (on tomato either) but at the end of a couple of pages of output:

webmon options:

just like tomato!

I found this snippet in /etc/iptables (on tomato), but I don't know how to do the equivalent on the command line:

:monitor - [0:0]
-A FORWARD -o vlan1 -j monitor
-A monitor -p tcp -m webmon --max_domains 2000 --max_searches 2000 --domain_load_file /var/webmon/domain --search_load_file /var/webmon/search -j RETURN

Colin, do you know the magic iptables commands to add the necessary rules?
 
Last edited:
Try this:

Code: 
# iptables -N monitor
# iptables -A FORWARD -o eth0 -j monitor
# iptables -A monitor -p tcp -m webmon --max_domains 2000 --max_searches 2000 -j RETURN


webmon_gargoyle.init (https://www.openwrt.org.cn/browser/...n-gargoyle/files/webmon_gargoyle.init?rev=151) has something similar, so:
Code: 
iptables -t filter -N web_monitor
iptables -t filter -I FORWARD -o eth0 -j web_monitor
iptables -t filter -I web_monitor -m webmon --max_domains 2000 --max_searches 2000
Although I don't really understand how the gargoyle script can work without an "iptables -t filter -A web_monitor -j RETURN" at the end.
 
Last edited:
:(

Code: 
# iptables -A monitor -p tcp -m asdf --max_domains 2000 --max_searches 2000 -j RETURN
iptables v1.3.8: Couldn't load match `asdf':File not found

Try `iptables -h' or 'iptables --help' for more information.
# iptables -A monitor -p tcp -m webmon --max_domains 2000 --max_searches 2000 -j RETURN
iptables: Invalid argument
# iptables -A monitor -p tcp -m webmon --max_domains 2000 --max_searches 2000
iptables: Invalid argument
# iptables -A monitor -p tcp -m webmon
iptables: Invalid argument
# iptables -A monitor -p tcp
#

(no error message after that last step)
 
Here is what i typed.

Code: 
modprobe ipt_webmon
iptables -t filter -N web_monitor
iptables -t filter -I FORWARD -o eth0 -j web_monitor
iptables -t filter -I web_monitor -m webmon --max_domains 2000 --max_searches 2000

then
cat /proc/webmon_recent_domains

I get this output from the cat command

1411439312 10.0.0.212 geo.yahoo.com

So it seems to work. sort of.
 
I get the same error I got before on that last command.

I think my /usr/lib/iptables/libipt_webmon.so might be bad. I got mine from a recent shibby build for the K24 kernel, since I flashed tomato to an old WRT54L. Where did you get yours?

By the way, that first column in a unix epoc, which can be converted to a date/time. See http://www.epochconverter.com/.
 
ColinTaylor said:
Although I don't really understand how the gargoyle script can work without an "iptables -t filter -A web_monitor -j RETURN" at the end.
Click to expand...
I've just read the documentation :eek: and the default action when reaching the end of a subchain is to RETURN.
 
@vdemarco
Can you tell us more about your config? What build/version are you running? If you copied over the library manually, where did you get it?

@those following along :)

I tried the iptables command on shibby (flashed to a K26 version) and if I don't include "-p tcp" that last step always returns "Invalid argument" and the system log will contain
Code: 
Dec 31 16:56:57 unknown user.warn kernel: ip_tables: webmon match: only valid for protocol 6

Otherwise, either variation of that command succeeds.

On Merlin, no matter what I try on that last step I get "Invalid argument", and system log will contain
Code: 
Sep 23 19:48:12 kernel: ip_tables: webmon match: invalid size 1176 != 16

Any clue what that means?
 
stickmonster said:
@vdemarco
Can you tell us more about your config? What build/version are you running? If you copied over the library manually, where did you get it?

@those following along :)
Click to expand...

I have stock merlin 3.0.0.4.376.47 on a RT-AC66U. the kernel module came with the build.
 
You must log in or register to reply here.

Similar threads

dev_null
  • Locked
[Beta] VNSTAT on Merlin; CLI, UI and email data use monitoring (install script offline for now)
5 6 7
Replies
121
Views
19K
dev_null
dev_null
dev_null
vnStat [Release] vnStat-on-Merlin - UI, CLI and email - data use and data limit monitoring - R1 and R2
20 21 22
Replies
429
Views
59K
Tarek Yag
Tarek Yag
R
RT-AC3100 won't restart with Asuswrt-Merlin 382.1
Replies
5
Views
4K
aploch
aploch
Shaun4BigBlocks
ASUS RT-AC88U VPN WAN Issue After Merlin Flash
Replies
4
Views
2K
Shaun4BigBlocks
Shaun4BigBlocks
RMerlin
  • Locked
  • Sticky
[FAQ] READ ME FIRST before posting a question
Replies
2
Views
91K
RMerlin
RMerlin
Similar threads
Thread starter Title Forum Replies Date
v.y.k Traffic Monitoring - Save frequency - issue Asuswrt-Merlin 7
delikid package to expand Parental Controls - Web & Apps Filters? Asuswrt-Merlin 4
L Is firmware reinstallation in recovery mode deeper than on the web interface? Asuswrt-Merlin 20
L Feature suggest: web notifications Asuswrt-Merlin 1
D Feature request: Two factor authentication web login. (TOTP) Asuswrt-Merlin 8
L dynamic cgnat ip is changing correctly with ddns custom script but shown ip in web interface never changes Asuswrt-Merlin 5
cyruz Reversing the web node reboot feature Asuswrt-Merlin 0
S restart web server from SSH Asuswrt-Merlin 13
U Solved Cannot upload self-signed cert for web UI Asuswrt-Merlin 11
M Allow admin web GUI access on another interface (tailscale0) Asuswrt-Merlin 0

Similar threads

Latest threads

Support SNBForums w/ Amazon

If you'd like to support SNBForums, just use this link and buy anything on Amazon. Thanks!

Sign Up For SNBForums Daily Digest

Get an update of what's new every day delivered to your mailbox. Sign up here!

Members online

Total: 842 (members: 27, guests: 815)
Top