Domain-based VPN Routing Script

[Kyjiep]

It is hard to say. I would need you to provide full memory specs. while you are in the thick of all of it for me to warrant the idea that it wasn't memory related. While there may be the presence of available memory, it does not necessarily mean there is enough for the initial launching of everything. AdGuardHome requires atleast 50mb of free ram. which typically is fine for our routers. However DNSMASQ log has been known to have buggy memory issues lately. Particularly around startup and spikes in memory usage.

take a look at this post:

Dnsmasq-full gets killed by out of memory on 21.02 (64 MB RAM)

Hi, I see dnsmasq causing OOM and getting killed at least once a day after upgrading to 21.02 from 19.07. Sometimes I see multiple OOM messages in a row (with the same timestamp) but different dnsmasq PID. My router is Asus N14U with MT7620/64 MB RAM/16 MB flash. I have dnsmasq-full which is...

forum.openwrt.org

This is how it was with working AdGuardHome. Immediately after rebooting the router, I have about 460 MB of free memory, over time the amount of free memory decreases, in a week it is reduced to 250-300 MB. That's why I have the system auto-rebooted once a week. When I turned on DNSMASQ log and rebooted the router, the free memory began to jump a little around 420-450 MB, and I described all the other symptoms earlier.
I don’t currently have AdGuardHome installed, but if necessary, I can install it again and experiment to provide you with the other necessary data.

 

[SomeWhereOverTheRainBow]

This is how it was with working AdGuardHome. Immediately after rebooting the router, I have about 460 MB of free memory, over time the amount of free memory decreases, in a week it is reduced to 250-300 MB. That's why I have the system auto-rebooted once a week. When I turned on DNSMASQ log and rebooted the router, the free memory began to jump a little around 420-450 MB, and I described all the other symptoms earlier.
I don’t currently have AdGuardHome installed, but if necessary, I can install it again and experiment to provide you with the other necessary data.

What all odd behaviors do you remember when issues started. Any strange dnsmasq tainted logs? It is hard to detect because dnsmasq restarts itself. AGuardhome will not start because it shutsdown by default in OOM situations due to behaviors of GO binaries, but DNSMASQ restarts itself and resumes like nothing every happen. The only indications of DNSMASQ OOM failures is in the system logs.

 

[Kyjiep]

What all odd behaviors do you remember when issues started. Any strange dnsmasq tainted logs? It is hard to detect because dnsmasq restarts itself.

If you don't mind, let's move the discussion of this issue to Conversations.

 

[SomeWhereOverTheRainBow]

If you don't mind, let's move the discussion of this issue to Conversations.

ok.

 

[SomeWhereOverTheRainBow]

Add the following lines to /jffs/configs/dnsmasq.conf.add and restart the dnsmasq service

log-queries
log-facility=/var/log/dnsmasq.log

It works. But, only if I uninstall Adguard Home. With Adguard Home installed, the router's processor goes crazy, all installed addons and DDNS do not work. Is it possible to somehow connect this with Adguard Home?

Just to let you know it now works.

log-queries
log-facility=/var/log/dnsmasq.log
port=553
local=/168.192.in-addr.arpa/
local=/10.in-addr.arpa/
local=//
dhcp-option=lan,6,0.0.0.0

I may add

log-queries
log-facility=/var/log/dnsmasq.log

for AdGuardHome. I am still tossing the idea around. But I can confirm what the problem was. Dnsmasq was being stubborn on giving up port 53, hence AdGuardHome was not able to start.

For anyone curious in knowing AdGuardHome is able to read IPsets from a file.

• ipset: List of domain-ipset_name associations for adding IP addresses ofthe specified domain names to an ipset list. Syntax:
  DOMAIN[,DOMAIN].../IPSET_NAME[,IPSET_NAME]
  IPv4 addresses are added to an ipset list with ipv4 family; IPv6addresses, to an ipv6 ipset list. ipset list must exist.
  This setting is supported on Linux OSs only. This feature is similar to--ipset in dnsmasq.
• ipset_file (since v0.107.13): Same as ipset, but the rules areread from a file. If this property is set, property ipset is ignored.

You can read abit more about IPsets on their wiki in the "other settings" section on the configuration page.

Configuration

Network-wide ads & trackers blocking DNS server. Contribute to AdguardTeam/AdGuardHome development by creating an account on GitHub.

github.com

 

[Ranger802004]

Just to let you know it now works.

log-queries
log-facility=/var/log/dnsmasq.log
port=553
local=/168.192.in-addr.arpa/
local=/10.in-addr.arpa/
local=//
dhcp-option=lan,6,0.0.0.0

I may add

log-queries
log-facility=/var/log/dnsmasq.log

for AdGuardHome. I am still tossing the idea around. But I can confirm what the problem was. Dnsmasq was being stubborn on giving up port 53, hence AdGuardHome was not able to start.

Is there anything you need me to do on my end or all is well?

 

[SomeWhereOverTheRainBow]

Is there anything you need me to do on my end or all is well?

At the moment, I currently don't have all these intricate VPN setups, so it is really hard for me to test when Adguardhome is conflicted with this type of script. The edits I have made should resolve a conflict issue with Adguardhome not being able to start due to not acquiring port 53. One piece of info I wouldn't mind sharing though is AdGuardHome can read IPsets from a file, or store individual Ipsets similar to how dnsmasq does. I edited the previous post sharing the details about it.

 

[Kyjiep]

Just to let you know it now works.

log-queries
log-facility=/var/log/dnsmasq.log
port=553
local=/168.192.in-addr.arpa/
local=/10.in-addr.arpa/
local=//
dhcp-option=lan,6,0.0.0.0

I may add

log-queries
log-facility=/var/log/dnsmasq.log

for AdGuardHome. I am still tossing the idea around. But I can confirm what the problem was. Dnsmasq was being stubborn on giving up port 53, hence AdGuardHome was not able to start.

For anyone curious in knowing AdGuardHome is able to read IPsets from a file.



You can read abit more about IPsets on their wiki in the "other settings" section on the configuration page.

Configuration

Network-wide ads & trackers blocking DNS server. Contribute to AdguardTeam/AdGuardHome development by creating an account on GitHub.

github.com

Unfortunately, nothing has changed for me. I also noticed that in addition to the dnsmasq.log file, the dnsmasq.logport=553 file also appeared, and dnsmasq logs began to be written to it. But it seems to me that this interferes with the work of Domain-based VPN Routing Script, because it will look for logs in the dnsmasq.log file.
In the next week, I will not be able to participate in searching for the source of this bug and testing ways to eliminate it, because as of today I am on a business trip 4000 kilometers from home. It’s not convenient for me to do this remotely, I don’t have enough time and I’m afraid I’ll break something in the router settings and my family will be left without the Internet until I return. In a week I will return home and can try again, but for now I uninstalled AdGuardHome again for the sake of normal operation of the Domain-based VPN Routing Script with DNSMASQ log.
And thank you for not leaving this problem unattended.

 

[Ranger802004]

At the moment, I currently don't have all these intricate VPN setups, so it is really hard for me to test when Adguardhome is conflicted with this type of script. The edits I have made should resolve a conflict issue with Adguardhome not being able to start due to not acquiring port 53. One piece of info I wouldn't mind sharing though is AdGuardHome can read IPsets from a file, or store individual Ipsets similar to how dnsmasq does. I edited the previous post sharing the details about it.

The only thing this script is doing is reading the dnsmasq log if one is configured, it’s an optional configuration.

 

[SomeWhereOverTheRainBow]

@chongnt Made a modification script that allows for transferring IPSETS from dnsmasq to AdGuardHome. Idk if he is still around, but he is welcome to share what he did. Maybe it can trigger a collaboration here.

​

 

[SomeWhereOverTheRainBow]

The only thing this script is doing is reading the dnsmasq log if one is configured, it’s an optional configuration.

I am having a hard time understanding if your feature would be "useful" when using AdGuardHome since AdGuardHome is taking over port 53. The only exchanges I would think would be in the dnsmasq log would be local requests from AdGuardHome back to dnsmasq requesting for client information e.g. arp requests..

here is the tail of DNSMASQ log with AdGuardHome.

Oct 26 07:54:43 dnsmasq[805764]: query[AAAA] amazon-a40008af0.Some-Network from ::1
Oct 26 07:54:43 dnsmasq[805764]: DHCP amazon-a40008af0.Some-Network is dXXX:b9XX:XXX:XXX:XXX:XXfX:XXXX:XXX
Oct 26 07:54:43 dnsmasq[805764]: query[A] amazon-a40008af0.Some-Network from ::1
Oct 26 07:54:43 dnsmasq[805764]: DHCP amazon-a40008af0.Some-Network is 192.168.1.170
Oct 26 07:54:57 dnsmasq[805764]: query[AAAA] osb-v1-alb.samsungqbe.com.Some-Network from ::1
Oct 26 07:54:57 dnsmasq[805764]: config osb-v1-alb.samsungqbe.com.Some-Network is NXDOMAIN
Oct 26 07:54:59 dnsmasq[805764]: query[AAAA] localhost.Some-Network from ::1
Oct 26 07:54:59 dnsmasq[805764]: config localhost.Some-Network is NODATA-IPv6
Oct 26 07:54:59 dnsmasq[805764]: query[AAAA] localhost from ::1
Oct 26 07:54:59 dnsmasq[805764]: config localhost is NODATA-IPv6

 

[SomeWhereOverTheRainBow]

If DNSMASQ is to be useful in this kind of way with AdGuardHome in the middle, then AdGuardHome would have to send all its request to DNSMASQ as its upstream. Either that, or IPSETS would have to be adapted to be usable with AdGuardHome.

e.g. If I change my upstream to below:

Then DNSMASQ becomes useful for your domain scripting the way it is now. However, this means users will not be able to use any other upstream than DNSMASQ since it is controlling the IPSet and query logs. Queries to any other upstream will break the chain.

 

[Ranger802004]

If DNSMASQ is to be useful in this kind of way with AdGuardHome in the middle, then AdGuardHome would have to send all its request to DNSMASQ as its upstream. Either that, or IPSETS would have to be adapted to be usable with AdGuardHome.

e.g. If I change my upstream to below:

View attachment 53840

Then DNSMASQ becomes useful for your domain scripting the way it is now. However, this means users will not be able to use any other upstream than DNSMASQ since it is controlling the IPSet and query logs. Queries to any other upstream will break the chain.

Is there a query log for AdGuardHome?

 

[SomeWhereOverTheRainBow]

Is there a query log for AdGuardHome?

Yes, but to have a readable (a.k.a script readable) query log, I believe it has to be enabled in the .yaml configuration file. Similar to how it is configured for other proxy type servers(e.g for dnscrypt-proxy it is configured in the .toml config file).

So far from what I can tell:

Before v0.107.24 query log settings were part of the dns object.
Since v0.107.24 query log settings are part of querylog object.

So I will have to investigate this to see if it is script readable.

 

[Kyjiep]

Perhaps this article will help you, although it is in Russian and for the Keenetic router. But, in my opinion, there is useful information there. Just in case, I’ll leave a link here, Google translator will help you if you want to read it. https://pimpmykeenetic.github.io/2022/12/17/adguardhome-and-selective-routing.html
If you think it doesn't belong here, then excuse me and just delete it.

 

[SomeWhereOverTheRainBow]

Perhaps this article will help you, although it is in Russian and for the Keenetic router. But, in my opinion, there is useful information there. Just in case, I’ll leave a link here, Google translator will help you if you want to read it. https://pimpmykeenetic.github.io/2022/12/17/adguardhome-and-selective-routing.html
If you think it doesn't belong here, then excuse me and just delete it.

So I get the direction of the guide. It is for a very specific vpn use case. However, it is helpful because it demonstrates the use of the adguardhome ipset file. It would still be some work adapting the script for asuswrt vpn usage. I am not saying it isn't doable, but the approach would be slightly different, but slightly similar.

 

[Ranger802004]

So I get the direction of the guide. It is for a very specific vpn use case. However, it is helpful because it demonstrates the use of the adguardhome ipset file. It would still be some work adapting the script for asuswrt vpn usage. I am not saying it isn't doable, but the approach would be slightly different, but slightly similar.

I just need to be able to get log entries for host name > IPs and then I can add them to my relative policies and ipsets.

 

[SomeWhereOverTheRainBow]

I just need to be able to get log entries for host name > IPs and then I can add them to my relative policies and ipsets.

From looking at the log, it appears to only show the requesters IP, and not the IP of the domain requested.

awk -F ',"IP":"' '{split($2, ip, "\""); if(!x[ip[1]]++)print ip[1]}' /opt/etc/AdGuardHome/data/querylog.json is what I used to determine this. It provided a list of local IP addresses.

here is an example of what a single log entry looks like.

{"T":"2023-10-26T19:58:43.882061771-04:00","QH":"nrdp-ipv4.prod.ftl.netflix.com","QT":"A","QC":"IN","CP":"","Upstream":"1.1.1.1:53","Answer":"kuKBgAABAAcAAAAACW5yZHAtaXB2NARwcm9kA2Z0bAduZXRmbGl4A2NvbQAAAQABwAwABQABAAFRgAARDm5yZHAtaXhhbnljYXN0wBvAPAAFAAEAAAEsABMEbnJkcARwcm9kBmRyYWRpc8AfwFkABQABAAAAPAAfBG5yZHAEcHJvZAl1cy1lYXN0LTEIaW50ZXJuYWzAY8B4AAUAAQAAADwARClhcGlwcm94eS1ucmRwLXByb2QtbmxiLTMtMGFlNzQ3MWQxYmRiYzE4MQNlbGIJdXMtZWFzdC0xCWFtYXpvbmF3c8AnwKMAAQABAAAAPAAEFxZMYsCjAAEAAQAAADwABDRXfuTAowABAAEAAAA8AARILC/J","IP":"192.168.1.94","Result":{},"Elapsed":62562067}

The query log is printed in a .json format. It may even be better to parse it with jq package but that depends on entware. I used awk in my test case.

The command above returns "192.168.1.94" for the output of the awk command.

RT-AX88U:/opt/etc/AdGuardHome# awk -F ',"IP":"' -v VAR='nrdp-ipv4.prod.ftl.netflix.com' '$0 ~ VAR{split($2, ip, "\""); if(!x[ip[1]]++)print ip[1]}' /opt/etc/AdGuardHome/data/querylog.json
192.168.1.94

You could technically take any domain match and find the requestor IP.

awk -F ',"IP":"' -v VAR="$DOMAIN" '$0 ~ VAR{split($2, ip, "\""); if(!x[ip[1]]++)print ip[1]}' /opt/etc/AdGuardHome/data/querylog.json

HTH.

If not, then atleast you know the location of the querylog for AdGuardHome.

 

[Ranger802004]

From looking at the log, it appears to only show the requesters IP, and not the IP of the domain requested.

awk -F ',"IP":"' '{split($2, ip, "\""); if(!x[ip[1]]++)print ip[1]}' /opt/etc/AdGuardHome/data/querylog.json is what I used to determine this. It provided a list of local IP addresses.

here is an example of what a single log entry looks like.

{"T":"2023-10-26T19:58:43.882061771-04:00","QH":"nrdp-ipv4.prod.ftl.netflix.com","QT":"A","QC":"IN","CP":"","Upstream":"1.1.1.1:53","Answer":"kuKBgAABAAcAAAAACW5yZHAtaXB2NARwcm9kA2Z0bAduZXRmbGl4A2NvbQAAAQABwAwABQABAAFRgAARDm5yZHAtaXhhbnljYXN0wBvAPAAFAAEAAAEsABMEbnJkcARwcm9kBmRyYWRpc8AfwFkABQABAAAAPAAfBG5yZHAEcHJvZAl1cy1lYXN0LTEIaW50ZXJuYWzAY8B4AAUAAQAAADwARClhcGlwcm94eS1ucmRwLXByb2QtbmxiLTMtMGFlNzQ3MWQxYmRiYzE4MQNlbGIJdXMtZWFzdC0xCWFtYXpvbmF3c8AnwKMAAQABAAAAPAAEFxZMYsCjAAEAAQAAADwABDRXfuTAowABAAEAAAA8AARILC/J","IP":"192.168.1.94","Result":{},"Elapsed":62562067}

The query log is printed in a .json format. It may even be better to parse it with jq package but that depends on entware. I used awk in my test case.

The command above returns "192.168.1.94" for the output of the awk command.

RT-AX88U:/opt/etc/AdGuardHome# awk -F ',"IP":"' -v VAR='nrdp-ipv4.prod.ftl.netflix.com' '$0 ~ VAR{split($2, ip, "\""); if(!x[ip[1]]++)print ip[1]}' /opt/etc/AdGuardHome/data/querylog.json
192.168.1.94

You could technically take any domain match and find the requestor IP.

awk -F ',"IP":"' -v VAR="$DOMAIN" '$0 ~ VAR{split($2, ip, "\""); if(!x[ip[1]]++)print ip[1]}' /opt/etc/AdGuardHome/data/querylog.json

HTH.

If not, then atleast you know the location of the querylog for AdGuardHome.

Yea I've worked with plenty of JSON parsing using awk instead of python or jq at main job so I can manage, lol.

 

[SomeWhereOverTheRainBow]

Yea I've worked with plenty of JSON parsing using awk instead of python or jq at main job so I can manage, lol.

Haha, and I thought you only played with awk as an enthusiast. JK. I know you know your stuff. While I don't have any VPN's to play along the way, I am free to answer questions and consult as needed on this matter.