AdGuardHome Latest Entware update possibly made AdGuard Home fail

[Markfree]

Today I had a weird issue with AdGuard Home in two AX86U routers.

This night, all of a sudden, AdGuard stopped working after a reboot and did not start at all.
Thus, I was mostly disconnected from the Internet.

When I checked the service, it was "dead" and nothing would revive it.

# /opt/etc/init.d/S99AdGuardHome check
 Checking AdGuardHome...              dead.

Rebooting the router had no effect.

When checking AdGuard logs, they only showed the last time the app was stopped. (during the night)

Mar 17 03:59:26 hostname AdGuardHome[5217]: 2023/03/17 03:59:26.819288 [info] Updated filter #1665644498.  Rules: 1861 -> 1861
Mar 17 03:59:26 hostname AdGuardHome[5217]: 2023/03/17 03:59:26.819306 [info] Updated filter #1678320236.  Rules: 50615 -> 50720
Mar 17 03:59:26 hostname AdGuardHome[5217]: 2023/03/17 03:59:26.819329 [info] Updated filter #1678321506.  Rules: 6180 -> 6180
Mar 17 04:00:01 hostname rc_service: service 886768:notify_rc stop_AdGuardHome

Only after restarting DNSMasq that I managed to access Internet again.

# service restart_dnsmasq
Done.

I tried to reinstall AdGuard Home but it froze when starting up again.

Info:  This operation will install AdGuardHome and related files (<6MB)
Info:  to ENTWARE, no other data will be changed.
Info:  Also some start scripts will be installed/modified as required.
=>  Do you want to install AdGuardHome to ENTWARE? [y/n]: y
Info:  installer is up to date. Skipping...
=>  Do you want to switch AdGuardHome builds? [y/n]: n
Info:  continuing without changing builds.
Info:  ADGUARDHOME_BUILD=Release
Info:  No new release version available.
Info:  ADGUARDHOME_VER=v0.107.26
Info:  AdGuardHome.sh is up to date. Skipping...
Info:  S99AdGuardHome is up to date. Skipping...
Info:  rc.func.AdGuardHome is up to date. Skipping...
Info:  init-start file already configured
Info:  services-stop file already configured
Info:  dnsmasq.postconf file already configured
Info:  Configure service-event-end file
Info:  Configuring AdGuardHome...
=>  Do you want to redirect all DNS resolutions on your network through to AdGuardHome? [y/n]: y
Info:  You can choose to keep any custom dnsfilter values by only redirect non-custom traffic or send all traffic through to AdGuardHome.
=>  Do you want to redirect only NON-CUSTOM DNS resolutions on your network through to AdGuardHome? [y/n]: n
Info:  DNS is set to redirect All DNS resolutions through to AdGuardHome.
Info:  Checking AdGuardHome configuration...
=>  Do you want to skip reconfiguration options? [y/n]: y
Info:  Skipping reconfiguration options.
Info:  Starting AdGuardHome...

After lots of searching, I remembered that yesterday there was an update through AMTM for many Entware packages. And only after rebooting the router that the error showed up.

So, I tried to updated the router again and found a new update for libopenssl.
The package was updated, I rebooted the router, then, finally, AdGuard started up.

I guess libopenssl was faulty.

Did anyone else have similar issues?

 

[The1Fear]

I'm having the exact same issue, after updating entware and rebooting the router, adguard stopped working.

Would you mind letting me know how did you update libopenssl package? If you are talking about one of entware packages, I tried updating them again but no luck and now I'm stuck.

 

[ryzhov_al]

AdGuardHome configuration schema has been changed. Please, rename original /opt/etc/AdGuardHome/AdGuardHome.yaml and start AGH to get a new one.

 

[Markfree]

I'm having the exact same issue, after updating entware and rebooting the router, adguard stopped working.
Would you mind letting me know how did you update libopenssl package? If you are talking about one of entware packages, I tried updating them again but no luck and now I'm stuck.

I upgraded Entware packages using AMTM, Entware option.
This updated libopenssl to version 3.0.8-2a. After such, AdGuard got back up.

I also tried to reinstall AdGuard. So, that might have updated the configuration file.

 

[StarkWiz]

I have posted a possible solution for this issue in a different thread.
Link to Fix

 

[zyxmon]

I have posted a possible solution for this issue in a different thread.
Link to Fix

adguard is linked staticaly (go binary) and does not depent on external *.so

In my case there was a problem in ipset file. One line had a wrong syntax - it passed in the previous AdGuard version and crashed the new one.

 

[SomeWhereOverTheRainBow]

adguard is linked staticaly (go binary) and does not depent on external *.so

In my case there was a problem in ipset file. One line had a wrong syntax - it passed in the previous AdGuard version and crashed the new one.

I concur, AdGuardHome doesn't actually use any of these other binaries from Entware. AdGuardHome breaking could be directly related to changes in the .yaml schema though.

as mentioned by:

AdGuardHome configuration schema has been changed. Please, rename original /opt/etc/AdGuardHome/AdGuardHome.yaml and start AGH to get a new one.

The installer will now use schema 16, actually 17. Next release it will need to be changed to 19 (the edge and beta most likely already use shema version 19).

Commits · jumpsmm7/Asuswrt-Merlin-AdGuardHome-Installer

The Official Installer of AdGuardHome for Asuswrt-Merlin - Commits · jumpsmm7/Asuswrt-Merlin-AdGuardHome-Installer

github.com

In accordance to these changes

AdGuardHome/CHANGELOG.md at master · AdguardTeam/AdGuardHome

Network-wide ads & trackers blocking DNS server. Contribute to AdguardTeam/AdGuardHome development by creating an account on GitHub.

github.com

 

[zyxmon]

Just run

. /opt/etc/AdGuardHome/adguardhome.conf ; AdGuardHome

To see what is wrong

 

[peepsnet]

Just run

. /opt/etc/AdGuardHome/adguardhome.conf ; AdGuardHome

To see what is wrong

This tells me cannot bind to :53

It seems it is not stopping dnsmasq???

 

[SomeWhereOverTheRainBow]

This tells me cannot bind to :53

It seems it is not stopping dnsmasq???

Or you are trying to run something else on port 53 like stubby or dnscrypt-proxy or nextdns. Adguardhome is not compatible with any 3rd party DNS softwares which are trying to control any bind port of 53 because adguardhome binds to 0.0.0.0:53 by default. For users, what this means is you cannot test stubby, nextdns, or dnscrypt-proxy along side adguardhome (atleast not on the same ports). Also, unbound cannot occupy any bind port of 53 either when running adguardhome on those same ports. The AMTM Adguardhome script should have no problem pushing any dnsmasq instance out of the way unless you have added your own entries to counter act this process, however it is not setup to push all of those other third party applications off port 53. The AdGuardHome AMTM script also does not use the new Entware AdGuardHome-go package. It still downloads the binaries straight from Adguardhome release locations.

Use the command below to confirm all occupants of port 53.

netstat -nlp | grep 53

If you use the adguardhome installation script in amtm then you avoid the fuss of having to fix the dnsmasq issue, but I am assuming the adguardhome-go package in entware does absolutely nothing to take care of all these configurations that must be accounted for such as dnsmasq controlling port 53.


For those users who have to stumble the path of using AdGuardHome-Go package introduced by Entware developers,
Here is a helpful tutorial that shows how to edit the AdGuardHome init.d scripts to account for RMerlin using Dnsmasq on port 53.

Tutorial - AdGuard Home - adblocker - Clean install on Asus Merlin (No 3rd party scripts !!!)

I have finally managed to install AdGuard Home on an Asus Router.... this is without using any special scripts. There was a previous forum post (here) on the same, but that involved installing using a custom script (that asked for your ssh login credentials, installed its own version of Samba...

www.snbforums.com

Any body trying to use the AdGuardHome AMTM script, you cannot use both the adguardhome-go, and the AMTM AdGuardHome installation script. It will fail. To resolve this issue, you have to do:

opkg remove adguardhome-go and possibly reinstall the AdGuardHome AMTM script.

 

[peepsnet]

Or you are trying to run something else on port 53 like stubby or dnscrypt-proxy or nextdns.

admin@router:/tmp/home/root# netstat -nlp | grep 53
netstat: showing only processes with your user ID
tcp        0      0 127.0.0.1:53            0.0.0.0:*               LISTEN      2182985/dnsmasq
tcp        0      0 192.168.1.1:53          0.0.0.0:*               LISTEN      2182985/dnsmasq
tcp        0      0 192.168.64.1:53         0.0.0.0:*               LISTEN      2182985/dnsmasq
udp        0      0 127.0.0.1:53            0.0.0.0:*                           2182985/dnsmasq
udp        0      0 192.168.1.1:53          0.0.0.0:*                           2182985/dnsmasq
udp        0      0 192.168.64.1:53         0.0.0.0:*                           2182985/dnsmasq

I originally installed adguardhome throught the AMTM install script some time ago. I always used the AMTM script to upgrade.

I upgraded the entware and the AGH bin about a week or so ago, along with moving to Asus-Merlin 388.2 beta. AGH never restarted from that and CURL failed. Even after many restarts

I was able to get curl to work again with a system link to the old file somehow(I don't remember exactly where I did created the symlink).

 

[SomeWhereOverTheRainBow]

admin@router:/tmp/home/root# netstat -nlp | grep 53
netstat: showing only processes with your user ID
tcp        0      0 127.0.0.1:53            0.0.0.0:*               LISTEN      2182985/dnsmasq
tcp        0      0 192.168.1.1:53          0.0.0.0:*               LISTEN      2182985/dnsmasq
tcp        0      0 192.168.64.1:53         0.0.0.0:*               LISTEN      2182985/dnsmasq
udp        0      0 127.0.0.1:53            0.0.0.0:*                           2182985/dnsmasq
udp        0      0 192.168.1.1:53          0.0.0.0:*                           2182985/dnsmasq
udp        0      0 192.168.64.1:53         0.0.0.0:*                           2182985/dnsmasq

I originally installed adguardhome throught the AMTM install script some time ago. I always used the AMTM script to upgrade.

I upgraded the entware and the AGH bin about a week or so ago, along with moving to Asus-Merlin 388.2 beta. AGH never restarted from that and CURL failed. Even after many restarts

I was able to get curl to work again with a system link to the old file somehow(I don't remember exactly where I did created the symlink).

Do you mind giving the output of

opkg list-installed | grep adguardhome-go

 

[peepsnet]

Do you mind giving the output of

opkg list-installed | grep adguardhome-go

admin@router:/tmp/home/root# opkg list-installed | grep adguardhome-go
admin@router:/tmp/home/root# opkg remove adguardhome-go
No packages removed.
admin@router:/tmp/home/root#

also,
I can stop dnsmasq and start adguardhome manually and it functions perfect till reboot or I close the SSH session

 

[SomeWhereOverTheRainBow]

admin@router:/tmp/home/root# opkg list-installed | grep adguardhome-go
admin@router:/tmp/home/root# opkg remove adguardhome-go
No packages removed.
admin@router:/tmp/home/root#

also,
I can stop dnsmasq and start adguardhome manually and it functions perfect till reboot or I close the SSH session

You may need to uninstall and reinstall adguardhome. There was recently an issue where the updated the schema and adguardhome stopped starting for all those who updated their adguardhome versions. The new version of the installer uses the latest schema in the .yaml file. By the proper schema not being present in the .yaml file, adguardhome will not start.

 

[peepsnet]

You may need to uninstall and reinstall adguardhome. There was recently an issue where the updated the schema and adguardhome stopped starting for all those who updated their adguardhome versions. The new version of the installer uses the latest schema in the .yaml file. By the proper schema not being present in the .yaml file, adguardhome will not start.

I did this once before a few days ago but between this issua and the libopenssl breaking curl it didn't work. I do not know which was first to fail.

I used the script DIRECTLY FROM GITHUB

GitHub - jumpsmm7/Asuswrt-Merlin-AdGuardHome-Installer: The Official Installer of AdGuardHome for Asuswrt-Merlin

The Official Installer of AdGuardHome for Asuswrt-Merlin - jumpsmm7/Asuswrt-Merlin-AdGuardHome-Installer

github.com

curl -L -s -k -O https://raw.githubusercontent.com/jumpsmm7/Asuswrt-Merlin-AdGuardHome-Installer/master/installer && sh installer

once I got curl working again and I was able to uninstall and reinstall AGH this time.

FIXED!!

 

[SomeWhereOverTheRainBow]

I did this once before a few days ago but between this issua and the libopenssl breaking curl it didn't work. I do not know which was first to fail.

I used the script DIRECTLY FROM GITHUB

GitHub - jumpsmm7/Asuswrt-Merlin-AdGuardHome-Installer: The Official Installer of AdGuardHome for Asuswrt-Merlin

The Official Installer of AdGuardHome for Asuswrt-Merlin - jumpsmm7/Asuswrt-Merlin-AdGuardHome-Installer

github.com

curl -L -s -k -O https://raw.githubusercontent.com/jumpsmm7/Asuswrt-Merlin-AdGuardHome-Installer/master/installer && sh installer

once I got curl working again and I was able to uninstall and reinstall AGH this time.

FIXED!!

Yep it is definitely a problem when so many dependencies change in these binaries. I am glad you got it working and I am sorry it caused you so much issue down the line.

 

[zyxmon]

I guess libopenssl was faulty.

AdGuardHome is linked statically (it go binary) and does not depend on other libraries.

 

[SomeWhereOverTheRainBow]

AdGuardHome is linked statically (it go binary) and does not depend on other libraries.

Yes, but the user had an issue with curl which is not statically linked to that library which resulted in a failure to download the next adguardhome version properly. The users curl broke when the binary libraries broke. Fixing the issue with curl allowed for adguardhome download issue to be fixed.

 

[Martinski]

... an issue with curl which is statically linked to that library ...

That's incorrect. The curl executable is not statically linked to the libopenssl library. The following screenshot clearly shows the dependencies on dynamically linked libraries:

The issues that arose during the last 2 weeks or so after the recent Entware upgrades were because of a missing dynamic library and/or loading the wrong dynamic library at runtime. Such problems could be avoided if executables (like curl) were to be compiled and linked with static libraries, but doing so has many other disadvantages, especially when trying to minimize the footprint of Entware packages installed onto embedded systems with very limited resources.

 

[SomeWhereOverTheRainBow]

That's incorrect. The curl executable is not statically linked to the libopenssl library. The following screenshot clearly shows the dependencies on dynamically linked libraries:

View attachment 48996

The issues that arose during the last 2 weeks or so after the recent Entware upgrades were because of a missing dynamic library and/or loading the wrong dynamic library at runtime. Such problems could be avoided if executables (like curl) were to be compiled and linked with static libraries, but doing so has many other disadvantages, especially when trying to minimize the footprint of Entware packages installed onto embedded systems with very limited resources.

Thank you for explaining that better. At either rate, I was correct with explaining that it was the failure of curl that broke the users ability to fetch a working adguardhome binary.

@zyxmon no one is trying to say adguardhome binary is statically linked to libopenssl.