Here are some basic comparisons I made between AGH (with abp.oisd and with dbl.oisd+mmotti filters) and Diversion on an Asus RT-AX86U with Asuswrt-Merlin 386.5_2
Methodology:
Note: for these tests the RT-AX86U was downstream from a pfsense box that was forcing all DNS to go through itself. The test laptop was downstream from the RT-AX86U on its own subnet and connected by gigabit ethernet. I don't know if this invalidates anything. For my DNS Bench tests, I set the DNS servers to be used as the upstream pfsense box-both on the router's upstream subnet (ethernet here was on its own vlan-x.x.99.x) and on the home subnet (x.x.11.x). I also had the RT-AX86U as one of the DNS Bench entries (x.x.50.x).
Note2: I realized after the fact that I left the 20 lookups per second throttling on in AGH when recording these values. I believe this is why the reliability isn't 100% in the DNS Bench when I was using AGH.
Note3: while I later went on to experiment with pixelserv-tls in the mix, these tests were with a pretty out of the box setup for AGH aside from changing the DNS filter lists.
Results:
Conclusion: I decided to use AGH with the dbl.oisd.nl block list + mmotti filters (as detailed elsewhere on this forum). This was based on the "all reload" metrics and the DNS Bench Avg uncached speeds. I also weighed ease of unblocking sites and flexibility in blocking individual devices in interesting ways in my decision.
CAUTION: If you switch to / start using AGH you should know there might be a memory leak somewhere. See: https://github.com/AdguardTeam/AdGuardHome/issues/4437
Methodology:
- I did a fresh install and update of the router (clean flash / reset to defaults after flash) and installed either Diversion or AGH
- I recorded the total RAM usage from the Asuswrt web page
- I used various real websites and firefox debug console (F12) and the network tab there and recorded the value for the field called "load."
- I did this 3x per site. I did not calculate averages (yet).
- I also opened all the sites in their own tabs and used the stopwatch on my phone to load them all at once. This is obviously subject to human error, but I did my best, as I am only human.
- Finally, I used DNS Bench to run some DNS benchmarks.
Note: for these tests the RT-AX86U was downstream from a pfsense box that was forcing all DNS to go through itself. The test laptop was downstream from the RT-AX86U on its own subnet and connected by gigabit ethernet. I don't know if this invalidates anything. For my DNS Bench tests, I set the DNS servers to be used as the upstream pfsense box-both on the router's upstream subnet (ethernet here was on its own vlan-x.x.99.x) and on the home subnet (x.x.11.x). I also had the RT-AX86U as one of the DNS Bench entries (x.x.50.x).
Note2: I realized after the fact that I left the 20 lookups per second throttling on in AGH when recording these values. I believe this is why the reliability isn't 100% in the DNS Bench when I was using AGH.
Note3: while I later went on to experiment with pixelserv-tls in the mix, these tests were with a pretty out of the box setup for AGH aside from changing the DNS filter lists.
Results:
diversion - around 58% ram with oisd / large
espn load speeds
8.32s / 7.52s / 7.85s
cnn load speeds
5.78s / 5.53s / 2.97s
reddit load speeds
11.65s / 9.88s / 6.88s
ign load speeds
3.90s / 4.61s / 3.90s
polygon load speeds
2.05s / 1.18s / 1.39s
wikipedia load speeds
716ms / 402ms / 435ms
youtube load speeds
3.37s / 2.19s / 2.20s
yahoo load speeds
4.15s / 5.54s / 4.56s
yahoo sports load speeds
5.61s / 3.00s / 4.46s
weather.com load speeds
2.13s / 1.64s / 735ms
accuweather.com load speeds
2.50s / 2.71s / 2.34s
all reload 19.92s / 20.13s / 21.09s
espn load speeds
8.32s / 7.52s / 7.85s
cnn load speeds
5.78s / 5.53s / 2.97s
reddit load speeds
11.65s / 9.88s / 6.88s
ign load speeds
3.90s / 4.61s / 3.90s
polygon load speeds
2.05s / 1.18s / 1.39s
wikipedia load speeds
716ms / 402ms / 435ms
youtube load speeds
3.37s / 2.19s / 2.20s
yahoo load speeds
4.15s / 5.54s / 4.56s
yahoo sports load speeds
5.61s / 3.00s / 4.46s
weather.com load speeds
2.13s / 1.64s / 735ms
accuweather.com load speeds
2.50s / 2.71s / 2.34s
all reload 19.92s / 20.13s / 21.09s
Code:
DNS BENCH
Final benchmark results, sorted by nameserver performance:
(average cached name retrieval speed, fastest to slowest)
192.168. 99. 1 | Min | Avg | Max |Std.Dev|Reliab%|
----------------+-------+-------+-------+-------+-------+
- Cached Name | 0.001 | 0.001 | 0.004 | 0.000 | 100.0 |
- Uncached Name | 0.042 | 0.113 | 0.523 | 0.109 | 100.0 |
- DotCom Lookup | 0.040 | 0.058 | 0.177 | 0.026 | 100.0 |
---<O-OOO-OO>---+-------+-------+-------+-------+-------+
Local Network Nameserver
192.168. 50. 1 | Min | Avg | Max |Std.Dev|Reliab%|
----------------+-------+-------+-------+-------+-------+
+ Cached Name | 0.001 | 0.001 | 0.001 | 0.000 | 100.0 |
+ Uncached Name | 0.040 | 0.117 | 0.503 | 0.114 | 100.0 |
+ DotCom Lookup | 0.041 | 0.060 | 0.136 | 0.024 | 100.0 |
---<O-OOO-OO>---+-------+-------+-------+-------+-------+
Local Network Nameserver
192.168. 11. 1 | Min | Avg | Max |Std.Dev|Reliab%|
----------------+-------+-------+-------+-------+-------+
- Cached Name | 0.000 | 0.001 | 0.001 | 0.000 | 100.0 |
- Uncached Name | 0.041 | 0.125 | 0.495 | 0.126 | 100.0 |
- DotCom Lookup | 0.043 | 0.054 | 0.081 | 0.009 | 100.0 |
---<O-OOO-OO>---+-------+-------+-------+-------+-------+
Local Network Nameserver
UTC: 2022-03-29, from 20:13:52 to 20:14:07, for 00:15.309
agh- around 62% ram with abp.oisd
espn load speeds
7.80s / 6.57s / 6.63s
cnn load speeds
5.61s / 4.36s / 3.78s
reddit load speeds
5.97s / 10.54s / 6.87s
ign load speeds
6.72s / 3.37s / 3.32s
polygon load speeds
1.32s / 1.59s / 1.30s
wikipedia load speeds
613ms / 472ms / 487ms
youtube load speeds
3.35s / 2.42s / 2.57s
yahoo load speeds
6.62s / 4.22s / 6.52s
yahoo sports load speeds
5.54s / 2.79s / 3.17s
weather.com load speeds
1.45s / 1.57s / 1.48s
accuweather.com load speeds
4.31s / 2.58s / 2.69s
all reload 21.11s / 17.35s / 17.58s
espn load speeds
7.80s / 6.57s / 6.63s
cnn load speeds
5.61s / 4.36s / 3.78s
reddit load speeds
5.97s / 10.54s / 6.87s
ign load speeds
6.72s / 3.37s / 3.32s
polygon load speeds
1.32s / 1.59s / 1.30s
wikipedia load speeds
613ms / 472ms / 487ms
youtube load speeds
3.35s / 2.42s / 2.57s
yahoo load speeds
6.62s / 4.22s / 6.52s
yahoo sports load speeds
5.54s / 2.79s / 3.17s
weather.com load speeds
1.45s / 1.57s / 1.48s
accuweather.com load speeds
4.31s / 2.58s / 2.69s
all reload 21.11s / 17.35s / 17.58s
Code:
DNS BENCH
Final benchmark results, sorted by nameserver performance:
(average cached name retrieval speed, fastest to slowest)
192.168. 50. 1 | Min | Avg | Max |Std.Dev|Reliab%|
----------------+-------+-------+-------+-------+-------+
+ Cached Name | 0.000 | 0.001 | 0.001 | 0.000 | 100.0 |
+ Uncached Name | 0.000 | 0.105 | 0.535 | 0.107 | 95.8 |
+ DotCom Lookup | 0.041 | 0.055 | 0.082 | 0.009 | 97.8 |
---<O-OOO-OO>---+-------+-------+-------+-------+-------+
Local Network Nameserver
192.168. 99. 1 | Min | Avg | Max |Std.Dev|Reliab%|
----------------+-------+-------+-------+-------+-------+
- Cached Name | 0.001 | 0.001 | 0.001 | 0.000 | 100.0 |
- Uncached Name | 0.039 | 0.107 | 0.354 | 0.081 | 100.0 |
- DotCom Lookup | 0.043 | 0.056 | 0.147 | 0.017 | 100.0 |
---<O-OOO-OO>---+-------+-------+-------+-------+-------+
Local Network Nameserver
192.168. 11. 1 | Min | Avg | Max |Std.Dev|Reliab%|
----------------+-------+-------+-------+-------+-------+
- Cached Name | 0.001 | 0.001 | 0.001 | 0.000 | 100.0 |
- Uncached Name | 0.040 | 0.107 | 0.364 | 0.086 | 100.0 |
- DotCom Lookup | 0.042 | 0.079 | 0.309 | 0.062 | 100.0 |
---<O-OOO-OO>---+-------+-------+-------+-------+-------+
Local Network Nameserver
UTC: 2022-03-29, from 21:31:26 to 21:31:48, for 00:22.245
agh2- around 61% ram with dbl.oisd + mmotti filters
espn load speeds
8.58s / 8.58s / 8.45s
cnn load speeds
4.82s / 3.15s / 2.56s
reddit load speeds
7.02s / 6.50s / 11.74s
ign load speeds
3.78s / 3.80s / 3.69s
polygon load speeds
1.84s / 1.52s / 1.81s
wikipedia load speeds
565ms / 568ms / 508ms
youtube load speeds
4.02s / 2.93s / 2.87s
yahoo load speeds
5.45s / 4.22s / 3.59s
yahoo sports load speeds
5.98s / 4.51s / 4.86s
weather.com load speeds
2.15s / 1.48s / 1.53s
accuweather.com load speeds
3.85s / 2.50s / 2.78s
all reload 15.86s / 13.01s / 12.30s
espn load speeds
8.58s / 8.58s / 8.45s
cnn load speeds
4.82s / 3.15s / 2.56s
reddit load speeds
7.02s / 6.50s / 11.74s
ign load speeds
3.78s / 3.80s / 3.69s
polygon load speeds
1.84s / 1.52s / 1.81s
wikipedia load speeds
565ms / 568ms / 508ms
youtube load speeds
4.02s / 2.93s / 2.87s
yahoo load speeds
5.45s / 4.22s / 3.59s
yahoo sports load speeds
5.98s / 4.51s / 4.86s
weather.com load speeds
2.15s / 1.48s / 1.53s
accuweather.com load speeds
3.85s / 2.50s / 2.78s
all reload 15.86s / 13.01s / 12.30s
Code:
DNS BENCH
Final benchmark results, sorted by nameserver performance:
(average cached name retrieval speed, fastest to slowest)
192.168. 50. 1 | Min | Avg | Max |Std.Dev|Reliab%|
----------------+-------+-------+-------+-------+-------+
+ Cached Name | 0.000 | 0.001 | 0.001 | 0.000 | 100.0 |
+ Uncached Name | 0.000 | 0.105 | 0.535 | 0.107 | 95.8 |
+ DotCom Lookup | 0.041 | 0.055 | 0.082 | 0.009 | 97.8 |
---<O-OOO-OO>---+-------+-------+-------+-------+-------+
Local Network Nameserver
192.168. 99. 1 | Min | Avg | Max |Std.Dev|Reliab%|
----------------+-------+-------+-------+-------+-------+
- Cached Name | 0.001 | 0.001 | 0.001 | 0.000 | 100.0 |
- Uncached Name | 0.039 | 0.107 | 0.354 | 0.081 | 100.0 |
- DotCom Lookup | 0.043 | 0.056 | 0.147 | 0.017 | 100.0 |
---<O-OOO-OO>---+-------+-------+-------+-------+-------+
Local Network Nameserver
192.168. 11. 1 | Min | Avg | Max |Std.Dev|Reliab%|
----------------+-------+-------+-------+-------+-------+
- Cached Name | 0.001 | 0.001 | 0.001 | 0.000 | 100.0 |
- Uncached Name | 0.040 | 0.107 | 0.364 | 0.086 | 100.0 |
- DotCom Lookup | 0.042 | 0.079 | 0.309 | 0.062 | 100.0 |
---<O-OOO-OO>---+-------+-------+-------+-------+-------+
Local Network Nameserver
UTC: 2022-03-29, from 21:31:26 to 21:31:48, for 00:22.245
Conclusion: I decided to use AGH with the dbl.oisd.nl block list + mmotti filters (as detailed elsewhere on this forum). This was based on the "all reload" metrics and the DNS Bench Avg uncached speeds. I also weighed ease of unblocking sites and flexibility in blocking individual devices in interesting ways in my decision.
CAUTION: If you switch to / start using AGH you should know there might be a memory leak somewhere. See: https://github.com/AdguardTeam/AdGuardHome/issues/4437
Last edited: