Kamoj Kamoj Add-on Beta testing

[Droidrat]

Sorry for being short!
Life is not always easy and harmonic, but I want to help as soon as I can.
top2 is router command showing all processes sorted with highest cpu load at top.
Try! And/or follow @sppmaster advice.

Got it. Top2 is great (now that I know what it is, and what it's for).

I'm not sure I would call my situation a "new install of firmware" since the installation occurred over 3 days ago. Top2 would have answered my question regarding high CPU utilization (until I destroyed the evidence by doing a reboot). Next time I will know.

Thank you for your help (and your patience).

 

[sppmaster]

Lots of this lines in DNSCrypt v2 log

[2021-02-09 23:04:29] [INFO] Server [dnscrypt.eu-nl] returned temporary error code SERVFAIL -- Invalid DNSSEC signature received or server may be experiencing connectivity issues [2021-02-09 23:04:29] [INFO] Server [cloudflare] returned temporary error code SERVFAIL -- Invalid DNSSEC signature received or server may be experiencing connectivity issues [2021-02-09 23:11:02] [INFO] Server [scaleway-fr] returned temporary error code SERVFAIL -- Invalid DNSSEC signature received or server may be experiencing connectivity issues [2021-02-09 23:21:36] [INFO] Server [dnscrypt.eu-dk] returned temporary error code SERVFAIL -- Invalid DNSSEC signature received or server may be experiencing connectivity issues [2021-02-09 23:21:36] [INFO] Server [cloudflare] returned temporary error code SERVFAIL -- Invalid DNSSEC signature received or server may be experiencing connectivity issues

 

[kamoj]

Try another NTP time server.
Else I don't know.
You can ask at the DNSCrypt community ?

Are you running Extended Logging? Do that while looking for errors.
(Toggling extended logging will restart dnscrypt).

Lots of this lines in DNSCrypt v2 log

[2021-02-09 23:04:29] [INFO] Server [dnscrypt.eu-nl] returned temporary error code SERVFAIL -- Invalid DNSSEC signature received or server may be experiencing connectivity issues [2021-02-09 23:04:29] [INFO] Server [cloudflare] returned temporary error code SERVFAIL -- Invalid DNSSEC signature received or server may be experiencing connectivity issues [2021-02-09 23:11:02] [INFO] Server [scaleway-fr] returned temporary error code SERVFAIL -- Invalid DNSSEC signature received or server may be experiencing connectivity issues [2021-02-09 23:21:36] [INFO] Server [dnscrypt.eu-dk] returned temporary error code SERVFAIL -- Invalid DNSSEC signature received or server may be experiencing connectivity issues [2021-02-09 23:21:36] [INFO] Server [cloudflare] returned temporary error code SERVFAIL -- Invalid DNSSEC signature received or server may be experiencing connectivity issues

 

[B-dog66]

Hey Kamoj,

I'm back with the issue of "Out of memory" at this time not the Adguard Home caused the issue but leaded to crash the adguard, all of this resulted temporary internet disconection till I restarted the Adguard home from the interface menu.
After this I obseved high CPU usage, I tried to check the System Information-Top but nothing came up so I copied to usb the top.txt from /var and I find out that the kamoj.sh vas using 40% CPU.
By the way the top.txt is 57Mb because lot of empty space it was in it what I deleted and I needed to split in multiple files to can upload.
Please let me know if you need more info.
Please see attached logs.

 

[kamoj]

Thank you.
I can not process the info now, but will of course look into it soon.

What is "the interface menu"?
Have you experienced hight cpu load when not using Adguard home?
Do you experinece problems when using Adguard using default configuration?

Please provide print of "ps -www" or "ps2" command.
And be sure to provide version of FW + add-on, since that is very important info!

Hey Kamoj,

I'm back with the issue of "Out of memory" at this time not the Adguard Home caused the issue but leaded to crash the adguard, all of this resulted temporary internet disconection till I restarted the Adguard home from the interface menu.
After this I obseved high CPU usage, I tried to check the System Information-Top but nothing came up so I copied to usb the top.txt from /var and I find out that the kamoj.sh vas using 40% CPU.
By the way the top.txt is 57Mb because lot of empty space it was in it what I deleted and I needed to split in multiple files to can upload.
Please let me know if you need more info.
Please see attached logs.

 

[B-dog66]

Sorry kamoj, I forgot to mention FW + addon, FW Voxel V1.0.2.82.1SF + Kamoj Add-on V5.4b20.

What is "the interface menu"?

I was meaning "Kamoj Menu->DNS Privacy/Ad-Blocking->DNS Filter/Encryption" and I switched to ⚠ None - Not recommended and back to Adguard Home.

Have you experienced hight cpu load when not using Adguard home?

Nope, however I was using Adguard home all the time, but I will give a try for a couple of days and report back

Do you experinece problems when using Adguard using default configuration?

After I finished testing without Adguard home I will run with default configuration.

By the way I restarted the router and hope it's not affecting the ps2 or ps-www log.

I noticed after restart that the router internet connection led was blinking amber not white, but the connection it was alright also run a speed test to check it and it was fine.
After reconnection (disconnect-> connect again) it's blinking again white.

Hope I give enough detail this time, if not please let me know.

Edit: I'm using Aegis also

 

[blueliner]

H

I'll add your wishes to the todo-list. I have not forgotten your previous suggestions!
1). Should be possible.
The reason for it not being so now is that the existing lists are of at least 4 different formats.
But it's doable, absolutely not too time consuming.
And maybe there already is an existing script to do the job? Do you know one?

#Type of hosts lists in adlists below:
#1: 0.0.0.0,      eg: 0.0.0.0 papoto.com
#2: 127.0.0.1,    eg: 127.0.0.1 crash.163.com
#3: pure domains, eg: fanclubmania.ru
#4: 0.0.0.0 unzip needed, contains eg: HOSTS file of type 1 together with other files...

2). All this with iptables and routing tables is too complicated to satisfy all needs/wishes.
For the moment it will remain as it is, suiting my own needs.
Unless someone comes a with a ready solution. If so I'll implement it.
3). See point 2 above.

Thank you for your suggestions and support and for using the add-on!Hello,

Kamoj, thanks for the feedback on my suggestions. I don't know of a script or other means for adding the blocking list entry but will keep my eyes open.

I certainly understand about the VPN bypass options. I can only imagine how complicated manipulating the iptables can become. Except for some basic firewall rules I leave them pretty much alone because I create more problems than I solve!

I did try the "Route to Other Network" option. My setup has a satellite router/modem running in bridge mode (router deactivated). The R9000 and another router connect to the modem...running two different/separate networks. I entered the modem ip in the Route to Other Network" section but was unable to connect. Perhaps I misunderstood or it is just something to do with my modem.

Best wishes,
BL

 

[B-dog66]

Have you experienced hight cpu load when not using Adguard home?

Back with new logs, I disabled Adguard home and still I see high cpu usage.

FW Voxel V1.0.2.82.1SF + Kamoj Add-on V5.4b20 + Aegis (default config)

Please see attached.

Regards,

 

[Warlord1981]

Guyz after a small research I see that one of the best dns blocklists is dbl.oisd.nl.

So i left all default Kamoj lists de-selected and pressed Get Own Blacklist and put the url there and pressed Save own blacklist and then Submit. However I see no blocking of ads as well as no space takin on system disk (this list is big). I tried either as dbl.oisd.nl or https://dbl.oisd.nl/ with the same result. Any advice?

 

[kamoj]

Thank you very good to get your log files.

Please enable functions/settings one by one until you find the culprit - if there really is any problem.
This is the least I expect from a beta tester.

You have some settings enabled that disturb your measurements:

• Aegis. Disable Aegis while doing these measurements.
• Set time stamps on dmesg logs. This setting also makes Aegis to bug out and stop logging after some 9000 seconds from boot!
  This Netgear bug is discussed elsewhere, so just switch this option off if you use Aegis.
• Router Information. Switch off auto update (every 20 seconds), and don't make manual refresh.
  Router Information consumes "much" cpu when run, as it collect and calculates "thousands" of measurements.
• Save logs. This function also consumes much cpu and disk i/o.
  It also writes a lot to disk and may effect memory. Switch it off, until you know what you do/want it for.
• If you don't use DLNA, switch it off!
• If you don't use WPS, switch it off!
• You seems to have some error with your routers memory.
  (msm_nand_read_oob 0 800 0 + msm_nand_read_oob c80000 800 0 failed)
  Please provide the "flash info" output!

Apart from all your settings causing artificial cpu load I must ask:
Do you experience any problem?
Any function that is not working?

Back with new logs, I disabled Adguard home and still I see high cpu usage.

FW Voxel V1.0.2.82.1SF + Kamoj Add-on V5.4b20 + Aegis (default config)

Please see attached.

Regards,

 

[kamoj]

I don't agree it's one of the best...
For sure it's a BIG one. It's actually too big for the R7800 limited memory, that's why I commented it out.
It's OK for the R9000 I suspect, but since I could not test that I left it out.
(My comment in current source code: "Too big for internal memory of 49.836MB" (2 years ago))
If you are adventurous you can patch addon_dnsprivacy.htm and see how your routers memory is eaten up by oisd.)

Your Own Blacklist shall be a list of URLs - not pointers to lists of URLs.

Guyz after a small research I see that one of the best dns blocklists is dbl.oisd.nl.

So i left all default Kamoj lists de-selected and pressed Get Own Blacklist and put the url there and pressed Save own blacklist and then Submit. However I see no blocking of ads as well as no space takin on system disk (this list is big). I tried either as dbl.oisd.nl or https://dbl.oisd.nl/ with the same result. Any advice?

 

[B-dog66]

Kamoj thanks for you patience, I will follow your guide and will report back to you.

Apart from all your settings causing artificial cpu load I must ask:
Do you experience any problem?
Any function that is not working?

I don't use openvpn/wireguard because it's a bit slow, so I cannot say anything about it.

DNSCrypt 2 I didn't used since you added Aguard Home, but I didn't have any problem with it till then, I need to think about to switch back to DNSCrypt 2

Apart from this everything it was just working fine and I didn't notice any issue.

See attached flash info.

Regards,

 

[kamoj]

It's good but strange that your router memories ar just fine!
I don't understand why you got those dmesg error messages.

Adguard Home is working for many other users, so I'm confused about your issues.

DNSCrypt is really much faster than Adguard, but Adguard has a nice interface and many functions.

Hope you find out something by yourself!

Kamoj thanks for you patience, I will follow your guide and will report back to you.

I don't use openvpn/wireguard because it's a bit slow, so I cannot say anything about it.

DNSCrypt 2 I didn't used since you added Aguard Home, but I didn't have any problem with it till then, I need to think about to switch back to DNSCrypt 2

Apart from this everything it was just working fine and I didn't notice any issue.

See attached flash info.

Regards,

 

[OcramSagev]

Changes in kamoj-addon beta version 5.4b7
--------------------------------------------------
- Removed automatic setting of dmesg time stamps (To align with Aegis)
- Advanced: Kamoj Menu: Settings: Added "Set time stamps on dmesg logs"
N.B.: Don't check this one if you want Aegis logs to work >10000 seconds
- OpenVPN Client: AzireVPN: Added: Spain-Madrid.
- Restart Supervision: Changed timeouts.
Delayed start 31 seconds to not start simultaneously with other cron jobs.
ping failure detection changed.
Added possibility to change ping failure timeout (default 1 sec) with nvram variable, e.g.:
nvram set kamoj_restart_ping_timeout=2 && nvram commit
- Router Information: Changed number of ping for DNS/Internet test from 1 to 3.
Added possibility to change ping failure timeout (default 3 sec) with nvram variable, e.g.:
nvram set kamoj_restart_ping_timeout=2 && nvram commit
- net-wan: added restart of Wireguard client if needed
- Advanced: Administration: Logs: Enlarged default log window size for R7800.
- Advanced: Administration: Logs: Enabled these log alternatives for R7800:
+ Automatic Internet connection reset
+ ReadySHARE Mobile Connect

Hello Guys:
My first post here in the forum. I've been following it (and read almost all the postings) > Great conversations and I'm impressed with the level of knowledge.
I'm an engineer (I can write code but not too familiar with networking engineering) and I own a R7800 . I've been using Voxel for a long time now.
I started doing research in these forums because my router was experiencing random reboots and it was driving me insane.

I'm using the R7800 as an AP connected to an Ubiquiti EdgeRouter X (Awesome device at a very affordable price).

I am completely unable to test Kamoj latest add-on.
1.- On 1.0.2.79 it completely blocks the internet, it makes the router super slow and the only solution is to do a factory reset.
2.- On 1.0.2.82.X it works well and I'm impressed on how powerful it is but this fw. makes my router reboot randomly.
3.- The 2019 Kamoj add on that appears on another thread (the one that can get downloaded with a curl command from Voxel's repository): This one has no issues with 1.0.2.79 but of course misses the new features.

Has anyone been able to run Kamoj latest beta on the R7800 (in AP mode) successfully? Does anyone have any clue how to stop the random reboots?

Thanks in advance. Looking forward to exchange ideas with all of you guys.

 

[R. Gerrits]

AdGuardHome v0.105.0 has been released.
For the people who were running a beta version of v0.105 -> uncheck the "Use BETA version" and then click on "Download latest version" to upgrade to this new stable version.

 

[sppmaster]

@kamoj
Yesterday there was a short blackout here. After the recovery of the power supply I saw that half on hour later the router R7800 CPU was at 60-70% load. The top was almost impossible to open from the menu and console command gave me no useful information. One time I saw that ksoftirqd was using 10%. The info page was impossible to open spitting the message - The CGI script took too long to produce a response. I've seen the messages here on the forum about that high CPU usage and decided to not reboot the router to pinpoint the issue. This morning R7800 CPU was still at 60-70% load. The top from console gave this info (13 hours later).

  PID USER     STATUS   RSS  PPID %CPU %MEM COMMAND
26609 root     R N     1296 26268 43.8  0.2 kamoj.sh
12425 root     SW         0     2  1.6  0.0 kworker/0:0
15994 root     S N      452     1  0.1  0.0 top
21537 root     S N      452     1  0.1  0.0 top
7789 root     S N      448     1  0.1  0.0 top
16578 root     S N      448     1  0.1  0.0 top
6838 root     S N      448     1  0.1  0.0 top
9696 root     S N      448     1  0.1  0.0 top
1887 root     S        956     1  0.0  0.1 haveged
6471 root     S        588     1  0.0  0.1 addon_info_upda
6472 root     S        572     1  0.0  0.1 addon_info_upda
1377 root     S        496     1  0.0  0.1 datalib
3704 root     S N      456     1  0.0  0.0 top
14584 root     S N      452     1  0.0  0.0 top
27066 root     S N      452     1  0.0  0.0 top
16618 root     S N      448     1  0.0  0.0 top
2368 root     S N      448     1  0.0  0.0 top
12463 root     S N      448     1  0.0  0.0 top
16644 root     S N      448     1  0.0  0.0 top
12865 root     S N      448     1  0.0  0.0 top
9547 root     S N      448     1  0.0  0.0 top

I'm on latest Voxel and Kamoj b20.
As I'm very recent with Voxel firmware and Kamoj add-on and I use them from b19 I can say that with b19 I didn't have such CPU usage. I was monitoring it during several days at least 10-20 times per day.
I use DNSCrypt v2 with adblock and 4 filters to block around 40k domains. I saw message above of high memory demand (RAM or NAND memory?) when using more filters and just wonder about it. I have years of experience with OpenWRT and adblock use on routers even with just 32MB of RAM and just 4MB Flash (blocking 30k domains) and blocking over 80k domains on router with 128 MB RAM without a single issue.

 

[kamoj]

Thank you for the interesting report. Well done.

What I would like is the output from console of the commands: ps2 and ps -www

Changes between b19 and b20 are minimal and should not cause these kind of issue.

Make sure to switch off QoS.

PS
The full oisd block list contains >1.1 million entries, so quite much more than you might expect.

@kamoj
Yesterday there was a short blackout here. After the recovery of the power supply I saw that half on hour later the router R7800 CPU was at 60-70% load. The top was almost impossible to open from the menu and console command gave me no useful information. One time I saw that ksoftirqd was using 10%. The info page was impossible to open spitting the message - The CGI script took too long to produce a response. I've seen the messages here on the forum about that high CPU usage and decided to not reboot the router to pinpoint the issue. This morning R7800 CPU was still at 60-70% load. The top from console gave this info (13 hours later).

  PID USER     STATUS   RSS  PPID %CPU %MEM COMMAND
26609 root     R N     1296 26268 43.8  0.2 kamoj.sh
12425 root     SW         0     2  1.6  0.0 kworker/0:0
15994 root     S N      452     1  0.1  0.0 top
21537 root     S N      452     1  0.1  0.0 top
7789 root     S N      448     1  0.1  0.0 top
16578 root     S N      448     1  0.1  0.0 top
6838 root     S N      448     1  0.1  0.0 top
9696 root     S N      448     1  0.1  0.0 top
1887 root     S        956     1  0.0  0.1 haveged
6471 root     S        588     1  0.0  0.1 addon_info_upda
6472 root     S        572     1  0.0  0.1 addon_info_upda
1377 root     S        496     1  0.0  0.1 datalib
3704 root     S N      456     1  0.0  0.0 top
14584 root     S N      452     1  0.0  0.0 top
27066 root     S N      452     1  0.0  0.0 top
16618 root     S N      448     1  0.0  0.0 top
2368 root     S N      448     1  0.0  0.0 top
12463 root     S N      448     1  0.0  0.0 top
16644 root     S N      448     1  0.0  0.0 top
12865 root     S N      448     1  0.0  0.0 top
9547 root     S N      448     1  0.0  0.0 top

I'm on latest Voxel and Kamoj b20.
As I'm very recent with Voxel firmware and Kamoj add-on and I use them from b19 I can say that with b19 I didn't have such CPU usage. I was monitoring it during several days at least 10-20 times per day.
I use DNSCrypt v2 with adblock and 4 filters to block around 40k domains. I saw message above of high memory demand (RAM or NAND memory?) when using more filters and just wonder about it. I have years of experience with OpenWRT and adblock use on routers even with just 32MB of RAM and just 4MB Flash (blocking 30k domains) and blocking over 80k domains on router with 128 MB RAM without a single issue.

 

[sppmaster]

@kamoj
The system doesn't allow more than 10k symbols. Where can I send you the output.
Splitting in several posts seems impractical to me.
I don't use QoS as my connection is 512Mbps/150Mbps down/up speed and I can hardly saturate it even when using torrents.
But an interesting fact is that when I use an USB 3 flash drive connected to R7800 the up speed is cut in half at least for 2.4GHz band from longer distance. This is probably because of interference caused by the USB 3 interface itself and is off-topic here but wanted to share it to the tech-savvy guys here.

 

[Warlord1981]

@kamoj
I just installed Adguard for the first time (v0.105.0) and currently use Quad9's quad9-dnscrypt-ip4-filter-pri sdns stamp on Upstream Servers with DNSSEC enabled and all green blocklists found here https://firebog.net/.

1. Some URLs (even youtube sometimes) often do not open on first ENTER. When immediately pressed second time (or refreshed) they do. Any thoughs?
2. How can I verify that my DNS queries & answers are indeed encrypted? Do i also need to Enable Encryption on Encryption Settings or this is only if i want secure certificate on my Adguard server webpage? Also what IPs do I need to input on DNS Address of Internet Setup menu of Netgear? Quad9 pri and sec?

 

[kamoj]

You are a beta tester.
I suggest you find out on an Adguard forum and post your findings here.
The DNS Netgear setup is valid until you start eg Adguard.

@kamoj
I just installed Adguard for the first time (v0.105.0) and currently use Quad9's quad9-dnscrypt-ip4-filter-pri sdns stamp on Upstream Servers with DNSSEC enabled and all green blocklists found here https://firebog.net/.

1. Some URLs (even youtube sometimes) often do not open on first ENTER. When immediately pressed second time (or refreshed) they do. Any thoughs?
2. How can I verify that my DNS queries & answers are indeed encrypted? Do i also need to Enable Encryption on Encryption Settings or this is only if i want secure certificate on my Adguard server webpage? Also what IPs do I need to input on DNS Address of Internet Setup menu of Netgear? Quad9 pri and sec?