Kamoj Kamoj Add-on 5.1 Beta testing poll

[foo man]

Hello Kamoj, could I try the beta please? Thanks!

 

[kamoj]

New beta available!

Changes in kamoj-addon beta version 5.3b3
-------------------------------------------------
- Bandwidth Usage: Updated to handle data larger than 53 G.
- Bandwidth Usage: Added All Devices Summary
- Settings: html code cleaned up a little again
- DDNS: Bug fixed in cmd_ddns (@R. Gerrits)
- OpenVPN Client: Moved killswitch to RAM
- OpenVPN Client: Apply killswitch after reboot
- OpenVPN Client: Remove killswitch at stop of client
- OpenVPN Client: Less verbose start/stop of client
- OpenVPN Client: Fixed (harmless) message: "Cannot get device feature names: No such device"
- Wireguard: Moved killswitch to RAM
- Wireguard: Apply killswitch after reboot
- Wireguard: Remove killswitch at stop of client
- Wireguard: Don't create killswitch if wg-client not installed
- net-wall Firewall: Added "Mutex" lock to solve simultaneous access
(From Netgear/DNI/Voxel/Kamoj code)
- Updated FAQ.txt

 

[foo man]

Just got the R9000 earlier in the week and promptly upgraded to Voxel's latest firmware. Finally got around to installing kamoj"s latest beta yesterday and all I can say is WOW! The options are endless, I am like a kid in a candy store! Thanks so much to @kamoj and @Voxel for providing the community with your work, it is fantastic!

 

[kamoj]

New beta available!

Changes in kamoj-addon beta version 5.3b4
-------------------------------------------------
- VPN Bypassing: Now support for Wireguard clients
- OpenVPN Bypass: Changed name to "VPN bypassing"
- AdGuard Home: Added many filters (Firehol + all from Kamoj DNSCrypt addblocking!, but none activated).
- OpenVPN Server: New "dynamic" version: vpn-firewall.sh (@R. Gerrits)
- net-wall Firewall: New firewall rules for VPN clients (@R. Gerrits)
- net-wall Firewall: Speeded up 2 seconds (Honestly: Fixed bug ;-)
- Bandwidth Usage: Sorting of "Last seen" to Summary line.
- BASIC: Changed icon for Netgear BASIC menu to famous router model

 

[R. Gerrits]

- AdGuard Home: Added many filters (Firehol + all from Kamoj DNSCrypt addblocking!, but none activated).

I see you added https://hosts.oisd.nl
The .nl domain caught my interest. And then via the reddit page of that blocklist, I saw that you can better use https://abp.oisd.nl as AdGuard Home supports ABP. (makes the list almost 4 times as small and probably more efficient)

 

[Droidrat]

@kamoj , on the "Bandwidth Usage" page, it gives the date and time when the information was generated. It would also be helpful to know when the measurement period commenced (for the page as a whole).

 

[Droidrat]

The Bandwidth Usage page is not updating for me in V5.3b4 (though it worked in the prior build). The date and time below the table refresh correctly, but the data in the table itself (including the "last seen" time) is not updating.

 

[kamoj]

Noted. It's working for me
Have you done something else? Upgraded firmware?
Done iptables changes?
How big amounts of data do you have?
It should work up to 15 digits of data...
Also at "off peak" hours the data is updated only every 30 minutes.

The Bandwidth Usage page is not updating for me in V5.3b4 (though it worked in the prior build). The date and time below the table refresh correctly, but the data in the table itself (including the "last seen" time) is not updating.

 

[Rustypouch]

Hi Kamoj and thanks for the great add-on for the great Voxel FW.

I don't understand what most of either does, but I am trying to learn more about it as I go. Are there any resources where I can read more? I would like to understand the wizardry behind it all.

Can I have access to the latest beta of your Add-On, please? By installing the latest Voxel FW does this delete the previous add-on? or should I remove the add-on prior to installing the new version?

Thanks again!

Rustypouch

 

[kamoj]

There is unfortunately no User's Guide for the add-on.
There is some documentation in the beta download area. Read that!

Installing new Firmware will delete the add-on.
If you update the add-on, you MUST uninstall the previous one before installing the new version.

Don't be afraid of creating your own thread in this forum, there are many helpful people here!

Hi Kamoj and thanks for the great add-on for the great Voxel FW.
I don't understand what most of either does, but I am trying to learn more about it as I go. Are there any resources where I can read more? I would like to understand the wizardry behind it all.
Can I have access to the latest beta of your Add-On, please? By installing the latest Voxel FW does this delete the previous add-on? or should I remove the add-on prior to installing the new version?
Thanks again!
Rustypouch

 

[blueliner]

Hello,

Can't say enough about how your addon and Voxel firmware have improved my R9000 - thank you. My setup is a R9000 with QOS active, OpenVPN (several devices bypassed) and DNSCrypt running with adblock. I don't use remote access.

5.3b3 ran great and for the first time I was getting the message showing the latest Voxel and Netgear firmware releases. However, that stopped working with version 5.3b4. A bigger issue is that my devices bypassing the VPN seem to loose internet connection (it acts more like a DNS issue). I've tried to reset and redo the VPN bypass but it eventually fails again. Shutting off the VPN fixes the connectivity problem.

Could this be related to the VPN bypass changes in 5.3b4? For now I've went back to 5.3b3 and all is well.

BL

New beta available!

Changes in kamoj-addon beta version 5.3b4
-------------------------------------------------
- VPN Bypassing: Now support for Wireguard clients
- OpenVPN Bypass: Changed name to "VPN bypassing"
- AdGuard Home: Added many filters (Firehol + all from Kamoj DNSCrypt addblocking!, but none activated).
- OpenVPN Server: New "dynamic" version: vpn-firewall.sh (@R. Gerrits)
- net-wall Firewall: New firewall rules for VPN clients (@R. Gerrits)
- net-wall Firewall: Speeded up 2 seconds (Honestly: Fixed bug ;-)
- Bandwidth Usage: Sorting of "Last seen" to Summary line.
- BASIC: Changed icon for Netgear BASIC menu to famous router model

 

[kamoj]

Thank you for the reports!

Can you try install todays version v. 1.0.4.41HF of Voxel's R9000 firmware, and then previous working Kamoj add-on 5.3b3?
(The changes in bypassing could hardly be the problem but quite big changes in firewall changes (for OpenVPN and Wireguard),
that are the same as in the latest Voxel release v. 1.0.4.41HF. )
(The bypassing change was only for Wireguard, not OpenVPN)

If this combo is working I must have slipped on the keyboard and apologize.
If you still have the problem, the iptables maestro himself must come to help!

For DNS issue, you can try Stubby or AdGuard Home or None.

Since my time and stamina is limited, that would be good if you could test this, I need help to help you.

 

[R. Gerrits]

The Bandwidth Usage page is not updating for me in V5.3b4 (though it worked in the prior build). The date and time below the table refresh correctly, but the data in the table itself (including the "last seen" time) is not updating.

I'm having the same issue.
And if I look at the iptables chain RRDIPT, then I have over 2000 entries, where it should only have 38 (if I only count the unique entries).
The counters will only increase for the first occurance of a rule. If the script is only looking at the last occurance, that would explain why they stay at zero after resetting them.

But of course the big question is where the duplicates come from.

Edit:
Did a net-wall restart -> this cleared out all the entries, and now I do see some data in the Bandwidth usage.
Will keep an eye on this, if duplicates start re-appearing.

Edit2:
After only a few minutes, the first duplicates started popping up in RRDIPT

 

[kamoj]

Thank you @R. Gerrits, we need you!

You can off Bandwidth usage in the Settings.
It should clear/delete the counters-iptables and restart net-wall.

 

[R. Gerrits]

Thank you @R. Gerrits, we need you!

You can off Bandwidth usage in the Settings.
It should clear/delete the counters-iptables and restart net-wall.

Also did tried that, but also then the duplicates reappear.

Had a look at the code you use to insert the rules, but at first glance I also cannot see anything wrong there....

 

[R. Gerrits]

I see that both firewall-start-bwusage.sh and addon_bwusage.sh have similar code to add entries.
So I added some logging to both locations:

root@R7800:~$ tail -f /tmp/bwusage.log 
Wed May 27 16:59:05 UTC 2020 adding 192.168.1.10 from firewall-start-bwusage.sh
Wed May 27 16:59:05 UTC 2020 adding 192.168.1.50 from firewall-start-bwusage.sh
Wed May 27 16:59:05 UTC 2020 adding 192.168.1.90 from firewall-start-bwusage.sh
Wed May 27 16:59:05 UTC 2020 adding 192.168.1.11 from firewall-start-bwusage.sh
Wed May 27 16:59:05 UTC 2020 adding 192.168.1.12 from firewall-start-bwusage.sh
Wed May 27 16:59:05 UTC 2020 adding 192.168.1.20 from firewall-start-bwusage.sh
Wed May 27 16:59:05 UTC 2020 adding 192.168.1.82 from firewall-start-bwusage.sh
Wed May 27 16:59:05 UTC 2020 adding 192.168.1.4 from firewall-start-bwusage.sh
Wed May 27 16:59:05 UTC 2020 adding 192.168.1.83 from firewall-start-bwusage.sh
Wed May 27 19:00:01 GMT 2020 adding 192.168.1.1 from addon_bwusage.sh
Wed May 27 19:00:01 GMT 2020 adding 192.168.1.100 from addon_bwusage.sh
Wed May 27 19:00:01 GMT 2020 adding 192.168.1.119 from addon_bwusage.sh
Wed May 27 19:00:01 GMT 2020 adding 192.168.1.10 from addon_bwusage.sh
Wed May 27 19:00:01 GMT 2020 adding 192.168.1.25 from addon_bwusage.sh
Wed May 27 19:00:01 GMT 2020 adding 192.168.1.100 from addon_bwusage.sh
Wed May 27 19:00:01 GMT 2020 adding 192.168.1.28 from addon_bwusage.sh
Wed May 27 19:00:02 GMT 2020 adding 192.168.1.119 from addon_bwusage.sh
Wed May 27 19:00:02 GMT 2020 adding 192.168.1.30 from addon_bwusage.sh
Wed May 27 19:00:02 GMT 2020 adding 192.168.1.4 from addon_bwusage.sh
Wed May 27 19:00:02 GMT 2020 adding 192.168.1.28 from addon_bwusage.sh
Wed May 27 19:00:02 GMT 2020 adding 192.168.1.40 from addon_bwusage.sh
Wed May 27 19:00:02 GMT 2020 adding 192.168.1.4 from addon_bwusage.sh
Wed May 27 19:00:02 GMT 2020 adding 192.168.1.80 from addon_bwusage.sh
Wed May 27 19:00:02 GMT 2020 adding 192.168.1.81 from addon_bwusage.sh
Wed May 27 19:00:02 GMT 2020 adding 192.168.1.83 from addon_bwusage.sh
Wed May 27 19:00:02 GMT 2020 adding 192.168.1.83 from addon_bwusage.sh
Wed May 27 19:00:02 GMT 2020 adding 192.168.1.100 from addon_bwusage.sh
Wed May 27 19:02:01 GMT 2020 adding 192.168.1.10 from addon_bwusage.sh
Wed May 27 19:02:01 GMT 2020 adding 192.168.1.10 from addon_bwusage.sh
Wed May 27 19:02:01 GMT 2020 adding 192.168.1.28 from addon_bwusage.sh
Wed May 27 19:02:01 GMT 2020 adding 192.168.1.11 from addon_bwusage.sh
Wed May 27 19:02:02 GMT 2020 adding 192.168.1.119 from addon_bwusage.sh
Wed May 27 19:02:02 GMT 2020 adding 192.168.1.12 from addon_bwusage.sh
Wed May 27 19:02:02 GMT 2020 adding 192.168.1.20 from addon_bwusage.sh
Wed May 27 19:02:02 GMT 2020 adding 192.168.1.28 from addon_bwusage.sh
Wed May 27 19:02:02 GMT 2020 adding 192.168.1.30 from addon_bwusage.sh
Wed May 27 19:02:02 GMT 2020 adding 192.168.1.1 from addon_bwusage.sh
Wed May 27 19:02:02 GMT 2020 adding 192.168.1.80 from addon_bwusage.sh
Wed May 27 19:02:02 GMT 2020 adding 192.168.1.81 from addon_bwusage.sh
Wed May 27 19:02:02 GMT 2020 adding 192.168.1.100 from addon_bwusage.sh
Wed May 27 19:02:02 GMT 2020 adding 192.168.1.90 from addon_bwusage.sh

some interesting observations:
the lines generated by the script that is initiated by net-wall uses a different timezone
the lines generated by addon_bwusage sometimes in "one loop" (looking at the time) add certain IPs twice.

So I'm almost thinking that addon_bwusage uses a different shell than firewall-start-bwusage, and that somehow (on my router) the command you use (iptables -nL RRDIPT | sed 's/[[:space:]]*$//' | grep "${IP}$" >/dev/null) does work in the one shell, but not in the other.

Perhaps one is using entware components or so ??

 

[Droidrat]

Noted. It's working for me
Have you done something else? Upgraded firmware?
Done iptables changes?
How big amounts of data do you have?
It should work up to 15 digits of data...
Also at "off peak" hours the data is updated only every 30 minutes.

My router is probably the most "plain vanilla" of any of your testers. I don't use any special features. I renamed the 5Ghz channel and gave it a different password, disabled 20/40 on the 2.4 channel, enabled IPv6 and that's about it. Devices are connected to both the 2.4Ghz and 5Ghz channels, and a Polycom unit is connected to one of the Ethernet ports. So far as I know, the operation of the router is flawless (and it is definitely fast).

I originally installed your add-on because I had lost my settings during the upgrade from Voxel 74.3SF to 74.4SF, and thought that I might have a defective RAM chip. I wanted your add-on as a means to identify the chip. But I haven't lost any settings since then, so I may not really have a chip issue. Most recently, I have continued to explore/test your add-on as an educational experience. I don't currently use any of the many features, but you never know. I'm curious and I like to learn.

As for the failure of "Bandwidth usage" to update properly, this morning (at 6h) I uninstalled your add-on. Rebooted. Used Telnet to clear your data. Then reinstalled your add-on. For about 2 hours, it appeared that this may have fixed the issue. But, as you can see from the attached screenshot (taken at 09h44), it did not. The Bandwidth Usage data has not updated since 08h22. I tried to force an update; the "this page was generated on" date and time incremented as it should, but the "last seen" did not (nor did any of the reported data).

This failure is in some ways consistent with other problems I have experienced (and previously reported to you). A feature will work for a period of time (between 2 and 4 hours) and then it will cease working. Some timer appears to be triggered, but I don't know what or how to identify it.

Fortunately, none of the add-on issues have affected the basic functionality of the router.

Please let me know if there are any tests that I can run for you.

 

[R. Gerrits]

occasionally I also see these messages when I do a net-wall restart:
Another app is currently holding the xtables lock. Perhaps you want to use the -w option?
Another app is currently holding the xtables lock. Perhaps you want to use the -w option?
Another app is currently holding the xtables lock. Perhaps you want to use the -w option?

So my guess, because you call iptables a lot in a loop, some of the attempts fail with the error above, and then it will insert duplicates.

I'm now trying with dumping the chain to a temporary file, and then during the loop, only grep that file instead of calling iptables.

Also noticed that if net-wall is restarted, while the cronjob is also running, then both are creating rules at the same time.
Perhaps is is better to remove the firewall-start-bwusage.sh and handle everything only from the cronjob?
(downside would be you loose a maximum of minute of data after a net-wall restart. (if cronjob runs every minute))

 

[kamoj]

I thought these errors came with Voxel R7800-V1.0.2.76.3SF.
That's why I implemented the mutex lock in net-wall...
It didn't help but I kept it in the code since it also take care of what you mention, and races between other code using the firewall.

So one clear mistake of me was to forget to use the mutex lock from within the bypassing script!
Still I don't know why it seems to work in the previous version of the add-on. It does not make sense.
The only change there was your eminent changes to the firewall rules.
And new Voxel release with new iptables/ipset handling etc.

(For me it's working good, no problems noticed with anything, but I'm still running (unofficial) R7800-V1.0.2.76.3SF.)

So if it works for you, please use the older add-on, or disable the Bandwidth usage/monitor!

I can not fix this now even if it's easy to add the mutex code to the bypassing. I just can't do anything know.

Thank you for checking things out and reporting.!!!

Be safe and keep reporting irregularities!
PS
The mutex lock restarts the process when it is locked, probably reason to different time zone?!

occasionally I also see these messages when I do a net-wall restart:
Another app is currently holding the xtables lock. Perhaps you want to use the -w option?
Another app is currently holding the xtables lock. Perhaps you want to use the -w option?
Another app is currently holding the xtables lock. Perhaps you want to use the -w option?

So my guess, because you call iptables a lot in a loop, some of the attempts fail with the error above, and then it will insert duplicates.

I'm now trying with dumping the chain to a temporary file, and then during the loop, only grep that file instead of calling iptables.

Also noticed that if net-wall is restarted, while the cronjob is also running, then both are creating rules at the same time.
Perhaps is is better to remove the firewall-start-bwusage.sh and handle everything only from the cronjob?
(downside would be you loose a maximum of minute of data after a net-wall restart. (if cronjob runs every minute))

 

[kamoj]

After you disabled the Bandwidth usage, did you check the cron jobs? They should be gone!

Also did tried that, but also then the duplicates reappear.

Had a look at the code you use to insert the rules, but at first glance I also cannot see anything wrong there....