Custom firmware build for R7800 v. 1.0.2.23SF & v. 1.0.2.24SF

[Voxel]

Continuation of

https://www.snbforums.com/threads/custom-firmware-build-for-r7800.36859/
https://www.snbforums.com/threads/custom-firmware-build-for-r7800-v-1-0-2-22sf.37222/

New version of my custom firmware build: 1.0.2.23SF.

What’s important in this release: mainly improvement of OpenVPN speed. I remind that I added OpenVPN v. 2.4.0 into my previous build 1.0.2.22SF, but now:

(1) It is possible from WebGUI to choose settings for pure OpenVPN v. 2.4.x clients, i.e. not compatible with v. 2.3.x clients. Users who still have 2.3.x clients should use compatible settings (as before, no changes here).
(2) 2.4.x mode uses AES-128-GCM instead of AES-128-CBC what leads to increase in speed and security.
(3) 2.4.x mode uses LZ4 compression algorithm, which is in general more fast and provides more high level of compression than LZO used in 2.3.x mode (improvements in speed).
(4) To provide more optimal use of LZ4 compression, I added external liblz4 library v. 1.7.5 for use instead of LZ4 compatible codes in OpenVPN 2.4.0 sources. According to info from LZ4 developers, starting from v. 1.7.3 they significantly increased its speed especially for ARMv6 and ARMv7 CPU, from their log:

Improved: Small decompression speed boost
Improved: Small compression ratio and speed improvement on small files
Improved: Significant speed boost on ARMv6 and ARMv7
​

So it is is strongly recommended to use 2.4.x mode if all of your clients are 2.4.x version. You should get increase in speed (theoretically ;-)).

Changes (vs 1.0.2.22SF):
1. WebGUI is changed to allow selection of OpenVPN 2.4.x clients.
2. WebGUI is changed to allow deselection of “Respond to Ping on Internet Port” after enabling OpenVPN server (WAN settings, thanks to staticfree).
3. External liblz4 v. 1.7.5 package is added (used in OpenVPN for LZ4 compression).
4. libusb package is upgraded 1.0.20->1.0.21.
5. libusb-compat package is upgraded 0.1.4->0.1.5.
6. zlib package is upgraded 1.2.9->1.2.11 (several fixes and improvement).
7. dnscrypt-proxy package is upgraded 1.7.0->1.9.4
8. dnscrypt-resolvers.csv is updated.
9. bridge-utils package is upgraded 1.5->1.6.
10. mtd-utils package is upgraded 1.5.0->1.5.2
11. Latest QoS Optimization DB is included into firmware.
12. Host tools: three components are upgraded to most recent versions (gmp, gperf and pkg-config).


Updated: new bug fixing version 1.0.2.24SF.
Changes (vs 1.0.2.23SF):
1. OpenVPN init script is updated (fixing bug reported by staticfree).
2. e2fsprogs package is upgraded 1.43.3->1.43.4.
3. e2fsprogs host tool is upgraded 1.43.3->1.43.4.
4. alsa-libs package is upgraded 1.1.0->1.1.3.
5. xz host tool is upgraded 5.2.2->5.2.3.


The link is:

https://www.mediafire.com/folder/tyj61i5uc610w/voxel-firmware

No reset is needed to upgrade from my previous versions.

(Thanks to vladlenas for additional testing).


Voxel.

 

[pege63]

Thanx m8

 

[vladlenas]

Thanks Voxel.
Another, great job!

 

[RMerlin]

To provide more optimal use of LZ4 compression, I added external liblz4 library v. 1.7.5 for use instead of LZ4 compatible codes in OpenVPN 2.4.0 sources. According to info from LZ4 developers, starting from v. 1.7.3 they significantly increased its speed especially for ARMv6 and ARMv7 CPU, from their log:

Interesting. That's something I was pondering at the time, but decided for now to go with the fastest route and stick to the built-in implementation, seeing that it was only a few months older than the latest libz4 release when I looked at their git repo. Thanks for the investigation.

 

[Csection]

Do to Norton Security. I am unable to d/l from that site.
It has a bad reputation with Norton. Any other way to get the d/l?

 

[Voxel]

Interesting. That's something I was pondering at the time, but decided for now to go with the fastest route and stick to the built-in implementation, seeing that it was only a few months older than the latest libz4 release when I looked at their git repo. Thanks for the investigation.

If it is interesting, I use two defines for LZ4 compilation:

-DLZ4_FORCE_MEMORY_ACCESS=1 -DXXH_FORCE_MEMORY_ACCESS=1

to make sure it will use ARMv7 optimization (I use option -mcpu=cortex-a15, but not -marh=armv7-a). Maybe not necessary for pure -march=armv7-a.

And "compress lz4-v2" in OpenVPN config files (server/client).

Voxel.

 

[Voxel]

Do to Norton Security. I am unable to d/l from that site.
It has a bad reputation with Norton. Any other way to get the d/l?

Probably you should just temporary disable your Norton. And enable it after downloading.

Voxel.

 

[Csection]

Probably you should just temporary disable your Norton. And enable it after downloading.

Voxel.

Yes! I was able to get it.
Thank you!

 

[TonyH]

Do to Norton Security. I am unable to d/l from that site.
It has a bad reputation with Norton. Any other way to get the d/l?

Can't you make an exception for that site. Or disable Norton just for downloading.

 

[rbird2]

Thank you Voxel for supporting our Netgear R7800's.

Installed your firmware and so far everything is working fine.

 

[RMerlin]

Do to Norton Security. I am unable to d/l from that site.
It has a bad reputation with Norton. Any other way to get the d/l?

I've been using Mediafire for years myself, and never heard of any issue with it and Norton Security (which I also use myself).

 

[Csection]

I've been using Mediafire for years myself, and never heard of any issue with it and Norton Security (which I also use myself).

I had to set the firewall to bypass to get it to d/l from that site.
I don't know why. Maybe something else was causing it to block or I have something set up wrong in my settings.

 

[sfx2000]

Thanks for the investigation.

Voxel is pretty sharp - still seems to be on the Cortex-A15 path with Krait (which I disagree with), but otherwise, he's doing good stuff...

We've chatted a bit with on-boarding here on SNB, and he's posting code on GitHub is a very good thing.

I have a fair amount of respect for Voxel...

 

[Voxel]

Voxel is pretty sharp - still seems to be on the Cortex-A15 path with Krait (which I disagree with), but otherwise, he's doing good stuff...

We've chatted a bit with on-boarding here on SNB, and he's posting code on GitHub is a very good thing.

I have a fair amount of respect for Voxel...

Thanks for your words.

Regarding Krait and Cortex-A15. Thought thrives on conflict

I respect opinions of experts and gurus, but I still prefer to trust my own eyes . Maybe, this is some side effect, let it be compiler-specific or whatever else, but… it works, and faster. Benchmarks. Moreover, produced by compiler binaries are smaller in size (what is important for firmware), in general of course it does mean nothing, not always optimization reduces a size of resulting binary, but in this concrete case I can conclude that produced binaries just more optimally use CPU/FPU instructions decreasing their amount for the same algorithm. Because of the same level of general optimization (-O2) and the same other options.

Voxel.

 

[sfx2000]

Regarding Krait and Cortex-A15. Thought thrives on conflict

Just don't get trapped in the rat-hole of over optimization

Mash bugs - make improvements - share code - everyone wins!

 

[sfx2000]

to make sure it will use ARMv7 optimization (I use option -mcpu=cortex-a15, but not -marh=armv7-a). Maybe not necessary for pure -march=armv7-a.

Might work for Krait - but BCM470x is still Cortex-A9, and a simple one at that (no VFP3 or Neon, so it's all core...) so better to just do -march=armv7-a

(FWIW - outside of NG or Asus - playing around with compiler options - Cortex-A7 and Cortex-A53 play very nicely with A15 optimizations to some degree, much better than A8/9 - and then play the VFP3/4 and Neon accordingly - has a lot to do with big.LITTLE configs there where A7/A15 or A53/A57 are paired up, so code needs to work for the little cores)

 

[Voxel]

Just don't get trapped in the rat-hole of over optimization

I know, I know . Reliablility in the first place.

Might work for Krait - but BCM470x is still Cortex-A9, and a simple one at that (no VFP3 or Neon, so it's all core...) so better to just do -march=armv7-a

BCM470x: Yes I know. I use AC68U and AC56U. I just had in mind that -march=armv7-a and -mcpu=cortex-a15 are incompatible options, and LZ4 codes check predefined __ARM_ARCH_7__ which is OK if using -march=armv7-a but not if to use -mcpu=cortex-a15. So I had to force "ARMv7 optimization explicitly with these two defines (-DLZ4_FORCE_MEMORY_ACCESS=1 -DXXH_FORCE_MEMORY_ACCESS=1).

Voxel.

 

[cybrnook]

Does this offer a VPN "Client" page? Also does it offer policy based routing similar to Merlins, routing only specific IP's over VPN?

 

[Voxel]

Does this offer a VPN "Client" page? Also does it offer policy based routing similar to Merlins, routing only specific IP's over VPN?

Sorry, no. Unfortunately Netgear does not provide full source codes in their GPL (I guess security reasons). Especially their net-cgi and net-wall (own firewall). So I am very limited in modification of WebGUI and in use of iptables. I had to make a lot of tricks just to add “2.4.x Clients” choice to WebGUI. And I am not sure that it even is possible to do even from console. Own Netgear’s precompiled firewall is called from several other precompiled binaries and spoils all iptables rules I set from console after some time of working. There should be some sophisticated tricks to overcome this.


So currently OpenVPN server is working the same way as in stock firmware, and only adds possibility to use some from OpenVPN 2.4.0 advantages such as LZ4 compression and AES-128-GCM instead of AES-128-CBC.

Voxel.

 

[staticfree]

Continuation of

https://www.snbforums.com/threads/custom-firmware-build-for-r7800.36859/
https://www.snbforums.com/threads/custom-firmware-build-for-r7800-v-1-0-2-22sf.37222/

New version of my custom firmware build: 1.0.2.23SF.

What’s important in this release: mainly improvement of OpenVPN speed. I remind that I added OpenVPN v. 2.4.0 into my previous build 1.0.2.22SF, but now:

(1) It is possible from WebGUI to choose settings for pure OpenVPN v. 2.4.x clients, i.e. not compatible with v. 2.3.x clients. Users who still have 2.3.x clients should use compatible settings (as before, no changes here).
(2) 2.4.x mode uses AES-128-GCM instead of AES-128-CBC what leads to increase in speed and security.
(3) 2.4.x mode uses LZ4 compression algorithm, which is in general more fast and provides more high level of compression than LZO used in 2.3.x mode (improvements in speed).
(4) To provide more optimal use of LZ4 compression, I added external liblz4 library v. 1.7.5 for use instead of LZ4 compatible codes in OpenVPN 2.4.0 sources. According to info from LZ4 developers, starting from v. 1.7.3 they significantly increased its speed especially for ARMv6 and ARMv7 CPU, from their log:

Improved: Small decompression speed boost
Improved: Small compression ratio and speed improvement on small files
Improved: Significant speed boost on ARMv6 and ARMv7
​

So it is is strongly recommended to use 2.4.x mode if all of your clients are 2.4.x version. You should get increase in speed (theoretically ;-)).

Changes (vs 1.0.2.22SF):
1. WebGUI is changed to allow selection of OpenVPN 2.4.x clients.
2. WebGUI is changed to allow deselection of “Respond to Ping on Internet Port” after enabling OpenVPN server (WAN settings, thanks to staticfree).
3. External liblz4 v. 1.7.5 package is added (used in OpenVPN for LZ4 compression).
4. libusb package is upgraded 1.0.20->1.0.21.
5. libusb-compat package is upgraded 0.1.4->0.1.5.
6. zlib package is upgraded 1.2.9->1.2.11 (several fixes and improvement).
7. dnscrypt-proxy package is upgraded 1.7.0->1.9.4
8. dnscrypt-resolvers.csv is updated.
9. bridge-utils package is upgraded 1.5->1.6.
10. mtd-utils package is upgraded 1.5.0->1.5.2
11. Latest QoS Optimization DB is included into firmware.
12. Host tools: three components are upgraded to most recent versions (gmp, gperf and pkg-config).

The link is:

https://www.mediafire.com/folder/tyj61i5uc610w/voxel-firmware

No reset is needed to upgrade from my previous versions.

(Thanks to vladlenas for additional testing).

Voxel.

Okay Voxel, I've loaded your new 23SF firmware over my original Netgear R7800 firmware and it started working fine for the most part.
It's been running for a couple of days with my original configuration. All the major functions I use are working solid and fine. Just one area I need to ask you about... the OpenVPN does not work for me any longer. I've tried using the standard version as well as your new v2.4.x setting for UDP and auto settings. My Android phone app which worked fine with the original Netgear firmware always connected immediately and fine. But I cannot get this 23F version to connect at all. Tried 2 different OpenVPN apps from Android playstore and they won't connect. Is there something else I need to do? Other than this quirk, I notice that the firmware does run quicker, the GUI pages appear more snappier than original firmware. So everything looks great for me except for the OpenVPN issue.