OpenVPN performance

[RMerlin]

A big surprise to me: the OpenVPN guys did a damn good job at optimising AES cipher for ARM! Forget about the rest use it if you're on ARM (regardless client or server).

The credit goes to the OpenSSL devs rather. AES is optimized on ARM-based devices by having the cipher code written directly in optimized assembly code. When I backported that ASM code from 1.0.2 to 1.0.0 I measured a very significant performance gain from it.

Only SHA and AES are ASM-optimized, that's why AES is strongly recommended over BF for performance reasons.

 

[kvic]

The credit goes to the OpenSSL devs rather. AES is optimized on ARM-based devices by having the cipher code written directly in optimized assembly code. When I backported that ASM code from 1.0.2 to 1.0.0 I measured a very significant performance gain from it.

Only SHA and AES are ASM-optimized, that's why AES is strongly recommended over BF for performance reasons.

You're right. On a second thought, openssl instead of openvpn makes much more sense. Didn't know they code AES in assembly..no wonder.

Entware has a version of openvpn linked with polarssl. What are the advantages of polarssl compared to openssl?

 

[kvic]

People who had started using openvpn+openssl 1.0.0 with Merlin backported ASM shall really thank him for doing that and got a head start on speed boost.

The assembly optimisation for AES is remarkable. Here is my benchmark result (AC56U 1200,667):

         type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
1 core:  des cbc          17532.92k    18784.28k    18721.31k    19165.58k    19153.94k
1 core:  blowfish cbc     31622.84k    36396.54k    37855.89k    37943.32k    38010.88k
1 core:  aes-128 cbc      38315.71k    42108.93k    43497.05k    44069.46k    44083.37k
2 cores: aes-128 cbc      76674.92k    84037.65k    86799.45k    87559.51k    87823.70k

This resonates with the throughput difference we saw earlier in #79. AES-128 beats blowfish and des across the board. Look at the doubling when both cores are used. That's where OpenVPN can be shooting once it's multithread'ed.

To benchmark, type

openssl speed <cipher>
## or for two cores
openssl speed -multi 2 <cipher>

I'm interested in seeing the boost for a higher memory clock. Can someone benchmark @1200,800 (@Calisro ? ) Would be interesting if someone can do stock clock at 800,553 too.

 

[Calisro]

1200/800. I doubt memoty overclock would make any difference here...

256_ASM -DSHA512_ASM -DAES_ASM -DBSAES_ASM -DGHASH_ASM
the 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
des cbc          17435.95k    18614.15k    19353.16k    19062.66k    19597.78k
blowfish cbc     31413.37k    36117.18k    37558.13k    37964.11k    37832.15k
aes-128 cbc      38896.32k    42045.16k    43117.68k    44340.97k    43843.81k

 

[malau]

Tested on my AC56U running OpenVpn server with a WAN up 100Mbps/down 100Mbps. Client external to LAN, raw 89Mbps/91Mbps. With OpenVpn 8Mbps / 13Mpbs.

Apparently AC56U CPU is NOT the bottleneck. Plenty idle cycles are there. Let's figure out something..

I have the same situation with r7000, after disable hw nat, speed goto 50mbs, but the wifi nat speed goes down 30%, what can I do to fix it, thx

 

[kvic]

I have the same situation with r7000, after disable hw nat, speed goto 50mbs, but the wifi nat speed goes down 30%, what can I do to fix it, thx

Sorry, I have no experience with R7000 nor a netgear router.

 

[malau]

Does the Nat performance affected without hw nat in asus router?

 

[kvic]

On my AC56U, I see zero impact on WAN-LAN throughput and non-observable increase on CPU load with HW NAT off. My WAN is 100Mbps.

I think people with 1Gbps WAN will observe hit on the throughput. Starting with 300Mbps WAN, people may observe increasing CPU load. A simple speedtest maybe give you a peace of mind.

 

[kvic]

I grabbed the low hanging fruit by setting "Hardware NAT" to Disable. Seems to me TUN device and Broadcom's CTF module not only cannot get along in the little kernel but get into intense litigation. My tests done on 378.55

For people having similar performance issue, may give it a try. Please provide feedback on your results. I would like to hear if you observe the same, in particular

1. Openvpn Server throughput is much lower when "Hardware NAT" is set to Auto
2. CPU utilisation is much higher (mostly by SIRQ up to 90%) when "Hardware NAT" is set to Auto (and Tool's page indicate it's indeed enabled).

EDIT: A better way to show CPU utilisation is to telnet/ssh/putty into Asus, type "top -d1" and then press "1". Please report both CPU0 & CPU 1 utilization at the top of the screen.

This issue has puzzled me ever since my discovery. With luck, I found a cure.

The issue was with HW NAT on, speedtest.net shows 5Mbps/15Mbps over OpenVPN (wire speed 100Mbps full duplex). With HW NAT off, speedtest net goes back to normal (60-ish Mbps down/up).

I had a weird idea last night and found a couple of CTF drivers in the firmware repository. What if playing a mix&match..? Boom, indeed one set is better!

I come to realise turning off HW NAT is really not a cool thing regardless how you may think it's not absolutely necessary. Now, I don't have to.

CPU utilisation of speedtest.net over OpenVPN. The chart is taken when speedtest net reports ~68Mbps down/up (core 2 runs OpenVPN server. CPU clock 1.4GHz).

Originally without HW NAT, I can achieve similar speed but core 1 will be near 100%.

I can't be happier with the little AC56U. Just not sure on its successor..

 

[ronan_zj]

I have the same issue. without OpenVPN client on, my speed test is 170Mbps, but the speed drops to 30Mbps when I use OpenVPN. In addition, I can get the full speed if I use VPN client software on my PC, so I guess the AC68R CPU can not handle too many tasks.

 

[kvic]

I have the same issue. without OpenVPN client on, my speed test is 170Mbps, but the speed drops to 30Mbps when I use OpenVPN. In addition, I can get the full speed if I use VPN client software on my PC, so I guess the AC68R CPU can not handle too many tasks.

This thread is mainly discussion on VPN server speed on ASUS routers 'cos VPN client speed also depends on your VPN provider, not fully in your control.

Given that said 30Mbps as vpn client is about right on AC68R with stock clock.

If you overclock to 1.4GHz (which may brick your router), you may hit near 70Mbps. If your router is overclocked already, then t's a config mismatch between your router and your VPN provider.

A vendor supplied PC client is usually fine tuned to work optimally with its server.

 

[Henk59]

My router is AC66
I have 200Mbps down and 20Mbps up from my ISP.
With OpenVPN then it is 14.5 down and 12.5 up (IPVanish).

 

[tyspeed42]

I'm able to get about 59/4 using AC68U router Overclocked to 1200/666 via ssl putty
"nvram set clkfreq=1200,666
"nvram commit"
"reboot"
no heat issues, overclock stable and no other issues with router with using Client 1 OpenVPN through PIA using AES-128-CBC, which I have been told anything higher is redundant encryption.

Only issue I run into is attempting to remote back home, just about everything I have tried using Open Vpn Server 2 has worked.

Anyone have some ideas to remote connect back home through? I normally use default settings, export certs to all devices.

Using Merlin Fork 374.43_2-14j9527 Router Firmware

 

[john9527]

Only issue I run into is attempting to remote back home, just about everything I have tried using Open Vpn Server 2 has worked.

Anyone have some ideas to remote connect back home through? I normally use default settings, export certs to all devices.

You need to add the ip address range of your server 2 clients to the be excluded from your VPN client (turn on policy based routing and route the server address range to WAN). Server 2 defaults will be 10.16.0.0/24 if you didn't change it.

 

[tyspeed42]

Hey John, thanks for replying. Thanks to you and Merlin for the work on the firmware and Merry Xmas to you both.
Sorry if these are retarded questions, never ran dual vpns, one client outgoing and one server for incoming on the same router before.
Do I route vpn server address 10.16.0.0-10.16.0.24 to the internal router ip, or external ip, for example 192.168.1.1 or the Public ip? Or The starting DHCP server ips 192.168.1.2-192.168.1.255 do I need to choose VPN, or WAN?
Or if you have a example to include so I understand.

 

[john9527]

Here's my setup for VPN Client 1 all local traffic through PIA, and the Server 1 bypassing the client.



BTW....I'm connected through my VPN Server to take that scrrenshot

 

[tyspeed42]

Here's my setup for VPN Client 1 all local traffic through PIA, and the Server 1 bypassing the client.

View attachment 5178

BTW....I'm connected through my VPN Server to take that scrrenshot

Thanks Much Appreciated.
We need to start a donation thread for you John like Merlins for all your hardwork.

 

[LexLuthor]

I have an Asus AC68U on FiOS 50M/50M and VPN Unlimted and I'm limited to around 30Mbps using "Default" encryption cipher. From reading this, it sounds like I might get some higher speeds using AES-128-CBC. Is that correct?

But, I've tried all of the AES options and I can't connect to the VPN using any of them. The only option that seems to work other than default is BF-128.

Does that mean that VPN Unlimited doesn't support AES and only supports Blowfish?

 

[joegreat]

I have an Asus AC68U on FiOS 50M/50M and VPN Unlimted and I'm limited to around 30Mbps using "Default" encryption cipher. From reading this, it sounds like I might get some higher speeds using AES-128-CBC. Is that correct?

Simple rule: The lower the encryption the higher the throughput of the small ARM CPU...

But it also depends on your VPN provider: I have 40-50 MBit downstream with Perfect Privacy and 10 MBit upstream using AES256 (default)...

 

[LexLuthor]

Simple rule: The lower the encryption the higher the throughput of the small ARM CPU...

But it also depends on your VPN provider: I have 40-50 MBit downstream with Perfect Privacy and 10 MBit upstream using AES256 (default)...

Yes, that makes sense, but I can't get OpenVPN to work using another other than Default or Blowfish encryption cipher. I assume default must be blowfish then. If I choose any of the AES options, I can't access the internet and the VPN disconnects after a few seconds or so. Would that be normal that I can't use any of the AES options?