OpenVPN performance

[EnF70]

Your existing rule does seem to do the trick.
Also, I've tested with normal VPN traffic (outside VPN client <-> internal server) and I don't see any improvements in terms of speed or CPU usage when marking packets.
The only case I see improvements is when testing with Speedtest (which means routing all internet traffic through the VPN server and accessing outside servers)
I will test some more.

 

[MoBlues]

Going to try that out. Thanks @DomFel

I finally got the UDP up and running and tested (also, it is faster), but one really strange thing happened. I had policy routing in place (TCP) for my main PC and all other devices would bypass the VPN, but when I switched to UDP the "Redirect Internet Traffic" setting reverted to NO and my WAN IP was still being reported as the VPN IP address on the internet. I checked the IP for my Phone, Tablet, etc, and they all showed the VPN's IP. Verified with a traceroute on the Router side to confirm as well. My use case is to get the highest speed out of the VPN and hide my IP address on torrents, but am I really protected and are the packets AES-256 encrypted, how do I know with the "NO" setting on? I know for 100% that Policy Based routing works, but this setting just confused me.

 

[joegreat]

The only case I see improvements is when testing with Speedtest (which means routing all internet traffic through the VPN server and accessing outside servers)
I will test some more.

I also used external speedtest (Ookla) which showed the same result.

By the way: I am using UDP protocol for my VPN (suggested by the provider for router and smartphones). Maybe this is the reason why CPU is low and no difference?

 

[EnF70]

I also used external speedtest (Ookla) which showed the same result.

By the way: I am using UDP protocol for my VPN (suggested by the provider for router and smartphones). Maybe this is the reason why CPU is low and no difference?

I am using UDP as well.
I think the reason in your case is the rule you already have in the PREROUTING chain.

 

[peraburek]

discussion is really interesting, but just one question regarding OpenVPN and CTF
does it only apply to OpenVPN server run on ASUS router (currently using AC68U)
or does it apply to OpenVPN client run on ASUS router as well?

am I right?
server rules:

iptables -t mangle -A PREROUTING -i tun21 -j MARK --set-mark 1
iptables -t mangle -A PREROUTING -i tun22 -j MARK --set-mark 1

client rules:

iptables -t mangle -A PREROUTING -i tun11 -j MARK --set-mark 1
iptables -t mangle -A PREROUTING -i tun12 -j MARK --set-mark 1

does skipping CTF makes sense if ASUS router is running OpenVPN client?
or skipping CTF only makes sense if ASUS router is running OpenVPN server
since I don't use external VPN service, I am connecting two sites (site-to-site) with ASUS routers thanks to Merlins magic (great firmware!)
Site1 - Asus OpenVPN server -> Internet <- Site2 - Asus OpenVPN client

 

[kvic]

I applied the latest code to my AC87U but I don't see any improvement in OpenVPN with CTF enabled.
kvic mentioned that he managed to get better performance making those changes in 378.55 with CTF enabled.
So, question is, how can I replicate this with the latest code?
Are you sure no changes are needed? Not even CTF_PPTP_L2TP=n in "src-rt-6.x.4708/target.mak"?

CTF_PPTP_L2TP=n is still required if it applies to your model. I believe it's the essential not even the ctf.o itself.

I only tried in 378.55 as I mentioned earlier in the thread and on AC56U. So YMMV..

 

[kvic]

@MoBlues use UDP protocol not TCP. Way faster.

Not a rule of thumb really..

If people mark packets in iptables, might try TCP as well...perhaps faster!

 

[Goobi]

[QUOTE="
EDIT: Just discovered that on a AC68U, using the OpenVPN server with the TAP device (instead of TUN) is valid workaround for the low performance with NAT acceleration enabled. Just remember to set the valid IP range for the OpenVPN server as the DHCP option didn't work for me.[/QUOTE]

When trying to change to TAP, I get the following error in the syslog and my clients cannot connect:

NOTE: unable to redirect default gateway -- VPN gateway parameter (--route-gateway or --ifconfig) is missing

I am running PIA, could it be something on their end or am I perhaps missing some other changes that should go with the TUN to TAP change?

Thanks in advance.

 

[kvic]

Both ends need to be TUN or TAP...can't mismatch.

Also TAP won't work for iOS, and not as efficient as TUN in terms of CPU cycles.

 

[kvic]

Thanks for testing. That way, you can keep CTF enabled, just need to mark the VPN traffic so it can bypass it.

Now the big question is whether this is a temporary bug that will eventually be fixed by Broadcom, or if it would be better to implement this as a long-term solution in the firmware itself.

Interesting, so I gave it a try on Alpha 3 + RT-AC56U.

It produces remarkable difference in Alpha3. Here are quick observations:

• Without marking, OpenVPN is painfully slow, not quite usable.
• With marking, speed is restored back to "normal", similar to stock 378.55 + CTF enabled that I recall from my memory.

I suspect something is wrong in Alpha 3 that upsets OpenVPN very much. Would be interesting to try marking on 378.55 and other combination of comparisons...but I don't have time nor the need. So leave to fellow members..

Though old and new SDK might share the same kernel version, I think broadcom continues to refine their changes in the source tree. Some bugfix/enhancement in CTF simply do not trickle down into AC56/AC68, especially when no complaints from users.

 

[spopinski]

This is my script:

cat /jffs/scripts/firewall-start

#!/bin/sh

iptables -t mangle -A PREROUTING -i tun21 -j MARK --set-mark 1
iptables -t mangle -A PREROUTING -i tun22 -j MARK --set-mark 1

Verify after reboot that you have the rules in the PREROUTING chain by running:
iptables -L -t mangle -vn

Chain PREROUTING (policy ACCEPT 1974K packets, 1492M bytes)
pkts bytes target     prot opt in     out     source               destination
    0     0 MARK       all  --  tun21  *       0.0.0.0/0            0.0.0.0/0            MARK set 0x1
188K   26M MARK       all  --  tun22  *       0.0.0.0/0            0.0.0.0/0            MARK set 0x1

Cool. Gonna test it later today. Thanks!

 

[DomFel]

• Without marking, OpenVPN is painfully slow, not quite usable.
• With marking, speed is restored back to "normal", similar to stock 378.55 + CTF enabled that I recall from my memory.

I suspect something is wrong in Alpha 3 that upsets OpenVPN very much.

Not for everybody. With or without markings doesn't make a difference here, CTF enabled. 24mbps connection, server 3500 miles away (to avoid censorship), I get steady 20/21mbps.

P.S.: forgot to mention the AC56U is overclocked: 1000,800.

 

[kvic]

Not for everybody. With or without markings doesn't make a difference here, CTF enabled. 24mbps connection, server 3500 miles away (to avoid censorship), I get steady 20/21mbps.

P.S.: forgot to mention the AC56U is overclocked: 1000,800.

I believe you're testing OpenVPN client. Joegreat above also didn't see any improvement. The packet marking thing seems killing some brain cells from a few folks. What if it turns out a red herring..

I'm going to give another try with the latest alpha 3 later tonight. Will report back..

EDIT:

TL;DR I cannot reproduce the same tremendous effect in the latest Alpha 3..

Previous Alpha 3 I built with the stock Buildroot toolchains. The latest Alpha 3 I built with NPTL (http://www.snbforums.com/threads/arm-toolchains-with-nptl-for-asuswrt-merlin.27938/). Other than that no major difference. I know latest Alpha 3 has marking rules added. So flush everything in iptables before my quick tests.

Test scenario

A kind of road warrior. That is the router acting as VPN server and the road warrior connects from Internet to the VPN server and access the Internet through the VPN server. Data access uses speedtest dot net. CTF on. RT-AC56U.

It's the same test scenario before and now. With marking or not, ultimately no difference in the maximum throughput in tonight's tests.

One caveat that I observed tonight is that without marking the first run usually shows bumpy speed and less than half of max achievable. But the second and sequent runs of the same speedtest server will bring back the speed.

With marking, the first run almost near maximum achievable throughput. Perhaps coincidence.. I'm not sure. I didn't do repeat enough number of times to statistically prove a thing.

 

[kvic]

Also worth noting that TCP seems capable of better coping with CTF on. As a quick reference, using TCP in the latest Alpha 3, I could get performance on par with the best I had seen in my custom 378.55 (kinda surprise!). Using UDP, download is comparable but upload is only slightly more than half of download.

I will keep the marking rule as I don't see any slow down. I've revised with more efficiency (I believe). Use on your own discretion:

iptables -t mangle -A PREROUTING -i tun21 -m conntrack --ctstate NEW -j MARK --or-mark 0

EDIT: revised the rule

 

[DomFel]

I believe you're testing OpenVPN client.

Yes, client here not server. Why don't you try dslreport's speedtest? They use Amazon AWS servers, in my case they work way better than speedtest dot net.

 

[peraburek]

@kvic - thank you for your marking rule

since I am using site to site VPN connection with 2x AC68U I have setup marking rules on both sides, changing interfaces on server (tun21) and on client (tun11)
rule of logic would say when OpenVPN client is uploading data to OpenVPN server packet should be marked to skip CTF
when OpenVPN server is uploading data to OpenVPN client than data should be marked to skip CTF

I will test it soon with 100/100 Mbps line, I think OpenVPN should be maxed out somewhere around 50-60Mbps (at the moment I can max out 32Mbps line with AES-256)

do you know how to assign openvpn client 1 to cpu 2 ?
I know how to do it with taskset and PID# but how to grep PID# and make script for OpenVPN to do taskset automatically

 

[EnF70]

iptables -t mangle -A PREROUTING -i tun21 -m state --state NEW -j CONNMARK --or-mark 0

Can you explain what this rule should do?
I don't understand the binary OR with 0 (zero)
Is this in addition to the original marking?

 

[kvic]

@kvic - thank you for your marking rule

I think I made a terrible mistake. Revised in my post. I believe that one shall work. Again, the effect is not noticeable in the latest Alpha 3.

People would see some benefit if load up an older firmware such as the stock 378.55. Even there the improvement is not astonishing as disabling CTF altogether.

I would assert the speed boost seen in latest Alpha 3 the magic in CTF improvement itself.

I will test it soon with 100/100 Mbps line, I think OpenVPN should be maxed out somewhere around 50-60Mbps (at the moment I can max out 32Mbps line with AES-256)

I saw close to 70Mbit/s throughput on a 1.4GHz CPU with AES-128-CBC.

do you know how to assign openvpn client 1 to cpu 2 ?
I know how to do it with taskset and PID# but how to grep PID# and make script for OpenVPN to do taskset automatically

taskset -p 2 $(pidof vpnclient1)

The process name is my guess. Check in top...

 

[kvic]

@EnF70, I think the mark set e.g. 0x1 in your case isn't consumed anywhere. I believe the idea is that we want the packets to bypass CTF with the actions of marking. What actually marked doesn't matter. Binary OR gives us a sweet deal.

I believe the difference is there but I can't measure it at hundreds of megabit speed.

 

[peraburek]

There is a reply above pending moderator approval..

can you please send it as PM "start a conversation" ?