VPN speed gradually slows down vs WAN

[markran]

Above is the VPN hourly speed tests

...and WAN speed tests for the same period. If I restart the OpenVPN connection it always immediately goes back to higher speeds and the same declining trend follows. I've also verified the slowing trend via manual speed tests from my laptop browser. The VPN provider (Surfshark) says they don't do any slowing down or shaping. I understand that VPN speeds are slower than the underlying WAN but I would expect the speed difference to be fairly consistent over time. I'm a newbie to Merlin but loving it so far.

My Question: What are some ways I could troubleshoot what's causing this?

Details / Logs
My AX3000 has Merlin 386.5_2 + spdMerlin/scMerlin and I'm using the .ovpn profile from Surfshark's website with no changes. There is only this one VPN connection and everything is going through it. The WAN connection is 4G cellular phone (Visible/Verizon). The signal strength is stable and good. The variation in raw WAN speed decline is likely just peak hour usage in this sparse rural area.

Log for the last two hourly speed tests.

Apr 28 08:12:01 spdMerlin: Starting speedtest using IdeaTek Telcom (Hutchinson, KS, United States) for WAN interface
Apr 28 08:12:33 spdMerlin: Speedtest results - Download: 63.30 Mbps (data used: 71.5 MB ) - Upload: 19.24 Mbps (data used: 23.5 MB )
Apr 28 08:12:33 spdMerlin: Connection quality - Latency: 58.30 ms (3.85 ms jitter) - Packet Loss: 0.0%
Apr 28 08:12:33 spdMerlin: Starting speedtest using CenturyLink (Phoenix, AZ, United States) for VPNC1 interface
Apr 28 08:12:58 spdMerlin: Speedtest results - Download: 12.86 Mbps (data used: 15.6 MB ) - Upload: 13.77 Mbps (data used: 16.6 MB )
Apr 28 08:12:58 spdMerlin: Connection quality - Latency: 78.06 ms (2.53 ms jitter) - Packet Loss: 0.0%
Apr 28 08:12:58 spdMerlin: Retrieving data for WebUI charts
Apr 28 08:14:37 ovpn-client1[15614]: WARNING: 'link-mtu' is used inconsistently, local='link-mtu 1633', remote='link-mtu 1581'
Apr 28 08:14:37 ovpn-client1[15614]: WARNING: 'auth' is used inconsistently, local='auth SHA512', remote='auth [null-digest]'
Apr 28 08:19:02 ovpn-client1[15614]: read UDP [EMSGSIZE Path-MTU=1428]: Message too long (code=90)
Apr 28 08:21:02 dnsmasq-dhcp[15747]: not giving name Pro-8 to the DHCP lease of 192.168.1.237 because the name exists in /jffs/addons/YazDHCP.d/.hostnames with address 192.168.1.206
Apr 28 08:29:06 ovpn-client1[15614]: read UDP [EMSGSIZE Path-MTU=1428]: Message too long (code=90)
Apr 28 08:32:58 dnsmasq-dhcp[15747]: not giving name Pro-8 to the DHCP lease of 192.168.1.237 because the name exists in /jffs/addons/YazDHCP.d/.hostnames with address 192.168.1.206
Apr 28 08:40:09 dnsmasq-dhcp[15747]: not giving name Pro-8 to the DHCP lease of 192.168.1.237 because the name exists in /jffs/addons/YazDHCP.d/.hostnames with address 192.168.1.206
Apr 28 08:55:51 ovpn-client1[15614]: read UDP [EMSGSIZE Path-MTU=1428]: Message too long (code=90)
Apr 28 09:05:31 kernel: eth3 (Int switch port: 3) (Logical Port: 3) (phyId: b) Link Up at 1000 mbps full duplex
Apr 28 09:07:27 ovpn-client1[15614]: read UDP [EMSGSIZE Path-MTU=1428]: Message too long (code=90)
Apr 28 09:12:01 spdMerlin: Starting speedtest using IdeaTek Telcom (Hutchinson, KS, United States) for WAN interface
Apr 28 09:12:29 spdMerlin: Speedtest results - Download: 31.64 Mbps (data used: 38.2 MB ) - Upload: 13.51 Mbps (data used: 15.7 MB )
Apr 28 09:12:29 spdMerlin: Connection quality - Latency: 57.51 ms (5.76 ms jitter) - Packet Loss: 0.0%
Apr 28 09:12:29 spdMerlin: Starting speedtest using CenturyLink (Phoenix, AZ, United States) for VPNC1 interface
Apr 28 09:12:54 ovpn-client1[15614]: WARNING: 'link-mtu' is used inconsistently, local='link-mtu 1633', remote='link-mtu 1581'
Apr 28 09:12:54 ovpn-client1[15614]: WARNING: 'auth' is used inconsistently, local='auth SHA512', remote='auth [null-digest]'
Apr 28 09:12:55 spdMerlin: Speedtest results - Download: 10.84 Mbps (data used: 13.1 MB ) - Upload: 11.36 Mbps (data used: 12.9 MB )
Apr 28 09:12:55 spdMerlin: Connection quality - Latency: 80.61 ms (12.09 ms jitter) - Packet Loss: 3.6%
Apr 28 09:12:55 spdMerlin: Retrieving data for WebUI charts

Here is the Custom Configuration from the SurfShark .ovpn file.

resolv-retry infinite
remote-random
tun-mtu 1500
tun-mtu-extra 32
mssfix 1450
ping 15
ping-restart 0
ping-timer-rem
remote-cert-tls server
pull
fast-io
cipher AES-256-CBC

 

[markran]

It gets stranger still. I disabled the Service State on the OpenVPN page for 30 seconds and then re-enabled it. Then did a manual speed test of VPN vs WAN.

The speed jumps up to more 'normal-expected' VPN slowdown (45Mbps vs 62Mbps). Yet it returns to the previous slowing trend at the next hourly auto-test!

Meanwhile the raw WAN speed is carrying on its pattern unaffected by whether I interrupt the VPN connection or not. BTW, during all of this the overall bandwidth usage is low except for the speed tests. I'm the only one home and just doing light web browsing trying to figure this out. Any thoughts on what could be happening?

It would be surprising (because they are so big) but I guess SurfShark could just be blatantly lying to the world in all their claims about not slowing down connections but why would they speed it back up after a brief disconnect yet *then* slow it back down to the same consistently diminishing rate trend they were previously on? If they were being deceptive wouldn't they mask any slowdown using an irregular pattern? Also, I'm barely putting any bandwidth through the connection except for speed tests, so I should be the last person they'd want to slow down.

Alternatively, something in the router is slowing down over time, maybe due to some buffer accumulation or ???

 

[CasualObserver]

There was a similar thread noted below touching on this issue in the past as well. I have updated it to reflect issue persist,

OpenVPN Speed Slows Over Time

Apologies if this has been answered before, I couldn't find it if it has. Running 386.2_4 on an Asus RT-AC5300. When I first boot the router, my connection speeds over the VPN are pretty good, around 60 mbps down, 11 up. Over time this speed degrades, dropping about 10 mbps per day...

www.snbforums.com

I will comment that as far as i am aware it has been present in all releases put out for at least a few years.

Below was my update to that thread:

After observing this further ... issue still persisted while on 386.4 and also behavior continued on subsequent releases including 386.5_2.

There is another thread included below that has reported similar issues:

- It included link to this thread


Below is data i can share that reflects behavior over 30 days with speeds returning to normal after reboots:

Traffic for 1 IP is shaped through VPN Director and routed to NordVPN provider, all other IP traffic remains responsive through normal WAN flow at a consistent band width rate.

 

[markran]

CasualObserver, thanks for posting the link to the other thread on this issue. In all my searching I never found that post. Probably because searches on terms for *this* issue get lost in all the posts incorrectly reporting the standard 'expected' VPN slowdown as an issue.

I'm sorry others are having the same issue but am happy to learn the issue arises with other providers and on other Merlin router hardware. At least it's not my particular unit, provider or something dumb I did in my very basic setup.

The consistent curves indicate it's a progressive degeneration more related to time than traffic. I'd be happy to help debug what's causing this but don't know where to start.

 

[CaptainSTX]

There was a similar thread noted below touching on this issue in the past as well. I have updated it to reflect issue persist,

OpenVPN Speed Slows Over Time

Apologies if this has been answered before, I couldn't find it if it has. Running 386.2_4 on an Asus RT-AC5300. When I first boot the router, my connection speeds over the VPN are pretty good, around 60 mbps down, 11 up. Over time this speed degrades, dropping about 10 mbps per day...

www.snbforums.com

I will comment that as far as i am aware it has been present in all releases put out for at least a few years.

Below was my update to that thread:

After observing this further ... issue still persisted while on 386.4 and also behavior continued on subsequent releases including 386.5_2.

There is another thread included below that has reported similar issues:

- It included link to this thread


Below is data i can share that reflects behavior over 30 days with speeds returning to normal after reboots:

Traffic for 1 IP is shaped through VPN Director and routed to NordVPN provider, all other IP traffic remains responsive through normal WAN flow at a consistent band width rate.

I don't see that on my AC86 using StrongVPN. The download speed tends to vary more (145 - 200 Mbps) than the WAN speed but no repetitive pattern as you are showing and no need for a reboot for it to recover. I attribute the fluctuations time of day network congestion.

 

[CasualObserver]

Not sure if behavior difference would be associated with VPN setup on Asus-WRT Merlin, setup in place which presents issue is where Redirect Internet Traffic Through Tunnel is setup via VPN Director as follows:

A redirect using policy is in place

Only one IP is assigend to flow through tunnel as configured below:

Not sure if the above type setup would be similar to what is in place on CaptainSTX's hardware, just sharing this in the event that it is wrapped into some form of rules constraint that causes what appears to be Client VPN performance degradation over time with setup noted above.

 

[CaptainSTX]

Not sure if behavior difference would be associated with VPN setup on Asus-WRT Merlin, setup in place which presents issue is where Redirect Internet Traffic Through Tunnel is setup via VPN Director as follows:

A redirect using policy is in place

View attachment 41029

Only one IP is assigend to flow through tunnel as configured below:

View attachment 41028


Not sure if the above type setup would be similar to what is in place on CaptainSTX's hardware, just sharing this in the event that it is wrapped into some form of rules constraint that causes what appears to be Client VPN performance degradation over time with setup noted above.

Here is my VPN director running three VPN clients. Also the 30 day chart from VPN client 3.

 

[redbird71]

@markran did you find out what the problem was?
I don't know if this is related but I've got two AC68U connected vs OpenVPN. Both have a 1000Mbps WAN connection but if I transfer files I get CONSISTENT ~25Mbps effective speed. This is a windows share that does 500Mbps easy when accessed from a local device.

 

[markran]

@markran did you find out what the problem was?
I don't know if this is related but I've got two AC68U connected vs OpenVPN. Both have a 1000Mbps WAN connection but if I transfer files I get CONSISTENT ~25Mbps effective speed. This is a windows share that does 500Mbps easy when accessed from a local device.

No, I worked at it but hit a wall and never made any more progress due to lack of anything else to try.

From what I understand of your post I'm not sure your issue is related since my issue is between my 58u and the entire Internet when connected via OpenVPN Client to Surfshark vs direct connect (no VPN). If you're running your own VPN server locally that's a lot of unrelated stuff that's different from the scenario discussed above. One apparent symptom that seems unique to the issue I (and some others) have is the gradual progressive slowing over time despite changing absolutely nothing. It starts out fast and over many hours & days gets slower and slower. Stop and reconnect the VPN connection and it's instantly fast again.