spdMerlin + CAKE [+ VPN]

[Lynx]

Fantastic add on Jack Yaz. AutoBW works perfectly with CAKE even with custom CAKE script override.

For those using auto bandwidth calculations - do you still have the tests run every hour on 12 and 42 as default? I can see one issue with this is the tests may get distorted by other general use of the connection at those times. Would be interested to know how others handle this.

Otherwise, in terms of the script I only changed the line:

echo "SELECT avg($metric) FROM (SELECT $metric FROM spdstats_WAN ORDER BY [Timestamp] DESC LIMIT $avgcalc);"

such that spdstats_WAN is replaced by spdstats_VPNC1:

echo "SELECT avg($metric) FROM (SELECT $metric FROM spdstats_VPNC1 ORDER BY [Timestamp] DESC LIMIT $avgcalc);"

because I want the auto bandwidth calculation based on average VPN results. Works with this tiny change. Kudos in terms of coding in facilitating such a straightforward change.

Albeit did take me a little while to figure that one out!

Jack Yaz might be nice to make that a configurable parameter in GUI for those like me who use VPN for everything.

 

[Jack Yaz]

Fantastic add on Jack Yaz. AutoBW works perfectly with CAKE even with custom CAKE script override.

For those using auto bandwidth calculations - do you still have the tests run every hour on 12 and 42 as default? I can see one issue with this is the tests may get distorted by other general use of the connection at those times. Would be interested to know how others handle this.

Otherwise, in terms of the script I only changed the line:

echo "SELECT avg($metric) FROM (SELECT $metric FROM spdstats_WAN ORDER BY [Timestamp] DESC LIMIT $avgcalc);"

such that spdstats_WAN is replaced by spdstats_VPNC1:

echo "SELECT avg($metric) FROM (SELECT $metric FROM spdstats_VPNC1 ORDER BY [Timestamp] DESC LIMIT $avgcalc);"

because I want the auto bandwidth calculation based on average VPN results. Works with this tiny change. Kudos in terms of coding in facilitating such a straightforward change.

Albeit did take me a little while to figure that one out!

Jack Yaz might be nice to make that a configurable parameter in GUI for those like me who use VPN for everything.

i forsee problems where the vpn running much slower than your WAN, e.g. a dodgy/overloaded VPN server causing you to shoot yourself in the foot and cripple your speeds for invalid reasons

I think its fine if users want to customise the script to do this, but I have no intention to make this configurable for the reason above


remember, any time you run an update a "hotfix" will be detected as your script no longer matches the remote, so you will lose the ability to ever get hotfixes from me. the alternative is to fork spdMerlin and make your changes, then change the necessary settings at the top of the script

 

[Lynx]

Good point. And actually if the VPN speeds up again I ought to work with WAN limit anyway, since then I can make use of the extra bandwidth.

In that case, with having set in OpenVPN client settings:

Force Internet traffic through tunnel: Yes

(I like the above because this forces all normal traffic through the VPN, including DNS lookups using non-VPN DNS servers, for both router and client devices, and this allows me to ensure DNS lookups set based on the DNS filter page will go over VPN - I would like to keep ability to use the DNS filter without exposing DNS lookups to my ISP),

the ASUS WAN test still works fine under the default bandwidth monitor in Adaptive QoS -> Internet Speed when setting:

Select interface to test: WAN

but spdMerlin WAN test does not work:

Starting speedtest using auto-selected server for WAN interface

[2021-07-20 15:46:31.393] [error] Configuration - Couldn't connect to server (Network is unreachable)
[2021-07-20 15:46:31.393] [error] Configuration - Cannot retrieve configuration document (0)
[2021-07-20 15:46:31.393] [error] ConfigurationError - Could not retrieve or read configuration (Configuration)
[2021-07-20 15:46:31.393] [error] ConfigurationError - Could not retrieve or read configuration (Configuration)
[error] Configuration - Could not retrieve or read configuration (ConfigurationError)
Error running speedtest for WAN

Any idea what the reason / fix for this is for spdMerlin in terms of getting the WAN speed test to work?

 

[Jack Yaz]

Good point. And actually if the VPN speeds up again I ought to work with WAN limit anyway, since then I can make use of the extra bandwidth.

In that case, with having set in OpenVPN client settings:



(I like the above because this forces all normal traffic through the VPN, including DNS lookups using non-VPN DNS servers, for both router and client devices, and this allows me to ensure DNS lookups set based on the DNS filter page will go over VPN - I would like to keep ability to use the DNS filter without exposing DNS lookups to my ISP),

the ASUS WAN test still works fine under the default bandwidth monitor in Adaptive QoS -> Internet Speed when setting:



but spdMerlin WAN test does not work:

Starting speedtest using auto-selected server for WAN interface

[2021-07-20 15:46:31.393] [error] Configuration - Couldn't connect to server (Network is unreachable)
[2021-07-20 15:46:31.393] [error] Configuration - Cannot retrieve configuration document (0)
[2021-07-20 15:46:31.393] [error] ConfigurationError - Could not retrieve or read configuration (Configuration)
[2021-07-20 15:46:31.393] [error] ConfigurationError - Could not retrieve or read configuration (Configuration)
[error] Configuration - Could not retrieve or read configuration (ConfigurationError)
Error running speedtest for WAN

Any idea what the reason / fix for this is for spdMerlin in terms of getting the WAN speed test to work?

i suspect the built-in speedtest is still using the VPN tunnel. i bind to WAN interface specifically, whereas builtin test doesn't and opts for no binding if using WAN. can you check the traffic counters in ifconfig on the tun interface vs. WAN to check if this is the case?

 

[Lynx]

Yes - it seems tun11 shows traffic with the ASUS speed test. So I think it is just going over the VPN as you supposed.

So is the fix for me to stop using 'Force Internet traffic through tunnel: yes'?

I may be wrong but I understand that when using PBR under OpenVPN Client settings + DNS Filter, DNS queries arising due to the DNS Filter get sent over WAN and not the VPN, exposing DNS requests to the ISP, which seems daft to me. Any way to ensure DNS requests will go over the VPN to help safeguard against such leaks? I think what happens is that the router gives out its own address as the DNS address to LAN clients, so when LAN client sends DNS query to router, then router performs DNS over WAN, and returns it to the LAN client. But I'd like to keep such DNS lookups over the VPN.

'Force Internet traffic through tunnel: yes' worked for that, but now I will need to try and find an alternative solution.

 

[Jack Yaz]

Yes - it seems tun11 shows traffic with the ASUS speed test. So I think it is just going over the VPN as you supposed.

So is the fix for me to stop using 'Force Internet traffic through tunnel: yes'?

I may be wrong but I understand that when using PBR under OpenVPN Client settings + DNS Filter, DNS queries arising due to the DNS Filter get sent over WAN and not the VPN, exposing DNS requests to the ISP, which seems daft to me. Any way to ensure DNS requests will go over the VPN to help safeguard against such leaks? I think what happens is that the router gives out its own address as the DNS address to LAN clients, so when LAN client sends DNS query to router, then router performs DNS over WAN, and returns it to the LAN client. But I'd like to keep such DNS lookups over the VPN.

'Force Internet traffic through tunnel: yes' worked for that, but now I will need to try and find an alternative solution.

i'm not sure of the fix, to be honest. I'll set a tunnel up with redirect yes and poke around the firewall and routing to see if spdmerlin can exclude itself temporarily without disrupting other traffic

 

[Lynx]

That would be super cool because 'Force Internet traffic through tunnel: Yes' is desirable to prevent DNS leaks. I prefer the idea of having all traffic default through VPN. And PBR seems to fail when you try to direct traffic originating through router to VPN.

Also see my question here: https://www.snbforums.com/threads/dnsfilter-vs-vpn-vpn-director.73765/

 

[Lynx]

I found an alternative way to do what I want is to just explicitly specify rules on PBR with the DNS server IP addresses set to destinations and interface as VPN - this seems to ensure that DNS lookups on the router get sent over the VPN. To get Divert to work properly I just set my desired DNS as my router DNS under WAN settings, then Global Filter Mode -> Router, then set the desired DNS as destinations in PBR to ensure that the lookups go over VPN.

This keeps spdMerlin happy.

By the way, I shape in the following way using CAKE (shaping on eth0 does not work when using VPN since CAKE cannot see what it is doing):

admin@RT-AX86U-4168:/tmp/home/root# tc qdisc ls

qdisc pfifo_fast 0: dev eth0 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: dev eth1 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: dev eth2 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: dev eth3 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: dev eth4 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: dev eth5 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: dev spu_us_dummy root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: dev spu_ds_dummy root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: dev eth6 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: dev eth7 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc cake 8074: dev br0 root refcnt 2 bandwidth 56095Kbit besteffort dual-dsthost nonat wash ingress no-ack-filter split-gso rtt 100ms noatm overhead 53
qdisc cake 8073: dev tun11 root refcnt 2 bandwidth 27875Kbit diffserv3 dual-srchost nat nowash no-ack-filter split-gso rtt 100ms noatm overhead 53

I set this up using a custom /jffs/scripts/qos-start and /jffs/configs/cake-qos.conf.add.

admin@RT-AX86U-4168:/tmp/home/root# cat /jffs/configs/cake-qos.conf.add

ULIF='tun11'
DLIF='br0'
DLOPTIONS='wash dual-dsthost ingress'

admin@RT-AX86U-4168:/tmp/home/root# cat /jffs/scripts/qos-start

#!/bin/sh


(
cat <<'ADDTEXT'
#!/bin/sh
source /etc/cake-qos.conf


case "$1" in
start)
        if [[ -d /sys/class/net/"$ULIF" ]]; then
                tc qdisc add dev $ULIF root cake $ULPRIOQUEUE $ULOPTIONS $ULBW $OVERHEAD $FRAMING 2>/dev/null
                tc qdisc add dev $DLIF root cake $DLPRIOQUEUE $DLOPTIONS $DLBW $OVERHEAD $FRAMING 2>/dev/null
        fi
        ;;
stop)
        tc qdisc del dev $ULIF root 2>/dev/null
        tc qdisc del dev $DLIF root 2>/dev/null
        ;;
*)
esac
ADDTEXT
) > /tmp/qos

Happily the spdMerlin QoS shutdown and restart process still seem to work OK (I suppose the calls to /tmp/qos get passed to, and handled by: /jffs/scripts/qos-start?). Anyway, it's working.

Using the spdMerlin auto bandwidth calculation settings in their default, my buffer bloat seems still under control, more or less, but it is erring slightly on the side of more bandwidth. How does one decide a good tradeoff between bandwidth sacrifice and latency? Overall I think the defaults that you have set seem to work well on my LTE 4G connection. How did you go about choosing them?

Also any thoughts on frequency of bandwidth testing? I am not sure what the optimal granularity would be.