spdMerlin The future of spdMerlin

[andresmorago]

This is a known issue a few have mentioned to jack. It seems if you uninstall a script that uses jq it will effectively render jq unavailable for spdmerlin. One script that uses jq is X3Mrouting, if this is removed jq gets removed. There may be other scripts that use jq as well but I haven't investigated. Ultimately it is up to script devs to maintain that dependencies are installed when required

You’re right about x3mrouting. I uninstalled it right before this error started to occur

 

[cplay]

spdMerlin will stop cake for the duration of the test

Is this the same for flexqos?

Cheers

 

[cplay]

I did a test today and realized that with Exclude speedtests from QoS set to No the script limits the results to the Upload Bandwidth limit set on the QoS page but not the Download Bandwidth limit. I use FlexQoS, I don't know how Cake QoS behaves in that scenario. I'm on V4.0.0 and have applied the latest hotfixes.

I have experienced exactly the same issue today on the latest version of spdmerlin @Jack Yaz

If I follow the exact same steps as NGI i get the same issue - issue flexqos in addition to spdmerlin on 384.18.

On another note; how exactly does spdmerlin circumvent qos download and upload limits when when exclude speedtests from QOS is set to "YES"?

Hope you are well.

 

[Jack Yaz]

Is this the same for flexqos?

Cheers

I have experienced exactly the same issue today on the latest version of spdmerlin @Jack Yaz

If I follow the exact same steps as NGI i get the same issue - issue flexqos in addition to spdmerlin on 384.18.

On another note; how exactly does spdmerlin circumvent qos download and upload limits when when exclude speedtests from QOS is set to "YES"?

Hope you are well.

spdMerlin adds some iptables rules to mark the traffic such that it should be excluded from the QoS engine. The marks may not work correctly anymore, they were given to me by FreshJR a long time ago. @dave14305 as the resident QoS guru will be able to check and advise

 

[dave14305]

spdMerlin adds some iptables rules to mark the traffic such that it should be excluded from the QoS engine. The marks may not work correctly anymore, they were given to me by FreshJR a long time ago. @dave14305 as the resident QoS guru will be able to check and advise

I don't really understand what those 3 rules are trying to do, other than mark everything with the download bit 8. This is odd because they do not discriminate based on interface. I suppose if they egress the eth0 (upload) interface with a download bit set, they won't be picked up by the tc filters expecting a 4 (upload bit) and may bypass the qdisc.

I don't yet see any purpose to the first and third rules. POSTROUTING happens after OUTPUT, so I don't think it's necessary to re-mark the packets. And I don't see it useful to mark the packet in the default filter table.

iptables -A OUTPUT -p "$proto" -j MARK --set-xmark 0x80000000/0xC0000000 2>/dev/null
iptables -t mangle -A OUTPUT -p "$proto" -j MARK --set-xmark 0x80000000/0xC0000000 2>/dev/null
iptables -t mangle -A POSTROUTING -p "$proto" -j MARK --set-xmark 0x80000000/0xC0000000 2>/dev/null

Did FreshJR leave a note?

 

[Jack Yaz]

I don't really understand what those 3 rules are trying to do, other than mark everything with the download bit 8. This is odd because they do not discriminate based on interface. I suppose if they egress the eth0 (upload) interface with a download bit set, they won't be picked up by the tc filters expecting a 4 (upload bit) and may bypass the qdisc.

I don't yet see any purpose to the first and third rules. POSTROUTING happens after OUTPUT, so I don't think it's necessary to re-mark the packets. And I don't see it useful to mark the packet in the default filter table.

iptables -A OUTPUT -p "$proto" -j MARK --set-xmark 0x80000000/0xC0000000 2>/dev/null
iptables -t mangle -A OUTPUT -p "$proto" -j MARK --set-xmark 0x80000000/0xC0000000 2>/dev/null
iptables -t mangle -A POSTROUTING -p "$proto" -j MARK --set-xmark 0x80000000/0xC0000000 2>/dev/null

Did FreshJR leave a note?

At the time I found that something was happening between OUTPUT and POSTROUTING so put the marks in both, but that was a while ago and may have been an oddity in my testing setup. I'll see if I can find it but I get a LOT of PMs!

It may be better to start from scratch to be honest. Given that, what rules would be necessary to exclude traffic? If it would work better, is there a tc modification to apply instead?

 

[dave14305]

At the time I found that something was happening between OUTPUT and POSTROUTING so put the marks in both, but that was a while ago and may have been an oddity in my testing setup. I'll see if I can find it but I get a LOT of PMs!

It may be better to start from scratch to be honest. Given that, what rules would be necessary to exclude traffic? If it would work better, is there a tc modification to apply instead?

I suppose it works to just use the middle rule, but add the -o eth0 just to avoid any oddities on the LAN side, which shouldn't need to be affected by the test. Since the download limiting only happens on the br0 interface, that's out-of-scope for a router-based speedtest.

This was my test:

iptables -t mangle -A OUTPUT -o eth0 -j MARK --set-xmark 0x80000000/0xC0000000
tc -s qdisc show dev eth0
iperf3 -4 -c iperf.scottlinux.com
tc -s qdisc show dev eth0
iptables -t mangle -nvL OUTPUT
iptables -t mangle -D OUTPUT -o eth0 -j MARK --set-xmark 0x80000000/0xC0000000

You will see the bypass in the direct_packets_stat:

qdisc htb 1: root refcnt 2 r2q 10 default 0 direct_packets_stat 61792 direct_qlen 1000

 

[Jack Yaz]

I suppose it works to just use the middle rule, but add the -o eth0 just to avoid any oddities on the LAN side, which shouldn't need to be affected by the test. Since the download limiting only happens on the br0 interface, that's out-of-scope for a router-based speedtest.

This was my test:

iptables -t mangle -A OUTPUT -o eth0 -j MARK --set-xmark 0x80000000/0xC0000000
tc -s qdisc show dev eth0
iperf3 -4 -c iperf.scottlinux.com
tc -s qdisc show dev eth0
iptables -t mangle -nvL OUTPUT
iptables -t mangle -D OUTPUT -o eth0 -j MARK --set-xmark 0x80000000/0xC0000000

You will see the bypass in the direct_packets_stat:

qdisc htb 1: root refcnt 2 r2q 10 default 0 direct_packets_stat 61792 direct_qlen 1000

i'll drop the other 2 rules then in my next dev cycle. thanks!

 

[dave14305]

i'll drop the other 2 rules then in my next dev cycle. thanks!

Be sure to verify I'm not lying.

 

[Jack Yaz]

Be sure to verify I'm not lying.

but I trust you not to

 

[cplay]

I don't really understand what those 3 rules are trying to do, other than mark everything with the download bit 8. This is odd because they do not discriminate based on interface. I suppose if they egress the eth0 (upload) interface with a download bit set, they won't be picked up by the tc filters expecting a 4 (upload bit) and may bypass the qdisc.

I don't yet see any purpose to the first and third rules. POSTROUTING happens after OUTPUT, so I don't think it's necessary to re-mark the packets. And I don't see it useful to mark the packet in the default filter table.

iptables -A OUTPUT -p "$proto" -j MARK --set-xmark 0x80000000/0xC0000000 2>/dev/null
iptables -t mangle -A OUTPUT -p "$proto" -j MARK --set-xmark 0x80000000/0xC0000000 2>/dev/null
iptables -t mangle -A POSTROUTING -p "$proto" -j MARK --set-xmark 0x80000000/0xC0000000 2>/dev/null

Did FreshJR leave a note?

The marks work correctly for BOTH upload AND download when you leave it on the default: "Exclude speedtests from QoS set to YES" - SPD merlin circumvents QOS rules.

The issue only occurs when you set "Exclude speedtests from QoS set to No" - The marks work correctly on the upload but not the download and download limit set in adaptive qos is ignored.

This is the same issue I have noticed when I save flex qos settings @dave14305 - download limiter for adaptive qos no longer works, only the upload (we were discussing this on another thread).

These two issues are surely related?

 

[L&LD]

Trustworthy liars are hard to find. Let's keep this one!

 

[dave14305]

but I trust you not to

It's just that I don't use any VPN clients or Guest networks or anything very fancy to ensure it covers advanced use-cases.

 

[L&LD]

@dave14305, that's what the beta testers are for!

 

[dave14305]

The issue only occurs when you set "Exclude speedtests from QoS set to No" - The marks work correctly on the upload but not the download and download limit set in adaptive qos is ignored.

For spdMerlin (or the new ASUS speedtest), it is true that Adaptive QoS Download limits will never restrict the download speed because the limiting occurs only on the br0 LAN interface. When the destination is the router itself (where the speedtest runs), the br0 interface isn't used. Only the upload direction limit applies.

When using CAKE, it's a different story because CAKE-QoS adds an ingress filter on the WAN interface to limit inbound traffic from the WAN.

 

[Jack Yaz]

It's just that I don't use any VPN clients or Guest networks or anything very fancy to ensure it covers advanced use-cases.

To be honest I'm just thrilled to finally see one of my scripts in your signature

 

[cplay]

For spdMerlin (or the new ASUS speedtest), it is true that Adaptive QoS Download limits will never restrict the download speed because the limiting occurs only on the br0 LAN interface. When the destination is the router itself (where the speedtest runs), the br0 interface isn't used. Only the upload direction limit applies.

When using CAKE, it's a different story because CAKE-QoS adds an ingress filter on the WAN interface to limit inbound traffic from the WAN.

got it, the only thing I don’t understand is why saving flexqos settings results in the exact same behaviour - download limit ignored yet the upload limit respected

so it is just a coincidence?

 

[dave14305]

got it, the only thing I don’t understand is why saving flexqos settings results in the exact same behaviour - download limit ignored yet the upload limit respected

so it is just a coincidence?

Applying settings in FlexQoS will restart QoS and wipe out the existing mangle iptables rules along with tc qdiscs, classes and filters. You'll have to explain the steps you take to reproduce the problem.

 

[cplay]

OK so it sounds like there is life yet

Thank you everyone for the encouragement

checking vpn tunnel speeds is insanely useful for benchmarking wan vs tunnel speed.

It also stops mehaving to disable vpnsto check wan speed.

The ability to auto adjust qos based on results deviating from certain thresholds is also huge.

Stop selling yourself short, your addon is superior in all aspects.

 

[Jack Yaz]

checking vpn tunnel speeds is insanely useful for benchmarking wan vs tunnel speed.

It also stops mehaving to disable vpnsto check wan speed.

The ability to auto adjust qos based on results deviating from certain thresholds is also huge.

Stop selling yourself short, your addon is superior in all aspects.

aw shucks. thanks!!