QoS-Cake Throttling USB?

[dave14305]

Egress/Ingress is from the router interfaces’ perspectives.

• Upload traffic from LAN to WAN is ingress on br0 and egress on eth0 (in a normal setup).
• Download traffic from WAN to LAN is ingress on eth0 and egress on br0.
• Upload traffic from LAN to router (i.e. Samba share on router) is ingress on br0 and does not egress anywhere.
• Download traffic from router to LAN (i.e. Samba share on router) is egress on br0.
• Upload/download traffic from LAN to LAN should be out of scope of Cake because it is not routed, but switched. But that might be not 100% true if going through the br0 bridge as in Wired to Wireless. I don’t know the details.

 

[Lynx]

@dave14305 I finally had some time to implement a solution. I went for the standard practice of setting up an IFB for the interface in question for ingress. So I set up an IFB for 'tun11' and apply CAKE on that. It seems to work very well. It does not suffer from the samba share issue and also now the download bandwidth and upload bandwidth match (before the download got throttled a little more).

I presume the CAKE 'nat' option determines the local destination IP from the 'tun11' IFB?

I tested by setting two simultaneous downloads and sure enough the split is held beautifully at 50/50 of the set bandwidth (in my case 30Mbit/s) between the two devices.

Do you see any issues with my script below? It looks to see if 'tun11' is up and if so then CAKE is applied to that and otherwise CAKE is applied to 'eth0' as standard. Either way an IFB is set up for ingress. The VPN overhead is taken into account.

#!/bin/sh

(
cat <<'ADDTEXT'
#!/bin/sh
source /etc/cake-qos.conf

TUN_PACKET_OVERHEAD=53

case "$1" in
start)
        if [[ -d /sys/class/net/tun11 ]]; then
                ULIF='tun11'
                OVERHEAD=$(echo $OVERHEAD | awk '$2=$2+'$TUN_PACKET_OVERHEAD)
        fi
                ip link add name ifb type ifb 2>/dev/null
                tc qdisc add dev $ULIF root cake $ULPRIOQUEUE $ULOPTIONS $ULBW $OVERHEAD $FRAMING 2>/dev/null
                tc qdisc add dev $ULIF handle ffff: ingress 2>/dev/null
                tc qdisc add dev ifb root cake $DLPRIOQUEUE $DLOPTIONS $DLBW $OVERHEAD $FRAMING 2>/dev/null
                ip link set ifb up 2>/dev/null
                tc filter add dev $ULIF parent ffff: matchall action mirred egress redirect dev ifb 2>/dev/null
        ;;
stop)
        tc qdisc del dev eth0 ingress 2>/dev/null
        tc qdisc del dev eth0 root 2>/dev/null
        tc qdisc del dev tun11 ingress 2>/dev/null
        tc qdisc del dev tun11 root 2>/dev/null
        tc qdisc del dev ifb root 2>/dev/null
        ip link set ifb down 2>/dev/null
        ip link del ifb 2>/dev/null
        ;;
*)
esac
ADDTEXT
) > /tmp/qos

admin@RT-AX86U-4168:/jffs/scripts# tc -s -d qdisc show
qdisc pfifo_fast 0: dev eth0 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
 Sent 5572208590 bytes 4480828 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc pfifo_fast 0: dev eth1 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
 Sent 37564540 bytes 201299 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc pfifo_fast 0: dev eth2 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
 Sent 37150619 bytes 194436 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc pfifo_fast 0: dev eth3 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
 Sent 38627883 bytes 207826 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc pfifo_fast 0: dev eth4 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
 Sent 5793426645 bytes 5157474 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc pfifo_fast 0: dev eth5 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc pfifo_fast 0: dev spu_us_dummy root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc pfifo_fast 0: dev spu_ds_dummy root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc pfifo_fast 0: dev eth6 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
 Sent 3561567911 bytes 7169931 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc pfifo_fast 0: dev eth7 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
 Sent 9598786919 bytes 8253987 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc cake 8015: dev tun11 root refcnt 2 bandwidth 30720Kbit diffserv3 dual-srchost nat nowash no-ack-filter split-gso rtt 100ms noatm overhead 91
 Sent 2062662749 bytes 1750275 pkt (dropped 2459, overlimits 3357335 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 445266b of 4Mb
 capacity estimate: 30720Kbit
 min/max network layer size:           29 /    1378
 min/max overhead-adjusted size:      120 /    1469
 average network hdr offset:            0

                   Bulk  Best Effort        Voice
  thresh       1920Kbit    30720Kbit     7680Kbit
  target         9.37ms          5ms          5ms
  interval        104ms        100ms        100ms
  pk_delay          0us       4.66ms        297us
  av_delay          0us       2.38ms         28us
  sp_delay          0us          7us          7us
  backlog            0b           0b           0b
  pkts                0      1752685           49
  bytes               0   2066000830         6833
  way_inds            0          301            0
  way_miss            0          555            2
  way_cols            0            0            0
  drops               0         2459            0
  marks               0            1            0
  ack_drop            0            0            0
  sp_flows            0            5            1
  bk_flows            0            1            0
  un_flows            0            0            0
  max_len             0         1378          203
  quantum           300          937          300

qdisc ingress ffff: dev tun11 parent ffff:fff1 ----------------
 Sent 448443329 bytes 1796242 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc cake 8016: dev ifb root refcnt 2 bandwidth 30720Kbit besteffort dual-dsthost nat wash ingress no-ack-filter split-gso rtt 100ms noatm overhead 91
 Sent 428388376 bytes 1781492 pkt (dropped 14713, overlimits 1576497 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 783872b of 4Mb
 capacity estimate: 30720Kbit
 min/max network layer size:           40 /    1378
 min/max overhead-adjusted size:      131 /    1469
 average network hdr offset:            0

                  Tin 0
  thresh      30720Kbit
  target            5ms
  interval        100ms
  pk_delay        447us
  av_delay        102us
  sp_delay          3us
  backlog            0b
  pkts          1796205
  bytes       448441849
  way_inds           49
  way_miss          574
  way_cols            0
  drops           14713
  marks               2
  ack_drop            0
  sp_flows            5
  bk_flows            1
  un_flows            0
  max_len          1378
  quantum           937

 

[dave14305]

I presume the CAKE 'nat' option determines the local destination IP from the 'tun11' IFB?

I tested by setting two simultaneous downloads and sure enough the split is held beautifully at 50/50 of the set bandwidth (in my case 30Mbit/s) between the two devices.

Not sure how nat behaves in your setup. If it works for you, that’s all that really matters. I don’t think it’s really working as you hope it is, but that’s a gut feeling and not factual.