(RT-AC88U) Traditional QoS fully fixed (I believe)

[cdufour]

EDITED: applies to 384.4.beta1

Looking closely at the output of the various tc -s ... commands when downloading/uploading files, I have discovered that the ingress (ffff:) qdisc on the eth0 interface behaves very strangely:

1. downloaded/ingress traffic DOES appear on the ingress (ffff:) qdisc (tc -s qdisc ls dev eth0)
2. BUT does NOT seem to be taken in account by the matching filter (tc -s filter ls dev eth0 parent ffff:); [EDITED] see comment #13 for reason why
3. and INSTEAD appears in the 1:40 ("low" priority) qdisc/class, which one would expect to handle only uploaded/egress "low" priority traffic (tc -s class ls dev eth0)

This mixing of ingress/egress traffic is very surprising. I use Linux traffic control on servers at work and do not witness similar behavior (unintuitive and contradicting all 'tc' documentation I have read).
On the other hand, it is also quite surprising to see eth0 - the WAN interface - also be the underlying interface for the vlan1@eth0, part or the br0 - LAN - bridge. This strange setup has to do with the underlying Broadcom Roboswitch; [EDITED] see comment #3 below for the detailed explanation.

I can not explain why ingress traffic is not filtered as per ad-hoc rules. No easy way to tell whether this strange behavior is a kernel/driver bug or results from the eth0/vlan1 mix.
[EDITED] On the other hand, that mix - along a bug in the 802.1Q qdisc filter rule - explains why downloaded/ingress traffic is eventually (wrongly) throttled as per upload settings, in the 1:40 qdisc/class:

• the 802.1Q filter rule - tc filter ... parent 1: prio 6 protocol 802.1q handle 6 fw flowid 1:60 - is intended to redirect LAN traffic to the ad-hoc 1:60 (1Gbits/s) qdisc/class
• in order for this rule to work, traffic must BOTH be VLAN(802.1Q)-tagged and (CONN)MARK-ed with value fw 6
• BUT looking at the QOSO (mangle) iptables chain, one can see that traffic will NOT be (CONN)MARK-ed with value fw 6 if it matches a previous classification; e.g. if one classifies HTTP (TCP 80) traffic to "medium" priority (fw 3), downloaded/ingress packets will NEVER be classified as fw 6, even when they egress as LAN traffic on vlan1@eth0
• in consequence, such 802.1Q packets match no other filter rules (they are not caught by protocol ip rules) and end up in the root HTB qdisc's default 1:40 sub-qdisc/class

Fortunately, one can fix that faulty rule by deleting and replacing it with a working one! And while we're at it, one might as well fix other QoS-related weirdness in the mangle iptable. E.g. in /jffs/scripts/firewall-start:

# QOS
# ... fix erroneous LAN qdisc filter (-> match all VLAN ID 1 - LAN - traffic)
tc filter del dev eth0 parent 1: prio 1 2>/dev/null
tc filter add dev eth0 parent 1: prio 1 protocol 802.1q u32 match u16 0x0001 0x0FFF at -4 flowid 1:60
#OR (for ALL VLAN-tagged traffic): ... protocol 802.1q u32 match u32 0 0 flowid 1:60
tc filter del dev eth0 parent 1: prio 6 2>/dev/null
tc filter add dev eth0 parent 1: prio 6 protocol ip handle 6 fw flowid 1:60
# ... fix erroneous CONNMARK mask
rnum="$(iptables --line-numbers -t mangle -nvL QOSO | grep 'connmark match ! 0x0/0xff00' | cut -d' ' -f1)"
[ -n "${rnum}" ] && iptables -t mangle -R QOSO ${rnum} -m connmark ! --mark 0x0/0x7 -j RETURN
# ... fix hardcoded /24 netmask (e.g. for a 172.16.0.0/12 segment)
rnum="$(iptables --line-numbers -t mangle -nvL QOSO | grep 'CONNMARK .* 172.16.0.0/24' | cut -d' ' -f1)"
[ -n "${rnum}" ] && iptables -t mangle -R QOSO ${rnum} -d 172.16.0.0/12 -j CONNMARK --set-xmark 0x6/0x7
rnum="$(iptables --line-numbers -t mangle -nvL QOSO | grep 'RETURN .* 172.16.0.0/24' | cut -d' ' -f1)"
[ -n "${rnum}" ] && iptables -t mangle -R QOSO ${rnum} -d 172.16.0.0/12 -j RETURN​

WARNING: This works on the RT-AC88U's kernel 2.6.36.4brcmarm (firmware 384.x); it may NOT work on 3.x or more recent kernels (tc syntax for VLAN filtering MUST then use basic match (meta vlan ID) syntax, which requires updated - ematch-able - tc/iproute2 tool)

This fixes:

• downloaded/ingress traffic no longer appears in the 1:40 ("low" priority) qdisc/class (throttled as per upload/egress bandwidth)
• INSTEAD, it appears in the 1:60 - LAN - qdisc/class (commissioned with a 1Gbit/s bandwidth)
• QOSO (mangle) chain properly RETURN when a connexion already has a (CONN)MARK
• other-than /24 LAN segments are properly (non-)QoS-ed (directed to the 1:60 - LAN - qdisc/class)

This does NOT fix:

• downloaded/ingress traffic throttling as per user-configured limits, since the ingress (ffff:) filter policies are apparently NOT used

'hope this might help some of you.

 

[RMerlin]

Part of the Traditional QoS issues have to do with Broadcom I suspect. It used to work properly, then suddenly things got broken a few years ago. I know some people insist that "download and upload values are just swapped on the webui", but the problems run much deeper than that as you've seen, so I've kept repeating.

I don't have the traffic classifier expertise to analyze the problem and be able to determine if it's fixable or not. At least the good news is, Traditional QoS rule generation is in the router/rc/qos.c source code, so someone more skilled than me at it might attempt a fix. But I suspect that ultimately, architectural decisions from Broadcom might be causing issues that are hard to fix. I've always been puzzled for example at the output from /proc/net/dev - packet count seem really off, possibly tied to that Broadcom architecture. Those odd numbers started to appear at around the same time Traditional QoS broke, which led me to suspect they were tied.

 

[cdufour]

Looking at the Broadcom Roboswitch setup:

admin@rt-ac88u:/tmp/home/root# robocfg show
VLANs: BCM5301x enabled mac_check mac_hash
1: vlan1: 0 1 2 3 5 7t
2: vlan2: 4 7​

Might explain the strange eth0/vlan1 setup:

• unless I'm utterly wrong, the "CPU" port number 7 is the one referred to as eth0 by the kernel
• VLAN ID 1 - LAN - is tagged on that port, thus matched by the vlan1@eth0 logical interface
• VLAN ID 2 - WAN - is untagged on that port, thus matched by the eth0 (no vlan) logical interface

The eth0 physical interface thus "sees" both the WAN and (wired) LAN traffic, with ingress WAN matching (summed wired) egress LAN and vice versa. This explains the weird ingress behavior witnessed at the tc level! One just can not use the ingress qdisc approach (on eth0) given the topology at hand.

I'll continue digging and see whether I can come with some patches (for qos.c, etc.) that definitely fixes that issue.

 

[cdufour]

Had a look at router/rc/qos.c:

• the erroneous qdisc filter is fixable but tricky: in order to stay safe, one ought to filter specifically for the LAN VLAN ID; is ID=1 to be considered the "hard-coded" ID for the LAN or should this be extracted from NVRAM ?
• the "erroneous" CONNMARK mask is actually related to incompletely implemented "min-max transferred (kB)" QoS rule: according to comments in the source code, rules with no such min-max limits ought to be marked 0x101-0x106 instead of 0x1-0x6; [EDITED] but comments also state Asus ought to use only first 8 bits (0xFF), because higher bits are for (TrendMicro) Adaptive QoS; fixing this issue would require significant changes to the code (BUT: if Traditional QoS is used, Adaptive QoS isn't, ain't it ?)
• the hard-coded /24 netmask ought to be fixable, thanks to the "lan_netmask" NVRAM value

Only difficulty: I'm not looking forward to install the entire build chain to verify changes pass compilation (or even "worse": installation).
Would someone with existing build chain/experience be willing to test the patch I would propose ?

 

[cdufour]

Just a note on ingress filtering: Linux/TC support for ingress infiltering is quite originally sparse. But it'd be worth looking into the RT-AC88U's ifb0/ifb1 interfaces (unused as far as I can tell, at least by Traditional QoS):
tc filter add dev eth0 parent ffff: protocol ip u32 match ... action mirred egress redirect dev ifb0

 

[FreshJR]

Just a note on ingress filtering: Linux/TC support for ingress infiltering is quite originally sparse. But it'd be worth looking into the RT-AC88U's ifb0/ifb1 interfaces (unused as far as I can tell, at least by Traditional QoS):
tc filter add dev eth0 parent ffff: protocol ip u32 match ... action mirred egress redirect dev ifb0

Can you correct my terminology in case I have it wrong.

Egress/Ingress does not mean Upload/Download.

Ingress = towards router
Egress = away from router

During download
-Ingress: Server to Router
-Egress: Router to LAN client

During upload
-Ingress: LAN client to Router
-Egress: Router to Server

TC only works on egress traffic. I was not aware it supports ingress traffic even sparsely.
I was confused by your interchanging of terms.

--

As for your second point

We can have a difference of opinion, but a majority of my buffer bloat was caused to download traffic.
Having TC drop a few of the initial packets so the server transmits below my speed works amazingly.

DSL reports has an amazing speedtest.

If your QOS download rules drop too many packets to lower rate, you get a poor QUALITY grade.
If your QOS download rules do not drop enough packets so your network is conjested, you get a poor BUFFERBLOAT grade.

Anyway this comment I madeis off topic.

--

Thanks for fixing bugs with the implementation.

--

I always imaged in working like this while using TC with the adpativeQOS configuration.
(Orange cones are TC filtering/queing)

 

[cdufour]

We can have a difference of opinion, but a majority of my buffer bloat was caused to download traffic.
Having TC drop a few of the initial packets so the server transmits below my speed works amazingly.

Agree. After further thinking, I came to see (and edited my original post) that ingress shaping isn't much different from what our ISP does to have the bandwidth match what we pay for. So it can not be that bad.

Can you correct my terminology in case I have it wrong.

Egress/Ingress does not mean Upload/Download.

Agree the Upload/Download terms may be misleading.

Ingress = towards router
Egress = away from router

Yes, with one clarification. With the Broadcom Roboswitch coming into play, it's essential to understand the Roboswitch is not "part" of the router but "outside" of it. It is "connected" to the router via its port #7 (aka. "CPU" port), known to us as eth0. And the router does all the job of routing/filtering/shaping from/to the WAN/LAN (and vice-versa) through that SAME interface, since the real (physical) WAN (VLAN 2) is on the switch port #4 and other wired LAN clients (VLAN 1) on ports #0-3,5 (5 being the additional Realtek switch).
That's what makes the TC configuration on those router a real casse-tête

During download
-Ingress: Server to Router
-Egress: Router to LAN client

During upload
-Ingress: LAN client to Router
-Egress: Router to Server

That's absolutely right.

For the purists, one might add that a "download" is made (principally) of packets ingressing from the WAN but also (marginally) packets egressing to the WAN (initial request, TCP ACKs, etc.). And vice-versa for an "upload".

TC only works on egress traffic.

Well, according to documentation I read, TC is supposed to work for ingress as well.
That's were the ingress (ffff: ) qdisc comes into play, along its "simplistics" police filters: tc filter ls dev eth0 parent ffff: (TC does not allow to do anything more fancy on the ingress qdisc). BUT that's what seems not to work on the RT-AC88U. I haven't given up yet on trying to find out why.
AND, according to the Internet, ifb interfaces can come very handy for shaping ingress traffic, by redirecting the ingress traffic to an "internal" ifb interface where it can be shaped as egress traffic. The RT-AC88U has two of those ifb interfaces, unused by Traditional QoS. I wonder whether they are here to be used by TrendMicro Adaptive QoS. I'd be curious to see the full tc and iptables output of a router when Adaptive QoS is active.

 

[FreshJR]

For the purists, one might add that a "download" is made (principally) of packets ingressing from the WAN but also (marginally) packets egressing to the WAN (initial request, TCP ACKs, etc.). And vice-versa for an "upload".

While yes that upload is essential to have the download take place, to simplify things, I just look at is as an absolute.
Data leaving your computer = upload
Data entering your computer = download

When dealing with the router, we can catch download/upload at either ingress/egress depending where we have access too.

For QOS it shouldn't make a different where the two bottlenecks are implemented as long as we have two streams that segregae upload/download in the absolute sense as I mentioned.

That's what makes the TC configuration on those router a real casse-tête

It seems ASUS figured out how to implement a segregation system like the image I posted.

I do not have the knowledge to see how the interfaces are mirrored/routed/link/act with each other, but couldn't we simply view and copy their working setup? Whatever they did can view both traffic streams correctly.

 

[cdufour]

Just for the sake of clarity, I paste below the original Traditional QoS configuration as performed by AsusWRT magic (but reflecting part of my personal rules).

/tmp/qos:

#!/bin/sh
#LAN/WAN
I=eth0
SFQ="sfq perturb 10"
TQA="tc qdisc add dev $I"
TCA="tc class add dev $I"
TFA="tc filter add dev $I"
case "$1" in
start)
#LAN/WAN
tc qdisc del dev $I root 2>/dev/null
$TQA root handle 1: htb default 40
# upload 1:1
$TCA parent 1: classid 1:1 htb rate 10240kbit ceil 10240kbit overhead 27 linklayer ethernet
# download 1:2
$TCA parent 1: classid 1:2 htb rate 1000000kbit ceil 1000000kbit burst 10000 cburst 10000
# 1:60 ALL Download for BCM
$TCA parent 1:2 classid 1:60 htb rate 1000000kbit ceil 1000000kbit burst 10000 cburst 10000 prio 6
$TQA parent 1:60 handle 60: pfifo
$TFA parent 1: prio 6 protocol 802.1q handle 6 fw flowid 1:60 # <-- RULE NO T ALWAYS MATCHED WHEN IT SHOULD
# egress 0: 50-100%
$TCA parent 1:1 classid 1:10 htb rate 5120kbit ceil 10240kbit prio 1 quantum 1500 overhead 27 linklayer ethernet
$TQA parent 1:10 handle 10: sfq perturb 10
$TFA parent 1: prio 10 protocol ip handle 1 fw flowid 1:10
# egress 1: 30-100%
$TCA parent 1:1 classid 1:20 htb rate 3072kbit ceil 10240kbit prio 2 quantum 1500 overhead 27 linklayer ethernet
$TQA parent 1:20 handle 20: sfq perturb 10
$TFA parent 1: prio 20 protocol ip handle 2 fw flowid 1:20
# egress 2: 15-100%
$TCA parent 1:1 classid 1:30 htb rate 1536kbit ceil 10240kbit prio 3 quantum 1500 overhead 27 linklayer ethernet
$TQA parent 1:30 handle 30: sfq perturb 10
$TFA parent 1: prio 30 protocol ip handle 3 fw flowid 1:30
# egress 3: 5-100%
$TCA parent 1:1 classid 1:40 htb rate 512kbit ceil 10240kbit prio 4 quantum 1500 overhead 27 linklayer ethernet
$TQA parent 1:40 handle 40: sfq perturb 10
$TFA parent 1: prio 40 protocol ip handle 4 fw flowid 1:40
$TFA parent 1: prio 14 protocol ip u32 match ip protocol 6 0xff match u8 0x05 0x0f at 0 match u16 0x0000 0xffc0 at 2 match u8 0x10 0xff at 33 flowid 1:10
$TFA parent 1: prio 15 protocol ip u32 match ip protocol 6 0xff match u8 0x05 0x0f at 0 match u16 0x0000 0xffc0 at 2 match u8 0x02 0x02 at 33 flowid 1:10
$TFA parent 1: prio 13 protocol ip u32 match ip protocol 1 0xff flowid 1:10
tc qdisc del dev $I ingress 2>/dev/null
$TQA handle ffff: ingress
# ingress 0: 100%
$TFA parent ffff: prio 1 protocol ip handle 1 fw police rate 40960kbit burst 20480kbit drop flowid ffff:1
# ingress 1: 100%
$TFA parent ffff: prio 2 protocol ip handle 2 fw police rate 40960kbit burst 20480kbit drop flowid ffff:2
# ingress 2: 100%
$TFA parent ffff: prio 3 protocol ip handle 3 fw police rate 40960kbit burst 20480kbit drop flowid ffff:3
# ingress 3: 100%
$TFA parent ffff: prio 4 protocol ip handle 4 fw police rate 40960kbit burst 20480kbit drop flowid ffff:4
;;
stop)
tc qdisc del dev $I root 2>/dev/null
tc qdisc del dev $I ingress 2>/dev/null
;;
*)
#---------- Upload ----------
tc -s -d class ls dev $I
tc -s -d qdisc ls dev $I
echo
esac

/tmp/mangle_rules:

*mangle
: PREROUTING ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:QOSO - [0:0]
-A QOSO -j CONNMARK --restore-mark --mask 0x7
-A QOSO -m connmark ! --mark 0/0xff00 -j RETURN # <-- RULE NEVER MATCHED
-A QOSO -p tcp -m multiport --dport 53 -j CONNMARK --set-mark 0x1/0x7
-A QOSO -p tcp -m multiport --dport 53 -j RETURN
-A QOSO -p udp -m multiport --dport 53 -j CONNMARK --set-mark 0x1/0x7
-A QOSO -p udp -m multiport --dport 53 -j RETURN
-A QOSO -p tcp -m multiport --dport 443 -j CONNMARK --set-mark 0x2/0x7
-A QOSO -p tcp -m multiport --dport 443 -j RETURN
-A QOSO -p tcp -m multiport --dport 80 -j CONNMARK --set-mark 0x3/0x7
-A QOSO -p tcp -m multiport --dport 80 -j RETURN
-A QOSO -d 224.0.0.0/4 -j CONNMARK --set-mark 0x6/0x7
-A QOSO -d 224.0.0.0/4 -j RETURN
-A QOSO -d 172.24.0.0/24 -j CONNMARK --set-mark 0x6/0x7 # <-- WRONG NETMASK
-A QOSO -d 172.24.0.0/24 -j RETURN # <-- WRONG NETMASK
-A POSTROUTING -o br0 -j QOSO
-A QOSO -j CONNMARK --set-mark 0x4/0x7
-A FORWARD -o eth0 -j QOSO
-A OUTPUT -o eth0 -j QOSO
-A QOSO -j RETURN
-A PREROUTING -i eth0 -j CONNMARK --restore-mark --mask 0x7
COMMIT

I'm working on the qos.c source code that generates those two files.

 

[cdufour]

GOOD NEWS:

tc qdisc del dev eth0 ingress # Reset all ingress stuff
ip link set dev ifb0 up # CRITICAL! (before redirecting traffic to it)
tc qdisc add dev eth0 ingress # Handle ALWAYS ffff: (can not be overridden)
tc filter add dev eth0 parent ffff: prio 1 protocol ip u32 match u32 0 0 action mirred egress redirect dev ifb0 # Redirect all ingress traffic to ifb0 as egress traffic

WORKS!

watch -n 1 grep ifb0 /proc/net/dev

While downloading, and (WAN ingress) traffic does indeed appear on the ifb0 interface!

AND, a very crude shaper built on top of it DOES work:

tc qdisc add dev ifb0 root handle 1: htb default 1
tc class add dev ifb0 parent 1: classid 1:1 htb rate 4096kbit ceil 4096kbit
tc qdisc add dev ifb0 parent 1:1 handle 11: sfq perturb 10

Downloads now peak at 4096kit!

This opens up great new horizons for ingress shaping!

EDITED: I expected upload (LAN ingress) traffic to also appear on ifb0, but it does NOT. The reason is the tc filter add dev eth0 parent ffff: ... protocol ip ... redirect, which does NOT match VLAN/802.1Q (vlan1@eth0) packets! This will make things simpler

 

[Vexira]

So would any of this help adaptive qos or is it strictly for traditional qos?

 

[RMerlin]

BTW, note that the ifb interfaces don't exist on all router platforms.

 

[cdufour]

TC is supposed to work for ingress as well.
That's were the ingress (ffff: ) qdisc comes into play, along its "simplistics" police filters: tc filter ls dev eth0 parent ffff: (TC does not allow to do anything more fancy on the ingress qdisc). BUT that's what seems not to work on the RT-AC88U. I haven't given up yet on trying to find out why.

Traditional QoS implements ingress qdisc filtering using firewall marks - set in the QOSO chain - like for the standard egress qdiscs.
BUT, when being processed in the ingress qdisc, a packet has NOT YET gone through netfilter (iptables) processing and thus lacks the necessary (CONN)MARKs.

BOTTOM LINE:

• Traffic can NOT be classified in the ingress qdisc using (CONN)MARKs (this also stands true for processing via the ifb interface)

UNLESS:

1. iproute2 is updated to version 4.1 (snapshot 150626); RT-AC88U currently uses version 3.x (snapshot 150210)
   PS: asuswrt-rmerlin.ng already provisions version 4.3 support
2. the matching updates are available in the kernel (from version 4.0)
   PS: see GitHub link below for corresponding commit

REF:

• https://wiki.archlinux.org/index.ph...#Example_of_ingress_traffic_shaping_with_SNAT
• https://lwn.net/Articles/649590/
• https://github.com/torvalds/linux/commit/22a5dc0e5e3e8fef804230cd73ed7b0afd4c7bae
  # git tag --contains 22a5dc0e5e3e8fef804230cd73ed7b0afd4c7bae | sort -V | head -n 1
  v4.0
  

 

[FreshJR]

So would any of this help adaptive qos or is it strictly for traditional qos?

Adaptive QOS already works properly.

 

[cdufour]

So would any of this help adaptive qos or is it strictly for traditional qos?

I'm concentrating here on Traditional QoS (TQ). But the underlying mechanisms - tc and iptables - are the same.

As far as I understood, download shaping does work when using Adaptive QoS (AQ). From what I've stumbled on here and there, it seems that AQ uses tc rules on the br0 bridge; TQ doesn't.
I wonder whether this is Asus solution to succeed in shaping WLAN ingress packets. As FleshJR stated, it doesn't really matter exactly on which interface we add tc shaping, as long as the job is done.
But I'm (still) a little reluctant at the idea of implementing WLAN-ingress shaping on the LAN (egress) bridge; it somehow doesn't seem right to me (but I'm a little of a purist sysadmin ); thus my looking thoroughly into the ingress qdisc before giving up.

 

[cdufour]

BTW, note that the ifb interfaces don't exist on all router platforms.

qos.c is riddled with #ifdef CONFIG_BCMWL5; would those intersect with routers that do have the ifb interfaces ?
Or is there another #ifdef I can use ?

 

[RMerlin]

qos.c is riddled with #ifdef CONFIG_BCMWL5; would those intersect with routers that do have the ifb interfaces ?
Or is there another #ifdef I can use ?

I don't remember which router have it and what it does specifically. But in my RT-AC88U's case, there's no traffic at all through it:

admin@Stargate88:/tmp/home/root# cat /proc/net/dev
Inter-|   Receive                                                |  Transmit
 face |bytes    packets errs drop fifo frame compressed multicast|bytes    packets errs drop fifo colls carrier compressed
    lo: 127692455  563385    0    0    0     0          0         0 127692455  563385    0    0    0     0       0          0
  ifb0:       0       0    0    0    0     0          0         0        0       0    0    0    0     0       0          0
  ifb1:       0       0    0    0    0     0          0         0        0       0    0    0    0     0       0          0
   agg:       0       0    0    0    0     0          0         0        0       0    0    0    0     0       0          0
  eth0: 975162147 4749065    0    0    0     0          0         0 1642396991 3404897    0    0    0     0       0          0
 dpsta:       0       0    0    0    0     0          0         0        0       0    0    0    0     0       0          0
  eth1: 1367676   11498    0    0    0     0          0       142 45331326  220272    0   39    0     0       0          0
  eth2: 81131952 1389686    0    0    0     0          0      6662 3700608537 2759767    0   48    0     0       0          0
 vlan1: 3763296453 3434537    0    0    0     0          0      5455 1420815930 2545035    0    0    0     0       0          0
 vlan2:       0       0    0    0    0     0          0         0        0       0    0    0    0     0       0          0
   br0: 245981964 1188424    0    0    0     0          0         0 1446302917 1346878    0    0    0     0       0          0
 tun21:       0       0    0    0    0     0          0         0        0       0    0    0    0     0       0          0

 

[RMerlin]

Tomato used IMQ to allow for ingress traffic shaping. However patching the kernel to add IMQ support requires changes to skbuf kernel structure, which breaks every precompiled networking components, such as the wireless drivers.

 

[cdufour]

Hurrah. Everything Traditional QoS now works: upload (WAN egress), download (WAN ingress) and transferred limits (I believe).
ingress qdisc is NOT used, given it can not make use of (CONN)MARKs. Instead, a standard (egress) qdisc on br0 does the job.
Below the files that ought to be created by by AsusWRT/qos.c magic, currently written by hand for review.
I would be would be nice if several pairs of eyes could verify everything makes sense.
(to be compared with original counterparts in comment #9)

/tmp/qos.tqfix:

#!/bin/sh
#LAN/WAN
WI=eth0 <-- set as per router model in qos.c
LI=br0 <-- set as per router model in qos.c
EQ="sfq perturb 10" <-- USER-specified queue
TQA="tc qdisc add dev"
TCA="tc class add dev"
TFA="tc filter add dev"
case "$1" in
start)
# WAN egress ($WI:egress)
tc qdisc del dev $WI root 2>/dev/null
$TQA $WI root handle 1: htb default 40
# 1:1
$TCA $WI parent 1: classid 1:1 htb rate 10240kbit ceil 10240kbit overhead 27 linklayer ethernet
# LAN 1:2 (*wired* LAN traffic also goes through $WI:egress, but 802.1Q/VLAN tagged)
$TCA $WI parent 1: classid 1:2 htb rate 1000000kbit ceil 1000000kbit burst 10000 cburst 10000
# LAN 1:60
$TCA $WI parent 1:2 classid 1:60 htb rate 1000000kbit ceil 1000000kbit burst 10000 cburst 10000 prio 6
$TQA $WI parent 1:60 handle 60: pfifo limit 1000
$TFA $WI parent 1: prio 6 protocol 802.1q u32 match u32 0 0 flowid 1:60 <-- Anything 802.1Q/VLAN tagged is NOT WAN traffic; we can thus classify all all 802.1Q/VLAN traffic as LAN
# egress 0: 50-100%
$TCA $WI parent 1:1 classid 1:10 htb rate 5120kbit ceil 10240kbit prio 1 quantum 1500 overhead 27 linklayer ethernet
$TQA $WI parent 1:10 handle 10: $EQ <-- Use the user-specified queue rather than hard-code SFQ
$TFA $WI parent 1: prio 10 protocol ip handle 1 fw flowid 1:10
# egress 1: 30-100%
$TCA $WI parent 1:1 classid 1:20 htb rate 3072kbit ceil 10240kbit prio 2 quantum 1500 overhead 27 linklayer ethernet
$TQA $WI parent 1:20 handle 20: $EQ
$TFA $WI parent 1: prio 20 protocol ip handle 2 fw flowid 1:20
# egress 2: 15-100%
$TCA $WI parent 1:1 classid 1:30 htb rate 1536kbit ceil 10240kbit prio 3 quantum 1500 overhead 27 linklayer ethernet
$TQA $WI parent 1:30 handle 30: $EQ
$TFA $WI parent 1: prio 30 protocol ip handle 3 fw flowid 1:30
# egress 3: 5-100%
$TCA $WI parent 1:1 classid 1:40 htb rate 512kbit ceil 10240kbit prio 4 quantum 1500 overhead 27 linklayer ethernet
$TQA $WI parent 1:40 handle 40: $EQ
$TFA $WI parent 1: prio 40 protocol ip handle 4 fw flowid 1:40
# egress: ACK
$TFA $WI parent 1: prio 14 protocol ip u32 match ip protocol 6 0xff match u8 0x05 0x0f at 0 match u16 0x0000 0xffc0 at 2 match u8 0x10 0xff at 33 flowid 1:10
# egress: SYN
$TFA $WI parent 1: prio 15 protocol ip u32 match ip protocol 6 0xff match u8 0x05 0x0f at 0 match u16 0x0000 0xffc0 at 2 match u8 0x02 0x02 at 33 flowid 1:10
# egress: ICMP
$TFA $WI parent 1: prio 13 protocol ip u32 match ip protocol 1 0xff flowid 1:10
# WAN ingress ($WI:ingress can NOT use FW marks; we MUST use $LI:egress)
tc qdisc del dev $LI root 2>/dev/null
$TQA $LI root handle 1: htb default 40
# 1:1
$TCA $LI parent 1: classid 1:1 htb rate 40960kbit ceil 40960kbit linklayer ethernet
# LAN 1:2 ($LI:egress being used in place of $WI:ingress, we MUST be careful of *wired* to *WLan* LAN traffic) <-- NOTA BENE: currently, this traffic remains "within" the bridge and is seen neither by tc nor iptables BUT, the day the bridge_netfilter kernel module is enabled, it will appear; better safe than sorry
$TCA $LI parent 1: classid 1:2 htb rate 1000000kbit ceil 1000000kbit burst 10000 cburst 10000
# LAN 1:60
$TCA $LI parent 1:2 classid 1:60 htb rate 1000000kbit ceil 1000000kbit burst 10000 cburst 10000 prio 6
$TQA $LI parent 1:60 handle 60: pfifo limit 1000
$TFA $LI parent 1: prio 6 protocol ip handle 6 fw flowid 1:60
# ingress 0: 50-100%
$TCA $LI parent 1:1 classid 1:10 htb rate 20480kbit ceil 40960kbit prio 1 quantum 1500 overhead 27 linklayer ethernet
$TQA $LI parent 1:10 handle 10: $EQ
$TFA $LI parent 1: prio 10 protocol ip handle 1 fw flowid 1:10
# ingress 1: 30-100%
$TCA $LI parent 1:1 classid 1:20 htb rate 12288kbit ceil 40960kbit prio 2 quantum 1500 overhead 27 linklayer ethernet
$TQA $LI parent 1:20 handle 20: $EQ
$TFA $LI parent 1: prio 20 protocol ip handle 2 fw flowid 1:20
# ingress 2: 15-100%
$TCA $LI parent 1:1 classid 1:30 htb rate 6144kbit ceil 40960kbit prio 3 quantum 1500 overhead 27 linklayer ethernet
$TQA $LI parent 1:30 handle 30: $EQ
$TFA $LI parent 1: prio 30 protocol ip handle 3 fw flowid 1:30
# ingress 3: 5-100%
$TCA $LI parent 1:1 classid 1:40 htb rate 2048kbit ceil 40960kbit prio 4 quantum 1500 overhead 27 linklayer ethernet
$TQA $LI parent 1:40 handle 40: $EQ
$TFA $LI parent 1: prio 40 protocol ip handle 4 fw flowid 1:40
# ingress: ACK
$TFA $LI parent 1: prio 14 protocol ip u32 match ip protocol 6 0xff match u8 0x05 0x0f at 0 match u16 0x0000 0xffc0 at 2 match u8 0x10 0xff at 33 flowid 1:10
# ingress: SYN
$TFA $LI parent 1: prio 15 protocol ip u32 match ip protocol 6 0xff match u8 0x05 0x0f at 0 match u16 0x0000 0xffc0 at 2 match u8 0x02 0x02 at 33 flowid 1:10
# ingress: ICMP
$TFA $LI parent 1: prio 13 protocol ip u32 match ip protocol 1 0xff flowid 1:10
;;
stop)
tc qdisc del dev $WI root 2>/dev/null
tc qdisc del dev $LI root 2>/dev/null
;;
*)
#---------- Upload (WAN egress) ----------
tc -s -d qdisc ls dev $WI
tc -s -d class ls dev $WI
tc -s -d filter ls dev $WI
echo
#---------- Download (WAN ingress) ----------
tc -s -d qdisc ls dev $LI
tc -s -d class ls dev $LI
tc -s -d filter ls dev $LI
echo
esac

[EIDTED]: removed the "flows 64" limit for fq_codel (keep the 1024 default); I thought each flow used a 10240-packet queue (as per limit parameter; that would have been a LOT of resources used), but looking at the (kernel) source code, it seems it is not the case

/tmp/mangle_rules.tqfix:

*mangle
: PREROUTING ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:QOSO - [0:0]
-A QOSO -j CONNMARK --restore-mark --mask 0x7 <-- The MARK used by tc
-A QOSO -m connmark ! --mark 0x0/0x10 -j RETURN <-- non transferred-limited rules can bail out immediately; I switch the mask from 0xff00 to 0x10 to account for the Asus comment stating than only the first 8 bits ought to be used for connmarks (higher bits being reserved for TrendMicro)
-A QOSO -s 172.24.0.0/16 -d 172.24.0.0/16 -j CONNMARK --set-mark 0x16/0x17 <-- Make sure LAN-to-LAN is identified, for use in the br0 qdisc
-A QOSO -s 172.24.0.0/16 -d 172.24.0.0/16 -j RETURN
-A QOSO -d 224.0.0.0/4 -j CONNMARK --set-mark 0x16/0x17 <-- Multicast traffic is LAN-only by definition (non-routable)
-A QOSO -d 224.0.0.0/4 -j RETURN
--> BEGIN: USER-specified rules <--
-A QOSO -p tcp -m multiport --dport 53 -j CONNMARK --set-mark 0x11/0x17
-A QOSO -p tcp -m multiport --dport 53 -j RETURN
-A QOSO -p udp -m multiport --dport 53 -j CONNMARK --set-mark 0x11/0x17
-A QOSO -p udp -m multiport --dport 53 -j RETURN
-A QOSO -p tcp -m multiport --dport 443 -j CONNMARK --set-mark 0x12/0x17
-A QOSO -p tcp -m multiport --dport 443 -j RETURN
-A QOSO -p tcp -m multiport --port 80 -m connbytes --connbytes 0:10000000 --connbytes-dir both --connbytes-mode bytes -j CONNMARK --set-mark 0x02/0x17 <-- Transferred-limited rule; note the 0x02 MARK (not 0x12), so this rule continues to be hit for the transferred bytes accounting to take place; also, we can NOT use d(estination) ports here or response packets will NOT get caught here and be unaccounted for
-A QOSO -p tcp -m multiport --port 80 -m connbytes --connbytes 0:10000000 --connbytes-dir both --connbytes-mode bytes -j RETURN
-A QOSO -p tcp -m multiport --dport 80 -j CONNMARK --set-mark 0x13/0x17 <-- The user OUGHT to define an "after-limit" rule, used once the transferred-limit is reached
-A QOSO -p tcp -m multiport --dport 80 -j RETURN
--> END: USER-specified rules <--
-A QOSO -m connmark ! --mark 0x0/0x7 -j RETURN <-- Make sure we do not wrongly re-classify (below) transferred-limited connections that have reached their limit (this rule is hit by response packets that do not match the d(estination) ports of the "after-limit" rule)
-A QOSO -j CONNMARK --set-mark 0x14/0x17
-A QOSO -j RETURN
-A POSTROUTING -o eth0 -j QOSO <-- No need for other mangle chains anylonger; POSTROUTING does all the job (before the packet reaches the eth0 or br0 egress qdisc)
-A POSTROUTING -o br0 -j QOSO
COMMIT

 

[cdufour]

Attached the qos.c patch file corresponding to the changes proposed in comment #19.

I managed to build AsusWRT (NG) from the latest GutHub pull and verified it passes compilation:

[...]
make[4]: Entering directory '/local/data/asuswrt-merlin/build/amng.ac88/release/src/router/rc'
[rc] CC qos.o
make[4]: Leaving directory '/local/data/asuswrt-merlin/build/amng.ac88/release/src/router/rc'
[...]
Creating ASUS RT-AC88U firmware to image/RT-AC88U_384.4_beta3-g.trx
TRX Image:
Total Size .... : 39473152 (38548.0 KB) (37.6 MB)
CRC-32 ........ : 74EB9903

Problem now: I dare not test further and risk bricking my (in use) AC88U.
PS: Has anyone ever tried running AsusWRT using QEMU emulator (for test purposes)?

@RMerlin: if this patch looks ok for you and you're willing to integrate it in your builds, I'll send you a pull-request via GitHub.