Does OFDMA Really Work? Part 2

[thiggins]

Our test of six Wi-Fi 6 routers found no significant benefit from OFDMA.

Continue reading on SmallNetBuilder

 

[JohnB_123]

Excellent piece @thiggins

So if I read this correctly, you've proven that cheap AX/Wifi6 routers like the AX-58U and the RAX15 aren't discernible--or even inferior-- to 4+ year old AC/Wifi5 routers like the R7800?

 

[Razor512]

It seems like in general all of the newer standards produced lower latency than the R7800, though the RAX15 ended up producing significantly worse results when OFDMA was enabled.

While it is nearly impossible too scientifically test, I wonder if OFDMA will have any improvements in normally congested environments, e.g., an urban area where it is normal to see 100+ access points in range and no one seems willing to use Ethernet, even when they have a desktop PC in the same room as their router. Over time as more people move to OFDMA, will WiFi become less bad in those areas?

 

[thiggins]

So if I read this correctly, you've proven that cheap AX/Wifi6 routers like the AX-58U and the RAX15 aren't discernible--or even inferior-- to 4+ year old AC/Wifi5 routers like the R7800?

No. All AX routers had lower latency than the two AC routers tested. But the lower latencies were not due to OFDMA.

AX routers can provide other advantages over AC routers, especially for 2.4 GHz AX devices. But OFDMA isn't one of the advantages.

 

[thiggins]

I wonder if OFDMA will have any improvements in normally congested environments

It's possible that OFDMA may some day evolve to provide some of its touted advantages. But I think it will take years, as it did with MU-MIMO.

 

[JohnB_123]

No. All AX routers had lower latency than the two AC routers tested. But the lower latencies were not due to OFDMA.

AX routers can provide other advantages over AC routers, especially for 2.4 GHz AX devices. But OFDMA isn't one of the advantages.

Very helpful, thanks Tim.

Since I've put ethernet to all my stationary devices, the only AX devices we have are mobile - and I haven't seen a discernible difference with our RT-AX88U vs. our Netgear AC router whatsoever.

 

[digital10]

It's possible that OFDMA may some day evolve to provide some of its touted advantages. But I think it will take years, as it did with MU-MIMO.

I thought there is an article here saying MUMIMO didnt show any real benefits and sometimes will give worse performance

 

[thiggins]

I thought there is an article here saying MUMIMO didnt show any real benefits and sometimes will give worse performance

Here is the link
https://www.smallnetbuilder.com/wireless/wireless-features/33100-why-you-don-t-need-mu-mimo

Three years later, I'm told by colleagues at octoScope that MU-MIMO can provide total throughput gain. I have not verified this myself. But MU-MIMO testing will be incorporated into the upcoming new Wi-Fi test suite.

 

[StR]

Here is the link
https://www.smallnetbuilder.com/wireless/wireless-features/33100-why-you-don-t-need-mu-mimo

Three years later, I'm told by colleagues at octoScope that MU-MIMO can provide total throughput gain. I have not verified this myself. But MU-MIMO testing will be incorporated into the upcoming new Wi-Fi test suite.

Do you know if those new findings about MU-MIMO have been published somewhere?
I was trying to find any followup on that initial report (thank you for posting it), and that seems to be the only real test of MU-MIMO performance.

Any pointers to the more recent tests/reports/... would be appreciated.
I am considering an upgrade from a non-MU-MIMO router to a more recent technology, and trying to decide which technology will actually yield a reasonable gain. Per your test, OFDMA doesn't seem to offer much of a gain for a real-life setting. The question now is whether MU-MIMO does.

Of course, the progressively higher QAM level provides the advantage ( n -> ac -> ax). But I have a suspicion that in my case working MU-MIMO could in principle offer a better advantage.

 

[coxhaus]

I heard Cisco enterprise guys mention it when talking on a video with the new catalyst wireless APs about wireless at an airport with 300 or 400 connections and getting benefit. No specifics. And I did not understand. Do you have that many connections with 1 wireless AP?

 

[thiggins]

And I did not understand. Do you have that many connections with 1 wireless AP?

That would not be good practice for normal APs. Maybe for a Xirrus AP that has multiple radios.

 

[coxhaus]

Maybe then it would be 300 or 400 users on a virtual AP. Does MU-MIMO make sense across a virtual AP? I would not think so since it is hardware.

I guess I will have to wait and see what they do.

 

[geminis3]

Maybe then it would be 300 or 400 users on a virtual AP. Does MU-MIMO make sense across a virtual AP? I would not think so since it is hardware.

I guess I will have to wait and see what they do.

IIRC master interface hardware settings are applied to VAPs

 

[CrystalLattice]

ARUBA ENGINEERS said, "here are also advantages for less-capable stations. As link- speeds have increased, some devices struggle to transmit at the maximum rates. Whereas with full-channel OFDM, they have to do the best they can, perhaps not filling the medium, OFDMA allows them to cap their maximum rates. This allows for simpler hardware implementations and potentially longer battery life, similar to the 20 MHz-only (see later) concept for IoT sensors.OFDMA also offers opportunities for applying QoS, particularly for traffic that demands low latency or jitter. While a device might have to wait a long while in a single-user OFDM system for a transmit opportunity, OFDMA allows it to transmit ‘little and often’, reducing latency and jitter."

 

[StR]

Disclaimer: The following is a pure speculation on my part, not supported by any tests or data.

I suspect there could be two reasons why in the original tests (very interesting and useful - thank you @
thiggins !) the results with the OFDMA enabled were not necessarily significantly better:
1. The actual implementation of the OFDMA in the tested routers in questions may not be optimized/tuned (by the manufacturer); and
2. The actual benefit of the OFDMA is [more] pronounced for a larger number of connecting clients. (It might not need to be 300-400, but just 20-40). Moreover, it might be more beneficial (latency-wise) for the situations when the each of the clients is nearly or even over-saturating the overall upstream router bandwidth, but when only the entire cohort collectively gets close to the total saturation, while the individual clients are well under that.
It is conceivable that with a larger number of clients, where each client would need to wait for its turn to "speak" for much longer, - the OFDMA would have a [more significant] benefit by allowing each specific device to get its turn sooner. The comment from Aruba engineers is consistent with this guess.

It is also possible that it is the combination of (1) & (2): depending on the number of the clients, different values of the parameters of the OFDMA would be maximizing its benefit, and the tested routers are optimized for a larger number of clients (say, 20-30).

It might be possible to test if (2) is the case, - if @thiggins could have some 15-20 (30? ) STApals trying to utilize the network simultaneously (and that can be tested with different bandwidth loads from each/all of them).
For (1) - as it was suggested in the article discussing the test results, - it might take some considerable time for ASUS to optimize their implementation of OFDMA. And that should be possible not just in theory: after all, LTE (4G wireless) has been successfully using OFDMA for years (see, e.g. here: https://www.gta.ufrj.br/ensino/eel879/trabalhos_vf_2014_2/rafaelreis/ofdma_scfdma.html )
(As far as I understand, in case of LTE, it is actually OFDMA for downlink and SC-FDMA for the uplink: https://www.electronics-notes.com/a...rm-evolution/ofdm-ofdma-scfdma-modulation.php )

 

[thiggins]

@StR No consumer AX router on the market today supports HE-MU frames to more than 8 STAs; most support only 4.

Yes, more optimization is needed for AX AP airtime scheduling. But it will take years and may never pan out to deliver all the glorious features promised by manufacturers.

OFDMA is yet another technology borrowed from the mobile world that doesn't translate to Wi-Fi very well. The two technologies are very different. The network is in control for. mobile; devices are in control for Wi-Fi.

I have little interest in testing with more than 4 AX STAs, since I have a consumer focus. At some point in the distant future the average consumer may have more than 4 Wi-Fi 6E devices. But if a $500 router can't provide measurable results of advertised improvements in typical use scenarios, they are just blowing smoke to attract naive buyers.

Believe me, I'd like to be proved wrong. But I've spent over a year trying many methods to find any evidence of consistent, significant performance improvement from AX and have yet to find it. Typical sniffs show very low percentages of HE-MU frames; most are HE-SU.

I've been working for the past month with octoScope on using video traffic with up to 12 STApals to see if OFDMA helps. I haven't seen consistent improvement yet.

 

[mbze430]

Here is the link
https://www.smallnetbuilder.com/wireless/wireless-features/33100-why-you-don-t-need-mu-mimo

Three years later, I'm told by colleagues at octoScope that MU-MIMO can provide total throughput gain. I have not verified this myself. But MU-MIMO testing will be incorporated into the upcoming new Wi-Fi test suite.

I was wondering if you have put a test bed together for AX MU-MIMO testing yet?

 

[thiggins]

Yes. The AX Testbed uses Intel AX200 STAs, which support AX MU-MIMO. However, AX MU-MIMO support in consumer routers was a work in progress when I stopped my OFDMA explorations back in May.

I won't be specifically looking for AX MU-MIMO effects when I resume testing. If a router supports it, I'll enable it, along with OFDMA and let throughput and latency test results tell the tale.

 

[Deleted member 22229]

Interesting looking forward to the review.

 

[cooloutac]

Excellent piece @thiggins

So if I read this correctly, you've proven that cheap AX/Wifi6 routers like the AX-58U and the RAX15 aren't discernible--or even inferior-- to 4+ year old AC/Wifi5 routers like the R7800?

I would take what this guy says with a grain of salt, since he also believes router rotation and antennae position on consumer routers like asus have no affect on signals. IN my real world experience and in my environment the ax58u blows away the ac86u. Especially on 2.4ghz and not just on ax. Maybe not related to ofdma, but there must be some other major advantages going on with these routers that may be be an after affect. At least he agrees on that.

But He says himself he can't test for mu-mimo cause he uses a testbed that can't separate the devices. So i would chalk up his ofdma tests too for the same reasons. which is unfortunate because I would like to see him to do tests and provide the blueprint of his room and distance between devices which is when these things are most discernible. This would be a more real world experience. Test them like you would use them in your house like amateurs do on youtube which I find more informative. I like the way this guy does tests--> Steen's Lab - YouTube

I'd also be interested in battery life tests and qos with the latest mobile phones and laptops on these newer routers as an above poster referred to.