2x2 AC Access Point Roundup - Part 2

[thiggins]

Jim Salter takes another run at our collection of 2x2 AC APs, this time testing multi-client load handling in open air, with roaming too!

Read on SmallNetBuilder

 

[ika]

Thank you very much for the great article.

We run two inexpensive Zyxel NXC2500 controllers in two separate buildings with Zyxel APs on each (we inherited the setup with the buildings). I can tell you that the setup with the controller is very easy and quick, and they are quite reliable ("set-up and forget") devices. You just plug in the APs and the controller picks it up automatically, upgrades it to the latest FW and does everything for you as you would expect from a WI-FI controller.

That being said, the performance of the APs are truly one of the worst I ever encountered, and I spent way too much time with Zyxel experts and also myself trying to set them up correctly . Even if you don't roam, and just sit under one AP with two phone devices at night, there is still a chance that you will get speeds resembling the good old US robotics BBS modem times. It's really unbelievable how bad these APs are, and things were even worse when we had the earlier N300 and N600 models. I highly recommend to avoid their AP products as much as possible.

ps.: An interesting note: They are quite (as much as 10dB) stronger when they fed with POE. That's quite a big difference imho.

 

[Internet Man]

Is the D-Link DAP-2610 really a v2? I can't find any references to any version other than the A1 covered in part 1 of the roundup. D-Link only lists hardware version A on their support page and there is only one FCC entry covering a DAP-2610.

The only place I could really ding the EAP-225's interface is in trying to identify the hardware version. This model has three hardware revs—v1, v2, and v3—and they have separate firmware lines. I eventually had to give up and go look at the physical label on the access point.

What is the FCC ID printed on the label of your TP-Link EAP-225 v2, if any?

I do not believe that the extra MIMO stream really gave the A60 any advantage with my strictly 2x2 test stations.

That's surprising to me since 4x4:4 devices typically outperform 3x3:3 devices in Tim's tests with a 2x2:2 STA.

 

[Jim Salter]

Is the D-Link DAP-2610 really a v2? I can't find any references to any version other than the A1 covered in part 1 of the roundup. D-Link only lists hardware version A on their support page and there is only one FCC entry covering a DAP-2610.

You know what, I think you're right, and I brain-farted their actual description of it - which is DAP-2610 wave 2, referring to its MU-MIMO support.

What is the FCC ID printed on the label of your TP-Link EAP-225 v2, if any?

Dunno off the top of my head; but I think the FCC ID on the label is what I went by eventually. Couldn't figure it out from the UI or from the box - having to go chase down the physical access point itself and hunt for labels was what ticked me off. =)

That's surprising to me since 4x4:4 devices typically outperform 3x3:3 devices in Tim's tests with a 2x2:2 STA.

In the real world, I've never seen any correlation between large number of MIMO streams on the AP and 2x2 STA performance. Granted, I've spent more time debunking that on consumer routers and mesh kits (which love to claim ridiculous "AC speed ratings" based on adding up all the MIMO streams multiplied by all the radios multiplied by some extra "just for fun" fudge factors). But nothing I've seen there, or any understanding I have of the technology, leads me to believe that it would be any different in these APs. And the OpenMesh A60 certainly did not seem to stand out here.

 

[thiggins]

You know what, I think you're right, and I brain-farted their actual description of it - which is DAP-2610 wave 2, referring to its MU-MIMO support.

I've removed references to v2 for the DAP-2610.

 

[Internet Man]

You know what, I think you're right, and I brain-farted their actual description of it - which is DAP-2610 wave 2, referring to its MU-MIMO support.

No worries!

Dunno off the top of my head; but I think the FCC ID on the label is what I went by eventually. Couldn't figure it out from the UI or from the box - having to go chase down the physical access point itself and hunt for labels was what ticked me off. =)

I'd hate to tick you off again but it would be handy to have some more details about the internals of the V2. V1 is 2x2:2 for 2.4GHz bgn while V3 is actually 3x3:3 for 2.4GHz bgn. The product page for V2 suggests that it is 2x2:2 so I'm guessing that V2 is pretty much identical to V1 and may even share its FCC id.

In the real world, I've never seen any correlation between large number of MIMO streams on the AP and 2x2 STA performance. Granted, I've spent more time debunking that on consumer routers and mesh kits (which love to claim ridiculous "AC speed ratings" based on adding up all the MIMO streams multiplied by all the radios multiplied by some extra "just for fun" fudge factors). But nothing I've seen there, or any understanding I have of the technology, leads me to believe that it would be any different in these APs. And the OpenMesh A60 certainly did not seem to stand out here.

My understanding is that additional tx/rx chains can help improve receive sensitivity and effective transmit power so a difference could be more evident at longer distances or in situations where there is more interference or reduced signal quality.

 

[Jim Salter]

My understanding is that additional tx/rx chains can help improve receive sensitivity and effective transmit power so a difference could be more evident at longer distances or in situations where there is more interference or reduced signal quality.

It's possible that that's an effect, but I don't think it's a very significant one - I've seen lower AC speed rated (which basically means fewer MIMO streams) devices kick the teeth out of much higher AC speed rated devices over and over and over again while testing gear for Ars Technica, the Wirecutter, etc.

 

[coxhaus]

I would like to know if using band steering whether you can maintain a FaceTime call. If not can you turn off the 2.4GHz radio then maintain a FaceTime call across APs. It seems like a simple test.

My wife uses FaceTime a lot while walking around the house and outside.

 

[Jim Salter]

I would like to know if using band steering whether you can maintain a FaceTime call. If not can you turn off the 2.4GHz radio then maintain a FaceTime call across APs. It seems like a simple test.

My wife uses FaceTime a lot while walking around the house and outside.

You ought to be able to, emphasis *ought*. Certainly Google Hangouts calls and the like are fine when I roam between APs and bands in my house. That said, I'm not an Apple person except at gunpoint; I've literally never made a Facetime call in my life, and Apple ... well, they frequently surprise and disappoint.

I have an iPad I use for test purposes; if you want to give me a number to Facetime call I suppose I could do that, then walk around the house? It'd be nice to have a definitive answer for your question, and the Googles are turning up nothing because all the results are packed chock full of people concerned about *cellular* roaming and telco charges.

 

[coxhaus]

Sounds like a plan. My wife is out of town until Monday some time so after that we can try it. She has all her gear, iPad, iPhone with her.

I can try my new Cisco WAP371 wireless units I just installed. My old Cisco three WAP321 units worked great. I am still missing one WAP371 as I only have 2 right now so my outside is not lit. I had to setup a SG300-10MPP switch to have enough power to run the new WAP371 unit since they require POE+. I have all the VLANs working from my layer 3 switch through the SG300-10MPP to WAP371 units. I did it while she was gone. It was much easier.

 

[Sineira]

Thank you, this is great!
I love the roaming tests. Might buy 2 WAC505 now

 

[sfx2000]

@Jim Salter - you are testing the clients, not the AP's...

The stations themselves are four identical Samsung Chromebooks running GalliumOS equipped with a built-in Intel Dual Band Wireless-AC 7265 and Linksys WUSB-6300 external wireless adapters. The WUSB-6300 is my reference adapter and used for most performance testing, because there's less variation between the four NICs than there is between the internal NICs in the Chromebooks, and also because they have exceptional TX performance.[/code]

Just saying - I've spoken about this before...

 

[YeOldeStonecat]

Just a quick note to the editor regarding Ubiquitis Cloud Keys. As much of a Ubiquiti fan that I am, yes the MongoDB is sensitive to rude shutdowns and easily corrupts. So the Cloud Keys themselves often have power cut to them, and it tanks the DB. So what we do when prepping networks at our office, before unplugging everything, I log into the CK and gracefully shut it down. And of course we always put network gear on battery UPS..so that helps. It happens even on controllers you build on PCs or servers, even virtualized cloud controllers. Granted..those rarely experience a rude shutdown.

Anyways, for repairing it when MongoDB does get tangled up...I use one of two methods.
*You can reset them remotely 99% of the time, without needing to go grab it and push the reset button. One method, the Cloud Key itself does have a local web admin login (not the Unifi login). You can log into it, and restore from the most recent backup (which is stored on the micro SD card that comes with the CK). Second method, SSH into CK and there is a command line you can run to repair the MongoDB.

 

[thiggins]

@Jim Salter - you are testing the clients, not the AP's...

How would you test roaming without a STA to roam?

 

[Jim Salter]

@Jim Salter - you are testing the clients, not the AP's...


Just saying - I've spoken about this before...

I'm testing both. I'm aware of the ambiguity involved and tried to address it in the article; for example I test roaming using the Intel 7265 rather than the Linksys WUSB-6300 because the former does roam much more readily than the latter. With that said, if the only thing I were testing were the STA, my results would have been identical or near-identical with every make of AP, router, or mesh kit - and they're anything but.

I'm working toward being able to catch deauth (AP-enforced minRSSI) and BSS transition frames in-flight, but I'm not there yet; in the meantime empirical testing is much better than no testing at all.

 

[sfx2000]

How would you test roaming without a STA to roam?

In an ideal world - it would be a representative client STA - ideally something like the OctoScope PAL - although I'm told it has limited support for 11k/11v/11r.

Once the AP's have been normalized, then one can try different clients, and there - alternate vendors, like QC-Atheros, Broadcom, Ralink (mediatek), RealTek, Intel and so forth.

I think it's also a challenge to get appropriate log information out of the chipset itself - not many adapters support monitor mode, and there are those that do, but report a lot of artifacts and odd behavior.

The challenge with the roaming testing in Jim's article is that it's not very conclusive from an objective point of view - the RF characteristics of a built-in Intel 7260 is one thing, as is the behavior of the firmware inside the chip - same would apply to the Realtek based WUSB-6300.

His findings would apply to his particular unit - but if I were to bring in a MacBook Air, likely the findings would be different - same with a Samsung Galaxy S8...

Hence my statement - the roaming tests were more about the client than the AP's - without a stable baseline, it's all subjective.

 

[sfx2000]

I'm working toward being able to catch deauth (AP-enforced minRSSI) and BSS transition frames in-flight, but I'm not there yet; in the meantime empirical testing is much better than no testing at all.

One thing to try is to set up a monitor STA just to listen to frames within the ESS - that can go a long way - also you should be observing the DS between the AP's, as this is where you can capture interesting things related to the handover between AP's - specifically the security items.

When testing - look at not just WPA2-Personal (PSK), but also WPA2-Enterprise, as the procedures there can be different - and I would include testing with no auth in place (e.g. open system), because again - it's different layers of how the client STA manages the transition, and how the AP's support it.

 

[Jim Salter]

Hence my statement - the roaming tests were more about the client than the AP's - without a stable baseline, it's all subjective.

You could say the same about throughput testing: it's also different for different clients. Like it or not, different equipment is different; the "stable baseline" here is the same client in the same STA being used for all tests. The fact that the different APs produce different - in some cases wildly different - roaming behavior despite the same client being used in the same STA moving in the same patterns is demonstrative of the fact that there are real differences in the APs themselves.

Worse, the kind of clarity you want isn't possible even if every frame is captured, because 802.11 WiFi isn't preemptive in the first place - even with 802.11k/v support on both ends, the infrastructure is merely "assisting" clients to make "better" choices - by design, the client is still ultimately in control of when, how, and where it roams. (This is not the way *I'd* have designed a network - reminds me of the abomination that was "cooperative multitasking" - but I wasn't consulted.)

The biggest actual problem with the roaming testing without being able to see what's happening at the frame level isn't anything to do with the STA not being a "stable baseline", it's that different TX strengths from the APs can also affect the STA's roaming behavior, completely irrespective of any actual roaming assistance in the form of BSS transition frames or enforced minRSSI (by way of de-authing the STA) from the AP.

 

[Jim Salter]

It happens even on controllers you build on PCs or servers, even virtualized cloud controllers. Granted..those rarely experience a rude shutdown.

It's *extremely* rare for the DB to get lunched on a commodity server, because it's stored on a journaling filesystem - ext4 or NTFS. For mongo to get lunched there, it needs to be actively being written to at the very moment the system crashes - and it's not a terribly active db.

The big problem with the cloud keys is that they use ext2, which *isn't* a journaling filesystem, for the backing filesystem on the sd card - so it's EXTREMELY easy for the card's filesystem to be inconsistent after a crash.

 

[thiggins]

SFX: Jim and I discussed the test methodology before and during the review. Yes, the roaming test was subjective and far from ideal. But, as Jim pointed out, so is every other method we or any other reviewer uses.

The octoScope Pal has its own quirks and is based on real Qualcomm chipsets. The advantages it provides, however, is the ability to monitor many parameters (RSSI of each stream, Tx rate, Rx rate, throughput, BSS, etc) and the ability to set many connection parameters.

Right now, Pal roaming controls include band preference (auto, 2.4, 5), roam RSSI (level to trigger roaming start) and target RSSI (signal level to qualify roaming targets). 11v is also supported, but only to advertise BSS Transition capability.

I'm working with octoScope to expand the Pal's capabilities and to develop roaming tests. You'll see more on this soon.