2.4 GHz and 5 GHz tests - Linksys WRT-1900AC vs. Asus RT-AC66U

[njweb]

UPDATED 2.4 GHz and 5 GHz tests - Linksys WRT-1900AC vs. Asus RT-AC66U

Hopefully this will help some of those on the fence in their decision as to which router to buy...

EDIT - APRIL 24 1:28 AM EDT - I have added a SECOND pdf with new results run tonight with a different WRT-1900AC config (see post #7 below). (end edit)

I have attached the results (see attached pdf at the bottom of this post) of my testing comparing the wireless performance of my Asus RT-AC66U router against my Linksys WRT-1900AC.

SummaryIn short (overall), the 5 GHz of the WRT-1900AC was impressive and 2.4 GHz was lackluster at the furthest test location.

Interesting Observations
1. The biggest anomaly is that in the toughest location (row 14) the WRT-1900AC had a stronger 5 GHz signal than the RT-AC66U yet failed to deliver when it came to speedtest.net download results (31.33 MBps down for WRT-1900AC versus 57.66Mbps down for the RT-AC66U, even though the WRT-1900AC’s signal was much stronger on that band in that location than the RT-AC66U -80 to -77 versus -71 TO -62 (mainly -69 to -66)!!!
Perhaps someone can explain this as I was not expecting such low speedtest results.

2. The 'LAN Speed Test', both read and write, test results of the
WRT-1900AC (row 13) on 2.4 GHz were half that of the RT-AC66U

3. Overall (in every test location), the ‘LAN Speed Test‘ read tests of the WRT-1900AC were lower than those of the RT-AC66U, despite the former having a stronger signal (as reported by Asus Wireless utility that came with the USB-AC53 wireless adapter).


SETUP: • USB wireless adapter (client): Asus USB-AC53. AC1300, dual-band .
• Computer: i7 quad core, 8 GB RAM laptop running Windows 8.1.
• Browser (for Speedtest.net testing): Firefox (no special reason).

TEST LOCATIONS (laptop with wireless adapter):

• Bedroom (door open): This room is adjacent to the room where the router is located.
  
• Living Room (no doors): This room is on the same floor as the router but on the other side of the house
  
• Family Room: This room is on the opposite side of the house and also one floor down from the room where the router is located (toughest location).

 

[thiggins]

Thanks for running the tests and sharing. A few questions:

1) What was 2.4 GHz bandwidth set to?

2) What is neighboring wireless activity like?

3) How close in time were the tests run for the different routers?

4) How did you measure signal strength?

 

[jlake]

And more questions....

1. What channel numbers did you use (on each router)

2. Did you have one of the routers unplugged from electrical outlet while you performed tests on the other router?

3. Did you use different/unique SSIDs for both routers? In other words, did linksys 5 ghz have the same (or different) 5ghz ssid as asus router? If same, did you remove wireless profiles?

 

[njweb]

Thanks for running the tests and sharing. A few questions:

1) What was 2.4 GHz bandwidth set to?

2) What is neighboring wireless activity like?

3) How close in time were the tests run for the different routers?

4) How did you measure signal strength?

No problem.


1) What was 2.4 GHz bandwidth set to?

'Auto' on both routers.

2) What is neighboring wireless activity like?

It is a single family home in a residential neighborhood with decent lot sizes, so there are not as many networks within range as one might have in an apartment building e.g., but I still see about
12 - 15 networks or so (don't quote me on this since I haven't counted in ages, but I am going by my visual recollection of what I typically see).

3) How close in time were the tests run for the different routers?
I ran most (if not all) of the RT-AC66U tests around 10 PM yesterday.
I ran the WRT-1900AC tests starting around 7:30 AM today
(Fell asleep last night at some point, so I did not get to run the rest as planned).

4) How did you measure signal strength?
I used the values returned by the Asus wireless utility that was bundled with the wifi adapter's driver (USB-AC53).


While we're discussing specifics:
I also made sure to place the laptop as close as possible to the same spot within each test location, as compared to the position the laptop was in for the prior router's run (had marked each of the 3 spots with pieces of paper /
non-metallic objects).


I may take fresh measurements again tonight (time permitting), although I suspect the results will be pretty similar.
I may focus on the furthest location (first) since that is where I observed the most statistically significant variations between the two routers that would have real world impact on my day to day experience.

 

[njweb]

And more questions....

1. What channel numbers did you use (on each router)

2. Did you have one of the routers unplugged from electrical outlet while you performed tests on the other router?

3. Did you use different/unique SSIDs for both routers? In other words, did linksys 5 ghz have the same (or different) 5ghz ssid as asus router? If same, did you remove wireless profiles?

1. I left the channel set to 'Auto' on both routers.
I may experiment with specific channels tonight, time permitting since I have a big event to plan for this weekend.


2. Yes, I only had one router plugged in at a given time.


3. I reused the same SSID's (since I was very short on time - that is an understatement).
Likewise I did not remove any wireless profiles.
IF I have enough time tonight, I will remove the wireless profiles and also reboot the laptop (as well as perform other 'cleanup' activities) between testing the two routers.

 

[jlake]

1. Yeah same channels would be important. (And channel width).

2.

3. From my experience, you can't switch back and forth between Broadcom and Marvell on same wireless profile (same ssid) when testing. ymmv.
Personally, I would give completely unique SSIDs (something you've never used before). Eg. Wrt24test, wrt5test, asus24test, asus5test
That would remove wireless profile from the equation.

 

[njweb]

Tonight's WRT-1900AC results, overall, much better than this morning

Tonight's WRT-1900AC results, overall, are much better than this morning.
I tried all sorts of reboots, a router reset and adapter profile refreshes etc., initially to no avail, tonight.
Eventually something paid off (nothing more than one more reboot e.g.).

I should mention that this morning's runs were done with the WRT-1900AC's DHCP server enabled and with it using the same IP address as my 'primary router' which servers most functions except wireless - I forgot, in my haste to get the tests runs before rushing to work, that I had to reconfigure the WRT-1900AC with a different IP address and disable the DHCP server.
I need to do this since I am running my ISP-supplied Actiontec cable modem / router combo as primary with wifi disabled (have to use theirs for my VOIP and VOD / channel guide etc.).
http://forums.smallnetbuilder.com/showpost.php?p=69660&postcount=8


Anyway, here are this evening's runs - see attached pdf.


Here are the results - see bottom half of PDF (I copied the sheet from this morning to another tab, in the interest of time, and replaced all the results with new runs performed tonight (Wednesday night into Thursday just after midnight) for the WRT-1900AC.
I decided to leave the RT-AC66u runs from this morning for a little reference, but changed the font of the AC66U cells to strikethrough to indicate they are not fresh runs like those of the WRT-1900AC...


The signal strength numbers are great, so hopefully with a bit of tweaking the 2.4 GHz numbers can improve a bit more.

 

[jlake]

If you did actually test while the linksys was double natting on the same subnet as primary router....that would be really bad......and a big no no.

It's unlikely that you were double natting on same subnet since linksys has been making their routers idiot proof for the last couple years. It will automatically detect the double NAT on same subnet, and automatically switch to a random 10.x.x.x subnet or 172.x.x.x subnet.

I can't really say for sure that wrt1900 has that idiot proof feature since I don't have the router. The reason I know that they are idiot proof....is because I have also accidentally double natted a newer linksys on same subnet too.

 

[thiggins]

If you did actually test while the linksys was double natting on the same subnet as primary router....that would be really bad......and a big no no.

Why? Do you think there is a big performance hit?

 

[njweb]

If you did actually test while the linksys was double natting on the same subnet as primary router....that would be really bad......and a big no no.

It's unlikely that you were double natting on same subnet since linksys has been making their routers idiot proof for the last couple years. It will automatically detect the double NAT on same subnet, and automatically switch to a random 10.x.x.x subnet or 172.x.x.x subnet.

I can't really say for sure that wrt1900 has that idiot proof feature since I don't have the router. The reason I know that they are idiot proof....is because I have also accidentally double natted a newer linksys on same subnet too.

Right, I suspect the Linksys indeed switched to another IP address and / or subnet.
It certainly was at one point using a funky IP address (way off the usual 192.168.1.1 etc.) last night when I was reconfiguring my setup as per method outlined in the DSL reports article.


Your comment reminded me of something:
I am hoping / assuming wireless bridge mode will work fine on the WRT-1900AC with the DHCP server disabled along with the fact that I am using 192.168.1.2 instead of 192.168.1.1 (Actiontec is using 192.168.1.1)...
I'll note it in the wireless bridge thread.
Updated the second PDF slightly.

 

[jlake]

Why? Do you think there is a big performance hit?

Hmmm. How can one really comment on a performance hit when he said he was double natting on same subnet as his primary router?

Double natting on the same subnet immediately invalidates the data. Good or bad.

If you could possibly check your wrt1900 and see if it has the idiot proof feature that I mentioned. Maybe he wasn't really double natting on same subnet as main router.

 

[njweb]

Hmmm. How can one really comment on a performance hit when he said he was double natting on same subnet as his primary router?

Double natting on the same subnet immediately invalidates the data. Good or bad.

If you could possibly check your wrt1900 and see if it has the idiot proof feature that I mentioned. Maybe he wasn't really double natting on same subnet as main router.

The results in my second pdf (i.e. the one attached to my post this morning around 1:16 AM I believe) were obtained after setting my WRT-1900as secondary (Actiontec - wireless off; WRT-1900AC DHCP Server off and IP changed to 192.168.1.2), as well as after having reset everything else last night * and after redoing everything until I finally observed the performance results one would expect (or higher of course).

*
Reset Linksys WRT-1900AC to defaults (hard reset).
Reconfigured WRT-1900AC (as noted above).
Deleted wireless profiles on the USB-AC53 wifi adapter side and recreated them.
Rebooted both machines (machine where configuration was done and laptop that is being used as the wireless client).
Shut down and restarted Actiontec cablem modem / router combo.
Etc...

Those results are far more representative of what i was expecting.


Overall I give the edge to the Linksys for a couple of reasons:
Wireless signal strength with the WRT-1900AC is higher in EVERY case (3 locations on both bands) than with the Asus, so with some driver tweaks, wireless performance could likely improve.
In the tougher locations, the speedtest.net results were higher overall on the WRT-1900AC.

I was most disappointed by the LAN Speed Test results, but I am more interested in the speedtest.net donwstream and upstream internet bandwidth results than with wireless file copy speeds.
Luckily the speedtest.net results were very good.

Now I just need to determine which band to use as the link between the two WRT-1900AC routers (one is still in the box) when I configure the wireless bridge.

 

[njweb]

third run with WRT-1900AC (results attached to THIS post

Here are the results from run 3 (see latest pdf attached)

 

[NightOwl326]

Here are the results from run 3 (see latest pdf attached)

These home self test seem pretty well useless since they would very from home to home and setup to setup. Even the results varied over the short time period tested. I am not sure anything is gleaned from these tests that already haven't been gleaned by more controlled tests already posted on this website. Seems kind of pointless. Just my humble opinion.

 

[KGB7]

WAN speeds are subjective thus should never be used in any testing....ever.

Stick to LAN and WLAN only.

I have suggested to Tim in the past, but if you want unbiased results, test the router in the open field miles away from nearest living soul.

There are too many unforseen variables that will sque results in a building or in a test chamber ( unless you can guaranty no side effects from materials or design that makes up a test chamber). You should see the test chamber that Apple uses to test their iPhone, its mind blowing. I know you don't have the access or the budget, so drive out in the.middle of the woods and retest it.

Thanks for your time.

 

[njweb]

Thanks for sharing your thoughts.

My goal was just to perform some informal * real world comparison testing to help me decide whether to stick with the two WRT-1900AC routers we bought (15 day return period expires this coming Sunday or Monday) and, if yes, sell our two RT-AC66U routers.
I figured I would share here to hopefully also help others.

* I controlled certain variables, such as (but not limited to):

using the same 3 specific locations for both sets of tests (marked the exact spots to allow placement within an inch or so)
using the same wireless client (laptop and adapter) for both sets of tests
selected speedtest.net servers that seemed the most stable (less fluctuation)
ran multiple tests to eliminate outlier results


What I can't control (same goes for cNet, PC World, pcmag.com, TrustedReviews, Tweaktown and Extremetech) is the following key variable, which is the key reason my tests are not scientific:

The network environment - i.e. neighboring networks and neighboring wireless clients present and in use at any given time on the same band.

That being said, if I were to test in the wide open (in a big wide open space miles from any other networks), I won't know how each router deals with real world interference from neighboring networks etc.

So from that perspective, I would rather test under real world conditions (within range of neighboring networks), as I did, in order to see how the router will perform in my home.

I also had limited time since I have been busy with personal matters.
If I have time tonight, I will hook the RT-AC66U up again and rerun the tests.


The one key constant is signal strength - the WRT-1900AC consistently bested the AC66U in all 6 scenarios (3 locations across the 2 wireless bands).
Given this, hopefully some driver tweaks will improve the performance.

The most interesting thing to me is the LAN Speed Test performance.

My interest (given my typical usage) is primarily on internet bandwidth.

 

[thiggins]

These home self test seem pretty well useless since they would very from home to home and setup to setup. Even the results varied over the short time period tested. I am not sure anything is gleaned from these tests that already haven't been gleaned by more controlled tests already posted on this website. Seems kind of pointless. Just my humble opinion.

All wireless tests, whether open field, in a test chamber or in a home/office are valid data points. The nature of wireless performance is that it is highly dependent on EVERYTHING.

NJWeb did a good job of trying to be consistent in his testing and documenting the results. If more people took the time to do what he did, there would be more data points to compare.

I agree with NJweb (and have told KGB7) that open field testing isn't of much practical use except for vendors to use as marketing tools for buyers who don't know better.

Lastly, don't put too much faith in signal levels as an indicator of performance. LAN based throughput measurements (between a Gigabit Ethernet wired client and wireless client) are always the best thing to go by.

 

[njweb]

Thanks for the insights Tim.

Yes, I hope others run similar tests. As you suggested, more data points would help!
We could, based on results that are common to the majority or all users, draw some conclusions about relative router performance.
For instance if across a reasonable number of users' tests (we may not get a statistically significant amount for some time at this rate), router A almost always performs best at distant locations on band A and router B usually performs best at both short and long distances on band B e.g., then we can draw meaningful conclusions about each router's relative strengths and weaknesses with a reasonable level of confidence.


My thinking, all other things being equal of course, was that with higher signal strength, a given router stood a better chance to perform better in a given situation with future driver code improvements - i.e. the line of thought was you can only affect performance so much with updated drivers if the signal is relatively weak to begin with.
Of course, at the end of the day for my actual performance comparisons, I will continue to use my LAN Speed Test results, combined with speedtest.net results to judge performance.
Signal strength could be a tie breaker and / or POTENTIAL indicator of the likelihood of possible wireless performance improvement with driver optimization - I may be way off base with my line of thought here so feel free to comment.

I am somewhat interested in better understanding signal strength and may do some research if I get a chance...
I should probably first focus on finishing the booking of an important trip though.

I found / made time to run the tests partially due to the return period nearing and partially due to my usual excitement when something new is introduced to the market...
Hopefully others can run similar WRT-1900AC tests this weekend, either against the Netgear R7000 and Asus AC68U) e.g. or perform standalone testing.
LAN Speed Test (not the Server version which I just read about this morning before work) is very affordable (no longer free) and VERY easy to use.
That may encourage others who don't have a lot of time / want to invest too much effort in router (comparison) testing...

 

[thiggins]

Yes, I hope others run similar tests. As you suggested, more data points would help!

I have considered setting up a database for people to do this, similar to what DSLReports does for ISPs.

But it would be a lot of work, for questionable return. Would also need a standard test tool to try to enforce some level of consistency.

Someday when I get time.....maybe...

 

[RogerSC]

I agree with NJweb (and have told KGB7) that open field testing isn't of much practical use except for vendors to use as marketing tools for buyers who don't know better.

Lastly, don't put too much faith in signal levels as an indicator of performance. LAN based throughput measurements (between a Gigabit Ethernet wired client and wireless client) are always the best thing to go by.

Totally agree...since the ratings and rankings at this site are based on comparing networking equipment using the same tests and the same physical testing points consistently, this is a reference site that I trust for reviews. The tests themselves and testing process are thoroughly explained, which gives the reader confidence that tests are controlled as much as possible, and the only variable is the equipment under test (as much as is possible). The reviews even give the firmware release that was used in testing, so I can calibrate the review relative to what I know about various firmware releases. Using tests in an actual house where people live makes the most sense to me, you get a better idea of what real-world behavior of these devices is.

Thanks for doing all this, it is really appreciated.