New Router Test Processes Are Coming

[thiggins]

Our new wired and wireless test processes will include tougher tests and more of them.

Read on SmallNetBuilder

 

[sfx2000]

Great news..

On the wireless side, the big news is we're retiring our trusty NETGEAR R7000 AC1900 test client. Although the exact new standard STA hasn't been selected yet, I can say it will be a 2x2 device. The reason is to more accurately represent the lower throughput users see from actual devices, most of which aren't 3x3.

The industry has pretty much settled on the 2*2:2 (and single stream on the low end) due to cost/performance on the client side.

 

[L&LD]

Tim, ty!

I'll be looking forward to all the 'retests' in the next few months.

It would be great if the retesting including the latest official firmware of old, but currently selling models like the RT-N66U, for example.

In any case, thank you once again for all your great efforts on the main site and the forums too.

 

[thiggins]

It would be great if the retesting including the latest official firmware of old, but currently selling models like the RT-N66U, for example.

Retests will be with latest released firmware.

 

[RMerlin]

If I can make a suggestion: replace the Startech HDD dock with a closed USB enclosure for USB disk sharing performance tests. An open dock is pretty much a worst-case scenario when it comes to EMF, and is most likely skewing results toward the worst. Might be a good time to replace the HDD with an entry level SSD as well, as I expect routers will start to move beyond single gigabit performance in the very near future.

 

[thiggins]

If I can make a suggestion: replace the Startech HDD dock with a closed USB enclosure for USB disk sharing performance tests. An open dock is pretty much a worst-case scenario when it comes to EMF, and is most likely skewing results toward the worst. Might be a good time to replace the HDD with an entry level SSD as well, as I expect routers will start to move beyond single gigabit performance in the very near future.

So you think the Router Storage test results are too low?

The key thing the dock provides is eSATA. That's needed for NAS testing and was needed for the new Linksys WRTs...

 

[L&LD]

So you think the Router Storage test results are too low?

The key thing the dock provides is eSATA. That's needed for NAS testing and was needed for the new Linksys WRTs...

eSATA is great when it was semi-popular in the late '90's. Besides the WRTs that has it now, what needs it besides old product that will be eol soon enough?

 

[thiggins]

So if router has an eSATA port you guys are good with me not testing performance through it?
eSATA is still common on new NASes.

 

[sfx2000]

I haven't heard of many negative concerns with the StarTech Dock - I think as long as it's outside of the enclosure (shield box) it'll be fine...

Having eSATA capabilities - WRT's were one of the few router devices - but it's good to keep around for NAS testing.

I agree with RMerlin on the SSD aspect - the recent crop of Phison based TLC drives are at a very attractive pricepoint, and on the good side of the price curve for capacity...

 

[RMerlin]

So you think the Router Storage test results are too low?

I've seen a few tests where my own results were much higher than yours, and that's despite the fact I was using a WD Caviar Green (5800 rpm inside a Vantec USB3.0 enclosure, versus your 10K RPM Velociraptor). Beside the RT-AC68U, I think my test results were also quite higher with the RT-AC88U.

(I never could test the Linksys WRT1900AC, it failed to recognize both USB enclosures I tested with it, and they didn't provide any system log for me to troubleshoot it.)

The key thing the dock provides is eSATA. That's needed for NAS testing and was needed for the new Linksys WRTs...

I checked, and unfortunately Vantec (the brand I personally prefer) seem to have been phasing eSata out, as it's only available on their USB 2.0 enclosures. Maybe another brand might still offer a USB 3.0 + eSata combo. However while I agree with sfx that eSata is being phased out in general (I rarely see it available on the motherboards I use for my customer's PCs), I guess that for the very rare case where you might encounter a router with an eSata port, the Startech dock might do the job. Dropping any eSata tests would make sense IMHO, however I'm sure you'll get people asking you why you didn't test it.

 

[RMerlin]

I just noticed that while they no longer offer eSata in the CX series, their NexStar 3 series still offers USB 3.0 + eSata capabilities:

http://www.vantecusa.com/en/product/view_detail/479

 

[Razor512]

I love having eSATA support on a router. Due to the lower cost, I have a bunch of USB 2/ eSATA enclosures and socks, mainly for cold backups, and slightly higher speeds on some routers which support it.

Hopefully we will begin to see smarphones move to 3 stream WiFi adapters, and laptops move to 4 stream WiFi adapters.

 

[L&LD]

So if router has an eSATA port you guys are good with me not testing performance through it?
eSATA is still common on new NASes.

Not even on the radar of any of my customers with router or NAS needs. USB 3.1 enclosures have greatly overshadowed eSATA's once great performance advantage (for backup use).

 

[thiggins]

Ok, guys, got the point on storage. Any other comments on the other aspects of the preview? No griping about lack of IPv6 testing?

 

[L&LD]

Ok, guys, got the point on storage. Any other comments on the other aspects of the preview? No griping about lack of IPv6 testing?

IPv6; we'll need to go there sooner than later. But testing today's mess is just a make work project, imo.

 

[sfx2000]

Ok, guys, got the point on storage. Any other comments on the other aspects of the preview? No griping about lack of IPv6 testing?

regarding IPv6 - always figured that would be an item as a 'point release' of the test process once that's sorted by the different test equipment vendors...

 

[thiggins]

regarding IPv6 - always figured that would be an item as a 'point release' of the test process once that's sorted by the different test equipment vendors...

There aren't really test process "point releases". Any process change resets the Charts and Rankers. I try to not do that often because then products can't be fairly compared and ranked.

I still don't see IPv6 as a big issue in the U.S. If it were, router makers would have better and more consistent IPv6 feature sets.

 

[sfx2000]

I still don't see IPv6 as a big issue in the U.S. If it were, router makers would have better and more consistent IPv6 feature sets.

The IPv6 rollout in the US has been a bit "odd" - there are many options, and some of the big cable companies are still working thru the details - gotta be tough on the Router/AP vendors chasing a moving target...

 

[RMerlin]

Ok, guys, got the point on storage. Any other comments on the other aspects of the preview? No griping about lack of IPv6 testing?

IPv6 is more about functionality than performance at this stage IMHO. Since your reviews tend to focus on performance, lack of IPv6 testing isn't an issue. Just mentionning what type of IPv6 support is present while writing a router review should be enough for now (i.e. report if it supports DHCP-PD, 6in4 tunneling, if it offers a configurable IPv6 firewall, etc...)

 

[thiggins]

Router reviews have focused mainly on performance. I'm now adding hundreds of functional checks that will be run on every router tested.

I'd love to also check how well (or whether) IPv6 functions. But there is no equivalent of the standard DHCP WAN configuration all routers support for IPv4 that is consistently supported for IPv6.

Having an IPv6 DHCP WAN type is necessary but not sufficient. How the LAN handles addressing is just as important and that's where things get messy...