How We Test Hardware Routers - Version 4

[thiggins]

Would be interesting to see the results, but I expect NDA's would perhaps prevent disclosure in case anything went wrong with the device during test case/test plan development.

I may have misinterpreted your comment. But I want to make clear SNB has no NDAs in place that prevent us from disclosing the results of any testing we do.

 

[sfx2000]

I may have misinterpreted your comment. But I want to make clear SNB has no NDAs in place that prevent us from disclosing the results of any testing we do.

Sorry - not what I meant - I was suggesting that CDRouter and Microtik may have NDA's in place regarding anything found during development of the test cases/scripts/plan...

This is normal stuff during development and validation.

 

[CDROUTER Jason]

Hey all, we're going to have a webinar on Tim's tests that he will be running against hardware routers using CDRouter next Thursday June 9th. You can join us here:

https://attendee.gotowebinar.com/register/1231467691481698819

 

[sfx2000]

Just signed up - looking forward to it.

(Tim probably is thinking -- oh no, not sfx, lol... - this is very interesting stuff for me, so don't be surprised if you get some good questions)

 

[sfx2000]

I think it's a great thing bringing this platform into the SNB Test Process, many sites that review customer gear do not get into this level of testing.

@thiggins - well done here along with the other test plan improvements.

 

[System Error Message]

When looking at routers and embedded computing there is one thing that we really forgot to review which is how a router compares to one given by an ISP. With so many consumer brands there really need to be more points to look at such as RF design, firmware quality and improvements over something given by an ISP. ASUS for example focuses on gaming. Virgin media in the UK has been aggressive and has released a superhub with the usual consumer router config with AC1900 wifi. Most users unfortunately tend to get satisfied with crappy ISP given routers that they dont know there are many better routers.

While @thiggins has always reviewed routers based on their specs, features and performance there really needs to be a mention on if it is an improvement over an ISP supplied router.

Funny enough it seems that ISPs in the UK are buying the same router i am using.

 

[thiggins]

I could do that if companies made the routers available for review AND if they have Ethernet WAN ports. I can't review routers with DSL, Fiber or Cable modem only WAN.

 

[System Error Message]

I could do that if companies made the routers available for review AND if they have Ethernet WAN ports. I can't review routers with DSL, Fiber or Cable modem only WAN.

You dont have to, you can find screenshots of what features they have (some websites have detailed descriptions of what configs you can do with them) and just compare to what benefits the consumer routers you have been reviewing have over them as part of your usual review. Price is also an interesting factor as ISPs usually factor in the price of the supplied equipment in some way. You can also use physical features as a comparison too.

 

[sfx2000]

I could do that if companies made the routers available for review AND if they have Ethernet WAN ports. I can't review routers with DSL, Fiber or Cable modem only WAN.

It's a fairly vertical niche - Premises Equipment - and even across different operators, there's a fair amount of custom work that is done per the operator's requirements - so testing a device from vendor X doesn't mean that it'll perform the same, or have the same feature set even from Operator C vs. Operator V as an example.

Tribal knowledge suggests that all Premises Equipment is inferior and less flexible that off the shelf -

Problem is that tribal knowledge is sometimes out of date - the latest crop of CPE I've seen from the operators is much better than what we saw even 18 months ago... I think much of this is due to a number of factors

1) technology pace - we're seeing AC1900 class CPE finally in the the sector, while it's been on store shelves for much longer - it's the requirements development and testing that takes the long time
2) services - the operators are rolling more and more over IP, so the CPE has to be more robust in the first place
3) Customer Demand - plainly put, customers are asking for more, and using it more

While it would be interesting to see something from a provider on the test bench, I'm thinking not much value if it could be tested, because it would only be representative for that particular operators configuration.

 

[System Error Message]

We dont have to compare all the features, for example when it comes to modems, how do you compare them properly? So just compare the features that relate to routers, stuff like QoS, firewall, parental, whether it has a DSL modem or a 1 MW wifi amplifier would be irrelevant in comparison (though you could still mention it).

Comparing hardware specs (like CPU, chips, RF design) is helpful. You can find the specs on wikidevi as well and other sites.

 

[sfx2000]

I can't speak for the direction - but what I can see is that we're in a period of innovation, and there's a lot of consumer gear out there that folks want to know about...

Perhaps Netgear could do a brief writeup, as they've had great success on the store shelves as well as an increasing amount of the Operator/Carrier CPE market.