Looking for Wireless Stress Test Cases

[thiggins]

Thanks bman. Do you try to do these things simultaneously?

 

[bman212121]

Thanks bman. Do you try to do these things simultaneously?

Lol nope, it's barely possible to do them by themselves. I have HT20 in 2.4ghz so if you want to do an import of photos its a lot faster to plug in temporarily. The streams do work at full strength with one client, but since they are not being transcoded on the fly like Netflix would there is no backing off on the bitrate. You need to have approx. 20000kbps of bandwidth at all times to watch an HD stream. Just walk across the house to a low signal area and you'll lose tv quickly. I'm hoping to upgrade to AC and then maybe this scenario would be possible to do at the same time.

 

[zviro]

*, 1x1 ac phone, facebook, mail, web
*, 1x1 ac phone, facebook, youtube, radio, web
*, 1x1 n phone, facebook, youtube, web
, 1x1 n phone, youtube, web
*, 2x2 n laptop, youtube, web
*, 3x3:2 n computer, youtube/twitch, torrent, gaming, skype/teamspeak

i hope i wrote that correctly.
basically we're 3 flatmates with a computer and phone each, i have a secondary phone i sometimes use as a mini tablet and for testing android roms.
there's also a computer and a moca adapter for the TV(vod) connected via wire.
by web i mean general web browsing - news, forums, 9gag, comic reading and the like.

 

[System Error Message]

but what about pushing wifi APs to their limits?

 

[avtella]

* 2x2 AC Laptop: YouTube, Web, Gaming, Radio, Hangouts, Netflix, Email
* 2x2 AC Laptop: Video Conferencing, Database usage, Email
2x2 N Laptop: Web, Email,
* 2x2 N Laptop: YouTube, Web, Skype, Email
* 2x2 N Laptop: YouTube, Web, Facebook, Skype, Netflix
* 1x1 AC Phone: YouTube, Web, Email, WiFi Calling
* 1x1 AC Phone: YouTube, Web, Email, WiFi Calling
1x1 AC Phone: YouTube, Web, Email, WiFi Calling
1x1 AC Phone: YouTube, Web, Email, WiFi Calling
* 2x2 N Tablet: YouTube, Web, Email
1x1 N Tablet: YouTube, Web, Email
* 1x1 N VoIP: Calls
* 1x1 N Streamer: YouTube, Netflix
* 1x1 N Streamer: YouTube, Netflix
2x2 N Streamer: YouTube, Netflix
* 4x4 AC Router: 5Ghz bridge for NAS (backups/sync/VHDs for VirtualBox)
* 1x1 G Speaker: Music, Audio for videos
1x1 N Watch: Google Fit
1x1 N Thermostat
1x1 N Printer/Scanner
1x1 N Security Camera (motion mode)
1x1 N Security Camera (motion mode)

4 users at home with 4 phones, 2 tablets, 4 personal laptops and 1 work laptop.

 

[thiggins]

* 2x2 AC Laptop: YouTube, Web, Gaming, Radio, Hangouts, Netflix, Email
* 2x2 AC Laptop: Video Conferencing, Database usage, Email
2x2 N Laptop: Web, Email,
* 2x2 N Laptop: YouTube, Web, Skype, Email
* 2x2 N Laptop: YouTube, Web, Facebook, Skype, Netflix
* 1x1 AC Phone: YouTube, Web, Email, WiFi Calling
<SNIP>

Thanks for your submission.
With four users, are 12 devices really in simultaneous use?

 

[thiggins]

but what about pushing wifi APs to their limits?

If you have a specific test case, please submit it.

 

[avtella]

Thanks for your submission.
With four users, are 12 devices really in simultaneous use?

Yes surprisingly, let me make it a bit more clear:
2 Chromecasts linked to two laptops for video so 4 devices there.
2 more laptops either surfing or on YouTube/Netflix with one of them occasionally using a WiFi speaker for audio out, so we are at 7 devices
1 work laptop always running database operations and other work stuff even when unattended, so now we are at 8 devices
1 Ooma VoIP device connected to our phones, in use a lot due to work reasons so at 9 devices now
All four of us use either our phones or the two tablets intermittently even when playing videos, to check mail, news, sports scores etc. One family member is always on Facebook on the phone lol. so by now 10-11 devices at peak.
I suppose even up to 12 if the NAS which is connected to a seperate AC Router (5Ghz bridge mode) is included

EDIT: It seems I forgot to mention the above was for the worst case scenario not regular usage.

Active devices in regular usage:
* 2x2 AC Laptop: YouTube, Web, Gaming, Radio, Hangouts, Netflix, Email
* 2x2 AC Laptop: Video Conferencing, Database usage, Email
* 2x2 N Laptop: YouTube, Web, Facebook, Skype, Netflix
* 1x1 AC Phone: YouTube, Web, Email, WiFi Calling
* 1x1 AC Phone: YouTube, Web, Email, WiFi Calling
* 2x2 N Tablet: YouTube, Web, Email

Oh and by the way I have been lurking around your forums for 2-3 years, your site was what got me more interested in network hardware and router testing, learned a lot from you guys, more than my high school Cisco class which a long time back lol, so I would like to say thanks.

 

[thiggins]

@avtella: thanks for the additional details. You guys need some time away from screens!

So what are you doing to handle all those devices?

 

[System Error Message]

If you have a specific test cas.e, please submit it.

1 use case would be SMB/cluster sync file transfers(such as dropbox/aerofs) + p2p file transfers (such as torrents/ game updates) + fps gaming + streaming + web browsing simultaneously. I would love to include compute clusters but i dont think thiggins would be able to set that up having both wire and wireless LAN fully stressed at the same time on the same wifi router. I think this would cover a broad usage spectrum for QoS.

Other than that, using bandwidth and latency testers to see the performance of the wifi chips even with range as well since different wifi chips perform differently. (maximum achievable bandwidth and packet loss when at that maximum bandwidth instead of the usual tests performed by SNB). its not supposed to be a practical test but it is to see how good the wifi implementation is in the AP.

 

[sfx2000]

Actually that streaming audio is exactly that use case. Airplay from Apple "radio" on iPhone 6 to access point and then over the wire to the airport extreme.

I'd add Airplay screen mirroring, but that is an almost never occasion (2, 3 times a year? to my Apple TV).

Should note the Peer-to-Peer Airplay can come into play depending on the HW/SW... even in Infrastructure Mode, peer-to-peer will be preferred if the stars line up...

• Airplay Receiver - Audio/Video/Screen Mirroring
  
  • Apple TV (3rd generation Rev A or later) running Apple TV software 6.1 or later
• iDevices running OS8 or later acting as sender
  
  • iPhone 4s or later
  • iPad Air 1st generation or later
  • iPad 3rd generation or later
  • iPad mini 1st generation or later
  • iPod touch 5th generation or later
• Macs running Mac OS X 10.10 or later acting as sender
  
  • Retina Macbook Pros 2013 or later
  • MacBook Air 2012 or later
  • iMac 2012 or later
  • MacBook (Retina) or later

Peer to Peer Airplay uses BTLE for discovery, and will use either 2.4GHz CH6 or 5GHz CH149+1 for peer to peer mode...

 

[Razor512]

My current setup with my Netgear R7000 is

* 1x1 N Tablet: Youtube, Verizon fios streaming (live TV), and using either BS player or Dice player to stream 1080p video from my NAS.

* 1x1 N Tablet: Verizon fios live TV streaming, 1080p media streaming from NAS

*1x1 N (cheap 10 inch) Tablet: Constant streaming from either the NAS or youtube, or Amazon prime (when they offer free trials) (used more as a kitchen TV and will stream non stop on the 2.4GHz band as well as to a speaker system with a bluetooth receiver connected)

* 1x1 N IP camera: (constant 20-30mbit/s stream to NVR (rest of cameras are wired)(Edit: depending on motion, and using tomato, it really does around 10-19mbit/s, higher if I just randomly shake the camera)

* 1x1 AC Smartphone: Youtube, Security camera streaming (constant 6-8mbit/s), bittorrent sync (~40 mbit/s when backing the phone up to the NAS)

*1x1 G (ancient) Smartphone: universal remote app, and occasional app updates

* 1x1 G Wireless printer/ scanner: (used daily for printing and scanning stacks of paperwork via the ADF, 20mbit/s while scanning)

* 2x2 AC Laptop: Media streaming from NAS, youtube, daily scheduled backups to NAS (300+mbit/s sustained during backup)

* 2x2 N Laptop: Security camera monitoring (~50mbit/s if viewing all cameras at the same time), Windows backup (daily)

All wireless devices except the low end tablet, printer, and ancient 802.11g smartphone are on the 2.4GHz band. (I have largely moved to 5 GHZ, especially since the 2.4GHz band is somewhat crowded and suffers a sharper drop in throughput over distance compared to the 5GHz band. http://i.imgur.com/meTZtwY.jpg

Router in the living room (first floor)

2x2 dual band N600 client (laptop in the basement, corner of the house)
2.4 GHz: http://i.imgur.com/dhATdOM.jpg
5 GHz: http://i.imgur.com/BE8nOts.jpg

Everything else is wired, and everything is rather easy to manage, especially since many of the devices serve a limited set of functions, e.g., a dirt cheap 10 inch tablet (with IPS display) makes a decent kitchen TV since it can be placed in a water tight bag and then placed in many different locations and connected to a bluetooth speakers and a waterproof keyboard.

 

[thiggins]

Thanks, Razor.

So all the devices you listed are connected to the R7000?
How well does the R7000 handle the load? What interactions do you notice?
For example, does the "kitchen TV" get affected when you are scanning with the
11G printer?

 

[sfx2000]

@Razor512 - if you have the 11n Laptop monitoring the security cameras, how does that affect the 11n Tablet in the Kitchen watching 1080P stream while BitTorrent Sync is running... all concurrent...

 

[Razor512]

For the load, it handles it pretty well, on the 2.4GHz band, the printer/ scanner doesn't seem to cause any issues, though when it scans, it sends data in chunks and when scanning multiple pages using the ADF, though it is fairly consistent.


To better load it up, I had it scan while also rapidly refreshing the web page that the printer creates. For my low end tablet which has a slow WiFi radio to begin with, it does not suffer too much of a performance hit, streaming 1080p 10 bit h.264 content does not suffer any ill effects.

I then tried benchmarking it while scanning and constantly loading stuff from the little webpage that the printer makes, and as expected based on the streaming (using Dice Player to stream from the NAS). speeds to not really drop low and stay low long enough to overrun the buffer.

In the picture, I start scanning and having a macro reload (Ctrl + F5 on firefox) the printer page rapidly. http://i.imgur.com/zy2Kg6W.jpg


On the 5GHz side, there are some noticeable slowdowns when which are mainly noticed in the form of slower backups to the NAS, but for general use, things work well. Video streaming is smooth and responsive the laptops simply open the media from a network share, and play in VLC.

Overall, the only annoyance is windows backups taking longer with multiple devices are using the 5GHz band (though I think going with a R8000 may partially fix that issue).
I was also interested in the R7500 (not really to buy one since it is hard to justify an upgrade so soon), but it is hard to tell if it will do a better job at handling many devices, as well as being ready for when we can finally have some 4 stream 802.11 ac client devices.


Edit: I will work on testing more devices at the same time. since it is only 3 of us in the house, we will not really have all of them actively doing bandwidth intensive stuff.

Most of the more automated stuff, happens for a relatively short while, e.g., if I backup data from my phone using bittorrent sync (mainly photos, videos, audio recordings), will happen for a few minutes when on the home network).

When monitoring the security cameras, I mainly do it when I am expecting a package delivery, or if there are jerks making noise outside and I want to be aware if they start causing more serious issues. Overall, it does a good job at balancing the load for everything.

 

[thiggins]

Thanks for the details, Razor. Interesting you have access to IxChariot.

You are not going to see 4x4 client devices other than routers that can be put into bridge modes and maybe a few 4x4 dedicated bridges. Not large enough market. Same reason you are seeing the USB WiFi adapter die off.

I'm working on new test capability that will enable serious load testing for SU and MU devices. The work done so far with MU devices has been interesting. Manfs have a lot of work to do...

 

[Razor512]

A while back, I was thinking about just bringing a high end router to class (college has a gigabit internet connection), and then asking if everyone could help stress test it, mainly to see how it would handle a large number of users as compared to the insanely overpriced cisco access points. I never really got around to it, and didn't think it would work unless I could beyond a reasonable doubt prove that no malicious man in the middle activity was taking place. It would have been cool to see a consumer router handle 50+ active wireless clients.

 

[sfx2000]

It would have been cool to see a consumer router handle 50+ active wireless clients.

I've seen a Linksys WRT1900acV1 take on 64 clients (across both radios) and live to tell about it, so my guess is most AC1900 class devices should be capable of at least 50 in real world use..

 

[thiggins]

I've seen a Linksys WRT1900acV1 take on 64 clients (across both radios) and live to tell about it, so my guess is most AC1900 class devices should be capable of at least 50 in real world use..

Do tell. What sort of traffic were those STAs presenting?

 

[sinshiva]

I may have missed it, but i'd like to see is how your wifi handles or recovers from neighbors streaming or torrenting over wifi with overlapping and nonoverlapping channels, how that affects your low latency apps and so on. I live in an apartment, so obviously one that handles these situations better than others would appeal to somebody like myself

I think ideally, you could duplicate a stress test setup and run one then both simultaneously, etc.