Somewhat advanced network question for new home.

[pw40b]

Hi -

The company I work for normally does networks for luxury homes and we always do it with and "industrial strength" kind of design, at a minimum using decent hardware.

However the current project is new territory, just because it's large and has a lot of devices.

There are multiple buildings involved (switches connecting via fiber).

There are 6 VLANs including separate ones for voice (>30 IP phones/intercoms), cameras (>20 1080p IP cameras), control system, guest, etc.

The general design includes 6 Cisco SG200 switches of various sizes all connecting to a SG300 switch acting as the "main" switch which will route between VLANs. The SG300 switch would also connect to the router, a Cisco RV320. Since this network will have a lot of internal traffic, having the layer three switch should keep most of it out of the router. An important point is that the SG300 switch is the only connection point between all the SG200 switches.

So here are my questions:

(1) Is it proper to connect the switches together? If the layer 3 switch goes down, then there will be no traffic between the switches. It seems like if I set up STP correctly then I could connect all the switches together but the ports would be shut down by STP as long as the root bridge (SG300) was up. As soon as that went down, STP would stop blocking the ports connecting the switches. There would be no communication between VLANs, but at least the cameras could all keep recording.

(2) Similar question, should the switches connect to the router in addition to the layer 3 switch? My rationale is that traffic destined for the internet is going to be heading for the router anyway, so why send it through another switch first?

I realize that with the amount of traffic involved I could probably be a lot less careful and it would still work fine, but I would really like to design it properly.

THANKS!

 

[azazel1024]

You'll have to ensure that STP is setup right so that it weights the connection to the SG300 higher than the interlinking of the SG200s.

Same thing if you connect any of the SG200 to the RV320.

I am not particularly familar with the Cisco gear, but I'd go with semi-managed/managed switches that can do VLANs on the edge switch level. Then you wouldn't lose anything if the SG300 went down. Or at least based on your questions it sounds like the SG200 cannot do VLANs.

The last question is, why L3? I hear what you are saying, but unless you are routing across IP blocks or using the L3 as a DHCP, it isn't "saving" any router overhead. If it is just switching at the router to interconnect the switches, that takes no CPU load on the router at all. DHCP is very little overhead other than needing the RAM space to hold a sufficiently large IP table. Of course if the router can't handle VLANs, that would be a reason not to switch traffic through the router.

I guess I am not seeing a reason to need an L3 in this setup. VLANs and enough semi/full managed L2s that support VLANs should be plenty with maybe a bit of redundant linking and RSTP should be all you need to handle everything and have some redundancy in there.

Since you are concerned about that, a beefy UPS is going to be a must. Power outage is a lot more likely than hardware failure. Even if there is a fail over generator for the property, most of those take a second or 5 to kick in. If the cameras are POE powered, make sure that the UPS is sized to handle the load of the PVRs, cameras, switches, router, etc in the network closet. Probably want to size it to at least 15 minutes of run time with the expected load if you have a generator on the property (auto fail-over or not. They don't ALWAYS come up, so you'd want to leave plenty of time for someone to manually go and start the generator). If there isn't, get a generator or install a UPS large enough to handle any expected or possible power outage (which would probably be cheaper to install a generator, even if just for the networking gear. $300-1,000 for a generator of 3-5KVA capacity is a lot cheaper than a UPS sized to run all that gear for several hours to a couple of days).

 

[abailey]

I don't know what your budget is but if your looking for redundancy and L3 routing on the switch you may want to look at the Cisco 500 series, like the SG500's. If you get two SG500's as your core then you can set them up as one switch and run dual connections from each SG200 in a port aggregation setup (one to each 500 series switch). This would give you redundancy on the core switch level and on a port/cable level coming from the SG200's. You also get the benefit of using two 1GB/s lines aggregated throughput to each SG200.
This is similar to how I have our network set up for my company, but we use the Catalyst series.

 

[pw40b]

Thanks for the replies.

AZAZEL1024 - The SG200 switches do VLANs. The reason for the L3 switch as the "core" is just for speed. Is the router-on-a-stick not more of a bottleneck than the L3 switch? If not then fine, that will save a piece of hardware. Thanks for the comments on the UPS, it's fully backed up.

abailey - I like that idea a lot, especially as I sit here looking at the $5 12V wall wart transformer that powers the SG300 - that HAS to be the weak link in this whole system and is kind of disappointing.

I wish there was a 10 port SG500, Seems a shame to have two 28 port switches with each only having 5-6 connections. Also it's a price increase of $1100... I'll have to think about that but it probably makes sense.

 

[htismaqe]

For your application, I would probably stay with the idea of the core L3 switch rather than putting in a router on a stick.

If you had a logical egress point that connected to the router, like a WAN connection or the Internet, that would be one thing. But it sounds like you're just wanting to facilitate inter-VLAN routing...

 

[pw40b]

Thanks - that is the plan. There is a WAN connection but 90% of the traffic will not be going through it.

 

[coxhaus]

Using an L3 core switch is a good idea in my way of thinking. I would assign an IP network to each VLAN. With this type of structure connecting the layer 2 switches together will not work well as you need a layer 3 core for the network to run properly. If you want redundancy run 2 core layer 3 switches.

A couple more things which come to mind is if your going to run redundancy then I would run DHCP on a multi-scoped server like Microsoft's DHCP server and use DHCP relay. If you don't run a redundant DHCP server then I would hard code network, phones & cameras IP addresses so if the DHCP server fails the network would still function. You would lose workstations but the ones needed could be hard coded until the server is fixed.
With redundancy you are going to use a lot more ports than you think so you need to figure it all out before you buy.

 

[thiggins]

I agree with azazel; no need for L3. Any switch in this class will be able to handle gigabit wire speed between all ports.

 

[coxhaus]

I don't like running VLANs without a separate IP network so I want to work at layer3. I would rather have a switch doing the routing rather than a firewall router doing the routing. By the time you add all the access lists required for the network your firewall router is going to be slow or expensive as the consumer routers tend not to do this well so you may have to move to a business class router. It just depends. Since your phones and security cameras are running in this network it needs to run 24/7 and not lock up ever so often.

Redundancy is not required if you want to hold the expense down as it adds a lot to the price tag.

 

[System Error Message]

L3 isnt exactly required but VLANs would definitely be needed. L3 switching is just switches that do L2 switching but can do switching based on some L3 features such as looking at the last IP block instead of MAC and having an IP table or maybe even layer 3 features such as L3 QoS, security, LAN firewall and so on. My mikrotik CRS switch lets me choose between MAC or IP based switching and implement layer 3 features like IP based QoS and IP based filtering. The reason to use IP or layer 3 based switching is that it reduces the latency for switching since it does less steps to switch packets but the latency difference isnt noticeable and it lets you use some layer 3 features on the switch. Routers have a switch chip on them (usually) that perform switching using L2 so there is no issue leaving things connected to the router if you are doing L2 switching.

Switches have come far that redundancy is not required and its easier using a single switch to prevent congestion between switches. The only reason to use different switch is if you want the security system to talk to each other but not rely on the rest of the network for example in which case you'd want a switch that has really good security (this means configuring things to prevent broadcast packets for instance). The main thing to look at when using switches if you want longevity and to prevent rebooting is to use the switches in a cool and dry environment since heat can cause problems on the long term and to use a switch that can handle load 24/7 without crashing when using advance features. Go for the industrial switches if you can for the security while you get away with consumer switches for customer devices. RSTP is a good feature to have if you want to interconnect switches but there are newer variants to that protocol.

It is important to configure it properly. Switches handle switching at wirespeed at L2 and L3 and when looking at rating theres 2 bandwidths. Port capacity and non blocking bandwidth. Non blocking bandwidth needs to be at least half of port capacity for a switch to be rated as wirespeed which is the amount of traffic it can forward. Some switches may have conditions that makes it unable to function at wirespeed if not following certain conditions. If it can switch at wirespeed with 64byte packets that would be even better since it is a measure of the switch cpu performance which is very different from a RISC CPU like ARM or MIPS. Cisco managed switches and mikrotik CRS switches do well but mikrotik switches have a high learning curve (different terminologies). Some mikrotik CRS switches have 10G ports alongside 24 gigabit ethernet ports.

 

[coxhaus]

I agree. The problem with the routers is when you hit them with L3 traffic they fall apart.

I guess I should ask. What are the newer variants to RSTP? I am retired so I have limited resources now to keep up.

 

[System Error Message]

VTP (VLANs), HSRP, VRRP
rapid PVST

Some are cisco only while some relate to routers.

 

[coxhaus]

VTP (VLANs), HSRP, VRRP
rapid PVST

Some are cisco only while some relate to routers.

Thanks. I will check them out.

 

[System Error Message]

A lot of the new STP protocols implement MSTP in them (Multiple STP). It is worth the research if you want to know but it isnt something you would directly see. Usually it is implemented in a bunch of other things from VLANs to different routing protocols.

 

[coxhaus]

Interesting my SG300-28 switch has MSTP support. I don't have it turned on as I know RSTP. It looke like MSTP works at the VLAN level. Normally with RSTP the loops are calculated per port and a port is shutdown if a loop is detected. With MSTP the Spanning Tree is calculated per VLAN not port. I guess you don't have root bridge with MSTP you use maxium hops. I think I will have to look at this more.

 

[Cloud200]

Well, in a case like this you can do it in a two tier setup, core and access.

Get 2 core switches,
Run double the fiber as interconnects.
Connect one uplink to SG300 A
Connect the other to SG300 B
The SG200 units will be acting as access switches.
I don't know if the Cisco RV320 supports STP or not, but if it does, you will set it up so one LAN port is connected to Core A and one is connected to Core B.

The only thing is, NO CLIENTS CONNECT TO THE CORE.
The only things that connect are routers, servers (with multiple NICs to connect one or more to each Core switch) and other switches.

STP will make it so if one of the SG300 goes down, you will still be up.

For true redundancy to ensure you never go down from a single point of failure in your core, you need a router that supports a Warm or Hot spare (warm being active/passive and hot being active/active)

 

[djgizmo]

Yea, what he said ^^


Sent from my iPhone using Tapatalk