Network hardware and config for 100 people LAN

[Smartnolf]

Hi guys!

So, me and a couple of friends are thinking about arranging a LAN for about 100 people and I have some questions regarding the setup of the network.

The current idea is that we are going to use a left-over computer running an AMD Phenom II X4 965 at 3.4GHz and 8GB RAM.
This will be complimented by a Intel I210-T1 for the WAN connection and Intel I350-T4 for 4xCisco SF300-24 switches for the LAN part.
The integrated NIC on the motherboard will handle an accesspoint for WiFi.

The thing is that we want to QoS the ports that are corresponding to a list of games and the SF300 series are capable of that, but so are the Intel NICs. Would it be better to use SF100 series (which are a lot cheaper) and handle all the QoS with the I350-T4?

The WAN connection is a solid 250/100 Mbit/s.

See the attached document for a chart of the network.

 

[abailey]

Do all the clients on a switch have the same priority? If so you could go with the less expensive switches and use the computer to QOS between the different switches. It also depends on what QOS or bandwidth shaping you are trying to do.

 

[Smartnolf]

Yes, all the clients have the same priority, but different traffic should be prioritized by their ports. My understanding is that QoS can be handled using ports so to speak.
So for example ports for games such as WoW and CS can be prioritized over all other traffic.

The question is if it would be best to do with the switches and then the NIC, or only on the NIC?

 

[abailey]

Yes, all the clients have the same priority, but different traffic should be prioritized by their ports. My understanding is that QoS can be handled using ports so to speak.
So for example ports for games such as WoW and CS can be prioritized over all other traffic.

The question is if it would be best to do with the switches and then the NIC, or only on the NIC?

If all clients on one switch played the same game, you could just do it at the router level. If your switch will have a mixture of game players on it then you will need to do QOS on the switch and the router.

 

[Smartnolf]

If all clients on one switch played the same game, you could just do it at the router level. If your switch will have a mixture of game players on it then you will need to do QOS on the switch and the router.

Yea I basically want our list of games to be prioritized over all other traffic, as I expect there will be a lot of torrenting and YouTube for example. As I don't want our Internet connection to get slow.

What do you think about the general setup though? I had some thoughts about going gigabit, but that increased the price by quite a big margin as we want to keep ourselves in the semi-enterprise area of hardware.

 

[Dreamslacker]

Yea I basically want our list of games to be prioritized over all other traffic, as I expect there will be a lot of torrenting and YouTube for example. As I don't want our Internet connection to get slow.

What do you think about the general setup though? I had some thoughts about going gigabit, but that increased the price by quite a big margin as we want to keep ourselves in the semi-enterprise area of hardware.

There's no need to have 4 interfaces on pfSense go to the 4 individual switches.

In fact, I highly do not recommend it because pfSense will not bridge the ports at wire speed.

Daisy-chain the switches via the GbE ports and set one uplink port to pfSense as LAN.

For QOS, just use what's in pfSense. Ultimately, you only need to be concerned with QoS through the WAN link. Use HFSC or PRIQ (easier to configure) and prioritize the games based on destination ports.

You can use a combination of Layer7 HTTP videos and Google ASN IPs (use firewall alias to group them) to catch Youtube video traffic and limit them accordingly.

Example:

• Create an alias containing all Google IP ranges.
• Create a Layer7 container for HTTP Video type Queue.
• Create 2 limiters (1 for upload, 1 for download). Set the bandwidth accordingly. E.g. 10Mb/s for upload, 100Mb/s for download.
• Create a child limiter with host mask for the upload limiter. Ditto for download but with destination mask instead.
• Create a Firewall rule on LAN with Destination Port HTTP, Destination network Google Alias.
• Set the Queue to the lower priority queue.
• Set the Layer7 to the container created above.
• Set the Limiter In/ Out to: Upload Child/ Download Child.

This will allow you to limit all total outgoing traffic for Youtube to 10Mb/s and shared equally amongst clients. All download traffic for Youtube streaming is limited to total 100Mb/s shared equally amongst clients.

Shaping for Torrent is tricky because of encryption. The only real way to do this is to explicitly prioritize all critical applications/ protocols (DNS, NTP, PING etc.), and games. Then you follow with default priority for all other traffic that is limited in bandwidth and does not have explicit realtime bandwidth/ delay commitment.

 

[azazel1024]

There's no need to have 4 interfaces on pfSense go to the 4 individual switches.

In fact, I highly do not recommend it because pfSense will not bridge the ports at wire speed.

Daisy-chain the switches via the GbE ports and set one uplink port to pfSense as LAN.

For QOS, just use what's in pfSense. Ultimately, you only need to be concerned with QoS through the WAN link. Use HFSC or PRIQ (easier to configure) and prioritize the games based on destination ports.

You can use a combination of Layer7 HTTP videos and Google ASN IPs (use firewall alias to group them) to catch Youtube video traffic and limit them accordingly.

Example:

• Create an alias containing all Google IP ranges.
• Create a Layer7 container for HTTP Video type Queue.
• Create 2 limiters (1 for upload, 1 for download). Set the bandwidth accordingly. E.g. 10Mb/s for upload, 100Mb/s for download.
• Create a child limiter with host mask for the upload limiter. Ditto for download but with destination mask instead.
• Create a Firewall rule on LAN with Destination Port HTTP, Destination network Google Alias.
• Set the Queue to the lower priority queue.
• Set the Layer7 to the container created above.
• Set the Limiter In/ Out to: Upload Child/ Download Child.

This will allow you to limit all total outgoing traffic for Youtube to 10Mb/s and shared equally amongst clients. All download traffic for Youtube streaming is limited to total 100Mb/s shared equally amongst clients.

Shaping for Torrent is tricky because of encryption. The only real way to do this is to explicitly prioritize all critical applications/ protocols (DNS, NTP, PING etc.), and games. Then you follow with default priority for all other traffic that is limited in bandwidth and does not have explicit realtime bandwidth/ delay commitment.

Exactly.

What you are talking about doesn't require QoS at the switch level. Switch level QoS is generally only needed for applications where the port is going to be near capacity and/or VoiP applications over the LAN where jitter is extremely important (and honestly, even then, you are still only talking about situations where the port might be close to load).

NOW, if you are going to have any internal servers that LAN users will be accessing, port based QoS on the switches comes back in to being important.

The issue here is that L2 switches (most/all) only support QoS based on L2 data (IE you can generally prioritize by MAC). You can't generally set QoS based on IP port being accessed unless you are running an L3 switch. Even in the later case, often times you are stuck with very basic QoS features and nothing in the way of real bandwidth shaping based on application in question (though again, both can generally do bandwidth shaping per switch port/MAC).

You are really going to need/want to do the QoS at the level of the router. For internal traffic, do not use multiple inputs to the router, as mentioned, that machine/pfsense will NOT be able to bridge at port speed between the switches, which might be important if you do have internal traffic going on. Do everything off the network switches.

 

[Smartnolf]

There's no need to have 4 interfaces on pfSense go to the 4 individual switches.

In fact, I highly do not recommend it because pfSense will not bridge the ports at wire speed.

Daisy-chain the switches via the GbE ports and set one uplink port to pfSense as LAN.

Interesting! How come they won't accomplish that (or at least near that)?

The idea that I had was that in future places we might want to/have to separate the different switches into different rooms or areas. I was also thinking that I might want separate QoS settings and the like.

For QOS, just use what's in pfSense. Ultimately, you only need to be concerned with QoS through the WAN link. Use HFSC or PRIQ (easier to configure) and prioritize the games based on destination ports.

You can use a combination of Layer7 HTTP videos and Google ASN IPs (use firewall alias to group them) to catch Youtube video traffic and limit them accordingly.

Example:

• Create an alias containing all Google IP ranges.
• Create a Layer7 container for HTTP Video type Queue.
• Create 2 limiters (1 for upload, 1 for download). Set the bandwidth accordingly. E.g. 10Mb/s for upload, 100Mb/s for download.
• Create a child limiter with host mask for the upload limiter. Ditto for download but with destination mask instead.
• Create a Firewall rule on LAN with Destination Port HTTP, Destination network Google Alias.
• Set the Queue to the lower priority queue.
• Set the Layer7 to the container created above.
• Set the Limiter In/ Out to: Upload Child/ Download Child.

This will allow you to limit all total outgoing traffic for Youtube to 10Mb/s and shared equally amongst clients. All download traffic for Youtube streaming is limited to total 100Mb/s shared equally amongst clients.

Shaping for Torrent is tricky because of encryption. The only real way to do this is to explicitly prioritize all critical applications/ protocols (DNS, NTP, PING etc.), and games. Then you follow with default priority for all other traffic that is limited in bandwidth and does not have explicit realtime bandwidth/ delay commitment.

I appreciate the elaborate answer, but I was actually thinking that I only need to prioritize the games. The rest is non-essential for the LAN imho.

EDIT: Possibly VoIP will get a priority aswell

 

[Smartnolf]

Exactly.

What you are talking about doesn't require QoS at the switch level. Switch level QoS is generally only needed for applications where the port is going to be near capacity and/or VoiP applications over the LAN where jitter is extremely important (and honestly, even then, you are still only talking about situations where the port might be close to load).

NOW, if you are going to have any internal servers that LAN users will be accessing, port based QoS on the switches comes back in to being important.

The issue here is that L2 switches (most/all) only support QoS based on L2 data (IE you can generally prioritize by MAC). You can't generally set QoS based on IP port being accessed unless you are running an L3 switch. Even in the later case, often times you are stuck with very basic QoS features and nothing in the way of real bandwidth shaping based on application in question (though again, both can generally do bandwidth shaping per switch port/MAC).

You are really going to need/want to do the QoS at the level of the router. For internal traffic, do not use multiple inputs to the router, as mentioned, that machine/pfsense will NOT be able to bridge at port speed between the switches, which might be important if you do have internal traffic going on. Do everything off the network switches.

So if I would do it like this:
Single port Intel NIC to Cisco SG100D-05 switch from router.
This switch will go out to each other switch. That would be preferable?

 

[azazel1024]

You are still going to need to prioritize in the router, or if an L3 switch...if it can do real prioritization by IP port.

As for why it won't, it has to do software switching and that machine you are talking about is probably not capable of switching 2Gbps between the various ports on the NIC. Deffinitely cannot handle 8Gbps of traffic for full duplex on all ports if the switches had a lot of cross switch traffic.

Switching is done in a dedicated ASIC normally, so it can do it at full port speed across the entire fabric (well, if the fabric is capable of it), but software based switching (which is what happens with a bridged connection) in a general purpose computer takes significant amounts more overhead, which is why a little 2.5w 5 port switch can handle a 10Gbps switching fabric, but a 100w full computer can't.

You can always dedicate a little 5 or 8 port semi-managed switch or something as the core that you plug each of the bigger switches in to and then plug that switch in to the PFsense box.

 

[azazel1024]

So if I would do it like this:
Single port Intel NIC to Cisco SG100D-05 switch from router.
This switch will go out to each other switch. That would be preferable?

Yes. I would do it that way. I have a preference for things that I can manage though, so I'd look at a Dlink DGS-1100-05 personally (I have one, and it is pretty nice, that not overawing, but a super low power, tiny semi-managed 5 port switch, which is exactly what I need for my entertainment unit as I have 3 IP devices plus the LAN drop to connect and it is the one place in the house where I can't do additional runs and didn't think to do more than 1 run when I wired it up months ago).

 

[Smartnolf]

I see! This makes sense, and I feel quite stupid for not thinking of that myself tbh haha.

But the QoS in PfSense should be able to handle QoS for the now single port to WAN? Sorry if I am asking basic questions about this, but haven't arranged anything this big before.

 

[abailey]

As far as your switches go and your pfSense nics, you need to plan accordingly if you want to take advantage of the 250Mbit download on your internet. I have not used pfSense but if you are using 100Mbit switches I would hope pfSense can switch near wire speed for that (even though it will not be hardware accelerated). Thus you could benefit from running 4 cards in the router. Personally I would uplink the switches to one switch and make sure that they have a couple of Gigabit ports to uplink with and then one attach to your router. If your looking at Cisco, something like this would work: http://www.newegg.com/Product/Product.aspx?Item=N82E16833150150
It will be much easier to just uplink to the router with a single port.
EDIT: Actually you would need one that has 4 Gigabit ports so you could uplink 3 switches and then uplink to the router.

 

[Smartnolf]

Created a fast chart of the network that is actually quite a bit cheaper than the other one

Is this something like what you would recommend?

EDIT: Possibly just run SF102-24 switches instead of the SF300-24 ?

 

[Smartnolf]

Yes. I would do it that way. I have a preference for things that I can manage though, so I'd look at a Dlink DGS-1100-05 personally (I have one, and it is pretty nice, that not overawing, but a super low power, tiny semi-managed 5 port switch, which is exactly what I need for my entertainment unit as I have 3 IP devices plus the LAN drop to connect and it is the one place in the house where I can't do additional runs and didn't think to do more than 1 run when I wired it up months ago).

Why would I want this manageable if I want to do all the QoS in PfSense? (I realize this looks aggressive but I assure you it's not )

 

[abailey]

Why would I want this manageable if I want to do all the QoS in PfSense? (I realize this looks aggressive but I assure you it's not )

Manageable switches can do many things for you. First it helps future proof your purchase if you ever need to run VLAN's, port aggregation, QOS, etc. Also they are very handy when you are trying to run down a problem or bottleneck. Most can also detect things like a bad cable. Many also have graphs and port stats so you can monitor traffic.
I would also advise you to use managed switches. If you want to save money you can use a less expensive brand like TP-Link (I really like their switches).
You could use 4 of these: http://www.amazon.com/dp/B00BMJEMFW/?tag=snbforums-20
and one of these: http://www.amazon.com/dp/B00KT30D0A/?tag=snbforums-20
Its up to you though. Your last pic looks okay as far as the setup. I would, though, use Cisco series 100's for the clients if your going to use a series 100 for the Gigabit uplink.

 

[Smartnolf]

Manageable switches can do many things for you. First it helps future proof your purchase if you ever need to run VLAN's, port aggregation, QOS, etc. Also they are very handy when you are trying to run down a problem or bottleneck. Most can also detect things like a bad cable. Many also have graphs and port stats so you can monitor traffic.
I would also advise you to use managed switches. If you want to save money you can use a less expensive brand like TP-Link (I really like their switches).
You could use 4 of these: http://www.amazon.com/dp/B00BMJEMFW/?tag=snbforums-20
and one of these: http://www.amazon.com/dp/B00KT30D0A/?tag=snbforums-20
Its up to you though. Your last pic looks okay as far as the setup. I would, though, use Cisco series 100's for the clients if your going to use a series 100 for the Gigabit uplink.

Ah I see! I can see how that could be useful, but I don't think we will be playing around with that in the foreseeable future. As of right now we just want a simple LAN for 100 people that can be in separate areas if need be, and QoS for the popular games.

Yea I saw that after I posted the pic haha, SF102-24 switches were intended ^^

EDIT: Would an AMD 1035T (6 cores) at 2.6GHz be better suited for the PfSense task?

 

[Dreamslacker]

But the QoS in PfSense should be able to handle QoS for the now single port to WAN? Sorry if I am asking basic questions about this, but haven't arranged anything this big before.

Yes, it can - provided you configure traffic shaping properly. There's no magic button to do this and the Wizard only takes you that far.

I've ran 40+ computers in a cybercafe using pfSense on a 10m/10m line (and a 4m/ 768k DSL before that) without any lag issues so you should have no problems with that big fat pipe you are getting.
And this was on a much older version of pfSense without the limiter/ layer7 capabilities you get these days.

Again, it's all about getting your traffic shaping right.

You do not actually need an additional switch - your SF300-24 has 4 gigabit ports each. You can actually trunk 2 GbE ports from each switch to the next and simply connect pfSense with 1 GbE link to any of the switches.

 

[Smartnolf]

Yes, it can - provided you configure traffic shaping properly. There's no magic button to do this and the Wizard only takes you that far.

I've ran 40+ computers in a cybercafe using pfSense on a 10m/10m line (and a 4m/ 768k DSL before that) without any lag issues so you should have no problems with that big fat pipe you are getting.
And this was on a much older version of pfSense without the limiter/ layer7 capabilities you get these days.

Again, it's all about getting your traffic shaping right.

You do not actually need an additional switch - your SF300-24 has 4 gigabit ports each. You can actually trunk 2 GbE ports from each switch to the next and simply connect pfSense with 1 GbE link to any of the switches.

I see!

Yea I know, but we might want to use those to expand if the amount of people exceeds what we have originally planned!

Seems like I am all set then! Thanks a lot guys, really appreciate the help!