Condo Assoc Networking Advice

[coleygm]

Hello...and thanks in advance as I'm new to this forum and this is my first post.

I am helping a condo assoc that has 84 units comprised in 11 separate buildings that wants to put an Ethernet jack in every unit so residents can simply plug in (or plug in their own router) and surf the internet. ...and Wife is not an option the assoc will consider. The buildings are situated in a line, so building 1 is approx 1/4 mile from building 11.

So, talking strictly of hard-wiring the association then (each building is connected to the next via direct Burial Cat5e)...the first question, is how much bandwidth would make sense assuming 84 units? My options from the local ISP (Charter) are 50Mbps and 100Mbps streams (5Mbps up in both cases). Let’s just say there are two connections per unit so 168 devices requesting bandwidth. We have no way of knowing what every connection would be requiring of the net, but this is assoc of mostly older people. Basic websites (news, weather, etc...) along with email and occasional facebook by some. In the summer grandkids visit, so god knows what that could bring but they are perfectly aware that it might be slower sometimes than others depending on the time (ie Labor Day Weekend would be slow).

Now where i start to get a little troubled is then if we have 2 internet streams coming into buildings 3 and 9, would there be the problem with cascaded switches for the two sections? IE the assoc's internet networking topology is broken into two sections (buildings 1-5 & 6-11). For example building 3 has the modem and routerh cascaded to a switch in building 2 which is cascaded to a switch in building 1...and going the other way cascading to 4 which is cascading to 5. Each buildings switch is then fed into 8 separate units. Within the unit then, a person would plug in their own personal router and go from there. Let me say that there isn't a budget for a stacked switch solution, but not sure if cascading like this would put definite breakdown on the outlying buildings (ie the further we get from the source, the more networking issues or slow internet). I would think that since no one device has more than a 3-4 hops to the internet it would be an issue, but again…this is where I get fuzzy on my knowledge.

Small visual with bold building numbers being where the internet source is
1-2-3-4-5 | 6-7-8-9-10-11

Assuming the above scenario would work, would then two 50Mbps internet transmissions feeding the two internet building sections work for the entire condo assoc? By my count each 50Mbps line would feed approx. 84 devices or 0.5Mbps per device which I’m thinking is fine. We’ll have no limiters, and doubt there would ever be a time that everyone is using the internet at the same time…so would think most times an individual would see a higher throughput….by the same token I guess a few cowboys streaming HD Netflix along with some Xbox would kill it for everyone.

Finally the routing and switching equipment that is planned on beings used are simple Cisco Small business devices (ie Routers = RV016 / Switches = sf100-16)

I know this is a lot of info, and hope I didn’t muddy it all to much. I appreciate any thoughts anyone may have regarding

Thanks again
greg

 

[stevech]

I looked at doing something like this for a luxury RV park. The contract for ISP service is the deal killer. (a) the cable TV guys won't do it at all. And metro ethernet is too expensive. There's no way I can see to get a contract where the Assoc. is the reseller/operator.

Then who operates this and takes the 11PM trouble calls?

I didn't go into the issue of retrofitting cat5.

I live in a condo complex, have been on the board. No way we'd want to become a private ISP. Nor could we compete with the prices from Time Warner, high as they are, and with 99.99% up time because so many people now have digital phone over cable TV IP service. Talking about phone service is another can of worms. I'm very pleased with TIme Warner's digital phone - it's equivalent to my long gone AT&T landline.

So, the big picture is what I'd look at, before cables and trenches.

 

[claykin]

Greg

You also need to consider the many wifi devices that will be connected. Phones, tablets, laptops, etc.. As Stevech said, trying to operate as a small ISP will be a daunting task. Check the contract terms on those 50Mb/s lines and see how many users they allow to connect. Also check for bandwidth caps. You may find you'll exceed or violate their terms on day 1 and risk having the ISP turn off your pipe(s).

In addition, you'll need enterprise gear to insure high reliability, isolate users from each other, handle load balancing during heavy use periods, etc... By the time you'll be done pricing this out doing it right (as well as professional oversight) you'll be more $ than just contracting with a local provider.

Plus every time there's a problem the tenants will be disliking building management for their decision to provide an internal solution. If you contract with a local ISP you can keep better peace with your tenants.

 

[coleygm]

I looked at doing something like this for a luxury RV park. The contract for ISP service is the deal killer. (a) the cable TV guys won't do it at all. And metro ethernet is too expensive. There's no way I can see to get a contract where the Assoc. is the reseller/operator.

Then who operates this and takes the 11PM trouble calls?

I didn't go into the issue of retrofitting cat5.

I live in a condo complex, have been on the board. No way we'd want to become a private ISP. Nor could we compete with the prices from Time Warner, high as they are, and with 99.99% up time because so many people now have digital phone over cable TV IP service. Talking about phone service is another can of worms. I'm very pleased with TIme Warner's digital phone - it's equivalent to my long gone AT&T landline.

So, the big picture is what I'd look at, before cables and trenches.

hey...thanks for the thoughts.

In our case, we'd be running Business Class service via Charter and I double checked with our rep today. They said there are no limits and they could care less who or how many people use the internet transmission. They bring us the line, and what we do with it after that is up to us.

So assuming we have no ISP issues, any thoughts as to pitfalls technically with the networking?

thanks again

 

[stevech]

hey...thanks for the thoughts.

In our case, we'd be running Business Class service via Charter and I double checked with our rep today. They said there are no limits and they could care less who or how many people use the internet transmission. They bring us the line, and what we do with it after that is up to us.

So assuming we have no ISP issues, any thoughts as to pitfalls technically with the networking?

thanks again

In case the sales rep is wrong, take a look at the business class model contract terms and conditions - if it says the service may not be resold or consigned.

Technical comment: I didn't see, in your architecture concept, a router that you own/operate. So do you think all the users' personal routers' WAN side DHCP requests will go to Cox's routers and give out public or net 10 type addresses, so you don't need to have your own enterprise edge router that does double-NAT? Or do you intend to have one or more routers and each WAN port gets a public IP address from Cox?

 

[coleygm]

Greg

You also need to consider the many wifi devices that will be connected. Phones, tablets, laptops, etc.. As Stevech said, trying to operate as a small ISP will be a daunting task. Check the contract terms on those 50Mb/s lines and see how many users they allow to connect. Also check for bandwidth caps. You may find you'll exceed or violate their terms on day 1 and risk having the ISP turn off your pipe(s).

In addition, you'll need enterprise gear to insure high reliability, isolate users from each other, handle load balancing during heavy use periods, etc... By the time you'll be done pricing this out doing it right (as well as professional oversight) you'll be more $ than just contracting with a local provider.

Plus every time there's a problem the tenants will be disliking building management for their decision to provide an internal solution. If you contract with a local ISP you can keep better peace with your tenants.

Thanks for the response...I'm not worried about the ISP as I've clarified with our rep and there shouldn't be any issues in those regards.

Regarding your thoughts on the technical networking setup though, I definitely understand the part about isolating users, but this would come in at everyone's unit as they'd have their own router....correct?

Otherwise, I'm not sure quite what you meant by 'enterprise gear'? I chose equipment that was the most cost effective, but still provided adequate NAT throughput, switching capacity's, MAC Tables, etc.... Clearly these switches don't have the customization options that more expensive models do, but didn't see the need for a managed switch or anything along those lines in this case. Let me know if I'm mistaken though and/or a specific model of router or switch you think would be needed and/or a specific element that a higher end model offers that I'd need.

thanks again for the thoughts

 

[coleygm]

In case the sales rep is wrong, take a look at the business class model contract terms and conditions - if it says the service may not be resold or consigned.

Technical comment: I didn't see, in your architecture concept, a router that you own/operate. So do you think all the users' personal routers' WAN side DHCP requests will go to Cox's routers and give out public or net 10 type addresses, so you don't need to have your own enterprise edge router that does double-NAT? Or do you intend to have one or more routers and each WAN port gets a public IP address from Cox?

gotcha...and great advice.

As for the routers, yes we plan on having two routers at both points of origin from Charter...then switches cascaded from there. The personal routers within the residence should would take care of individual device addressing per residence then and our routers would only need to deal out as many addresses are we have routers per unit...far beneath the 200+ address capacity if I'm thinking about it right.

let me know if I'm off here though...and thanks again

 

[claykin]

Trust me when I say you are barking up the wrong tree, especially when trying to use the level of equipment you selected.

Regarding Cox, have you thought about what will happen if you have P2P users violating DMCA? Continued violations will get your pipe disconnected and worse, legal letters coming to the landlord since you own the public IP. I can think of other bad scenarios too....

Another thing that comes to mind is when using unmanaged switches is you have no control. The first time someone connects a downlink only switch port (ex: port without auto uplink. Linksys for example) They will begin broadcasting on your entire LAN and take everyone down.

Also using unmanaged switches you put everyone on same LAN. Don't expect people to connect routers. Many will plug directly into your wall jacks and say, "wow I don't even need a router". The first worm on your LAN will have fun propagating from unit to unit.

And with the way you are double/triple Nat, its possible people will have trouble using certain services on the Net.

You absolutely need bandwidth management on a per port basis. Without this one user can suck up 100% of your available bandwidth.

I can go on and on....

 

[coleygm]

Trust me when I say you are barking up the wrong tree, especially when trying to use the level of equipment you selected.

Regarding Cox, have you thought about what will happen if you have P2P users violating DMCA? Continued violations will get your pipe disconnected and worse, legal letters coming to the landlord since you own the public IP. I can think of other bad scenarios too....

Another thing that comes to mind is when using unmanaged switches is you have no control. The first time someone connects a downlink only switch port (ex: port without auto uplink. Linksys for example) They will begin broadcasting on your entire LAN and take everyone down.

Also using unmanaged switches you put everyone on same LAN. Don't expect people to connect routers. Many will plug directly into your wall jacks and say, "wow I don't even need a router". The first worm on your LAN will have fun propagating from unit to unit.

And with the way you are double/triple Nat, its possible people will have trouble using certain services on the Net.

You absolutely need bandwidth management on a per port basis. Without this one user can suck up 100% of your available bandwidth.

I can go on and on....

Great stuff...and you definitly make me rethink the swtich portion. if we went to the 200 series of managed smart switches we could just create the vlan's we'd need so every unit would be on it's own ip if i'm thinking of it right.

regarding bandwidth, yes that is something we've mulled over...but with the managed switch does it have the abilit with QOS or something to say if 1 person is using...give them 100%. if 2 people then 50% each, etc... or is it only static in that each person only gets their portion regardless of who else is using at that time?

thanks again

 

[sfx2000]

Look into "hospitality" solutions, similar to what you find at Hotel's, etc...

There are a number of companies that do this, and have the support staff in place. More importantly, this is what they do... they have the knowledge and experience to do it right the first time and to keep the residents happy...

 

[Mark Uhde]

Ugh, this is a difficult situation, and you're definitely in over your head, but I'm not one to bite the person in over their head For one thing, you definitely want a "friendly" ISP that knows you're providing public service and can help with things like DMCA issues. Thankfully, none of the small businesses I've helped - even ones with public wireless - have been in those shoes, but some content filtering is done to reduce (NOT eliminate) risks. Obviously, you can't content filter someone's residential connection.

Beyond this, you'll need to manage bandwidth per-user, etc. Application-level QoS (though that may run into legal issues if they charge - with the whole network neutrality thing), etc. IFF I had to do this, I'd use a pfSense server as the firewall/gateway. Multi-WAN support (it sounds like you're getting two connections from Charter), traffic shaping, etc. Plenty of routing power on good hardware with good NICs (Intel work best). Then you need good managed switches. I'd literally probably VLAN EVERY SINGLE SWITCH PORT back to it's own interface in pfSense. This way, you're creating a "network" for each condo. NO ONE CONNECTS THEIR OWN ROUTER (give them instructions not to, tho if they do, worst case they get a double-NAT environment, but if they obey they don't get double NAT and it's as good as a normal home connection). Instead, they would connect plain old wireless access points. No one can damage the network for anyone else by connecting a DHCP server or something, and every virtual network gets it's own dedicated bandwidth guarantees and caps in the QoS/traffic shaping.

Honestly, the biggest scary thing with this is the day you have to show records and logs in a court case against one of your users. C'est la vie. It wouldn't stop me from doing the project in your shoes, but I'd make sure I had good logs, and a good terms of service (reviewed by a lawyer) everyone had to agree to before their switch port got enabled.

Anyways, to give you an idea of your costs, since you're cascading switches like you are (and I'm assuming all wiring is in place so I won't judge the fact you have copper when you should have fibre between buildings...) I just priced it out at just over $2000 in equipment. That's for the hardware to build a decent little pfSense server yourself and for 11 dirt-cheap TrendNET "smart" (web managed) switches that have 24 10/100 ports and 4 gigabit ports (so each condo gets a 10/100 port and you cascade them with the gigabit ports).

Again, my ONLY concerns in this are:

1. The liability, does the association know the liability they're taking on? Does your contract make it clear that you assume no liability?

2. Your plan is majorly under-building. But if you do what I suggested here, it should work very, very well, and still be extremely inexpensive.

I won't recommend specific places to buy, but I priced out a MicroATX LGA 1155 motherboard with PCI-Express slots, 2 2GB DDR3 DIMM's, an Intel Pentium G630 processor (low-end but way overkill for your needs since you're not doing any UTM or anything), a case with power supply, a decent quality hard drive, two Intel dual-port NICs (avoiding the motherboard's worse performing on-board NIC that gives you ports for up to 3 WAN connections and your network on the 4th port), and 11 of the cheap Trendnet smart switches mentioned above (NOT a recommendation, just the cheapest junk that'll work for you). If you can figure out how to get this all connected and finally get pfSense setup on it (basically, install pfSense, setup 2 WAN connections on two of the physical interfaces, and setup 85 VLANs on a third of the physical interfaces - one for DHCP and one for each unit [hint - make VLAN tag the unit # to make life easy], setup the gigabit switch ports as trunk ports and setup the 10/100 ports to be access ports for the VLAN tag that is that unit number, setup DHCP on each of those VLANs, setup a traffic shaper with a zillion (okay, 85) LAN type queues, setup shaping and limits on each of those 85 interfaces, and go get yourself an iced latte to relax)