Shared Wireless bridge

[Warb]

I wasn't sure whether to post this in wireless or "other", but I figured that the fact that it is a wireless link makes no real difference, so.....

It's a few years since I retired from IT, and whilst I do an occasional project for fun I'd like an up to date opinion on this one! I live on a farm, outside the reach of "wired" broadband and in a dead spot for mobile phones. As a result I'm limited to an expensive and very slow satellite connection which is OK for overnight downloads but ranges from awful to impossible for anything sensitive to latency.

However.... I own a building in town, from which there is line of sight to the top of a hill next to my farm. With two wireless PtP links I could get broadband with only a few ms extra latency, which must be better than the 800ms I currently live with. The wireless aspect isn't hard, I already run such links between buildings on my farm. The problem is that the hill is on my neighbours property, and he is also restricted to satellite. He's happy for me to "use" his hill, but obviously would like to speed up his internet too! I'd like him to benefit, and it will spread the costs.

I can get 2 broadband connections in town, a PtP from there to the top of the hill, and 2 further PtP's, 1 to each farm. The routers would, presumably, still be located at each farm such that only internet traffic would hit the wireless links. The wireless links themselves are Layer 2 devices, something like Ubiquiti AirFibers (details undecided).

The question is what is the best way to connect 2 broadband links (static IP addresses) to two domestic routers via a shared PtP link (which acts like any other bridge). Obviously the routers can't use DHCP on the WAN (bridge) side, but are there any other issues? Or is it just a matter of a switch (with or without VLANs) at each end of the bridge to connect 2-1-2?

 

[Cloud200]

Any chance you can get a diagram in something like Visio or Draw.io ?

It would help a lot in understanding what it is exactly that you wish to accomplish.

 

[Warb]

Any chance you can get a diagram in something like Visio or Draw.io ?

It would help a lot in understanding what it is exactly that you wish to accomplish.

Edit. This forum removes multiple spaces and won't let me use tabs, which makes ASCII diagrams very hard! Please interpret ...... as spaces!

ISP modem1}....................................................{}---------wireless bridge--------{C domestic router1 + LAN
................A}-----------wireless bridge-----------{B
ISP modem2}....................................................{}---------wireless bridge--------{D domestic router2 + LAN



Location A is in town, with two ISP "boxes" with fixed public IP addresses. These share a wireless point to point to location B, where they are "split" to provide service to two farms (C and D). The two domestic routers (at locations C and D) would be configured with fixed IP addresses to talk to "their" ISP connection (avoiding DHCP issues of selecting the "wrong" ISP box).

If devices A and B were repeaters this would work perfectly, but all (routed) traffic would be repeated on all 3 wireless links. I am simply wondering, given that these days "hubs" are usually VLAN capable switches rather than old fashioned repeaters, whether there is any advantage to devices A and B being configured (VLAN etc.) in a more advanced manner than straight "repeating".

 

[Cloud200]

https://drive.google.com/file/d/0B4opHQDEV9aNZVZ1TW1BN2V0VDQ/view?usp=sharing

Just confirm for me that this is the setup.
If this is the case the benefit from using VLANs would be minimal but there.
The main benefit would be if your router supports WAN failover and the ISPs offer more than a single static IP.
That way if one connection is down you are not both down.

If not you can still do this by adding a dual WAN router to the "Hill" and then creating a static route to router A and B along with firewall rules to prevent networks A and B from talking to each other.

 

[Warb]

That drawing is correct, assuming that the town to hill link is a single shared point to point (I'm guessing you understand that and have drawn both yellow and purple "logical", rather than physical, links).

It was my thought that the only benefit of using VLANs would be to remove traffic from the non-shared links, but in fact these links will likely be much faster than the shared town-hill link (because they are shorter) which to some extent negates the benefit.

Given the restrictions of the broadband market here, other than local equipment failure it is unlikely that one WAN link would be down and the other still working, so failover from one to the other would probably be pointless. My experience (though a few years out of date now!) is that unless different vendors and preferably different physical routes are available, multiple connections are normally all or nothing. Additionally it adds a layer of "human" complexity if one farm starts using the other farms broadband. With a shared ptp link and otherwise complete separation, if one ISP account is (for arguments sake) unpaid/lapsed then that farm loses its broadband but the other is unaffected either financially or technologically.

I am trying to keep the configuration and routing simple, to allow a single NAT and also to minimise the hardware on the hill, because it will be solar powered so minimal power usage is beneficial.

 

[Cloud200]

That drawing is correct, assuming that the town to hill link is a single shared point to point (I'm guessing you understand that and have drawn both yellow and purple "logical", rather than physical, links).

It was my thought that the only benefit of using VLANs would be to remove traffic from the non-shared links, but in fact these links will likely be much faster than the shared town-hill link (because they are shorter) which to some extent negates the benefit.

Given the restrictions of the broadband market here, other than local equipment failure it is unlikely that one WAN link would be down and the other still working, so failover from one to the other would probably be pointless. My experience (though a few years out of date now!) is that unless different vendors and preferably different physical routes are available, multiple connections are normally all or nothing. Additionally it adds a layer of "human" complexity if one farm starts using the other farms broadband. With a shared ptp link and otherwise complete separation, if one ISP account is (for arguments sake) unpaid/lapsed then that farm loses its broadband but the other is unaffected either financially or technologically.

I am trying to keep the configuration and routing simple, to allow a single NAT and also to minimise the hardware on the hill, because it will be solar powered so minimal power usage is beneficial.

You are correct on the first part; assumptions of the drawing. The drawing assumes only 1 single RF link between "town" and "hill"

On the second count of failover. Correct, if say ISP A uses Coax and ISP B uses ADSL . . . smaller chance of both being down at the same time. Sometimes however you are just buying two different routes to the same main CO. If the CO has an issue both your lines are down.

On the last count of separation and security. If that is a concern, use VLANs to header the traffic at the "town" on a switch with 3 VLANs. 1 for native just for management of the wireless network. 2 for Farm A. 3 for Farm B. At the "hill" the switch there gets a trunk port coming in from the "town" but access ports going downstream to the Farms. This way you never have to worry about Farm A touching Farm B's traffic at any point in time.

edit;
A decent unit for powering up the devices in location "hill" are from a company called Netonix.
If you are supplying conditioned DC power it should cost you about $150 for a switch able to power 3 radios. If the power is unconditioned the unit you want runs almost $400 but the power input is anywhere from 9-60v. It really depends on the solar panels and batteries you are using.

 

[sfx2000]

Just thinking - with sources A/B - bond them, so one has one pipe over the bridge link, then one can VLAN out from there on the same box - each tenant then gets their one drop they can do stuff with....

Over simplifying things - it's a complicated setup, but it can be done...

 

[Warb]

On the last count of separation and security. If that is a concern, use VLANs to header the traffic at the "town" on a switch with 3 VLANs. 1 for native just for management of the wireless network. 2 for Farm A. 3 for Farm B. At the "hill" the switch there gets a trunk port coming in from the "town" but access ports going downstream to the Farms. This way you never have to worry about Farm A touching Farm B's traffic at any point in time.

edit;
A decent unit for powering up the devices in location "hill" are from a company called Netonix.
If you are supplying conditioned DC power it should cost you about $150 for a switch able to power 3 radios. If the power is unconditioned the unit you want runs almost $400 but the power input is anywhere from 9-60v. It really depends on the solar panels and batteries you are using.

Whilst there needs to be a degree of security, keeping the traffic separate is really only a financial issue - the two farms may have ISP packages with different speeds or quota's etc.

The 3 VLAN setup you describe is exactly what I had in mind, and for exactly the reasons you mentioned.

At present I am looking at PoE equipment, with the concept being that a VLAN switch could power all three radios on the hill. It depends on the power needs of the radios, but if it works out then a single PoE switch could potentially power all three radios. It just comes down to what radios and hence what power is required. Ubiquiti PoE injectors will run (from memory) on a very wide range of input voltage. Nothing is decided yet!

 

[Cloud200]

Look at a Netonix switch model WS-12-250-DC.
I have used it to power a pair of AirFibers, 4x 90 degree sectors and a router. Everything was being powered by a pickup truck at a beach during this past summer.

 

[Warb]

Just thinking - with sources A/B - bond them, so one has one pipe over the bridge link, then one can VLAN out from there on the same box - each tenant then gets their one drop they can do stuff with....

Over simplifying things - it's a complicated setup, but it can be done...

Bonding the two connections would give higher single-user WAN speeds, but would also require more management because local quota's would have to be set on the VLANs to prevent one farm burning through both ISP data quotas. Such local quota's would need to be updated if the ISP deal was changed, meaning the farms would lose some autonomy.

I can also see issues arising from removing the farms local routers - DHCP etc. would be via the wireless bridge or otherwise locally provided (and configured), and any other router provided functions (Wifi, printers, NAS etc.) would need to be reprovisioned.

Left unbonded, with a router at each farm, no special configuration of the routers is required and each farm is completely autonomous. In fact whilst I have not yet 100% thought it through, I suspect that a VLAN setup will allow the farm routers to use DHCP on their WAN ports, meaning that no local config is required once the farm-to-hill PtP link is installed. Each farm just plugs in to their local "radio box" and get a virtual connection to their ISP's modem in town.

But it's been a few years since I've done any significant network design, so it's entirely possible that I'm missing something obvious...!

 

[sfx2000]

Bonding the two connections would give higher single-user WAN speeds, but would also require more management because local quota's would have to be set on the VLANs to prevent one farm burning through both ISP data quotas. Such local quota's would need to be updated if the ISP deal was changed, meaning the farms would lose some autonomy.

I can also see issues arising from removing the farms local routers - DHCP etc. would be via the wireless bridge or otherwise locally provided (and configured), and any other router provided functions (Wifi, printers, NAS etc.) would need to be reprovisioned.

Please do not confuse business decisions (accounts, etc) with technical/engineering solutions...

The business side - the farmers can form a co-op solution, delivering a proposed technical solution to the ISP/ISP's...

Building/maintaining the bonds - that's an admin task that while a bit complex, is something that provided cooperation between the different customers and the ISP, is likely something that would only happen occasionally.

Think of it as forming an informal carrier/operator solution - perhaps from a business side, an LLC with the different farmsteads as shareholders and customers - pooling resources...

 

[Warb]

Please do not confuse business decisions (accounts, etc) with technical/engineering solutions...

[Edit: I meant also to say that "confusing" technical decisions with engineering decisions is not the same as "balancing" them. Allowing either to be made in isolation is almost always a recipe for disaster.]

No indeed. It's just that I can't honestly see any significant benefit for what would seem to be a fair amount of invested time and money, and I can see significant downsides.

Option 1:
As described above, 2 ISP boxes.
Shared Investment= 2 VLAN switches, 3 PtP links.
Per Farm investment = none.
Shared config = 3 VLANs (2 switches), 6 radios.
Per farm config = none

Advantages:
Each farm keeps its existing LAN completely unaltered and is free to pick it's own ISP package.
Each farm can ditch/change it's ISP agreement with no impact or reconfiguration required.
Each farm can work (albeit with no internet) given any failure of the wireless links.
Each farm invests 50% of the capital and then can walk away at any time (cancelling own ISP agreement).

Disadvantages:
Potentially users will get lower performance than option 2, though this depends on the wireless PtP being fast enough to make best use of bonded WAN ports.


Option 2:
Bonded WANs.
Shared Investment = Dual WAN router plus everything in option 1, deleting 1 VLAN switch
Per Farm investment = replacement of any functions currently on router
Shared config = As option 1, plus VLAN quota, central DHCP, dual WAN router
Per farm config = Everything previously on local router

Advantages:
Potentially users will get higher performance than option 1, though this depends on the wireless PtP being fast enough to make best use of bonded WAN ports.
If a single WAN connection fails, connection should remain active. Given that both WAN ports are likely to be via the same fibre etc., is it likely that one only will fail?

Disadvantages:
All router functions via wireless, so no DHCP etc. (with knock-on impacts) etc. if wireless fails.
Requires more shared investment (I'm guessing a dual WAN router etc. >$ than a switch).
Requires significantly more config, both per farm and shared.
Requires enforceable agreement or other arrangement to cover shared ISP agreements.



In fact the configuration is made more complicated by the fact that some of the daytime traffic on the PtP will not go to the WAN, and so cannot be included in the VLAN quota system. This is because traffic that previously used satellite internet to get between the farm and the location in town will now, given the direct link, become LAN traffic rather than WAN traffic. I have not really thought through the config required for that, except that it is effectively zero for "option 1", and more significant for option 2.

However, as I said I retired from IT a few years ago so this is the first bit of network design (excepting LAN and associated short-hop wireless PtP) that I've done for a while and I may well be missing something. But the main point here is to create a simple and as much as possible "commitment free" means for two farms to share the upfront costs of moving from almost unusable (slow and expensive) satellite connections to something better and (on an ongoing basis) cheaper. It's not a business venture, and I'm having too much fun doing other things to want to dedicate a great deal of time to supporting it, so minimal ongoing config requirements are important. If there are significant gains to be made from more complex configs then that's OK, but we're starting out with 800ms latency, 300KBps and 10GB/month (in waking hours) for $110, so a couple of bean cans and some string would be an improvement.....

 

[Warb]

Look at a Netonix switch model WS-12-250-DC.
I have used it to power a pair of AirFibers, 4x 90 degree sectors and a router. Everything was being powered by a pickup truck at a beach during this past summer.

That would be about perfect. Given the PoE aspect I can understand why they don't quote a maximum power consumption, but I'd love to know its efficiency. Given it appears to be able to step up voltage from 9V to 48V and also step down from 60V to 24V, some efficiency graphs would be useful to allow the user to choose an optimum battery bank voltage!

 

[Cloud200]

The efficiency is ~95% depending on load at 24v.
It drops to ~89% at around 12v.
It drops to ~93% at around 54v

The sweet spot is 24v - 48v depending on the voltage of the radios you are trying to power. For example if you got 5x 24v radios and 1x 48v, go with 24v as the source.

 

[Warb]

The efficiency is ~95% depending on load at 24v.
It drops to ~89% at around 12v.
It drops to ~93% at around 54v

The sweet spot is 24v - 48v depending on the voltage of the radios you are trying to power. For example if you got 5x 24v radios and 1x 48v, go with 24v as the source.

Thanks, that looks like an ideal solution for the hill.