pfsense + DIY Router ?

[Clive.B]

Hi need to run some ideas across the more enlightened !

I see many people like pfsense lately it also had openvpn support, I was looking at a few high end routers the new AC66/N7000 look great but expensive, so was just thinking why not build a super Router with pfsense for a bit more !

So my idea was simple at first, a SAPPHIRE EDGE VS8 or Intel Nuc, I was leaning on the intel nuc barebones 1.8ghz celeron, but the sapphire edge has 1.6ghz quad cpu it also supports AES support. So I think this would really give less cpu overhead with openvpn support ? (I use openvpn client with my vpn)

Either way yes the DIY router may cost me £100-150 more then the top routers but its about future proof and stability and performance, also if it breaks down can easily repair it!

Any thoughts or projects on what hardware or cpu to choose ?

 

[PrivateJoker]

Hi need to run some ideas across the more enlightened !

I see many people like pfsense lately it also had openvpn support, I was looking at a few high end routers the new AC66/N7000 look great but expensive, so was just thinking why not build a super Router with pfsense for a bit more !

So my idea was simple at first, a SAPPHIRE EDGE VS8 or Intel Nuc, I was leaning on the intel nuc barebones 1.8ghz celeron, but the sapphire edge has 1.6ghz quad cpu it also supports AES support. So I think this would really give less cpu overhead with openvpn support ? (I use openvpn client with my vpn)

Either way yes the DIY router may cost me £100-150 more then the top routers but its about future proof and stability and performance, also if it breaks down can easily repair it!

Any thoughts or projects on what hardware or cpu to choose ?

Since you're going to want multiple gigabit NICs in it (one from WAN, one to an WLAN AP?, one or more to LAN?) I guess you could do those through USB 3.0 GbE adapters put it would be sort of a kludge.

If it was me, I would grab a Dell, HP, or IBM server (there's a lot of them in 1-2U rack size, you don't have to actually put it in a rack if you don't want to) on eBay with a Xeon 5160-ish processor that will have, minimally, dual built in GbE NICs and just run your Pfsense on that. There is a massive glut of them on eBay in the $100-$200 USD range, then maybe get an RT-N66U or any of the N600-N900 routers listed here (high performance N router, that's not so new you'll get sticker shock, but still fast enough that you could run OpenVPN on it if you needed to, etc. assuming you don't have any AC clients today that could benefit from going AC). I called out the N66U specifically because I am familiar with it and it has a thriving community here on these boards, but read the reviews on the main site, there may be something less expensive that will suit your needs, especially if you have another computer doing the heavy lifting for sophisticated firewall and/or VPN options.

Just my $0.02. I'll probably be doing a similar project once I move and get resituated.

 

[Dreamslacker]

Hi need to run some ideas across the more enlightened !

I see many people like pfsense lately it also had openvpn support, I was looking at a few high end routers the new AC66/N7000 look great but expensive, so was just thinking why not build a super Router with pfsense for a bit more !

So my idea was simple at first, a SAPPHIRE EDGE VS8 or Intel Nuc, I was leaning on the intel nuc barebones 1.8ghz celeron, but the sapphire edge has 1.6ghz quad cpu it also supports AES support. So I think this would really give less cpu overhead with openvpn support ? (I use openvpn client with my vpn)

Either way yes the DIY router may cost me £100-150 more then the top routers but its about future proof and stability and performance, also if it breaks down can easily repair it!

Any thoughts or projects on what hardware or cpu to choose ?

What setup you should choose will really depend on what you intend to run on the router (Snort, OVPN, IPSEC, Squid etc).

A D2500 will give about 500-700Mbps of NAT throughput. VPN throughput will be about 50-60Mbps. AES-NI will help if you specifically use AES encryption for OVPN rather than Blowfish. Even so, you will need to actually push that kind of rates over VPN (mobile broadband is generally a limiting factor for road warriors).

The main issue you will face is whether the NIC is supported by pfSense (2.1 at the moment). Some of the newer Intel NICs (200 series) and Realtek NICs (8111G) will not work.

A single NIC isn't too much of a problem if you pair it up with a VLAN capable switch such as the Netgear GS108T or a HP Procurve 1810-24G. I wouldn't use the USB ethernet adapters. They are flaky in operation and do not support ALTQ (required for traffic shaping).

 

[Clive.B]

thx so far for the tips, was not aware some nics are not compatible with pfsense !

Would have to watch out for what versions I would get.

I was considering getting usb 3.0 gigabit dongles but if they are flaky as suggested then it may not be the best idea....

I did not really want any routers or racks tbh, just one simple device I could run it all from. Even if I ran a gigabit switch or dual ethernet power plugs.

It was more for openvpn support and perhaps 3 machines(pc + nas), 1 would not be much in use and very little bandwith required.

Any other suggestions and ideas are welcome, I have never come across so much small yet important details even on other tech forums ! so thx

 

[Dreamslacker]

The NUC's NIC will work with pfSense 2.1.

You can use a USB thumbdrive with the NanoBSD VGA builds to run the unit. This eliminates the need for a pricey Msata SSD but does require a slight hack to get it running.

See what I posted here for the instructions (Note that I'm using serial console rather than VGA which you want on the NUC because it doesn't have a native serial port):
http://forum.pfsense.org/index.php?topic=28707.0

Aside from that, you will want to edit /boot/loader.conf.local instead (this was on an older install where the .local file wasn't retained on upgrades, it now is).

You will want to get a VLAN capable switch if using a single NIC setup. Delegate 2 VLANs such as VLAN 100 & VLAN 200 for LAN and WAN respectively.

On the initial boot, enter 'Y' when prompted to setup VLANs. Setup the 2 VLANs and then assign the VLANs interfaces as your LAN and WAN respectively.

On the switch side, setup the port connected to pfSense unit as trunk port with members VLAN 100 & 200.
Setup one more port (to connect to modem) as access port (aka Untagged port, member of VLAN 200, default PVID 200). Connect this to your modem or ONT (if using FTTH).
Setup the rest of the ports used for LAN as access ports (Untagged, member of VLAN 100, default PVID 100).

The rest of pfSense configuration should be quite straight forward after that if you follow the guides on the pfSense website.

 

[Clive.B]

thanks ds

I was only considering the intel nuc, but as suggested above using Ethernet usb dongles may prove unreliable.

I was going to use the pfsense live cd iso, run it from ram or just install onto a thumb drive to avoid any issues via yumi or bootable iso programs, I think if that did not work id use the cheapo 8gig msatas on ebay for £20-30 if none of the above worked.

Still it is a complex and time consuming setup, I think I will first wait to find confirmation if openvpn support under pfsense will not be bottlenecked like they appear to be on even the top end and most fastest routers like netgears R7000.

If it is bottlenecked, then for myself pfsense + diy mini pc makes sense and better then an R7000 even

 

[Dreamslacker]

thanks ds

I was only considering the intel nuc, but as suggested above using Ethernet usb dongles may prove unreliable.

I was going to use the pfsense live cd iso, run it from ram or just install onto a thumb drive to avoid any issues via yumi or bootable iso programs, I think if that did not work id use the cheapo 8gig msatas on ebay for £20-30 if none of the above worked.

Still it is a complex and time consuming setup, I think I will first wait to find confirmation if openvpn support under pfsense will not be bottlenecked like they appear to be on even the top end and most fastest routers like netgears R7000.

If it is bottlenecked, then for myself pfsense + diy mini pc makes sense and better then an R7000 even

A D2500 Atom will do about 50-60Mbit/s of OVPN throughput on BF-CBC.
The Celeron 847 will likely do more.

Don't fret over the single NIC, just grab a Netgear GS108T or Mikrotik RB250GS (or RB260GS) and run VLANs. Unless you've a 500Mbit/s or 1Gbit/s FTTH connection, using VLANs and a smart switch to 'multiply' the interfaces will suffice.

Personally, I've managed about 40Mbit/s sustained (connection upload speed limitation, my FTTH upload at the time peaked out around 43Mbit/s) on mine (Intel Core 2 Duo T7200). This chip should handle about 75-100Mbit/s though I've not had the opportunity to test this. Screenshot attached of file transfer over File sharing through OpenVPN.

Edit: One thing pfSense won't do better than any of the N or AC routers is Wifi AP. The choice of Wifi interfaces is limited and even when supported, it's limited to G only (even on supported N cards).

 

[Clive.B]

A D2500 Atom will do about 50-60Mbit/s of OVPN throughput on BF-CBC.
The Celeron 847 will likely do more.

Don't fret over the single NIC, just grab a Netgear GS108T or Mikrotik RB250GS (or RB260GS) and run VLANs. Unless you've a 500Mbit/s or 1Gbit/s FTTH connection, using VLANs and a smart switch to 'multiply' the interfaces will suffice.

Personally, I've managed about 40Mbit/s sustained (connection upload speed limitation, my FTTH upload at the time peaked out around 43Mbit/s) on mine (Intel Core 2 Duo T7200). This chip should handle about 75-100Mbit/s though I've not had the opportunity to test this. Screenshot attached of file transfer over File sharing through OpenVPN.

Edit: One thing pfSense won't do better than any of the N or AC routers is Wifi AP. The choice of Wifi interfaces is limited and even when supported, it's limited to G only (even on supported N cards).

thx a wealth of info, are you suggesting not to worry about the intel nucs single gigabit port then and just to use a netgear 8port switch instead ?

I am still new to all this, but figured you need 2 ports one for wan and one for lan, If a switch allowed one to plug in a router and then the extras for others.... it is interesting.

What type of cpu would you advise to get around the openvpn bottleneck ?
I was considering one with AES instructions since I hear it offloads much of the work especially if your VPN is using AES encryption, but I guess considering openvpn clients use single core only, a fast 3ghz may do 100meg+ while under openvpn.

Its good to hear you are aware of the cpu limitations under openvpn, this is really my reasoning to go pfsense+diy route + I maybe getting 60meg or 100meg bb and just wanna use it to the fullest.

 

[Dreamslacker]

thx a wealth of info, are you suggesting not to worry about the intel nucs single gigabit port then and just to use a netgear 8port switch instead ?

I am still new to all this, but figured you need 2 ports one for wan and one for lan, If a switch allowed one to plug in a router and then the extras for others.... it is interesting.

What type of cpu would you advise to get around the openvpn bottleneck ?
I was considering one with AES instructions since I hear it offloads much of the work especially if your VPN is using AES encryption, but I guess considering openvpn clients use single core only, a fast 3ghz may do 100meg+ while under openvpn.

Its good to hear you are aware of the cpu limitations under openvpn, this is really my reasoning to go pfsense+diy route + I maybe getting 60meg or 100meg bb and just wanna use it to the fullest.

Yes, I'm suggesting using a smart switch with VLANs if you are looking to use the NUC or any other small single NIC setup without expansion slots.

If you're in Msia, check out Lowyat forums on using smart switches with VLANs for tutorials. Plenty of those floating around from the guys using Unifi (The FTTH service, not the Ubiquiti product).

The packet filter process is still locked to single core at the moment but other processes like OpenVPN, Snort and Squid are free to utilize other cores.

I wouldn't worry too much about the firewall process though - as I mentioned earlier, even an Atom would push 500-700mbps of NAT traffic.
A Core i3 at 3GHz will easily provide >1Gbps of routing.

If you are confident that you can push 100mbps or more of VPN traffic, go for a Core i3 ITX setup. It's rare to do that for road warriors though I won't discount the fact (FTTH at the office and FTTH at home for the mobile client).

 

[Clive.B]

Yes, I'm suggesting using a smart switch with VLANs if you are looking to use the NUC or any other small single NIC setup without expansion slots.

If you're in Msia, check out Lowyat forums on using smart switches with VLANs for tutorials. Plenty of those floating around from the guys using Unifi (The FTTH service, not the Ubiquiti product).

The packet filter process is still locked to single core at the moment but other processes like OpenVPN, Snort and Squid are free to utilize other cores.

I wouldn't worry too much about the firewall process though - as I mentioned earlier, even an Atom would push 500-700mbps of NAT traffic.
A Core i3 at 3GHz will easily provide >1Gbps of routing.

If you are confident that you can push 100mbps or more of VPN traffic, go for a Core i3 ITX setup. It's rare to do that for road warriors though I won't discount the fact (FTTH at the office and FTTH at home for the mobile client).

thx will consider all the options, a mini itx build is appealing

 

[Terry Kennedy]

If it was me, I would grab a Dell, HP, or IBM server (there's a lot of them in 1-2U rack size, you don't have to actually put it in a rack if you don't want to) on eBay with a Xeon 5160-ish processor that will have, minimally, dual built in GbE NICs and just run your Pfsense on that. There is a massive glut of them on eBay in the $100-$200 USD range[...]

I'm quite familiar with the Dell PowerEdge server line, and I have a few comments about drawbacks for using these for home use. Many of these probably apply to other brands as well:

• They're quite large (and heavy) for this sort of lightweight application. And since they're shaped like oversize pizza boxes, they may not be the easiest thing to find a spot for.
• Being servers, they have loud fans. Particulary at system startup, before the BMC takes control of them. How loud? Well, the upper warning limit for the Generation 10 units is over 34,000RPM. And fan speed tends to fluctuate with minor variations in temperature, which can be particularly annoying.
• They can be quite power-hungry, particularly the older generations. The newer ones are a lot better, but are more expensive on eBay. I have a Generation 11 R710 with dual X5680 Xeons, 48GB of RAM, and 6 15K SAS drives and it idles at about 260 watts, which is amazingly low.

On the other hand, something like a R310 can probably handle just about all needs a user might have - NAS, web server, firewall, tunnel endpoint, P2P, etc.

I actually got that R710 for free from a datacenter that closed, when the landlord dumpstered everything. In this picture you can see dumpsters full of Cisco 7200VXRs, etc. The switch standing on its side in the back dumpster is a 10GbE SFP+ switch. You can see from all of the bent metal that these were just ripped out and trashed. Sorry, it's all gone now and no, I can't tell you where.

 

[Clive.B]

This was why originally I was looking at the AMD sapphire or intel nuc as an option, something that uses very little electricity and can just be left 24/7 on. However as suggested before I would need to use switches which means more wires/power. USB network dongles are hit and miss sadly.

I am also looking in the cubie i4 pro, 1ghz quad arm cpu. Could perhaps run ubuntu or linux, and then run pfsense from there via vbox. Few options available, something that small and tiny.

Then again am considering I need to install pfsense and perhaps get another nic and see performance and how openvpn runs and sets up for myself, R7000 is tempting me also !