Confessions of a pfSense Newbie ...

[sfx2000]

Which switch are you using? Your DHCP statement makes no sense to me, it says to me your layer 3 switch is running as layer 2? Which means you are not running a layer 3 switch so calling it that is confusing to readers. It is not called a layer 3 switch if you don't run it that way.

Without the VLAN defintions on pfsense how are you using pfsense for DHCP? When you run your switch as layer 3 the network VLANs are only defined to the switch and pfsense will not be aware or care about the network VLANs on the switch other than having a static route statement pointing to the layer 3 switch so packets can be forwarded. You could also use a routing protocol but I doubt your network gear is that sophisticated. And routing protocols don't work that well on pfsense from my reading.

@coxhaus - you are entitled to your own opinion, and I respect that...

Yes, I have an L3-Lite switch standing behind pfSense, and that's ok - DHCP is actually handled on a separate host from the GW, and that goes for DNS as well. There is no hard/fast rule to say that DNS/DHCP needs to be handled by the edge router on a small network - makes things easier, but it's not mandatory.

Anyways - I know you have had past issues with pfSense and getting best performance out of it - and that's ok, everyone's network is different, and folks have their past experience to go with.

 

[coxhaus]

I was just responding to your statement about pfsense. pfsense will not provide DHCP services for networks not controlled by pfsense. This goes back to my statement that pfsense likes to be in control or it does not play well with others.

A L3-Lite switch sounds like another marketing term for an L2 switch so your statement that pfsense works well with a layer 3 switch is maybe stretching the truth a little. The statement you made above about pfsense using a layer 3 switch will not work because pfsense does not support it. I have setup pfsense with a layer 3 switch. I can tell you have not.

Running DHCP and/or running DNS from another server is not a problem I see at all. You can not setup pfsense to provide DHCP services for a network unless all the networks are defined to pfsense. It might be a nice feature to add to pfsense but they seem to want pfsense to control everything so I doubt it will happen.

 

[sfx2000]

I was just responding to your statement about pfsense. pfsense will not provide DHCP services for networks not controlled by pfsense. This goes back to my statement that pfsense likes to be in control or it does not play well with others.

How I build out my little home network is my business - it works...

pfsense plays well, once one gets the gist of what and how things work there.

A L3-Lite switch sounds like another marketing term for an L2 switch so your statement that pfsense works well with a layer 3 switch is maybe stretching the truth a little. The statement you made above about pfsense using a layer 3 switch will not work because pfsense does not support it. I have setup pfsense with a layer 3 switch. I can tell you have not.

I've got a fair amount of experience in dealing with L3 switches - Cisco, Juniper and others - some have small busines experience, some enterprise, some carrier-grade - I'm in the carrier grade group there.

Running DHCP and/or running DNS from another server is not a problem I see at all. You can not setup pfsense to provide DHCP services for a network unless all the networks are defined to pfsense. It might be a nice feature to add to pfsense but they seem to want pfsense to control everything so I doubt it will happen.

Think what you want... I know you have a h**d**n for pfSense as it didn't rock your particular boat, but that's you... and that's ok

 

[coxhaus]

In my instance - I moved all the VLAN's over to a Layer 3 switch - DHCP was kept on pfSense, and there, it's pretty flexible.

Why would you make this statement? You can not do this with pfsense and a layer 3 switch…… The statement is not true. I explained above.

I don't care how you setup your network. It is your network.

 

[kvic]

And routing protocols don't work that well on pfsense from my reading.

pfSense was forked from m0n0wall. So its emphasis is more as a firewall in the beginning. Like any FOSS projects, they could add routing protocols later but I doubt that many people are using pfSense outside the role of firewall/edge routers. For SOHO, everyone basically needs a firewall/edge router to the Internet, pfsense with a pretty GUI looks like a natural fit.

OPNsense breaking from pfSense has personal emotion involved perhaps. I'm not interested in that. The primary reason as I understand from reading is that pfSense guys refused to move forward with better GUI technologies. A few years after OPNsense forking from pfSense, pfSense adopted the same GUI technologies as OPNsense guys originally proposed but got rejected.

m0n0wall stop development relatively recently. Its author endorses OPNsense and encourages m0n0wall users migrate to OPNsense.

 

[coxhaus]

How does openwrt fit into this? I saw it mentioned in Jim's article. I have not played with either opensense or openwrt.

PS
I figured it out. Never mind.

 

[kvic]

How does openwrt fit into this? I saw it mentioned in Jim's article. I have not played with either opensense or openwrt.

PS
I figured it out. Never mind.

Yeah, think of OpenWRT as Debian-like for a Linux router.

 

[sfx2000]

Yeah, think of OpenWRT as Debian-like for a Linux router.

That's actually not an accurate statement - it's more like buildroot on steroids...

It's a very nice platform for supported HW - and bringing in support for new SoC's and hardware is pretty straightforward...

Now that the OpenWRT and LEDE split is over, things are improving - 18.06 brought forward quite a few changes for the better

They're cleaning up older platforms, and someone has stepped in to clean up the Wiki - the OpenWRT forums were partially lost some time back, they've restored some of it... check the LEDE forums for current user/dev stuff...

One of the challenges for the new OpenWRT is a lack of maintainers - I've done a gentle Jedi force nudge on the Asus and Netgear 3rd party devs here to check their interest, as their experience and insight would be very helpful to the project.

 

[sfx2000]

pfSense was forked from m0n0wall. So its emphasis is more as a firewall in the beginning. Like any FOSS projects, they could add routing protocols later but I doubt that many people are using pfSense outside the role of firewall/edge routers. For SOHO, everyone basically needs a firewall/edge router to the Internet, pfsense with a pretty GUI looks like a natural fit.

OPNsense breaking from pfSense has personal emotion involved perhaps. I'm not interested in that. The primary reason as I understand from reading is that pfSense guys refused to move forward with better GUI technologies. A few years after OPNsense forking from pfSense, pfSense adopted the same GUI technologies as OPNsense guys originally proposed but got rejected.

m0n0wall stop development relatively recently. Its author endorses OPNsense and encourages m0n0wall users migrate to OPNsense.

Interesting to note that Netgate - which is the main corp sponsor of pfSense, is moving towards Linux and away from FreeBSD with their TNSR project...

https://www.netgate.com/products/tnsr/

I don't take it as a no confidence vote on FreeBSD, but a reality check where it's recognized that Linux just has a lot more 3rd party contributions going in, so the development pace is faster - less work on foundational stuff, and that allows them to focus on some special sauce that they've developed in the past with less porting effort back to FreeBSD.

they did some great heavy lifting to get pfSense (and FreeBSD) up to 10Gbe, but at a big cost - 10Gbe in a production system is an expensive option, as it demands very high clock rates to get the packets per second into the useful range - with Linux, and the various SDK's (think DPDK and QAT for example), it's less work...

 

[sfx2000]

Why would you make this statement? You can not do this with pfsense and a layer 3 switch…… The statement is not true. I explained above.

Yes, I can, and I can do it without pfSense... even with a Layer3-Lite switch like Netgear's GS-108T line - this is not rocket science - yes, one does need to have a plan, and be methodical about it, but definitely do-able.

One can let the pfSense GW manage the VLAN's, but it's not required, and with some platforms, not desired...

For most looking at networks of the SNB scale - pfSense and a lightweight switch like the GS-108T is a good path - and the GS-108T is very affordable, sub-$100 in most markets.

 

[coxhaus]

If you change your statement to a layer 2 switch instead of a layer 3 switch then your statement is correct but as you wrote it, it is wrong. The concepts of layer 3 switches are different than layer 2. The layer 3 switches are working at the routing level not the switch level so layer 3 switches are routers with lots of networks which they route. People are having a hard time understanding this.

 

[sfx2000]

If you change your statement to a layer 2 switch instead of a layer 3 switch then your statement is correct but as you wrote it, it is wrong. The concepts of layer 3 switches are different than layer 2. The layer 3 switches are working at the routing level not the switch level so layer 3 switches are routers with lots of networks which they route. People are having a hard time understanding this

My network manages two IP ranges thru the switch - I'm not going to get into what color the bike shed should be painted, but it is was it is... layer 3

I know you are deeply invested in the Ci$co$phere - and that's ok...

 

[coxhaus]

My network manages two IP ranges thru the switch

Your statement is saying your switch is layer 2. Otherwise your statement should say my layer 3 switch manages 2 networks or 2 networks are defined to my layer 3 switch which it manages. The layer 3 switch is the network. It controls the networks defined to the layer 3 switch. There is no pass thru, that is a layer 2 switch concept.

The problem I have between a layer 2 switch and a layer 3 switch is with default gateway. With a layer 2 switch you have a default gateway. With a layer 3 switch you have a default route. I say default gateway when I mean default route.

 

[tman222]

Interesting to note that Netgate - which is the main corp sponsor of pfSense, is moving towards Linux and away from FreeBSD with their TNSR project...

https://www.netgate.com/products/tnsr/

I don't take it as a no confidence vote on FreeBSD, but a reality check where it's recognized that Linux just has a lot more 3rd party contributions going in, so the development pace is faster - less work on foundational stuff, and that allows them to focus on some special sauce that they've developed in the past with less porting effort back to FreeBSD.

they did some great heavy lifting to get pfSense (and FreeBSD) up to 10Gbe, but at a big cost - 10Gbe in a production system is an expensive option, as it demands very high clock rates to get the packets per second into the useful range - with Linux, and the various SDK's (think DPDK and QAT for example), it's less work...


View attachment 14011

Have been using pfSense for a little while and even did some basic performance testing at 10Gbit/s. Based on my finding, I will say at that at 10Gbit/s one does require very beefy hardware to be able push enough PPS across the firewall. Having said that, I'm quite excited to see where things head next - tnsr does look very promising.

 

[sfx2000]

Have been using pfSense for a little while and even did some basic performance testing at 10Gbit/s. Based on my finding, I will say at that at 10Gbit/s one does require very beefy hardware to be able push enough PPS across the firewall. Having said that, I'm quite excited to see where things head next - tnsr does look very promising.

tsnr looks very interesting - time will tell, but I'm optimistic that it's a good decision to leverage into all the development that has been done in the linux realm around software defined networking and the cloud in general. That, and just better driver support in general for various NIC's compared to where things are with FreeBSD (not a ding on FreeBSD, it's just a lower priority than Linux for many of the silicon vendors)

with 10gbe - pfsense is really sensitive to a few things - clocks are more important than number of cores, memory speed is also very important, and hyperthreading doesn't help matters at all... with NIC's, the chelsio 520 line seems to be the NIC of choice, followed by intel's 10gbe NIC's.

 

[tman222]

tsnr looks very interesting - time will tell, but I'm optimistic that it's a good decision to leverage into all the development that has been done in the linux realm around software defined networking and the cloud in general. That, and just better driver support in general for various NIC's compared to where things are with FreeBSD (not a ding on FreeBSD, it's just a lower priority than Linux for many of the silicon vendors)

with 10gbe - pfsense is really sensitive to a few things - clocks are more important than number of cores, memory speed is also very important, and hyperthreading doesn't help matters at all... with NIC's, the chelsio 520 line seems to be the NIC of choice, followed by intel's 10gbe NIC's.

Fully agree with you. FreeBSD has great support for Intel and Chelsio based NIC hardware. I'm currently running pfSense baremetal on a Supermicro 5018D-F8NT 1U server with Chelsio T520-SO-CR SFP+ add-on NIC and never had any issues (the server comes with Dual Intel I210 and 4x Intel I350 1Gbit ports, along with 2x Intel SFP+ ports). Performance has been outstanding, but unfortunately the quad core 2.2GHz Xeon D CPU does not not quite have enough horse power to drive a 10Gbit internet load. Based on some testing and back of the envelope calculations I have done, I think the ceiling of the machine is probably around 3 - 4Gbit/s, anything more than that will require a faster CPU. That being said, it's not really much of a concern unless one is planning on doing high bandwidth inter-VLAN routing across the firewall - the majority of consumer internet connections are still 1Gbit/s or less (and even if they are that fast, they are rarely maxed out for an extended period of time).

 

[sfx2000]

Performance has been outstanding, but unfortunately the quad core 2.2GHz Xeon D CPU does not not quite have enough horse power to drive a 10Gbit internet load. Based on some testing and back of the envelope calculations I have done, I think the ceiling of the machine is probably around 3 - 4Gbit/s, anything more than that will require a faster CPU.

Like I mentioned - clocks with pfSense - your numbers are spot on in my opinion - pfSense is SW based routing, so the core has to touch every packet, the faster it can do it and still apply the pf rules and nat in most cases.

With Linux, things can move over to userland drivers, and bypass the kernel completely - memory speed can still be a limit there obviously, and this is interesting as Intel's HEDT platform helps there, on both single core and memory speed, along with updated bus speeds across the PCI-e.

That's the interesting aspect of tnsr...

 

[tman222]

Thanks @sfx2000 - I really appreciate your insight. I have a quick follow up question for you related to NIC tuning and configuration: I've read in several places that it makes sense to disable TCP offloading (TOE) on NIC's that are used within routers (e.g. pfSense). Is this because in a router/firewall like pfSense each packet has to be processed by the CPU anyway, so there is really no point in off loading that work to the NIC (i.e. it just adds extra complexity or latency), or is it mainly for speed reasons (i.e. the CPU can process packets faster than the NIC's ASIC)? Or are there other reasons? Thanks in advance for your help, I appreciate it.

 

[sfx2000]

This used to be a sticky in the forums...

@thiggins - why the change?

 

[RMerlin]

Probably because the OP is 7 years old, and most likely outdated by now.