Using pfSense with a L3 core switch

[unmesh]

I've been running pfSense with a TP-Link T1600G-28TS in managed L2 mode as my core switch for a while now where interVLAN routing including mDNS is managed by the pfSense box. Since that switch has L3 capabilities, I'd like to learn how to use them should the day arise when video streams need to cross between VLANs and the pfSense struggles with it.

From several posters here, it seems that pfSense does not play well with L3 switches and that extreme care should be taken setting up a network with L3 switches. Where could I read more about the issues and possible solutions?

If this is too dangerous to attempt in my "production" home network, I could run pfSense on my ESXi server and could borrow a Juniper EX2300-C-12P L3 switch to create an isolated homelab network and use that as the learning vehicle.

Though I may soon realize that this is too much work to solve what is today a non-existing problem

Thanks

 

[coxhaus]

I suggest you buy a Cisco SG350 or a SG350X switch if you want help. I have already posted how to do it with examples. Just follow the examples. I would not count on TP-Link really working. The 1 device I used of TP-Link the coding was not very good.

 

[Tech9]

Though I may soon realize that this is too much work to solve what is today a non-existing problem

You most likely will. What hardware your pfSense firewall runs on? On a home network for routing between VLAN's I would rather increase the bandwidth between the firewall and the switch. Anything i5 2nd Gen and above can route multi-gigabit, if you have proper NIC's to transport the bits in and out.

 

[unmesh]

It is only a fanless Intel 1.5GHz 2-core Rangeley box with two i354 integrated NICs and two i211 NICs.

iperf3 shows it can forward at about 900Mbps between any two ports or between VLANs on the same port. Lightweight pfBlockerNG-devel config running but not much else.

CPU is usually at 15%

 

[unmesh]

I suggest you buy a Cisco SG350 or a SG350X switch if you want help. I have already posted how to do it with examples. Just follow the examples. I would not count on TP-Link really working. The 1 device I used of TP-Link the coding was not very good.

I did a search for your postings on these switches but could not find the ones with the examples. Since the CLIs are supposedly similar between Cisco and Juniper, it would be great if I could read them and see if the instructions map over easily to a Juniper EX that I wouldn't have to buy.

Thanks

 

[coxhaus]

You most likely will. What hardware your pfSense firewall runs on? On a home network for routing between VLAN's I would rather increase the bandwidth between the firewall and the switch. Anything i5 2nd Gen and above can route multi-gigabit, if you have proper NIC's to transport the bits in and out.

You know using a router is not efficient. If you want traffic from port 1 to go to port 10 on a different network in the same switch using layer 2 you send all traffic out port 1 to the router so it can tag the traffic with the network ID and then send it back to the switch on port 10. The traffic is going out a 1 gig port with all out bound traffic going to your internet and coming back down to the switch. This is slow and has more latency than using layer 3 where the L3 switch just switches the traffic straight to port 10 on a high-speed backplane.

 

[wolf5150]

I suggest you buy a Cisco SG350 or a SG350X switch if you want help. I have already posted how to do it with examples. Just follow the examples. I would not count on TP-Link really working. The 1 device I used of TP-Link the coding was not very good.

Where would I find the examples you referenced in your post?

 

[Tech Junky]

@wolf5150

Don't buy into the hype he's spouting about Cisco vs anything else. Just find a switch that meets your needs. Start your own thread with the requirements you're looking for and we'll be able to advise you better that way.

 

[coxhaus]

Yes, they don't understand layer 3 switching.

 

[Tech Junky]

they

Who is they?

I've worked on nationwide networks and data centers. I think I have a pretty good grasp on networking to the point where I've went head to head with a 5 x CCIE that couldn't figure out a simple IP subnet issue. There's a difference between operations minded people and conceptual dullards.

 

[sfx2000]

From several posters here, it seems that pfSense does not play well with L3 switches and that extreme care should be taken setting up a network with L3 switches. Where could I read more about the issues and possible solutions?

If this is too dangerous to attempt in my "production" home network, I could run pfSense on my ESXi server and could borrow a Juniper EX2300-C-12P L3 switch to create an isolated homelab network and use that as the learning vehicle.

Use the Juniper switch you have on hand, it's more than sufficient for your L3 tasks.

pfSense plays just fine with L3 switches, just need to ensure that you're using the right components for the purpose at hand - the switch likely will be all you need for simple VLAN's and possibly access control lists

 

[coxhaus]

Did you ever get layer 3 switching working with pfsense? I just set up pfsense again using my Cisco L3 switch so I could test pfsense 23.01 using FreeBSD 14 and my 10 gig NIC. It took about 45 minutes to set it up.

If you are using trunking on your L3 switch to pfsense then your switch is not doing layer 3 switching. pfsense is doing the L3.

 

[coxhaus]

Well, it turns out if you change NICs in pfsense 23.01 you need to register again and get a new token. I added my 10 gig Broadcom NIC after upgrading to 23.01. There were no NIC drivers for my 10-gig card in pfsense 2.6 CE so I started with an Intel gig 2 port card. I then changed NIC cards after upgrading to version 23.01.

I also read the gateway names in pfsense need to be all capitals no lower case. I had to redo that. I had to delete static routes and then the gateway as you cannot change just the gateway name. You have to set it up again.

I have another issue where when I change the WAN interface for IPv6 to none instead of DHCP my internet access quits for IPv4 network local to pfsense. My L3 switch vlans still have internet access. I have not tried it again since I made the above changes. I am waiting for my wife to leave to break the network.

I also implemented power D in pfsense to where the cpu will self-regulate cpu speed based on load. I think it uses the speed steps in the cpu. It saves a little heat and power for home users. Current: 800 MHz, Max: 3400 MHz

I also switched from unbound DNS to DNS forwarding DNS to QUAD9 9.9.9.9

 

[avtella]

Well, it turns out if you change NICs in pfsense 23.01 you need to register again and get a new token. I added my 10 gig Broadcom NIC after upgrading to 23.01. There were no NIC drivers for my 10-gig card in pfsense 2.6 CE so I started with an Intel gig 2 port card. I then changed NIC cards after upgrading to version 23.01.

I also read the gateway names in pfsense need to be all capitals no lower case. I had to redo that. I had to delete static routes and then the gateway as you cannot change just the gateway name. You have to set it up again.

I have another issue where when I change the WAN interface for IPv6 to none instead of DHCP my internet access quits for IPv4 network local to pfsense. My L3 switch vlans still have internet access. I have not tried it again since I made the above changes. I am waiting for my wife to leave to break the network.

I also implemented power D in pfsense to where the cpu will self-regulate cpu speed based on load. I think it uses the speed steps in the cpu. It saves a little heat and power for home users. Current: 800 MHz, Max: 3400 MHz

I also switched from unbound DNS to DNS forwarding DNS to QUAD9 9.9.9.9

Yeah PowerD is disabled on custom installs by default for compatibility sake, I turn it on as well. It’s enabled by default on Netgate hardware. Also make sure to disable the DNS override in general settings otherwise it will use the one from ISP.

Also have you tried pfblocker and ntopng on it?

 

[coxhaus]

So, I made progress on being able to change IPv6 to none instead of DHCP on the WAN interface. My wife left so I started playing with my firewall after the changes last night when I could not sleep. It turns out blowing away my static routes and gateway caused my default gateway to point to my L3 switch which was not the case before changing my gateway to all caps and redefining them from initial setup because I had looked before.

Any way it is now a simple change changing the default gateway to point to my WAN interface for IPv4. I have successfully turned off IPv6. IPv6 on the WAN interface is now set to none.

avtella, I am not going to load any packages until I get everything working my way. And probably I will wait for 23.05 which should be out in a couple of weeks. I am most comfortable using port 53 for my DNS so TLS is out for me. I plan at some point to load SNORT. If I load SNORT do, I really need pfblocker?

 

[avtella]

Glad things are working for you now.

Probably don’t need pfblocker, with snort running though some do use both together as one blocks traffic based on origin/exit and the other based on heuristics from my understanding though there some overlap. Feel free to correct me if I’m wrong.

Let me know how Snort goes, last time I used it was like 2 years ago with some helpful videos from Lawrence tech. You probably have a better understanding/patience than me to properly optimize the rules/suppression list to your needs to enable blocking mode.

 

[ddaenen1]

I also switched from unbound DNS to DNS forwarding DNS to QUAD9 9.9.9.9

Very interested to understand why you chose DNS forwarded instead of using unbound?

 

[ddaenen1]

I plan at some point to load SNORT. If I load SNORT do, I really need pfblocker?

I have been running pfBlockerNG for a couple years now. I am quite happy with it. Foremost, it is much easier to set up than SNORT (which i tried for several weeks) and SNORT requires much more time to tune and tweak whereas pfBlockerNG works almost out of the box with IPV4 and DNSBL. Maybe in the end SNORT can do more but for now, i am still contemplating if i need more security such as IDS/IPS. For now, i am not convinced yet. Maybe it will come one day.

 

[coxhaus]

Very interested to understand why you chose DNS forwarded instead of using unbound?

I want QUAD9 to be totally responsible for DNS.

 

[coxhaus]

I have been running pfBlockerNG for a couple years now. I am quite happy with it. Foremost, it is much easier to set up than SNORT (which i tried for several weeks) and SNORT requires much more time to tune and tweak whereas pfBlockerNG works almost out of the box with IPV4 and DNSBL. Maybe in the end SNORT can do more but for now, i am still contemplating if i need more security such as IDS/IPS. For now, i am not convinced yet. Maybe it will come one day.

With pfblocker you are still running blind. You set a bunch of rules and hope for the best. SNORT is showing real time what is happing. I know SNORT is much more involved as I spent 2 weeks setting it up once and I could have spent more time on it. I plan to work on and off with SNORT as I have time. We will see how it works.