Upgraded Opnsense Firewall Router hardware

[Christos]

L3 switches help inter-vlan traffic.
Do we have inter-vlan traffic at home or in a small business? Probably not, as we put guests/IoT on a separate network in order to prevent them from communicating with other networks, not helping them with L3 routing.

 

[sfx2000]

I don't think I would trust TP-Link trying L3 switching as their programing code is weak.

For many of the consumer class managed L2/L3 switches - it's all vendor code with an Vendor Wrapper/Skin...

 

[coxhaus]

L3 switches help inter-vlan traffic.
Do we have inter-vlan traffic at home or in a small business? Probably not, as we put guests/IoT on a separate network in order to prevent them from communicating with other networks, not helping them with L3 routing.

If you are happy with that simple of network.

I want a separate networks. I want to be able to share printers. Have my servers in a separate high-speed network. Be able to control access at wire speed. I want the flexibility of an L3 network with no load on my router except outbound traffic which means it is faster with no network bogging it down. This means if I saturate a server segment local maybe doing a backup or restore or whatever it will have no impact on my internet traffic as it will just as fast as ever. My clients are in a separate vlan network working on the internet with no impact whatsoever and my server segment bandwidth is saturated because my server or NAS is backing up from 1 to another. I have to have L3 switching. I am very unhappy in a L2 network as I see the issues that could be fixed using L3 switching.

 

[sfx2000]

If you are happy with that simple of network.

Most home networks are best as a flat network on the LAN side - and if one wants a Guest WLAN, let the router build the VLAN there...

Just saying...

 

[coxhaus]

Most home networks are best as a flat network on the LAN side - and if one wants a Guest WLAN, let the router build the VLAN there...

Just saying...

They work lightly used. Once you push one aspect of networking then everything is affected. Whereas you can network around it using vlans and L3 switching.

I know too much about networking to run a flat network. It is boring to me.

 

[sfx2000]

It is boring to me.

Boring is good - from someone that has also managed carrier grade networks...

 

[coxhaus]

Everybody gets have their own network. Mine is with a L3 switch.

 

[Christos]

I know too much about networking to run a flat network. It is boring to me.

You can play with IPv6 which is more exciting than L3 routing

 

[coxhaus]

You can play with IPv6 which is more exciting than L3 routing

Not to me. I block IPv6 on my network. By the time we are ready for IPv6 there will be something else.

And my Cisco L3 switch is a great place to block IPv6. It supports IPv6 but I don't route it plus I pull it out of the Windows network stack so IPv4 has priority in Windows.

 

[Maverick009]

L3 switching also comes in to play when your outside router LAN port bandwidth becomes saturated. Using L2 slows your network down regardless of processor speed on the router as it is a bandwidth issue that does not happen using a L3 switch.

If you lightly use your local network then L2 will work fine.

I don't think I would trust TP-Link trying L3 switching as their programing code is weak.

Comparing Layer 3 and Layer 2 Switches

This article discusses the difference between layer 2 and layer 3 switches and the appropriate use cases for each.

documentation.meraki.com

Sorry been a moment since I looked and was able to respond due to some health issues and network issues. There is actually no real slow down in L2 switch vs L3 switch. The big difference is in the hardware abilities based on the switch. L2 does great with MAC Address assigning while L3 does it at the IP address level. Also L3 has extra incentives with using multiple VLANs and having intercommunication done, something an L2 Switch cannot fully do. Now if you have a Router, that can do the L2 and L3 capabilities. In my case I have an L3 Switch (Soon to be a mix of L3 and VLAN/L2 Switches) and a Pfsense firewall with 1G/2.5G/10G NICs and the router specs is probably overkill but at the same time, no worries about expandability or what I am limited too.

TPLink was cheap and through Dell at the time I purchased it. My Next major switch upgrades are looking to be between Netgear, Ubiquiti, and a few others. I agree, unless you can find a 3rd party firmware, TP-Links firmware is lacking and, in some areas, weak.

 

[Maverick009]

So I have been talking about it for a while now and finally did a fully upgrade revamp of my Opnsense Firewall. Major upgrade to the firewall that will last for a very long time. I have the Intel NICs also setup in Bridge mode as it made complete sense with my home network and the CPU is more than capable of running it that way. I have 1 port of the 10G NIC running directly into the 10G Aquantia card on my gaming and NAS server that also host my Lancache. Currently have the main feed of my devices running from one of the I350-T4 gigabit ports to a 24-port L3 Switch that route out to the various hardwired devices and to 2 Asus Routers in AP mode for the wireless devices. I will probably be making some changes to the way devices are connected at some point and possibly look at adding multigig 2.5G/5G/10G Switches to the mix within the next year. The difference in hardware alone and specifically the CPUs, sees a nice power reduction and with that a huge heat difference, which is a good thing. As for the OS, Opnsense has been working quite well and really gets the job done and for anyone on the fence, this has been a great project I can speak too as well and wanted to share.

Previous hardware
Intel Q6600 2.4Ghz CPU (95W)
4GB DDR3 Dual-Channel Memory
240G SATA SSD (OS)
Gigabyte G41MT-USB3 motherboard
Intel I350-T4 Quad Gigabit NIC
Dual 2.5G Realtek NIC

Upgraded hardware
AMD Ryzen 5700G 3.8Ghz Base/4.6Ghz (Single Core)/4.2Ghz (All Core) (65W)
Asrock B550M Steel Legend Motherboard
16GB DDR4-3200Mhz Dual-Channel Memory
512GB M.2 NVMe SSD (OS)
2.5G Onboard Realtek NIC (Modem/Internet connection)
1G Intel I350-T4 Quad Gigabit NIC
10G Intel X540-T2 Dual NIC
Opnsense 23.7.6

Been dealing with some health issues, otherwise would of provided an update to this sooner. So all the hardware from the upgrades is the same. The big change was switching out Opnsense for Pfsense. After using it for a little over a year now, I was slowly running into a few issues. Most of them were minor and able to isolate. The straw that broke it, was the compatibility issues I was having with Opnsense and my Unraid server with a Lancache docker on it. The lancache kept running into issues communicating to steam servers and pulling game content onto my lancache for fast acceess and installation of games between my systems. Also had a few issues with the Jellyfin media docker and opnsense too. I also found out through the switch, that my Intel X540-T2 card that I bought and thought may of been DOA, as it was constantly giving errors and problems on Opnsense, actually turned out to be good and works perfectly with Pfsense. I actually have my cable modem plugged into the one 10G port and I have not seen a single in/out error nor any random disconnects, confirming that Opnsenses constant updates, may have broken the Intel drivers. Although Pfsense is slower with updates, it seems to be a better option bringing a little more stability while still giving major enough updates about every 6 months to a year. MY 10G NIC and Lancache have been working perfectly ever since. Now I can get back to gaming and making other hardware upgrades and adjustments.

 

[L&LD]

@Maverick009, I hope you are well on your way to a full recovery.

 

[coxhaus]

Been dealing with some health issues, otherwise would of provided an update to this sooner. So all the hardware from the upgrades is the same. The big change was switching out Opnsense for Pfsense. After using it for a little over a year now, I was slowly running into a few issues. Most of them were minor and able to isolate. The straw that broke it, was the compatibility issues I was having with Opnsense and my Unraid server with a Lancache docker on it. The lancache kept running into issues communicating to steam servers and pulling game content onto my lancache for fast acceess and installation of games between my systems. Also had a few issues with the Jellyfin media docker and opnsense too. I also found out through the switch, that my Intel X540-T2 card that I bought and thought may of been DOA, as it was constantly giving errors and problems on Opnsense, actually turned out to be good and works perfectly with Pfsense. I actually have my cable modem plugged into the one 10G port and I have not seen a single in/out error nor any random disconnects, confirming that Opnsenses constant updates, may have broken the Intel drivers. Although Pfsense is slower with updates, it seems to be a better option bringing a little more stability while still giving major enough updates about every 6 months to a year. MY 10G NIC and Lancache have been working perfectly ever since. Now I can get back to gaming and making other hardware upgrades and adjustments.

Yes, pfsense is using newer drivers as they are on the newer FreeBSD 14 vs FreeBSD 13 for Opensense.

I hope you get well.

 

[coxhaus]

Sorry been a moment since I looked and was able to respond due to some health issues and network issues. There is actually no real slow down in L2 switch vs L3 switch. The big difference is in the hardware abilities based on the switch. L2 does great with MAC Address assigning while L3 does it at the IP address level. Also L3 has extra incentives with using multiple VLANs and having intercommunication done, something an L2 Switch cannot fully do. Now if you have a Router, that can do the L2 and L3 capabilities. In my case I have an L3 Switch (Soon to be a mix of L3 and VLAN/L2 Switches) and a Pfsense firewall with 1G/2.5G/10G NICs and the router specs is probably overkill but at the same time, no worries about expandability or what I am limited too.

There is a difference between L2 switch routing using a router and an L3 switch doing line rate switching.
The L2 data needs to traverse to a router to route from one network VLAN to another one so it is chewing line bandwidth up traversing up to the router and back. If you push your system then you will run out of line bandwidth to the router because it has to go up and back down for routing that is done right in the layer 3 switch from port to port no traversing up to the internet routing which leaves plenty of line bandwidth because the L3 switch is doing all the work with no line bandwidth being used.

And if the upload line going to the router from the L2 switch happens to be on the internet path which more than likely, you will slow down your internet speed using a L2 switch to route L3 traffic to a router whereas you will not using a L3 switch. Yes, if you lightly use your L2 switch you can get away with it.

 

[Maverick009]

@Maverick009, I hope you are well on your way to a full recovery.

Thanks. Yes I am doing better.

 

[Maverick009]

Yes, pfsense is using newer drivers as they are on the newer FreeBSD 14 vs FreeBSD 13 for Opensense.

I hope you get well.

Yeah I saw that they were on BSD 14. I also saw some reports about driver issues with the latest Opnsense builds. Mix that with my network DNS issues and Lancache problems, and it was creating more headaches. Ran a test with Pfsense and even after some reboots and a cleanup of the server/network rack, it still running flawlessly.

Thanks, I am getting better day by day.

 

[Maverick009]

There is a difference between L2 switch routing using a router and an L3 switch doing line rate switching.
The L2 data needs to traverse to a router to route from one network VLAN to another one so it is chewing line bandwidth up traversing up to the router and back. If you push your system then you will run out of line bandwidth to the router because it has to go up and back down for routing that is done right in the layer 3 switch from port to port no traversing up to the internet routing which leaves plenty of line bandwidth because the L3 switch is doing all the work with no line bandwidth being used.

And if the upload line going to the router from the L2 switch happens to be on the internet path which more than likely, you will slow down your internet speed using a L2 switch to route L3 traffic to a router whereas you will not using a L3 switch. Yes, if you lightly use your L2 switch you can get away with it.

Good to know. I figured home network, a mix of cheap L2 and powerful routing with custom hardware makes sense. For me, I still have an L3 Managed Switch, but I know currently, I am not fully utilizing it. With the cleanup of the rack and rerouting cabling, I may start putting more load on the network and adding VLANS. Also so far, all the ports are high-end Intel I350-T4 or X540-T2 on the Pfsense Router (I have the Realtek 2.5G port currently not in any use) with a 5700G 8C/16T Ryzen and 16GB DDR4 memory to feed bandwidth and any burst loads. Overall, I do not think I will saturate the home network even with a few devices at load on the network but agree on dedicated L3 switches for bigger expandability and performance and would use them in any enterprise environment 100%. I did just recently add a 4 Port 2.5G POE capable switch to the network that also has 2 10G SFP ports. Mainly going to use this with my Main desktop and the Gaming-NAS Unraid Server that has the 10G Ethernet card in it. For now, I just have them both routed through the 2.5G ports with 1G uplink connection to the Switch. I will eventually end up getting 2 SFP to Ethernet adapters so I can connect that Server up with a dedicated 10G connection. All in due time.