pfSense (or other dedicated router) questions

[System Error Message]

I did say before that both pfsense and mikrotik would suit you well so im not sulking that you didnt choose mikrotik, just happy you didnt make the wrong choice such as choosing ubiquiti. Mikrotik is more of a dedicated router whereas pfsense is more of a desktop OS focused into a router. While ubiquiti would've let you tinker a lot it doesnt let you tinker at your WAN speeds.

@Nullity, yes linux forums arent noob friendly. Ubuntu still hasnt fixed their mdadm boot loop bug which has been comfirmed over 2 years ago because linux communities are filled with people like "you shouldnt use fake RAID, use mdadm software RAID instead or a dedicated RAID instead". I can only say FU to them because a lot of people dual boot and use fake RAID as all other OSes just need chipset drivers and play well for booting from fake RAID and also caused significant delays to my project. That fake RAID 1 would save you from data loss if one drive failed and you would still be running normally. Without fake RAID getting an OS to boot would be more complicated and you couldnt dual boot or share a RAID 1 data partition between different OSes. Ubuntu's reply is, use another drive for ubuntu, i mean try telling that to a home user that he needs to buy another hard drive or to someone whos server's drive slots are already full with no more drive bays/space available for more drives.

Essentially linux like ubuntu have turned toxic because of their arrogance which is why linux popularity have dropped. Its become like the medieval ages where you have clans and racism.

 

[garyd9]

Essentially linux like ubuntu have turned toxic because of their arrogance which is why linux popularity have dropped. Its become like the medieval ages where you have clans and racism.

LOL. I still consider ubuntu to be "entry level" linux. Real men use Gentoo and compile everything themselves. (Although, to be completely fair, ubuntu is a really quick/easy system to spin up a VM for short-term use.)

I usually google to death before asking questions, because I can't stand when I get non-answers, useless answers, lectures (novels), etc. Sometimes, the answers I get from google searches aren't definitive enough, so I end up asking... In this case, however, I'm not doing as much research beforehand as I should due to time constraints: I only have 1 week left to get this Asus AC3200 returned to amazon. (For an AP, I'll be using a netgear R7000 with tomato firmware.. that fully supports EAP-PEAP, guest networks, and separate VLANs for each.)

Switching gears a bit...

Confirming that I like to do things backwards, today I downloaded the pfsense ISO image and installed it in a VM under hyper-V. After playing for about an hour, I think I made a large mistake in my planning for the new system: I only took 1 day off work to play with it. It's going to take me at least 2-3 days to tinker with everything (and then re-install a few times after I learn from some mistakes.)

This thing has TONS of dials, switches and buttons to play with.. even before installing packages! Oh, and I'll also get to play with this "IPMI" thing that the supermicro board has. I might end up having to call in sick (cough, cough) Friday after taking Thursday off. That, along with the weekend, should be enough time to at least gain competency with pfSense.

 

[System Error Message]

99% of the time at snbforums you will get an answer and i am sure the members with highest post counts (except me, RMerlin and thiggins) would be happy to help you with pfsense. For me is because i dont have much experience with pfsense lol as it turned out all my intel boards (1st gen intel iseries xeon based) were unreliable and none of them worked.

 

[sfx2000]

Essentially linux like ubuntu have turned toxic because of their arrogance which is why linux popularity have dropped. Its become like the medieval ages where you have clans and racism

Ubuntu is still fairly approachable - but they've had their issues - and that's that

It is a "go to" distro for some things, and I use here on several platforms at the house, but I'm ok with debian, arch, gentoo, and of course the RedHat universe of releases (Fedora/RHEL/CentOS, etc...)

I wouldn't put Linux into the same general category though - there's much variety there - and there's millions of people using Linux that don't even know they are - everything from ChromeBooks to Android to various emdedded linux devices...

 

[sfx2000]

Mikrotik is more of a dedicated router whereas pfsense is more of a desktop OS focused into a router. While ubiquiti would've let you tinker a lot it doesnt let you tinker at your WAN speeds.

pfSense is a router focused build - just like VyOS, RouterOS, OpenWRT, and many others...

 

[garyd9]

pfSense is a router focused build - just like VyOS, RouterOS, OpenWRT, and many others...

I'm sorry... what color (colour?) did you say the bikeshed was?

 

[sfx2000]

I'm sorry... what color (colour?) did you say the bikeshed was?

I don't care - as long as it holds bikes

 

[garyd9]

I don't care - as long as it holds bikes

Oh.. no.. What do you consider a REAL bike? (lol.. sorry.. couldn't help myself.)

 

[garyd9]

Oh.. wow. I just realized that pfSense can do bridging over interfaces. I haven't explored it completely, but I'm hoping that means it can also bridge VLAN interfaces.

Why is this important?

As I'm sure most people know, many network services don't quite work properly when devices are on different subnets. Two examples: if my two TiVo DVR's are on different subnets, they can't find each other (and avahi doesn't help.) Also, if my kids have friends over to play minecraft PE, they can't play together if their devices are on different subnets. A lot of "consumer" stuff works via network broadcast.

This always created a security concern for me... My kids use EAP to get on wifi and therefore have access to some "core" things (such as network file storage.) On the other hand, I don't want the friends of my kids popping on the "guest" wifi also having access to the NAS.

In the past (pre-Asus), I was able to use bridge filtering to accomplish this. If the AP/Router assigned "wl0.2" to the guest wifi network, I could use ebtables to block IPv6 completely, and then block access to all IP's < *.128. (192.168.1.0 / 25)

# block all ipv6 traffic to/from the guest wifi as ipv6 routing might get around restrictions
ebtables -I FORWARD -i wl0.2 -p ipv6 -j DROP
ebtables -I FORWARD -o wl0.2 -p ipv6 -j DROP
# block all traffic to/from internal ipv4 address less than 128.  Non-reserved DHCP pool starts at 128
ebtables -A FORWARD -i wl0.2 -p ip --ip-dst 192.168.1.0/25 -j DROP
ebtables -A FORWARD -o wl0.2 -p ip --ip-src 192.168.1.0/25 -j DROP

This wouldn't work with the Asus router as the broadcom "CTF" skips and ignores ebtables for non-ARP packets. Without CTF, the AP/Router is CPU limited on routing.

Anyway, I've never really liked this solution. It required that I was careful to make DHCP reservations for any "core" machines < 128, and I've always felt I was becoming too dependent on a specific IP range/network. Changing to a different network address would be a nightmare in scripts, etc.

Enter pfsense: My plan (hope) (and this still a thought in progress) is to bring up a pure AP (netgear R7000 running tomato in AP only mode) and assign each class of SSID to a different vlan. The guest SSID's would get, for example, vlanGuest, I'd then have a vlan for "core" devices (that shouldn't be accessed by guests) and then another for normal devices (that are authenticated, but not hidden from guests.)

On pfsense, I could bridge vlanGuest, vlanCore and vlanNormal, but insert filtering to block certain traffic from being bridged.. Basically, the exact same thing I did with ebtables above (but the filtering would be for interfaces instead of IP masks.) This would effectively insert a transparent firewall between the vlanGuest and vlanCore. Everyone could be on the same network address, so broadcast traffic would still work properly!!

Normally, it might be a concern having the router also act as a bridge related to the primary network, but with the hardware I'm putting into the pfSense box, it SHOULD be able to handle things.

My only (real) concern with this idea is that if the router goes down, it'll take the bridge down with it... which would cripple the LAN. I don't think my switch (netgear GS724Tv4) can do bridging like that. It's only a L2 managed switch.

Something like this:
https://docs.google.com/drawings/d/1kVDYNT-89ClvtnZVR4yDIKA55a_TlPSJHiAtBGgpXf4/edit?usp=sharing

 

[sfx2000]

Oh.. wow. I just realized that pfSense can do bridging over interfaces. I haven't explored it completely, but I'm hoping that means it can also bridge VLAN interfaces.

Why is this important?

As I'm sure most people know, many network services don't quite work properly when devices are on different subnets. Two examples: if my two TiVo DVR's are on different subnets, they can't find each other (and avahi doesn't help.) Also, if my kids have friends over to play minecraft PE, they can't play together if their devices are on different subnets. A lot of "consumer" stuff works via network broadcast.

This always created a security concern for me... My kids use EAP to get on wifi and therefore have access to some "core" things (such as network file storage.) On the other hand, I don't want the friends of my kids popping on the "guest" wifi also having access to the NAS.

pfSense, RouterOS (Microtik) and I believe the EdgeRouter's are all capable of this - with assigning VLAN's to SSID's, a consumer AP might be able to do this - OpenWRT can if I recall, and perhaps DDWRT, not too sure about factory firmware releases...

Airports do assign VLAN1003 to the "guest" network SSID, as that is how they "extend" the guest networks to additional AP's in a homogenous Airport roaming network - and I've used this in the past in conjunction with a managed switch.

 

[System Error Message]

You can have devices on different subnets still being able to communicate without bridging by using routing as routing is the layer 3 equivalent of bridging. However by routing you will need IGMP to forward those layer 2 stuff.

You can have 2 unswitched/unbridged interfaces on the same subnet. Essentially you need to assign IPs to both interfaces within the same network and make sure that no 2 devices get the same IP address. RouterOS will let you do this easily as will a linux/unix distro but ubiquiti wont as they have purposely restricted this.

 

[sfx2000]

However by routing you will need IGMP to forward those layer 2 stuff.

Specifically the IGMP proxy is what can be used to handle the multicast group across two subnet's - RouterOS, pfSense, and of course, most managed switches can handle that part without much trouble...

Quick tip - before hooking everything up - design it on paper first to get the design right for a setup like this, and each network might be a little bit different - but drawing things out will be very useful when troubleshooting... memory sucks

 

[garyd9]

Quick tip - before hooking everything up - design it on paper first to get the design right for a setup like this, and each network might be a little bit different - but drawing things out will be very useful when troubleshooting... memory sucks

I did.. well, not on paper, but on google docs. (See the link in my post above.) It's still a "I wonder if this will work out" type thing at the moment, however.

To start, I'll just put everything on the network on a default (unspecified) vlan without any rules (and leaving the guest wifi turned off.) After I get the initial setup working, I'll be able to start experimenting with different ideas so I at least know what might be possible (and what might not be.)

However by routing you will need IGMP to forward those layer 2 stuff

IGMP (and IGMP forwarding) is something I'm not familiar with. I know my L2 switch has something called IGMP snooping, but I never played with it. If it can transparently forward broadcasts across subnets (and vlans that are routed to each other) then it might be a better solution than bridging. (Time to hit google...)

 

[mtganzer]

This thing has TONS of dials, switches and buttons to play with.. even before installing packages! Oh, and I'll also get to play with this "IPMI" thing that the supermicro board has. I might end up having to call in sick (cough, cough) Friday after taking Thursday off. That, along with the weekend, should be enough time to at least gain competency with pfSense.

Does Supermicro IPMI still require running an unsigned Java applet (which Oracle JRE now blocks) to get console access? Their "fix" (at least for my Haswell-Xeon MB) is a version of ipmitool wrapped with an older insecure JRE version. IPMI is one of those tools that is really nice when you need, it, but it really needs to have very strict access controls.

 

[garyd9]

Does Supermicro IPMI still require running an unsigned Java applet (which Oracle JRE now blocks) to get console access? Their "fix" (at least for my Haswell-Xeon MB) is a version of ipmitool wrapped with an older insecure JRE version. IPMI is one of those tools that is really nice when you need, it, but it really needs to have very strict access controls.

It is wrapped in a JRE that's probably out-dated. I certainly wouldn't leave the IPMI tool running on a machine "in the wild", but it's suitable for a machine that is as physically secure as the machine that supports IPMI to begin with.

 

[sfx2000]

It is wrapped in a JRE that's probably out-dated. I certainly wouldn't leave the IPMI tool running on a machine "in the wild", but it's suitable for a machine that is as physically secure as the machine that supports IPMI to begin with.

ILO (Integrated Lights Out) management interfaces (vendors call these different things, HP refers to them as ILO, IBM for example, calls them IMM in non-blades, and AMM in blade centers) - these are never intended to be exposed to the public internet - in a data center environment, they're usually put on a separate LAN/VLAN outside of the primary traffic, esp, if the primary traffic is facing outside, but best practices is to keep them out of band from the primary traffic in any event.

And yes, the ongoing Java issues, which are primarily the KVM functionality, whether the applet is unsigned (common problem), older versions (common problem), outdated manifests (common problem), or expired certificates (again, common problem)....

Got to a point where I just kept an old WinXP VM handy to manage my lab development servers (tip from my Ops teams that were dealing with similar issues).

Now for the positives - IPMI is a two part system, the host, and an agent that can run on the primary machine - and that agent can send data pretty much anywhere - e.g. to a centralized box even where one is monitoring hundreds of servers...

With any monitoring agent - whether it's nagios, xymon, snmp, ipmi, etc... one has to be aware that they are little services themselves - and protect them accordingly - many of these are not very sophisticated with authentication or encryptions (some do have basic options like snmp v3, but that's not common in use)...

 

[garyd9]

To those familiar with pfSense... I feel like I'm missing something obvious here, but how can I set up a rule based on a MAC address?

For example, let's say I want to block MAC address A:B:C:E:F from any internet (LAN <> WAN) traffic. I prefer not using IP addresses for a couple reasons:
1. I'd have to set up multiple rules for IPv4 and IPv6.
2. IP addresses change... between IPv4 dynamic assignments and Windows constantly rotating IPv6 addresses, there's no constant.
3. Even if not for #1 and #2, most non-techies find it much easier to spoof an IP address than to spoof a MAC address.

However, I can't find a way to do MAC filtering in the pfSense firewall... surely I'm missing something obvious. I hope.

 

[garyd9]

To those familiar with pfSense... I feel like I'm missing something obvious here, but how can I set up a rule based on a MAC address?

Okay... nevermind. I just did a search on the pfsense forum and apparently pfsense doesn't support it. The amusing part is that TONS of people ask about it, and the typical response ranges from "why would you want to do that?" to actively attacking the person asking for wanting The All-Mightly PFSense to do Something Trivial.. (sarcasm added by me, obviously.)

Powerful as it might be, I have to admit that the attitudes on the pfsense forums are pretty High and Mighty. I found several different topics in searching for how to do something... a user would ask a question, and a long-term "hero member" (often the same one) would attack the user for wanting to do something so stupid and telling them that they are wrong for wanting something like that (and never bothering to answer the question... most likely because they were just as clueless as the user.)

Back to the MAC filtering.. the predominate response on the pfsense forum seems to be that pfsense isn't a L2 device, so shouldn't be dealing with MAC addresses. It's a shame that none of those people have stopped to consider that: First, most L3 devices also have L2 functionaly. Second, that making a DHCP reservation for every single device is a PITA and is useless when most modern equipment will actually use IPv6 in preference to IPv4 if it's available (and that DHCPv6 is next to useless these days for reservations.)

Wish me luck in the battles I'm about to rage over there...

https://forum.pfsense.org/index.php?topic=115949.0

Take care
Gary

 

[sfx2000]

Okay... nevermind. I just did a search on the pfsense forum and apparently pfsense doesn't support it. The amusing part is that TONS of people ask about it, and the typical response ranges from "why would you want to do that?" to actively attacking the person asking for wanting The All-Mightly PFSense to do Something Trivial.. (sarcasm added by me, obviously.)

Powerful as it might be, I have to admit that the attitudes on the pfsense forums are pretty High and Mighty. I found several different topics in searching for how to do something... a user would ask a question, and a long-term "hero member" (often the same one) would attack the user for wanting to do something so stupid and telling them that they are wrong for wanting something like that (and never bothering to answer the question... most likely because they were just as clueless as the user.)

Back to the MAC filtering.. the predominate response on the pfsense forum seems to be that pfsense isn't a L2 device, so shouldn't be dealing with MAC addresses. It's a shame that none of those people have stopped to consider that: First, most L3 devices also have L2 functionaly. Second, that making a DHCP reservation for every single device is a PITA and is useless when most modern equipment will actually use IPv6 in preference to IPv4 if it's available (and that DHCPv6 is next to useless these days for reservations.)

Wish me luck in the battles I'm about to rage over there...

https://forum.pfsense.org/index.php?topic=115949.0

Take care
Gary

Routing is IP Layer - and pfSense is very good at it...

MAC layer, or more succinctly perhaps in your case, ethernet layer, this is a switching function - and pfSense does not do switching... it has some VLAN capability, goes without saying...

pfSense with a managed switch behind it - you've got an incredible amount of flexibility here...

uTik's - some of them are Router/Switches, so one has a single place to configure both...

pfSense isn't for the uneducated (not saying dumb) that do not understand networking fundamentals... and that perhaps is an unsettled expectation with many Router only devices in the SOHO space - one sees all those ports and treats them like switch ports, when they're actually not...

That's the nice thing with the all-on-one consumer AP's - they've do the homework, and in 99 percent of the cases, the solution works as everything is pre-configured to do so...

 

[sfx2000]

So getting smart on networking is a bit like gaming... think about Mario...

You move forward, get some coins (nuggets of knowledge), learn how to jump, bounce the bad guys - and then you hit the boss level - it's still the same rules, but there's a barrier in that not all the rules are known when you're starting out in the game - so it's a barrier...

Figure out the unknown rules - and you beat the boss... and level up to the next one...

Seems like you're at that Boss level, and you're fighting that barrier of understanding - and that's ok... you're a smart guy and you'll sort it - until you hit the next boss level, and you'll sort that one as well...

Most folks don't get beyond level 1 as they don't need to - the all-in-one consumer routers have already sorted that problem. But if you can get past it, and you can - you'll be better off...

How's that for some advice from a pfSense user and a data center guy that does a bit more complicated solutions... and I've been there - know where you're at... push thru that barrier, beat the boss...