Another firewall advice

[L&LD]

My network behaves as it should and as it's expected to behave (i.e. it is in a good/known state).

If I introduce a client device that makes this network misbehave, it's the client that is dumped, not the network.

Using the internet to me is having an unlimited reading supply of information. It is not turning on/off a lightbulb that is 5" from me or 500 miles away.

 

[coxhaus]

This is completely unrelated. The data still has to travel over Internet.

Not if you block it at the firewall. I don't use cloud-based products.

The only access is my AppleTV. I have an 85-inch Sony TV and it has no internet access. If at some point if I need a firmware update it will be done manually through a 1 time access. Same thing with cameras and EVO light switches. My Bluetooth JBL Party box 100 speakers run off my AppleTV with no cables as they use Bluetooth and stereo Bluetooth between speakers. These are the speakers for my 85-inch Sony TV. They fill my large living room with loud sound. I use a lot of Bluetooth. I am sure there is more.

My Bluetooth EVO light switches only work when you are within range which does not even cover all my house. The back half of my house covers some and the front half covers the other. Very short range.

 

[Tech9]

What’s the best solution please ? You can’t say no IoT!

You figure it out. My own home automation doesn't need Internet.

 

[SSri]

You figure it out. My own home automation doesn't need Internet.

Sure. Will do.

I only asked as you commented “not best but works”!

Thx for all the inputs guys.

 

[ifisher]

I spent some time recently evaluating home firewalls including SimpleWall, opnSenses, pfSense, Untangle, and Sophos XG.
I also used ClearOS in the past but had some challenges with it being overly disruptive (when configured to actually provide decent security).

I have a lot of enterprise security experience and work/ have worked with Fortinet, CheckPoint, PAN, ForcePoint, Cisco, Sophos, and Dell/SonicWall professionally in the last 20+ years.

In my opinion, if you're looking for a simple home firewall with a great user interface and don't care much about security - Untangle is great. Super simple to setup and configure, low resource usage, quite fast. However, its IPS is useless in protecting against HTTP evasion, which means it can easily be bypassed. (Google HTTP evasion and run the EICAR test on your setup to see for yourself) and the subscription costs are kinda high for a home product.

I also have experience with the Ubiquiti UDM Pro but while it has a lot of nice features (especially for the price) - it doesn't have an Anti-Virus, its IPS is very limited, and it can't do TLS decryption - so it's more of a router than a firewall.

Out of all the home firewalls I tested - Sophos was the only one who actually provided protection against evasion attempts. It actually did better out of the box than some commercial firewalls I tested as part of my work.

pfSense and ClearOS failed the evasion test but did better than Untangle and opnSense, etc. I think it might be possible to get pfSense to pass but it would take a lot of fiddling.

Balancing security without sacrificing usability and performance is not simple. Even some of the best enterprise solutions can struggle.

The other thing to look for is the ability to do SSL/TLS decryption and make sure TLS 1.3 doesn't get downgraded in the process.
With that, Sophos is better too but Untangle is actually quite good at providing flexible configuration.
With SSL/TLS decryption you need to take into account that it's:
1. Very resource intensive
2. Breaks a lot of applications (due to certificate pinning)
So this has to be setup correctly (build a smart policy to use it sparingly).

To me, the importance of good event/traffic logging (for troubleshooting), good dashboard/reporting to gain quick visibility into the security posture are integral part of any security solution. Again, Sophos and Untangle are better at it. With some of the others you can get close but it will take a lot of time and tinkering. Some are just too simplistic in their approach and really limit your ability to drill down the logs. (I'm spoiled after years of using CheckPoint, Palo-Alto, and Forcepoint who have amazing logging/reporting capabilities).

Finally, there's the issue of QoS and ensuring the router can provide a high MOS for VoIP, low latency jitter on load, and prioritize traffic without making you spend days writing QoS rules. In addition, as QoS processing can conflict with fast-path (CTF/NAT acceleration) - it's important to test your choice of firewall and make sure it can actually deliver the bandwidth you need on the hardware platform you run it on. With that I did notice that Sophos struggled a little on my test VM (1 Gig fiber connection) but it could have been the VM implementation.

I'm going to do a little more research into other possible firewall solutions and test them - but I will also install Sophos and test on my bare metal machine: Core i5-8500, 8GB RAM, 256G SATA3 SSD. Hopefully it will be powerful enough to provide close to 1G with all protections enabled (but using SSL decryption sparingly, of course) especially since Sophos limits the home XG to 4 cores 6G RAM.

If anyone here has another suggestion for me to test, I'd be happy to give it a try and share my impression.

 

[SSri]

I spent some time recently evaluating home firewalls including SimpleWall, opnSenses, pfSense, Untangle, and Sophos XG.
I also used ClearOS in the past but had some challenges with it being overly disruptive (when configured to actually provide decent security).

I have a lot of enterprise security experience and work/ have worked with Fortinet, CheckPoint, PAN, ForcePoint, Cisco, Sophos, and Dell/SonicWall professionally in the last 20+ years.

In my opinion, if you're looking for a simple home firewall with a great user interface and don't care much about security - Untangle is great. Super simple to setup and configure, low resource usage, quite fast. However, its IPS is useless in protecting against HTTP evasion, which means it can easily be bypassed. (Google HTTP evasion and run the EICAR test on your setup to see for yourself) and the subscription costs are kinda high for a home product.

I also have experience with the Ubiquiti UDM Pro but while it has a lot of nice features (especially for the price) - it doesn't have an Anti-Virus, its IPS is very limited, and it can't do TLS decryption - so it's more of a router than a firewall.

Out of all the home firewalls I tested - Sophos was the only one who actually provided protection against evasion attempts. It actually did better out of the box than some commercial firewalls I tested as part of my work.

pfSense and ClearOS failed the evasion test but did better than Untangle and opnSense, etc. I think it might be possible to get pfSense to pass but it would take a lot of fiddling.

Balancing security without sacrificing usability and performance is not simple. Even some of the best enterprise solutions can struggle.

The other thing to look for is the ability to do SSL/TLS decryption and make sure TLS 1.3 doesn't get downgraded in the process.
With that, Sophos is better too but Untangle is actually quite good at providing flexible configuration.
With SSL/TLS decryption you need to take into account that it's:
1. Very resource intensive
2. Breaks a lot of applications (due to certificate pinning)
So this has to be setup correctly (build a smart policy to use it sparingly).

To me, the importance of good event/traffic logging (for troubleshooting), good dashboard/reporting to gain quick visibility into the security posture are integral part of any security solution. Again, Sophos and Untangle are better at it. With some of the others you can get close but it will take a lot of time and tinkering. Some are just too simplistic in their approach and really limit your ability to drill down the logs. (I'm spoiled after years of using CheckPoint, Palo-Alto, and Forcepoint who have amazing logging/reporting capabilities).

Finally, there's the issue of QoS and ensuring the router can provide a high MOS for VoIP, low latency jitter on load, and prioritize traffic without making you spend days writing QoS rules. In addition, as QoS processing can conflict with fast-path (CTF/NAT acceleration) - it's important to test your choice of firewall and make sure it can actually deliver the bandwidth you need on the hardware platform you run it on. With that I did notice that Sophos struggled a little on my test VM (1 Gig fiber connection) but it could have been the VM implementation.

I'm going to do a little more research into other possible firewall solutions and test them - but I will also install Sophos and test on my bare metal machine: Core i5-8500, 8GB RAM, 256G SATA3 SSD. Hopefully it will be powerful enough to provide close to 1G with all protections enabled (but using SSL decryption sparingly, of course) especially since Sophos limits the home XG to 4 cores 6G RAM.

If anyone here has another suggestion for me to test, I'd be happy to give it a try and share my impression.

Wow! That’s a thorough review analysis. I do not recollect, from my memory, anyone putting Sophos a bit ahead of the pack. Did you test them on a VM or a bare metal? We would appreciate if you could test and post your findings of PfSense, Untangle and Sophos on bare metal machines.

So, your recommendation for me is Sophos Home XG? What about IDS/IPS — Suricata — on Sophos please ? This is important for me.

While 4 cores are enough, I do not understand the 6GB RAM limit.

I had almost placed the order for Optiplex, one of the 6 cores machines; I had asked about the warranty, they said it is voided if I the OS is removed from machine. I did not after they confirmed it in writing. So, I am back to square one.

Thx!

Edit:!

When did you test them please ? I think this is important as they may have been performance improvements.

 

[coxhaus]

I have heard good things about Sophos but at the time of my testing they had just got bought and it was a slow change over so I never tested it. Just peeking at it seemed more difficult to setup over Untangle.

 

[ifisher]

Wow! That’s a thorough review analysis. I do not recollect, from my memory, anyone putting Sophos a bit ahead of the pack. Did you test them on a VM or a bare metal? We would appreciate if you could test and post your findings of PfSense, Untangle and Sophos on bare metal machines.

So, your recommendation for me is Sophos Home XG? What about IDS/IPS — Suricata — on Sophos please ? This is important for me.

While 4 cores are enough, I do not understand the 6GB RAM limit.

I had almost placed the order for Optiplex, one of the 6 cores machines; I had asked about the warranty, they said it is voided if I the OS is removed from machine. I did not after they confirmed it in writing. So, I am back to square one.

Thx!

Edit:!

When did you test them please ? I think this is important as they may have been performance improvements.

So I tested over the last couple of weeks running on a VMWare ESXi 7 host. It's running on an old HP Proliant DL380 G7 (Xeon 5600 series, dual 6 cores, 12-threads CPU, with 32GB RAM, and 6 x 1Gbe interfaces. The 'WAN' side was connected to a Cogent 1Gbit Fiber DIA. Tests were done after hours when line utilization was less than 1%.

"Firewall" machine was configured with 8 Cores, 8GB RAM running VMXnet3 interfaces
"Client" machine was a Windows 11 x64, 10 Cores, 12GB RAM running VMXnet3 interface

Since this is an older server, you can't really get more than 600-700Mbit on VM. Although I verified the WAN circuit can deliver a solid 920Mbit (when directly connecting a Linux laptop to the vLAN).

As for the testing, most home firewalls distros were able to be pretty close to direct WAN connection on the VM (so ~600Mbit). Sophos was only able to get to 350 down (upload wasn't affected) but it only had 4 cores to use (license restriction) and I set it to scan everything. I'm also planning to do some additional testing using a Linux VM as my client and dial down the settings a bit.

I didn't really spend time taking notes on each platform - I just got it configured and if it failed the evasion testing, I moved on to the next solution. I haven't gotten to using the bare metal PC. I want to finalize all testing on VM first. I'm going to give ClearOS another shot since I last used in 2 years ago.

Once I finish my VM testing I'm going to use an old HP prodesk SFF with dual 10GbE hooked up to my home Google Fiber (1G) setup and test bare metal performance but I'm going to install sophos first and only if it fails to deliver on performance will I try other distros.

I'm sure there's tons of people here running Untangle and pfSense (and pfSense flavors) - they can simply go to http evasion test (HTTP Evader Test-site (noxxi.de)) and report back how their solution performed.

 

[SSri]

Sophos was only able to get to 350 down

Sorry, is this the internet download speed out of the 1 gig connection?

I'm sure there's tons of people here running Untangle and pfSense (and pfSense flavors) - they can simply go to http evasion test (HTTP Evader Test-site (noxxi.de)) and report back how their solution performed.

You have raised an important aspect of the security. I am not sure if this figures prominently on relevant forums, at least I am struggling to find threads related to this test. There is one here and another here hereLet’s hope the users test and report the results here, which will be very useful for the community.

You have been using / testing Sophos. How do you find the IDS/IPS?

You said the IPS is ineffective on Untangle; I believe, it uses Suricata for IDS/IPS!

test bare metal performance but I'm going to install sophos first and only if it fails to deliver on performance will I try other distros.

That will be good. Thanks. However, I will be surprised if testing over the bare metal will produce a significantly different outcome.

 

[coxhaus]

So I tested over the last couple of weeks running on a VMWare ESXi 7 host. It's running on an old HP Proliant DL380 G7 (Xeon 5600 series, dual 6 cores, 12-threads CPU, with 32GB RAM, and 6 x 1Gbe interfaces. The 'WAN' side was connected to a Cogent 1Gbit Fiber DIA. Tests were done after hours when line utilization was less than 1%.

"Firewall" machine was configured with 8 Cores, 8GB RAM running VMXnet3 interfaces
"Client" machine was a Windows 11 x64, 10 Cores, 12GB RAM running VMXnet3 interface

Since this is an older server, you can't really get more than 600-700Mbit on VM. Although I verified the WAN circuit can deliver a solid 920Mbit (when directly connecting a Linux laptop to the vLAN).

As for the testing, most home firewalls distros were able to be pretty close to direct WAN connection on the VM (so ~600Mbit). Sophos was only able to get to 350 down (upload wasn't affected) but it only had 4 cores to use (license restriction) and I set it to scan everything. I'm also planning to do some additional testing using a Linux VM as my client and dial down the settings a bit.

I didn't really spend time taking notes on each platform - I just got it configured and if it failed the evasion testing, I moved on to the next solution. I haven't gotten to using the bare metal PC. I want to finalize all testing on VM first. I'm going to give ClearOS another shot since I last used in 2 years ago.

Once I finish my VM testing I'm going to use an old HP prodesk SFF with dual 10GbE hooked up to my home Google Fiber (1G) setup and test bare metal performance but I'm going to install sophos first and only if it fails to deliver on performance will I try other distros.

I'm sure there's tons of people here running Untangle and pfSense (and pfSense flavors) - they can simply go to http evasion test (HTTP Evader Test-site (noxxi.de)) and report back how their solution performed.

Now I also remember Sophos had an IP limit on the home version which now I guess changed to a number cores limited.
So should report to Untangle forums what test is failing. They may have an answer that you aren't seeing. They had every answer when I switched from a transparent bridge mode to router mode running with a Cisco L3 switch.

 

[SSri]

should report to Untangle forums

This Eicar failed test thread is old on the untangle forum, but discussed here.

In short, they are saying it’s expected to fail. One may click the link for a detailed discussion.

 

[coxhaus]

Yes, Untangle knows what's going on. They are not failing. It is the way they see it.

 

[SSri]

@coxhaus Do you still use untangle ?

I was surprised to read about @ifisher observations, considering the 20+years of experience in the space.

At the same time, I would also think untangle, netgate and many others are aware of it and don’t think they wouldn’t do anything to fix this failed test if the packers / files were to pose a threat.

It’s all in the interpretations, I guess. Thx

Edit:

Bitdefender does block the whole site (https://secure.eicar.org) when I tried to download the file on my workstation, which uses windows native advanced firewall, MBAM real time and Bit Defender AV.

 

[ifisher]

Yes, Untangle knows what's going on. They are not failing. It is the way they see it.

They are gravely mistaken. The way to prove it is simple.

A little note first - currently the EICAR test site (not the evader) only allows HTTPS testing so you should use SSL inspection.

For my testing, I wanted to specifically test the IPS ability to detect protocol anomalies that are used in evasion attempts.

I configured Untangle (and the other distros) in a way where when I check the EICAR test sample - it does get blocked.
Then I used the evader testing tool. If I was able to download the EICAR file locally - this means it evaded the firewall blocking capabilities.

This is a severe finding that industry leaders (Checkpoint, Forcepoint, Palo Alto Networks, Fortinet) have published knowledgebase articles to address. You can actually get a special evader tool from Forcepoint (requires NDA etc) and test your firewall using real threats (not an EICAR sample). Bottom line, if I am able to download a piece of malware through the firewall (one it should easily block) - it means any vulnerability (malicious script, drive by download, etc) can get to the end client undetected and unhindered by the perimeter device.

I don't mean to offend anyone, but most of these distros are basically the same. They are on-par with solutions we used in the 1990s.
The landscape today is dramatically different. I can probably write 3 pages worth of what would be required from a good firewall solution and how to test it - but while it might be an interesting read for some, it's not the topic of this thread.

The advice I'm providing is based on my personal and professional experience and observations / opinions only. There are countless of best-selling solutions that have miserably failed my testing through the years.

In fact, I am currently conducting a Cloud-based firewall (SASE/ZTNA) evaluation for my work and so far, ZScaler, Cisco Umbrella (the full SWG), ContentKeeper, and CATO failed it. I actually used ZScaler in production for a year too. I'm currently evaluating Versa Networks and Palo Alto's Prisma Access and hope to have some results in the next few weeks.

 

[ifisher]

Sorry, is this the internet download speed out of the 1 gig connection?

You have raised an important aspect of the security. I am not sure if this figures prominently on relevant forums, at least I am struggling to find threads related to this test. There is one here and another here hereLet’s hope the users test and report the results here, which will be very useful for the community.

You have been using / testing Sophos. How do you find the IDS/IPS?

You said the IPS is ineffective on Untangle; I believe, it uses Suricata for IDS/IPS!

That will be good. Thanks. However, I will be surprised if testing over the bare metal will produce a significantly different outcome.

I did some testing this afternoon with a Linux client instead of Windows 11.

First, a direct connection (of the 1G circuit) can do about 700-800Mbit (vs 600 Mbit on W11) I think it's just a limitation of the ancient hardware it's running on and the VM implementation for W11.

I repeated the test through the Sophos and the results were much better. I was getting very close to the direct number even with the very high protection settings.

I'm actually not going to be able to make any more progress on this the next week or so since I'm travelling but when I get back I'll give the bare metal a shot and see how it works as my daily driver at home (1 Gbit internet, 50+ devices including IoT devices, laptops, PCs, IOS and Android phones and tablets, smart TVs, etc..)

 

[SSri]

I did some testing this afternoon with a Linux client instead of Windows 11.

First, a direct connection (of the 1G circuit) can do about 700-800Mbit (vs 600 Mbit on W11) I think it's just a limitation of the ancient hardware it's running on and the VM implementation for W11.

I repeated the test through the Sophos and the results were much better. I was getting very close to the direct number even with the very high protection settings.

I'm actually not going to be able to make any more progress on this the next week or so since I'm travelling but when I get back I'll give the bare metal a shot and see how it works as my daily driver at home (1 Gbit internet, 50+ devices including IoT devices, laptops, PCs, IOS and Android phones and tablets, smart TVs, etc..)

Thx; we look forward to it.

So, in a nutshell, it is Sophos XG for home is what you would suggest ahead of PfSense and Untangle.

but most of these distros are basically the same. They are on-par with solutions we used in the 1990s.

The question is these vendors should be aware of their tools have failed the evasion testing. One would wonder why they haven’t addressed it. It can’t be the lack of awareness or skills.

 

[Paliv]

Bitdefender does block the whole site ([URL]https://secure.eicar.org[/URL]) when I tried to download the file on my workstation, which uses windows native advanced firewall, MBAM real time and Bit Defender AV.
[/QUOTE]

Just a side note, running MBAM and Bitdefender AV at the same time is a bit overkill. I know MBAM is able to run next to other AVs, but these days either of those options offers the home user plenty of protection. Both have great web filtering. However, it is probably costing you some performance. Also, Malwarebytes and Bitdefender both have browser extensions that can get you the web filtering if you want double layers. Anyway, quite off topic, but just a thought.

 

[SSri]

Just a side note, running MBAM and Bitdefender AV at the same time is a bit overkill. I know MBAM is able to run next to other AVs, but these days either of those options offers the home user plenty of protection. Both have great web filtering. However, it is probably costing you some performance. Also, Malwarebytes and Bitdefender both have browser extensions that can get you the web filtering if you want double layers. Anyway, quite off topic, but just a thought.

Thx for your thought. I have not noticed any performance impact, tbh. It is a very old machine anyway. I don’t use it often these days.

I work from home most of the days; I use the work laptop. The iPad Pro and iPhones are enough for every day browsing unless I need the PC for some serious personal work.

Cheers!

 

[coxhaus]

They are gravely mistaken. The way to prove it is simple.

A little note first - currently the EICAR test site (not the evader) only allows HTTPS testing so you should use SSL inspection.

For my testing, I wanted to specifically test the IPS ability to detect protocol anomalies that are used in evasion attempts.

I configured Untangle (and the other distros) in a way where when I check the EICAR test sample - it does get blocked.
Then I used the evader testing tool. If I was able to download the EICAR file locally - this means it evaded the firewall blocking capabilities.

This is a severe finding that industry leaders (Checkpoint, Forcepoint, Palo Alto Networks, Fortinet) have published knowledgebase articles to address. You can actually get a special evader tool from Forcepoint (requires NDA etc) and test your firewall using real threats (not an EICAR sample). Bottom line, if I am able to download a piece of malware through the firewall (one it should easily block) - it means any vulnerability (malicious script, drive by download, etc) can get to the end client undetected and unhindered by the perimeter device.

I don't mean to offend anyone, but most of these distros are basically the same. They are on-par with solutions we used in the 1990s.
The landscape today is dramatically different. I can probably write 3 pages worth of what would be required from a good firewall solution and how to test it - but while it might be an interesting read for some, it's not the topic of this thread.

The advice I'm providing is based on my personal and professional experience and observations / opinions only. There are countless of best-selling solutions that have miserably failed my testing through the years.

In fact, I am currently conducting a Cloud-based firewall (SASE/ZTNA) evaluation for my work and so far, ZScaler, Cisco Umbrella (the full SWG), ContentKeeper, and CATO failed it. I actually used ZScaler in production for a year too. I'm currently evaluating Versa Networks and Palo Alto's Prisma Access and hope to have some results in the next few weeks.

Why don't you go argue with the Untangle folks? They understand what is going on and they do it their way. Nothing is broken in Untangle. Read the link posted above in the Untangle forums.

 

[SSri]

These vendors can’t be ignorant or incompetent in their security-risk assessment / implications of Eicar evastion test failure. Some of us keep asking answers for these questions.

I understand the implications of a potential security breach, but it is a little technical for me to understand, raise this question on the untangle forum as a third person — as I haven’t done this test.

As a security expert, I hope @ifisher would post his findings on the untangle forum and see how they respond? Of course, it is @ifisher’s prerogative to post or otherwise!

I do not think we can find a convincing response here as the answers must come from Untangle, Netgate and so on.

In the mean time, I am sure, other knowledgeable and experienced members will chip in and perhaps, explain these in detail as well.

Thx