Confessions (part2) of a PFSENSE 2.0 newby..

[Dennis Wood]

I've been reporting in here from time to time on pfsense and our adventures with it in this thread. Today was a big day as we took the two boxes "live" with version 2.0 incorporating:

1. SNORT IDS (detects and blocks intrusion)
2. HAVP antivirus
3. SQUID caching proxy
4. SQUIDQUARD web content filtering
5. LIGHTSQUID web activity reporting
6. IPSEC VPN

With several NAS replicating over 2 WAN connections, VOIP, remote VOIP, VPN/remote access and plenty of specialized traffic, swapping in routers has never been something I've looked forward too. That said, after messing with pfsense and many, many updates/rebuilds tweaks over the last six months, I can say the conversion from Draytek 2950s to the pfsense boxes was .... the easiest to date. The proof was in the fact that there were pretty much zero complaints, and network performance for the first time in weeks was flawless.

A big, big surprise is how quickly we got VPN working. It was almost scary how quickly VPN/remote access was up and working, particularly given zero time with it so far, and the fact that SHREW was working so well for us previously. This link: http://dekapitein.vorkbaard.nl/tech...ling-in-pfsense-2-0-release-for-road-warriors
quided me through the very complex series of options. VPN worked first try using the guide. Very big props to "Kapitein Vorkbaard" for taking the time to post this up. I'll post up some more once we get VPN via iOS5 working.

We've managed to do everything the draytek 2950s were doing, but are doing with one box (instead of a router (25watts 24/7) and computer (50 watts 24/7) and have added IDS, proxy and AV to the mix. All of this in one little box using 18 watts with much better performance. Truly impressive and a testament to the huge amount of work the pfsense developers and users have put into the system.

 

[Dennis Wood]

A few tips to add based on tweaks that had to be done to get things working properly.

Rather than updating pfsense (if you're updating from an older version), I found it less problematic to save my configuration (using the pfsense GUI) and reload the router with a new install using the latest USB stick version (memstick versions on the mirror sites). Win32diskimager.exe is what works well for me to take the .img file and write a bootable USB stick. Once you have a basic psfense built and assign an IP you can disconnect the monitor/keyboard from the router and carry on via browser access to the router's web gui. Restore using the saved configuration file, and sit back for a few minutes as packages are reinstalled. This makes sure you have all the latest package code and are not afflicted by straggler bits of code.

SNORT

1. Snort blocked IMAP from email provider, a white list entry had to be added.

2. Snort required this entry under suppression to stop an HTTP inspect trigger that blocked a lot of sites: suppress gen_id 120, sig_id 3

3. Snort blocks SKYPE by default. If you want it to work, you need to check the snort_p2p.rules and disable the four rules blocking SKYPE.

One thing that caught me is that when adding whitelists and suppression to SNORT, you need to edit each interface and change the default for the interface to use your new whitelist and suppression list. There is dropdown list that normally only shows "default" but will show the names of your suppression/whitelist as choices after you've created them.

3. Many people forget to check off the options under the "Preprocessor" tab that shows up after you add your WAN interfaces to SNORT. Nothing is blocked until you do this. You also need to get an OINK code and update the rules before anything will be blocked.

4. Once you choose rules (assuming you choose all of them) to filter on each interface, SNORT will often fail to start. Looking the log will often show that SNORT choked on a rule...so deselect and restart the snort interface. One thing that helps is to only choose one of the rule sets, make sure everything is working ok, add a few more, restart etc.

5. Taking a SNORT install live without understanding how to read the alert logs, create suppression rules and whitelist is a mistake as you will need to respond quickly when a typical web/network function stops working in the live environment. You can take stop the SNORT interfaces when required to troublehoot in a live setting.

HAVP

1. HAVP blocked virtually every exe download (like flash) with a broken executable error. The problem is fixed by doing this:
http://forum.pfsense.org/index.php/topic,43160.msg223416.html#msg223416

2. Youtube would not work from iOS devices until the checkbox for "Http range requests" was checked off. This is under HAVP's HTTP Proxy tab.

3. Adding these to the HAVP whitelist (again, under HAVP's HTTP Proxy tab) greatly improved media streaming performance:

*.adobe.com/*
*.youtube.com/*
*.vimeo.com/*

IPSEC VPN

I used this guide: http://dekapitein.vorkbaard.nl/tech...ling-in-pfsense-2-0-release-for-road-warriors

2. One very important change required (for stable IPSEC reconnects using SHREW as we are) is something I found poking around in older posts. I needed to edit the IPSEC phase 1 tunnel and change the "Policy Generation" option from "Default" to "Unique" . This was the conclusion after seeing configuration file errors coming from the IPSEC error log.

3. An IPSEC tab shows up under the network rules interface after you add IPSEC. As per the guide, you need to allow traffic on it, otherwise all is blocked. What is cool about this is that you can tighten down on where your IPSEC traffic is allowed in from etc. I also needed to allow port 500 (ISAKMP) inbound traffic on the WAN interface by adding a simple rule.

General Settings

1. Under System, Advanced, Miscellaneous: To enable reliable outbound VPN, I found "Sticky Connections" needed to be checked off.

2. If you are trying to troubleshoot with HAVP, Squid, Squidguard and SNORT, you should change the default log settings to check off "Show raw filter logs", "Log packets blocked by default rule", and you may want to increase the log count to 1000 entries, reverse order.

3. Having a laptop on hand using a tethered iPhone is a great way to be on site but have an external connection to test/troubleshoot VPN.

As an example of where to start when troubleshooting IPSEC VPN after a connection fails.

1. Check the firewall log to make sure a port is not being blocked by the default rule. If you're blocked here, no point in looking further ... at least for now. Add the appropriate firewall rules to permit the blocked ports/traffic. In my case, I did not allow port 500 inbound on the WAN for IPSEC so I could see my external IP being blocked on port 500.

2. If you're not blocked by the default rule, check to make sure SNORT has not blocked the external IP for some reason. If so, you'll need to look at the alert and possible add a suppression or whitelist rule.

3. Finally, if nothing shows up in the default firewall or SNORT log, look to the IPSEC log. This will show you what's going in in great detail so you can see failed connections.

 

[GregN]

Thanks for the Update!

How is load balancing and dual WAN under the Full Rel 2.0?

Any issues in traffic shaping?

 

[Dennis Wood]

Load balancing works great. The only (big for some) caveat is that once you install Squid, all port 80 traffic goes to 1 WAN. All other port traffic is balanced. Currently HTTPS traffic is load balancing but this is already an issue as any secure page is choking on IP changes..have already added a port 443 rule to force this to one WAN. Same thing for SQL traffic over the WAN.

In practice I've found port 80 load balancing previously caused a lot of complaints as we use a lot of web based tech. Nothing session based will tolerate IP changes..so previously we ended up directing ip ranges to specific WAN connections. IMAP so far is happy load balancing, so this is good.

With respect to traffic shaping, all I can say is that I have it setup, queues are being managed, and no one is complaining. VOIP calls appear to be of noticeably better quality over the Draytek setup. We do a lot of SKYPE so we'll see how well we fare in that regard. I'll be adding a rule to priorize SKYPE packets which was something I could not do with the Draytek 2950s.

Btw, Greg, your avatar made me laugh. Beaker was one of my favs...I've loaded an avatar just for you

 

[bezant]

pfsense throughput

According to pfSense, to achieve throughput of 201-500 Mbps, you need server class hardware with PCI-X or PCI-e network adapters, or newer desktop hardware with PCI-e network adapters. And, no less than 2.0 GHz CPU.

Yet the ASUS RT-N56U is capable of 800+ Mbps.

How is that possible?

 

[Dennis Wood]

Pfsense uses PC based hardware, but a router manufacturer can do whatever they like with custom processors. Throughput measurement is something that frequently falls down in real world testing. That said, you're comparing apples to oranges here.

Pfsense is capable of DPI (deep packet inspection) from layer 2-7, where the Asus product advertises SPI which only looks at packet footers and headers along with session information. It makes sense that if you want to maintain throughput with DPI, you need much faster hardware. Even pFsense published throughput numbers are very much compromised as you add packages like SNORT, Squidguard etc. Another major difference is the ability to simply add interfaces and therefore build a dual, triple, quadruple WAN box with VLANs, multiple LAN ports etc.

The up side is that for some users, performance will seem to increase despite all the inspection if you've implemented SQUID which can deliver content very quickly from an SSD drive used for cache on the LAN side of things. If you allow that pfsense (at least in our implementation) is used for IDS (SNORT), antivirus, proxy (SQUID cache), proxy filtering (SQUIDGUARD), traffic reporting (Darkstat) and web use reporting (Lightsquid) all in one small high performance box using 18 watts and costing under $400....then from my perspective, nothing compares. The problem is that if you want a "packaged" setup with pfsense, support etc, the price tag climbs sharply.

One of the truly eye opening experiences is firing up SNORT and seeing just how much traffic is being blocked as a result of DPI. An interesting study would be look at how many of these attacks are coming from compromised home/business networks, and how much of this traffic would be stopped by integrated IDS systems. In essence, operating without deep packet inspection means you're still fairly naked with respect to network exploits. Another basic essential (at least in my book) is that a router be able to report on LAN traffic by IP address. This alone (providing you look at the data regularly) allows you to ask a few questions that may need to asked when one IP stands out in terms of trafffic and/or open sessions.

 

[Dennis Wood]

VPN setup

So after some playing around with VPN, here's what I came up with.

Setting up IPSEC on both WAN interfaces is not obvious, nor is it simple to setup.


IPSEC VPN using SHREW (best for Windows remote access)

So for IPSEC and SHREW on one WAN interface, the aforementioned link is good:

http://dekapitein.vorkbaard.nl/tech...ling-in-pfsense-2-0-release-for-road-warriors

... With the same added tweak: One very important change required (for stable IPSEC reconnects using SHREW as we are) is something I found poking around in older posts. I needed to edit the IPSEC phase 1 tunnel and change the "Policy Generation" option from "Default" to "Unique" . This was the conclusion after seeing configuration file errors coming from the IPSEC error log.

PPTP (iOS5 VPN access)

For iOS5 (ipad iphone vpn) PPTP is simpler to setup and therefore you don't need to mess with the IPSEC setup that works so well with SHREW software. PPTP setup is easy using this guide: https://discussions.apple.com/thread/1776886?start=0&tstart=0

So that's it...a few weeks in full swing and everything is working great. There was a SNORT update a few days ago that addressed a few issues once the package was reinstalled...so recommended to update if you're using it.