pfSense (or other dedicated router) questions

[garyd9]

The more I play, the less I'm appreciating this Asus AC3200 "router" I'm using. It does wifi great, and I have to admit I really like the live "adaptive QOS" meter and "traffic analyzer" statistic page (where I can see just how many GB's of "youtube" my daughter is watching.)

However, I'm reaching limits: Regardless of what LAN->WAN bandwidth it's supposed to be rated for, it's starting to fall behind. CTF doesn't work whatsoever with IPv6, and if I have CTF enabled, it prevents me from using any of the fancy ebtables tricks I usually enjoy (as CTF seems to completely bypass ebtables!!) If CTF isn't enabled (and with other QOS stuff going on), it can barely keep up with 170 megabit/sec.

So, I'm considering the idea of a standalone router... I don't know what (if anything) can accommodate my wish list. pfSense seems to be highly regarded, but I'm clueless as to what it can/can't handle (and while I'm fluent with linux, I'm NOT fluent with BSD.)

So, on to my wish list:

Inexpensive: If I have to purchase everything new, it can't cost more than $300-$400.

Routing speeds: It has to realistically handle a LAN <--> WAN speed of a sustained 500 megabits/second. (No, my cable modem isn't that fast, but I'd like to plan ahead a bit...) This is actual speeds - not "theoretically rated for" speeds. Also, it should handle that speed without any "tricks" that bypass normal operations (such as CTF.)

VLAN Tagging support.

Pretty realtime "what is using my bandwidth" reporting.

Access to hourly/daily reports on WTF my daughter is doing to blow through 10 GB of data in 3 hours.

Would be nice: LACP support on the LAN side.

....

So, can pfSense do this? Can ANYTHING do this?

Thanks
Gary

 

[sfx2000]

yes - just keep in mind that it's a mind reset with how to config pfSense...

Minor nit - pfSense 2.3 introduced a new monitoring process, moving away from the legacy RRD graphs, and this has been an issue with many - there's an open ticket and someone is working the issue.

 

[garyd9]

yes - just keep in mind that it's a mind reset with how to config pfSense...

Are you sure? After YEARS, I'm still trying to come to grips with the idea that my "router" and my "wifi AP" are the same box. It annoys the &^%@#% out of me that wifi traffic doesn't pass through my switch... (Is that what you meant by it being a mind reset?)

(I've actually been trying to think of a way to put all the wifi traffic on a vlan, send it to my switch, and then have it come back to the "router" as a different vlan.. just so I feel like I have a separate AP and router...)

pfSense 2.3 introduced a new monitoring process, moving away from the legacy RRD graphs

I'm not familiar with pfSense, so I really don't understand what this statement means. Can you point me to before/after pictures that might better show?

Thanks
Gary

 

[garyd9]

Not an endorsement whatsoever, but would something like this meet my wishlist: https://www.amazon.com/dp/B00JS6YZYO/?tag=snbforums-20

 

[sfx2000]

I'm not familiar with pfSense, so I really don't understand what this statement means. Can you point me to before/after pictures that might better show?

pfSense 2.2 had a nice summary presentation based on RRDGraph of bandwidth on ports over time, and this went away in 2.3 - for better or worse, the new monitoring stuff has more flexibility, but the current implementation doesn't graph this particular item...

 

[L&LD]

Not an endorsement whatsoever, but would something like this meet my wishlist: https://www.amazon.com/dp/B00JS6YZYO/?tag=snbforums-20

I wouldn't even consider that at that price. You can build one that is easily 10x more powerful for the same or a few more dollars and then install pfSense yourself.

https://www.freebsd.org/releases/10.3R/hardware.html


Just make sure you buy what's on the recommended list for no headaches during setup or use.

 

[sfx2000]

Not an endorsement whatsoever, but would something like this meet my wishlist: https://www.amazon.com/dp/B00JS6YZYO/?tag=snbforums-20

This is a bit spendy for what you get - check the QOTOM thread - it's a bit more build in yourself, but it's plenty of power there - probably overkill for a home network.

Netgate has a few purpose designed boxes that are co-branded with pfSense - but they also offer the boxes without, and it's a roll your own solution from there... I'm running pfSense 2.3.1r5 on a Netgate RCE-C2440, and it runs quite nicely, but again, it's a bit of a premium compared to x86 boxes out of Shenzen that will run pfSense well..

 

[sfx2000]

This is a bit spendy for what you get - check the QOTOM thread - it's a bit more build in yourself, but it's plenty of power there - probably overkill for a home network.

The QOTOM build thread starts here - http://www.snbforums.com/threads/confessions-of-a-pfsense-newbie.5379/page-3#post-267939

 

[garyd9]

Thanks for the links... I'm not sure if I want to build it myself or just order something pre-built. Regardless of if I do a pre-built or DIY, I don't think I want to get into another overseas ordering situation. (Been there, lost money, extreme amounts of time, and won't do it again for technology.) I actually prefer these days to either buy from brick and mortar or via amazon prime.

(When amazon ships it, I actually get it in a predictable amount of time - and returning defective stuff for a full refund is trivial.)

I do see on amazon a couple of pre-builts using that same QOTOM barebones that might be an option. (Sure, they are more expensive, but worth the extra price for the above reasons.) Then there's always the Netgate solution 4 port at 350 USD. That seems to have the advantage of being "known good." (I could also save a few bucks with a dual LAN box instead of quad-lan, but then I'd give up LACP and the dual-LAN boxes seem to have slower processors - which could possibly start to impact router throughput.)

Both the QOTOM and netgate have mSATA, which is perfect as I just upgraded my kid's NUC machine from a 120GB mSATA to a 250GB... so I have this 120 GB device sitting around collecting dust. (No, I don't really need 120GB on a router, but I'd feel much better if the spare equipment was somehow being utilized.)

I really need to (ab)use google and research this some more. Both to ensure that pfSense will give me what I want, and to decide on the best h/w choice.

 

[Nullity]

@garyd9

Considering that you have more experience with Linux, you should check out some of the Linux-based firewall/router distributions like IPFire or VyOS.

If you are interested in features, Linux wins (though... do routers need a huge amount of features?).

 

[sfx2000]

Then there's always the Netgate solution 4 port at 350 USD.

I like the Netgate solution for a couple of reasons... (other than it being awesome)

1) Designed/Made in the USA - Jobs for America - which is always good compared to cheap Shenzen boxen...
2) Netgate is a pfSense partner and funds/supports pfSense, and that's always a good thing.

Just my thoughts... for those reasons, I invested in the Netgate directly...

 

[garyd9]

@garyd9

Considering that you have more experience with Linux, you should check out some of the Linux-based firewall/router distributions like IPFire or VyOS.

If you are interested in features, Linux wins (though... do routers need a huge amount of features?).

I suppose it depends on what kind of "features." If you're referring to a bittorrent client, then it's meaningless to me on a router. Same thing for a NAS. However, if it means better reporting/statistics, then it means something. VPN is also a "feature" that seems appropriate to be on a router/gateway.

What I'd really, really, really like to find on one of these stand-alone routers is the stats reporting that I have on the asus firmware. I can look at a single page and see that, in the past 24 hours, my daughter has dumped 3 GB on youtube, 100 MB on HTTPS, 22 MB with Amazon, etc. My was having a slow day - he only dumped 1.3 GB in youtube. There must have been a firmware update on his WiiU, because it downloaded 720MB today. Even better, I can see WHEN this all happens, so I can know if one of my kids is up past their bedtime watching youtube, etc.

All that in a nice graph..

For those not familiar with it, there's a photo of it on this stackexchange topic:

http://hardwarerecs.stackexchange.com/questions/1289/home-router-that-logs-per-device-internet-usage

That's the first time I've seen something fairly reliably narrow down traffic usage to specific applications... and I've already found it useful.

 

[Nullity]

Traffic monitoring is a very OS-agnostic feature. There are literally hundreds of great apps out there that work on most Unix-based Operating Systems. I would hesitate to choose a system simply because of the apps that are bundled with it.

RRDtool, ntop/ntopng, many netflow related projects, etc.
You might also check out https://github.com/firehol/netdata, which is an incredibly sexy (and free) monitoring app that is very popular over on github. You can find some live demos of netdata here.



By "features" I was referring to hardware support (802.11ac, USB3, etc) and networking features like more modern traffic-shaping capabilities (fq_codel, cake, PIE, etc).

 

[ericnix]

Have you looked at what pfSense store offers? https://store.pfsense.org/systems/

 

[sfx2000]

You might also check out https://github.com/firehol/netdata, which is an incredibly sexy (and free) monitoring app that is very popular over on github. You can find some live demos of netdata here.

Neat.. linux only, so pfSense would probably need some coding work (upside is that it's bootstrap based, so is netdata) - thanks for the tip

 

[System Error Message]

aside from pfsense the CCR1009 is something for considerations. Unlike ubiquiti who only lists layer 3 routing speeds with hardware acceleration, the CCR does not rely on hardware acceleration and has speeds rated with various configs. Each CCR core will do 2Gb/s of NAT if using a CPU connected interface without any PPPOE or overheads. 1Gb/s with PPPOE. It has enough processing power to run all the firewall and QoS rules you want while being within the range of your budget.

Since it has SFP you could just plug fiber optics right into it without a modem.

 

[Nullity]

Neat.. linux only, so pfSense would probably need some coding work (upside is that it's bootstrap based, so is netdata) - thanks for the tip

I dunno if pfSense even offers what OP wants (per client monitoring) out of the box. The darkstat (GUI) or iftop (CLI) pfSense packages may have that feature... I dunno. The softflowd package can surely export the information he wants, but setting up a netflow collector is probably not something he wants to do.


I am not sure what course I would take...

The Firewall Rules tab (since pfSense 2.3) records the number of active states & total traffic (in bytes), which is useful but you must create a rule.

According to my pfSense firewall rules it seems like my Chromecast has sent 21MByte of pings this month ...

 

[sfx2000]

pfSense has a rich SNMP implementation so one can do off machine and run something like Cacti, mrtg, or something similar without have to resort to netflow gathering...

 

[System Error Message]

mikrotik routerOS can do what you ask, as for the user logging you will have to use transparent configuration of hotspot/radius with mikrotik. The CCR1009 will handle gigabits worth of NAT without needing any tricks while applying all the rules you want. If you use IPTables than using routerOS will be familar only that they use a GUI/CLI that uses a different syntax.

CCR1009 can do LACP very easily with different modes on the CPU interfaces. Switch interfaces vary depending on chip but is pointless as switch to CPU is only 1Gb/s link so the CCR1009 has a combination of switched and cpu connecting ports. The CPU is fast enough that the switch isnt really necessary but the 9 core CPU itself doesnt have enough networking interface that are directly on it (2 pipes instead of 4 on the more core models). Each pipe supports 4 1Gb/s ports or 1 10Gb/s port. Again see the diagram as it shows you what port connects where.

Every interface and user (not using the built in routerOS accounting that you use to login router management) has its own stats and graphs and such. You also have the option of storing them as well (micro sd recommended for this).

Supports vlan tagging.

Obviously it is capable of much more but mikrotik is marketed as a cisco alternative so if you want those enterprise features mikrotik has them but not quite as good as one of those cisco routers that they use in internet exchange. So features like BGP and so on dont work as great on mikrotik as they do on cisco but all other non cisco related features work well and it works way better than consumer routers.

The CCR1009 costs between $400-$500 and is more cost effective per throughput than other routers even in unfair comparisons especially if you dont want any sort of CTF or hardware acceleration for your multi gigabit WANs.

All that you would need to do is connect CCR to modem/SFP module and connect your AC3200 behind it.

Go to demo.mt.lv to see how the interface looks like but i suggest winbox instead as you wont have to worry about what ip address and can still access the router even if you mess up your config.

Other routers like pfsense and ubiquiti can do the same things as well but ubiquiti can only do gigabit NAT speeds using hardware acceleration. Their wirespeed claims is bogus because any router can do wirespeed in routing mode with no config/hardware acceleration. Pfsense just depends on what NICs you use and having a good overall system and requires x86 which while some have embedded, not as nice as the ccr in the embedded space unless snort is important to you.

Think of mikrotik as a focused router which you cant install other software on, so theres no printer sharing, no torrent clients and so on. Ubiquiti is more of an embedded linux server that use marketing gimmicks, their speed claims are nothing special as any router can do layer 3 routing (not NAT) at wirespeed (2Gb/s in the case of consumer routers as they only have 2 CPU connected ports with switch chip behind one). Pfsense is a specialised x86 OS focused on being a router but between mikrotik and pfsense they each have features the other doesnt. PFsense cannot be used for things like file sharing, printer sharing (unless you can use cups and xsane) and so on.

 

[garyd9]

I dunno if pfSense even offers what OP wants (per client monitoring) out of the box. The darkstat (GUI) or iftop (CLI) pfSense packages may have that feature... I dunno. The softflowd package can surely export the information he wants, but setting up a netflow collector is probably not something he wants to do.

Would something called "ntop" do it? I found a youtube video from 2013 that shows all kinds of pfsense statistic packages, but it's narrated in a language I don't understand. Around 16 minutes into the video, it looks like there's a per-app breakdown:

On the other hand, doesn't something like this require level7 DPI? Did pfsense REMOVE level7 in a recent build? If it was removed, would that imply that I couldn't possibly get the stats I want from it in a current build?