pfSense box or Edgerouter lite?

[Nullity]

...

I wish pfsense would give me some sense of direction. It seems hard to get forum support any more.

pfSense's forum users are pretty unforgiving if you ask a question that can be answered with the wiki, forum, or FreeBSD/OpenBSD's documentation... which means many questions go unanswered over there.


Disabling off-loading may help, if those particular NICs are buggy.

 

[coxhaus]

Well I made progress.

IPv6 is having a drastic effect on NIC speed. I went under the WAN interface General Config selected IPv6 configuration type and selected none.

My DSLReport's speedtest now is running 345 megabit.

 

[Nullity]

Well I made progress.

IPv6 is having a drastic effect on NIC speed. I went under the WAN interface General Config selected IPv6 configuration type and selected none.

My DSLReport's speedtest now is running 345 megabit.

Your ISP supports native IPv6?

Was the speedtest running over IPv6?

 

[sfx2000]

Did you notice the list they have 8254 and 8257 but no 8256. I have 82563EB so does that mean it's not supported?

FreeBSD has two intel drivers, the EM series, and if I recall, the igb series - anyways, ssh into the box, and check ifconfig and your dmesg output - see how the NIC is being configured - the intel driver defaults very conservative, and one might have to do the mode select to get most out of the card...

With Hardware, you're better off searching the BSD communities, FreeBSD first as that's the core of pfSense...

 

[sfx2000]

I dug around up in my attic and found an Intel FW82546GB dual port old server card. Do think I should try a reinstall with the dual port card? Some how I need to come up with the offload the drivers part without killing my system.

I wish pfsense would give me some sense of direction. It seems hard to get forum support any more.

Can always try that - drop the card in, go into BIOS and disable the onboards, and the kernel should pick up the new card - shouldn't have to reinstall for that.

 

[Nullity]

FreeBSD has two intel drivers, the EM series, and if I recall, the igb series - anyways, ssh into the box, and check ifconfig and your dmesg output - see how the NIC is being configured - the intel driver defaults very conservative, and one might have to do the mode select to get most out of the card...

With Hardware, you're better off searching the BSD communities, FreeBSD first as that's the core of pfSense...

Yeah, even pfSense says to do that.

It shows the driver in the GUI though. No need for SSH.

 

[coxhaus]

Yeah, even pfSense says to do that.

It shows the driver in the GUI though. No need for SSH.

OK. Where in the GUI is it located? Maybe I can get a little more speed out of this thing.

 

[sfx2000]

Yeah, even pfSense says to do that.

It shows the driver in the GUI though. No need for SSH.

yeah, but to configure it, he might have to get into the shell/terminal - so Putty or other ssh client may be needed...

 

[Nullity]

OK. Where in the GUI is it located? Maybe I can get a little more speed out of this thing.

Click on Interfaces. It shows the driver (em0, em1, igb0, etc).

 

[Nullity]

yeah, but to configure it, he might have to get into the shell/terminal - so Putty or other ssh client may be needed...

It kinda saddens me, but CLI configuration (beyond the initial setup) is actively discouraged because unless you directly edit the pfSense-centric config file, your ifconfig/pfctl/etc changes will be erased by the GUI's php config script. The php scripts can be directly executed via the php shell I think, but that is nothing a beginner should be messing with.

Most of the pre-2.3 config is php. I think 2.3 migrates away from it.

 

[sfx2000]

yes, and no - going into the shell can at least confirm configuration - I'd be hesitant to make changes inside the shell with pfSense, as, like you mentioned, there's a fair amount of scripts...

 

[coxhaus]

Click on Interfaces. It shows the driver (em0, em1, igb0, etc).

When I click interfaces I only have LAN and WAN. When I select either one I can't find a driver showing anywhere.

PS
When I did the install the WAN was em0 and LAN was em1. Is that what you mean?

 

[sfx2000]

We probably need to get Tim Higgins to branch this thread off, as we've gone very much off topic...

em0 should be mapped to the WAN interface, so em1 makes sense to be the inward facing one...

 

[coxhaus]

Sounds good to me. Maybe pfSense Tunning thread?

 

[sfx2000]

@thiggins - can you prune and migrate this thread - items based on coxhaus's implementation of pfSense?

 

[coxhaus]

Here are some setting I have changed today under System->Advanced. I think my NICs must be supported as I have been reading pfSense all day and tunning.

1. I changed kern.ipc.nmbclusters="1000000" to increase mbufs. I had to add this entry to system tunables.

2. The settings for Hardware TCP Segmentation Offload (TSO) and Hardware Large Receive Offload (LRO) under System > Advanced on the Networking tab default to checked (disabled) for good reason. Nearly all hardware/drivers have issues with these settings, and they can lead to throughput issues. Ensure the options are checked. Sometimes disabling via sysctl is also necessary. I enabled these by unchecking them.

My system seems smoother now. What do you guys think? Any ideas about more tunning?

I can tell you I have not reached the same low level latency of my RV320 router. My RV320 is 18ms response and my pfsense is 21ms. I am still looking for 3ms better response time.

 

[Nullity]

I wonder if software routing is even capable of the same latency as hardware routing. (Assuming the RVxxx is a hardware router.)


Anyway, if latency is paramount, increase kern.hz by adding "kern.hz=2000" (or more?) to /boot/loader.conf.local via SSH.

Also, I would not be concerned with a variance of 3ms in idle latency because no human could even notice that. Much more important is the worst-case latency which can be easily optimized with traffic-shaping/CoDel. Without CoDel my ping averages 650ms during a fully saturating upload. With CoDel I average ~35ms with 55ms maximum. My idle ping is ~10ms.

Internet browsing during an upload is painful without CoDel, but with it I can upload for hours and not even notice any difference.

 

[coxhaus]

I just checked and kern,hz is not available under system->advanced->system tunables. If I change it with SSH will it go away with any updates? I will try it tomorrow as I am drinking Bourbon right now.

I don't want to give ICMP high priority to see lower ping rates which I am guessing CoDel is.

In the old days x86 were faster than router based hardware. But now I am thinking that may not be so true any more. There probably is a lot of code bloat with IPv6. The x86 CPUs are so much faster than router CPUs. But if it is all in hardware maybe software cannot keep up. This is a interesting project.

 

[Nullity]

I just checked and kern,hz is not available under system->advanced->system tunables. If I change it with SSH will it go away with any updates? I will try it tomorrow as I am drinking Bourbon right now.

I don't want to give ICMP high priority to see lower ping rates which I am guessing CoDel is.

In the old days x86 were faster than router based hardware. But now I am thinking that may not be so true any more. There probably is a lot of code bloat with IPv6. The x86 CPUs are so much faster than router CPUs. But if it is all in hardware maybe software cannot keep up. This is a interesting project.

1. /boot/loader.conf.local will not be affected by updates.

2. CoDel is an AQM, meaning it dynamically adjusts the networking queue/buffer to keep latency as low as possible without affecting bandwidth. In many ways CoDel removes the need for QoS since all traffic has optimal latency. There is no known drawback to using CoDel that I am aware of.

Edit: Meant "bandwidth" but repeated "latency".

 

[sfx2000]

CoDel is good... advanced monkey on the QoS keyboard there...