Today's delightful new problem (10gbit/s connection incoming)

[sfx2000]

Second thing I really am curious about is all of these offload features, and how much of a performance difference the make. For the software routers, and general network stacks, I assume they need to fall back to the lowest common denominator, which means they avoid using vendor specific features like this unless they're highly integrated in the drivers. Is BSD better on this part, abstracting away offloading from the networking stack and drivers, compared to Linux, where it seems it's more of a wild west what offload features you can trust to be enabled?

Hmmm - the BSD's - they do tend to be stable... slow but steady - pfSense is FreeBSD based, and the team there is pretty focused on packets per second, which perhaps is the important benchmark.

Linux - performant, but it's a bit of the wild west with drivers and API's at the moment, as Linux is a foundation for a lot of the cloud stuff and under heavy development - everything from kernel to userspace drivers...

with a 1G connection - it's an easy answer, with a 10G connection on the WAN... look for what works.

 

[Samir]

Very interesting set up you have brewing there. Can't wait to hear more in the saga.

As far as obtaining legit nics at good prices, check out the servethehome forums--lots of great people there that are knee deep into the type of set up you're working on and usually know where to find deals as well as sell off their gear at reasonable prices.

 

[Magebarf]

Very interesting set up you have brewing there. Can't wait to hear more in the saga.

As far as obtaining legit nics at good prices, check out the servethehome forums--lots of great people there that are knee deep into the type of set up you're working on and usually know where to find deals as well as sell off their gear at reasonable prices.

Thanks for the pointer, it's a very good one. I actually came across the page itself when I started researching the potential use of Xeon D and Atom C3000 I've mentioned earlier in this thread, so I've been trying to keep up daily, at least with the Good Deals section as well as a good eye towards the market.

The market should be reasonably safe, as the users there are normally very knowledgeable in all their undertakings, but even they seem a bit stumped on the question of validity or clone.
The whole field of OEM models that seems to exist does not really make it easier identifying clones either.

As said, I'm not so sure if my concern really is whether the NIC is legit, but I know for sure that I'm expecting it to follow the specs and feature lists of the "original" product.
I guess some clones and otherwise unauthorized builds of NICs really do manage to meet those standards, but I also hear of a few different models that seems to have quite the issue with overheating etc.
And who knows about the offloading features and how well they work in the knockoffs.

 

[sfx2000]

Thanks for the pointer, it's a very good one. I actually came across the page itself when I started researching the potential use of Xeon D and Atom C3000 I've mentioned earlier in this thread, so I've been trying to keep up daily, at least with the Good Deals section as well as a good eye towards the market.

I follow the servethehome folks - just keep in mind that one can get into a real rat hole and spend money that is not needed looking for small benefit...

 

[Magebarf]

Another recommendation, that kind of strikes the balance in between SNB and Servethehome, is TinkerTry IT @ Home (https://tinkertry.com/)
A lot less of community and more review/guide/analytical posts, with the main interaction with readers being comments on the posts.
It tends to cover the area of slightly more buffed home server while not going the full length of the Homelab community at Servethehome.
I end up there every now and then when researching virtualization topics, Xeon D/Atom C3000, as well as NICs, including external (Thunderbolt/USB-C) 10GbE NICs.

 

[Magebarf]

The ketchup effect in action; Just as my NICs for the softrouter I'm planning to configure is about to be delivered, I get a notification from the Swedish postal service that the ISP just has sent me a package, so today I was able to pick up the Huawei HN8255Ws and bring it home.

Exact model number is HN8M8255Ws04, which may be related to the customizations ordered by the ISP, and the box is marked with "10GE CPE ... For Bahnhof of Sweden".
Apparent modifications is the inclusion of the ISPs logo on the front of the router.

On the side of the unit there are three buttons, toggles for lights and WLAN, and also a WPS button.

Looking at the back of the device you have the 4 x 1000Base-T1 ports on the LAN side, as well as telephony and two USB ports.

Tearing away the cover on the back side reveals the two 10GbE ports. 1 SFP+ slot for the WAN side, and a 10GBASE-T on the LAN side.

Next step here is to setup the device and test it, and then after my NICs for the intended softrouter has arrived, I will put this unit under test to see if it will pull its weight.

With some assistance from a friend, we've managed to find a possible SoC matching the capabilities of the device, and that is the HiSilicon ONT SOC.
If this is the SoC being used here (which we will hopefully be able to verify soon) the marketing material for it promises "Line rate forwarding capability even in minimum packet size". Whether this holds true when adding NAT and firewall rules is a whole other question.

 

[coxhaus]

Hmmm - the BSD's - they do tend to be stable... slow but steady - pfSense is FreeBSD based, and the team there is pretty focused on packets per second, which perhaps is the important benchmark.

They may focus on packets per second but their latency sucks with a layer 3 switch.

 

[sfx2000]

They may focus on packets per second but their latency sucks with a layer 3 switch.

The latency issue you were working on with your pfSense installation might be site/configuration specific...

 

[sfx2000]

With some assistance from a friend, we've managed to find a possible SoC matching the capabilities of the device, and that is the HiSilicon ONT SOC.
If this is the SoC being used here (which we will hopefully be able to verify soon) the marketing material for it promises "Line rate forwarding capability even in minimum packet size". Whether this holds true when adding NAT and firewall rules is a whole other question.

Might find somethere here that could be helpful..

https://support.huawei.com/enterprise/en/access-network/echolife-hn8255ws-pid-22280247

 

[coxhaus]

The latency issue you were working on with your pfSense installation might be site/configuration specific...

I doubt it. pfsense wants to be in control of all the routing. It does not play well with others on the local LAN.

 

[sfx2000]

I doubt it. pfsense wants to be in control of all the routing. It does not play well with others on the local LAN.

It does... your statement about latency holds little water...

Properly configured - that's why I suggested a possible site misconfig in your case...

Live Data from inside my LAN...

WAN side - dns lookups have to happen - using DNS Resolver with forwarder disabled - resolver is configed as DNS over TLS using Unbound going to Cloudflare DNS.

LAN side - This is looking outward from my LAN - first hop is the gateway (Netgate), second is the modem (Surfboard), and the third is my providers local PoP

Note - Modem <-> pfSense <-> Netgear GS-108T <-> Monitor Host

Inside - TimeCapsule is connected via Ethernet as an AP (routing disabled)

AP Extreme is a wireless extension of TimeCapsule these days... so that's going to be a bit slow as they're paired up over 2.4GHz.

Instrumenting one's network rocks...

 

[sfx2000]

Instrumenting one's network rocks...

And FWIW - I was doing some testing/eval on an upcoming 802.11ax Router/AP (Broadcom based)

First half is the Router/AP, second is me putting PFSense back into the role as the primary router...

 

[coxhaus]

It does... your statement about latency holds little water...

Properly configured - that's why I suggested a possible site misconfig in your case...

Live Data from inside my LAN...

WAN side - dns lookups have to happen - using DNS Resolver with forwarder disabled - resolver is configed as DNS over TLS using Unbound going to Cloudflare DNS.

View attachment 16204

LAN side - This is looking outward from my LAN - first hop is the gateway (Netgate), second is the modem (Surfboard), and the third is my providers local PoP

Note - Modem <-> pfSense <-> Netgear GS-108T <-> Monitor Host

Inside - TimeCapsule is connected via Ethernet as an AP (routing disabled)

AP Extreme is a wireless extension of TimeCapsule these days... so that's going to be a bit slow as they're paired up over 2.4GHz.

View attachment 16205

Instrumenting one's network rocks...

You have to setup gateways on pfsense to route to the layer 3 switch. It seems kind of clumsy.

PS
My guess is you don't fully understand layer 3. There should be a hop for your layer 3 switch since it is a router. You have stated in the past you run a layer 3 switch.

PSS
If you want to try understand what I am saying about using a layer 3 switch with pfsense. Here is an example I found.
https://forum.netgate.com/topic/959...sense-internet-from-a-vlan-behind-a-l3-switch
This is clumsy in my mind and does not work well. I ran it for a year. I will not go back.

And no you are not running at layer 3 with your switch if you do not have a hop on your traceroute for your layer 3 switch.

So your comments about me holds NO water.

 

[Magebarf]

After been using the Huawei HN8255Ws for a while, it seems to work quite good.

I have encountered an issue where using an external AP (a repurposed Netgear R6300v1) would make all traffic going over WLAN to be dropped every 10 minutes, while all other traffic going over the same cable (integrated switch) goes through great. Using the integrated AP functionality works without any issues however.

I've added a Netgear GS810EMX, the Nighthawk "gaming switch" with double NBASE-T ports, and have with that one been able to test the performance from end points in my network.
All these tests run on my Windows 10 computer, with a Asus NBASE-T NIC. These were run through Chrome, other than the Ookla Speedtest which was run using the Windows 7 desktop client. For Ookla, it seems Chrome and Edge are both bottlenecks, and running the Windows 10 Desktop version (from Microsoft Store) seems to have the same bottleneck. The older desktop client still available for download on their webpage is what gives best results.

Still haven't given this a run for the money with smaller packets etc. but hope I will be able to test that out soon. My issue with external AP requires me to have it plugged in while the ISP support helps me pinpoint the issue.