Upgrade to Symmetrical Gigabit - Need to Upgrade my Asus RT-AC3100 (RMerlin FW: 384.17)

[Roadspill]

So I've recently upgraded my Internet to Symmetrical Gigabit Fiber.

Equipment for Tests:
Test Computer -> Late 2013 Macbook Pro w/ Thunderbolt Gigabit Ethernet Adapter
Test Router -> Asus RT-AC3100 (RMerlin FW 384.17)

Speeds:
- ETH to ONT == 850-900 down // 900ish up
- ONT > ETH > Asus > ETH > MBPro -> 550 down // 700-800 up
- Wifi 5g AC (80MHz Channel) -> 300ish down // 500-600 up

Asus Router Factory Reset + JFFS Reset and followed guide on setting up standard settings

Thinking that the router just isn't quite able to handle the symmetrical gigabit speeds.

Info for upgrade suggestions:
- Almost 1700sqft Home (Approx 1200-1400sqft per floor, Total does not include garage)
- 2 Floors
- Mainly Drywall
- No Ethernet Drops (Yet - Future Project in the works. Approx 20-24 drops upon completion)
- 100% Wifi Currently
- All Devices Support Wireless AC and a few Wifi6 AX Devices
- Several IoT 2.4G only devices (Willing to separate to own AP) (Thermostat, temp sensors, etc.)
- 2x Wyze Cam 2's
- 1x Ring Doorbell 2
- More Cam's being introduced soon
- 2x iPhones
- 4x iPads
- 2x Active Macs
- 3x Active PC's
- 1x PS4
- 1x XBox 360
- Anywhere from 0-20 guest devices at any given time (All support 5G AC minimum).
- One PC is a work PC that connects via software based Cisco VPN
- (Total of about 18-20+ devices active at any given time, and upto about 40-50 when we have guests over.)

Needs:
1) Can utilize as much of the Gigabit service as possible.
2) Can handle the Cisco Software VPN traffic with minimal impact to regular network traffic. (Cannot offload onto the router due to Smart Card/PIV/CAC style certificates)
3) Can handle 2x people gaming and streaming to Twitch/Youtube/Mixer/TikTok/Likee/etc. at the same time. (Potentially 2-4 streams at the same time depending on who's streaming in the house and what platform/s they are streaming too. Yes I know about Restream and utilize it for myself, but not available for apps used by the other user in the house)
4) Can be locked down as good as or better than the Asus+RMerlin routers.
5) Preferably some AX (WiFi6) AP Suggestions to go along with it as several devices in the home offer this.
6) I would like to isolate my my network into roughly 3-6 separate VLANs to isolate my work PC, IoT devices, guests and IP Cameras from my main network.

Consumer AiO's I feel are on the way out for what I'm going to need to do with my home network.

I'm willing to get my hands dirty and learn things like PFSense, Linux, etc. as long as I'm pointed in the right direction.

Budget: Unknown, but if I had to throw a number out there ideal $300-400 range // $500 Max (Willing to up depending on suggestions)

I have seen the new UDM Pro, although I also see the mixed reviews.

Side Note - I do also have a Dell Optiplex 7060 Micro i7 (16gb ram) available w/ 1x onboard gigabit port + 2x USB3 Gigabit Ethernet adapters I can use for now until I can find something more suited to my needs as well.

I will more than likely be setting the RT-AC3100 up as an extra AP along with

If I get an AX AP/Router I will probably place the Asus RT-AC3100

I'm open to suggestions and know that I probably left something out.

 

[Roadspill]

I am definitely more willing to invest in a separate router and switch/s and keep the RT-AC3100 as an AP, and just wait for WiFi 6 to be more widely adopted and more AP options available.

 

[Trip]

I'd throw a $30 Intel I340 and an SSD in that Optiplex and run pfSense or OpenWRT as your firewall, then re-purpose the 3100 as just an AP (factory-reset + latest Merlin + manual re-config). If WLAN still under-performs, I'd rip and replace with Qualcomm hardware, either an all-in-one to blast the whole house (R7800, RT2600ac) or two controller-based APs (TP-Link EAP series or Ubiquiti UniFi) set at very low transmit power, one on the top floor, one on the bottom, remote AP hard-wired if possible, or wirelessly uplinked if needed. The latter will get you VLAN support out-of-the-box, so you can properly segment your private, guest and IoT traffic; plus, if you can hard-wire both APs, more fronthaul channel space to serve your growing collection of clients.

 

[Roadspill]

I'd throw a $30 Intel I340 and an SSD in that Optiplex and run pfSense or OpenWRT as your firewall, then re-purpose the 3100 as just an AP (factory-reset + latest Merlin + manual re-config). If WLAN still under-performs, I'd rip and replace with Qualcomm hardware, either an all-in-one to blast the whole house (R7800, RT2600ac) or two controller-based APs (TP-Link EAP series or Ubiquiti UniFi) set at very low transmit power, one on the top floor, one on the bottom, remote AP hard-wired if possible, or wirelessly uplinked if needed. The latter will get you VLAN support out-of-the-box, so you can properly segment your private, guest and IoT traffic; plus, if you can hard-wire both APs, more fronthaul channel space to serve your growing collection of clients.

The Optiplex that I have is an USFF PC, so it can't take any PCIe cards. Only USB3 to ETH adapters.

 

[Trip]

The Optiplex that I have is an USFF PC, so it can't take any PCIe cards. Only USB3 to ETH adapters.

Whoops. Gotcha. I wouldn't monkey with adapters, then. I'd save the box for client/media/micro-server duty and instead just get a sub-$100 SFF with a multi-NIC card off eBay or cheap embedded box like a PC-Engines APU2 (with Intel I210AT, not the 211 -- 4 queues per port on the 210, only 2 with the 211), or a Qotom/Protectli unit off Amazon.

 

[mackworth]

I am in a similar boat. I have a symmetric gigabit connection and would love to get rid of my buffer bloat with something with fq_codel or cake. So that leaves PFSense or Openwrt. The APU2 @Trip mentioned is almost perfect, but its just not quite fast enough for an SQM on gigabit but would be perfect if you don't care about SQM. Other popular machines are the HP T730 and HP T620 Plus for PFSense, can find those on eBay for decent prices. Just need to add a PCIe network card.

But on a side note, I am surprised your Asus router can't handle basic routing of gigabit. My 68U can.

 

[Trip]

@mackworth - With a symmetric gig WAN, I'm surprised you've got noticeable bloat. If the 68U is in the router role, then I'd wager it probably can't truly NAT 2Gb/s, which is what would be required in order not to buffer any traffic from LAN-to-WAN and back again. Getting something in there can could might solve your bufferbloat issues alone, without SQM being needed. (Remember bufferbloat occurs in rough proportion to how differing the link speeds are across interfaces -- a 1Gb full duplex LAN egressing to a 1Gb full duplex WAN should theoretically have no bloat, provided the router can NAT 2Gb/s continually)

 

[Roadspill]

I figured it should have handled it, but I'm seeing about 1.2Gbps combined max throughput from the router that it's been able to handle. I'm going to play with it a little more this weekend, but truly at a loss that the crappy router they supplied is able to handle it with less specs. Oh well, off to the drawing board or the purchasing of a new router/hardware here soon.

 

[doczenith1]

For what it's worth the AC3100 is more than capable of attaining the full speeds of your fiber service. I had a 3100 for a little bit and could easily hit 940 Mbps both ways. That said, I could also hit 940 both ways on my older 68U as @mackworth mentioned. I'm curious of what "standard settings" you used other than some of the wireless settings tweaks to avoid wireless issues.

That said, as you mentioned you'll need more than a consumer all in one router to do you VLANs.