Slow LAN speed bonded 802.3ad

[D34DC3N73R]

I recently upgraded my internet to gigabit through spectrum (940/40) and noticed I'm not getting good speeds when running speedtest-cli on my home server. My router is the RT-AC88u running Merlin 384.11_beta1 with Bonding/ Link aggregation enabled. I'm using a Rosewill RNG-407-Dualv2 NIC with Ubuntu 18.04. I have a cat8 cable from the modem to the router and cat7 cables from the router to the server. Currently, I'm getting around 350-400 mbps down.

$ ethtool bond0
Settings for bond0:
Supported ports: [ ]
Supported link modes: Not reported
Supported pause frame use: No
Supports auto-negotiation: No
Supported FEC modes: Not reported
Advertised link modes: Not reported
Advertised pause frame use: No
Advertised auto-negotiation: No
Advertised FEC modes: Not reported
Speed: 2000Mb/s
Duplex: Full
Port: Other
PHYAD: 0
Transceiver: internal
Auto-negotiation: off
Cannot get wake-on-lan settings: Operation not permitted
Link detected: yes

$ cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)

Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer2 (0)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0

802.3ad info
LACP rate: fast
Min links: 0
Aggregator selection policy (ad_select): stable

Slave Interface: enp5s0
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 3
Permanent HW addr: 68:1c:a2:12:5f:fa
Slave queue ID: 0
Aggregator ID: 1
Actor Churn State: none
Partner Churn State: none
Actor Churned Count: 0
Partner Churned Count: 1

Slave Interface: enp4s0
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 3
Permanent HW addr: 68:1c:a2:12:5f:f9
Slave queue ID: 0
Aggregator ID: 1
Actor Churn State: none
Partner Churn State: none
Actor Churned Count: 1
Partner Churned Count: 1

Tools > sysinfo reports that HW acceleration is enabled. LAN > Switch Control, NAT acceleration is set to auto and reporting "CTF (Cut Through Forwarding) is enabled."

Running analiti on an nvidia shield gets me about 500 mbps down. When I upgraded to the beta, I did a full wipe to see if that resolved any issues and even before enabling DoT, or firewall etc, my speeds weren't what I'd expect them to be. I guess I'm wondering what I can do to achieve speeds closer to the 940mbps I should be getting.

I know there's a lot that could be causing the slowdown so please let me know what other info might be needed to diagnose this issue.

 

[ApexRon]

I would plug a laptop into Spectrum directly to confirm speed. If correct speed then plug laptop into port of ASUS and check speed.

 

[sfx2000]

Bonding does not double speed...

It doubles capacity only

The 88u should be ok with gigabit speed - check QoS settings, worst case would be to reset everything and start over, layering on features that you might find useful

 

[D34DC3N73R]

I would plug a laptop into Spectrum directly to confirm speed. If correct speed then plug laptop into port of ASUS and check speed.

I don't have a laptop with an Ethernet port, but using a usb3 Ethernet adapter, I get 350 down. I'll be able to test direct from the modem shortly.

 

[D34DC3N73R]

Bonding does not double speed...

It doubles capacity only

The 88u should be ok with gigabit speed - check QoS settings, worst case would be to reset everything and start over, layering on features that you might find useful

Right, I get that. My thought was it'd be nice to still get near gigabit speed, while a local file transfer is occuring. I did reset the router after upgrading to the beta and still had the same issue. The only settings changed were the bonding setting, and a port forward.

 

[sfx2000]

but using a usb3 Ethernet adapter, I get 350 down

that might explain much...

350 is good for gigabit ethernet over a USB2 port...

 

[D34DC3N73R]

that might explain much...

350 is good for gigabit ethernet over a USB2 port...

It was a usb3 adapter and a usb3 port. If what I've read is correct, the capable throughput should be around 5gbps. So I shouldn't have an issue reaching gigabit speeds with that setup. Test was done using a cat5e cable.

To clarify, usb3 throughput should be around 5gbps. It's still a gb Ethernet adapter.

 

[sfx2000]

It was a usb3 adapter and a usb3 port. If what I've read is correct, the capable throughput should be around 5gbps. So I shouldn't have an issue reaching gigabit speeds with that setup. Test was done using a cat5e cable.

To clarify, usb3 throughput should be around 5gbps. It's still a gb Ethernet adapter.

so sorry to disappoint you... there's two common chipsets out there for USB gigabit adapters - one is RealTek, the other is Asix - and 350 is what you're going to get with USB3 on them

I've got a Thunderbolt adapter on MacOS - and it's real gigabit - but that's expected, as it's a PCI-e interface there.

 

[D34DC3N73R]

so sorry to disappoint you... there's two common chipsets out there for USB gigabit adapters - one is RealTek, the other is Asix - and 350 is what you're going to get with USB3 on them

I've got a Thunderbolt adapter on MacOS - and it's real gigabit - but that's expected, as it's a PCI-e interface there.

It's the ugreen adapter which seems to have the AX88179 chipset. Reading other reviews seems to indicate it'll top out around 550. I find it suspicious that it's getting the same speed (350) as my server. I'll test it out connected directly to the modem, as well as connecting the shield directly to the modem.

 

[D34DC3N73R]

I think I'm putting too much faith in the speedtest. I'm getting over 500 with usenet on the server, even though I'm topping out around 350 with the speedtest. I'm building a home office computer now, so I'll have a windows box I can test with soon. I'll update within the week with results.

 

[D34DC3N73R]

So, I finally got this running at high speeds. After multiple resets with no resolution, I noticed cpu0 would spike up to 100% when downloading even at 200 mbps. Checked out top and had a high sirq on cpu0 when downloading as well. CTF was reported as enabled, but it didn't seem like it was working. I didn't have any entries for ctf_fa_mode or ctf_fa_cap when running nvram show | grep ctf. I disabled adaptive QOS and set ctf_fa_mode and ctf_fa_cap to 2. Now I have CTF and FA enabled, and I'm getting around 850 mbps using speedtest-cli on my server.

 

[Makaveli]

Borrow a laptop from a friend with an Ethernet port on it. And then test I wouldn't trust any usb adapters regardless of what they put on the box.

 

[D34DC3N73R]

Borrow a laptop from a friend with an Ethernet port on it. And then test I wouldn't trust any usb adapters regardless of what they put on the box.

No need as it's working now near the advertised speed. I did just build an office computer for my girlfriend and the day I got a 20' cat6 cable is the same day I figured out the hardware acceleration settings.