RT-BE88U - 10G speeds are 1G

[vanarebane]

Currently I have set up like this:

From router, SFP+ 10G going to my PC trough fiber.
From router, 10G RJ45 going to my NAS, to ASUS XG-C100C card.
Maximum speeds I get with this are 1G exactly, from PC to NAS.

When I connect NAS to 2.5G, I get 2.5G speeds exactly.
I checked my PC and NAS, and they have MTU of 9000.
The router has no Jumbo Frames settings (it's hidden option, as also mentioned in other thread here)

When I SSH into the router, I see MTU's only 1500 for everything.
The eth0 and eth9, that are the 10G ports, have MTU 1500 and is reported 10G speeds.
With that MTU, I should still expect speeds around 6~7G.
Manually tiring to increase the MTU trough SSH, I get "ifconfig mtu SIOCSIFMTU: Invalid argument", only values up to 1500 are accepted.

I'm not sure why the speed between two 10G ports is just 1G

 

[Squall Leonhart]

sounds like a contact asus situation.

Dong managed nearly 9Gbit out of his review part.

 

[jzchen]

Have you tried turning off Jumbo Packets and testing?

 

[vanarebane]

I just got new RT-BE88U back from RMA (bad WiFi 2.4 radio).

sounds like a contact asus situation.

Dong managed nearly 9Gbit out of his review part.

I did contact ASUS, they are asking almost every detail about the setup to understand the problem.

Have you tried turning off Jumbo Packets and testing?

In my RT-BE88U, the Jumbo Packets are hidden in the admin menu and looks like disabled by default.

In a more in-depth analysis, the problem seems to be with the ASUS XG-C100C v2 NIC card, in the NAS. As it does not like to be bonded and the Proxmox software I'm using, needs to make all networks bonded.
By using iperf3 I now get 1G/9G speed in one direction though. From unRaid (that runs in VM) > Proxmox > RT-BE88U > Fiber to MikroTik CRS305-1G-4S+IN > PC with Mellanox Connectx-3.
But between Proxmox and PC I get 2.5G/2.5G speeds. And between Proxmox and RT-BE88U I get 1G/2.5G

So iperf3 is just limited to the device performance, not networks... so my complaints are invalid here and I'll continue figuring why the PC>NAS is 1G/10G right now.

Thanks for thinking with me here!

 

[jzchen]

Jumbo packets (as I understand) are tricky. Is there a network switch in the middle, and if so does it support the same 9000 packet size?

(As a confession I've never tried SSH and I suspect you have a better understanding than I)....

I just verified on my BE98 Pro WebGUI

Under Advanced Settings -> LAN -> Switch Control (tab) is the Jumbo frame/packet setting

I'd be very surprised if yours does not have it?

It's my understanding that both ends and any configurable switches in between must have the exact same 9000 in order for it to work properly.

 

[vanarebane]

In RT-BE88U, there is only Bonding option (this seems to be setting for 2x LAN to NAS only)

Rest of the options are hidden in HTML. I can manually un-hide them, but not sure if there was a important reason why those options are disabled?

Poking around in SSH, I see that the 10G links are not MTU 9000

And those can't be set to 9000 manually either. So I'd rather look for other reason before breaking the router.