Performance between Orbi devices

[mith_y2k]

There is a thread about the performance of Orbis over ethernet that raised a question in my mind: how fast is my ethernet connection between the router and my two satellites? In particular one of them is connected via ethernet, so I expected speeds close to 1Gb. I tried a couple of times using iperf3 as described in http://www.snbforums.com/threads/orbi-rbr50-rbs50-ethernet-performance.78176/ and the performance is consistently around 450Mb/s. That's about half I would have expected. Am I wrong? Is the Orbi Ethernet not 1Gb? Do I have a flaky connection? Curious what others are seeing especially since Voxel has iperf3 pre-installed maybe we can all run some tests! Geek-party

 

[HELLO_wORLD]

I don't have any Orbi, so I cannot test it.

For reference, between a R7800 (server) and an Odroid N2 (client) plugged to it via ethernet Gb, I get
an average of 922 Mbits/sec in IPv6 and 940 Mbits/sec in IPv4.

 

[thiggins]

I'd be careful using the iperf3 instance installed on the Orbi unless you first run a test using an iperf3 client directly connected to it to establish a baseline measurement.

 

[HELLO_wORLD]

I'd be careful using the iperf3 instance installed on the Orbi unless you first run a test using an iperf3 client directly connected to it to establish a baseline measurement.

It is always good practice to make a baseline measurement.

Unless the Orbi has a very poor cpu or some kind of hardware bottleneck problem, iperf3 should perform nicely. @Voxel 's firmware is always extremely optimized.

 

[mith_y2k]

I'd be careful using the iperf3 instance installed on the Orbi unless you first run a test using an iperf3 client directly connected to it to establish a baseline measurement.

I'm not sure I get this. I am using the RBK50 with Voxel as server and RBS50 with Voxel as client. They are connected with an Ethernet wire that I assume is cat 6, but I will be checking.

What would you suggest the baseline to be?

 

[Tech Junky]

@mith_y2k

With a 450 reading I suspect it's operating in 1/2 duplex. The other thing with these node type devices is they typically use wireless backhaul and since it's plugged in that may need to be disabled for full performance.

 

[HELLO_wORLD]

I'm not sure I get this. I am using the RBK50 with Voxel as server and RBS50 with Voxel as client. They are connected with an Ethernet wire that I assume is cat 6, but I will be checking.

What would you suggest the baseline to be?

I would try RBK50 as server, and a device you know is at max capability as client (sbc, PC…). Then the same reversing server and client.
Then same procedure with RBS50.
By testing one device at a time instead of both, you will know if one is limited, or both.

I doubt iPerf3 is at fault, and I think @Tech Junky is up to something with the half duplex. You should be able to check that with ethtool command

root@HERMES:~$ ethtool ethwan
Settings for ethwan:
Supported ports: [ ]
Supported link modes: Not reported
Supported pause frame use: No
Supports auto-negotiation: No
Supported FEC modes: Not reported
Advertised link modes: Not reported
Advertised pause frame use: No
Advertised auto-negotiation: No
Advertised FEC modes: Not reported
Speed: 1000Mb/s
Duplex: Full
Port: Twisted Pair
PHYAD: 0
Transceiver: internal
Auto-negotiation: off
MDI-X: Unknown
Supports Wake-on: d
Wake-on: d
Current message level: 0x00000000 (0)

Link detected: yes

You need to find the physical interface with ifconfig (probably starting with eth, not br).

 

[thiggins]

I'm not sure I get this. I am using the RBK50 with Voxel as server and RBS50 with Voxel as client. They are connected with an Ethernet wire that I assume is cat 6, but I will be checking.

What would you suggest the baseline to be?

Now I understand your setup better. You are dealing with three unknowns: performance of the iperf3 server, performance of the iperf3 client and the Wi-Fi connection between them.

Start by connecting a computer with iperf3 via Ethernet to the Orbi router and run a baseline test. You should get ~ 940 Mbps. Repeat on the Orbi satellite. Then connect router and satellite via Ethernet and run a measurement. Finally, run your test with a Wi-Fi connection between the Orbis. Move them close together and see if throughput improves.

I don't agree with the "half duplex" theory. Wi-Fi by its nature only operates in one direction at a time and you are not running a bi-directional test,

 

[HELLO_wORLD]

To check full/half duplex status (to make sure this is not about it), there is also these commands:

detcable show rbr

or

detcable show rbs

it shows current link, example:
----------------------------------
root@RBR50:~# detcable show rbr
---------------Port 1 UP---------------
---------------Port 2 UP---------------
---------------Port 4 UP---------------
LAN0 : Plug in, 1000M, Full duplex
LAN1 : Plug in, 100M, Full duplex
LAN2 : Plug off
LAN3 : Plug in, 1000M, Full duplex
----------------------------------

 

[mith_y2k]

Now I understand your setup better. You are dealing with three unknowns: performance of the iperf3 server, performance of the iperf3 client and the Wi-Fi connection between them.

Router and satellite are connected via ethernet backhaul. According to the Orbi web interface ethernet is in use.

I will follow up with the steps suggested by @HELLO_wORLD for more debugging

 

[HELLO_wORLD]

For reference / info @Voxel tested in the past RBR<->LBR (WAN<->LAN) and his speeds were about 650Mbit, both sides.
With stock he had similar results or slightly lower.

 

[moecre]

Ahh...somebody was hijacking my topic I see

Just joking. It's great this topic get's a bit of traction though.

I have one RBR and 4 RBS around the house. Was thinking of running wires to every one of them to replace the wireless backhaul with a wired one.

BUT there's no point in doing this if the ethernet performance of the Orbi's don't reach 9xx Mbit/s.

I'm curious where this thread is going...

[EDIT]: I just did another test from my Macbook (client) to the RBR50 (server), 284 Mbits/sec and 647 Mbits/sec if I reverse the direction. That's ridiculous! I do the same test to my little HP server which is just on a different port on the same switch than the RBR and I get the full 950 Mbits/sec. Just through cables no WIFI involved.

root@RBR50:~# detcable show rbr
---------------Port 1 UP---------------
LAN0 : Plug in, 1000M, Full duplex
LAN1 : Plug off
LAN2 : Plug off
LAN3 : Plug off

 

[Tech Junky]

while convenient there's still more to be done with getting these types of devices up to par with their advertising. ~$500 for an experiment doesn't work in my book. It's supposed to be under promise and over deliver.

for the price of these node setups you can get a couple of decent AP's and hardwire them for line speed WIFI.

 

[thiggins]

I just did another test from my Macbook (client) to the RBR50 (server), 284 Mbits/sec and 647 Mbits/sec if I reverse the direction. That's ridiculous! I do the same test to my little HP server which is just on a different port on the same switch than the RBR and I get the full 950 Mbits/sec. Just through cables no WIFI involved.

What do you get when you connect your little HP server running iperf3 to a port on the RBR50 and connect your MacBook via Ethernet to another RBR port? Take the performance of the internal iperf3 server out of the equation. It's not in a stock Orbi anyway.

 

[moecre]

What do you get when you connect your little HP server running iperf3 to a port on the RBR50 and connect your MacBook via Ethernet to another RBR port? Take the performance of the internal iperf3 server out of the equation. It's not in a stock Orbi anyway.

Hi @thiggins, I just did exactly that, together with a few more tests. See below.

All tests are focused on ethernet performance, so using Cat6 wires throughout and ensured that all components are able to handle gigabit. All Orbis run Voxel's current firmware.

These tests confirm what @thiggins was suggesting all the time: the measurements we do on the Orbis (whether it's a RBR or RBS) with iperf3 are not representative. They don't really display the performance of the ethernet ports. As you can see on test 3, 6 and 7.

I'm glad that in the other tests, where the measures were taken on client (MacBook) and Server (HP MicroServer) it didn't matter how I connected them, the gigabit performance went through RBR50 and RBS50. So they are able to pass traffic at these speeds (as advertised). That's good to see.

I stuffed a test (4) in the middle which shows the performance of the WIFI Backhaul, just for comparison to ethernet (might not be conclusive though).

Test 1: Client <> Switch <> Server (baseline w/o Orbi)
1:  [ ID] Interval           Transfer     Bitrate
1:  [  7]   0.00-60.00  sec  1.52 GBytes   218 Mbits/sec                  sender
1:  [  7]   0.00-60.00  sec  1.52 GBytes   218 Mbits/sec                  receiver
1:  [  9]   0.00-60.00  sec  1.53 GBytes   218 Mbits/sec                  sender
1:  [  9]   0.00-60.00  sec  1.52 GBytes   218 Mbits/sec                  receiver
1:  [ 11]   0.00-60.00  sec  1.52 GBytes   218 Mbits/sec                  sender
1:  [ 11]   0.00-60.00  sec  1.52 GBytes   218 Mbits/sec                  receiver
1:  [ 13]   0.00-60.00  sec  1.52 GBytes   217 Mbits/sec                  sender
1:  [ 13]   0.00-60.00  sec  1.51 GBytes   217 Mbits/sec                  receiver
1:  [SUM]   0.00-60.00  sec  6.09 GBytes   872 Mbits/sec                  sender
1:  [SUM]   0.00-60.00  sec  6.08 GBytes   871 Mbits/sec                  receiver

Test 2: Client <> RBR50 <> Switch <> Server (with Orbi in the middle)
1:  [ ID] Interval           Transfer     Bitrate
1:  [  7]   0.00-60.00  sec   991 MBytes   139 Mbits/sec                  sender
1:  [  7]   0.00-60.00  sec   991 MBytes   139 Mbits/sec                  receiver
1:  [  9]   0.00-60.00  sec  1.09 GBytes   155 Mbits/sec                  sender
1:  [  9]   0.00-60.00  sec  1.08 GBytes   155 Mbits/sec                  receiver
1:  [ 11]   0.00-60.00  sec  2.01 GBytes   288 Mbits/sec                  sender
1:  [ 11]   0.00-60.00  sec  2.01 GBytes   288 Mbits/sec                  receiver
1:  [ 13]   0.00-60.00  sec  1.96 GBytes   280 Mbits/sec                  sender
1:  [ 13]   0.00-60.00  sec  1.96 GBytes   280 Mbits/sec                  receiver
1:  [SUM]   0.00-60.00  sec  6.02 GBytes   862 Mbits/sec                  sender
1:  [SUM]   0.00-60.00  sec  6.02 GBytes   861 Mbits/sec                  receiver

Test 3: Client <> RBR50 (Server)
1:  [ ID] Interval           Transfer     Bitrate
1:  [  5]   0.00-60.00  sec   503 MBytes  70.3 Mbits/sec                  sender
1:  [  5]   0.00-60.01  sec   500 MBytes  69.9 Mbits/sec                  receiver
1:  [  7]   0.00-60.00  sec   490 MBytes  68.6 Mbits/sec                  sender
1:  [  7]   0.00-60.01  sec   488 MBytes  68.3 Mbits/sec                  receiver
1:  [  9]   0.00-60.00  sec   513 MBytes  71.7 Mbits/sec                  sender
1:  [  9]   0.00-60.01  sec   511 MBytes  71.4 Mbits/sec                  receiver
1:  [ 11]   0.00-60.00  sec   511 MBytes  71.5 Mbits/sec                  sender
1:  [ 11]   0.00-60.01  sec   507 MBytes  70.9 Mbits/sec                  receiver
1:  [SUM]   0.00-60.00  sec  1.97 GBytes   282 Mbits/sec                  sender
1:  [SUM]   0.00-60.01  sec  1.96 GBytes   281 Mbits/sec                  receiver

Test 4: Client <> RBS50 <> WIFI Backhaul (same room, 3 metres apart) <> RBR50 <> Switch <> Server
1:  [ ID] Interval           Transfer     Bitrate
1:  [  7]   0.00-60.00  sec   993 MBytes   139 Mbits/sec                  sender
1:  [  7]   0.00-60.00  sec   993 MBytes   139 Mbits/sec                  receiver
1:  [  9]   0.00-60.00  sec   766 MBytes   107 Mbits/sec                  sender
1:  [  9]   0.00-60.00  sec   765 MBytes   107 Mbits/sec                  receiver
1:  [ 11]   0.00-60.00  sec  1.84 GBytes   264 Mbits/sec                  sender
1:  [ 11]   0.00-60.00  sec  1.84 GBytes   264 Mbits/sec                  receiver
1:  [ 13]   0.00-60.00  sec  1.05 GBytes   150 Mbits/sec                  sender
1:  [ 13]   0.00-60.00  sec  1.05 GBytes   150 Mbits/sec                  receiver
1:  [SUM]   0.00-60.00  sec  4.61 GBytes   659 Mbits/sec                  sender
1:  [SUM]   0.00-60.00  sec  4.60 GBytes   659 Mbits/sec                  receiver

Test 5: Client <> RBS50 <> Ethernet Backhaul <> RBR50 <> Switch <> Server
1:  [ ID] Interval           Transfer     Bitrate
1:  [  7]   0.00-60.00  sec  1.67 GBytes   240 Mbits/sec                  sender
1:  [  7]   0.00-60.00  sec  1.67 GBytes   239 Mbits/sec                  receiver
1:  [  9]   0.00-60.00  sec  1.01 GBytes   145 Mbits/sec                  sender
1:  [  9]   0.00-60.00  sec  1.01 GBytes   145 Mbits/sec                  receiver
1:  [ 11]   0.00-60.00  sec  1.85 GBytes   265 Mbits/sec                  sender
1:  [ 11]   0.00-60.00  sec  1.85 GBytes   265 Mbits/sec                  receiver
1:  [ 13]   0.00-60.00  sec  1.56 GBytes   224 Mbits/sec                  sender
1:  [ 13]   0.00-60.00  sec  1.56 GBytes   224 Mbits/sec                  receiver
1:  [SUM]   0.00-60.00  sec  6.10 GBytes   874 Mbits/sec                  sender
1:  [SUM]   0.00-60.00  sec  6.10 GBytes   873 Mbits/sec                  receiver

Test 6: Client <> RBS50 (Server)
1:  [ ID] Interval           Transfer     Bitrate
1:  [  5]   0.00-60.00  sec   434 MBytes  60.6 Mbits/sec                  sender
1:  [  5]   0.00-60.02  sec   432 MBytes  60.3 Mbits/sec                  receiver
1:  [  7]   0.00-60.00  sec   435 MBytes  60.8 Mbits/sec                  sender
1:  [  7]   0.00-60.02  sec   433 MBytes  60.5 Mbits/sec                  receiver
1:  [  9]   0.00-60.00  sec   437 MBytes  61.0 Mbits/sec                  sender
1:  [  9]   0.00-60.02  sec   435 MBytes  60.8 Mbits/sec                  receiver
1:  [ 11]   0.00-60.00  sec   494 MBytes  69.1 Mbits/sec                  sender
1:  [ 11]   0.00-60.02  sec   494 MBytes  69.1 Mbits/sec                  receiver
1:  [SUM]   0.00-60.00  sec  1.76 GBytes   252 Mbits/sec                  sender
1:  [SUM]   0.00-60.02  sec  1.75 GBytes   251 Mbits/sec                  receiver

Test 7: Client <> RBS50 <> Ethernet Backhaul <> RBR50 (Server)
1:  [ ID] Interval           Transfer     Bitrate
1:  [  5]   0.00-60.00  sec   485 MBytes  67.8 Mbits/sec                  sender
1:  [  5]   0.00-60.00  sec   484 MBytes  67.7 Mbits/sec                  receiver
1:  [  7]   0.00-60.00  sec   556 MBytes  77.8 Mbits/sec                  sender
1:  [  7]   0.00-60.00  sec   555 MBytes  77.6 Mbits/sec                  receiver
1:  [  9]   0.00-60.00  sec   501 MBytes  70.0 Mbits/sec                  sender
1:  [  9]   0.00-60.00  sec   500 MBytes  69.9 Mbits/sec                  receiver
1:  [ 11]   0.00-60.00  sec   732 MBytes   102 Mbits/sec                  sender
1:  [ 11]   0.00-60.00  sec   729 MBytes   102 Mbits/sec                  receiver
1:  [SUM]   0.00-60.00  sec  2.22 GBytes   318 Mbits/sec                  sender
1:  [SUM]   0.00-60.00  sec  2.22 GBytes   317 Mbits/sec                  receiver

 

[HELLO_wORLD]

Interesting. That confirms that routers are good at routing (hardware optimized for forwarding), but not so much for anything else…
It is exactly like trying to do a Speedtest from a router… it maxes the CPU and gives bad results.

 

[thiggins]

Hi @thiggins, I just did exactly that, together with a few more tests. See below.

All tests are focused on ethernet performance, so using Cat6 wires throughout and ensured that all components are able to handle gigabit. All Orbis run Voxel's current firmware.

These tests confirm what @thiggins was suggesting all the time: the measurements we do on the Orbis (whether it's a RBR or RBS) with iperf3 are not representative. They don't really display the performance of the ethernet ports. As you can see on test 3, 6 and 7.

I'm glad that in the other tests, where the measures were taken on client (MacBook) and Server (HP MicroServer) it didn't matter how I connected them, the gigabit performance went through RBR50 and RBS50. So they are able to pass traffic at these speeds (as advertised). That's good to see.

I stuffed a test (4) in the middle which shows the performance of the WIFI Backhaul, just for comparison to ethernet (might not be conclusive though).

Test 1: Client <> Switch <> Server (baseline w/o Orbi)
1:  [ ID] Interval           Transfer     Bitrate
1:  [  7]   0.00-60.00  sec  1.52 GBytes   218 Mbits/sec                  sender
1:  [  7]   0.00-60.00  sec  1.52 GBytes   218 Mbits/sec                  receiver
1:  [  9]   0.00-60.00  sec  1.53 GBytes   218 Mbits/sec                  sender
1:  [  9]   0.00-60.00  sec  1.52 GBytes   218 Mbits/sec                  receiver
1:  [ 11]   0.00-60.00  sec  1.52 GBytes   218 Mbits/sec                  sender
1:  [ 11]   0.00-60.00  sec  1.52 GBytes   218 Mbits/sec                  receiver
1:  [ 13]   0.00-60.00  sec  1.52 GBytes   217 Mbits/sec                  sender
1:  [ 13]   0.00-60.00  sec  1.51 GBytes   217 Mbits/sec                  receiver
1:  [SUM]   0.00-60.00  sec  6.09 GBytes   872 Mbits/sec                  sender
1:  [SUM]   0.00-60.00  sec  6.08 GBytes   871 Mbits/sec                  receiver

Test 2: Client <> RBR50 <> Switch <> Server (with Orbi in the middle)
1:  [ ID] Interval           Transfer     Bitrate
1:  [  7]   0.00-60.00  sec   991 MBytes   139 Mbits/sec                  sender
1:  [  7]   0.00-60.00  sec   991 MBytes   139 Mbits/sec                  receiver
1:  [  9]   0.00-60.00  sec  1.09 GBytes   155 Mbits/sec                  sender
1:  [  9]   0.00-60.00  sec  1.08 GBytes   155 Mbits/sec                  receiver
1:  [ 11]   0.00-60.00  sec  2.01 GBytes   288 Mbits/sec                  sender
1:  [ 11]   0.00-60.00  sec  2.01 GBytes   288 Mbits/sec                  receiver
1:  [ 13]   0.00-60.00  sec  1.96 GBytes   280 Mbits/sec                  sender
1:  [ 13]   0.00-60.00  sec  1.96 GBytes   280 Mbits/sec                  receiver
1:  [SUM]   0.00-60.00  sec  6.02 GBytes   862 Mbits/sec                  sender
1:  [SUM]   0.00-60.00  sec  6.02 GBytes   861 Mbits/sec                  receiver

Test 3: Client <> RBR50 (Server)
1:  [ ID] Interval           Transfer     Bitrate
1:  [  5]   0.00-60.00  sec   503 MBytes  70.3 Mbits/sec                  sender
1:  [  5]   0.00-60.01  sec   500 MBytes  69.9 Mbits/sec                  receiver
1:  [  7]   0.00-60.00  sec   490 MBytes  68.6 Mbits/sec                  sender
1:  [  7]   0.00-60.01  sec   488 MBytes  68.3 Mbits/sec                  receiver
1:  [  9]   0.00-60.00  sec   513 MBytes  71.7 Mbits/sec                  sender
1:  [  9]   0.00-60.01  sec   511 MBytes  71.4 Mbits/sec                  receiver
1:  [ 11]   0.00-60.00  sec   511 MBytes  71.5 Mbits/sec                  sender
1:  [ 11]   0.00-60.01  sec   507 MBytes  70.9 Mbits/sec                  receiver
1:  [SUM]   0.00-60.00  sec  1.97 GBytes   282 Mbits/sec                  sender
1:  [SUM]   0.00-60.01  sec  1.96 GBytes   281 Mbits/sec                  receiver

Test 4: Client <> RBS50 <> WIFI Backhaul (same room, 3 metres apart) <> RBR50 <> Switch <> Server
1:  [ ID] Interval           Transfer     Bitrate
1:  [  7]   0.00-60.00  sec   993 MBytes   139 Mbits/sec                  sender
1:  [  7]   0.00-60.00  sec   993 MBytes   139 Mbits/sec                  receiver
1:  [  9]   0.00-60.00  sec   766 MBytes   107 Mbits/sec                  sender
1:  [  9]   0.00-60.00  sec   765 MBytes   107 Mbits/sec                  receiver
1:  [ 11]   0.00-60.00  sec  1.84 GBytes   264 Mbits/sec                  sender
1:  [ 11]   0.00-60.00  sec  1.84 GBytes   264 Mbits/sec                  receiver
1:  [ 13]   0.00-60.00  sec  1.05 GBytes   150 Mbits/sec                  sender
1:  [ 13]   0.00-60.00  sec  1.05 GBytes   150 Mbits/sec                  receiver
1:  [SUM]   0.00-60.00  sec  4.61 GBytes   659 Mbits/sec                  sender
1:  [SUM]   0.00-60.00  sec  4.60 GBytes   659 Mbits/sec                  receiver

Test 5: Client <> RBS50 <> Ethernet Backhaul <> RBR50 <> Switch <> Server
1:  [ ID] Interval           Transfer     Bitrate
1:  [  7]   0.00-60.00  sec  1.67 GBytes   240 Mbits/sec                  sender
1:  [  7]   0.00-60.00  sec  1.67 GBytes   239 Mbits/sec                  receiver
1:  [  9]   0.00-60.00  sec  1.01 GBytes   145 Mbits/sec                  sender
1:  [  9]   0.00-60.00  sec  1.01 GBytes   145 Mbits/sec                  receiver
1:  [ 11]   0.00-60.00  sec  1.85 GBytes   265 Mbits/sec                  sender
1:  [ 11]   0.00-60.00  sec  1.85 GBytes   265 Mbits/sec                  receiver
1:  [ 13]   0.00-60.00  sec  1.56 GBytes   224 Mbits/sec                  sender
1:  [ 13]   0.00-60.00  sec  1.56 GBytes   224 Mbits/sec                  receiver
1:  [SUM]   0.00-60.00  sec  6.10 GBytes   874 Mbits/sec                  sender
1:  [SUM]   0.00-60.00  sec  6.10 GBytes   873 Mbits/sec                  receiver

Test 6: Client <> RBS50 (Server)
1:  [ ID] Interval           Transfer     Bitrate
1:  [  5]   0.00-60.00  sec   434 MBytes  60.6 Mbits/sec                  sender
1:  [  5]   0.00-60.02  sec   432 MBytes  60.3 Mbits/sec                  receiver
1:  [  7]   0.00-60.00  sec   435 MBytes  60.8 Mbits/sec                  sender
1:  [  7]   0.00-60.02  sec   433 MBytes  60.5 Mbits/sec                  receiver
1:  [  9]   0.00-60.00  sec   437 MBytes  61.0 Mbits/sec                  sender
1:  [  9]   0.00-60.02  sec   435 MBytes  60.8 Mbits/sec                  receiver
1:  [ 11]   0.00-60.00  sec   494 MBytes  69.1 Mbits/sec                  sender
1:  [ 11]   0.00-60.02  sec   494 MBytes  69.1 Mbits/sec                  receiver
1:  [SUM]   0.00-60.00  sec  1.76 GBytes   252 Mbits/sec                  sender
1:  [SUM]   0.00-60.02  sec  1.75 GBytes   251 Mbits/sec                  receiver

Test 7: Client <> RBS50 <> Ethernet Backhaul <> RBR50 (Server)
1:  [ ID] Interval           Transfer     Bitrate
1:  [  5]   0.00-60.00  sec   485 MBytes  67.8 Mbits/sec                  sender
1:  [  5]   0.00-60.00  sec   484 MBytes  67.7 Mbits/sec                  receiver
1:  [  7]   0.00-60.00  sec   556 MBytes  77.8 Mbits/sec                  sender
1:  [  7]   0.00-60.00  sec   555 MBytes  77.6 Mbits/sec                  receiver
1:  [  9]   0.00-60.00  sec   501 MBytes  70.0 Mbits/sec                  sender
1:  [  9]   0.00-60.00  sec   500 MBytes  69.9 Mbits/sec                  receiver
1:  [ 11]   0.00-60.00  sec   732 MBytes   102 Mbits/sec                  sender
1:  [ 11]   0.00-60.00  sec   729 MBytes   102 Mbits/sec                  receiver
1:  [SUM]   0.00-60.00  sec  2.22 GBytes   318 Mbits/sec                  sender
1:  [SUM]   0.00-60.00  sec  2.22 GBytes   317 Mbits/sec                  receiver

Thanks for doing the work and sharing the results.

 

[Tech Junky]

That confirms that routers are good at routing

On the consumer level at least. Enterprise devices might fair better but, typically wouldn't be speedtesting from their devices but through apps that monitor and accumulate the data for reporting purposes.

DIY though can get beyond the limitations of ineffective off the shelf equipment. All it takes is Linux and some reverse engineering to make it work. Using a proper CPU / RAM kills even enterprise systems depending on how much $ you throw at the issue.

As to the routing / switching... ASICS do the grunt work for passing packets fast w/o hitting the CPU constantly and for routing it doesn't take much effort. The issue comes into play trying to get them to do something you would normally use a PC / Phone / Tablet to do. Most consumers aren't smart enough to know the specs of the internal HW or from what I've seen subscribed to plans high enough in speed to show the weakness of their $500 router.

 

[Atko]

I've had similar experience with the Stock and Voxel's firmware, but I don't believe that it's a specific hardware problem. When I was running Voxel's firmware, I was surprised to see that running iperf3 from the RBR50 to the RBS50 over an ethernet backhaul could sometimes slow down to 250Mbits/s. If I had to guess it's related in some way on how Netgear is managing the VLAN's on the switch ports or how it is doing the firewalling/QoS. The reason I belive this is that I can get near gigabit speeds with stock OpenWrt, no VLANs just bridging all of the interfaces:

Test 1: Client <-CAT5-> RBR50
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   276 MBytes   231 Mbits/sec  155             sender
[  5]   0.00-10.01  sec   274 MBytes   230 Mbits/sec                  receiver
[  7]   0.00-10.00  sec   298 MBytes   250 Mbits/sec  342             sender
[  7]   0.00-10.01  sec   296 MBytes   248 Mbits/sec                  receiver
[  9]   0.00-10.00  sec   275 MBytes   231 Mbits/sec  380             sender
[  9]   0.00-10.01  sec   273 MBytes   229 Mbits/sec                  receiver
[ 11]   0.00-10.00  sec   275 MBytes   230 Mbits/sec  160             sender
[ 11]   0.00-10.01  sec   273 MBytes   229 Mbits/sec                  receiver
[SUM]   0.00-10.00  sec  1.10 GBytes   942 Mbits/sec  1037             sender
[SUM]   0.00-10.01  sec  1.09 GBytes   936 Mbits/sec                  receiver

Test 2: RBS50 <-CAT5-> RBR50
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  6.18 MBytes  5.18 Mbits/sec   11             sender
[  5]   0.00-10.02  sec  6.07 MBytes  5.08 Mbits/sec                  receiver
[  7]   0.00-10.00  sec   604 MBytes   507 Mbits/sec  281             sender
[  7]   0.00-10.02  sec   604 MBytes   506 Mbits/sec                  receiver
[  9]   0.00-10.00  sec   493 MBytes   414 Mbits/sec  310             sender
[  9]   0.00-10.02  sec   493 MBytes   413 Mbits/sec                  receiver
[ 11]   0.00-10.00  sec  6.64 MBytes  5.57 Mbits/sec   16             sender
[ 11]   0.00-10.02  sec  6.50 MBytes  5.45 Mbits/sec                  receiver
[SUM]   0.00-10.00  sec  1.08 GBytes   932 Mbits/sec  618             sender
[SUM]   0.00-10.02  sec  1.08 GBytes   929 Mbits/sec                  receiver

Test 3: Client <-CAT5-> RBR50 <-CAT5-> RBS50
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   287 MBytes   241 Mbits/sec  146             sender
[  5]   0.00-10.01  sec   286 MBytes   240 Mbits/sec                  receiver
[  7]   0.00-10.00  sec   263 MBytes   220 Mbits/sec  256             sender
[  7]   0.00-10.01  sec   261 MBytes   219 Mbits/sec                  receiver
[  9]   0.00-10.00  sec   282 MBytes   237 Mbits/sec  591             sender
[  9]   0.00-10.01  sec   281 MBytes   235 Mbits/sec                  receiver
[ 11]   0.00-10.00  sec   281 MBytes   235 Mbits/sec  512             sender
[ 11]   0.00-10.01  sec   279 MBytes   234 Mbits/sec                  receiver
[SUM]   0.00-10.00  sec  1.09 GBytes   933 Mbits/sec  1505             sender
[SUM]   0.00-10.01  sec  1.08 GBytes   928 Mbits/sec                  receiver

Test 4: RBS50 <-5Ghz-> RBS50 (variable throughput due to 5Ghz interference from neighbors)
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   141 MBytes   118 Mbits/sec    0             sender
[  5]   0.00-10.00  sec   138 MBytes   116 Mbits/sec                  receiver
[  7]   0.00-10.00  sec  92.7 MBytes  77.7 Mbits/sec    0             sender
[  7]   0.00-10.00  sec  90.9 MBytes  76.2 Mbits/sec                  receiver
[  9]   0.00-10.00  sec  72.9 MBytes  61.2 Mbits/sec    0             sender
[  9]   0.00-10.00  sec  71.3 MBytes  59.8 Mbits/sec                  receiver
[ 11]   0.00-10.00  sec  87.9 MBytes  73.7 Mbits/sec    0             sender
[ 11]   0.00-10.00  sec  86.4 MBytes  72.5 Mbits/sec                  receiver
[SUM]   0.00-10.00  sec   394 MBytes   331 Mbits/sec    0             sender
[SUM]   0.00-10.00  sec   387 MBytes   324 Mbits/sec                  receiver

Test 5: RBR50 <-CAT5-> RBS50 <-5Ghz-> RBS50 (variable throughput due to 5Ghz interference from neighbors)
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  83.4 MBytes  69.9 Mbits/sec  179             sender
[  5]   0.00-10.01  sec  82.8 MBytes  69.4 Mbits/sec                  receiver
[  7]   0.00-10.00  sec  83.7 MBytes  70.2 Mbits/sec  298             sender
[  7]   0.00-10.01  sec  83.3 MBytes  69.9 Mbits/sec                  receiver
[  9]   0.00-10.00  sec  77.2 MBytes  64.7 Mbits/sec  206             sender
[  9]   0.00-10.01  sec  76.5 MBytes  64.1 Mbits/sec                  receiver
[ 11]   0.00-10.00  sec  91.1 MBytes  76.4 Mbits/sec  184             sender
[ 11]   0.00-10.01  sec  90.3 MBytes  75.7 Mbits/sec                  receiver
[SUM]   0.00-10.00  sec   335 MBytes   281 Mbits/sec  867             sender
[SUM]   0.00-10.01  sec   333 MBytes   279 Mbits/sec                  receiver

Test 6: Client <-CAT5-> RBR50 <-CAT5-> RBS50 <-5Ghz-> RBS50 (variable throughput due to 5Ghz interference from neighbors)
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  95.3 MBytes  79.9 Mbits/sec   94             sender
[  5]   0.00-10.01  sec  94.3 MBytes  79.0 Mbits/sec                  receiver
[  7]   0.00-10.00  sec  66.0 MBytes  55.4 Mbits/sec  311             sender
[  7]   0.00-10.01  sec  65.4 MBytes  54.8 Mbits/sec                  receiver
[  9]   0.00-10.00  sec   104 MBytes  87.5 Mbits/sec  133             sender
[  9]   0.00-10.01  sec   103 MBytes  86.2 Mbits/sec                  receiver
[ 11]   0.00-10.00  sec  87.5 MBytes  73.4 Mbits/sec  331             sender
[ 11]   0.00-10.01  sec  86.6 MBytes  72.5 Mbits/sec                  receiver
[SUM]   0.00-10.00  sec   353 MBytes   296 Mbits/sec  869             sender
[SUM]   0.00-10.01  sec   349 MBytes   292 Mbits/sec                  receiver

Test 7: Client <-CAT5-> RBR50 <-CAT5-> Server
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   277 MBytes   232 Mbits/sec  263             sender
[  5]   0.00-10.00  sec   275 MBytes   231 Mbits/sec                  receiver
[  7]   0.00-10.00  sec   312 MBytes   262 Mbits/sec  260             sender
[  7]   0.00-10.00  sec   310 MBytes   260 Mbits/sec                  receiver
[  9]   0.00-10.00  sec   255 MBytes   214 Mbits/sec  256             sender
[  9]   0.00-10.00  sec   254 MBytes   213 Mbits/sec                  receiver
[ 11]   0.00-10.00  sec   291 MBytes   244 Mbits/sec  267             sender
[ 11]   0.00-10.00  sec   289 MBytes   243 Mbits/sec                  receiver
[SUM]   0.00-10.00  sec  1.11 GBytes   952 Mbits/sec  1046             sender
[SUM]   0.00-10.00  sec  1.10 GBytes   946 Mbits/sec                  receiver

 

[Atko]

Working on a hunch, it seems that all ports are not equal on an RBS50. As you may, or may not ,already know the RBS50 and RBR50 are the exact same hardware inside. The stock firmware relies on a setting in the eeprom which tells the operating system which device type it is. The architecture of the RBS50/RBR50 is such that it has two ethernet ports attached, eth0 and eth1. For me, on OpenWrt, the WAN port shows up as eth1 and the LAN ports show up as eth0, attached to eth0 is a 4 port switch (Qualcomm Atheros QCA8075). Software can control VLAN tagging/trunking, QoS etc on the WAN port as well as the individual ports of the integrated switch. When the device is configured as a sattellite it bridges eth0 and eth1 in software, I'll refer to the ports as WAN and LAN below, but just know that in a satellitle effectively they are all LAN ports.

If I use a satellite and pass traffic across one of the 3 exposed LAN ports to another LAN port, I get consistent performance. The integrasted switch manages the traffic forwarding in hardware:

[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   280 MBytes   235 Mbits/sec  225             sender
[  5]   0.00-10.00  sec   279 MBytes   234 Mbits/sec                  receiver
[  7]   0.00-10.00  sec   264 MBytes   221 Mbits/sec  237             sender
[  7]   0.00-10.00  sec   263 MBytes   220 Mbits/sec                  receiver
[  9]   0.00-10.00  sec   324 MBytes   272 Mbits/sec  255             sender
[  9]   0.00-10.00  sec   323 MBytes   271 Mbits/sec                  receiver
[ 11]   0.00-10.00  sec   266 MBytes   223 Mbits/sec  227             sender
[ 11]   0.00-10.00  sec   264 MBytes   222 Mbits/sec                  receiver
[SUM]   0.00-10.00  sec  1.11 GBytes   951 Mbits/sec  944             sender
[SUM]   0.00-10.00  sec  1.10 GBytes   947 Mbits/sec                  receiver

[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   295 MBytes   247 Mbits/sec   39             sender
[  5]   0.00-10.00  sec   294 MBytes   246 Mbits/sec                  receiver
[  7]   0.00-10.00  sec   266 MBytes   223 Mbits/sec   41             sender
[  7]   0.00-10.00  sec   265 MBytes   222 Mbits/sec                  receiver
[  9]   0.00-10.00  sec   312 MBytes   262 Mbits/sec   26             sender
[  9]   0.00-10.00  sec   311 MBytes   260 Mbits/sec                  receiver
[ 11]   0.00-10.00  sec   262 MBytes   220 Mbits/sec   46             sender
[ 11]   0.00-10.00  sec   261 MBytes   219 Mbits/sec                  receiver
[SUM]   0.00-10.00  sec  1.11 GBytes   952 Mbits/sec  152             sender
[SUM]   0.00-10.00  sec  1.10 GBytes   948 Mbits/sec                  receiver

[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   280 MBytes   235 Mbits/sec  186             sender
[  5]   0.00-10.00  sec   279 MBytes   234 Mbits/sec                  receiver
[  7]   0.00-10.00  sec   280 MBytes   235 Mbits/sec  210             sender
[  7]   0.00-10.00  sec   279 MBytes   234 Mbits/sec                  receiver
[  9]   0.00-10.00  sec   272 MBytes   228 Mbits/sec  203             sender
[  9]   0.00-10.00  sec   271 MBytes   227 Mbits/sec                  receiver
[ 11]   0.00-10.00  sec   296 MBytes   248 Mbits/sec  168             sender
[ 11]   0.00-10.00  sec   295 MBytes   247 Mbits/sec                  receiver
[SUM]   0.00-10.00  sec  1.10 GBytes   946 Mbits/sec  767             sender
[SUM]   0.00-10.00  sec  1.10 GBytes   943 Mbits/sec                  receiver

If I use a satellite and pass traffic across one of the 3 exposed LAN ports to the WAN port, I get slighlty variable performance. The CPU needs to forward the traffic across eth0<->eth1 in software:

[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   275 MBytes   231 Mbits/sec  228             sender
[  5]   0.00-10.00  sec   273 MBytes   229 Mbits/sec                  receiver
[  7]   0.00-10.00  sec   270 MBytes   226 Mbits/sec  292             sender
[  7]   0.00-10.00  sec   269 MBytes   225 Mbits/sec                  receiver
[  9]   0.00-10.00  sec   280 MBytes   235 Mbits/sec  135             sender
[  9]   0.00-10.00  sec   278 MBytes   233 Mbits/sec                  receiver
[ 11]   0.00-10.00  sec   264 MBytes   222 Mbits/sec  215             sender
[ 11]   0.00-10.00  sec   262 MBytes   220 Mbits/sec                  receiver
[SUM]   0.00-10.00  sec  1.06 GBytes   914 Mbits/sec  870             sender
[SUM]   0.00-10.00  sec  1.06 GBytes   908 Mbits/sec                  receiver

[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   206 MBytes   173 Mbits/sec  192             sender
[  5]   0.00-10.00  sec   205 MBytes   172 Mbits/sec                  receiver
[  7]   0.00-10.00  sec   227 MBytes   190 Mbits/sec  155             sender
[  7]   0.00-10.00  sec   225 MBytes   189 Mbits/sec                  receiver
[  9]   0.00-10.00  sec   258 MBytes   216 Mbits/sec  325             sender
[  9]   0.00-10.00  sec   257 MBytes   215 Mbits/sec                  receiver
[ 11]   0.00-10.00  sec   289 MBytes   243 Mbits/sec  596             sender
[ 11]   0.00-10.00  sec   288 MBytes   242 Mbits/sec                  receiver
[SUM]   0.00-10.00  sec   980 MBytes   822 Mbits/sec  1268             sender
[SUM]   0.00-10.00  sec   976 MBytes   818 Mbits/sec                  receiver

[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   324 MBytes   271 Mbits/sec   81             sender
[  5]   0.00-10.01  sec   321 MBytes   269 Mbits/sec                  receiver
[  7]   0.00-10.00  sec   233 MBytes   196 Mbits/sec   91             sender
[  7]   0.00-10.01  sec   233 MBytes   195 Mbits/sec                  receiver
[  9]   0.00-10.00  sec   320 MBytes   269 Mbits/sec   70             sender
[  9]   0.00-10.01  sec   319 MBytes   268 Mbits/sec                  receiver
[ 11]   0.00-10.00  sec   253 MBytes   213 Mbits/sec   93             sender
[ 11]   0.00-10.01  sec   252 MBytes   212 Mbits/sec                  receiver
[SUM]   0.00-10.00  sec  1.10 GBytes   949 Mbits/sec  335             sender
[SUM]   0.00-10.01  sec  1.10 GBytes   944 Mbits/sec                  receiver

Adding an additional VLAN tagging would probably decrease the throughput again and on the stock firmware it uses the ethernet backhaul as a VLAN trunk adding additional overhead.