Edgerouter X speeds with QoS on? (+VPN/worst case)

[Twice_Shy]

I've seen benchmarks but I don't recall seeing any with quality of service turned on. I'm even further curious whether other features can slow it down further - ie whats an absolute worst case that this router is capable of handling? What slows it down the most?

I'm looking for something that will be capable of handling up to perhaps 300 down/100up in the future and 150 down/50 up for now.

 

[System Error Message]

If you plan to handle 400Mb/s total with NAT, QoS and VPN there is no edgerouter capable of such. I suggest you look at mikrotik instead at the RB1100AHx2 at least for performing NAT + QoS + VPN at 400Mb/s or a PC based router.

QoS performance is the bane for the edgerouters as even the ERPRO is only capable of 200Mb/s QoS without adding VPN into the mix.

 

[mlg321]

Here are the estimates for for Smart Queue and bandwidth.

 

[System Error Message]

when you're doing NAT you're looking at the lowest number as thats the confirmed case and even more skilled/experienced/high ranked members on SNB will also confirm it. It took a lot of shaming ubiquiti by someone like me before they actually started showing numbers relevant for consumers that is NAT and even than they did so in a shameful way.

There is no Edegerouter that will do what you want fast enough.

 

[sfx2000]

when you're doing NAT you're looking at the lowest number as thats the confirmed case and even more skilled/experienced/high ranked members on SNB will also confirm it.

Getting to Gigabit on a NAT - it can be done - but it takes a lot of resources (e.g. money)

I'm ok with an Intel C2000 series chip on pfSense, but I'm 150/10, and it's running easy... Put the same box in a hotel with 250 rooms, it'll probably do the same - but it ain't gonna do a gigabit on a single session, but 750 sessions at 150 Mbit, it'll do it just fine...

 

[Twice_Shy]

Thank you to everyone for being direct and to the point, the answers I got from ubiquiti were pretty vague...

What would people suggest the clear upgrade path is then?

If I leave out VPN (or have that client side for now, so the top tier router on the internet only has to sort QoS for VOIP and streaming dependant media), should an Edgerouter X be able to keep up with 120down/10up? Which router would you recommend for 200down/20up?

There are several speed tiers available all the way up to gigabit - if it's too hard to do gig and 300 speeds, i'm still curious about best for 120/10 and 200/20 speeds which are the next ones down.

 

[System Error Message]

for QoS the ERPRO can handle up to 200Mb/s so it'd be fine there and the platform was made for use with VPNs as well. Honestly those VPN routers were terrible because not only did their firmwares suck but they came with very slow CPUs. Ubiquiti on the other hand uses the same platform but with much faster CPU but even than many edgerouters are very old. The ERPRO will do 200Mb/s of QoS, NAT and VPN only if it is l2tp/ipsec with AES. If you need more speed you're gonna need mikrotik or a PC based router.

For mikrotik the RB1100AHx2 is slightly better than the CCR when it comes to single tunnel performance for VPNs otherwise the CCRs have way more performance. PC based routers tie with mikrotik CCRs but since mikrotik cant assign multiple cores to a single tunnel this is where PCs pull ahead. A PC based router will get you gigabit NAT + QoS + VPN. a mikrotik CCR1009 will do NAT + QoS at 1Gb/s and 300Mb/s of any VPN that uses AES per tunnel.

With only NAT and no QoS even the ubiquiti ERL is capable of gigabit but so are many of mikrotik routerboards with the use of hardware acceleration on both brands.

The ERL and ER-X are good for 100Mb/s. A lot have criticised the ER-X as ubiquiti hasnt been able to implement hardware acceleration but in my opinion hardware NAT is worthless for anyone but the basic user. The ER-X may be faster than the ERPRO with QoS as it should be a newer MIPS CPU running at decent frequency with 2 cores and 4 threads. So far only SNB praises the ER-X, everywhere else on the net criticises it.

I still think ubiquiti isnt right for you if you are considering a future ISP upgrade but with your current ISP it will keep up. Hands down a mikrotik CCR1009 will do NAT + QoS at 1Gb/s will still having another 7 cores at least to use for other things (-1 core for 300Mb/s of VPN using AES, 2 cores for 1Gb/s NAT + QoS) and it even has SFP as well so you wont need a modem. a PC based router running something like pfsense is a solid consideration for both performance and features in various places that neither mikrotik and ubiquiti can provide.

 

[sintro]

If you plan to handle 400Mb/s total with NAT, QoS and VPN there is no edgerouter capable of such. I suggest you look at mikrotik instead at the RB1100AHx2 at least for performing NAT + QoS + VPN at 400Mb/s or a PC based router.

QoS performance is the bane for the edgerouters as even the ERPRO is only capable of 200Mb/s QoS without adding VPN into the mix.

Yes. I have a EdgeRouter Pro ERPro-8 with a 400Mbps symmetric link. Whenever I turn on QoS of any type the speed drops down to about 230Mbps.

No good.

 

[DanH]

The USG-Pro which is the same hardware as the ERPRO does 250ish Mbps with QoS.

The hEXr3 which is the same hardware as the ER-X does about 180ish Mbps with QoS.

The ERL does about 60ish Mbps with QoS.

That's been my personal experience with using them. That is just the simplest firewall rules, and very simple queueing, so I think that is what best case scenario would be. I don't see anyone getting even close to those upper numbers provided earlier.

Really though I will give my honest opinion, QoS at 300 Mbps isn't going to do much but make bufferbloat in Speedtests look good. In reality you are very rarely going to see bufferbloat if ever. Speaking of home use.

 

[System Error Message]

actually QoS improves bufferbloat and user experience. Its essential for gaming in a mixed environment and for multiple apps to not fight for bandwidth. For example even with 1Gb/s bandwidth and gaming using less than 1Mb/s but when a download gets initiated the game suffers a lot of lag and latency spikes. If someone is watching a video at the same time it will start buffering as well.

 

[DanH]

actually QoS improves bufferbloat and user experience. Its essential for gaming in a mixed environment and for multiple apps to not fight for bandwidth. For example even with 1Gb/s bandwidth and gaming using less than 1Mb/s but when a download gets initiated the game suffers a lot of lag and latency spikes. If someone is watching a video at the same time it will start buffering as well.

Not seeing that at all ever, in a home with 3 others. Two or three of which constantly stream or download and me pretty much gaming constantly. Didn't see it using 300 Mbps cable and don't see it on 1gig fiber. That's been my experience, the only issues I ever had with ping spikes was when I had a SB6190.

I have been out of town since Saturday, two people are home right now and just Netflix and Amazon VoD are showing 99Gigs, so when my kids aren't working or going to college or at a party they are streaming something. No issues ever.

 

[System Error Message]

Not seeing that at all ever, in a home with 3 others. Two or three of which constantly stream or download and me pretty much gaming constantly. Didn't see it using 300 Mbps cable and don't see it on 1gig fiber. That's been my experience, the only issues I ever had with ping spikes was when I had a SB6190.

Thats because their downloads werent enough to hit the cap, thanks the ISP level QoS (which hates download/sharing/p2p).

GTA 5 is a p2p based game.

 

[DanH]

So when you download something that saturates the 300Mbps and are playing a P2P based game at the same time? Yup, we rarely do that.

Typical usage is two devices streaming movies/TV from Netflix, Hulu, etc..., one person watching U-verse IPTV, and me playing some MP game which may be server based or P2P. Maybe I just never happened to be playing a P2P game when one of them downloaded something at 300Mbps or now at 900Mbps with all of that going on.

Seems to be a fringe case (for me anyway) that hardly would necessitate buying a $500 CCR1009, or something more expensive.

I play a lot,a lot, a lot of Mp games and am very conscious of lag spikes. Think I have 900 hours in R6, a few thousand in BF4, etc..Like I said it was noticeable on that SB6190, but when I got a CM600, and now I'm fiber I don't see it anymore.

I did see it in Speedtest from DSL reports. With QoS I got an A+, without a C on cable . On fiber it's A+ with QoS, and either an A+ or A without. I also notice by watching it that, the spike during the test only happpend when completely saturating on the upload. Saturate the download nothing, upload less than full cap and nothing. Upload >cap slight rise.

That is when I realized, one I am very rarely saturating the download, and two I am never saturating the upload anyway. On gigabit fiber it wouldn't be until I hit ~1300 Mbps on upload until I would see something.

I was really intent on chasing that bufferbloat tiger trying different routers, until I realized you know what other than this speed test, I don't ever notice it.
So yeah now I guess if I am playing a game at 9ms ping, and someone happens to go full hog upstairs on some rare occasion, my ping might go to 29ms for a few seconds, I'm not seeing it effect me.

By the way I was going to get a CCR-1009, but got a good deal on a USGPRO. The reason I made that choice is I read the CCR had issues with IPTV. Ended up I don't use the USG for the IPTV anyway, so I probably should have just gone for the CCR anyway...life.

 

[System Error Message]

So when you download something that saturates the 300Mbps and are playing a P2P based game at the same time? Yup, we rarely do that.

Typical usage is two devices streaming movies/TV from Netflix, Hulu, etc..., one person watching U-verse IPTV, and me playing some MP game which may be server based or P2P. Maybe I just never happened to be playing a P2P game when one of them downloaded something at 300Mbps or now at 900Mbps with all of that going on.

Seems to be a fringe case (for me anyway) that hardly would necessitate buying a $500 CCR1009, or something more expensive.

I play a lot,a lot, a lot of Mp games and am very conscious of lag spikes. Think I have 900 hours in R6, a few thousand in BF4, etc..Like I said it was noticeable on that SB6190, but when I got a CM600, and now I'm fiber I don't see it anymore.

I did see it in Speedtest from DSL reports. With QoS I got an A+, without a C on cable . On fiber it's A+ with QoS, and either an A+ or A without. I also notice by watching it that, the spike during the test only happpend when completely saturating on the upload. Saturate the download nothing, upload less than full cap and nothing. Upload >cap slight rise.

That is when I realized, one I am very rarely saturating the download, and two I am never saturating the upload anyway. On gigabit fiber it wouldn't be until I hit ~1300 Mbps on upload until I would see something.

I was really intent on chasing that bufferbloat tiger trying different routers, until I realized you know what other than this speed test, I don't ever notice it.
So yeah now I guess if I am playing a game at 9ms ping, and someone happens to go full hog upstairs on some rare occasion, my ping might go to 29ms for a few seconds, I'm not seeing it effect me.

By the way I was going to get a CCR-1009, but got a good deal on a USGPRO. The reason I made that choice is I read the CCR had issues with IPTV. Ended up I don't use the USG for the IPTV anyway, so I probably should have just gone for the CCR anyway...life.

you dont need a CCR1009. If you need 2Gb/s or more throughput and QoS and firewall with multi hundred VPN performance than thats what you get. In your case the RB1100AHx2 will do which is around the same price as the ERPRO. The RB1100AHx2 is confirmed to be able to do 1Gb/s NAT without acceleration, has the same IPSEC hardware acceleration as the edgerouters but the architecture is so much better than MIPS as its closer to x86 in terms of complexity. The internal design though is weird as 1 of the ports uses PCI, 2 of the ports are CPU connected and 10 ports are in 2x5 switch chips.

There is no doubt the RB1100AHx2 will do 400Mb/s of QoS. Whats even better is that mikrotik allows you to use hardware acceleration as a form of QoS, for example streams and games can be set to be hardware accelerated to enjoy faster processing and throughput while everything else goes through the CPU.

 

[DanH]

I found when setting up a hex that any mangle rules were disabling acceleration. Maybe the other models work differently? Anyway. I have what I have now, it works perfectly fine, so no need to buy anything else. Only thing left to get is some APs to replace the Orbis when we move in our next place.. I think we finally found the house we want, and they are supposed to complete building on 5/31, so yeah! Now if I can just get my college age twins to move out...

Side note- I will say the unifi gear has a lot of broken functionality, I was surprised how a lot of touted functions really don't work or don't provide accurate info.

 

[System Error Message]

I found when setting up a hex that any mangle rules were disabling acceleration. Maybe the other models work differently? Anyway. I have what I have now, it works perfectly fine, so no need to buy anything else. Only thing left to get is some APs to replace the Orbis when we move in our next place.. I think we finally found the house we want, and they are supposed to complete building on 5/31, so yeah! Now if I can just get my college age twins to move out...

Side note- I will say the unifi gear has a lot of broken functionality, I was surprised how a lot of touted functions really don't work or don't provide accurate info.

it doesnt disable hardware NAT, what happens is that the packets are not being accelerated. For hardware acceleration to work you must set the connection not packets to be sent to fasttrack. If a packet/connection goes through the firewall first and isnt sent to fasttrack than it does not get accelerated.

For mikrotik hardware acceleration is part of its firewall so it can be configured with the firewall if you still need to do the filtering. What you do is filter like normal first than you select the connection of the packets you want to be sent to fasttrack.

No other brand lets you configure hardware NAT in such a flexible way

 

[sfx2000]

I still think ubiquiti isnt right for you if you are considering a future ISP upgrade but with your current ISP it will keep up. Hands down a mikrotik CCR1009 will do NAT + QoS at 1Gb/s will still having another 7 cores at least to use for other things (-1 core for 300Mb/s of VPN using AES, 2 cores for 1Gb/s NAT + QoS) and it even has SFP as well so you wont need a modem. a PC based router running something like pfsense is a solid consideration for both performance and features in various places that neither mikrotik and ubiquiti can provide.

Actually - I had a demo unit for testing - and while the CCR1009 is pretty nice little device, it's not what you suggest or promise - spent some time with my uTik contact, and did some tuning...

It's pretty close to the Netgate/PfSense SG-2440 - sometimes a bit faster, sometimes not - the key strength of the 2440 is it has the horsepower - it doesn't depend on HW accel except for AES-NI (which has some benefit for VPN stuff, esp. L2TP/IPSec)

The CCR1009 is pretty effective though at a 500/500 megabit symmetric connection... might be nice in a network closet, but the fans are a bit noisy, and it's a power hungry little beastie for what it does...

For home use - or home office use even - it's hard to recommend the unit - for small business, perhaps yes...

RouterOS is very powerful, but again, the learning curve with it is a bit high, even for someone with a fair amount of enterprise/carrier experience, as they tend to use their own terms and methods compared to other vendors... pfSense has a bit of a curve there as well - but this isn't much different than deal with the diffs between juniper/cisco/broadcade/etc... but this is well beyond the expectations of most "Joe Six-Pack" home users...

 

[System Error Message]

Actually - I had a demo unit for testing - and while the CCR1009 is pretty nice little device, it's not what you suggest or promise - spent some time with my uTik contact, and did some tuning...

It's pretty close to the Netgate/PfSense SG-2440 - sometimes a bit faster, sometimes not - the key strength of the 2440 is it has the horsepower - it doesn't depend on HW accel except for AES-NI (which has some benefit for VPN stuff, esp. L2TP/IPSec)

The CCR1009 is pretty effective though at a 500/500 megabit symmetric connection... might be nice in a network closet, but the fans are a bit noisy, and it's a power hungry little beastie for what it does...

For home use - or home office use even - it's hard to recommend the unit - for small business, perhaps yes...

RouterOS is very powerful, but again, the learning curve with it is a bit high, even for someone with a fair amount of enterprise/carrier experience, as they tend to use their own terms and methods compared to other vendors... pfSense has a bit of a curve there as well - but this isn't much different than deal with the diffs between juniper/cisco/broadcade/etc... but this is well beyond the expectations of most "Joe Six-Pack" home users...

Actually without hardware NAT the CCR1009 is capable of handling up to 5Gb/s of NAT. So it is good for dual gigabit WANs. Its good for users who need performance + configurability.