CPU load and its impact on latency.

[reerden]

I have a Ac66u and a 120/12mbit connection.

I was never able to reach my actual speeds with Qos due to NAT acceleration being disabled. So I disabled it.

However, I recently noticed when testing something out, that when I simplify my QOS rules, I am able to reach 120/12mbit with QOS without NAT acceleration.

However, the CPU hovers between 95%-99% at those speeds. I'm worried this will negatively impact latency when simultaneously maxing out my download and upload.

So should it be safe to use QOS with these speeds? Or is it never a good idea to run a router with speeds that pushes it to its limits, since it will then start buffering packets since it can't handle the load, causing high ping times.

 

[L&LD]

The device and the hardware inside are meant to be used at their maximums. If they fail sometime in the future to provide that level of performance, that is when I would worry about it.

Use it now with QoS and CTF disabled. That is the optimum settings for you, right now.

And I'm sure you don't peg the connection 24 / 7 / 365.

 

[reerden]

The device and the hardware inside are meant to be used at their maximums. If they fail sometime in the future to provide that level of performance, that is when I would worry about it.

Use it now with QoS and CTF disabled. That is the optimum settings for you, right now.

And I'm sure you don't peg the connection 24 / 7 / 365.

I can reach my speeds as long as I keep my Qos rules simple, so why disable it? Is it because I might not reach my upload speed when downloading at full speed?

Also, why disable CTF? What are the downsides of CTF when you're not using anything that's incompatible with it?

 

[L&LD]

If the simple rules allow your network usage to be better than with CTF enabled, that is one reason to disable CTF (enabling QoS disables CTF).

There are no downsides in normal setups with CTF. But again, if your user experience is enhanced with a feature that disables CTF and still gives you your maximum throughput on the 'net, the answer is obvious to me.

I can reach my speeds as long as I keep my Qos rules simple, so why disable it? Is it because I might not reach my upload speed when downloading at full speed?

Also, why disable CTF? What are the downsides of CTF when you're not using anything that's incompatible with it?

 

[reerden]

If the simple rules allow your network usage to be better than with CTF enabled, that is one reason to disable CTF (enabling QoS disables CTF).

There are no downsides in normal setups with CTF. But again, if your user experience is enhanced with a feature that disables CTF and still gives you your maximum throughput on the 'net, the answer is obvious to me.

Thanks for the answer. I was just concerned that the high CPU usage would cause too much buffering (bufferbloat), which would make QOS useless.

So the high CPU usage won't cause any issues besides a slightly lower speed?

 

[L&LD]

Thanks for the answer. I was just concerned that the high CPU usage would cause too much buffering (bufferbloat), which would make QOS useless.

So the high CPU usage won't cause any issues besides a slightly lower speed?

Again, it is in the testing that the answer will be found; use what works in practice, not what theory may or may not indicate what the truth will be.

 

[netwrks]

QoS on, automatically disables CTF, which you want enabled (CTF) for 100+mb links. Check out the link below..

http://www.snbforums.com/threads/broadcoms-hardware-acceleration.18144/

 

[reerden]

Again, it is in the testing that the answer will be found; use what works in practice, not what theory may or may not indicate what the truth will be.

Alright I tested it again, and it seems that it can reach 120/12. However, every time another process on the router requests only a little CPU time, the speed immediately dropped to 107mbps. So I think I'm going to leave Qos disabled for the sake of stability and CTF on to keep CPU load low.

It seems that my earlier tests where kind off skewed because it was late in the evening and the ISPs network is busy with everyone streaming Netflix and downloading who knows what.

QoS on, automatically disables CTF, which you want enabled (CTF) for 100+mb links. Check out the link below..

http://www.snbforums.com/threads/broadcoms-hardware-acceleration.18144/

I know. This was more about the impact of high CPU load, as I was able to reach my maximum speed without CTF and with Qos, but with 100% CPU load which could possibly negatively impact latency.

 

[L&LD]

That depends on the router and how powerful the cpu it uses (or can overclock to) is.

It also indicates what the maximum for a single user or device will be, not what the overall performance of the network for all users is.

For example, if QoS gave each user a smoother and lag free network / internet experience, for many people that would beat having QoS disabled and giving even much faster performance for a single user. Of course, the goal is to have both highest possible speed with smooth and lag free performance for all, but when there is a choice to make, the better choice is usually for the network as a whole and not for a single device or user.

QoS on, automatically disables CTF, which you want enabled (CTF) for 100+mb links. Check out the link below..

http://www.snbforums.com/threads/broadcoms-hardware-acceleration.18144/

 

[L&LD]

What other process?

A drop of a few percent in Mbps for a single device does not mean latency increased. The total Mbps of the two processes should be the same, but more importantly, the latency to each should be in the acceptable range with QoS on, which it may not be with it off.

Alright I tested it again, and it seems that it can reach 120/12. However, every time another process on the router requests only a little CPU time, the speed immediately dropped to 107mbps. So I think I'm going to leave Qos disabled for the sake of stability and CTF on to keep CPU load low.

It seems that my earlier tests where kind off skewed because it was late in the evening and the ISPs network is busy with everyone streaming Netflix and downloading who knows what.

 

[Calisro]

simple. CPUs are on/off mechanisms. high loads will delay processing which will slow throughput. The question here though is will it impact it enough for you to notice? not sure as that depends on YOUR needs. If the cpu gets pegged at 100%, likely you will notice for that duration as it will queue the workload.

 

[reerden]

simple. CPUs are on/off mechanisms. high loads will delay processing which will slow throughput. The question here though is will it impact it enough for you to notice? not sure as that depends on YOUR needs. If the cpu gets pegged at 100%, likely you will notice for that duration as it will queue the workload.

The idea was to reduce bufferbloat and reserve bandwidth for latency sensitive programs (gaming, VOIP). My question was mainly if those issues also occur when the router is experiencing slowdown due to high CPU load. After some more testing and the answers in this thread, I've come to the conclusion that it does. So I'll leave Qos disabled.

 

[System Error Message]

with QoS enabled depending on the router and settings has buffer which introduces a slight latency. Hardware processing obviously has lower latency (but also limited resources) and latency can increase based on the number of rules that it has to go through such as firewall, QoS,etc which is mainly evident on configurable routers however the CPU is fast that the latency increase is insignificant.

The AC66U uses a single core MIPS CPU so when it gets other tasks it slows down current tasks which would obviously be network processing. If you want to use some firewall features and QoS you will have to disable other features on the router. High CPU load doesnt cause lag. What increases latency is buffering (not enough bandwidth/some QoS settings), slow memory that is filled with transfers and CPU that takes cycles. Usually when a CPU is fully loaded it will drop new packets assuming it is fully loaded since the 100% CPU use is because of the networking software keeping the CPU busy with useless instructions to make sure that it doesnt clock down or go into power save and wake up each time it has to process a packet.

So if you see high CPU load from normal internet use it is totally normal on many MIPS based CPUs that use software NAT and doesnt affect latency unless you burden the router with too many tasks.

The difference between software and hardware NAT aside from features is that hardware NAT causes issues when its memory is fully used and has very limited memory (from kilobytes to a few megabytes) whereas software NAT uses RAM to keep track of connections so it has a lot more memory.

 

[reerden]

with QoS enabled depending on the router and settings has buffer which introduces a slight latency. Hardware processing obviously has lower latency (but also limited resources) and latency can increase based on the number of rules that it has to go through such as firewall, QoS,etc which is mainly evident on configurable routers however the CPU is fast that the latency increase is insignificant.

The AC66U uses a single core MIPS CPU so when it gets other tasks it slows down current tasks which would obviously be network processing. If you want to use some firewall features and QoS you will have to disable other features on the router. High CPU load doesnt cause lag. What increases latency is buffering (not enough bandwidth/some QoS settings), slow memory that is filled with transfers and CPU that takes cycles. Usually when a CPU is fully loaded it will drop new packets assuming it is fully loaded since the 100% CPU use is because of the networking software keeping the CPU busy with useless instructions to make sure that it doesnt clock down or go into power save and wake up each time it has to process a packet.

So if you see high CPU load from normal internet use it is totally normal on many MIPS based CPUs that use software NAT and doesnt affect latency unless you burden the router with too many tasks.

The difference between software and hardware NAT aside from features is that hardware NAT causes issues when its memory is fully used and has very limited memory (from kilobytes to a few megabytes) whereas software NAT uses RAM to keep track of connections so it has a lot more memory.

How do you know that HW processing has such a small memory? The CTF module is closed source.

Also, wouldn't that cause massive connection problems for people using torrents or gaming clients? Since those tend to open a large number of UDP connections. (this was often a problem back in the days when I still had my good old WRT54G router).

 

[System Error Message]

The hardware NAT is quick to drop older connections to get around the issue since it uses things like the switch chip which has around <2MB to a few MB of memory which is technically enough for connections that are currently going through. During hardware NAT inactive connections are kept in ram or dropped but you can still overload the router if you have too many active connections (it would require many many thousands). some high end consumer routers that have accelerator chips in them have a few MB of ram for them that can easily handle many thousands of connections.

Its easy to know even though it is closed source because of the hardware.

 

[Nullity]

The general rule is that QoS becomes important (assuming you need QoS at all) during time-spans with 80% or higher throughput utilization.


Honestly, I do not know if an overloaded CPU is considered bufferbloat. I believe it is a seperate topic, though I cannot articulate why.


Most people do not need QoS, since 99% of internetworked services can cope with multi-second delays without ever glitching or suffering in any way.


BTW, run a router with CoDel and all your bufferbloat problems will instantly vanish.

 

[NGRhodes]

The general rule is that QoS becomes important (assuming you need QoS at all) during time-spans with 80% or higher throughput utilization.

If your not hitting your bandwidth limits, then QoS won't need to buffer/drop any packets, every connection is already getting enough bandwidth.

Honestly, I do not know if an overloaded CPU is considered bufferbloat. I believe it is a seperate topic, though I cannot articulate why.

It is adding delays to, but is not bufferbloat. Bufferbloat is phenomenon of excessively buffering tcp/ip connections. An overloaded is just not able to process the packets fast enough.

Most people do not need QoS, since 99% of internetworked services can cope with multi-second delays without ever glitching or suffering in any way.

These multi-second delays are what bufferbloat aims to prevent. The services can cope, but it adds to a poor user experience.

BTW, run a router with CoDel and all your bufferbloat problems will instantly vanish.

I find FQ-CoDel is not perfect, for example it does not work at all with torrents saturating a connection, I need a bit of QoS to help.
Properly configured Max bandwidth settings up and down usually helps reduce buffering delays by helping prevent buffers getting full (which is needed for QoS to work properly).

 

[Nullity]

Codel should work fine with torrents, but only on egress. Ingress should really only use traffic policing (bufferless) or perhaps traffic shaping (buffered) and not QoS or AQMs like Codel.

I have tried to find data regarding whether buffering (QoS) at a low bandwidth to high bandwidth interchange (ex. 100mbit WAN to 1GbE LAN) has any benefits, and found nothing. Why buffer (delay) data that can be transmitted?