QOS causing high pings

[GregS]

I'm having several issues with QOS but one that's easy to reproduce is with ping times. With no real traffic my pings are 10-30ms; when there's traffic that's being actively shaped by QOS my ping shots up to 200+ms with the occasional 1+ sec response and a few timeouts.

To simplify testing I am using Filezilla FTP connected via ethernet to upload a single file to a server with much higher downstream bandwidth than my upstream allows. I have QOS on the Low (default priority) to 50% on the upload bandwidth. Within Merlin and Filezilla I can see the upload is correctly being restricted to roughly 50% while the pings are erratic and 10x what is normal. The same happens on any other setting, for example 10% or 100%. If I set Filezilla to throttle and use the same 10% or 50% setting there's a slight 10-20ms rise in ping before dropping back down to normal after a few seconds. The CPU on the router is flat lined at 0-1%, RAM is only at 28% used.

I'm using Merlin's latest version (378.51) on an RT-AC68W which I got new and setup a few days ago so most things are still set to the default except for the following. I enabled AIProtection, traffic stats with apps analysis and per-client enabled, with stats saving to a usb drive every hour. I setup a few nat-start rules, mainly to specify a source IP. JFSS and SNMP are enabled. For QOS I'm using Traditional Type and have tried many different settings including much lower bandwidth numbers than I actually get (18/3 mbps). I have a few port forwards and a extra few QOS rules for a few games and newsgroups; none of which were in use during the testing. My old router an RT-N16 using tomato didn't have this problem and I tried replicating the settings I had on it but it didn't help. As suggested in another thread I unchecked all the "Highest Priority Packets" options but that made no difference.

Some other problems which perhaps could help point to non-QOS issues:
2) Occasionally after a website triggered router reboot none of the LAN clients can connect to the router or anything else; the lights are blinking like normal and the Wireless clients have no problems. This has happened with QOS enabled or disabled; turning the router on/off resolves the issue. While this is happening if I login to the router from a wireless client the Network Map shows a total of 0 clients, -5 LAN, 5 Wireless but it shows an error instead of listing any of them. The system log shows the DHCP requests from some of the LAN clients along with the IP returned but none of the LAN clients report hearing back from DHCP.
3) Occasionally I'll have a problem where any LAN traffic is limited to around 2.5mbit/s up and down, doesn't matter if it's Wireless to LAN, or WAN to LAN. Wireless to WAN traffic and LAN to LAN traffic was not impacted. This just happened again after ejecting a USB flash drive; not sure what preceded the other times. The issue went away after disabled QOS but that also causes a reboot. Just happened again after tweaking some QOS rules; rebooting fixed it.


Any help/suggestions would be greatly appreciated.

 

[GregS]

I just tried resetting to factory defaults, loading the new firmware (378.52) and resetting to defaults again. I then restored my prior configuration via http://www.snbforums.com/threads/user-nvram-save-restore-utility.19521/ and I'm still seeing the same QOS issue as well as issue #3. I've yet to see #2 return.

 

[GregS]

I ran with QOS turned off for a week and saw no issues. Then I turned on Adaptive QOS (not traditional) and immediately saw issue #2 but it was worse than before as no device Wireless or LAN could connect to the router even though all the lights were blinking like normal. A power cycle resolved the issue. Since switching to Adaptive QOS I haven't had an issue with high pings nor any real problem since the power cycle.

 

[Nullity]

There is no such thing as simple QoS, at least, not yet. There is fq_codel and a few other worth-while "fair queuing" algorithms, but there is nothing that will magically know what packets you want sent quickly, and which packets are tolerant of delay. Sadly, real QoS (guarantees on worst-case delay/bandwidth) is only accomplished by strict traffic guidelines. Fair queueing, though, is practically a universal improvement in worst-case delay with no configuration, as is CoDel, which is a huge improvement for casual users. Just to give you an example of CoDel's awesomeness: I ran two ping tests, while fully saturating my upload.
Without CoDel: ~600ms ping
With CoDel: ~50ms ping
and that is with no configuration at all. Just enable CoDel on my outgoing queue and bam... Really, this is what people should be focusing on getting to the average consumer. Oh, my ping during low-traffic is 10-20ms.


but yeah, stay away from Adaptive QoS. Unless it is telepathic, there's nothing special happening there. If you have some understanding of iptables, you could setup SFQ easily. Even on my aging RT-N66U I have an implementation of SFQ, HFSC, and TBF. They are all powerful qeueuing algorithms.

admin@merlin:/tmp/home/root# ls /lib/modules/*/kernel/net/sched/
sch_esfq.ko  sch_hfsc.ko  sch_tbf.ko

I have not personally used ESFQ but if it adheres to the SFQ paper, then it is simple to setup. I would go find a router with (fq_)CoDel though...


Always make sure to set your QoS bandwidth lower than your real-world throughtput, otherwise the buffering/queueing will happen at your ISP (most likely) and their buffers will be deep FIFO buffers. You must be the pinch point for bandwidth to control the queueing.

 

[RMerlin]

I always found pings to be a bad way of testing QoS's efficiency. Because in a proper QoS environment, you would typically want ICMP traffic to have a lower priority than regular traffic, which means ping times should be expected to be higher.

 

[Nullity]

I have read that ICMP should be prioritized the same as as the bulk of your traffic, so that ping's RTT will be an accurate representation for a majority of your traffic.

I have also read that you should prioritize ICMP and open your firewall to ping responses and other ICMP packets to assure you get the best service possible.

And then there's deprioritzing ICMP, as well.



I suppose it just depends on the user/admin. None of them seem blatantly wrong.

 

[GregS]

If ping isn't a good way of testing, what do you suggest? My two latency sensitive applications are VOIP and gaming which aren't very easy to run tests and quantify.

The reason I enabled Adaptive QOS yesterday is because I was on a VOIP call while something was downloading in the background and the call quality was terrible. After enabling QOS I was able to continue the call at a more typical quality level but it did have occasional issues; though it's possible they were outside of my network.

 

[Nullity]

Naw, ping is fine if you understand it's limitations.

I would just use standard QoS. If there are too many games to prioritize, maybe it would be easier to deprioritize the most common ports, like 20, 21, 80, 443, torrents, etc. Well over 90% of my throughput is in just a few ports.


Just as an example, 10-15 years ago I played Counter Strike on a backwoods 768Kbit/128Kbit ADSL connection and I had more than enough bandwidth and processing power. I think your network is slightly fubhuggered.


Hmm... actually, could some jackasses from those games be (D)DoSing you? That could explain why lag only happens during gameplay...

 

[GregS]

I can't get the standard/Traditional QOS working right; seeing high pings and packetloss with a single FTP upload. Even without an upload I'm seeing packetloss while connected to my work VPN. Neither was an issue with Adaptive mode. So maybe it's my setup; you see anything wrong?

 

[Nullity]

Looks fine to me. When you say packet loss, where are getting this information? Technically, TCP is working properly if it is dropping packets. That is how the TCP congestion avoidance algorithms throttle their speed. Some packet loss is perfectly normal.

Personally, I would probably simplify your QoS. Practically just have "VIP" and "do not care". Attempting to do much more than that without fully comprehending how your QoS is interacting with your connection leads to unpredictable problems.

 

[GregS]

For my ping testing I typically ping my Internet gateway (first public IP shown in a traceroute) or one the public dns servers like 4.2.2.2.

I'd actually only need 3 categories, VIP, default and bulk that can be slowed with no real impact.

 

[drawz]

I always found pings to be a bad way of testing QoS's efficiency. Because in a proper QoS environment, you would typically want ICMP traffic to have a lower priority than regular traffic, which means ping times should be expected to be higher.

I would argue that a proper QoS environment doesn't require prioritizing applications. It just works. And that is pretty much what OpenWrt QoS/SQM with fq_codel does.