fluctuating latency behavior

[Pierre Nakashian]

i have RT-AC5300 running merlin 386.2_6, latency behavior is not new, just getting on my nerves i guess.

the router ac5300 is behind att NVG599 modem 1GB plan.

ping from NVG599 to amazon gives me
--------------------------------------------
PING www.amazon.com (13.249.56.40): 56 data bytes
64 bytes from 13.249.56.40: seq=0 ttl=244 time=11.914 ms
64 bytes from 13.249.56.40: seq=1 ttl=244 time=11.265 ms
64 bytes from 13.249.56.40: seq=2 ttl=244 time=11.300 ms
64 bytes from 13.249.56.40: seq=3 ttl=244 time=11.278 ms

--- www.amazon.com ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max = 11.265/11.439/11.914 ms

ping from inside RT-AC5300 via ssh gives me
--------------------------------------------
ping -c 4 www.amazon.com
PING www.amazon.com (54.230.30.152): 56 data bytes
64 bytes from 54.230.30.152: seq=0 ttl=231 time=24.347 ms
64 bytes from 54.230.30.152: seq=1 ttl=231 time=24.026 ms
64 bytes from 54.230.30.152: seq=2 ttl=231 time=25.967 ms
64 bytes from 54.230.30.152: seq=3 ttl=231 time=23.937 ms

--- www.amazon.com ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max = 23.937/24.569/25.967 ms


a sudden extra 13ms latency


traceroute from NVG599 gives me
--------------------------------------
traceroute to www.amazon.com (13.249.56.40), 30 hops max, 38 byte packets
1 99-67-236-1.lightspeed.austtx.sbcglobal.net (99.67.236.1) 1.539 ms 1.082 ms 1.172 ms
2 71.149.77.250 (71.149.77.250) 2.790 ms 2.921 ms 3.270 ms
3 12.240.220.238 (12.240.220.238) 12.103 ms 13.247 ms 11.138 ms
4 dlstx21crs.ip.att.net (12.122.1.209) 13.325 ms 13.315 ms 17.635 ms
5 12.122.2.198 (12.122.2.198) 15.593 ms 16.666 ms 18.304 ms
6 12.123.212.177 (12.123.212.177) 10.789 ms 10.183 ms 10.433 ms
7 32.140.105.242 (32.140.105.242) 11.559 ms 17.269 ms 12.154 ms
8 *

traceroute with -I from inside RT-AC5300 gives me
--------------------------------------------------------
traceroute to www.amazon.com (13.32.230.20), 30 hops max, 38 byte packets
1 192.168.1.254 (192.168.1.254) 0.617 ms 0.401 ms 0.345 ms
2 99-67-236-1.lightspeed.austtx.sbcglobal.net (99.67.236.1) 1.768 ms 1.669 ms 1.657 ms
3 71.149.77.250 (71.149.77.250) 3.223 ms 3.850 ms 3.971 ms
4 12.240.220.238 (12.240.220.238) 27.612 ms 29.447 ms 31.912 ms
5 attga21crs.ip.att.net (12.122.2.109) 28.141 ms 28.638 ms 31.941 ms
6 gbr8.attga.ip.att.net (12.122.96.101) 26.800 ms 26.972 ms 26.446 ms
7 12.244.0.26 (12.244.0.26) 23.240 ms 23.283 ms 23.181 ms
8 *


from the traceroute it seems the issue is not between RT-AC5300 and NVG599
the extra latency seems to be happening upstream at 12.240.220.238 IP along the hops
RT-AC5300 is configured with IP passthrough mode in the NVG599 modem, closest thing to bridge mode.

I know traffic from my router will have public IP address, traffic from NVG599 not sure but they could be traversing upstream with ip 192.168.1.254 instead
as far as I know, since i have no SSH access inside the NVG599.


i stopped using the ATT DNS, i noticed it was having packet loss, a month ago, also packet loss with cloudflare DNS IP,
but no packet loss with google dns, or quad 9 dns.

last night ping to ATT DNS IP every 4 pings ping latency would double up, I rebooted my router and the ping latency start behaving better again.
i have the RT-AC5300 normally scheduled for reboot once a week, so it had been only 2 days since it had been up.

I know my router is overworked, i have alot of LAN devices around 96 leases, i'm upgrading the router shortly, i've also run some dslreport.com speed test with my computer
directly connected to RT-AC5300, i get around 700mbs for the 1GB plan with buffer bloat. I can't use QOS at my speed, even if i replace my router with RT-AX88U i hear
consumer router cpu isn't fast enought for 1Gbit plans.


I've played with -> ifconfig eth0 txqueuelen 20000 it seems to help a little, it doesn't make ping response time completely consistent, and it doesn't eliminate
bufferbloat completely but it improves it enough not to get C/D anymore.

is my issue my ISP doing something different when traffic is from my public IP address versus, traffic from NVG599
with private address 192.168.1.0/24?
or is my issue too many LAN devices for my router RT-AC5300?
or is there a firewall rule my router has that can affect latency behavior upstream, even though my ping test had no packet loss?

I have turned off all firewall rules on the NVG599 modem.

Thanks

 

[ColinTaylor]

Ping an IP address (like 104.26.9.66) rather than a CDN host name like www.amazon.com otherwise you're going to different destinations. Your first ping was going to Texas and the second went to Georgia. Also, don't ping from the router, ping from a client on your LAN.

 

[Pierre Nakashian]

didn't think of CDN ip's going all over.

ping from windows physical connection
--------------------------------------------
ping -c 4 13.249.56.40
PING 13.249.56.40 (13.249.56.40): 56 data bytes
64 bytes from 13.249.56.40: icmp_seq=0 ttl=242 time=15.476 ms
64 bytes from 13.249.56.40: icmp_seq=1 ttl=242 time=19.920 ms
64 bytes from 13.249.56.40: icmp_seq=2 ttl=242 time=12.676 ms
64 bytes from 13.249.56.40: icmp_seq=3 ttl=242 time=14.596 ms
--- 13.249.56.40 ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max/stddev = 12.676/15.667/19.920/2.656 ms

tracert from windows physical connection
------------------------------------------------------
TRACERT.EXE 13.249.56.40

Tracing route to server-13-249-56-40.iah50.r.cloudfront.net [13.249.56.40]ages automake,pkg-config,make,gcc-core,libncurses-devel,lib
over a maximum of 30 hops:

1 <1 ms <1 ms <1 ms RT-AC5300-2F10.rtac5300 [192.168.2.1]
2 1 ms <1 ms 1 ms 192.168.1.254
3 2 ms 2 ms 3 ms 99-67-236-1.lightspeed.austtx.sbcglobal.net [99.67.236.1]
4 4 ms 3 ms 4 ms 71.149.77.250
5 16 ms 15 ms 15 ms 12.240.220.238
6 19 ms 15 ms 18 ms dlstx21crs.ip.att.net [12.122.1.209]
7 14 ms 15 ms 14 ms 12.122.2.198
8 12 ms 11 ms 11 ms 12.123.212.177


latencies are more comparable,



physical connections on pc also has no buffer bloat, only wifi connections has buffer bloat.

The only strange thing left is pinging from the router to my ISP DNS i get packet loss around 30%,
--- 68.94.156.9 ping statistics ---
10 packets transmitted, 7 packets received, 30% packet loss
round-trip min/avg/max = 8.981/9.274/10.104 ms

i've switched to 9.9.9.9 as my new DNS,

pinging to my ISP DNS IP from PC i still had packet loss but only 2%

--- 68.94.156.9 ping statistics ---
548 packets transmitted, 534 packets received, 2% packet loss
round-trip min/avg/max/stddev = 9.158/12.612/141.176/11.754 ms


quad9 dns no packet loss

--- 9.9.9.9 ping statistics ---
285 packets transmitted, 285 packets received, 0% packet loss
round-trip min/avg/max/stddev = 39.091/40.608/124.543/5.204 ms


too bad, my ISP dns having little latency is so eratic, switching to 9.9.9.9 is not eratic but has more consistent latency.

 

[przemekwawa]

Have you read this?

Wi-Fi Ping Spikes: Causes and Fixes - SmallNetBuilder

Wi-Fi ping spikes aren't always caused by a poor connection. Your router settings could be to blame.

www.smallnetbuilder.com

Another thing you could use is e.g. raspberry pi with pi-hole and unbound - I mean, unbound is more important for this topic, but generally I am happy with those two for my whole network.

 

[Pierre Nakashian]

i did implement adguard on a raspberry pi after I saw packet loss to my ISP dns.
I'm not too familiar with unbound, but with adguard and raspberry pi setup using quad9 dns it still did not help with my wifi latency fluctuation and packet loss even from my windows 10 notebook. I did follow that link then search for similar topics in google, i did disable a few windows 10 services/startup apps, it helped a little. Today I finished configuring RT-AX88u to replace RT-AC5300 as the main router, and installed a newer driver for my windows 10 wifi interface, the latency fluctuations is normal now (new router and windows 10 now support wifi 6).

ping over wifi from notebook
--- 68.94.156.9 ping statistics ---
81 packets transmitted, 81 packets received, 0% packet loss
round-trip min/avg/max = 8.549/9.294/15.815 ms

bufferbloat over wifi seems to be slightly better dslreports.com gives me from grade A to C. I recall I had B to D sometimes F before
I'll have read more topics how to configure WIFI optimally for RT-AX88u.

I can't complain on speed and bufferbloat over physical network connection now, it is consistent A, and
pinging from inside the router to my isp DNS IP i don't have any packet loss.

ping from inside the RT-AX88U
--- 68.94.156.9 ping statistics ---
82 packets transmitted, 82 packets received, 0% packet loss
round-trip min/avg/max = 8.904/10.096/47.866 ms

Maybe my RT-AC5300 had some incompatibility issues with my ISP internet service. RT-AX88U has a new options in the WAN section
NAT Type, i set that to Fullcone, maybe that helped or the 4 core cpu is helping handle 1 GB traffic speed.

 

[przemekwawa]

Unbound is recursive and caching dns resolver, so at beginning it is slower (but still fast imho), but later, when it has everything in cache its super fast on lan. You can try it, but it can only make your dns resolving from cache faster.

 

[Vexira]

Are you using any form of QoS?

 

[Pierre Nakashian]

No, I know I can't with a 1gbit internet plan, it'll cause high cpu usage, prior experience.

wired connections now on RT-AX88U I virtually have no issues, except with a small percent packet loss when i ping from inside the router to my ISP DNS ip address and strangely with cloudflare.com ip (1.1.1.1) but not with quad9 ip (9.9.9.9) or google dns ip (8.8.8.8).

A number of AT&T customers have complained at AT&T forums on the AT&T dns ip packet loss without any actions by AT&T.

cpu usage on the new RT-AX88u is much lower with less cpu spikes than the RT-AC5300.
i recall on the RT-AC5300 I had high cpu spikes (sometimes 50% for a few seconds) coming from "networkmap --bootwait" even when I wasn't logged in the gui. my guess is I have too many lan devices for the RT-AC5300 to handle, so i just upgraded to a quad core CPU ASUS router.

 

[Pierre Nakashian]

this may help someone else, disabling "Enable WMM APSD" on 5G wifi significantly improved buffer bloating.
then switching 5g channels to ch 36 significantly improved download speed to 559 upload speed went up to 428,
dslreport score for buffer bloat score went up to A, speed score went up to A+.

I started on rt-ac5300 when I had these issues, when I moved to rt-AX88u I copied over settings to the rt-ax88u where it was applicable put settings in manually of course.

for wireless on a fiber optic gigabit plan the performance now is very good I'm happy.

 

[Svenskmat]

I had very similar problems with my RT-AC5300. You may not want to hear this, but when I did a complete factory reset and then a new setup ( but kept the same SSID). After I did the factory reset, the latency issue went away. I would suggest using 386.2_4 firmware to troubleshoot this.