Zero-win packets

[NogNeetMachinaal]

Team,

For performance reasons I'm tracking Zero-win packets (among a few other things).
To make sure everybody is on the same page: a zero-win packet indicates that its sender is suffering from a lack of resources; one way or the other.

I'm running Merlin version 386.1_2 on an Asus RT-AC68U.
Over the past few months, the router is sending dozens of zero-win packets per minute.

I tried reducing the # of TCP connections going from 300k (factory default?) to 10k.
But the problem remains - despite the fact that the CPU usage is far less then 10% and memory usage is less then 30%.

What could be the trigger of this? Anyone?


Kind regards - Will

 

[Make WiFi Great Again]

how are you tracking this?

 

[NogNeetMachinaal]

With a packet-analyzer-on-steroids (aka Allegro 500 from Allegro Packets) connected to a network tap. This network tap is connected between the WAN-port of the router and the LAN-port of the ISP-modem.

The Allegro 500 is configured in a way that an alert is sent each time a zero-win packet is passing-by (among a few other triggers).

A few examples of these alerts:

One or more TCP zero window packets from IP 192.168.2.240 to IP 54.201.82.66 (api.accounts.firefox.com) were seen.
One or more TCP zero window packets from IP 192.168.2.240 to IP 2.19.195.233 (assets.msn.com) were seen.
One or more TCP zero window packets from IP 192.168.2.240 to IP 157.240.201.60 (pps.whatsapp.net) were seen.
One or more TCP zero window packets from IP 192.168.2.240 to IP 18.184.233.249 (elb-fra-amz.nimbus.bitdefender.net) were seen.

Each of these alerts comes with 192.168.2.240; which is the WAN-port of the Asus router.

Other alerts are triggered by TCP retransmissions and responsetimes of the 3-way handshakes. An example of the latter:

TCP handshake threshold exceeded for IP 18.176.92.115 (elb-tky-amz.nimbus.bitdefender.net).
Handshake time of 257 ms exceeds threshold of 250 ms.

Hope this helps?


Kind regards - Will

 

[ColinTaylor]

What about the client side? Are these messages generated by a single client on the LAN? It would appear to be the LAN client(s) that are sending the zero window rather than the router itself.

 

[NogNeetMachinaal]

What about the client side? Are these messages generated by a single client on the LAN? It would appear to be the LAN client(s) that are sending the zero window rather than the router itself.

Perhaps - but the same Allegro Packet setup applies to the LAN-side of the router (this time as a VM with the same capabilities). Within that, there are no zero-win packets; not from the client and not from the server => the "problem" must be on the Asus router itself.

 

[ColinTaylor]

Perhaps - but the same Allegro Packet setup applies to the LAN-side of the router (this time as a VM with the same capabilities). Within that, there are no zero-win packets; not from the client and not from the server => the "problem" must be on the Asus router itself.

I think I would start by doing packet captures on both sides of the router and then compare the packets being sent by the client to those same packets being forwarded by the router.

Are you using QoS on the router?

Is hardware acceleration enabled on the router? If so try disabling it and seeing if that has an effect.

 

[NogNeetMachinaal]

I think I would start by doing packet captures on both sides of the router and then compare the packets being sent by the client to those same packets being forwarded by the router.

Are you using QoS on the router?

Is hardware acceleration enabled on the router? If so try disabling it and seeing if that has an effect.

I don't understand.
The clients are NAT-ed. Meaning packets before and after the router are different on OSI-L2 and OSI-L3 => once on the WAN-side of the Asus router, there is nothing that points to a certain client - let alone the client IP.

Please advise - how do I compare? What should I look out for?

NAT acceleration is enabled => disabled this => wait and see.
There is no QoS.


Kind regards - Will

 

[ColinTaylor]

I don't understand.
The clients are NAT-ed. Meaning packets before and after the router are different on OSI-L2 and OSI-L3 => once on the WAN-side of the Asus router, there is nothing that points to a certain client - let alone the client IP.

To associate a client's traffic from one side of the router to the other you would have to look at the conntrack table on the router. That might be quite awkward because of the transitory nature of what you're looking at. Given your example in post #3 and the apparent frequency it sounds like it would be much easier to deduce the client device by its traffic's destination, e.g. pps.whatsapp.net. Then monitor all traffic to and from that particular destination (on both sides of the router).

 

[NogNeetMachinaal]

To associate a client's traffic from one side of the router to the other you would have to look at the conntrack table on the router. That might be quite awkward because of the transitory nature of what you're looking at. Given your example in post #3 and the apparent frequency it sounds like it would be much easier to deduce the client device by its traffic's destination, e.g. pps.whatsapp.net. Then monitor all traffic to and from that particular destination (on both sides of the router).

Mmm... I still don't understand the reasoning...

Why is the router sending zero-win's to devices on the internet? Usually this only happens if these devices are sending to much packets in one go => not enough memory for storing these packets before they are processed? While memory usage is less then 30%?

In addition, on the LAN side, there is not a single trace of zero-win packets. Which client should I pick for this traffic deduction? How do I do that?

Btw - disabling NAT acceleration didn't help - problem remains.

 

[ColinTaylor]

Mmm... I still don't understand the reasoning...

Your assumption is that the router is somehow rewriting the TCP traffic being sent by the clients. I don't believe that's the case. To confirm what's actually happening you need capture the traffic as it leaves the client (or enters the router) and then compare it with the same traffic as it leaves the router. Obviously the source address (and maybe the port) will be different because of NAT. Seeing what else is different might provide a clue as to what is going on.

 

[NogNeetMachinaal]

Your assumption is that the router is somehow rewriting the TCP traffic being sent by the clients

Can you elaborate?
It is my understanding that typically, routers rewrite OSI-L2 and OSI-L3 of any traffic - TCP or UDP?
Because this is what routers are designed for - is it not...?

 

[ColinTaylor]

Can you elaborate?
It is my understanding that typically, routers rewrite OSI-L2 and OSI-L3 of any traffic - TCP or UDP?
Because this is what routers are designed for - is it not...?

That is true. I'm just suggesting one possible way of gaining a more complete picture of what is happening to the packets that you are concerned about.

 

[NogNeetMachinaal]

1. Tools -> Other Settings
- Wan: Use local caching DNS server as system resolver: No
- Disable Asusnat tunnel: Yes

2. WAN -> Internet Connection
- Connect to DNS Server automatically: No
DNS Server1: 8.8.8.8
DNS Server1: 8.8.4.4
- Enable DNS Rebind protection: No
- Enable DNSSEC support: No
- DNS Privacy Protocol: None

3. Administration -> Privacy -> Click in Withdraw

4. Firewall -> General -> Enable DoS protection: No

5. Reboot

Thank you Gato.

Can you elaborate a bit?
As far as item (3) goes: I can not click in it?

 

[ColDen]

Thank you Gato.

Can you elaborate a bit?
As far as item (3) goes: I can not click in it?

If you do not see the "Withdraw" button then it means that you haven't enabled any of the listed functions (AiProtection, Traffic Analyzer, Apps analyzer, Adaptive QoS/Game boost, Web history).