Repeater DHCP Issues

[agilani]

Came across something interesting troubleshooting the DHCP errors in my system logs.

One particular device shows in the dhcp logs over and over. Resetting the device does not fix the problem. Rebooting the internet router running dhcp also does not fix the problem. After resetting the repeater however, the device immediately sends a DHCPREQUEST and DHCPACK after the DHCPOFFER.

Before repeater reboot

Jun 30 15:12:16 dnsmasq-dhcp[333]: DHCPDISCOVER(br0) 00:d0:2d:38:74:b0
Jun 30 15:12:16 dnsmasq-dhcp[333]: DHCPOFFER(br0) 192.168.1.245 00:d0:2d:38:74:b0
Jun 30 15:12:23 dnsmasq-dhcp[333]: DHCPDISCOVER(br0) 00:d0:2d:38:74:b0
Jun 30 15:12:23 dnsmasq-dhcp[333]: DHCPOFFER(br0) 192.168.1.245 00:d0:2d:38:74:b0
Jun 30 15:12:25 dnsmasq-dhcp[333]: DHCPDISCOVER(br0) 00:d0:2d:38:74:b0
Jun 30 15:12:25 dnsmasq-dhcp[333]: DHCPOFFER(br0) 192.168.1.245 00:d0:2d:38:74:b0
Jun 30 15:12:30 dnsmasq-dhcp[333]: DHCPDISCOVER(br0) 00:d0:2d:38:74:b0
Jun 30 15:12:30 dnsmasq-dhcp[333]: DHCPOFFER(br0) 192.168.1.245 00:d0:2d:38:74:b0
Jun 30 15:12:37 dnsmasq-dhcp[333]: DHCPDISCOVER(br0) 00:d0:2d:38:74:b0
Jun 30 15:12:37 dnsmasq-dhcp[333]: DHCPOFFER(br0) 192.168.1.245 00:d0:2d:38:74:b0
Jun 30 15:12:45 dnsmasq-dhcp[333]: DHCPDISCOVER(br0) 00:d0:2d:38:74:b0
Jun 30 15:12:45 dnsmasq-dhcp[333]: DHCPOFFER(br0) 192.168.1.245 00:d0:2d:38:74:b0
Jun 30 15:12:48 dnsmasq-dhcp[333]: DHCPDISCOVER(br0) 00:d0:2d:38:74:b0
Jun 30 15:12:48 dnsmasq-dhcp[333]: DHCPOFFER(br0) 192.168.1.245 00:d0:2d:38:74:b0
Jun 30 15:12:54 dnsmasq-dhcp[333]: DHCPDISCOVER(br0) 00:d0:2d:38:74:b0
Jun 30 15:12:54 dnsmasq-dhcp[333]: DHCPOFFER(br0) 192.168.1.245 00:d0:2d:38:74:b0

After repeater reboot

Jun 30 15:14:32 dnsmasq-dhcp[333]: DHCPDISCOVER(br0) 00:d0:2d:38:74:b0
Jun 30 15:14:32 dnsmasq-dhcp[333]: DHCPOFFER(br0) 192.168.1.245 00:d0:2d:38:74:b0
Jun 30 15:14:33 dnsmasq-dhcp[333]: DHCPREQUEST(br0) 192.168.1.245 00:d0:2d:38:74:b0
Jun 30 15:14:33 dnsmasq-dhcp[333]: DHCPACK(br0) 192.168.1.245 00:d0:2d:38:74:b0 honeywellthermostat

 

[ColinTaylor]

Without knowing any details about the repeater (make, model, etc.) it's hard to speculate, but I'm not entirely surprised that resetting the repeater would fix it. I've seen plenty of wireless devices that for no apparent reason start constantly doing DHCPDISCOVERs (iPhones, smart TVs, etc.). There seems to be no rhyme or reason to it. Rebooting the repeater would force the client to reinitialise its network connection and solve the problem, until the next time...

 

[JDB]

Sounds like this RT-AC5300 x2 Wi-Fi Bridging Issue
https://r.tapatalk.com/shareLink?sh...rums.com/index.php?posts/362399/&share_type=t


I definitely think there is something wrong with the Asus routers ability to handle broadcast traffic with repeaters (including other Asus routers running as bridges/extenders) in some cases.

That link is the thread I have been putting my analysis in, however, if you read over the forums, MANY people have problems which you could explain with my theory, just not enough people have the time/skill/patience to track the problem down.

Posting in your thread as you seem to have drawn the same conclusion, that ARP (broadcast msg) has a problem!

As Colin suggested, please post more info about your devices and setup.


Sent from my iPhone using Tapatalk

 

[agilani]

Without knowing any details about the repeater (make, model, etc.) it's hard to speculate, but I'm not entirely surprised that resetting the repeater would fix it. I've seen plenty of wireless devices that for no apparent reason start constantly doing DHCPDISCOVERs (iPhones, smart TVs, etc.). There seems to be no rhyme or reason to it. Rebooting the repeater would force the client to reinitialise its network connection and solve the problem, until the next time...

Power cycling the device will also will reinitialize its network connection, but the problem continued.

 

[kfp]

When you powered cycled the repeater, is the signal strong enough that the device connected directly to the main router instead?

 

[agilani]

When you powered cycled the repeater, is the signal strong enough that the device connected directly to the main router instead?

I have different ssid's for everything. No chance that the device would connect directly back to anything other than the repeater.

The repeater is connecting back to the internet router at 1300Mbps RSSI -54dBm
The device is connecting back to the repeater -37dBm using an SSID broadcast only by the repeater

I originally thought this was the cause or symptom of my wireless issues. But now that everything is working well, i still see these errors in my logs. There is still some sort of issues with broadcast and the repeaters. Perhaps as a result of the aimesh changes. I gave up on trying to work with asus tech support. Apparently they can only communicate with their engineers via e-mail.

So part of dhcp handshake is udp broadcast, part is unicast. I'm not even sure how to get a decent network capture of the handshake on wireless without being on the actual device which is a thermostat. Maybe do a capture on the repeater? WPA encryption is per packet / device right? It's not like a mirrored port where i can see all traffic on a vlan.

 

[kfp]

I have different ssid's for everything. No chance that the device would connect directly back to anything other than the repeater.

Gotcha. I original thought the thermostat might’ve connected back to the router and hit some code path that properly reset itself out of that weird state.

And yeah, could be due to AiMesh changes though with the latest GPL they seem to have cleaned it up somewhat. (avoid running AiMesh code when in repeater/media bridge modes)

Do you see those symptoms if you connect the thermostat directly to the router? (bypassing the repeater)

 

[agilani]

Gotcha. I original thought the thermostat might’ve connected back to the router and hit some code path that properly reset itself out of that weird state.

And yeah, could be due to AiMesh changes though with the latest GPL they seem to have cleaned it up somewhat. (avoid running AiMesh code when in repeater/media bridge modes)

Do you see those symptoms if you connect the thermostat directly to the router? (bypassing the repeater)

Good question, will have to try that but changing the wifi settings for it was pretty painful last time.

 

[agilani]

So this time the issue started happening with a windows 10 workstation that was connected directly to the internet router. I had replaced the nic card on this the last time this happened on this device to see if it was hardware related. The new nic card is doing the same thing. I was able to do a network capture, and the network caputre is showing the same thing the asus logs are.

Jul  4 10:29:14 dnsmasq-dhcp[333]: DHCPREQUEST(br0) 192.168.1.137 00:e0:4c:1f:a5:00
Jul  4 10:29:14 dnsmasq-dhcp[333]: DHCPACK(br0) 192.168.1.137 00:e0:4c:1f:a5:00 dellxps
Jul  4 10:29:31 dnsmasq-dhcp[333]: DHCPREQUEST(br0) 192.168.1.137 00:e0:4c:1f:a5:00
Jul  4 10:29:31 dnsmasq-dhcp[333]: DHCPACK(br0) 192.168.1.137 00:e0:4c:1f:a5:00 dellxps
Jul  4 10:29:53 dnsmasq-dhcp[333]: DHCPREQUEST(br0) 192.168.1.137 00:e0:4c:1f:a5:00
Jul  4 10:29:53 dnsmasq-dhcp[333]: DHCPACK(br0) 192.168.1.137 00:e0:4c:1f:a5:00 dellxps
Jul  4 10:30:02 dnsmasq-dhcp[333]: DHCPREQUEST(br0) 192.168.1.137 00:e0:4c:1f:a5:00
Jul  4 10:30:02 dnsmasq-dhcp[333]: DHCPACK(br0) 192.168.1.137 00:e0:4c:1f:a5:00 dellxps
Jul  4 10:34:45 dnsmasq-dhcp[333]: DHCPREQUEST(br0) 192.168.1.137 00:e0:4c:1f:a5:00
Jul  4 10:34:45 dnsmasq-dhcp[333]: DHCPACK(br0) 192.168.1.137 00:e0:4c:1f:a5:00 dellxps
Jul  4 10:34:50 dnsmasq-dhcp[333]: DHCPREQUEST(br0) 192.168.1.137 00:e0:4c:1f:a5:00
Jul  4 10:34:50 dnsmasq-dhcp[333]: DHCPACK(br0) 192.168.1.137 00:e0:4c:1f:a5:00 dellxps
Jul  4 10:34:54 dnsmasq-dhcp[333]: DHCPREQUEST(br0) 192.168.1.137 00:e0:4c:1f:a5:00
Jul  4 10:34:54 dnsmasq-dhcp[333]: DHCPACK(br0) 192.168.1.137 00:e0:4c:1f:a5:00 dellxps
Jul  4 10:35:03 dnsmasq-dhcp[333]: DHCPREQUEST(br0) 192.168.1.137 00:e0:4c:1f:a5:00
Jul  4 10:35:03 dnsmasq-dhcp[333]: DHCPACK(br0) 192.168.1.137 00:e0:4c:1f:a5:00 dellxps
Jul  4 10:36:12 dnsmasq-dhcp[333]: DHCPREQUEST(br0) 192.168.1.137 00:e0:4c:1f:a5:00
Jul  4 10:36:12 dnsmasq-dhcp[333]: DHCPACK(br0) 192.168.1.137 00:e0:4c:1f:a5:00 dellxps
Jul  4 10:40:45 dnsmasq-dhcp[333]: DHCPREQUEST(br0) 192.168.1.137 00:e0:4c:1f:a5:00
Jul  4 10:40:45 dnsmasq-dhcp[333]: DHCPACK(br0) 192.168.1.137 00:e0:4c:1f:a5:00 dellxps
Jul  4 10:43:43 dnsmasq-dhcp[333]: DHCPRELEASE(br0) 192.168.1.137 00:e0:4c:1f:a5:00
Jul  4 10:43:47 dnsmasq-dhcp[333]: DHCPDISCOVER(br0) 192.168.1.137 00:e0:4c:1f:a5:00
Jul  4 10:43:47 dnsmasq-dhcp[333]: DHCPOFFER(br0) 192.168.1.137 00:e0:4c:1f:a5:00
Jul  4 10:43:48 dnsmasq-dhcp[333]: DHCPREQUEST(br0) 192.168.1.137 00:e0:4c:1f:a5:00
Jul  4 10:43:48 dnsmasq-dhcp[333]: DHCPACK(br0) 192.168.1.137 00:e0:4c:1f:a5:00 dellxps
Jul  4 10:46:22 dnsmasq-dhcp[333]: DHCPREQUEST(br0) 192.168.1.137 00:e0:4c:1f:a5:00
Jul  4 10:46:22 dnsmasq-dhcp[333]: DHCPACK(br0) 192.168.1.137 00:e0:4c:1f:a5:00 dellxps

Lots of DHCP Request followed by DHCPACK, but all to broadcast. I haven't figured out how to export the wireshark data in flow log format to post here. Towards the end i did a dhcp release and renew to see what happens. That worked just fine, but the workstaiton resumed the dhcprequests.

One big difference that i see is that the response from the asus server for a workstation that is not having this problem is unicast, where the workstation that is having the problem is multicast to the broadcast address:

working workstation dhcp offer
17214 19.617698 192.168.1.1 192.168.1.102 DHCP 345 DHCP Offer - Transaction ID 0x7e908361
17216 19.620171 192.168.1.1 192.168.1.102 DHCP 357 DHCP ACK - Transaction ID 0x7e908361


problem workstation dhcp offer
4695 678.316570 192.168.1.1 255.255.255.255 DHCP 345 DHCP Offer - Transaction ID 0xac0f0a37
4697 678.624374 192.168.1.1 255.255.255.255 DHCP 357 DHCP ACK - Transaction ID 0xac0f0a37

now i'm going to look at what determines how the client asks for the dhcp address.

If the 'giaddr' field in a DHCP message from a client is non-zero, the server sends any return messages to the 'DHCP server' port on the BOOTP relay agent whose address appears in 'giaddr'. If the 'giaddr' field is zero and the 'ciaddr' field is nonzero, then the server unicasts DHCPOFFER and DHCPACK messages to the address in 'ciaddr'. If 'giaddr' is zero and 'ciaddr' is zero, and the broadcast bit is set, then the server broadcasts DHCPOFFER and DHCPACK messages to 0xffffffff. If the broadcast bit is not set and 'giaddr' is zero and 'ciaddr' is zero, then the server unicasts DHCPOFFER and DHCPACK messages to the client's hardware address and 'yiaddr' address. In all cases, when 'giaddr' is zero, the server broadcasts any DHCPNAK messages to 0xffffffff.

I'll have to do some more checking, may be that dhcp for wireless acts differently than wired.

 

[kfp]

Don’t have a firm idea on what might be causing this, but can you double check your static DHCP leases and make sure the IPs used there are out of the DHCP pool/scope?

 

[kfp]

I don’t think the unicast vs broadcast is the cause of the issue, but symptom of the DHCP problem you’re experience.

They will be broadcast when the client does not set a source IP (when it’s getting IP first time), if source IP is set (like in a renewal request) the server knows where to send it so it uses unicast instead.

 

[kfp]

Maybe you can look further into the client side as to WHY it’s still sending a request after a renewal.

Running any VMs/containers on the Dell XPS?

 

[agilani]

I don’t think the unicast vs broadcast is the cause of the issue, but symptom of the DHCP problem you’re experience.

They will be broadcast when the client does not set a source IP (when it’s getting IP first time), if source IP is set (like in a renewal request) the server knows where to send it so it uses unicast instead.

I did a release, then renew so both of the machines were running windows 10 1803 without an ip address making a new dhcp request. I do have a reservation for the workstation that did not have the problem and it was on the wired network. So not exactly apples to apples comparison. I'll need to do some more testing. I think on wireless the dhcp reply for a non-renewal would have to be broadcast because the wpa2 encryption is per device per packet right? I'll have to look at the capture and see what flags the dellxps sets once it does have an ip address.

No vm containers on the xps and the dhcp scopes and reservations are clearly within the subnet.

 

[kfp]

I think on wireless the dhcp reply for a non-renewal would have to be broadcast because the wpa2 encryption is per device per packet right? I'll have to look at the capture and see what flags the dellxps sets once it does have an ip address.

No vm containers on the xps and the dhcp scopes and reservations are clearly within the subnet.

The router would transmit the broadcast to each wireless device, the wpa2 encryption does not stop that.

I’m not thinking about whether it is in scope or not, just want to double check your static leases are not IN the dhcp scope.

 

[ColinTaylor]

I’m not thinking about whether it is in scope or not, just want to double check your static leases are not IN the dhcp scope.

Asus doesn't use the term "scope" so I assume you mean the DHCP "IP pool". In which case it doesn't matter whether the DHCP reservations are inside or outside of the pool, so long as they are within the subnet.

Note that there is an assumption by Asus that the the reservations will be taken from inside the pool (similar to the way Microsoft does it), but it's not enforced by dnsmasq.

 

[coxhaus]

The client does a DHCP broadcast to 0.0.0.0. The first DHCP server to respond with an offer is used.

PS
My guess is there is some bad DHCP code which is seen at the bridge of the networks. It could be on either side of the bridge.

 

[kfp]

Asus doesn't use the term "scope" so I assume you mean the DHCP "IP pool". In which case it doesn't matter whether the DHCP reservations are inside or outside of the pool, so long as there are within the subnet.

Note that there is an assumption by Asus that the the reservations will be taken from inside the pool (similar to the way Microsoft does it), but it's not enforced by dnsmasq.

Cool, didn’t know the implementation details, sounds like IP conflict is not at play here.

 

[kfp]

Maybe you can look further into the client side as to WHY it’s still sending a request after a renewal.

@agilani That’s what I would track down first before anything else.

 

[penguin22]

FYI, I have been battling this same issue for some time when using a router (e.g. RT-N66, RT-AC66, RT-AC68) in media bridge mode. I didn't see this issue when using a dedicated media bridge (EA-N66), however it limited the functionality of the network such as parental controls, MAC address filtering, QoS.

In my network I have

RT-AC86U (router @ 2.4GHz - 20MHz - CH1) <--> RT-AC68P (MB) <--> RT-AC3200 (AP 2.4GHz - 20/40MHz - CH6 Above & 5GHz)

Having used each version of @RMerlin firmware and John's firmware, I can say that there were stability issues with the MB prior to the 384.4 releases where it rebooted frequently; this stopped occurring and I had someone better stability in 384.5, but there was a wireless issue. The wireless issue is fixed in 384.6 Alpha2, however this shows the DHCP issue much better as a result.

I have ~30 clients connecting, 3 to the router, which includes the MB, and 27 to the AP. Most clients pull an IP from DHCP without issue straight away from a clean start of the network, however within 1-day, devices may lose their IP configuration and have trouble renewing from DHCP. This is true of manually defined addresses under LAN or automatically assigned addresses. If you leave it long enough, the client does appear to eventually be able to obtain an IP, but it varies per client and what the client refresh period is. For example, a battery powered Nest Protect will not keep retrying and will instead report an error until manually triggered via the physical button.

I have found that restarting the AP has no effect, and also restarting the router has no effect. I have stripped down router services to the fundamentals to ensure it is not related to services running and also see no difference. When the issue does block a Windows client and is occurring, a restart of the MB restored the ability to obtain an IP immediately once it comes back online and re-established communication to the router.

I'm assuming that buying a more recent MB than the EA-N66 could fix the DHCP issue, but am reluctant if the other functionality does not work. It would be better for Asus to find and fix this issue since MB is a stock function of these routers.

 

[agilani]

FYI, I have been battling this same issue for some time when using a router (e.g. RT-N66, RT-AC66, RT-AC68) in media bridge mode. I didn't see this issue when using a dedicated media bridge (EA-N66), however it limited the functionality of the network such as parental controls, MAC address filtering, QoS.

In my network I have

RT-AC86U (router @ 2.4GHz - 20MHz - CH1) <--> RT-AC68P (MB) <--> RT-AC3200 (AP 2.4GHz - 20/40MHz - CH6 Above & 5GHz)

Having used each version of @RMerlin firmware and John's firmware, I can say that there were stability issues with the MB prior to the 384.4 releases where it rebooted frequently; this stopped occurring and I had someone better stability in 384.5, but there was a wireless issue. The wireless issue is fixed in 384.6 Alpha2, however this shows the DHCP issue much better as a result.

I have ~30 clients connecting, 3 to the router, which includes the MB, and 27 to the AP. Most clients pull an IP from DHCP without issue straight away from a clean start of the network, however within 1-day, devices may lose their IP configuration and have trouble renewing from DHCP. This is true of manually defined addresses under LAN or automatically assigned addresses. If you leave it long enough, the client does appear to eventually be able to obtain an IP, but it varies per client and what the client refresh period is. For example, a battery powered Nest Protect will not keep retrying and will instead report an error until manually triggered via the physical button.

I have found that restarting the AP has no effect, and also restarting the router has no effect. I have stripped down router services to the fundamentals to ensure it is not related to services running and also see no difference. When the issue does block a Windows client and is occurring, a restart of the MB restored the ability to obtain an IP immediately once it comes back online and re-established communication to the router.

I'm assuming that buying a more recent MB than the EA-N66 could fix the DHCP issue, but am reluctant if the other functionality does not work. It would be better for Asus to find and fix this issue since MB is a stock function of these routers.

Are they not pulling an ip address or do they keep showing up in the logs? At least on the machine i tested yesterday, it clearly got an ip address form the dhcp scope but it kept requesting the ip again and again. On the one that was connected to the repeater, rebooting the repeater was the only thing that fixed the symptoms. You are right though, before with the wireless instability, i didn't even get this far. I attributed the problems to wireless issues.

I'll spend some more time doing network captures and comparing the packets and see if i find anything. So far the only difference i've found was that the asus reply for the devices that had the issues were broadcast instead of unicast.

In testing a few other wireless devices, the dhcp offer and dhcp ack are both unicast. Now I have to find the part of the packet where the client specifies weather the server should send it as unicast or multicast.