Solved WAN DHCP renewal + VPN drops connection

[Morris]

I've seen problems with 1 hour leases before. Somehow it breaks the normal 60% DHCP renewal for some systems. Renewing at expiration is bad as the request must go through quickly or it's a mess. Is there a command that can be run from CRON to do a DHCP renewal every 30 min?

 

[ColinTaylor]

Is there a command that can be run from CRON to do a DHCP renewal every 30 min?

There is, it's the one I got him to try earlier. Unfortunately it doesn't work - that's the problem.

 

[Morris]

The 60% renewal is directed to the IP address. At 100% it is a network broadcast. Could this be a firewall issue? Possibly bad QOS rules preventing the directed DHCP renewal from going out?

Those are the only things I can think of a user could do to cause this. Otherwise it is a router code issue.

 

[ColinTaylor]

The 60% renewal is directed to the IP address. At 100% it is a network broadcast. Could this be a firewall issue? Possibly bad QOS rules preventing the directed DHCP renewal from going out?

Those are the only things I can think of a user could do to cause this. Otherwise it is a router code issue.

I thought about a firewall problem but there is already an explicit rule to allow DHCP traffic. Even when I deleted that rule and blocked all DHCP unicast traffic on my router I still couldn't recreate the problem. So it's something very odd.

 

[Morris]

I saw exactly the same symptoms 22 years ago when the lease was changed to 1 hour. It was Windows hosts exhibiting the issue yet Microsoft "Borrowed" the code from a Unix implementation. The easiest fix for us was to increase the lease. Getting an ISP to do this will not be that easy unless a lot of people have the issue.

 

[drinkingbird]

I saw exactly the same symptoms 22 years ago when the lease was changed to 1 hour. It was Windows hosts exhibiting the issue yet Microsoft "Borrowed" the code from a Unix implementation. The easiest fix for us was to increase the lease. Getting an ISP to do this will not be that easy unless a lot of people have the issue.

No but I'm sure the ISP will be happy to lease you their "supported" router.

 

[Deleted member 83610]

I am very happy and relieved to report that the issue now is solved!

The details follow below...

killall udhcpc; /sbin/udhcpc -i eth0 -S -p /var/run/udhcpc0.pid -s /tmp/udhcpc_wan -O33 -O249

The command line above led me in the right direction and confirmed my growing suspicion:

Nov 15 10:34:13 udhcpc[7743]: sending renew
Nov 15 10:34:20 udhcpc[7743]: sending renew
Nov 15 10:34:23 udhcpc[7743]: sending renew
Nov 15 10:34:25 udhcpc[7743]: sending renew
Nov 15 10:34:25 udhcpc[7743]: lease lost, entering init state

Aren't the renew requests reaching my ISP's DHCP server?

I tried adding this into the configuration of the VPN client:

redirect-gateway def1 bypass-dhcp

That did not do the trick since the NordVPN server was already pushing a redirect-gateway setting itself.

Then I remembered I was having some issues a while back where my traffic all the sudden was going unprotected outside the tunnel when I switched between different VPN connections. At that time I decided to discard my VPN Director policy rules and instead just use this client setting:

Big mistake! This now also meant that DHCP renewals was sent through the VPN tunnel and never reached the DHCP server of my ISP.

So earlier today I reverted back to using VPN Director and that magically solved everything:

Now my DHCP lease is renewed every 30 minutes and my connection is yet again rock stable (despite the very short one hour lease of my ISP). Yay!

So all in all this was a user error by yours truly and luckily there's nothing wrong with the Merlin firmware. I can only blame myself. :\

Perhaps it would be advantageous if renewals appeared in in the system log by default with the Merlin firmware?

Thanks to especially @ColinTaylor for his time and effort and others that have been chiming in. I appreciate it.

 

[ColinTaylor]

That's good news @raritywrt !

I must admit I'm rather confused why the solution works, but I'm not going to complain.

The reason I'm confused is because you had tried turning off the VPN client completely but still had the problem. I guess there's some strange routing going on somewhere. Still, as long as it works...

 

[ColinTaylor]

I thought I'd post an update just for future reference in case it helps someone.

After @raritywrt's discovery regarding VPN routing I did some packet captures on my router and discovered some surprising behaviour from my own ISP.

My router's WAN subnet is 80.7.84.0/22. However, if I check the address of my ISP's DHCP server, both through packet capture and by looking in /tmp/wan0_bound.env, I can see it is 80.1.16.73 !!! This address is not in the WAN subnet and is (obviously) not publicly accessible.

So after enabling a VPN client with the option to redirect all internet traffic through the tunnel udhcpd will attempt to send its first renew request via the VPN. This will obviously fail. Subsequent renew attempts change from unicast to broadcast and do go out of the WAN interface.

The slightly surprising thing here is that the DHCP broadcast request via the WAN actually works, even though the ISP's DHCP server isn't in that broadcast domain. I can only assume that my ISP is using a DHCP relay.

This is still not quite the same as @raritywrt experienced. In his case even the broadcast requests failed, so there's some other differences there as well.

 

[drinkingbird]

I thought I'd post an update just for future reference in case it helps someone.

After @raritywrt's discovery regarding VPN routing I did some packet captures on my router and discovered some surprising behaviour from my own ISP.

My router's WAN subnet is 80.7.84.0/22. However, if I check the address of my ISP's DHCP server, both through packet capture and by looking in /tmp/wan0_bound.env, I can see it is 80.1.16.73 !!! This address is not in the WAN subnet and is (obviously) not publicly accessible.

So after enabling a VPN client with the option to redirect all internet traffic through the tunnel udhcpd will attempt to send its first renew request via the VPN. This will obviously fail. Subsequent renew attempts change from unicast to broadcast and do go out of the WAN interface.

The slightly surprising thing here is that the DHCP broadcast request via the WAN actually works, even though the ISP's DHCP server isn't in that broadcast domain. I can only assume that my ISP is using a DHCP relay.

This is still not quite the same as @raritywrt experienced. In his case even the broadcast requests failed, so there's some other differences there as well.

That's normal - DHCP on large networks uses "IP Helper" to redirect DHCP broadcast packets to a DHCP server somewhere else on the network. Large ISP and corporate networks won't have a DHCP interface on every LAN, they relay/redirect the packets to central DHCP servers.

So in VPN you'd have to add exceptions for the DHCP/BOOTP ports (if possible) since you really don't know how many DHCP servers they have, what their IPs are, if/when they'll change, etc. I guess I'd kinda expect VPN client to know/handle this by default, but guess it depends on the VPN.

Other than home or small office networks I've rarely seen the DHCP server in the same subnet as the IP it hands out.

 

[ColinTaylor]

That's normal - DHCP on large networks uses "IP Helper" to redirect DHCP broadcast packets to a DHCP server somewhere else on the network. Large ISP and corporate networks won't have a DHCP interface on every LAN, they relay/redirect the packets to central DHCP servers.

Yes, I'm more than familiar with DHCP helpers as I used to set them up on Cisco enterprise gear as part of my day job. I was just a little surprised to see it being used on my home connection as it's not something they did in the past (not that I've looked for a very long time).

So in VPN you'd have to add exceptions for the DHCP/BOOTP ports (if possible) since you really don't know how many DHCP servers they have, what their IPs are, if/when they'll change, etc. I guess I'd kinda expect VPN client to know/handle this by default, but guess it depends on the VPN.

The VPN client in asuswrt is based on routing tables so there's not really a way of making exceptions for specific ports.

 

[drinkingbird]

Yes, I'm more than familiar with DHCP helpers as I used to set them up on Cisco enterprise gear as part of my day job. I was just a little surprised to see it being used on my home connection as it's not something they did in the past (not that I've looked for a very long time).

Guess I've been stuck with monopoly ISPs for a long time, have always seen centralized DHCP server IPs.

The VPN client in asuswrt is based on routing tables so there's not really a way of making exceptions for specific ports.

It must have some sort of built in rule that lets DHCP broadcasts bypass the VPN then? Otherwise I'd think there'd be a lot of people having issues.

Even back in the days when company VPN wouldn't even let you print to your own network printer (no split tunneling) DHCP would get through.

 

[ColinTaylor]

It must have some sort of built in rule that lets DHCP broadcasts bypass the VPN then? Otherwise I'd think there'd be a lot of people having issues.

I speculated earlier in this thread that unless people are issued really short leases like the OP they're not likely to notice this problem. It's not that they can't get an IP address, they can. It's just that the DHCP renew fails which causes the router to "bounce" the WAN interface. This shuts down the VPN client, allows the router to get an IP and then restarts the VPN. The whole process only takes a couple of seconds. If this happens once a week would anybody notice. They'd also have to be using the "redirect all traffic" option in the client, if they're using any of the other routing options it's not an issue. It's also not an issue if the ISP DHCP server is in the WAN subnet. So it's a quite specific scenario.

 

[drinkingbird]

I speculated earlier in this thread that unless people are issued really short leases like the OP they're not likely to notice this problem. It's not that they can't get an IP address, they can. It's just that the DHCP renew fails which causes the router to "bounce" the WAN interface. This shuts down the VPN client, allows the router to get an IP and then restarts the VPN. The whole process only takes a couple of seconds. If this happens once a week would anybody notice. They'd also have to be using the "route all traffic" option in the client, if they're using any of the other routing options it's not an issue. It's also not an issue if the ISP DHCP server is in the WAN subnet. So it's a quite specific scenario.

Verizon (and possibly Frontier) FIOS uses 2 hour lease times in most areas so it isn't uncommon. Not sure what the other big ones use but as IPs get more and more scarce they keep lowering their lease times.

That few seconds where the VPN disconnects and reconnects could potentially leak some traffic, not a big deal for some but others wouldn't be thrilled. You'd think a VPN client should be smart enough to allow both DHCP broadcast and unicast outside the tunnel, like I said corporate ones have allowed it for many years.

But then again, corporate hardware/software, this is not.