WAN was exceptionally disconnected

[Kanji-San]

Here is the situation: this week my fiber Internet service was upgraded from 100Mbs up and down to 1Gbit up and down. The provider, Gigamonster, replaced the media converter with an ONT. Speeds are great but occasionally the router (RT-AX88U running 384.19) reports WAN disconnects: WAN was exceptionally disconnected. Sometimes: ISP's DHCP did not function properly.

I also run connmon and whenever this happens the Quality goes down to 0%. This happens very infrequently, one time it was three times within 12 minutes. Yesterday, there was no issue at all. Sometimes the connection comes backs automatically, twice I had to reboot the router to regain the WAN connection.

The only thing I changed on the RT-AX88U was that I withdraw the privacy consent which turned off QoS (and the fantastic FlexQoS), Traffic Analyzer, etc in order to enable both Runner and Flow Cache to take advantage of the full gigabit speed.

I searched older threads for these WAN disconnect problems, but did not find anything that clarifies my situation.
Before the upgrade, I had never seen WAN was exceptionally disconnected. Therefore I am leaning towards the conclusion, it's not the router. Any ideas how I could debug this problem further? Any other ideas or solutions?

 

[ColinTaylor]

Contact your ISP as that appears to be where the fault lies.

 

[Kanji-San]

Contact your ISP as that appears to be where the fault lies.

Thank you, I will contact the ISP tomorrow.

The WAN connection went down again I tried to turn off the WAN interface (in the GUI) and on again which made no difference. I also tried different DHCP frequency modes which also did not help. The only thing that seems to work is rebooting the router. I wonder what is different when I reboot the router compared to toggling the WAN interface?

 

[ColinTaylor]

The only difference I can think of is that the reboot will "electrically" (for want of a better word) reset the WAN connection rather than just changing its configuration. That still doesn't explain the apparent problem with the ONT. You could try changing the Ethernet cable between the ONT and the router. Some people have experienced problems where two Ethernet connected devices simply refused to play nice with each other and putting a dumb switch between them fixed it. I don't know anything about ONTs but if it has a log file that would be the first place to look for clues as to the cause.

 

[Kanji-San]

The only difference I can think of is that the reboot will "electrically" (for want of a better word) reset the WAN connection rather than just changing its configuration. That still doesn't explain the apparent problem with the ONT. You could try changing the Ethernet cable between the ONT and the router. Some people have experienced problems where two Ethernet connected devices simply refused to play nice with each other and putting a dumb switch between them fixed it. I don't know anything about ONTs but if it has a log file that would be the first place to look for clues as to the cause.

Unfortunately, I have no access to the ONT, I think it must be in bridge mode.
I am more and more convinced that there is a problem in the ONT or maybe an incompatibility between my Asus router the and the ONT.

 

[Rassal]

I did experience the "Dreaded" ISP's DHCP did not function properly. To resolve this, all i did was change the DHCP query frequency to "Continous mode" instead of "Aggressive Mode". I do not have an ONT, but a Docsis 3 cable modem with 500/500 speed, so like you had to get around some config to get full speed, especially with upload speed. But since i am using "Continous mode" i never got disconnected on the WAN side anymore, beside when the ISP has problem which takes down the cable mode with it, guess it's the same with your ONT i guess... if you can match the ONT having issues and loosing your fiber signal, then there is no doubt, the WAN on your router will go down with it...

 

[Kanji-San]

I did experience the "Dreaded" ISP's DHCP did not function properly. To resolve this, all i did was change the DHCP query frequency to "Continous mode" instead of "Aggressive Mode". I do not have an ONT, but a Docsis 3 cable modem with 500/500 speed, so like you had to get around some config to get full speed, especially with upload speed. But since i am using "Continous mode" i never got disconnected on the WAN side anymore, beside when the ISP has problem which takes down the cable mode with it, guess it's the same with your ONT i guess... if you can match the ONT having issues and loosing your fiber signal, then there is no doubt, the WAN on your router will go down with it...

Thanks for the tip. I'll try this.

 

[drinkingbird]

Here is the situation: this week my fiber Internet service was upgraded from 100Mbs up and down to 1Gbit up and down. The provider, Gigamonster, replaced the media converter with an ONT. Speeds are great but occasionally the router (RT-AX88U running 384.19) reports WAN disconnects: WAN was exceptionally disconnected. Sometimes: ISP's DHCP did not function properly.

I also run connmon and whenever this happens the Quality goes down to 0%. This happens very infrequently, one time it was three times within 12 minutes. Yesterday, there was no issue at all. Sometimes the connection comes backs automatically, twice I had to reboot the router to regain the WAN connection.

The only thing I changed on the RT-AX88U was that I withdraw the privacy consent which turned off QoS (and the fantastic FlexQoS), Traffic Analyzer, etc in order to enable both Runner and Flow Cache to take advantage of the full gigabit speed.

I searched older threads for these WAN disconnect problems, but did not find anything that clarifies my situation.
Before the upgrade, I had never seen WAN was exceptionally disconnected. Therefore I am leaning towards the conclusion, it's not the router. Any ideas how I could debug this problem further? Any other ideas or solutions?

I had similar issues with Verizon FIOS. Nothing fixed it for me, had to revert back to 384 code base. Even the stock Asus 386 had the issue. There are others with the same problem, it seems to be people with short DHCP lease times (mine is 2 hours) on the WAN. I did not have a chance to run a sniffer on the WAN but my suspicion is that the WAN DHCP is not sending the DHCP ACK back to the ISP thus your IP is not getting reserved on their server and gets handed out to another customer. Again, just a suspicion. After going back to 384 my uptime is 3 weeks with no disconnects. There seems to be a bug in WAN DHCP on the 386 code base.

 

[Kanji-San]

I had similar issues with Verizon FIOS. Nothing fixed it for me, had to revert back to 384 code base. Even the stock Asus 386 had the issue. There are others with the same problem, it seems to be people with short DHCP lease times (mine is 2 hours) on the WAN. I did not have a chance to run a sniffer on the WAN but my suspicion is that the WAN DHCP is not sending the DHCP ACK back to the ISP thus your IP is not getting reserved on their server and gets handed out to another customer. Again, just a suspicion. After going back to 384 my uptime is 3 weeks with no disconnects. There seems to be a bug in WAN DHCP on the 386 code base.

Interesting. However I am on 384.19! Is yours still working? Maybe it is a certain setting.

 

[netware5]

I had the same problem (once) three days after upgrade to 386.2. Re-connection of WAN was possible only after reboot. It is not DHCP related as my WAN is with fixed IP address. I have Ethernet connection to the ISP's ONT. Such event NEVER happened until now during last 13 years I am using ASUS routers with Merlin with the same ISP.

 

[Kanji-San]

More and more I believe there is a hardware incompatibility issue with the DASAN ONT and possibly its Ethernet controller chip. Since only a reboot of the router resolves this WAN drop and @ColinTaylor remarked that a router reboot should also cause an electrical reset.

This post points out that there are ONTs out there that have hardware incompatibilities with Ethernet chips. All my problems started after the upgrade and the installation of this new ONT.
I will mention this to my ISP and see if there's possibly a different ONT brand / model that can be installed.

It looks like a small number of my neighbors who were also upgraded have the same WAN drop issues.

 

[netware5]

More and more I believe there is a hardware incompatibility issue with the DASAN ONT and possibly its Ethernet controller chip. Since only a reboot of the router resolves this WAN drop and @ColinTaylor remarked that a router reboot should also cause an electrical reset.

This post points out that there are ONTs out there that have hardware incompatibilities with Ethernet chips. All my problems started after the upgrade and the installation of this new ONT.
I will mention this to my ISP and see if there's possibly a different ONT brand / model that can be installed.
A
It looks like a small number of my neighbors who were also upgraded have the same WAN drop issues.

I think most probably the issue is FW related. My ISP didn't replaced its ONT recently. So the HW is the same. Also the probability that two different ISPs on two different continents are using the same "incompatible" ONTs and all this happens in the same time we upgraded our routers to 386.2 is very, very low ...

 

[Kanji-San]

I think most probably the issue is FW related. My ISP didn't replaced its ONT recently. So the HW is the same. Also the probability that two different ISPs on two different continents are using the same "incompatible" ONTs and all this happens in the same time we upgraded our routers to 386.2 is very, very low ...

My ISP told me there are some network equipment issues which are still being tracked down. Unclear if the ONT is affected or not. Nothing one can do at the router side.

I think behind the WAN was exceptionally disconnected or ISP's DHCP did not function properly messages many different root causes are hidden. It just means something is wrong with the Internet connection.

 

[ColinTaylor]

I think most probably the issue is FW related. My ISP didn't replaced its ONT recently. So the HW is the same. Also the probability that two different ISPs on two different continents are using the same "incompatible" ONTs and all this happens in the same time we upgraded our routers to 386.2 is very, very low ...

@netware5 Your situation is the opposite of his. His hardware did change, and he hasn't updated his firmware (he's on 384.19).

@Kanji-San You are correct about the error messages meaning "something" is wrong. Given the reply from your ISP and that you said your neighbours were also having problems it's sounding more like an area fault than something with your equipment.

 

[drinkingbird]

My ISP told me there are some network equipment issues which are still being tracked down. Unclear if the ONT is affected or not. Nothing one can do at the router side.

I think behind the WAN was exceptionally disconnected or ISP's DHCP did not function properly messages many different root causes are hidden. It just means something is wrong with the Internet connection.

Yes the DHCP message is just generic and can point to many issues. I didn't realize you hadn't upgraded to 386 so likely a different issue than I and several others had. I have been back on 384.19 (AC68U) for a few weeks and no disconnects still.

Have you made sure the cable between your router and the ONT is CAT5e or better? CAT5 cable will appear to work but there can be a lot of interference/packet loss (depending on distance, proximity to electrical wires, etc)? 100 meg is fine over CAT5 but gig is far more sensitive and requires 5e at minimum.

May be an equipment issue as you say, but something that is worth checking regardless. Even if it is cat 5e or higher you can check the ends to make sure they are clean and not loose etc.

 

[meistadieb]

General question: Are you using powerlan/dlan in your networks? If so, try to unplug and run your network without them for some time. Especially with VDSL they are really disruptive (in the german telekom forum they are really often mentioned)

 

[Kanji-San]

General question: Are you using powerlan/dlan in your networks? If so, try to unplug and run your network without them for some time. Especially with VDSL they are really disruptive (in the german telekom forum they are really often mentioned)

No, I am not.

 

[Kanji-San]

My ISP has not been clear what is going on. I kept digging and changed the DHCP query frequency setting from Aggressive to Continuous. Interestingly I had not had any WAN disconnects for 5 days. Then it happened again yesterday. I had to reboot. Now I am using tcpdump on port 67 and 68 to see what is going on. I suspect there is a problem renewing the IP address.

For a 12-hour lease, udhcpc tries to renew (Request) the lease after 6 hours but there is no answer (Ack) from the upstream DHCP server. udhcpc tries again after 3 hours with the same result. Only after another 1.5 hours is the renewal (Request) answered (Ack) and the lease renewed.

Looking at the Continuous setting:

/sbin/udhcpc -i eth0 -p /var/run/udhcpc0.pid -s /tmp/udhcpc -t1 -T5 -A0 -O33 -O249

-t1 means: send 1 discover packet
-T5 means: wait 5 seconds between packets
-A0 means: wait for 0 seconds before trying again

When do these settings kick in and how? What is the meaning behind Continuous?

 

[ColinTaylor]

For a 12-hour lease, udhcpc tries to renew (Request) the lease after 6 hours but there is no answer (Ack) from the upstream DHCP server. udhcpc tries again after 3 hours with the same result. Only after another 1.5 hours is the renewal (Request) answered (Ack) and the lease renewed.

So that's normal client behaviour but not a typical server behaviour.

Looking at the Continuous setting:

/sbin/udhcpc -i eth0 -p /var/run/udhcpc0.pid -s /tmp/udhcpc -t1 -T5 -A0 -O33 -O249

-t1 means: send 1 discover packet
-T5 means: wait 5 seconds between packets
-A0 means: wait for 0 seconds before trying again

When do these settings kick in and how? What is the meaning behind Continuous?

Normally these setting would only be relevant when udhcpc starts up and is trying to obtain the initial lease. I have no idea whether they would come into effect if the current lease expires without obtaining a renewal. Presumably your tcpdump will answer that question should the situation arise.

 

[Kanji-San]

My ISP has not been clear what is going on. I kept digging and changed the DHCP query frequency setting from Aggressive to Continuous. Interestingly I had not had any WAN disconnects for 5 days. Then it happened again yesterday. I had to reboot. Now I am using tcpdump on port 67 and 68 to see what is going on. I suspect there is a problem renewing the IP address.

For a 12-hour lease, udhcpc tries to renew (Request) the lease after 6 hours but there is no answer (Ack) from the upstream DHCP server. udhcpc tries again after 3 hours with the same result. Only after another 1.5 hours is the renewal (Request) answered (Ack) and the lease renewed.

Looking at the Continuous setting:

/sbin/udhcpc -i eth0 -p /var/run/udhcpc0.pid -s /tmp/udhcpc -t1 -T5 -A0 -O33 -O249

-t1 means: send 1 discover packet
-T5 means: wait 5 seconds between packets
-A0 means: wait for 0 seconds before trying again

When do these settings kick in and how? What is the meaning behind Continuous?

To answer my own question, I played around with udhcpc (not on the router) and learned what these paramters (-t -T -A) really do:
DHCP Query Frequency:

Aggressive Mode (default): send 3 discover packets with 3 seconds pause, wait for 20 seconds if this fails then repeat forever

> sudo ./busybox udhcpc -v -i ens33 -O33 -O249
udhcpc: started, v1.32.1
udhcpc: executing /usr/share/udhcpc/default.script deconfig
udhcpc: entering listen mode: raw
udhcpc: created raw socket

udhcpc: sending discover
udhcpc: waiting 3 seconds
udhcpc: sending discover
udhcpc: waiting 3 seconds
udhcpc: sending discover
udhcpc: waiting 3 seconds
udhcpc: executing /usr/share/udhcpc/default.script leasefail
udhcpc: waiting 20 seconds

udhcpc: sending discover
udhcpc: waiting 3 seconds
udhcpc: sending discover
udhcpc: waiting 3 seconds
udhcpc: sending discover
udhcpc: waiting 3 seconds
udhcpc: executing /usr/share/udhcpc/default.script leasefail
udhcpc: waiting 20 seconds

udhcpc: sending discover
udhcpc: waiting 3 seconds
udhcpc: sending discover
udhcpc: waiting 3 seconds
udhcpc: sending discover
udhcpc: waiting 3 seconds
udhcpc: executing /usr/share/udhcpc/default.script leasefail
udhcpc: waiting 20 seconds

...

Normal Mode: send 2 discover packets with 5 seconds pause, wait 160 seconds if this fails then repeat forever

> sudo ./busybox udhcpc -v -i ens33 -O33 -O249 -t2 -T5 -A160
udhcpc: started, v1.32.1
udhcpc: executing /usr/share/udhcpc/default.script deconfig
udhcpc: entering listen mode: raw
udhcpc: created raw socket

udhcpc: sending discover
udhcpc: waiting 5 seconds
udhcpc: sending discover
udhcpc: waiting 5 seconds
udhcpc: executing /usr/share/udhcpc/default.script leasefail
udhcpc: waiting 160 seconds

udhcpc: sending discover
udhcpc: waiting 5 seconds
udhcpc: sending discover
udhcpc: waiting 5 seconds
udhcpc: executing /usr/share/udhcpc/default.script leasefail
udhcpc: waiting 160 seconds

udhcpc: sending discover
udhcpc: waiting 5 seconds
udhcpc: sending discover
udhcpc: waiting 5 seconds
udhcpc: executing /usr/share/udhcpc/default.script leasefail
udhcpc: waiting 160 seconds

...

Continuous Mode: send discover packets every 5 seconds, if this fails then repeat forever

> sudo ./busybox udhcpc -v -i ens33 -O33 -O249 -t1 -T5 -A0
udhcpc: started, v1.32.1
udhcpc: executing /usr/share/udhcpc/default.script deconfig
udhcpc: entering listen mode: raw
udhcpc: created raw socket

udhcpc: sending discover
udhcpc: waiting 5 seconds
udhcpc: executing /usr/share/udhcpc/default.script leasefail

udhcpc: sending discover
udhcpc: waiting 5 seconds
udhcpc: executing /usr/share/udhcpc/default.script leasefail

udhcpc: sending discover
udhcpc: waiting 5 seconds
udhcpc: executing /usr/share/udhcpc/default.script leasefail

udhcpc: sending discover
udhcpc: waiting 5 seconds
udhcpc: executing /usr/share/udhcpc/default.script leasefail

...