Solved MTU Issues

[RachaelSeven]

So I ran into an issue where certain websites suddenly wouldn't load and I could not get an upload speed test to work (download tests did work). Things had been working fine previously and I do not know what precipitated the problem, perhaps a router firmware upgrade or maybe a change at my ISP. My setup is RT-AC68U running the latest Merlin, with a variety of LAN and WiFi clients. I tried all manner of resetting winsock, resetting the router, etc, etc, and nothing worked. I finally tracked the issue down to an MTU problem. Apparently my ISP needs an MTU of 1492 rather than 1500, even though it is a static IP and not a PPPoE, or at least that's what my ping testing indicated. So I set the MTU at 1492 in the WAN settings on the AC68U and voila, my upload speed tests work and the websites are reachable. But, to my dismay, a bunch of other websites stopped working. The only way I could get this new batch of websites to start working again was to force the devices on my local network to also use an MTU of 1492, by setting it using netsh from a command prompt. I didn't think I was supposed to have to do that - I thought MTU could be different on the LAN and WAN side, but apparently not. I also thought MTU was supposed to negotiate itself down through MTU discovery, but also apparently not. The only problem left was that those devices where I couldn't force the MTU, like my iPhone, still couldn't reach the second set of websites, which included among others, duckduckgo.com (my preferred search engine), squareup.com (my payment process for my business), and a whole lot of other major sites. In desperation, I tried enabling jumbo frames on the Switch Control page of the LAN settings and eureka, my phone (and all my other devices) could go everywhere again.

That's what I went through, and that's how I made it work in my near total ignorance of what I was doing. I feel like a monkey randomly pushing buttons until something just happened to work. I'm particularly concerned about having enabled jumbo frames, since everything I read indicated that was not a great idea. Can anyone explain what actually went wrong and what I should have done to fix it? Any advice or enlightenment would be most appreciated. Thank you.

 

[chongnt]

So I ran into an issue where certain websites suddenly wouldn't load and I could not get an upload speed test to work (download tests did work). Things had been working fine previously and I do not know what precipitated the problem, perhaps a router firmware upgrade or maybe a change at my ISP. My setup is RT-AC68U running the latest Merlin, with a variety of LAN and WiFi clients. I tried all manner of resetting winsock, resetting the router, etc, etc, and nothing worked. I finally tracked the issue down to an MTU problem. Apparently my ISP needs an MTU of 1492 rather than 1500, even though it is a static IP and not a PPPoE, or at least that's what my ping testing indicated. So I set the MTU at 1492 in the WAN settings on the AC68U and voila, my upload speed tests work and the websites are reachable. But, to my dismay, a bunch of other websites stopped working. The only way I could get this new batch of websites to start working again was to force the devices on my local network to also use an MTU of 1492, by setting it using netsh from a command prompt. I didn't think I was supposed to have to do that - I thought MTU could be different on the LAN and WAN side, but apparently not. I also thought MTU was supposed to negotiate itself down through MTU discovery, but also apparently not. The only problem left was that those devices where I couldn't force the MTU, like my iPhone, still couldn't reach the second set of websites, which included among others, duckduckgo.com (my preferred search engine), squareup.com (my payment process for my business), and a whole lot of other major sites. In desperation, I tried enabling jumbo frames on the Switch Control page of the LAN settings and eureka, my phone (and all my other devices) could go everywhere again.

That's what I went through, and that's how I made it work in my near total ignorance of what I was doing. I feel like a monkey randomly pushing buttons until something just happened to work. I'm particularly concerned about having enabled jumbo frames, since everything I read indicated that was not a great idea. Can anyone explain what actually went wrong and what I should have done to fix it? Any advice or enlightenment would be most appreciated. Thank you.

WAN MTU has to match with your service provider. They should have provided this information during service provisioning and you need to follow it. MTU mismatch will cause a lot of issues. Anyway, WAN port and LAN port MTU are totally different. A quick way to see is ssh into your router and run the command "ifconfig". By default br0 and ethx ports all should have a default MTU of 1500. Packets larger that 1500 bytes will get fragmented and there should be no issue. You can do a ping test from either your router or pc to verify the MTU. Just remember to use ping payload size of 1472 (plus 20 bytes ip header and 8 bytes icmp ping header is equal to 1500 bytes). Just ping your LAN devices with and without do-not-fragment option. With do-not-fragment, the maximum payload is 1472, anything more you will get packet too long message. Now ping without do-not-fragment and you can ping with a payload higher than 1472 because the packet will get fragmented.

 

[Adooni]

In most situation MTU should stay at default value 1500 but few years ago I had location that was not a standard one. To investigate and set it correctly I used this guide.
https://www.tp-link.com/us/support/faq/190/

 

[itpp20]

It depends, for example see this https://www.snbforums.com/threads/optimal-lte-4g-mtu-wan-packet-overhead.74046/
Allas this is a (3g)4g/5g issue and solved between the asus router and the 4g router with a low mtu and dmz to asus. Even an iperf test showed a more stable speed (from the jojo before) and more important no lower speed as a single block of 1300 is faster then blocks split in two nearly constantly arguing about its size.

 

[RachaelSeven]

WAN MTU has to match with your service provider. They should have provided this information during service provisioning and you need to follow it. MTU mismatch will cause a lot of issues. Anyway, WAN port and LAN port MTU are totally different. A quick way to see is ssh into your router and run the command "ifconfig". By default br0 and ethx ports all should have a default MTU of 1500. Packets larger that 1500 bytes will get fragmented and there should be no issue. You can do a ping test from either your router or pc to verify the MTU. Just remember to use ping payload size of 1472 (plus 20 bytes ip header and 8 bytes icmp ping header is equal to 1500 bytes). Just ping your LAN devices with and without do-not-fragment option. With do-not-fragment, the maximum payload is 1472, anything more you will get packet too long message. Now ping without do-not-fragment and you can ping with a payload higher than 1472 because the packet will get fragmented.

That's just it, my provider provided no such information and it worked for years without any adjustment to MTU. A week ago, something changed and it stopped working correctly. So I did ping tests, as you suggest, and determined the 1492 number experimentally, only to later find that was the value often used by PPPoE connections. At any rate, that was the number that worked, so I used SSH to set the MTU for eth0 to 1492 with the command:

ip link set dev eth0 mtu 1492

I confirmed with ifconfig that it had been set correctly, and like I said, a bunch of problem websites started working again, but new group started having problems. Looking again at ifconfig, it appears that br0, br1, and br2 were also set to 1492. I'm not sure what those are, perhaps the bridges for LAN, 2.4GHz, and 5GHz? In any case, my Win10 machine connected over LAN wouldn't talk to some sites until I set it's MTU to 1492 as well. I didn't think I should have to do that, but it wouldn't work until I did. Where did I go wrong?

 

[chongnt]

That's just it, my provider provided no such information and it worked for years without any adjustment to MTU. A week ago, something changed and it stopped working correctly. So I did ping tests, as you suggest, and determined the 1492 number experimentally, only to later find that was the value often used by PPPoE connections. At any rate, that was the number that worked, so I used SSH to set the MTU for eth0 to 1492 with the command:

ip link set dev eth0 mtu 1492

I confirmed with ifconfig that it had been set correctly, and like I said, a bunch of problem websites started working again, but new group started having problems. Looking again at ifconfig, it appears that br0, br1, and br2 were also set to 1492. I'm not sure what those are, perhaps the bridges for LAN, 2.4GHz, and 5GHz? In any case, my Win10 machine connected over LAN wouldn't talk to some sites until I set it's MTU to 1492 as well. I didn't think I should have to do that, but it wouldn't work until I did. Where did I go wrong?

I have no idea about br1 and br2. I only have br0 in my AC86U. I think you should revert MTU for all ports (other than WAN) to default value of 1500. It should just works.
I think the problem is when you lower your router LAN eth MTU from default 1500 to 1492. Windows by default has a MTU of 1500 as well. From Win 10 machine point of view, packet fragmentation will happen when the packet is larger than 1500 Bytes. But now the router LAN port cannot support 1500 bytes anymore, it can only support a Maximum of 1492 bytes. Meaning any packet larger than 1492 bytes will be dropped by router LAN port. After you change windows MTU to match your router LAN MTU of 1492 it will works again because now the MTU matches for both ends.

 

[John Fitzgerald]

So I ran into an issue where certain websites suddenly wouldn't load and I could not get an upload speed test to work (download tests did work). Things had been working fine previously and I do not know what precipitated the problem, perhaps a router firmware upgrade or maybe a change at my ISP. My setup is RT-AC68U running the latest Merlin, with a variety of LAN and WiFi clients. I tried all manner of resetting winsock, resetting the router, etc, etc, and nothing worked. I finally tracked the issue down to an MTU problem. Apparently my ISP needs an MTU of 1492 rather than 1500, even though it is a static IP and not a PPPoE, or at least that's what my ping testing indicated. So I set the MTU at 1492 in the WAN settings on the AC68U and voila, my upload speed tests work and the websites are reachable. But, to my dismay, a bunch of other websites stopped working. The only way I could get this new batch of websites to start working again was to force the devices on my local network to also use an MTU of 1492, by setting it using netsh from a command prompt. I didn't think I was supposed to have to do that - I thought MTU could be different on the LAN and WAN side, but apparently not. I also thought MTU was supposed to negotiate itself down through MTU discovery, but also apparently not. The only problem left was that those devices where I couldn't force the MTU, like my iPhone, still couldn't reach the second set of websites, which included among others, duckduckgo.com (my preferred search engine), squareup.com (my payment process for my business), and a whole lot of other major sites. In desperation, I tried enabling jumbo frames on the Switch Control page of the LAN settings and eureka, my phone (and all my other devices) could go everywhere again.

That's what I went through, and that's how I made it work in my near total ignorance of what I was doing. I feel like a monkey randomly pushing buttons until something just happened to work. I'm particularly concerned about having enabled jumbo frames, since everything I read indicated that was not a great idea. Can anyone explain what actually went wrong and what I should have done to fix it? Any advice or enlightenment would be most appreciated. Thank you.

Have a try at this: https://www.intel.com/content/www/us/en/support/articles/000058982/ethernet-products.html

MTU Size: What Is It and What Should It Be Set To? - Home Network Geek

MTU size is the maximum packet size that can be transmitted over your network. Changing it is an easy way of optimizing your home network's performance.

homenetworkgeek.com

 

[heysoundude]

Hmm...static IP and jumbo frames https://en.wikipedia.org/wiki/Jumbo_frame

If your CPE/bridge/gateway let jumbo frames through, did your ISP possibly roll out Native IPv6? can you ping6 your provider?
If that returns a positive result, you need to enable/set up IPv6 on the router...and enjoy

IPv6 is not scary -> go get certified: https://ipv6.he.net/certification/ and since you're running a business, don't hesitate to claim your /48

 

[RachaelSeven]

Hmm...static IP and jumbo frames https://en.wikipedia.org/wiki/Jumbo_frame

If your CPE/bridge/gateway let jumbo frames through, did your ISP possibly roll out Native IPv6? can you ping6 your provider?
If that returns a positive result, you need to enable/set up IPv6 on the router...and enjoy

IPv6 is not scary -> go get certified: https://ipv6.he.net/certification/ and since you're running a business, don't hesitate to claim your /48

You may be onto something. When I gave my ISP (a small municipal broadband company) my findings about MTU, they admitted the problem was on their end and reversed whatever they had changed. So perhaps they were testing ipv6 in preparation for a future rollout or something. I'll ask them about it. Thanks for the lead!

 

[RachaelSeven]

I have no idea about br1 and br2. I only have br0 in my AC86U. I think you should revert MTU for all ports (other than WAN) to default value of 1500. It should just works.
I think the problem is when you lower your router LAN eth MTU from default 1500 to 1492. Windows by default has a MTU of 1500 as well. From Win 10 machine point of view, packet fragmentation will happen when the packet is larger than 1500 Bytes. But now the router LAN port cannot support 1500 bytes anymore, it can only support a Maximum of 1492 bytes. Meaning any packet larger than 1492 bytes will be dropped by router LAN port. After you change windows MTU to match your router LAN MTU of 1492 it will works again because now the MTU matches for both ends.

Thanks for the help. Turns out it was something screwy at my ISP (a small municipal broadband company). When I gave them my findings about MTU, they immediately recognized it as their problem and fixed whatever they had changed. Everything works fine again with the normal MTU of 1500. So the reason none of what I had to do made sense was that the ISP had something set up incorrectly at their end. It's a young company and they're still sorting shirt out, so this kind of screw-up isn't too surprising. At any rate, it's working now. Thanks again for the help!

 

[heysoundude]

You may be onto something. When I gave my ISP (a small municipal broadband company) my findings about MTU, they admitted the problem was on their end and reversed whatever they had changed. So perhaps they were testing ipv6 in preparation for a future rollout or something. I'll ask them about it. Thanks for the lead!

Be sure to encourage them, and do the homework I suggested in advance.