Large HTTP Downloads failing many sites but not all

[TanyaC]

For the last 5 months I've been trying to isolate a problem with my WSUS server where updates remain in a "Downloading" State forever. After many rebuilds and some great support from Spiceworks members we found that it's an error with downloads failing part way through.

My PCs boots up connected to a VPN, and downloads work perfectly. It was by accident that I found when my VPN had disconnected and I was attempting to download an .ESD file that it too failed. I subsequently connected the WSUS server to the VPN and for sure, all downloads then work perfectly.

So, any HTTP download over around 750MB fails on any PC (Windows 7, 10, 2012 R2) when not connected to a VPN. I've tried Microsoft updates, Linux sites, and various third party sites with the same results. But some downloads work, for example, I can download the Big Buck Bunny HD AVI file no problems.

FTP and P2P downloads don't have a problem. For example, I can download the Linux Mint distro using P2P and it's fine.

Of course, my ISP denies that they are interfering with my traffic. We have a 100mb/s down, 40mbp/s up fiber service (NBN). I just get the standard response of "It's either a Microsoft problem or your equipment is faulty". Optus habitually interfere with customer traffic in many nefarious ways, but I moved from them to Vocus Group in March 2017. It's possible the problem has been around for as long as I've been with Vocus, but went undetected because all PCs except the server use a VPN, and the Windows updates were quite small.

Windows 10 updates though are now cumulative and 950MB+ and growing.

I've power cycled all equipment in the chain (NTD, Router, Switch). My router was on 380_68, it's now on 380_68_4. It's an ASUS RT-AC87U and it's about 18 months old. I don't have another router available to test with.

Could this be a router related problem. If so, I'd appreciate any thoughts and ideas on what to look for and test.

EDIT: I've turned off URL and keyword filtering (I had stuff to try and block out "objectionable" content), and network services filters.

 

[degrub]

Sounds like the connection to the server on the other end is timing out or too busy with HTTP protocol. Could be something limiting bandwidth or too much HTTP traffic load so it gets throttled.

 

[john9527]

Make sure your base router MTU is correct...try adjusting it down.

 

[TanyaC]

Thanks for your replies.
As I said, the ISP denies that they are limiting traffic, or interfering with it in any way.
I've tried several MTU values (The default for Merlin is 1492). Nothing I've tried helps.
I did some more investigations - downloads are failing at different places. The same file will always fail at the same spot, but for different downloads that failure is after a different amount transferred.

In trying to track this down I factory reset the router. I then added back my DHCP reservations, Wireless MAC filters and DNS servers (OpenDNS). Now when I use Firefox about 90% of the time I get an error telling me the website is insecure when trying to visit catalog.update.microsoft.com, and I cannot add an exception.

Q: What router setting might been causing the insecure website error?

In frustration I purchased a new router - The ASUS RT-AC88U and loaded the latest Merlin firmware on it. Applied the same settings I did to the AC87U and it gets the same error trying to visit Microsoft update catalog.

Using IE11 I can access the site, but the downloads still fail even on the new router. When I can access the site on Firefox, the downloads also still fail.

I'm no network or routing expert, but when I tracert the wsus update site off the VPN this is what I get...

C:\Users\Tanya>tracert wsus.ds.download.windowsupdate.com

Tracing route to a767.dscd.akamai.net [122.149.7.49]
over a maximum of 30 hops:

1 <1 ms <1 ms <1 ms RT-AC87U-5BC0 [192.168.1.254]
2 2 ms 2 ms 2 ms 252.mel0308.mel.iprimus.net.au [203.134.50.252]
3 3 ms 2 ms 2 ms 209.mel0308.mel.iprimus.net.au [203.134.50.209]
4 2 ms 3 ms 3 ms be6.bsr02.melbvoc.vic.m2core.net.au [203.134.25.106]
5 16 ms 16 ms 16 ms 253.1.2.123.network.m2core.net.au [123.2.1.253]
6 15 ms 16 ms 16 ms 49.7.149.122.sta.dodo.net.au [122.149.7.49]

Dodo is a local ISP that is part of the M2/VOCUS group, as is my ISP, Barefoot Telecom.

When I use a VPN this is the route (And download ALWAYS complete successfully)...

C:\Users\Tanya>tracert wsus.ds.download.windowsupdate.com

Tracing route to c-0001.c-msedge.net [13.107.4.50]
over a maximum of 30 hops:

1 15 ms 15 ms 14 ms 10.10.1.1
2 15 ms 15 ms 15 ms 137.59.252.254
3 15 ms 14 ms 15 ms 137.59.252.40
4 14 ms 14 ms 15 ms 137.59.252.32
5 36 ms 36 ms 41 ms 8075.syd.equinix.com [45.127.172.36]
6 * * * Request timed out.
7 * * * Request timed out.
8 15 ms 15 ms 15 ms 13.107.4.50

I'm wondering if Dodo is hosting an Akamai cache server, and that's where the problem lies???

 

[TanyaC]

Found the problem.
The issue with the insecure website just seems to have stopped.
However, the downloads from Microsoft update catalog, Windows update and via WSUS were all related to one keyword in my filters. Had many, and had to try each one, one at a time.

My understanding was that the keyword filters were to block http websites with content matching the filter.

The filter actually kills downloads from dozens of sites I tried. I guess the list is endless.

I had a four letter word in my filters. It seems this four letter word is contained within hundreds of what should be totally innocuous files. I'd rather not post the word as it's obscene, but if you want to try it for yourself, PM me and I'll give you the word.

Then go to Microsoft Update Catalog and try and download KB4038788. It should fail at around 75 - 80mb. Or KB4032188, which will fail around 740 -750mb.

Taken this word out of my filters and all downloads from all sites not work as expected.