Help with some playback issues since replacing my router

[Malaromane]

I'm hoping someone can offer some concrete suggestions on how to troubleshoot my setup.

My Plex setup has been working flawlessly for years. No playback stalls, no buffering. Just rock solid playback on all devices both local and remote.

My setup:

Plex Server 1: Windows 10 box running PMS connected to remote storage (Samba shares on a RAID array connected to a linux box)

Plex Server 2: Nvidia Shield TV Pro connected to an HD Homerun Connect. Content stored on Shield's local storage.

Playback devices: Mix of Roku 4 and Roku 3 boxes.

All devices are hardwired with Cat 6 cable connected through 10/100/1000 switches to the router.

As I mentioned at the top of the post, this has been working flawlessly for years. My troubles started when I replaced my Netgear R7000 Nighthawk (running DD-WRT) with an ASUS AC86U (running ASUSWRT-Merlin 384.5). Now, I get constant buffering after around 7 to 10 minutes of playback no matter what device (Roku 4, Roku 3, or web player) I use. Direct play is enabled for all devices.

I'm reasonably sure that the problem is some router setting but I'm stumped as to where to look or what to try. I've tried enabling/disabling QoS and prioritising video streaming all to no avail. I've also tried boosting the UDP Timeout: Assured and UDP Timeout: Unreplied settings to 600. This also did not help.

I'm stumped as to how to proceed.

 

[ColinTaylor]

All devices are hardwired with Cat 6 cable connected through 10/100/1000 switches to the router.

Can you describe what switches you have (make and model) and how they are connected together.

If all your devices are hard-wired then it can't be related to QoS or any other routing function because the traffic is not being routed. Check the link speed on each and every switch port in case they have not negotiated the correct speed. Make sure you haven't connected your switches in such a way as to create loops. Do you have spanning tree protocol enabled on the router?

Test the speed you are getting when you copy files to and from the Windows 10 box and the Samba shares.

 

[Malaromane]

Thanks very much for the reply @ColinTaylor. I'll report back on the actual switch makes and models when I get home from work and can gather up the necessary information. I will also confirm link speeds at that time.

In general, though, the router in the basement feeds a switch on each floor (basement, main floor, top floor) and devices on each floor are connected to the switch for that floor. The exception is the top floor where a second switch (in Room 2) is daisy-chained to the first switch (in Room 1) because I was too lazy/stupid to run a second Cat 6 line from the basement to the top floor. There are no loops.

Spanning tree protocol is enabled on the router but jumbo packets are not.

I was able to remotely test file transfers between the Win 10 server and the Samba shares and the results are thus:

A 7 GB .mkv file copied from the Samba share to the Win 10 server maintained an average throughput of 79.5 MB/s (as reported by Windows) and never dropped below 78.9 MB/s.
A 3 GB .zip file copied from the Windows 10 server to the Samba share averaged 48.3 MB/s and vacillated a bit more than the prior test but was still reasonably steady.

From a network perspective, those two devices are just about as far apart as they can get on my local network:
Linux box-> 10/100/1000 switch in Room 2-> 10/100/1000 switch in Room 1->ASUS Router in Basement->10/100/1000 switch in Basement-> Plex Server

EDIT: All switches on my network are Cisco 10/100/1000 unmanaged switches (either an SD2005 ver.3 5-port switch or an SG100D-08 v2 8-port switch).

 

[Malaromane]

Everything seemed to check out fine so I tried another test by setting up an Emby server pointing to the same files that the Plex server was serving. I've been able to watch a number of shows on the Roku 3 and Roku 4 boxes with no playback stalls at all on Emby. So I think this lets the network hardware AND the router settings off of the hook.

I still think that my issues were introduced by the replacement of the router but I'm thinking now that it caused the Plex servers to update something in their library databases and that's what is causing my issues. I'll try resetting the Plex servers and recreating the libraries from scratch to see if that clears things up.

Thank you again for the suggestions and guidance.