I know it's been a few weeks, but I think I identified the culprit of this strange problem in case anyone is interested. To reiterate, the home server I have configured has a couple purposes. It's a plex media server, a backup server and I also have iSpy agent on there to record video on a couple of my security cameras (of various makes and models). When I originally thought the media server was the problem, it was because it would go away if I unplugged it from the network. However, I have been been investigating if there were specific services that might be the problem. I came to discover that the issue would go away if I stopped the iSpy agent server that watched the cameras. Aha! So this thing watches 5 Cams, 2 are PoE connected ethernet, the other 3 are wyzecams that use the 2.4ghz band to connect. I also didn't start using iSpy agent until about a week before I attempted the AImesh that started this whole thing (the timing of which seems to have all been a coincidence at this point).
To continue, in order to get the Wyzecams to show up in the iSpy software, you need to install special RTSP firmware from Wyze as this ability isn't available on stock firmware. After looking specifically at issues with RTSP on Wyzecams, I've seen many threads complaining about how the RTSP firmware kills WiFi networks, even though the bandwidth they actually use when streaming isn't even close to taxing the network in any significant way. Wyze has also said they don't put many resources into maintaining this firmware. So, now that I'm fairly confident I have found what device/s are causing this, I started to think of some solutions. Preferably that don't involve throwing out the cameras (though Wyze cams are pretty inexpensive).
First what I did was bust out my old router again (AC66U), and add it as an AP in the network. I connected the Wyzecams to that with a unique SSID so they have a dedicated 2.4g radio for themselves. Oddly enough, when I turned the iSpy agent service back on, the 2.4ghz network plunged once again. So, ok, there must be something these Cameras are doing that is pinging everything in the subnet, some protocol I'm not familiar with. So my last idea was to configure my old router as a router, give it its own subnet (192.168.2.0), and connect the wyzecams to that. That way, whatever junk they're sending out (if it is indeed isolated to the subnet) shouldn't effect all my other 2.4g devices. I went ahead and flipped the Agent service back on, watched the Cams come back up, measured the speed of the 2.4g network..... No drop. In the past I would have needed to change the IP of the server to get it to work temporarily with everything turned on.
I wanted to post this update in case anyone has a similar issue, as niche as it probably is. I still don't know what those cams are doing that causes it, but isolating them on their own subnet has saved me the trouble of replacing with another brand. Thanks to everyone that tried to help.