Agree with @bbunge Kill the QOS. You don't need QOS, which was really designed to prioritize bandwidth amongst devices sharing a lower speed connection.
And for the record, most gaming is NOT very bandwidth intensive. As an example, the Sony PS5 only "requires" 3 Mbps. @Incubus32 if you have a 500 Mbps connection after an upgrade, I would first suggest to reboot your premise equipment such as a cable modem or ONU so your faster settings in a config file are downloaded from your ISP. If you are already realizing the 500 Mbps then you do not need to do this. From your description of a symmetrical connection, it sounds like you have fiber. Fiber has low latency so you should score an A on bufferbloat.
I will also say that QOS is an inherently STUPID function for faster connections because most ISPs over dimension your speeds to avoid subscriber complaints. As an example, my 600 Mbps connection is set to a bit over 700 Mbps by my ISP. If I were using QOS I would never know this. So turn off the QOS and unburden your router and your connection. This will probably fix your problem.
And for the record, most gaming is NOT very bandwidth intensive. As an example, the Sony PS5 only "requires" 3 Mbps. @Incubus32 if you have a 500 Mbps connection after an upgrade, I would first suggest to reboot your premise equipment such as a cable modem or ONU so your faster settings in a config file are downloaded from your ISP. If you are already realizing the 500 Mbps then you do not need to do this. From your description of a symmetrical connection, it sounds like you have fiber. Fiber has low latency so you should score an A on bufferbloat.
I will also say that QOS is an inherently STUPID function for faster connections because most ISPs over dimension your speeds to avoid subscriber complaints. As an example, my 600 Mbps connection is set to a bit over 700 Mbps by my ISP. If I were using QOS I would never know this. So turn off the QOS and unburden your router and your connection. This will probably fix your problem.