QOS, highly variable upload speed, and buffer bloat.

[CharlieTX]

TLDR, skip to the last paragraph. I currently have a Dell PowerEdge R210 II running Ubuntu as my Firewall router. It has an additional network interface to give me a total of 3. I have what used to be a very fast cable modem connection (1 gbps x 50 mbps), and a DSL connection (50 x 10 mbps). The cable connection has unlimited data, the DSL connection is capped at 1TB.

But my cable connection has become terrible. My uploads range from 1.8 mbps to about 9 mbps, and if I saturate the connection the bufferbloat will send my latency over 4 seconds, or worse. A single upload to Google Drive is all it takes to kill the network for everyone. Unless I use QOS. But I don't want to cap myself at 1.7mbps and never use the rest of the connection.

I thought the DOCSIS-PIE was supposed to solve this if I used a 3.1 modem on my 3.0 connection. It didn't.

Are there any tools that will monitor my latency and lower the bandwidth allocated using tc, and preferably (somehow) raise it back up again later? I realize that raising it would result in the occasional latency spikes, but I'll try to put the most sensitive traffic onto the DSL line for now.

 

[Trip]

While what you're proposing is potentially doable, why try solving a problem that likely doesn't have to exist at all? Upload that variable would suggest to me you've got a physical (layer 1) problem somewhere along the last mile.

Have you systematically tested the patch cable going to modem, the modem itself and/or your premise cabling, and/or also involved the ISP to ensure the feed to the house is adequate, plus their own upstream infrastructure? I would make sure to do all of that, twice even.

 

[coxhaus]

I would go over to DSLreports and post under your cable provider as to your modem and specs. As Trip said it is going to something basic.

As far as load balancing goes if that is what you are planning then you will need someone that knows Ubuntu and how to setup the rules for load balancing. Load balancing is not going to be auto magic. There are going to be limits with your rules.

 

[CharlieTX]

I've posted in DSL reports. We also have a very close neighborhood facebook group where we actually avoid politics, events, and anything that's not important. We are all suffering with terrible cable service, and the cable company just doesn't care. They are also only running 4 upstream channels, and 2 of them are half speed. The leaves 90mbps of shared upload capacity for the node. A handful of people running bittorrent could ruin it for everyone.

I don't know why I posted about load balancing. I've got it working very well. I'm using DNSMasq, ipset, and iptables to mark connections to sites I know will have highly asymmetric traffic. I really should write a howto for everything I have done. Then with an open source project I could collaborate with others to classify the traffic.

Maybe I should dump the cable company and settle for the slow DSL. But my list of sites to route connections to keeps growing as the kids want to download games on the cable modem, not DSL.

 

[CharlieTX]

Here is a snapshot of my cable modem usage for the last week, speedtest results, and all the stats from the cable modem. Yes, there is a problem. But the correctable and uncorrectable error spikes don't correlate with the performance issues.

I deleted one speedtest result from influxdb because it ran during sustained upstream use and gave a clearly invalid result. The speedtest is also used to configure my upstream bandwidth at 90% until the next one, when the script clears the qdisc from eno2 and starts anew. As a result of usage, the speedtest result came in by half, and I could see the upload usage get cut by the speedtest and the new, wrong limit.

https://snapshot.raintank.io/dashboard/snapshot/tpxXOV5MQE2dNlb8M2E5L02VTrdEsoMl

 

[coxhaus]

What about getting AT&T gig service? You would have much faster upload and lower latency. Sounds like a win win to me.

I sure wish I could get AT&T gig service. Not in my area even though they ran fiber right past me for the new neighborhoods. My only choice is DSL 18 meg service which is too slow.

 

[CharlieTX]

What about getting AT&T gig service? You would have much faster upload and lower latency. Sounds like a win win to me.

I have the fastest service they sell in my section of the neighborhood. Last year the best they would offer me was 18mbps x 1.5mpbs, so at least it went up. Yes, it would be a win/win. I feel like 5G might be my only hope, but at the same time, it's pending rollout is probably why I don't have fiber coming down my street now. New sections of my neighborhood have it, and they retrofit about 2/3s of the rest a couple of years ago. Supposedly the cable company also has FTTH plans, although they have been doing RFOG and still using docsis cable modems. And this plan seems to be what is holding them back from DOCSIS 3.1.

 

[coxhaus]

My gut feel is the latency on 5G is going to be worst than fiber or cable once they load it down. I don't see 5G in my future for home use.

 

[CharlieTX]

Back to my original question. I'm thinking a utility that would continuously ping the next hop on the cable network and build a model to find the standard deviation. As soon as the ping times went past average + N * stdDev (probably tune this, but thinking N = 2 to start with) that, it would drop the upload bandwidth by 10%.

Then it could increase the speed by 1% every minute that the ping times are conformant. I'm also thinking it should disregard the ping statistics for the minute prior to any reductions when calculating the average and standard deviation.

I'll write this, but it seemed like I shouldn't be the first to run into this, so I was hoping I could find someone else's work to start with.

 

[coxhaus]

What your tracert look like to Spectrum's DNS. My hops are pretty clean. This is while my wife is doing video conferencing right now using Zoom.

race complete.
C:\Users\lee>tracert 209.18.47.61
Tracing route to dns-cac-lb-01.rr.com [209.18.47.61]
over a maximum of 30 hops:
1 3 ms 8 ms 4 ms 192.168.0.254
2 2 ms 1 ms 1 ms router44BEE6 [192.168.10.1]
3 28 ms 13 ms 19 ms cpe-72-133-80-1.sw.res.rr.com [72.133.80.1]
4 27 ms 25 ms 29 ms tge0-0-0.elgntx0802h.texas.rr.com [66.68.3.173]
5 16 ms 16 ms 15 ms agg38.ausxtxir02r.texas.rr.com [24.175.42.140]
6 34 ms 36 ms 35 ms agg22.hstqtxl301r.texas.rr.com [24.175.41.48]
7 26 ms 27 ms 30 ms agg1.ausxtxir06r.texas.rr.com [24.175.42.172]
8 30 ms 35 ms 31 ms 24.175.43.66
9 28 ms 30 ms 27 ms dns-cac-lb-01.rr.com [209.18.47.61]

 

[CharlieTX]

I’m stuck with Suddenlink out here in Pflugerville. But pings and traceroutes are stable as long as my upstream isn’t saturated

 

[coxhaus]

Yes upstream is cable's problem child. I am in Elgin just south down the road about 25 miles.

 

[CharlieTX]

Yes upstream is cable's problem child. I am in Elgin just south down the road about 25 miles.

Your traceroute gives it away. Although I will admit I overlooked that line and thought Austin.

 

[coxhaus]

It's funny how I seem to flip flop between Houston and Dallas on the trace routes.

I thought of another place to check is maybe Reddit if there is a cable heading.

 

[Trip]

Back to my original question. I'm thinking a utility that would continuously ping the next hop on the cable network and build a model to find the standard deviation.

PingPlotter all day long. Run multiple sessions and/or from multiple hosts, to multiple WAN target IPs and see if you can triangulate on an upstream hop where things go haywire.