Best setting to reduce latency for real-time audio applications

[drinkingbird]

I beg to differ --- I find ping to be really useful for debugging/optimizing a home network. I agree that pinging random servers out on the net isn't going to give you much beyond a basic connectivity check. But there's nothing you can do about conditions out on the net anyway. What you can do something about is your home net, and logging ping results over time can give you a lot of insight into that.

As an example, awhile back I set up a cron script to steadily ping various other machines (particularly my routers) from my primary workstation, and discovered that the wireless connection to my FiOS router was dropping close to 1% of pings. That's frickin' awful, and it accounted for a lot of the visible performance problems I was having, such as slow web page loads. That eventually drove me to pay somebody to pull ethernet cable to replace that wireless link, something that's improved matters quite a bit. I've also used ping drop rates and round-trip-time stats to determine whether wireless AP placements are good and whether wireless interference is tolerable.

Sure, you are not going to see consistent-to-sub-millisecond ping times across a wireless link, but that's not the point. It's the fraction of dropped or very slow pings that matters.

To clarify, I'm talking about ICMP ping being unreliable/inaccurate to a certain extent. If you're using CRON, I'm assuming you were using TCP Ping which is far better (I believe that is the default for Linux).

I at one time used ICMP ping tests (graphed via cacti/rrdtool) to diagnose a packet loss problem my ISP was having. Turned out an electrical wire was laying on the coax somewhere in my neighborhood. They would not even look into it until I showed them the graphs. In that case, finding significant packet loss, ICMP was perfectly fine.

A consistent 1% loss is a reliable result even with ICMP. A drop every now and then, even to your own internal router, may just be the nature of ICMP (or rather the devices it passes through not treating it with the same priority as other protocols).

For latency, if you're trying to figure out which settings can shave off a msec, ICMP simply isn't accurate enough. TCP ping is better (the default for linux I believe), but having a reliable probe/listener setup is the best way. Not always practical obviously.

 

[drinkingbird]

Working for an ISP back in the day and troubleshooting customers the worst connections were ViaSat @ 2000-5000ms pings from our backbone to the customer. Making changes on those devices was painful waiting for the text to catch up to the screen. Scripting things out in notepad and pasting them into the buffer made more sense.

It could always be worse.

I have to ssh into devices in the middle east and Asia, through a jump server that sits in London. Even though it is wired the whole way, it is painful typing anything. I can only imagine how bad satellite is.

 

[anotherengineer]

Most times lag spikes are worse than ping. Is an article here about it

Wi-Fi Ping Spikes: Causes and Fixes - SmallNetBuilder

Wi-Fi ping spikes aren't always caused by a poor connection. Your router settings could be to blame.

www.smallnetbuilder.com

 

[heysoundude]

See, I warned you it was a can of worms @Zacharybinx34 lol

 

[Zacharybinx34]

The situation is for use with this software:

Audiofusion | Pro-Level Wireless In-Ear Monitoring

audiofusionsystems.com

 

[heysoundude]

My weekend gig will be upgrading to a digital console soon - performers have been asking for this for a while. Thank YOU!!
I'd guess you'd need to create a subnet for the server, or a seperate guest network just for the stage, possibly both...I'll dig in and suss it out, but as to latency between making an adjustment and having it translate to the receiver...I revert to my crusty old soundguy answer about cables ;-p {fistshake - you kids and your everything wireless toys mumble grumble...}

Another thing: you have to be mindful of AD and processing latency BEFORE going wireless too. Have you looked at where the server gets its sources? The more tech/conversions/processing you put between mics and ears, the more latency, even when it's "compensated" for, remember.

you'll have to inbox me to dig deeper, because this can go way past networking.

 

[Zacharybinx34]

My weekend gig will be upgrading to a digital console soon - performers have been asking for this for a while. Thank YOU!!
I'd guess you'd need to create a subnet for the server, or a seperate guest network just for the stage, possibly both...I'll dig in and suss it out, but as to latency between making an adjustment and having it translate to the receiver...I revert to my crusty old soundguy answer about cables ;-p {fistshake - you kids and your everything wireless toys mumble grumble...}

Another thing: you have to be mindful of AD and processing latency BEFORE going wireless too. Have you looked at where the server gets its sources? The more tech/conversions/processing you put between mics and ears, the more latency, even when it's "compensated" for, remember.

you'll have to inbox me to dig deeper, because this can go way past networking.

It's a Behringer Wing into a Mac Studio into the Wifi Router.

 

[drinkingbird]

Unfortunately WIFI and low latency (at least the low latency that pro audio requires) are mutually exclusive. Wireless mics or instrument connections use their own technology which was designed for keeping low and consistent latency, even the newest bluetooth standards have improved on latency quite a bit. But WIFI, especially with more than 1 client connected to a radio, is not focused on that. For the vast majority of things, the latency is fine, but for low latency audio, the extra latency and the jitter inherent in WIFI is going to be hard to overcome. Even with a single client connected, interference plays a part. Like I said before, using 5Ghz with only 1 client connected is probably the best you're going to be able to do. Two or three 5Ghz clients with excellent signal strength may be workable too. Limiting the client to a single stream and disabling MU-MIMO etc may yield latency benefits too, any time multiplexing is involved latency and jitter go up.

As has been mentioned, when you start chaining multiple wireless things (wireless mic going into a console that then goes to a computer then to wireless monitors, etc) it just compounds the issue. The measures that pro audio has to compensate (adding delay in other places to keep things in synch) can only do so much and is very hard to get dialed in just right, especially if you have a lot of jitter/fluctuation in latency.

I'm not an audio engineer (but have dabbled in pro audio at various times) but I'd think any computer connection should be hard wired, and any thing that must be wireless (such as in-ear monitors) should be running on their own dedicated tx/rx setup where one end is is hardwired into the console, not over a wireless router. In a studio environment, there really isn't any need for anything to be wireless. In a stage environment, I'd think it should be limited to just mic, in-ear monitors, and possibly instrument connections, again each using their respective dedicated TX/RX setup designed for the purpose.

 

[heysoundude]

Start here (It's worth it): https://www.soundonsound.com/techniques/ethernet-audio
And then maybe see if the CapEx budget can stand an Aviom system

Then go read https://www.aes-media.org/sections/uk/Conf2011/Presentation_PDFs/07 - Al Walker - Applications in Live Convert Sound.pdf
(and bear with me while I remember meeting Uli Behringer at AES-NYC in 1992 when he was just starting to disrupt things and AT&T had Tuvan throat singers to demonstrate the capability of their DSP-addon system for the SSL4000 console. I told you I was old and crusty and it's a can of worms!!)

 

[heysoundude]

Unfortunately WIFI and low latency (at least the low latency that pro audio requires) are mutually exclusive. Wireless mics or instrument connections use their own technology which was designed for keeping low and consistent latency, even the newest bluetooth standards have improved on latency quite a bit. But WIFI, especially with more than 1 client connected to a radio, is not focused on that. For the vast majority of things, the latency is fine, but for low latency audio, the extra latency and the jitter inherent in WIFI is going to be hard to overcome. Even with a single client connected, interference plays a part. Like I said before, using 5Ghz with only 1 client connected is probably the best you're going to be able to do. Two or three 5Ghz clients with excellent signal strength may be workable too. Limiting the client to a single stream and disabling MU-MIMO etc may yield latency benefits too, any time multiplexing is involved latency and jitter go up.

As has been mentioned, when you start chaining multiple wireless things (wireless mic going into a console that then goes to a computer then to wireless monitors, etc) it just compounds the issue. The measures that pro audio has to compensate (adding delay in other places to keep things in synch) can only do so much and is very hard to get dialed in just right, especially if you have a lot of jitter/fluctuation in latency.

I'm not an audio engineer (but have dabbled in pro audio at various times) but I'd think any computer connection should be hard wired, and any thing that must be wireless (such as in-ear monitors) should be running on their own dedicated tx/rx setup where one end is is hardwired into the console, not over a wireless router. In a studio environment, there really isn't any need for anything to be wireless. In a stage environment, I'd think it should be limited to just mic, in-ear monitors, and possibly instrument connections, again each using their respective dedicated TX/RX setup designed for the purpose.

I am one (audio engineer) and our world has been crashing into IT/Networking for quite some time as evidenced by that first link I posted. And the reason the big boys that do the big events/concerts have dedicated TR/RX systems that relay wireless-ly in the RF bands is because that's what works best. Just this past weekend, I had 2 old-school floor monitor mixes and 2 (but I can do up to 4) IEM mixes, and those were analog signal to proprietary transmitters on stage. (Don't get me started about the complexities of clocking a number of freewheeling digital audio devices to a common system clock for sample accuracy...I'd love to see clarity (that I can understand) in the networking standards about how that's handled...and may be a factor in the original query because it's same-same AFAIC)

I'll still try the software, because it may be spectacular for small things, but I'll bet now it'll be more trouble than solution/option...still, thanks for opening the doors to learning, @Zacharybinx34

 

[drinkingbird]

I am one (audio engineer) and our world has been crashing into IT/Networking for quite some time as evidenced by that first link I posted. And the reason the big boys that do the big events/concerts have dedicated TR/RX systems that relay wireless-ly in the RF bands is because that's what works best. Just this past weekend, I had 2 old-school floor monitor mixes and 2 (but I can do up to 4) IEM mixes, and those were analog signal to proprietary transmitters on stage. (Don't get me started about the complexities of clocking a number of freewheeling digital audio devices to a common system clock for sample accuracy...I'd love to see clarity (that I can understand) in the networking standards about how that's handled...and may be a factor in the original query because it's same-same AFAIC)

I'll still try the software, because it may be spectacular for small things, but I'll bet now it'll be more trouble than solution/option...still, thanks for opening the doors to learning, @Zacharybinx34

Well analog and digital is a whole other can of worms.

Once everything is in the digital domain (you've done your A/D conversions etc) then a single clock source is very easy. But as you say, getting to that point is where the challenge lies. Analog, A/D conversion, Digital, all use different standards, sampling rates, etc.

I would have hoped at this point that A/D converters at least would be able to clock off the same master clock as everything on the digital side. As long as your A/D converters are using the same sampling rate as everything else they're interacting with in the digital domain, then it should be easy. But of course, nothing is ever that cut and dry. Any setup I've seen used dozens of different brands of equipment (since each brand excels at certain things) and they don't get together and collaborate on making their stuff all follow the exact same standard and interoperate perfectly.

Within the networking world, the clocking is actually quite easy and accurate. Everything in digital requires pretty accurate clocking and it all uses the same standard in networking.

Please at least tell me the days of like 5 A/D and D/A conversions in the path are gone - I remember being in a studio with analog inputs to the board/console, but the board was digital with an A/D on each input, but then analog outputs from the board (with D/A on them) going into processors (some analog, some digital, all daisy chained), which then fed into a digital recorder that used DAT for the final mix. Ugh.

 

[heysoundude]

Well analog and digital is a whole other can of worms.

Once everything is in the digital domain (you've done your A/D conversions etc) then a single clock source is very easy. But as you say, getting to that point is where the challenge lies. Analog, A/D conversion, Digital, all use different standards, sampling rates, etc.

I would have hoped at this point that A/D converters at least would be able to clock off the same master clock as everything on the digital side. As long as your A/D converters are using the same sampling rate as everything else they're interacting with in the digital domain, then it should be easy. But of course, nothing is ever that cut and dry. Any setup I've seen used dozens of different brands of equipment (since each brand excels at certain things) and they don't get together and collaborate on making their stuff all follow the exact same standard and interoperate perfectly.

Within the networking world, the clocking is actually quite easy and accurate. Everything in digital requires pretty accurate clocking and it all uses the same standard in networking.

Please at least tell me the days of like 5 A/D and D/A conversions in the path are gone - I remember being in a studio with analog inputs to the board/console, but the board was digital with an A/D on each input, but then analog outputs from the board (with D/A on them) going into processors (some analog, some digital, all daisy chained), which then fed into a digital recorder that used DAT for the final mix. Ugh.

lol yes those days are mostly behind us - OP's control surface is the mother ship for routing, with AD and DA happening on ethernet-connected stage boxes. AD at the source, DA at the destination, processing happening - in this scenario, the "console" is a router that has remote control capabilities.

The Sound on Sound link suggests that there are specific routers/APs available for Audio Networks. In this case, OP's AX-series router will be just a WAP for one of those, which I suspect is the WING control surface, and THAT's what should be running the IEM-wireless app. (I'd like to see the config tutorial, if I'm right, but I'd wager it's coming in the next upgrade/generation, now that we've got this model happening prevalently)

For people following along with this convo - if you want a career in media/entertainment/"broadcast," go get some networking credentials (and for dog's sake, learn how to terminate UTP cable to look like a real bad@$$) and look for gigs keeping networks up and running on-set/on-location, wrangling data, making massive amounts of it move in real time over (significant) distance. I would, if I had to do it over again starting now. This Is The Way: The talent can't show off/get paid if they can't get their show/performance out to the world.

 

[drinkingbird]

lol yes those days are mostly behind us - OP's control surface is the mother ship for routing, with AD and DA happening on ethernet-connected stage boxes. AD at the source, DA at the destination, processing happening - in this scenario, the "console" is a router that has remote control capabilities.

The Sound on Sound link suggests that there are specific routers/APs available for Audio Networks. In this case, OP's AX-series router will be just a WAP for one of those, which I suspect is the WING control surface, and THAT's what should be running the IEM-wireless app. (I'd like to see the config tutorial, if I'm right, but I'd wager it's coming in the next upgrade/generation, now that we've got this model happening prevalently)

For people following along with this convo - if you want a career in media/entertainment/"broadcast," go get some networking credentials (and for dog's sake, learn how to terminate UTP cable to look like a real bad@$$) and look for gigs keeping networks up and running on-set/on-location, wrangling data, making massive amounts of it move in real time over (significant) distance. I would, if I had to do it over again starting now. This Is The Way: The talent can't show off/get paid if they can't get their show/performance out to the world.

Interesting. I get the reason for doing A/D back to D/A in a stage environment (reducing interference and going longer distances), but hopefully they're using really high sampling and bit rates to preserve quality. I mean if a consumer blu-ray can do 192/24 hopefully pro audio is doing at least that. I would have guessed that they'd do the A/D and keep it in the digital domain through the board and processing until it leaves to go to the amps. But I guess we're not there yet (and some processing admittedly just sounds better in analog, though usually that's guitar heads and effects pedals which would be before the A/D anyway).

I'm a network engineer/architect and one of the first things I learned 20+ years ago was terminating RJ45 and Coax. Maybe I can make a career change

 

[heysoundude]

Interesting. I get the reason for doing A/D back to D/A in a stage environment (reducing interference and going longer distances), but hopefully they're using really high sampling and bit rates to preserve quality. I mean if a consumer blu-ray can do 192/24 hopefully pro audio is doing at least that. I would have guessed that they'd do the A/D and keep it in the digital domain through the board and processing until it leaves to go to the amps. But I guess we're not there yet (and some processing admittedly just sounds better in analog, though usually that's guitar heads and effects pedals which would be before the A/D anyway).

I'm a network engineer/architect and one of the first things I learned 20+ years ago was terminating RJ45 and Coax. Maybe I can make a career change

I would encourage you to look into the integration of Samsung/Harman/JBL-Crown-BSS-Soundcraft- etc family at the Pro/Touring level: there are probably more RJ45's than XLRs in a rack/roadcase...so you'll fit right in on the crew/bus!
Even the amplifiers are digital (and the amount of processing built in them is frankly staggering). You might be gobsmacked by how far the code has come in terms of making the aural discernment between analog and digital processing strikingly difficult (and comparatively inexpensive!). Converters have come a VERY long way, and if PCM gives way to DSD conversion, the experience will be surreal.