I’ve had an AI voice agent running against Asterisk for a few months now and I’d like a sanity check on the transport before I build anything else on top of it.
Here’s how it’s wired up.
The dialplan answers the call and runs AudioSocket(<uuid>,<host>:9092), with the persona name passed in a channel variable. A sidecar accepts the TCP connection, reads the UUID frame, and runs the loop: caller audio into speech to text (Whisper, or ElevenLabs Scribe), with WebRTC VAD deciding when the caller has stopped talking. The transcript goes to an LLM streamed token by token, and the reply comes back out through text to speech (Piper locally, or ElevenLabs), resampled to slin16 8 kHz and written as AUDIO frames. Barge-in just drops the outbound queue the moment VAD fires again. If the model asks for a tool, the sidecar POSTs it to a webhook and feeds the result back into the same turn.
One thing caught me out, in case it saves somebody an afternoon. app_audiosocket writes each AUDIO frame at the channel the moment it arrives. So if you synthesize a whole sentence and write it in one go, you overrun the far end’s jitter buffer and the caller hears only the tail of every phrase. Nothing logs an error. It just sounds broken. What fixed it was metering the outbound audio to the 20 ms frame clock and re-clamping the deadline every frame, so a slow TTS response can’t make the next bit burst to catch up.
So, my question. For bidirectional real-time audio like this, is AudioSocket still the transport you’d point someone at, or should new work be targeting a WebSocket based external media path instead?
The specific things I’m unsure about:
Is AudioSocket considered stable and maintained for this kind of load, or is it more of a simple building block that people are expected to outgrow?
As far as I can tell AudioSocket is fixed at slin16 8 kHz. Is there a supported way to get wideband without transcoding twice?
Has anyone compared AudioSocket with ARI external media over WebSocket on latency and on CPU per concurrent call? The number I care about most is the gap between the caller going quiet and the first frame of the reply going out.
AudioSocket gives me no call control, so a transfer has to go back out over AMI or ARI on a separate connection. Is that what people normally do, or does everyone who needs both just run ARI for the whole thing?
I did my own little AudioSocket example in the fastagi_audio_player_async script in my seaskirt_examples repo. These are demos of my Python wrapper for every IPC-based API that Asterisk supports. Yes, you have to do your own pacing of writing to the AudioSocket connection.
The script is both a FastAGI and and AudioSocket server, with the call control done in the AGI part.
AudioSocket seems to hold up well enough. And it has been around longer than the WebSocket stuff.
The FastAGI-for-control, AudioSocket-for-media split is basically what we landed on too, just with the control side living in the dialplan and a REST hop instead of AGI.
And agreed on longevity — that’s a big part of why we went with it. The protocol’s small enough that there’s very little to break, and it runs on the older
boxes our customers are actually on.
Just another comment, I believe you are using the AudioSocket UUID to choose the persona.
This works, I guess, but note that you can pass arbitrary information in the AGI connection arguments and variables. My example uses this to pass the names of channel variables in which to return the generated UUID and address on which the newly-created AudioSocket listener is awaiting a connection. The UUID itself is then used as a kind of authentication check against spurious connection attempts from unauthorized sources.
You’ve got it exactly: the UUID picks the persona. The dialplan derives it as SHA1(${UNIQUEID}), then pre-registers {uuid → persona, caller, uniqueid} with the sidecar over a small POST /register before AudioSocket connects; the sidecar looks that UUID up when the UUID frame arrives. So it’s really a correlation key handed to both sides.
The auth angle is a good call-out, and one we don’t fully exploit yet: right now an unregistered UUID falls back to a demo persona instead of being rejected, so it’s not a hard gate. Turning the registry into a strict allowlist, dropping any UUID that was never pre-registered, is a clean hardening and I’ll add it (the listener also binds to 127.0.0.1 by default, so the surface is small unless you move the sidecar off-box). And returning the listener address back through channel variables is nicer than what we do: we compute the UUID in the dialplan and let AI_HOST/AI_PORT be set there (default 127.0.0.1:9092), but for picking the media host at call time, handing the address back dynamically the way your example does is clearly the better pattern, worth adopting for the multi-host case.
Shipped it. An AudioSocket connection whose UUID was never pre-registered now gets dropped instead of falling through to the demo persona, so the UUID is doing real work as an auth check rather than just correlation.
I added a test that drives the connection handler over a real UUID frame in both directions, mostly because I wanted to watch the old behaviour fail. With the guard taken back out it logs “persona=demo caller=None” and cheerfully builds the whole pipeline for a UUID nobody ever registered. That was easy to miss by reading the code, which is roughly how it survived this long.
It’s on main now and goes out with the next release. Thanks for the nudge, and for the seaskirt examples; the channel-variable return trick is on my list for the multi-host case.
That next release is out now, so the allowlist is live rather than just sitting on main. 0.1.2 on PyPI and Docker Hub.
It picked up a second fix on the way out: we were never setting TCP_NODELAY on the AudioSocket socket, so Nagle was free to sit on those 20 ms writes and quietly undo the pacing. Someone on r/Asterisk spotted it. Two separate people reading the same transport and finding two different holes in a week has been worth more than any testing we did ourselves.