Why Latency Is the Most Important Number in Voice AI

Latency is not a technical vanity metric. It is the variable that determines whether a caller experiences a conversation partner or a broken system — and the threshold at which it starts to matter is lower than most teams expect.

Research on human conversational turn-taking shows that in natural conversation, the average response offset is 0–200ms. Humans formulate responses before a speaker has finished their turn. Voice AI cannot do that — it must wait for end-of-turn detection before it can begin STT. That structural gap means telephony infrastructure latency is not a small overhead; it is a direct subtraction from an already tight budget.

The thresholds that production data consistently confirms:

The ITU-T G.114 recommendation for voice telephony specifies no more than 150ms of one-way transmission delay for good interactive quality. A voice AI stack has to fit STT inference, LLM generation, TTS synthesis, and all network traversal inside a window that telephony engineers have spent decades defending.

Your telephony infrastructure controls one specific segment of that budget: the carrier leg. That segment is what Vobiz is built to minimize.

The Latency Budget: Where Every Millisecond Goes

A typical voice AI pipeline for a production phone call looks like this:

As analysis across 4M+ production voice agent calls shows, median production voice AI today delivers 1,400–2,500ms — 2–6x over the 500ms conversational ceiling. The gap is not primarily in the AI models. It is in how the components are assembled, and specifically in the network traversal stages at either end.

There is also a hidden latency tax that teams rarely account for: audio format conversion. The PSTN delivers audio at 8kHz G.711 µ-law. STT models typically expect 16kHz PCM; TTS models often use 24kHz. Any mismatch triggers resampling, and resampling from 8kHz to 16kHz costs 50–150ms each way — every turn, every call, quietly spending budget that was never allocated. Vobiz handles this in the infrastructure layer, before audio reaches your AI pipeline.

Why 500–800ms Is Not a Benchmark Worth Celebrating

Industry blogs from large CPaaS platforms frequently cite "sub-800ms" as a meaningful achievement. Hitting 800ms consistently across carrier infrastructure is genuinely non-trivial engineering. But 800ms is not a bar that produces natural conversations. It is a bar that produces conversations that do not cause callers to immediately hang up.

The budget arithmetic explains the problem clearly. If the carrier leg alone costs 200ms — a number that many generic CPaaS platforms produce when handling Indian PSTN traffic — then the remaining pipeline (STT + LLM first token + TTS) has 600ms to complete to hit the 800ms ceiling. LLM inference alone, on a non-streaming model, routinely consumes 350–1,000ms. The math does not work.

The teams building production voice AI in 2026 — on Vapi, Retell AI, ElevenLabs, LiveKit, Pipecat — are investing heavily in streaming STT, streaming LLM token generation, and early TTS initiation to claw back milliseconds. That work is wasted if the carrier leg is eating 200ms before the AI has received a single audio packet.

Vobiz sets a stricter bar: average <80ms end-to-end RTT on the carrier leg, measured in production across thousands of concurrent calls. That leaves the budget where it belongs — available to the AI pipeline.

The Vobiz Latency Stack: Four Layers

The <80ms figure is not the result of a single optimization. It is the product of four distinct infrastructure decisions that compound.

Why India Is a Harder Problem

India's telecom environment compounds every latency challenge that exists in any market. Three specific factors make the problem structurally harder here than in the US or EU:

Carrier fragmentation. India's mobile market is served primarily by Jio, Airtel, and Vodafone Idea — but the interconnection agreements, routing tables, and quality-of-service policies between these carriers and enterprise SIP providers are not standardized in the way they are in North American or European markets. A call that routes cleanly between two parties in one carrier combination may traverse multiple additional hops in a different combination, with latency that is invisible to the application layer.

Urban–rural network variability. India's wireless tele-density varies sharply: 8.25% urban vs 0.61% rural as of mid-2025 per TRAI data. An agent optimized for calls originating in Mumbai or Bengaluru will behave differently on calls from Tier 2 and Tier 3 cities — with higher packet loss rates, more variable jitter, and codecs that may not match what the AI pipeline expects. Robust RTP handling is not optional; it is the difference between a product that works across India and one that works in metro demos.

TRAI routing compliance. TRAI regulations govern how automated outbound calls are routed, what caller IDs can be presented, and what series numbers (140/160 series) are permitted for commercial calling. Generic global CPaaS platforms, built primarily for US/EU compliance frameworks, typically do not ship with TRAI-aware routing configurations. Teams using them end up either non-compliant or building compliance tooling on top of infrastructure that was not designed for it — adding complexity, cost, and latency in the process.

Vobiz was built specifically for this environment, not adapted to it after the fact.

What <80ms RTT Actually Gives Your AI Pipeline

The practical implication of a <80ms carrier leg is not an abstract benchmark improvement. It is a reallocation of latency budget that directly affects what your AI pipeline can do.

A carrier leg that costs 200ms forces your STT, LLM, and TTS to complete in 300ms to hit a 500ms total response time. That is a near-impossible constraint on any non-trivial LLM inference. Teams working under that constraint end up making compromises: smaller models with lower reasoning quality, aggressive response caching that reduces conversational flexibility, or accepting 800ms+ total latency that customers notice.

A carrier leg that costs 80ms gives the same pipeline 420ms to work with — enough headroom for a streaming LLM generating first tokens in 150ms, a TTS synthesis stage initiated concurrently, and a natural-feeling conversation that does not require the AI to be artificially constrained.

Specifically, <80ms RTT enables:

• Streaming STT and LLM token generation to work as designed, without the AI pipeline sitting idle waiting for audio to traverse unnecessary network hops
• More capable LLM models — larger context windows, tool calls, multi-step reasoning — without blowing the total latency budget
• Consistent behavior across urban and rural Indian callers, not just metro callers on ideal networks
• Headroom to absorb genuine network variability without the total response time crossing the perceptible-delay threshold

The Vobiz AI voice agent architecture is designed so that none of this requires custom configuration. The infrastructure behaves this way by default, across 130+ countries for DID numbers and outbound reach into 190+ countries.

Observability: Full Visibility Into the Call Path

Latency problems in voice AI are notoriously difficult to diagnose because the failure mode is silent. A call where the carrier leg added 300ms looks identical to a successful call from inside your application logs. The CRM records a connected call. The webhook fires. The STT transcription arrives. Nothing in the data shows that 300ms of the caller's patience was spent waiting for audio to traverse a poorly optimised SIP route.

Vobiz provides:

• Real-time call flow debugging — per-call visibility into SIP signaling, RTP stream quality, and media path routing, accessible from the Vobiz console
• Performance monitoring — latency percentiles, packet loss rates, and codec negotiation outcomes across all active calls, not just aggregates
• Webhook delivery on every call event — ring, answer, hangup, duration, and quality metrics per call, so downstream systems have the data to correlate telephony performance with AI pipeline outcomes
• WebRTC SDK for browser-based calling paths, with the same observability layer applied to WebSocket audio streams as to SIP

If your AI voice agents are producing unnatural pauses, dropped media, or audio quality degradation on Indian numbers, the cause is almost always in the carrier leg — not in the model. Observability at the infrastructure layer is how you confirm that, and how you fix it.

FAQs