Boson AI logo
More articles

Higgs Realtime: cost-efficient, real-time speech-to-speech

The Boson AI TeamAugust 3, 2026
Real conversations are fluid, and real work rarely fits into a single request. Higgs Realtime is a real-time, audio-native model and API built for the way people actually talk—following the conversation, handling interruptions, adapting mid-sentence as tasks change, and carrying workflows through to completion. Trained across multiple stages on 10M+ audio hours, it leads major deployed voice models with public APIs on real-time audio benchmarks and is compatible with the OpenAI Realtime API.
  • #1on tool-call & interruption benchmarks
  • ~700msspeech-in to speech-out latency
  • 100+languages, one voice across all of them
  • $0.0023/min (audio input)$0.014/min (audio output)
Higgs Realtime isn’t a research preview. It is a production-quality voice model, available as an API service today to anyone with a Boson API key, and it changes the economics of deploying voice AI at scale, putting production voice agents within reach for use cases where per-call cost has been the blocker.

Hear it in Action

Plan a tripA travel agent that holds the whole itinerary in mind, takes changes mid-sentence, and books while you are still talking.
Tech supportA support line that reads the account, runs the checks and resolves the issue on the call, instead of routing you onward.

Built for Real Conversation

So far, most voice AI is turn-based: the model waits for you to finish speaking, processes what you said, generates a response, and plays it back. Real conversation is bidirectional. We listen while we talk, notice when someone pauses or barges in, and adjust mid-sentence based on what we hear.
Higgs Realtime is an end-to-end, instruction-tuned, audio-native model, trained for live conversation. It understands audio, text, or both, and generates high-quality responses. It functions as a text LLM when given text alone, while bringing native audio understanding to speech and multimodal inputs.

Leading accuracy, sub-second latency

Typically 0.70s to the first audio frame when your turn ends, and 0.125s to stop talking when you barge in.

A fraction of the cost

Audio in at $0.0023 a minute and audio out at $0.014, which changes the economics of running voice agents at scale.

A drop-in for the OpenAI Realtime API

The same session events, the same audio deltas, the same tool-calling shape. Change a few lines and your code runs.

100+ languages, code-switching included

No per-language setup, and the model follows a speaker who changes language mid-sentence.
In contrast to general-purpose omni models, Higgs Realtime is trained specifically for voice-agent reflexes: natural turn-taking, conversational cadence, and tone adaptation throughout an interaction. It supports audio-native tool calling with spoken preambles, multi-turn state tracking, and instruction following that remains robust through interruptions and changing requests. Semantic turn detection helps the model distinguish a completed turn from a pause, while multilingual and code-switched understanding lets users switch languages naturally within the same utterance.

Benchmarks

On real-time audio benchmarks, Higgs Realtime leads major deployed voice models with an accessible API. Scores are on a 0–100 scale, higher is better; the best score in each benchmark is highlighted.
ComplexFuncBench (Audio)
Higgs Realtime
83.4
gpt-realtime-2
77.7
gpt-realtime-1.5
77.3
grok-voice-think-fast-1.0
65.1
gemini-3.1-flash-live
56.6
The headline benchmark: complex, multi-step tool calling, rendered to speech.
Figures from our 2026-07-02 evaluation run. All models are compared in their no-thinking configuration, matching Higgs Realtime: gpt-realtime-2 with thinking set to none, gemini-3.1-flash-live at minimal thinking, and grok-voice-think-fast-1.0 with thinking disabled. N/A indicates that the benchmark could not be run, because the grok API does not support prefilling the prior assistant turns that multiturn audio benchmarks like IHBench require. ComplexFuncBench (Audio) is a speech-rendered version of ComplexFuncBench — no official audio release exists, so published scores use our speech renderings of the same prompts.

100+ Languages

Higgs Realtime understands and speaks 100+ languages. There is no per-language setup and no language flag to pass: the model works out what it is hearing and answers in kind. Code-switched understanding means it follows a speaker who changes language mid-sentence, so a caller moving between English and Chinese stays one conversation.

Pricing

Higgs Realtime is priced per minute of audio and metered on tokens underneath. There is no platform fee and no per-seat pricing.
Audio input
$0.0023/ min
Audio output
$0.014/ min
Billing eventsPrice
Input tokens
$0.75 / 1M tokens
Cached input tokens
$0.25 / 1M tokens
Output tokens
$4.50 / 1M tokens
Transcription input
$0.0025 / min
* Estimated rates reflect common usage. Actual cost is calculated from tokens consumed.
* Usage estimates exclude applicable taxes.
* See the pricing page for current rates across all Higgs models.

How We Did It

Four stages, each building a different layer of the model: the data it learns from, the curriculum it learns on, the reflexes it is trained for, and the evaluation that keeps the other three honest.

A data engine built at audio scale

Our internal pipeline processed and labeled 100M+ hours of raw audio, producing over 10M hours of model-ready training data through quality filtering, rich annotation, background-sound separation, source separation, and channel separation. The resulting corpus combines broad acoustic and linguistic coverage with the consistency production voice agents need.

A multi-stage curriculum

Training progresses from continued pretraining to teacher-guided, on-policy post-training. Each stage builds a different layer: audio understanding, instruction following, natural interaction, and real-time agent behavior. Together they let an efficient model deliver capabilities usually associated with much more expensive systems.

Post-training for voice-agent reflexes

The behaviors that make dialogue feel natural — sub-second turn-taking, interruption handling, spoken tool preambles, and adaptation across a conversation — are trained directly into the model. Much of the data for this stage is synthetic, from a multi-environment pipeline that generates high-quality question-answer pairs and multi-turn dialogues.

Evaluation grounded in real interaction

Our evaluation infrastructure runs competing real-time models under matched conditions, so comparisons hold across task capability, latency, and interaction quality. Internal suites derived from real voice-agent workflows connect that research progress to what customers actually experience in production.

Get Started

Higgs Realtime is available today as a hosted API on the Boson AI platform, at the rates above. Sign up, get a key, and start building.
The API is compatible with the OpenAI Realtime API: the same session events, the same audio deltas, the same tool-calling shape. Point the official OpenAI SDK at https://api.boson.ai/v1, set the model to higgs-realtime, and the rest of your code stays as it is.
# The official OpenAI SDK — pointed at Higgs
client = AsyncOpenAI(
- api_key=os.environ["OPENAI_API_KEY"],
+ api_key=os.environ["BOSON_API_KEY"],
+ base_url="https://api.boson.ai/v1",
)
-async with client.realtime.connect(model="gpt-realtime-2") as conn:
+async with client.realtime.connect(model="higgs-realtime") as conn:
await conn.session.update(session={
+ "model": "higgs-realtime", # required; the connect() arg is SDK-only
"output_modalities": ["audio"],
"audio": {
"input": {"turn_detection": {"type": "server_vad"}},
"output": {"voice": "default"},
},
})
# stream mic audio in; server VAD handles turn-taking
async for chunk in microphone():
await conn.input_audio_buffer.append(
audio=base64.b64encode(chunk).decode(),
)
# play assistant audio as it streams back
async for event in conn:
if event.type == "response.output_audio.delta":
speaker.play(event.delta)

Try Higgs Realtime Now

We can’t wait to see what you build.
Acknowledgments
Contributors: Dongming Shen, Ahmad Salimi, Jixuan Wang, Weisu Yin, Huapeng Zhou, Yuzhi Tang, Peter Yu, Xiling Zhao, Sepehr Harfi, Silin Meng, Ke Bai, Xinyu Wang, Wentao Ma, Yizhi Liu, Xiaoyi Li, Alex Chen, Lindsey Allen, Mu Li
Special thanks: The broader Boson team for supporting data, training, evaluation and serving.
#higgs-realtime
#speech-to-speech
#voice-agents
#realtime-api
#multilingual