Use case · Mobile & consumer products

Live captions that never send audio anywhere

Caption speech as it happens — in calls, classrooms, and venues — with a streaming recognizer that runs on the device doing the captioning. Captions keep working with no connection, and the conversation being captioned never leaves the hardware it was spoken into.

→ the quick → the quick brown → the quick brown fox
Why teams do this

Why captions belong on the device

Accessibility without an asterisk

Captions that depend on connectivity fail exactly where they're needed — trains, basements, crowded venues, planes. On-device captioning works everywhere the product does, with no "requires internet" footnote.

Private by construction

Captioning a call, a lecture, or a meeting means processing everything everyone says. When recognition is local, that audio never transits a captioning vendor — a claim your privacy policy can make in one sentence.

Real-time, actually

Partial captions appear as words are spoken and finalize when the sentence ends — streaming with an 80 ms attention look-ahead, not a batch job pretending to be live.

Where it shows up

Captions across products

Anywhere speech needs to be readable the moment it's spoken.

Calling & communication

In-call captions

Live captions inside voice and video calls — including the calls too sensitive to route through a third-party captioner.

Events & venues

Caption displays on local boxes

A Linux box beside the stage drives the caption screen — the venue's audio never streams out of the building.

Education

Lectures, captioned and kept

Classroom capture that produces readable text in real time and a transcript after — on school-owned hardware.

Deaf & hard-of-hearing tools

Personal captioning

A phone on the table captions the conversation around it — offline, private, no account required.

Media & players

Captions for uncaptioned content

Live caption of audio and video that shipped without subtitles, rendered on the viewer's own device.

Workplace software

Meetings with a no-upload guarantee

Caption and transcribe internal meetings while keeping a promise security teams can verify: nothing leaves the laptop.

How it works

From audio to readable text

Add the SDK and model

One dependency brings the runtime and the 32M-parameter streaming FastConformer — English with punctuation and capitalization at v1.

Stream the audio source into the engine

Microphone, call audio, or system audio — partial transcripts stream out as words are spoken, with an 80 ms attention look-ahead.

Render partials, replace with finals

Show the partial line as it grows, then swap in the finalized sentence when the recognizer commits it on end of speech — the familiar live-caption feel.

Keep or discard the transcript

The finalized text is yours: display-only for pure captioning, or stored as a transcript — either way it never had to leave the device.

For technical evaluators

The numbers behind the promise

Accuracy and streaming speed are measured and published — see the benchmark methodology and the on-device ASR comparison.

3.27%
word-error rate on LibriSpeech test-clean (RNN-T decoder) — publishable accuracy, on a phone
80 ms
attention look-ahead in cache-aware streaming — captions trail the voice by well under a second
0.08–0.10
streaming real-time factor on iPhone 13 Pro Max — ~90% of the core stays free for your product
0
bytes of audio leaving the device — captioning works in airplane mode

Pipeline. Audio source → on-device VAD → streaming FastConformer (CTC/RNN-T) → partial captions while speaking, finalized lines on end of speech. At a streaming real-time factor of 0.08–0.10 on an iPhone 13 Pro Max, roughly 90% of the core remains free for the rest of the product.

Platforms. iOS 16+ (Swift Package Manager) and Android 8.0+/API 26 (JitPack/Gradle), plus Linux aarch64 for device-class hardware — validated on Raspberry Pi 3/4/5 and Pi Zero 2 W, NVIDIA Jetson Nano and Orin Nano, AWS Graviton and similar Cortex-A53/A55 boards — shipped as a single ~480 KB shared library with Python, Node.js, Go, C/C++ and Rust wrappers. Quickstarts: iOS · Android.

Licensing. The published English model and SDK are free for commercial use with no per-user or per-minute fees; domain or language tuning is a paid engagement — see licensing.

FAQ

Live captions, answered

How far do captions lag behind the speaker?

The recognizer streams with an 80 ms attention look-ahead and processes a 1.12 s chunk in about 90 ms on an iPhone 13 Pro Max, so partial captions trail the voice by well under a second and finalize as each sentence ends.

How accurate are the captions?

3.27% word-error rate on LibriSpeech test-clean with the RNN-T decoder — within floating-point noise of the upstream NVIDIA NeMo reference. Real-world accuracy depends on audio quality and domain; the methodology page explains how the numbers are produced.

What languages can be captioned?

English at v1, with punctuation and capitalization. Additional languages are roadmap items driven by customer demand.

Does any of the captioned audio leave the device?

No — recognition is entirely local. That's the point: captioning everything everyone says is only acceptable when nothing anyone says is uploaded.

Caption everything, upload nothing

Tell us where captions should live — calls, venues, classrooms — and which devices have to carry them.

Get started Explore Streaming ASR