Streaming ASR for Linux

Streaming speech recognition on the VoxRT custom on-device inference runtime. NeMo FastConformer (32 M parameters), 16 kHz mono in, P&C-aware text out, cache-aware streaming with ~1.1 s chunks.

• Current version: v0.1.2
• Target platforms: x86_64 (Haswell 2013+, Excavator 2015+ — modern EC2 / Hetzner / GCP / Azure fleets) and aarch64 (Pi 4B / 5, Jetson family, AWS Graviton c6g/c7g/c8g, Ampere Altra, industrial SBCs)
• glibc baseline: 2.17 (manylinux2014 equivalent — Ubuntu 18.04+, Debian 10+, RHEL/CentOS 7+, Amazon Linux 2, Raspberry Pi OS Bullseye/Bookworm, Jetson L4T 32.x/35.x/36.x, AWS EC2 mainline fleet)
• Languages shipped: C / C++ (tarball + pkg-config + CMake), Python (PyPI wheel), Node.js (npm), Go (go get), Rust (git dependency). Same coverage as voxrt-wake-word-linux.
• License: Apache-2.0 (wrapper sources) · proprietary (compiled runtime, redistribution allowed via this SDK)
• Upstream model license: CC-BY-4.0 (NVIDIA NeMo)

Minimum CPU requirements (please read before you install)

The x86_64 build runtime-detects AVX2 + FMA and dispatches every hot kernel — Conformer subsampling, MHA batched matmul, LayerNorm, softmax, LSTM gates, CTC log-softmax — into 8-wide AVX2 + FMA. This is the industry-standard x86-64-v3 baseline (RHEL 9 default, Ubuntu 24.04 optimises for it, all mainline cloud fleets ≥ 2015 satisfy it).

Supported x86_64 CPUs (AVX2 + FMA):

Excluded (fall back to scalar path, not realtime):

• Intel Sandy Bridge (2011, 2nd gen Core), Ivy Bridge (2012, 3rd gen). Common on cheap 10+-year-old VPS from budget providers (some Kimsufi / OVH bare-metal Xeon E5-2670 v1/v2 lines).
• AMD Bulldozer / Piledriver / Steamroller (K15h, 2011-2014). Effectively retired from server fleets.
• Older Intel Atoms (Bay Trail Silvermont 2013, Cherry Trail Airmont 2015) — mostly cheap tablets / entry embedded.

The library still runs on excluded CPUs via a compiled scalar fallback path — it's just multiple-x slower and will not sustain realtime. If you need Sandy/Ivy Bridge servers as first-class targets, contact [email protected].

On aarch64 every ARMv8-A device has NEON, so no runtime check is needed — the NEON 4-wide FMA path is always active.

What is VoxRT?

VoxRT is a from-scratch inference runtime for on-device speech models. No ONNX Runtime, no PyTorch, no LiteRT — a custom Rust core sized and tuned for streaming voice workloads on phone-class, edge-Linux, and server-class x86_64 hardware.

voxrt-asr is the streaming-ASR product on that runtime, alongside voxrt-asr-android (JitPack AAR) and voxrt-asr-ios (Swift Package Manager). All three share the same runtime crate, the same .vxrt model file, the same C ABI, and the same NEON / AVX2 kernel set. The runtime is the product; the models are what it runs.

Sister products voxrt-silero (VAD) and voxrt-wake-word (always-on wake-phrase detection) share the same runtime and can be composed with ASR into a full agent pipeline.

Domain-tuned acoustic models, additional languages, on-premise fine-tuning, and bulk-device deployments live at voxrt.com.

Performance

Measured at ship time. RTF = wall-time-per-chunk ÷ chunk audio duration (lower is better).

x86_64

WSL2 Ubuntu 22.04 on Intel host, streaming session over 200 ms audio blocks, 500-utterance LibriSpeech test-clean subset:

On a bare-metal server CPU (Xeon Silver, EPYC Rome+) the numbers are typically 30-60 % lower still — single-thread runtime, no thread pool.

aarch64

The runtime uses the same NEON kernel set as voxrt-asr-android, whose reference-device measurements are:

Edge-Linux (Pi 4B/5, Jetson family, industrial SBCs) uses the same NEON code path as the phone builds above; RTF scales with core class + clock, but per-device numbers are not published here — measure on your target hardware via examples/c/wav-quickstart/.

CTC mode is ~15 % cheaper per chunk than RNN-T at the cost of ~1.6 pp lower accuracy on LibriSpeech test-clean (CTC 4.895 % vs RNN-T 3.267 %).

How it compares

voxrt-asr targets the same on-device streaming-ASR slot as Picovoice Cheetah and the various Whisper.cpp deployments. On Linux specifically, whisper.cpp (aarch64/x86_64) is the closest direct competitor for self-hosted server use.

Where we win unambiguously: published WER on LibriSpeech test-clean (3.27 % vs Cheetah 5.4 % vs Whisper base.en ~4.4 %), disclosed multi-tier RTF (server / phone / edge), streaming chunk granularity (80 ms vs Cheetah's 590 ms word emission latency), and license clarity (CC-BY-4.0 model, redistributable-via-SDK runtime).

Cheetah has a smaller model file (34 MB vs 61 MB) but no published streaming RTF; Whisper.cpp is more flexible for multi-language but has ~2-6× the runtime memory of voxrt-asr and no cached-streaming implementation.

Full sourced analysis: voxrt.com.

Binary footprint

• C tarball per arch (voxrt-asr-0.1.2-{aarch64,x86_64}-linux-gnu.tar.gz): ~600 KB stripped libvoxrt_asr.so + voxrt_asr.h + pkg-config + CMake configs + examples + LICENSE × 2 + README. Compressed tarball ~250-300 KB.
• ASR model streaming_medium_pc.vxrt (downloaded separately from voxrt-asr-models): ~61 MB fp16.
• Runtime heap at inference time: ~150 MB steady state (weights expand to f32 for inference; mmap'd zero-copy at load time so the .vxrt file itself is not double-buffered).

Net effect on a shipped app: roughly 62 MB once the runtime .so + .vxrt are bundled. Weight file dominates; the runtime itself is small enough to bundle inside a container image without noticeable size impact.

Install

C / C++

Download the tarball for your arch from the latest GitHub Release and extract under /usr/local:

# x86_64 host
curl -L https://github.com/VoxRT/voxrt-asr-linux/releases/download/v0.1.2/voxrt-asr-0.1.2-x86_64-linux-gnu.tar.gz \
    | sudo tar -xz -C /usr/local --strip-components=1
# aarch64 host
curl -L https://github.com/VoxRT/voxrt-asr-linux/releases/download/v0.1.2/voxrt-asr-0.1.2-aarch64-linux-gnu.tar.gz \
    | sudo tar -xz -C /usr/local --strip-components=1

sudo ldconfig

After extract:

gcc main.c $(pkg-config --cflags --libs voxrt-asr) -o my_app

Or in CMake:

find_package(VoxRTAsr 0.1.2 REQUIRED)
target_link_libraries(my_app PRIVATE VoxRT::Asr)

See examples/c/ for wav-quickstart (universal, C + C++), tcp-transcribe-server (web-demo backend skeleton, C + C++), and cmake-consumer (find_package skeleton).

Python

pip install voxrt-asr

One abi3 wheel per arch covers Python 3.9 / 3.10 / 3.11 / 3.12 / 3.13. See examples/python/quickstart for a runnable WAV → streaming-transcribe demo, and examples/python/tcp-transcribe-server for a socket server.

Node.js

npm install @voxrt/asr

TypeScript definitions ship with the package. Two prebuilt native addons per version — linux-arm64 and linux-x64. See examples/nodejs/quickstart for the runnable demo and examples/nodejs/tcp-transcribe-server for a socket server.

Go

go get github.com/VoxRT/voxrt-asr-linux/[email protected]

The go/ subdirectory of this repository IS the Go module — no separate publish step. cgo picks up the bundled .so (both arches) via ${SRCDIR}-relative rpath at build time; no LD_LIBRARY_PATH or system install required. See examples/go/quickstart for the runnable demo and examples/go/tcp-transcribe-server for a socket server.

Rust

Pin the crate via git for now:

[dependencies]
voxrt-asr = { git = "https://github.com/VoxRT/voxrt-asr-linux", tag = "v0.1.2" }

Get the streaming model

The streaming_medium_pc.vxrt model file (AES-256-GCM encrypted, ~61 MB fp16) is distributed separately from voxrt-asr-models so the SDK ships without weights and the two version independently. All v0.1.x SDK releases target the model v0.1.2.

curl -LO https://github.com/VoxRT/voxrt-asr-models/releases/download/v0.1.2/streaming_medium_pc.vxrt

SHA-256: 0d723e429157a8a8cb58739a1f090574f2f23db311ca7916b43411f5f727c79c

Three common deploy patterns:

• Bundle in your app / container — copy the .vxrt alongside your binary. Works offline from first launch, adds ~61 MB to your distribution.
• Download on first run + SHA-256 verify — recommended if you want to swap models without a full redeploy. Compare against the SHA above.
• Embed as a build-time resource — for C: any resource embedder (e.g. xxd -i). Container images: COPY into image at build time.

Quick start

C — the smallest possible end-to-end streaming loop (full working code lives at examples/c/wav-quickstart/main.c):

#include <voxrt_asr.h>

voxrt_asr_streaming_t *sess = NULL;
voxrt_asr_streaming_create(model_bytes, model_len,
                           VOXRT_ASR_DECODE_RNNT,    /* recommended default */
                           &sess);

for (;;) {
    /* Read 3200 mono S16 samples (200 ms @ 16 kHz), convert to f32 [-1, 1]. */
    read_pcm(&buf_i16);
    for (int i = 0; i < 3200; ++i) buf_f32[i] = buf_i16[i] / 32768.0f;

    /* Push; text_buf receives the transcript emitted this call (may be empty). */
    size_t written = 0;
    voxrt_asr_streaming_push_audio(sess, buf_f32, 3200,
                                   text_buf, sizeof text_buf, &written);
    if (written > 0) {
        text_buf[written] = '\0';
        printf("[hyp] %s\n", text_buf);
    }
}

/* Flush the tail on stop. */
voxrt_asr_streaming_stop(sess, text_buf, sizeof text_buf, &written);
voxrt_asr_streaming_destroy(sess);

Emit frequency is ~1 line / ~1.12 s of audio (streaming chunk = 105 mel frames). Each returned text_buf is the incremental text emitted this call — not a rolling transcript. Concatenate them yourself if you want the full utterance.

Audio contract

• Sample rate: 16 000 Hz. No automatic resampling. Server / desktop / edge audio pipelines usually give you 44.1 or 48 kHz; convert down to 16 kHz Float32 mono before feeding voxrt_asr_streaming_push_audio. Feeding the wrong rate is the #1 source of "transcript is gibberish" bugs.
• Sample format: float PCM in [-1, 1], mono, native endian. If you have int16_t from any capture path (ALSA, PulseAudio, PortAudio, container-piped WAV), divide by 32768.0 as the wav-quickstart example does.
• Buffer size: any. The session internally accumulates to the steady-state chunk size (voxrt_asr_streaming_chunk_audio_samples() = 17 920 samples ≈ 1.12 s) and emits text every chunk boundary. 200 ms blocks (3200 samples) is the recommended default — matches the wer_streaming_replay cadence, so any perf numbers you take are directly comparable to the published ones.
• Latency: one chunk (~1.12 s) of inherent buffering. Output text becomes available chunk-by-chunk from push_audio's written text.

Threading

• The session is a synchronous, stateful function. It does NOT own a worker thread. Each voxrt_asr_streaming_push_audio call blocks on the calling thread for the duration of the inference work. On x86_64 servers this is ~5-8 ms per 200 ms block for CTC and ~10-15 ms for RNN-T at RTF < 0.1; on Pi 4B / 5 it's proportionally longer but still sub-realtime.
• One session instance is single-thread-at-a-time. Serialise push_audio / stop / reset against each other on a given handle.
• One session instance handles a stream of utterances. Between utterances, call voxrt_asr_streaming_reset(sess) to zero the K/V cache + LSTM state without paying the weight-load cost again.
• If you need concurrent transcriptions on the same host (multi-tenant server), create one session per stream — they don't share state and the ~150 MB heap is per-session.

Decoder selection

Recommended: RNN-T — higher accuracy, modest extra cost. This is the SDK-wide default (matches voxrt-asr-android + voxrt-asr-ios); pass VOXRT_ASR_DECODE_CTC only when you specifically want the cheaper head.

Both decoders run the same Conformer encoder; the head is selected at voxrt_asr_streaming_create time. TDT is not supported on streaming-medium-pc (no duration head) — passing VOXRT_ASR_DECODE_TDT fails session creation.

Project layout

voxrt-asr-linux/
├── README.md                    # this file
├── LICENSE                      # Apache-2.0 (wrapper sources)
├── LICENSE-BINARY               # proprietary (compiled runtime)
├── examples/
│   ├── c/
│   │   ├── wav-quickstart/            # WAV → streaming transcribe (+ RTF)
│   │   ├── wav-quickstart-cpp/        # C++17 RAII variant
│   │   ├── tcp-transcribe-server/     # PCM-in / text-out socket server
│   │   ├── tcp-transcribe-server-cpp/ # C++17 socket-server variant
│   │   └── cmake-consumer/            # find_package(VoxRTAsr) skeleton
│   ├── python/
│   │   ├── quickstart/                # WAV → streaming transcribe (Python)
│   │   └── tcp-transcribe-server/     # socket server (Python)
│   ├── nodejs/
│   │   ├── quickstart/                # WAV → streaming transcribe (Node.js)
│   │   └── tcp-transcribe-server/     # socket server (Node.js)
│   └── go/
│       ├── quickstart/                # WAV → streaming transcribe (Go)
│       └── tcp-transcribe-server/     # socket server (Go)
└── go/                          # Go module (cgo over C ABI)
    ├── go.mod
    ├── voxrt_asr.go
    ├── include/voxrt_asr.h
    ├── lib/aarch64-linux-gnu/libvoxrt_asr.so
    └── lib/x86_64-linux-gnu/libvoxrt_asr.so

The compiled libvoxrt_asr.so + voxrt_asr.h + pkg-config + CMake configs for C consumers live on the GitHub Release page for this tag, packaged inside the per-arch C tarballs. The Python wheels + npm tgz live on the same GH release. The Go module lives in-tree under go/ — fetched by go get.

License

• Wrapper sources (C headers, examples, docs): Apache-2.0.
• Compiled binaries (libvoxrt_asr.so, prebuilt tarball artefact): proprietary VoxRT runtime code owned by Elephant Enterprises LLC, redistributable as part of this unmodified SDK. See LICENSE-BINARY.
• Streaming ASR model weights (streaming_medium_pc.vxrt): derived from nvidia/stt_en_fastconformer_hybrid_medium_streaming_80ms_pc, released under CC-BY-4.0. Attribution travels with the model on the voxrt-asr-models repo.
• Commercial licensing for custom models, additional languages, on-premise fine-tuning, or bulk-device deployments: [email protected].

Links

• Brand + docs: voxrt.com
• Android counterpart: voxrt-asr-android
• iOS counterpart (Swift Package Manager): voxrt-asr-ios
• ASR model weights & versions: voxrt-asr-models
• VAD companion: voxrt-silero-android · voxrt-silero-ios
• Wake-word companion (Linux): voxrt-wake-word-linux
• Bugs / questions: <https://github.com/VoxRT/voxrt-asr-linux/issues>

Synced from voxrt-asr-linux on 2026-07-20 — the GitHub repo is the source of truth. Found a mismatch? The repo wins.