Same weights, 21–23% lower WER across CUDA, MLX and CPU
Runtime engineering across NVIDIA NeMo, Apple MLX and CPU—without retraining the model.
Word error rate (WER) fell 21–23% versus our earlier runtime results. We pinned a better CUDA recipe, fixed feature parity on MLX and made long-audio cuts safer on CPU.
The result
We expected the checkpoint to set the quality ceiling. It didn’t. Feeding and decoding the same weights more carefully improved every platform.
Earlier rows used older runners. The current results are reproducible.
CUDA is now our choice for the best WER and throughput, MLX q8 for the best canonical M-WER on Apple Silicon, and CPU as the portable fallback.
No dictionary, biasing or transcript cleanup. The MIT runtime is on GitHub, the CC-BY-4.0 weights are on Hugging Face, and the package is on PyPI.
Accuracy
| Runtime | Hardware | WER | M-WER | Drug M-WER |
|---|---|---|---|---|
| NVIDIA NeMo BF16 | L4 | 6.54% | 2.23% | 4.75% |
| Apple MLX q8 | M4 Max | 6.65% | 2.12% | 4.52% |
| CPU GGUF q8_0 | Linux / Windows CPU | 7.10% | 2.16% | 4.30% |
On this benchmark, CUDA and MLX beat every open model we tested, plus the tested configurations of OpenAI GPT-Transcribe, Google Gemini 3.1 Pro Preview and AssemblyAI Universal-3.5 Pro Medical.
Speed
Times faster than audio duration. Hardware-specific.
How we tested
We tried ideas on 112 non-board clips, stress-tested long and sequential audio, then ran the final recipes once on the 1,513-clip benchmark.
NVIDIA: pin the whole recipe
NeMo could load the checkpoint, but its defaults were not our best recipe. The adapter now pins precision, attention, decoding, batching and audio normalization.
BF16Faster than FP32 on the full board, with a slightly better WER point estimate.
attention = [256,256]Bounded context that keeps arbitrary recording lengths practical.
greedy_batch · max_symbols=10The selected TDT decoder. Beam size two was slower and less accurate.
batch ≤ 8 · audio ≤ 900 sDuration-sorted work with caller order restored after decoding.
PCM16 · mono · 16 kHzOne FFmpeg normalization contract for every supported input format.
whole-record transcribeThe normal command no longer inserts a boundary after 240 seconds.
The selected L4 run scored 6.54% WER and 2.23% M-WER. Short and long slices ran at 216× and 201× realtime respectively.
Apple Silicon: fix the input, then bound memory
The biggest Apple fix happened before the model. MLX placed the Hann window at the edge of the FFT buffer; the training and GPU paths center it. We made them match.
centered Hann windowRestores model-input parity instead of trying to repair words after decoding.
local attention [256,256]Keeps long recordings bounded. Full attention gained only 0.03 WER points.
mx.clear_cache()Clears allocator state between files and prevents sequential long-run collapse.
U+2047 unknown tokenMatches the reference runtime’s visible output and scoring contract.
Clearing the MLX cache stopped sequential long-file failures. Local attention then ran 23% faster and cut the longest-recording allocation high-water by about 71%, while moving WER only from 6.62% to 6.65%.
Why q8 stayed the Mac default
The 0.94 GB q8 weights matched the 2.50 GB unquantized export on the development gate, so the larger export did not earn a full run.
CPU: make necessary cuts less harmful
CPU handles recordings up to 240 seconds whole. For longer audio, it uses overlapping 180-second chunks and moves each cut to the quietest nearby 300-millisecond window. CUDA and MLX transcribe the recording whole.
Silence-aware cuts moved full-board WER from 7.17% to 7.10% and reduced missed medical and drug mentions, with no measurable speed cost.
parakeet.cpp still builds the full attention matrix. The next meaningful CPU gain needs real windowed attention in C++, not another flag.
Run the qualified runtime
The CLI applies the complete NVIDIA recipe automatically.
# NeMo 3.0 is pinned by the package extra
pip install -U "omi-med-stt[nemo]"
omi-med-stt recording.m4a --runtime nemoThe same adapter works inside Python.
from omi_stt.nemo_runtime import transcribe_nemo
text = transcribe_nemo(
["recording.wav"],
repo_id="omi-health/omi-med-stt-v1",
)[0]
print(text)Apple Silicon and CPU use the same package.
# Apple Silicon
pip install -U "omi-med-stt[mlx]"
omi-med-stt recording.wav --runtime mlx
# Linux / Windows CPU
pip install -U omi-med-stt
omi-med-stt install-cpp --cpp-backend cpu
omi-med-stt recording.wav --runtime cpp- NVIDIA GPU recipe: exact configuration and release checks.
- Apple MLX q8 recipe: frontend, memory and cache measurements.
- CPU GGUF recipe: silence-aware chunking and statistical caveat.
What we tested
- 1,513 prerecorded English medical clips—not streaming or multilingual audio.
- An NVIDIA L4, an Apple M4 Max with 64 GB memory, and Linux / Windows CPU hosts.
- Speech-to-text accuracy, not clinical validity. Transcripts still require review.
Inspect the runtime.
Run it or inspect the settings in source.