Speaker diarization answers “who spoke when?” It usually produces anonymous, consistent labels such as Speaker 1 and Speaker 2. It is separate from speech recognition, which answers “what was said,” and from speaker identification, which tries to establish a person's identity.
What is speaker diarization?
A diarization system divides an audio recording or stream into speaker-attributed segments. The useful output is not only a transcript. It is a sequence of turns:
The product can later map “Speaker 1” to “clinician” and “Speaker 2” to “patient” when it has reliable workflow context. The diarization label itself does not prove identity.
Why speaker attribution matters in healthcare
A medical transcript can contain the right words and still produce the wrong downstream record if the speaker is wrong. “I stopped taking metformin” has a different meaning when spoken by a patient, a caregiver quoting the patient or a clinician confirming a history.
Speaker turns help products build role-aware notes, preserve questions and answers, distinguish instructions from reported symptoms, and let reviewers return to the relevant moment in the audio. They also create a visible failure surface: a clean paragraph can hide that two people were merged.
Why clinical conversations are difficult
- Short replies: “yes,” “right” and “okay” can be absorbed into another speaker's turn.
- Overlap: interruptions and simultaneous speech do not fit a simple one-speaker timeline.
- Extra people: caregivers, interpreters and staff can appear mid-conversation.
- Changing acoustics: movement, masks and room noise change the speaker signal.
- Anonymous labels: Speaker 1 does not inherently mean clinician or patient.
Live and batch diarization are different product contracts
Batch processing can use the complete recording before deciding speaker boundaries. Live processing has to produce useful output while the conversation is still happening. Labels or boundaries may therefore be provisional until more context arrives.
When evaluating a live API, ask what remains stable across draft text, confirmed text and the final transcript. Test whether a label can change, whether the product sends corrections, and how your interface or downstream agent applies those corrections. A live transcript that looks fast but cannot reconcile revised speaker turns can create silent state errors.
For batch jobs, inspect how the system handles long silences, people leaving and returning, and the same speaker being split into several labels.
How diarization is measured
Diarization error rate, or DER, is a common summary. It combines missed speech, false speech and speaker confusion relative to reference speaker time. But one DER can hide the failure your product cares about.
- Missed speech: reference speech was not assigned.
- False alarm: non-speech was treated as speech.
- Speaker confusion: speech was assigned to the wrong reference speaker.
For a healthcare product, add task-level measures: how often clinician/patient turns are swapped, whether medication statements are attributed correctly, and whether a note or agent changes behavior because of a speaker error.
The current public medical benchmark measures words, medical terms, drug names and dosages. It does not measure speaker attribution. Do not infer diarization accuracy from Omi's WER or M-WER result.
A practical clinical diarization test plan
- Define the roles and output. Decide whether anonymous consistency is enough or the product needs clinician/patient role mapping.
- Collect representative rooms. Include the microphones, distances, specialties and speaker counts the product will actually encounter.
- Include difficult turns deliberately. Mark overlaps, short acknowledgements, interruptions, quiet voices and people entering or leaving.
- Create a time-aligned reference. A reviewer should label who spoke and when, including the policy for overlap.
- Score batch and live separately. Do not transfer a batch result to a live product contract.
- Measure label stability. For live systems, record revisions and whether downstream state is corrected.
- Inspect role-critical statements. Review symptoms, denials, drug names, dosages and instructions by speaker.
- Test failure handling. Decide what the product does when speaker capacity is unavailable or confidence is inadequate. Silent removal of labels is not acceptable.
Test the actual room
Omi includes speaker labels and timestamps on every hosted plan, but the public benchmark does not score diarization. Use the playground for an initial check, then test the people, microphones and live corrections your product depends on.