Speaker diarization, explained
A transcript of five people talking, with no indication of who said what, is close to useless. Diarization is the part of the pipeline that answers "who spoke when" — a separate question from "what was said", solved separately, and with a legal dimension that the transcription half does not have.
Two different questions
Transcription asks: what words are in this audio? Diarization asks: how many distinct voices are here, and which stretch of time belongs to which? They run alongside each other and are merged at the end, which is why adding speaker labels does not double the wait.
How the system decides
- 1Find the speech. Silence and background noise are separated out, leaving the stretches where someone is actually talking.
- 2Describe each voice as numbers. Every short stretch is turned into a compact numerical description of how that voice sounds — pitch, timbre, the shape of the vocal tract. Two stretches from the same person land close together; two different people land apart.
- 3Cluster. The stretches are grouped by similarity. The number of groups is the number of speakers the system believes it heard.
- 4Merge with the words. The groups are aligned against the transcript's word timestamps, producing text split by speaker.
Why the labels start as numbers
Nothing in a recording says "this is Dana". The system can tell that the third voice is consistent throughout, not who it belongs to. You supply that once, and from then on the transcript, the summary, the action items and the search results all use the real name. It takes about fifteen seconds and it is the difference between a document people read and a document people skim.
Inside one meeting versus across meetings
This distinction matters more than it sounds, and it is the reason the product has two separate switches.
| Inside one meeting | Across meetings | |
|---|---|---|
| What it does | Marks where the voice changes and groups the segments | Recognises the same person in a later recording by their voice |
| What it stores | Nothing beyond that meeting | A stored voiceprint linked to a named person |
| Legal status | Not biometric identification | Biometric data under Israeli privacy law |
| Default | Always on | Off — the organization turns it on, and only consented speakers are matched |
| Retention | — | Unconsented voiceprints deleted after 30 days; unused ones after 365 |
Why we drew the line there
Where it goes wrong, and what to do
- Overlapping speech. When two people talk at once the audio contains both voices; every system has to choose. Expect errors exactly there.
- One microphone, far away. A laptop at the end of a table flattens the differences between voices. A phone in the middle of the table is often better than a laptop at the end of it.
- Echo. A hard room makes one voice arrive twice, slightly different. Soft furnishings genuinely help.
- Someone on speakerphone. A remote participant played through a room speaker is acoustically the room, not a person.
- Very similar voices. Two speakers with close pitch and delivery are the hardest honest case. Naming them once fixes the labels; the boundaries may still wobble.
The practical fix for all of these is the same and it is not software: get the microphone closer to the people. If you can record each participant on their own track, do that — it is the best input any transcription system will ever get.
What you can do with the result
Once a transcript is split by speaker, a set of things become possible that were not before: searching for what one person said across every meeting, seeing who actually spoke and for how long, and turning "he promised to send it" into an action item with a name attached. That is the point of the exercise — not the labels themselves, but everything they make possible afterwards.
Frequently asked questions
Does it know people’s names?
No. It detects that the voice changed and groups the segments; the labels start as Speaker 1, Speaker 2. You rename them once, and the names flow through the transcript and the summary.
How many speakers can it handle?
An ordinary meeting with several participants is routine. Accuracy falls as voices get more numerous, more similar and more overlapping — a crowded round table is genuinely hard.
Do I have to tell it how many people are in the room?
No. You are never asked how many people are speaking; that is inferred from the recording.
Is a voiceprint personal data?
When it is used to recognise a specific person, yes: that is biometric data. Separating voices inside a single meeting is not the same thing, which is why the two features have different defaults.
What ruins diarization?
People talking over each other, a single microphone far from the table, a hard-walled room, and one participant coming through another participant's laptop speakers — that last one makes two people sound like one.
Try it on your own recording
IvreetMeet transcribes and summarizes Hebrew meetings. The free plan covers 7 meetings and 120 minutes per organization each month.
Start free