Why general models miss spoken Hebrew

"Did he say fifty-nine, or fifty-nine thousand?" A transcription model does not hear the way a person does. It picks the sequence of words that is most likely given the sounds — and what counts as likely depends entirely on what it was trained on. A model trained mostly on English learned Hebrew the way a tourist does: it follows a clean news reader, and fills in a noisy sales call with guesses.

How a transcription model guesses

Every speech recognition system works roughly the same way: it takes a stretch of audio and searches for the word sequence that best explains both the sound and the words already written. That is not a weakness — it is the method, and it is why transcripts read like language instead of like a phonetic soup.

The question is what taught it which sequences are likely. When the Hebrew audio is clear, almost any modern system does well. When it is not — a swallowed suffix, a name, a number, two people at once — the model falls back on its sense of what usually comes next. A model whose sense of "usually" was formed on English, with Hebrew as a small fraction of the mix, will confidently insert a common Hebrew word in place of the one that was actually said.

The error that costs you

A missing word is visible. You read the sentence, it does not parse, you check the audio. A substituted word is invisible: the sentence is grammatical, the meaning is plausible, and nothing signals that it is wrong. In a sales call it changes a price. In a legal conversation it changes who agreed to what. In a medical context it can change a dose.

This is why the headline accuracy number of a transcription system tells you less than the shape of its errors. Two systems with identical error rates are not equally useful if one of them mostly drops filler words and the other mostly swaps meaningful ones.

Where spoken Hebrew is hardest

• Swallowed endings. Spoken Hebrew compresses word endings that written Hebrew keeps. The distinction the model needs is often barely in the audio.
• Names. People and company names are the least predictable words in any language, and Hebrew name inventories are not well represented in general training data.
• Numbers. Hebrew numerals carry gender agreement, and the difference between two forms can be a single unstressed syllable.
• Code switching. A Hebrew sentence with three English words inside it is normal here and unusual in most training data.
• Two people at once. Universal, but it compounds everything above.

Read a news broadcast and every model looks good. Sit in a team meeting and the differences open up immediately.

What we did instead

We trained our own transcription model on spoken Hebrew, and we keep training it. Each new version is compared blind against the one currently serving customers, on the same recordings of real meetings, and it only takes over if it makes fewer errors. On the public eval-d1 benchmark published by ivrit.ai, our model is level with the best public system, by our own measurement. On our own meeting sets, the largest improvement in the most recent comparison was in the hardest audio — room echo, background noise, compression — which is to say, in real meetings rather than in clean readings. Most of that improvement was fewer substituted words: the error described above.

There is a fuller description of how the pieces fit together in how IvreetMeet is built.

And still, check the names and the numbers

Every automatic transcript has errors, and they concentrate in names and numbers. That is why transcripts here are editable, why a speaker gets named once and stays named, and why every line carries a timestamp back to the audio. For anything that matters, verify the passage against the recording — it takes seconds, and it is the difference between a working document and a liability.