We wrote once about a transcript that came back nearly empty. A user uploaded a recording, the system guessed the wrong language, and the file finished with almost no text in it. Nothing failed. No error was raised, because from the software's point of view nothing had gone wrong. We built an arithmetic check in response: count the words, compare them against the length of the audio, and refuse to call a job healthy when the two do not agree.
Last week a file walked straight through that check and it failed without giving an error, so we went back, measured what went wrong, and changed the check.
The recording
Nine minutes of two people talking about Montessori teaching. The conversation runs mostly in Tunisian Derja, the everyday Arabic spoken in Tunisia, with French threaded through it the way it is in most Tunisian conversation. It arrived as an ordinary upload, in the ordinary way.
It came back with 317 words.
Two people talking for nine minutes should produce somewhere over a thousand. The transcript that was stored read perfectly well. Punctuation in the right places, sentences that made sense, nothing that looked broken. It was simply missing most of the conversation, and there was nothing on the page to say so.
Our check looked at that and passed it, because the bar we had set was 24 words per minute and the file came in at 35.
Four settings, one file
We ran the same recording four times through the same engine and changed one thing: which language we told it to expect.
Told to expect English, it returned twenty words, most of them "Mhmm". Told to expect Spanish, it returned an empty file.
Told to expect multiple languages, the setting built for recordings that switch between them, it returned 318 words and covered 41 percent of the audio. That is the run that produced the transcript we stored, and it left six minutes of the recording blank. It is also the setting most people would reasonably choose for a file like this one.
Told to expect Tunisian Arabic, it returned 781 words and covered 90 percent.
Where the missing audio actually was
Word counts tell you how much text came back. They do not tell you which parts of the recording produced it, so we cut the nine minutes into five second blocks and asked, for each block, whether any setting had heard anything in it.
The multiple languages setting heard something in 44 blocks out of 108. Tunisian Arabic heard something in 97. Between them, Tunisian Arabic recovered 300 seconds that the other setting had left blank, the other setting recovered 35 seconds that Tunisian Arabic had left blank, and 20 seconds produced nothing under either.
Five full minutes of that conversation existed in one configuration and not in the other. Same file, same engine, same speakers, same minute of speech.
The failure is silence, not error
This is the part worth carrying away, and it is not what the word accuracy suggests.
When the language is wrong, the engine does not write down the wrong words. It writes down nothing. You do not get a transcript full of visible mistakes you can go and correct. You get a shorter transcript that reads correctly, with whole minutes absent and no mark where they used to be.
An ordinary mistake shows itself on the page, in a word you can see is wrong. This one leaves nothing behind at all. If you were not in the room when the recording was made, there is nothing in the output that would tell you a third of the conversation is gone.
The engine guesses, and does not say it is guessing
The multiple languages run labels each word with the language it believes it heard. On this file the tally came out as French, then Portuguese, then Spanish, then English, then German.
There is no Portuguese in that recording. There is no German.
Handed sounds it could not place, the model did not decline to answer. It assigned them to the nearest language it had and reported the result with nothing attached to say it was unsure. A confidence score would not have saved us either, because the words it did return, it returned confidently.
One word with two correct answers
In the Arabic transcript there is a word written سيكاونس.
That is séquence. A French word, spoken in French, written in Arabic script, because an engine listening for Tunisian Arabic heard Tunisian sounds and wrote them the way a Tunisian writes them.
Call that an error and you are wrong. Call it correct and you are also wrong, depending on whether the person reading the transcript needed the French kept as French. It is one word scoring differently against two reasonable definitions of right, in a recording where both languages are the point.
That is a large part of why we do not publish an accuracy percentage. The number needs a definition of correct that this recording does not have.
What we changed
The bar moved from 24 words per minute to 60. Conversational speech runs at 120 to 150. The old figure was low enough that a file missing most of its audio passed as healthy, which is exactly what happened here. Sixty is still under half of slow speech, so a genuinely quiet recording does not get dragged into a needless retry.
The retry now runs whatever the user picked, not only when the language was detected automatically. Previously the safety net engaged only when nobody had named a language, on the reasoning that an explicit choice deserved to be respected. That reasoning was wrong. An explicit choice can be mistaken, and the person who picked it is the least likely to suspect it.
We had to fix a floor while raising the bar. Rounding every clip up to a whole minute made an ordinary thirty second voice note at eighty words per minute look sparse and buy itself a second transcription it did not need. It now floors at a quarter minute.
And we removed the multiple languages option from the spoken language list entirely. We tested it again on an unrelated file, a Thai narration, where it covered 19 percent of the audio while naming the language covered 100. Twice tested, twice last, on exactly the kind of recording it was named after. Leaving it in the list invited people to choose the wrong option for their own file, so we took it out.
What this does not prove
Two recordings. One engine. This is a measurement, not a benchmark, and we are not presenting it as one.
It also does not flatter us. The multiple languages mode was our own feature, and on this file it was the worst of the three settings that returned anything at all. The check that should have caught the problem was ours, and we had set it too loose.
What we are confident of is the shape of the failure rather than the size of it, because the shape follows from how these systems work rather than from which vendor built them. A model listening for one language does not transcribe another badly. It passes over it.
Three things worth doing with your own recordings
Compare the word count against the duration before you read the text. Nine minutes of conversation should exceed a thousand words. It is the fastest way to tell whether a transcript is whole, and almost nobody checks it, because the transcript reads fine.
Treat a gap as a symptom. If the text skips a stretch you remember as conversation, change the language setting before you blame the microphone.
Remember that a dialect is not an accent. We had assumed Arabic was Arabic. Tunisian Derja differs from Modern Standard Arabic considerably more than any two English accents differ from one another, and naming it produced two and a half times the words.