AI & ML
What I learned building speech tech for a language with zero datasets
Servin Osmanov DEV Community
1 views
The word was qara, "black". I fed it to an off-the-shelf TTS model, hit play, and got back a soft, polite, very Turkish kara.
In Crimean Tatar къ and к are two separate sounds, and къ sits inside words we say every day: yoq, qız, vaqıt, qadar. Getting it wrong is not an accent. It is the model quietly speaking a neighbouring language and hoping nobody notices.
I noticed. That one syllable is why I spent the next several months building speech models for Crimean Tatar on a single consumer GPU, in the evenings, with no dataset to start from. What follows is the engineering residue: what I would tell a solo developer about to do the same for their own low-resource language. Most of it cost me a week per lesson.
The chicken and the egg were both endangered
The standard way to build a speech dataset is to take audio and run ASR over it to get text. Except we had no usable ASR for Crimean Tatar, and you cannot build good ASR without a corpus. Classic chicken and egg, except the chicken is also endangered.
I stared at that loop longer than I want to admit before noticing I was holding the problem the wrong way round. Audiobooks come with the text. It is printed. It exists. There is nothing to recognise.
So I stopped trying to transcribe and started trying to align: known text, audio, find where each word lands, cut on those boundaries. Forced alignment instead of recognition. That flip is the reason the project exists at all, because it sidesteps the exact thing the language did not have.
If you take one thing from this post, take that one. Before training a recogniser, check whether the ground truth is already sitting on a shelf. If it is, you have an alignment problem, and alignment is enormously cheaper.
The run that lied to me for three and a half hours
The first big alignment run covered seven and a half hours of audio. It ran, it did not error, and it produced clean, confident, well-formatted output.
The first two hours were correct. The remaining three and a half hours were garbage. Not obviously-broken garbage either: plausible timestamps, plausible segments, looking exactly like the good ones. Out of seven hours I got a bit under two of usable material, and I only found out because I went listening.
I have come to think of a crash as a gift. A crash points at the line. A process that returns confident nonsense takes a week off your life, slowly.
What fixed it was not a code change but a granularity change. Instead of feeding the aligner three enormous blocks, I split the material into forty chapters and located each independently. Same audio, same text, same tooling, and over five hours of clean speech instead of under two.
That is the rule I kept: if a stage can fail silently, never run it on big units. Small units fail visibly, and they let you bin the bad ones instead of the whole batch.
Words do not line up. Letters do.
That chapter-matching step had its own trap waiting. Locating a chapter inside a book sounds like a string search problem, and my first instinct was word-level similarity. It found nothing. I tested it on a pair I knew matched, checked by hand, and the similarity came back under one percent.
The reason will bite anyone working outside English. Crimean Tatar is agglutinative, so the "same" word carries different suffixes in the two sources. Add two alphabets in circulation (Cyrillic and Latin), and word tokens simply do not line up.
Switching the comparison to short character sequences fixed it in one go: 38 of 40 chapters located. The characters survive the suffixes. The words do not. For anything morphologically rich or multi-script, character-level matching is the default and word tokens are an English-shaped assumption.
The base model had already decided
Back to qara. My first assumption was the obvious one: the model had never heard the sound. Easy to test, so I checked. In my data the letter q showed up in 88% of recordings, more often than k. The model had been shown the correct pronunciation thousands of times and had cheerfully ignored every one.
The problem sat underneath the fine-tuning. In the base model, before it ever saw a word of my language, "q" was already wired to /k/ by English, Spanish and French. Fine-tuning was not overwriting that prior, it was decorating it. Which leads somewhere unpleasant: more data would not have helped. Not ten more hours, not a hundred. The fix had to happen at the foundation.
So I went base-model shopping with a different question. Not "which base speaks a language close to mine" but "which base already produces the sound I keep losing".
Everybody's first suggestion is Turkish, the big well-resourced relative. But Turkish has no /q/ at all, only one k, which is exactly how you get kara. I verified that twice, because I did not trust my own ears the first time. Azerbaijani looks promising until you listen: its q leans towards /g/, then softens further into something breathy. The languages that keep the sound, with a dedicated letter of its own, are Kazakh, Uyghur and Bashkir.
The multilingual base I ended up choosing had roughly 1500 hours of Kazakh, 400 of Uyghur and 250 of Bashkir in its training mix, against 125 hours of Turkish. Exactly upside down from what "pick the closest big language" would tell you.
It still had no Crimean Tatar in it. I got around that at the writing-system level: our къ, гъ and нъ each map onto a single letter the model already knows (ҡ, ғ, ң), so you hand it Crimean Tatar text and it reads something familiar. One trick, two problems.
The nice part is that this is verifiable before you spend a single GPU-hour on training. Out of the box, with no fine-tuning at all, the new base pronounced our къ correctly 88–100% of the time. I was no longer teaching the model a new sound. I had stopped fighting it.
I took the Kazakh and Bashkir variants all the way to the end, because I wanted the comparison rather than the argument, and they tied to the third decimal place. I shipped Bashkir: at a certain point in training the Kazakh one returned a few empty phrases and let the /q/ slide back towards k. Given equal numbers, I take the one that breaks less often.
For a language with no data of its own, picking the base is a phonetics decision, not a taxonomy one. And you can smoke-test the prior in an afternoon.
Three times the data, and the number would not move
My favourite negative result of the project. The corpus kept growing: 5.9 hours, then 11.5, then 15.3. I tracked character error rate by synthesising speech and running it back through a recogniser, averaged over three runs each:
corpus
CER (mean of 3)
base, no Crimean Tatar at all
0.173–0.198
5.9 h
0.136
11.5 h
0.138
15.3 h
0.141
Two and a half times the data and the number goes slightly worse. Meanwhile the run-to-run spread on one unchanged voice was wide enough to swallow that whole difference, so the honest reading is that all three runs are indistinguishable by this metric. And yet in blind listening, not knowing which sample was which, I picked the newer voice every time.
The corpus grew two and a half times over. The automatic metric flatlined. Blind listening kept preferring the newer voice anyway.
The explanation matters more than the numbers. Intelligibility saturates early, around six hours. Prosody, breathing, the liveliness of a sentence keep improving well past the point where CER stops moving. The metric goes quiet exactly where the remaining quality lives, so anyone reading that table alone would have stopped collecting data, reporting the truth while making the wrong call.
The cheaper form of the same lesson: one measurement is not a measurement. The same voice gave one number on the first run and a noticeably different one averaged over three. I have deleted several "wins" that turned out to be variance wearing a hat. Measure three times, and never give a single automatic metric the only vote.
The bug you can only hear if you click one sentence
I was listening to a narrated chapter, clicked on sentence 195 to check something, and heard the wrong sentence. Not slightly wrong, a different sentence. The drift was beautifully linear:
sentence
timestamp offset
1
0 s
49
+10.7 s
98
+21.8 s
147
+33.0 s
195
+43.9 s
My first theory was the fashionable one: the model falls apart over long text. Wrong, and worth checking rather than assuming. Speaker similarity to the opening held at 0.96–0.98 throughout, and the tempo never shifted. The audio was fine.
The actual cause: two parts of my pipeline computed time independently, and one counted the pauses between phrases twice. Forty-three seconds is exactly the sum of those double-counted pauses across the chapter. Correct audio, wrong labels.
I did not debug this by reading code. I debugged it with a knife: cut the audio at the old timestamps and at the new ones, push both through recognition, compare. New marks came back at CER 0.008, so the text matches. Old marks came back at 0.712, so those are other sentences entirely. On the final line the old timestamp had run past the end of the file, and the recogniser dutifully "heard" its own boilerplate captions in the silence.
The part that should worry you: played straight through, the file sounds perfect. Every consumer of those timestamps, a word-highlighting reader or a training pipeline, would have inherited garbage from a file that passes any listening test you can run. Whole-file playback validates whole-file playback and nothing else.
The exam the model had already seen
The ASR side of the project produced numbers I liked:
WER
CER
the model I started from
34.6%
11.9%
+ fine-tuning (about 1.5 h on one GPU)
20.1%
9.4%
+ tuning how the model decodes its answer
17.0%
7.0%
Left to right: starting model, after fine-tuning, after decoding changes.
The third row involved no training at all, not a single weight touched, only a more careful decoding setup, and it moved WER by three points. The entire adapter I added to that model is 126 MB.
Now the part that nearly ruined it. The same four audiobooks existed twice in my material. Cut differently, segmented differently, stored under unrelated filenames, zero overlap by name. For one book I had picked as an eval set, 96.9% of the segments had a twin sitting in the training data.
If I had missed it, I would have been examining the model on questions it had already been handed the answers to. A lovely number, published, believed, with the next six months built on top.
Filenames are not identity. I check for near-duplicates by content now, always, and I treat a suspiciously good eval score as a bug report. Same reflex later, when a synthesised voice read a whole book: I proved by measurement, not by reasoning, that the book's text never entered training. Zero word-sequence matches against a corpus of nearly one and a half million words. Otherwise it is not reading, it is reciting.
What it costs now
A synthesised voice read an entire book, Merdiven by Şamil Alâdin. Not a demo paragraph. All sixteen chapters, 15,444 words, one hour fifty-eight minutes of audio, word timings verified across 100% of the words, per-chapter CER median 0.022 (worst 0.027, best 0.015).
The timings got a control that was required to fail: cut 30 words per chapter at the recorded marks and recognise them, 447 of 480 matched; shift the same fragments by 0.4 seconds and 4 of 480 match. That gap is what makes the check worth running.
And the number I care about most: roughly three hours of machine time per two hours of finished audio, everything included. An audiobook in Crimean Tatar now costs one night of one home computer. Not a studio, not a year of waiting.
Earlier work is public if you want a starting point: the dataset and the model.
Where the months actually went
Training was hours. An hour and a half here, a few hours there. It is not the line item anybody thinks it is.
The months went into collecting and aligning data, repairing the parts that broke without saying anything, proving the evaluation was honest, re-measuring, and listening with headphones, blind, one sentence at a time.
That last one produced the finding I did not expect. Across the project the bugs split roughly half and half between instruments and a human ear. Instruments caught the drifting timestamps, which no amount of listening would ever have revealed. The ear caught breaths in wrong places, bad annotations, and this one:
I heard ниже where the book said нидже. The cause took a minute to find. My letter-mapping rule, the same trick that gave me ҡ and ғ, was also collapsing дж into ж. The model had been handed the string "ниже" and pronounced it flawlessly. Perfect execution of a wrong input.
The rule had erased a distinction the language genuinely has: дж in our own words like джан, оджа, нидже, versus ж in loanwords. After that collapse nothing downstream could have said them differently. That chapter held fifty-five such words. I caught exactly one, the case that happened to land on another real word. The other fifty-four sounded "a bit off", and the ear slides right over "a bit off".
Fixing it barely moved any metric. I fixed it anyway, because it is our language, not a leaderboard.
Which is the most useful thing I can hand anyone doing this for their own language: your cheapest and strongest tool is one native speaker, fifteen minutes and headphones. It beats the automation, and it means you do not have to be a programmer to contribute. You need to hear your language well.
First post in a series. Coming up: teaching a recogniser to hear Crimean Tatar; narrating a book chapter by chapter; running blind A/B tests so you stop fooling yourself; and the one where I set money on fire renting somebody else's GPUs and computed almost nothing.
Voice samples and project page: https://ai.ana-yurt.dev/blog/2026-09-kak-rodilas-ideya/.
Two ways to help. If you're a Crimean Tatar speaker, the highest-value contribution is fifteen minutes with headphones — blind listening and proofreading beat any tool in my stack; write to support@ana-yurt.com. If you'd rather fuel the work directly, there's ko-fi.com/anayurt.
Read original: https://dev.to/servin_osmanov/what-i-learned-building-speech-tech-for-a-language-with-zero-datasets-2gka
← Previous
A Practical Checklist for an Agent-Ready API
Next →
plcmt=5 With playbackmethod=1 Validates. Those Two Integers Name Different Formats.
Related
Inside the SFPU: How a 40-Year-Old Rounding Trick Breaks on Modern AI Accelerators
AI & ML
1
DEV Community
We built an AI security layer and kept AI out of the decision
AI & ML
1
DEV Community
Address the row, not the line: when the checker is the first reader in disguise
AI & ML
5
Dev.to (EN Zone)
‘It Answers in Poetry Now’: A Week of Users Saying Their AI Got Wordier and Worse
AI & ML
2
Dev.to (EN Zone)
Comments0
No comments yet — be the first