📝 Originally published (in Japanese) at forge.workstyle.tech. This is a record of designing voices from single-line captions, automatically creating a learning corpus, and passing all 12 role-specific voices (narrator/counselor/sales/presenter/operator/MC for both men and women) through full inspection. I wrote about the failures I encountered during approximately one month of actual work, divided into 18 articles. This article is the table of contents. The Conclusion Upfront Voice design, voice manufacturing, and voice operation are different technologies with different failures. Design uses diffusion TTS. The voice is determined by the caption and random seed, making it fully reproducible. Manufacturing is primarily about corpus generation. The design of the quality gate directly determines the voice quality. Operation relies on lightweight pre-trained models. Diffusion TTS is too slow for conversation (2.5 times slower on the same GPU). The biggest lesson boils down to one point: Having a quality gate and it being effective are two different things. Six of these 18 articles are about gates that existed but weren’t effective. Reading Order The articles are arranged in the order of design → manufacturing → inspection → operation. Reading from the top will take you through the journey of a single voice being created and deployed into production. Chapter 1: Design — How to Determine the Voice The TTS Chosen for Sound Quality Was Too Slow for Conversation A 2.5x real-time factor (RTF) difference. How we settled on a two-stage approach: designing voices with diffusion TTS and using pre-trained models for speech. Drawing Voices Like a Gacha Voices are determined by captions and random seeds. By keeping a ledger of design values, voices can be recreated even if the model is lost. Letting a Machine Choose "Narrator-like Voices" from 24 Candidates Listening to all candidates is unsustainable. Automatically measure speech rate, intonation, and stability to only listen to the top candidates. Also, the story of how all male candidates were eliminated due to metric limitations. Chapter 2: Manufacturing — Corpus Quality Directly Becomes the Voice The Stricter the Quality Gate, the More Monotonous the Takes Survive The reason all emotion-infused corpora ended up monotonous was the quality gate itself. This is the central story of this series. Speech Rate Cannot Be Changed After Training Endings and speech rate are baked into the corpus. They cannot be adjusted with synthesis parameters. TTS That Changes "Recording Location" Every Time Even with the same model and speaker, frequency characteristics differ for each clip. If not standardized, style switching results in inconsistent sound quality. One Rough Clip Ruins the Entire Style If one out of five clips is rough, the entire style becomes hoarse. Dilution into the average doesn’t help. Chapter 3: Inspection — Having a Gate and It Being Effective Are Different Where Did the AI’s Habit of Stretching "Hello" Come From? Verification discarded long vowels due to kana normalization, making the defect fundamentally undetectable. "A Little" Becomes "Shomo" The model and parameters were fine, but the input text was broken. The symbol removal list was dropping characters like "々", "〆", and "髙". The Hallucination Guard Code Only Failed During Hallucinations A single-line bug made the guard ineffective only in necessary situations. The "Three Characters" Allowed by the Quality Gate Became a Verbal Tic The size passing through the gate matched the size of the sound reproduced by the model. Rejecting Candidates for Fixable Defects Is the metric measuring the product’s characteristics or the process’s state? Defects Unseen in Transcription STT-only inspection misses 0.1-second additive sounds with silence in between. Capture them with waveform envelopes. Chapter 4: Operation — Running as a Factory 70 Minutes of Training Material Lost to a Network Blink One disconnection led to 205 retries. Finding areas without retries and designing idempotent resumable processes. From "ja" to "JP": Creating a Babbling Model Language code case sensitivity led to a completed babbling model. A catalog of pitfalls encountered. Four Registration Paths, Zero Management Screens Asset catalogs became unmanageable. Only after creating an inventory API could we see the full picture. Each Deployment Overwrote the Other’s Work Two parallel workflows alternately deleted each other’s changes from production. Branch checks didn’t prevent it; only matching sha256 with running Pods stopped it. Chasing Unmeasured Targets with Thresholds Always Leads to Failure Interpreting zero metrics as "no issues" and tightening thresholds. Not realizing what wasn’t being observed until the end. Three Articles for Those Short on Time Interested in quality gate design? → Article 4 (selection bias) → Article 11 (three characters) → Article 13 (defects unseen by STT). These three cover all patterns of "gates existing but not being effective." Planning to train a speech model? → Article 15 (pitfall catalog) → Article 9 (broken input) → Article 5 (style baked into the corpus). Reading as an MLOps/manufacturing story? → Article 12 (process defects vs. product traits) → Article 14 (idempotent resume) → Article 18 (not chasing unmeasured targets). Common Thread Across These 18 Articles Most failures occurred not in models or GPUs, but in inspection design. It wasn’t that gates were too loose; it was that gates weren’t measuring what mattered, and those unmeasured aspects got baked into the product. Unlike text, with audio, defects that slip through aren’t noticed until heard. What ultimately worked wasn’t finding smarter models, but systematically addressing what wasn’t being measured. The work notes that formed the basis of these insights are compiled in Manufacturing Pipeline for Practical Voices from Diffusion TTS.