We shipped the first three Lina pilots on the GPT-4o transcription API. It was the obvious default in late 2025. Three months in we ripped it out and replaced it with Whisper large-v3 on the Hetzner GPU box. Here is what actually broke, what the numbers looked like, and why "obvious default" lost.
The original transcription path was: Telnyx audio in, 20 ms chunks streamed over WebSocket to the OpenAI realtime transcription endpoint, partials surfaced as they arrived, finals committed to the conversation log. End to end this was thirty lines of code. The German word error rate on test calls was about 4.5% headline. The medical vocabulary held up reasonably well. There was no diarisation to write, no model to host, no GPU to feed. For the first three pilots this was the entire point.
The cost was around 0.006 EUR per minute of audio. For a clinic averaging 90 minutes of call time a day that landed around 16 EUR per month per practice on transcription alone. Defensible against the 199 EUR founding price we charged at the time (today: 449 EUR).
Three things broke at roughly the same time, none of them catastrophic, all of them compounding.
The latency tail got worse, not better. Median first-token latency held at about 280 ms, which is fine. The p95 drifted from 600 ms in November to 1.4 seconds by mid-February with no public incident from OpenAI. A p95 over a second on a live phone call is audible silence. Callers start repeating themselves. That repetition then gets transcribed and confuses downstream logic.
Pricing changed once and we re-modelled the math. Not a huge jump, but enough that the unit economics on a scaled-out Essential tier stopped being obviously net-positive. We were now sensitive to a vendor we did not control.
The DSGVO conversation got harder, not easier. Clinic data protection officers asked two questions we struggled to answer cleanly. Where exactly does the audio land for processing. What is the retention guarantee. OpenAI's enterprise terms cover both, but for a one-engineer studio routing audio for a German Facharzt through a US-headquartered provider, the conversation took 90 minutes instead of 15. Multiply that across a sales pipeline.
Whisper large-v3 on a single H100 on the Hetzner Falkenstein box. NeMo for streaming partials with a 200 ms chunk size. The same WebSocket interface to the rest of the stack so the downstream call graph did not change. Audio never leaves Germany. The retention answer is now "we delete the transcript at call end unless the practice opted into 30-day storage" and we can point at the actual delete query.
The numbers we measured side by side over a two-week shadow run, same calls replayed against both pipelines:
It is not. We reversed because three things lined up: we already owned the hardware for unrelated reasons (Lucid's model training), the buyer was a German medical practice for whom data residency is not a feature but a contractual fact, and the call shape is narrow enough that fine-tuning the decoder bias on a glossary actually pays for itself.
For a US ecommerce chatbot serving English calls under no medical-secrecy regime, GPT-4o transcription is still the right default. The decision is not "which technology is better". It is "which path makes the next twelve conversations easier".
Convenience defaults are right until your domain catches up to you. The mistake was not shipping GPT-4o; the mistake was leaving it in place after the DSGVO conversations had told us twice it was costing us deals. The signal was loud and we waited for it to get louder.
The standing rule from this reversal: if the same vendor question comes up in three procurement conversations in a row, the next architecture decision should remove the question, not answer it faster.
Reply: [email protected]. Real replies. No comments section. Back to all essays.