Even LLMs like Gemma 4 used by voice agents are not supported.
Then I figured that the inference platform needs to be optimized differently for the kind of usecase you are using. Lets take an example for LLMs, not even STT and TTS. - Coding agents -> lot of cached input, needs to optimize for KV cache - Creation slides/blogs -> lots of output, needs to optimize for speculative decoding - Voice LLMs -> Cached input small output, not yet figured out on how to optimize this.
So TTS and STT is a completely different ballgame.
What I don't know is the timing, do people want to use open source models like kokoro, parakeet, Qwen etc RIGHT NOW?
0 comments