Voice pipeline · inputs, outputs, and control

One local session · one GPU STT worker · one isolated GPU TTS worker · streaming cloud conversation

Voca animated voice pipelineMicrophone audio passes through VAD and rolling-window Whisper transcription. Final text and local history go to Ollama Cloud. Streamed tokens enter a bounded phrase buffer, CPU Kokoro synthesis, and ordered browser playback. Completed playback updates local history. New speech cancels response tasks and stops playback. 01 / LISTEN + UNDERSTAND · LOCAL COMPUTER02 / STREAM + SPEAK · LOCAL COMPUTER OLLAMA CLOUD · ONLY REMOTE STEP BROWSER / AUDIOWORKLETMicrophoneIN: your English speechOUT: 512 float32 samples / 32 msINOUT CPU / VAD + ENDPOINTSilero VADIN: 16 kHz mono PCM framesOUT: speech turn after ~704 msINOUT GPU / 1 SPAWNED WORKERWhisper small.enIN: rolling window or full turnOUT: preview / final English textINOUT LOCAL + OPTIONAL CLOUD CHECKIntent + historyIN: raw text + supplied meaningsOUT: validated text + historyINOUT GEMMA 4 31BStreaming LLMIN: text messages · thinking offOUT: streamed English text tokensINOUT LOCAL / ASYNC TASKPhrase buffer + queueIN: tokens · capacity 32OUT: 40–180 chars · queue 3INOUT GPU / ISOLATED VOICE RUNTIMEKokoro 82MIN: text phrase · af_heartOUT: 24 kHz PCM16 audioINOUT BROWSER / WEBAUDIOOrdered playbackIN: audio chunks · up to 3 pendingOUT: speech + played ACKINOUT PCM frames speech audio raw final HTTPS: interpreted text + bounded history (12 turns) text tokens phrases PCM16 audio preview + final → UI only streamed text → UI played complete → append heard assistant phrase to local history New speech → cancel response tasks · clear pending playback · stop speaker cancel HTTP cancel queues discard stale result BROWSER TRANSCRIPT UIPreviews + final text + assistant tokensExplicit clarifications: optional cloud word check; raw transcript stays visible. Unresolved meanings ask for spelling.VAD: threshold 0.5 · speech start 256 ms · silence 700 ms (704 ms in 32 ms frames) · max turn 30 sWhisper: 10 s previews, beam 1 / 48 tokens; full-turn final decode uses beam 3. Running TTS results are discarded on interruption.
Forward dataInterruption control

Click an entity to inspect it. The animation illustrates dependencies and concurrent streaming, not measured latency. Only text messages leave this computer; audio processing stays local.