[● POC]
R&D prototype · GCP
Streaming Multilingual Voice Agent
A low-latency, fully GCP-native voice agent that understands and replies in 10+ languages — under 1.5 seconds end-to-end.
// overview
A real-time voice agent built entirely on Google Cloud. Audio streams in over WebSocket, Chirp transcribes it with automatic language detection, Gemini resolves intent and drafts a reply, and Journey speaks it back — with response generation starting on interim transcripts to shave perceived latency below 1.5s.
// the problem
Multilingual voice support usually means stitching together third-party STT/TTS with high latency and per-vendor cost. The goal was a single-cloud stack that handled mid-conversation language switching natively and felt instant.
// architecture
01 · stream
Audio chunks
WebSocket
02 · stt
Chirp STT v2
auto-detect lang
03 · reason
Gemini Flash
intent + reply
04 · speak
Journey TTS
natural voice
05 · scale
Cloud Run
Pub/Sub + Firestore
// side rail
Interim streaming
speculative
↳ generation starts on interim transcripts, before the final one lands
// what i built
- ✓Streaming pipeline: audio → Cloud Run → Chirp STT v2 with automatic language detection across 10+ languages.
- ✓Gemini Flash for intent resolution and response generation, kicked off on interim transcripts to cut perceived latency.
- ✓Cloud Text-to-Speech (Journey) for natural-sounding replies streamed back to the client.
- ✓Pub/Sub decoupling audio ingestion from processing; Cloud Run autoscaling per session with state in Firestore.
- ✓Native mid-conversation language switching via Chirp — no explicit language pre-selection needed.
- ✓Designed for multilingual customer support and enterprise helpdesk use cases with no third-party STT/TTS.
// impact
<1.5s
end-to-end latency
0+
languages
0
third-party STT/TTS
// tech stack
GCP Speech-to-Text (Chirp)Vertex AI GeminiCloud Text-to-SpeechCloud RunPub/SubFirestorePythonWebSocket
next project
NL-to-SQL Facility Intelligence Platform