Streaming support for Open Source Models

Short Course Q&A Building Live Voice Agents with Google’s ADK
1

Hi everyone,

I’m working with ADK streaming agents using Gemini Live models successfully, but I need to enable streaming for open-source models beyond Gemini.

Issue: I can run open-source models for general inference, but can’t configure them to interface with ADK in streaming mode.

Looking for:

  • Documentation on streaming interfaces for open-source LLMs in ADK

  • Configuration examples

  • Any community experiences

Haven’t found any resources on this specific integration. Has anyone done this successfully?

Thanks!

2

@robinrob55 Good question

I try with NVIDIA hosted model nvidia_nim/nvidia/llama-3.1-nemotron-nano-8b-v1

image
image1912×756 49.4 KB

Yes voice streaming is not working due to web socket connection error but, SSE is working for other model also

image
image1853×777 67.3 KB

1 Like
3

Thanks for sharing your experience. My focus is on multimodal support (video and audio included), but I think implementing it with an ADK is complicated for now outside of the live API.