Managing One-to-One Voice Communication in a WebRTC LiveKit Conversational UI

Short Course Q&A Building AI Voice Agents for Production
1

If I build a conversational user interface using WebRTC with LiveKit:

  1. The agent server runs as a service and joins a room.
  2. The client also joins the same room to receive audio.
  3. The agent server and LiveKit server communicate peer-to-peer.
  4. The client and LiveKit server also communicate peer-to-peer.
  5. Audio is transmitted using WebRTC.

When another client joins the room to listen to the agent server’s audio, all participants in the room will receive and hear the audio.

However, we want each client to have a separate voice conversation with the agent server.