Gemini Live Api DevSAFE

Skills for the Gemini API, SDK and model/agent interactions

The instruction file, verbatim from the audited commit — this is the text the model reads, and the surface the audit's instruction layer examines. Quoted here so you can judge it without cloning anything.

---
name: gemini-live-api-dev
description: Use this skill when building real-time, bidirectional streaming applications with the Gemini Live API, or migrating legacy Live models (2.0/2.5/3.1) to Gemini 3.8 Live. Covers WebSocket-based audio/video/text streaming, voice activity detection (VAD), background reasoning (extended thinking), asynchronous function calling, session management, ephemeral tokens, live transcription, and live translation. SDKs covered - google-genai (Python), @google/genai (JavaScript/TypeScript).
---

# Gemini Live API Development Skill

## Overview

The Live API enables **low-latency, real-time voice and video interactions** with Gemini over WebSockets. It processes continuous streams of audio, video, or text to deliver immediate, human-like spoken responses and background reasoning.

Key capabilities:
- **Bidirectional audio streaming** — real-time mic-to-speaker conversations
- **Background reasoning (extended thinking)** — multi-step background reasoning with spoken conversational fillers
- **Live streaming transcription** — real-time speech-to-text with interim and finalized streams
- **Video streaming** — send camera/screen frames alongside audio
- **Text input/output** — send and receive text within a live session
- **Audio transcriptions** — get text transcripts of both input and output audio
- **Voice Activity Detection (VAD)** — automatic server VAD, client-side Hybrid VAD, and manual Push-to-Talk
- **Asynchronous function calling** — non-blocking tool execution while audio continues streaming
- **Full-session client content** — inject and update conversation turns mid-stream
- **Session management** — context compression, session resumption, GoAway signals
- **Ephemeral tokens** — secure client-side authentication

> [!NOTE]
> The Live API connects directly via **WebSockets**. For WebRTC support or simplified integration, use a [partner integration](#partner-integrations).

## Models

### Current Models (Use These)

- `gemini-3.8-live` — Default option for most low-latency voice agent experiences and real-time dialogue without reasoning delays. Supports interleaved reasoning, asynchronous function calling by default (`behavior: NON_BLOCKING`), and full-session client content updates.
- `gemini-3.8-live-extended-thinking` — High-reasoning audio-to-audio model recommended when higher background reasoning is required during live interactions. Processes background reasoning and async tool calls (`behavior: NON_BLOCKING` required) while streaming continuous spoken conversational fillers; lifecycle managed via `interaction_status` (`IN_PROGRESS` vs `IDLE`).
- `gemini-3.5-transcribe-live` — Real-time streaming speech-to-text with interim hypotheses, finalized transcripts, smart formatting, and Hybrid VAD.
- `gemini-3.5-live-translate-preview` — Real-time speech-to-speech streaming translation across 70+ languages.

> [!WARNING]
> **Legacy Models (`gemini-3.1-flash-live-preview`, `gemini-2.5-flash-native-audio-*`, `gemini-live-2.5-flash-preview`, `gemini-2.0-flash-live-001`)**: Read [`references/migration.md`](references/migration.md) for breaking protocol changes (`behavior: "NON_BLOCKING"`, `thinking_level`, `interaction_status`, `send_client_content`).

## SDKs

- **Python**: `google-genai` >= `2.3.0` — `pip install -U google-genai`
- **JavaScript/TypeScript**: `@google/genai` >= `2.3.0` — `npm install @google/genai`

> [!WARNING]
> Legacy SDKs `google-generativeai` (Python) and `@google/generative-ai` (JS) are **deprecated**. Never use them.

## Partner Integrations

To streamline real-time audio/video app development, use a third-party integration supporting the Gemini Live API over **WebRTC** or **WebSockets**:

- [LiveKit](https://docs.livekit.io/agents/models/realtime/plugins/gemini/) — Use the Gemini Live API with LiveKit Agents.
- [Pipecat by Daily](https://docs.pipecat.ai/guides/features/gemini-live) — Create a real-time AI chatbot using Gemini Live and Pipecat.
- [Fishjam by Software Mansion](https://docs.fishjam.io/tutorials/gemini-live-integration) — Create live video and audio streaming applications with Fishjam.
- [Vision Agents by Stream](https://visionagents.ai/integrations/gemini) — Build real-time voice and video AI applications with Vision Agents.
- [Voximplant](https://voximplant.com/products/gemini-client) — Connect inbound and outbound calls to Live API with Voximplant.
- [Firebase AI SDK](https://firebase.google.com/docs/ai-logic/live-api?api=dev) — Get started with the Gemini Live API using Firebase AI Logic.

## Audio Formats

- **Input**: Raw PCM, little-endian, 16-bit, mono. 16kHz native (will resample others). MIME type: `audio/pcm;rate=16000`
- **Output**: Raw PCM, little-endian, 16-bit, mono. 24kHz sample rate.

> [!IMPORTANT]
> Use `send_realtime_input` / `sendRealtimeInput` for all real-time streaming user input (audio, video, **and text**). On Gemini 3.8 models, `send_client_content` / `sendClientContent` is supported across the full session lifecycle with explicit roles (`user` or `model`) to inject conversation context (`turn_complete=true` unconditionally interrupts active generation).

> [!WARNING]
> Do **not** use `media` in `sendRealtimeInput`. Use the specific keys: `audio` for audio data, `video` for images/video frames, and `text` for text input.

---

## Quick Start

### Authentication

#### Python

```python
from google import genai

client = genai.Client(api_key="YOUR_API_KEY")
```

#### JavaScript

```js
import { GoogleGenAI } from '@google/genai';

const ai = new GoogleGenAI({ apiKey: 'YOUR_API_KEY' });
```

### Connecting to the Live API

#### Python
```python
from google.genai import types

config = types.LiveConnectConfig(
    response_modalities=[types.Modality.AUDIO],
    system_instruction=types.Content(
        parts=[types.Part(text="You are a helpful assistant.")]
    )
)

async with client.aio.live.connect(model="gemini-3.8-live", config=config) as session:
    pass  # Session is active
```

#### JavaScr

SAFEgrade B · trust 89/100 Nothing in the source contradicts what it says it does. Grade A is reserved for packages that have also passed the behavioural sandbox.

What the source does

Filesystem

none-observed

Network

none-observed

Shell

none-observed

Dependencies

pinned

Secrets in source

none-found

Findings (1)

- [ ] **Client content updates**: `send_client_content` is supported throughout the entire session lifecycle with explicit roles (`user` or `model`). Setting `turn_complete=true` unconditionally inter

Gates applied: no_behavioural_pass.

[![ModelsAtlas audit: Gemini Live Api Dev](https://modelsatlas.com/skills/google-gemini/gemini-live-api-dev/badge.svg)](https://modelsatlas.com/skills/google-gemini/gemini-live-api-dev/)

Not your listing, or something out of date? Claim or update your listing.