Media GenerationSAFE

🤖 Type-safe, provider-agnostic TypeScript AI SDK for streaming chat, tool calling, agents, and multimodal apps across OpenAI, Anthropic, Gemini, React, Vue, Svelte, and Solid.

The instruction file, verbatim from the audited commit — this is the text the model reads, and the surface the audit's instruction layer examines. Quoted here so you can judge it without cloning anything.

---
name: ai-core/media-generation
description: >
  Image, audio, video, speech (TTS), and transcription generation using
  activity-specific adapters: generateImage() with openaiImage/geminiImage/byteplusImage,
  generateAudio() with geminiAudio/falAudio, generateVideo() with async
  polling (openaiVideo/geminiVideo/grokVideo/falVideo/byteplusVideo/openRouterVideo,
  per-model typed durations), generateSpeech() with openaiSpeech/byteplusSpeech/elevenlabsSpeech,
  generateTranscription() with openaiTranscription/byteplusTranscription,
  generateVoice() with elevenlabsVoiceDesign (create a voice, then speak with it).
  React hooks:
  useGenerateImage, useGenerateAudio,
  useGenerateSpeech, useTranscription, useGenerateVideo.
  TanStack Start server function integration with toServerSentEventsResponse.
type: sub-skill
library: tanstack-ai
library_version: '0.42.0'
sources:
  - 'TanStack/ai:docs/media/generations.md'
  - 'TanStack/ai:docs/media/generation-hooks.md'
  - 'TanStack/ai:docs/media/image-generation.md'
  - 'TanStack/ai:docs/media/audio-generation.md'
  - 'TanStack/ai:docs/media/video-generation.md'
  - 'TanStack/ai:docs/media/text-to-speech.md'
  - 'TanStack/ai:docs/adapters/elevenlabs.md'
  - 'TanStack/ai:docs/media/voice-creation.md'
  - 'TanStack/ai:docs/media/transcription.md'
  - 'TanStack/ai:docs/advanced/debug-logging.md'
---

# Media Generation

> **Dependency note:** This skill builds on ai-core. Read it first for critical rules.

All media activities (image, speech, transcription, video) follow the same
server/client architecture: a `generate*()` function on the server, an SSE
transport via `toServerSentEventsResponse()`, and a framework hook on the
client.

## Setup -- Image Generation End-to-End

### Server (API route or TanStack Start server function)

```typescript
// routes/api/generate/image.ts
import { generateImage, toServerSentEventsResponse } from '@tanstack/ai'
import { openaiImage } from '@tanstack/ai-openai'

export async function POST(req: Request) {
  const { prompt, size, numberOfImages } = await req.json()

  const stream = generateImage({
    adapter: openaiImage('gpt-image-1'),
    prompt,
    size,
    numberOfImages,
    stream: true,
  })

  return toServerSentEventsResponse(stream)
}
```

### Client (React)

```tsx
import { useGenerateImage, fetchServerSentEvents } from '@tanstack/ai-react'
import { useState } from 'react'

function ImageGenerator() {
  const [prompt, setPrompt] = useState('')
  const { generate, result, isLoading, error, reset } = useGenerateImage({
    connection: fetchServerSentEvents('/api/generate/image'),
  })

  return (
    <div>
      <input
        value={prompt}
        onChange={(e) => setPrompt(e.target.value)}
        placeholder="Describe an image..."
      />
      <button
        onClick={() => generate({ prompt })}
        disabled={isLoading || !prompt.trim()}
      >
        {isLoading ? 'Generating...' : 'Generate'}
      </button>

      {error && <p>Error: {error.message}</p>}

      {result?.images.map((img, i) => (
        <img
          key={i}
          src={img.url || `data:image/png;base64,${img.b64Json}`}
          alt={img.revisedPrompt || 'Generated image'}
        />
      ))}

      {result && <button onClick={reset}>Clear</button>}
    </div>
  )
}
```

### TanStack Start: Server Function Streaming (recommended)

When using TanStack Start, return `toServerSentEventsResponse()` from a
server function. The client fetcher receives a `Response` and the hook
parses it as SSE automatically:

```typescript
// lib/server-functions.ts
import { createServerFn } from '@tanstack/react-start'
import { generateImage, toServerSentEventsResponse } from '@tanstack/ai'
import { openaiImage } from '@tanstack/ai-openai'
import type { OpenAIImageModel } from '@tanstack/ai-openai'

export const generateImageStreamFn = createServerFn({ method: 'POST' })
  .inputValidator((data: { prompt: string; model?: OpenAIImageModel }) => data)
  .handler(({ data }) => {
    return toServerSentEventsResponse(
      generateImage({
        adapter: openaiImage(data.model ?? 'gpt-image-1'),
        prompt: data.prompt,
        stream: true,
      }),
    )
  })
```

```tsx
import { useGenerateImage } from '@tanstack/ai-react'
import { generateImageStreamFn } from '../lib/server-functions'

function ImageGenerator() {
  const { generate, result, isLoading } = useGenerateImage({
    fetcher: (input) => generateImageStreamFn({ data: input }),
  })

  return (
    <button
      onClick={() => generate({ prompt: 'A sunset over mountains' })}
      disabled={isLoading}
    >
      {isLoading ? 'Generating...' : 'Generate'}
    </button>
  )
}
```

---

## Core Patterns

### 1. Image Generation

Supported adapters: `openaiImage` (dall-e-2, dall-e-3, gpt-image-1,
gpt-image-1-mini, gpt-image-2), `geminiImage` (gemini-3.1-flash-image,
gemini-3.1-flash-lite-image, gemini-nano-banana-2.1, gemini-3-pro-image,
imagen-4.0-generate-001, etc.)
and `byteplusImage` (Seedream — `seedream-4-0-250828`, `seedream-4-5-251128`,
the 5.0 family).

> **Use the GA Gemini image ids.** `gemini-3.1-flash-image-preview` and
> `gemini-3-pro-image-preview` were shut down on 2026-06-25 and now 404. They
> survive in the type union only as deprecated aliases so existing code keeps
> compiling — a call to them typechecks and then fails at runtime. Use
> `gemini-3.1-flash-image` / `gemini-3-pro-image` instead.

> **Seedream quirks:** `watermark` defaults to **`true`** (pass
> `modelOptions: { watermark: false }` for a clean image), `size` is a token
> (`'1K'` | `'2K'` | `'4K'`) **or** explicit `'2048x2048'` pixels but never a
> mix, and `numberOfImages` is an **upper bound** — Seedream has no `n`, so it
> maps onto group-image mode and the model may return fewer. Reads
> `ARK_API_KEY`.

```typescript
import { generateImage } from '@tanstack/ai'
import { openaiImage } from '@tanstack/ai-openai'
import { geminiImage } from '@tanstack/ai-gemini'

// OpenAI with quality/backg

SAFEgrade B · trust 89/100 Nothing in the source contradicts what it says it does. Grade A is reserved for packages that have also passed the behavioural sandbox.

What the source does

Filesystem

none-observed

Network

none-observed

Shell

none-observed

Dependencies

pinned

Secrets in source

none-found

Findings (0)

No findings outside the package's declared scope.

Gates applied: no_behavioural_pass.