Ai Video ToolsCAUTION

A framework for building agentic apps

The instruction file, verbatim from the audited commit — this is the text the model reads, and the surface the audit's instruction layer examines. Quoted here so you can judge it without cloning anything.

---
name: ai-video-tools
description: >-
  All AI features in Clips — titles, summaries, chapters, tags, filler-word
  removal — delegate to the agent chat via sendToAgentChat except the narrow
  media pipeline path: transcription. Use when adding any
  AI-powered feature.
---

# AI Video Tools

## Rule

Every AI feature in Clips goes through the agent chat unless it is the narrow media-pipeline exception below. The UI and server should not add broad shadow agents or inline chat workflows. **Exception:** transcription. Transcription takes audio, not prompts — the `request-transcript` action calls the transcription API directly. Provider priority for transcription: **native** first (browser Web Speech API, desktop local Whisper/macOS SFSpeech when available, and desktop Web Speech fallback on non-mac, saved via `save-browser-transcript`, no key required) → **cloud fallback** when native text is missing: **Builder.io managed** Gemini (via `BUILDER_PRIVATE_KEY` or a connected Builder account, no extra key needed) → **Groq** `whisper-large-v3-turbo` via `GROQ_API_KEY` (fast, ~$0.04/hr). Clips never routes recording/meeting audio to OpenAI for transcription.

Builder.io is the primary setup path: it brings managed AI credits, object
storage, uploads, and transcription together. Bring-your-own-key setup belongs
in the agent sidebar's **API Keys & Connections** panel; Clips settings can
signpost that existing panel, but should not add a second key-management
surface.

## Why

The agent is already the user's primary interface — it has full project context, can chain tool calls, and can ask follow-up questions. Shadow LLM calls inside UI components create a second AI that doesn't know what the agent knows and can't coordinate with it. See the framework `delegate-to-agent` skill for the full argument.

## Features and how they delegate

| Feature                   | Trigger                                                                               | What the action does                                                                                  |
| ------------------------- | ------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------- |
| Include full video        | User toggles "Include full video" in the AI tools menu                                | `update-clips-ai-prefs --includeFullVideoInAi=true\|false` → stored in `clips-user-prefs`; default off |
| Default title             | Native/cloud transcript becomes ready and the recording still has an automatic title   | `regenerate-title --recordingId=<id> --includeSummary=true` tries the transcript-only Gemini fast path, keeps any local heuristic title replaceable, and queues a `generate-metadata` agent refinement that writes a specific title plus description. If Include full video is on, it always delegates so the agent watches the recording |
| Manual title suggestion   | User asks the agent "rename this"                                                     | Agent reads transcript and calls `update-recording --id=<id> --title=...`                             |
| Summary / description     | Transcript becomes ready, or user clicks "Summarize" / "Regenerate description"       | The automatic metadata handoff or `regenerate-summary` asks the agent to write `update-recording --description=...`. Existing user-authored descriptions are preserved. With Include full video, the agent must watch the clip, not transcript alone |
| Chapters                  | User clicks "Add chapters" or transcript > 3 minutes                                  | `generate-chapters --id=<id>` / `regenerate-chapters` → agent writes `chapters_json` (same full-video preference) |
| Tags                      | On upload complete                                                                    | `generate-ai-metadata --id=<id> --kind=tags` → agent inserts `recording_tags` rows                    |
| Filler-word removal       | User clicks "Remove ums and uhs"                                                      | `generate-filler-removal --id=<id>` → agent writes proposed cuts into `editor-draft` for user review  |
| Comment auto-reply        | User types "reply with ..." in the agent chat                                           | agent calls `add-comment` directly                                                                    |
| **Transcription**         | On upload complete (automatic) + live during recording                                | `request-transcript` → native (Web Speech / macOS SFSpeech) first, then cloud fallback Builder.io managed Gemini → Groq; `save-browser-transcript` for instant Web Speech result — see "Transcription" section below |

## Include full video

Screen recordings often have thin or misleading audio ("real quick", "um this
thing"). Titles, descriptions, chapters, and workflow docs (PR / SOP / ticket /
email) are more accurate when the agent can **see** the UI, product names, and
on-screen text.

- Preference key: `includeFullVideoInAi` on user setting `clips-user-prefs`.
- UI: checkbox at the top of the recording-page **AI tools** dropdown (copy
  notes Gemini-only).
- Actions: `get-clips-ai-prefs`, `update-clips-ai-prefs`.
- **Gemini only:** sending / understanding the full recording video requires a
  Gemini model (Builder Gemini or `GEMINI_API_KEY`). Claude and OpenAI cannot
  ingest the MP4/WebM. When the preference is on, the AI-request bridge prefers
  Builder `gemini-3-5-flash` for that turn.
- When on, queued `clips-ai-request-*` messages include instructions to attach
  or upload the recording to Gemini when possible, otherwise use
  `get-recording-player-data` / `create-recording-agent-link` and follow
  recommended frames / the frame API across the timeline — not transcript only.
- Default title generation also honors this: `regenerate-title` skips the
  transcript-only Gemini cleanup path 

CAUTIONgrade B · trust 89/100 Install with care. The audit found things worth knowing before you trust its output.

What the source does

Filesystem

none-observed

Network

none-observed

Shell

none-observed

Dependencies

pinned

Secrets in source

none-found

Findings (5)

.agents/skills/an

.agents/skills/content-product-development

.agents/skills/design-exploration

.agents/skills/turn-into-app

.agents/skills/visual-edit

Gates applied: no_behavioural_pass.

[![ModelsAtlas audit: Ai Video Tools](https://modelsatlas.com/skills/builderio/ai-video-tools/badge.svg)](https://modelsatlas.com/skills/builderio/ai-video-tools/)

Not your listing, or something out of date? Claim or update your listing.