Agent EnginesCAUTION

A framework for building agentic apps

The instruction file, verbatim from the audited commit — this is the text the model reads, and the surface the audit's instruction layer examines. Quoted here so you can judge it without cloning anything.

---
name: agent-engines
description: >-
  How to inspect and configure the AI engine (model provider) powering the
  agent. Use when the user asks to switch models, check which engine is active,
  test a new provider, or register a custom engine.
---

# Agent Engines

## Overview

The framework supports pluggable AI engines beneath the agent loop. The **Anthropic engine** is the default and best-in-class path (Claude models). Additional engines can be added via the Vercel AI SDK (OpenAI, Google Gemini, Groq, Mistral, Cohere, Ollama).

## Available Tools

| Tool | Purpose |
|---|---|
| `list-agent-engines` | List all registered engines, their capabilities, the current selection, and whether you can change the organization default (`canUpdateDefault`) |
| `set-agent-engine` | Set the organization's default engine and model (owners and admins only) |
| `test-agent-engine` | Send a trivial prompt to verify the engine works (connectivity + API key) |
| `check-provider-key` | Check a provider API key by listing the models it reaches, before saving it |
| `get-provider-models` | Read which models each provider shows in the model picker and default-model select |
| `manage-provider-models` | Choose those models (`set`) or go back to the recommended ones (`reset`) |
| `list-model-providers` | What Settings › Model shows: each provider's organization and personal key (masked), rejections, the restriction, and the stored default |

## Checking the Current Engine

```
list-agent-engines
```

Returns the registry of all engines (name, label, capabilities, supported models) plus the currently active engine and model. `supportedModels` is what the model picker offers: the provider's checked models when `modelSelection.state` is `"selected"`, otherwise its `recommendedModels`. `credentialRejected: true` means the provider rejected that engine's saved key (`credentialRejectedAt` says when); chats with it stop until the key is replaced, so tell the user.

## Switching Engines

```
set-agent-engine --engine "ai-sdk:openai" --model "gpt-4o"
```

Changes take effect on the next conversation. The default belongs to the organization (the `agent-engine` org setting), and only owners and admins can change it. A member's call is refused with an error to relay: ask an owner or admin, or pick a model for this chat in the model picker. A user with no organization sets their own default.

Resolution order (highest priority first):
1. Explicit `engine` option passed to `createAgentChatPlugin()` in the server plugin
2. The organization's default (`agent-engine` org setting)
3. `AGENT_ENGINE` environment variable
4. Default: `"anthropic"` (requires `ANTHROPIC_API_KEY`)

## Settings › Model

The Model page (`/settings/model`, page id `model`) has three groups:

- **Organization providers:** the organization's Builder.io connection and each organization key. Owners and admins manage them; members see only "N models".
- **Personal providers:** a member's own Builder.io connection, their own keys, and the ChatGPT subscription while its lab is on. A personal key is used before the organization's, for that person only.
- **Organization settings:** Default model (`manage-agent-engine` `set`, organization models only), Restrict personal API keys (`manage-provider-key-policy`), and Max iterations (`manage-agent-loop-settings`, 1 to 1000). Owners and admins change them; members see the values.

Read the page's state with `list-model-providers`, and the models each provider shows with `get-provider-models`. Keys are added and replaced in the provider dialog, which checks a pasted key by listing its models; you can't save a key for the user, so check one with `check-provider-key` and send them to Model (`open-settings-page`) to paste it. Before they remove a provider, run `preview-secret-removal` with the provider's key name and scope and tell them what stops.

## Restricting Personal API Keys

`manage-provider-key-policy` reads or changes the organization's "Restrict personal API keys" setting. While it is on, members (not owners or admins) can't use or save their own provider keys or a personal Builder.io connection, so their chats use organization providers; with none, a chat fails with "Owners and admins restricted personal API keys." Nothing is deleted, and turning it off restores their keys.

```
manage-provider-key-policy            # read; owners/admins also get affectedMembers
manage-provider-key-policy --set true # owners and admins only
```

Read it first and tell the user which members lose which providers before turning it on.

## Testing a New Engine

Before switching, verify the engine is working:

```
test-agent-engine --engine "ai-sdk:openai" --model "gpt-4o"
```

Returns `{ ok, latencyMs, response, capabilities }`. If `ok: false`, the error message explains what's wrong (missing API key, package not installed, etc.).

To check a key without sending a prompt, for example one the user just gave you:

```
check-provider-key --provider "anthropic" --key "<the key>"
```

Returns `{ ok: true, models }` or `{ ok: false, code, reason }` (`rejected`, `wrong-provider`, `missing-key`, `invalid-endpoint`, `unreachable`, `provider-error`). Relay the reason. Omit `--key` to re-check the saved key; it is only checked against its saved endpoint, so `--baseUrl` needs `--key` (except for Ollama). `--scope org` is for owners and admins.

## Choosing Which Models Show

Each provider (Builder.io included) keeps the models its owner checked. Only checked models appear in the chat model picker and the default-model select; Builder.io and providers with their own key show side by side.

```
get-provider-models --provider "openai"
manage-provider-models --action "set" --provider "openai" --models '["gpt-6-sol"]'
manage-provider-models --action "reset" --provider "openai"
```

- A selection lives at the same scope as the provider's key: `--scope user` is the caller's own, `--scope org` is the organization's (owners and admins only; members

CAUTIONgrade B · trust 89/100 Install with care. The audit found things worth knowing before you trust its output.

What the source does

Filesystem

none-observed

Network

none-observed

Shell

none-observed

Dependencies

pinned

Secrets in source

none-found

Findings (5)

.agents/skills/an

.agents/skills/content-product-development

.agents/skills/design-exploration

.agents/skills/turn-into-app

.agents/skills/visual-edit

Gates applied: no_behavioural_pass.

[![ModelsAtlas audit: Agent Engines](https://modelsatlas.com/skills/builderio/agent-engines/badge.svg)](https://modelsatlas.com/skills/builderio/agent-engines/)

Not your listing, or something out of date? Claim or update your listing.