caveman-manage

What the source does

• FilesystemNone observedNo reads or writes in the source
• NetworkNone observedNo outbound calls in the source
• ShellNone observedNo commands run in the source
• DependenciesPinnedNothing floats
• SecretsNone foundNo keys in source

Five audit layers

Gates applied: no behavioural pass.

Audit 0.4.1 · source sha ee5548342062 · 31 Oct 2026Report an issue or request a re-scan

• Publisherjuliusbrussee GitHub userOwner of the repository this skill is read from
• Source folderjuliusbrussee/caveman/skills/caveman-manageThe exact folder we audited
• LicenceApache-2.0As the repository states it
• Commit pinned for this verdictee5548342062A new audit runs when the source changes
• Repository statusNot archivedPushed today
• Sandboxed runNot runSkills are read, not executed; layer L4 is skipped

Its description: Inspect Caveman Cloud's experiment lifecycle and block unsafe execution. Use when asked to start, approve, cancel, promote or roll back a Caveman experiment.

---
name: caveman-manage
description: >
  Inspect Caveman Cloud's experiment lifecycle and block unsafe execution. Use
  when asked to start, approve, cancel, promote or roll back a Caveman
  experiment.
---

# Manage eval-gated experiments

Treat every lifecycle change as a production control action. Read current state
and results, then report one supported recommendation or block.
Current agent MCP is intentionally read-only: control-api does not yet enforce a
complete lifecycle transition table and evidence gate atomically.

## Non-negotiable gates

1. A request to review, inspect, explain, or recommend authorizes reads only.
2. Never approve an experiment whose results are pending, whose required
   guardrails are absent, or whose evidence reports a breach.
3. Never convert experiment lift into `verified_savings`. Only active real
   traffic plus provider-causal, provider-complete ledger evidence can do that.
4. Never supply an organization id. Project and tenant scope come from the
   logged-in Caveman identity and server RBAC.
5. Never execute a lifecycle mutation, even after user approval. Exact
   `<action>:<experiment_id>` strings are agent-generatable and are not proof of
   human intent.
6. Unknown states and server errors fail closed. Report exact
   `cave_snake_code`.

## Step 1 — Load project and experiment

Prefer MCP:

```text
caveman_context {}
caveman_experiment_get {"action":"get","experiment_id":"<id>"}
caveman_experiment_get {"action":"results","experiment_id":"<id>"}
```

Use `{"action":"list"}` when the user has not named an id.

CLI fallback:

```bash
caveman cloud experiments list
caveman cloud experiments show <id>
caveman cloud experiments results <id>
```

Stop if login, project, experiment, or results are unavailable.

## Step 2 — Evaluate evidence

Report:

- current lifecycle state and safety class;
- control and candidate sample sizes;
- quality or eval result;
- latency, error, cost, retry, drop, and escalation guardrails when present;
- evidence cost;
- rollback or hold reason;
- whether result is pending, failed, promotable, or active.

Absence is not a pass. If a required field is absent, state
`evidence incomplete` and do not propose approval.

## Step 3 — Propose one action

Allowed actions:

- `start` — only from a startable draft or queued state with configured graders;
- `approve` — only with complete passing evidence and a safety class the
  current role may approve;
- `cancel` — stop a non-active experiment the user no longer wants;
- `rollback` — revert an active or harmful change through the server's linked
  policy path. Current deployments may reject this honestly with
  `cave_not_implemented`; never describe that response as a rollback.

Show recommendation and id:

```text
Proposed action: approve experiment 7f...
Reason: candidate passed quality and every configured guardrail.
Execution: blocked until server-authoritative lifecycle and evidence gates ship.
```

Do not treat earlier generic statements such as "manage it" or "do what is best"
as mutation approval.

## Step 4 — Block unsafe execution

Do not emit or run an executable lifecycle command. Explain that current server
does not yet enforce every evidence/state transition atomically. CLI and MCP
agent surfaces therefore expose experiment reads only.

## Step 5 — Re-read after external operator action

If operator says they executed command, read detail and results again. Report
server-observed post-state, audit or result response, and any policy-delivery
status returned. Never infer success from operator intent alone.

Use this close:

```text
Action: <action> <experiment-id>
Before: <state>
Server response: <status and cave_snake_code if any>
After: <re-read state>
Basis: experiment evidence only. Verified savings unchanged unless the signed
ledger independently records active, provider-causal real-traffic savings.
```

Excerpt, first 3,897 characters of SKILL.md, verbatim from the audited commit. It is text from someone else’s repository, quoted so you can judge it before your agent reads it.

git clone https://github.com/juliusbrussee/caveman
git -C caveman checkout ee5548342062

Works in

1. 2026-10-31
   ✓SAFE B 89 / 100

   first audit · source ee5548342062

Questions about caveman-manage

• github.com/juliusbrussee/caveman/tree/main/skills/caveman-manage — the audited source at commit ee5548342062 and every finding aboveObserved · 31 Oct 2026

Spotted something wrong? Report a correction · Methodology