App Control

What the source does

• FilesystemUndeclared5 observations, not in its own description
• NetworkNone observedNo outbound calls in the source
• ShellNone observedNo commands run in the source
• DependenciesPinnedNothing floats
• SecretsNone foundNo keys in source

Five audit layers

Gates applied: no behavioural pass.

assistant/CLAUDE.md

assistant/src/approvals/CLAUDE.md

assistant/src/notifications/CLAUDE.md

assistant/src/runtime/CLAUDE.md

gateway/CLAUDE.md

import { forwardAppControlProxyTool } from "../../../../tools/app-control/skill-proxy-bridge.js";

} from "../../../../tools/types.js";

import { forwardAppControlProxyTool } from "../../../../tools/app-control/skill-proxy-bridge.js";

} from "../../../../tools/types.js";

import { forwardAppControlProxyTool } from "../../../../tools/app-control/skill-proxy-bridge.js";

Audit 0.4.1 · source sha 19bc86343b63 · 14 Oct 2026Report an issue or request a re-scan

• Publishervellum-ai GitHub organizationOwner of the repository this skill is read from
• Source foldervellum-ai/vellum-assistant/assistant/src/config/bundled-skills/app-controlThe exact folder we audited
• LicenceMITAs the repository states it
• Commit pinned for this verdict19bc86343b63A new audit runs when the source changes
• Repository statusNot archivedLast push 1 day ago
• Sandboxed runNot runSkills are read, not executed; layer L4 is skipped

Its description: Drive a specific named macOS app via raw input bypassing the Accessibility tree

---
name: app-control
description: Drive a specific named macOS app via raw input bypassing the Accessibility tree
compatibility: "Designed for Vellum personal assistants"
metadata:
  emoji: "🎯"
  vellum:
    display-name: "App Control"
    category: "system"
    feature-flag: "app-control"
    activation-hints:
      - "User explicitly directs the assistant to drive a specific named app via raw input (emulator, game, OpenGL canvas, custom-rendered Electron app)"
      - "User says the macOS Accessibility tree is unhelpful or empty for the target app"
    avoid-when:
      - "Task can be done via the computer-use skill (general macOS UI navigation)"
      - "Task can be done via a CLI / API alternative"
---

This skill exposes the `app_control_*` proxy tools for driving a single
named macOS application via raw input — keyboard, mouse, screenshot — that
bypasses the system Accessibility tree. Use it only when explicitly directed
to a specific app where the AX tree is unhelpful (emulators, games, OpenGL
canvases, custom-rendered Electron apps). For general macOS UI navigation
prefer the `computer-use` skill.

Tools in this skill are proxy tools — execution is forwarded to the connected
macOS client, never handled locally by the assistant.

## Cadence

Take 2-3 actions per turn, then yield with a short narration so the user can
interject. Do not chain long sequences without surfacing what you are doing.

## Always observe before acting

Call `app_control_observe` before your first input action whenever the screen
state matters (e.g. you need to know what is on screen, where a UI element is,
or whether the app is even running). Re-observe after actions that may have
moved the window or changed visibility.

`observe` waits a short settle delay (default ~200ms) before capturing so the
target app and the WindowServer can flush pending input and composite a fresh
frame. If the captured screenshot looks one input behind the latest state
(common with emulators or other slow-feedback apps), pass a larger
`settle_ms`. For static UIs where you just want a quick snapshot, pass
`settle_ms: 0` to skip the wait.

## Input choice

- Prefer `app_control_sequence` over multiple back-to-back `app_control_press`
  calls when sending an ordered batch of presses (e.g. menu navigation,
  repeated movement). Sequence runs in a single round-trip — the target app is
  activated once at the start and the keys are sent serially without any
  window for keyboard focus to drift to another app between presses. Each step
  may carry its own `duration_ms` (hold) and `gap_ms` (pause after).
- Prefer `app_control_combo` over rapid sequential `app_control_press` for
  simultaneous inputs (e.g. cmd+shift+4). `combo` holds every key at once;
  sequential presses interleave key-down and key-up events.
- Use `app_control_type` for literal text into a focused field.

## Coordinate caveat

`app_control_click` and `app_control_drag` use **window-relative** coordinates.
The window may move or resize between observation and click — if you are
uncertain whether the window has shifted, re-observe first.

## App targeting

Use bundle IDs (e.g. `com.example.app`) when possible — they are the most
reliable identifier. Fall back to localized process names if a bundle ID is
unavailable.

## Ending the session

Call `app_control_stop` when you are done. Do **not** auto-quit the controlled
app — `stop` only ends the app-control session, leaving the app running.

Excerpt, first 3,470 characters of SKILL.md, verbatim from the audited commit. It is text from someone else’s repository, quoted so you can judge it before your agent reads it.

git clone https://github.com/vellum-ai/vellum-assistant
git -C vellum-assistant checkout 19bc86343b63

Works in

1. 2026-10-14
   ✓SAFE B 89 / 100

   first audit · source 19bc86343b63

Questions about App Control

• github.com/vellum-ai/vellum-assistant/tree/main/assistant/src/co — the audited source at commit 19bc86343b63 and every finding aboveObserved · 14 Oct 2026

Spotted something wrong? Report a correction · Methodology