Vellum Github App Setup

What the source does

• FilesystemNone observedNo reads or writes in the source
• NetworkDeclared4 observations
• ShellDeclared2 observations
• DependenciesPinnedNothing floats
• SecretsNone foundNo keys in source

Five audit layers

Gates applied: no behavioural pass.

**Non-secret fields** — read them from the credentials JSON and store them inline. These values come from the manifest API exchange (never typed or pasted by the user), so pass `--generated`:

**Secret fields** (`client_secret`, `webhook_secret`, and the private key) — never read these into the conversation, ask for them in chat, or pass them inline to `assistant credentials set` (the CLI r

assistant/CLAUDE.md

assistant/src/approvals/CLAUDE.md

assistant/src/notifications/CLAUDE.md

assistant/src/runtime/CLAUDE.md

gateway/CLAUDE.md

Audit 0.4.1 · source sha 19bc86343b63 · 14 Oct 2026Report an issue or request a re-scan

• Publishervellum-ai GitHub organizationOwner of the repository this skill is read from
• Source foldervellum-ai/vellum-assistant/skills/vellum-github-app-setupThe exact folder we audited
• LicenceMITAs the repository states it
• Commit pinned for this verdict19bc86343b63A new audit runs when the source changes
• Repository statusNot archivedLast push 1 day ago
• Sandboxed runNot runSkills are read, not executed; layer L4 is skipped

Its description: Create and configure a GitHub App so the assistant can push commits, open PRs, and comment under its own bot identity. Use when the user wants the assistant to have its own GitHub identity, or when setting up git push…

---
name: vellum-github-app-setup
description: Create and configure a GitHub App so the assistant can push commits, open PRs, and comment under its own bot identity. Use when the user wants the assistant to have its own GitHub identity, or when setting up git push access for the first time.
compatibility: "Designed for Vellum personal assistants. Requires Python 3, bun, and the assistant credentials CLI."
metadata:
  icon: assets/icon.svg
  emoji: "🤖"
  author: vellum-ai
  version: "1.0"
  vellum:
    category: "development"
    display-name: "GitHub App Setup"
---

## Overview

This skill creates a **GitHub App** under a GitHub organization, giving the assistant its own bot identity for git operations. After setup, commits, PRs, and comments will attribute to `<app-name>[bot]` instead of the user's personal account.

**Total manual effort: 7 interactions** — Continue to GitHub → Create App → 3 secure credential prompts → Install on repo → (optional) upload avatar. Everything else is automated.

> **Note:** This skill currently supports GitHub **organizations** only, not personal accounts.

## What Gets Created

- A **GitHub App** owned by the org (not the user's personal account)
- **Installation** on selected repositories with configurable permissions (default: `contents:write`, `pull_requests:write`, `checks:read`, `metadata:read`)
- **Credentials** stored in the assistant's encrypted vault (7 fields)
- **Token helper script** at `bin/gh-app-token.mjs` (in the workspace root) for refreshing auth tokens

## Prerequisites

- User must be an **admin** of the GitHub organization
- User must be **logged into GitHub** in their browser
- The `assistant credentials` CLI must be available
- Python 3 on the host machine (for the manifest flow server)
- `bun` in the container (for the token helper script)

## Setup Flow

### Step 1: Create the GitHub App

Run the manifest flow script on the **user's host machine** (it needs to open a browser and catch a localhost callback):

```bash
python3 scripts/create-github-app.py \
  --org=ORG_NAME \
  --name=APP_NAME \
  --url="https://example.com" \
  --output=/tmp/github-app-credentials.json
```

This starts a local server on `localhost:29170`, opens the browser, and guides the user through two clicks:

1. **"Continue to GitHub"** — submits the pre-filled manifest form
2. **"Create GitHub App for ORG"** — confirms on GitHub's page

The callback server catches the redirect and exchanges the code for credentials automatically. The credentials JSON is saved to `--output`.

**Important manifest details** (learned the hard way):

- `hook_attributes` MUST include a nested `url` field, even if webhooks are disabled. Without it, GitHub rejects the manifest with a misleading `"url" wasn't supplied` error — it's not about the top-level `url`.
- The form must use `<input type="text">` (not hidden) with the value set via `JSON.stringify()` in JavaScript.
- A `state` query parameter must be included on the form action URL.
- The `redirect_url` must point to the local callback server.

### Step 2: Store Credentials

The credentials JSON from Step 1 lives on the user's host. Store the non-secret identifier fields inline, then collect the secret fields from the user via the secure prompt.

**Non-secret fields** — read them from the credentials JSON and store them inline. These values come from the manifest API exchange (never typed or pasted by the user), so pass `--generated`:

```bash
assistant credentials set --service github-app --field app_id "APP_ID" --generated
assistant credentials set --service github-app --field app_slug "APP_SLUG" --generated
assistant credentials set --service github-app --field client_id "CLIENT_ID" --generated
```

**Secret fields** (`client_secret`, `webhook_secret`, and the private key) — never read these into the conversation, ask for them in chat, or pass them inline to `assistant credentials set` (the CLI refuses inline user-supplied secrets). Collect each one with `assistant credentials prompt` — a secure input whose value never enters the chat transcript — and have the user copy the values out of the credentials JSON on their host:

```bash
assistant credentials prompt --service github-app --field client_secret \
  --label "GitHub App Client Secret" \
  --description "Copy the client_secret value from the credentials JSON (e.g. /tmp/github-app-credentials.json)"
assistant credentials prompt --service github-app --field webhook_secret \
  --label "GitHub App Webhook Secret" \
  --description "Copy the webhook_secret value from the credentials JSON"
```

For the PEM (private key), the secure prompt input is single-line — pasting a multi-line key strips its newlines and the mangled key fails JWT signing. Have the user base64-encode the key to a single line on their host and paste **that** into the secure prompt. It is stored as field `pem_b64`; `scripts/gh-app-token.mjs` (copied into the workspace in Step 4) is the sole decode site — it reveals and base64-decodes the key internally at JWT-signing time, so the plaintext PEM never appears anywhere. If a step genuinely needs the decoded key in a shell, capture it in a variable — `PEM="$(assistant credentials reveal --service github-app --field pem_b64 | base64 -d)"` — and **never run the reveal|decode pipeline bare: it prints the private key into the transcript**, evading every redaction layer. Print the single-line value for copying:

```bash
jq -r .pem /tmp/github-app-credentials.json | base64 | tr -d '\n'
```

```bash
assistant credentials prompt --service github-app --field pem_b64 \
  --label "GitHub App Private Key (base64-encoded PEM)" \
  --description "Paste the single-line output of: jq -r .pem /tmp/github-app-credentials.json | base64 | tr -d '\n'"
```

> **Note:** The `installation_id` credential is stored in Step 3 after installing the app.

### Step 3: Install the App on Repositories

Open the installation page for the user:

```
https://github.com/apps/APP_SLUG/installatio

Excerpt, first 6,001 characters of SKILL.md, verbatim from the audited commit. It is text from someone else’s repository, quoted so you can judge it before your agent reads it.

git clone https://github.com/vellum-ai/vellum-assistant
git -C vellum-assistant checkout 19bc86343b63

Works in

1. 2026-10-14
   ✕BLOCK D 69 / 100

   first audit · source 19bc86343b63

Questions about Vellum Github App Setup

• github.com/vellum-ai/vellum-assistant/tree/main/skills/vellum-gi — the audited source at commit 19bc86343b63 and every finding aboveObserved · 14 Oct 2026

Spotted something wrong? Report a correction · Methodology