QaBLOCK

A framework for building agentic apps

The instruction file, verbatim from the audited commit — this is the text the model reads, and the surface the audit's instruction layer examines. Quoted here so you can judge it without cloning anything.

---
name: qa
description: >-
  Autonomous multi-app QA sweep that drives template apps with Playwright MCP.
  Use only when the user explicitly runs /qa or asks for an end-to-end QA sweep.
  Do not load it for ordinary feature work, single-page checks, or one-off bug
  fixes.
user-invocable: true
scope: dev
metadata:
  internal: true
---

# QA Testing

Autonomous QA testing that spins up template apps, tests them with Playwright in parallel, fixes issues, retests, and reports findings. Only alerts the user when blocked.

This is an explicitly requested sweep, not a default verification step. Do not run any part of it — dev servers, browser automation, tester agents — as follow-up to a normal code change.

## Usage

```
/qa                                            # test mail, calendar, content, forms
/qa --apps mail,forms                          # test specific apps
/qa --focus "test form submission and compose"  # prioritize specific flows
```

## Browser MCP Readiness

QA can use the framework's built-in browser MCP capabilities instead of a hand-written `mcp.config.json`. The built-ins are off by default and are toggled through `/_agent-native/mcp/builtin`.

- Prefer `browser-playwright` for automated QA sweeps: it runs `npx -y @playwright/[email protected]`.
- Use `browser-chrome-devtools` only when the test specifically needs to attach to a live Chrome session. It runs `npx -y [email protected] --autoConnect --no-usage-statistics` and requires Chrome 144+ with remote debugging enabled. Do not assume it signs into the user's Chrome profile.
- Browser built-ins are exclusive per scope: enabling Chrome disables Playwright and enabling Playwright disables Chrome.
- `computer-use` runs `npx -y [email protected]` and is macOS-only.

**Args:**

- `--apps` — comma-separated app names (default: `mail,calendar,content,forms`)
- `--focus` — natural language guidance for what to prioritize

## Orchestrator Steps

You (the agent running this skill) are the orchestrator. Follow these steps in order.

### Step 1: Parse Args

Parse the user's invocation to determine:

- **apps**: which apps to test (default: mail, calendar, content, forms)
- **focus**: optional test priority guidance

### Step 2: Check Credential Readiness

For each app, check if required credentials exist:

| App      | Check                                                    | Can test without? |
| -------- | -------------------------------------------------------- | ----------------- |
| forms    | No credentials needed                                    | Yes               |
| content  | No credentials needed (Notion is opt-in)                 | Yes               |
| calendar | `templates/calendar/.env` has `GOOGLE_CLIENT_ID` present | Partially — local events work, Google sync won't |
| mail     | `templates/mail/.env` has `GOOGLE_CLIENT_ID` present     | Partially — UI renders, Gmail features won't |

Read each app's `.env` file (if it exists) only to check whether required names
are present. Never print, copy, summarize, paste, or pass `.env` values into
tester prompts, reports, screenshots, logs, or chat. If credentials are missing:

- Still test the app — many features work without external APIs
- Include in the tester's instructions: "No Google credentials found. Test local features. Flag any feature that crashes without credentials as 'needs credentials' rather than a bug."
- Only alert the user (via SendMessage to team lead or AskUserQuestion) if the app **cannot start at all**

### Step 3: Start Dev Servers

Start each app's dev server on a dedicated port using `run_in_background`:

| App      | Port | Command |
|----------|------|---------|
| mail     | 9201 | `cd templates/mail && PORT=9201 pnpm dev` |
| calendar | 9202 | `cd templates/calendar && PORT=9202 pnpm dev` |
| content  | 9203 | `cd templates/content && PORT=9203 pnpm dev` |
| forms    | 9204 | `cd templates/forms && PORT=9204 pnpm dev` |

Start all servers in parallel (multiple Bash calls with `run_in_background: true`).

Then verify each server is ready by polling with curl:

```bash
for i in {1..30}; do curl -s -o /dev/null -w "%{http_code}" http://localhost:9201 && break; sleep 1; done
```

If a server fails to start within 30 seconds, skip that app and note it in the report.

### Step 4: Generate Test Plans

For each app, read these files to understand what to test:

1. `templates/<app>/app/routes/` or `templates/<app>/app/routes.ts` — discover all pages
2. `templates/<app>/CLAUDE.md` — features, API routes, data model
3. `templates/<app>/actions/` — domain operations the UI and agent share
4. `templates/<app>/server/routes/api/` — route-only endpoints such as uploads, streaming, webhooks, and OAuth callbacks

Combine with any `--focus` guidance to produce a test plan. The test plan is a numbered list of user-facing flows to verify. Example:

```
1. Home page loads without errors
2. Create a new item via the UI
3. Edit an existing item
4. Delete an item
5. Check all navigation links work
6. Verify no console errors on any page
7. Verify no failed network requests
```

### Step 5: Create Team and Spawn Testers

Create a team and spawn one tester agent per app, all in parallel:

```
TeamCreate: name="qa", description="QA testing sweep"
```

Create one task per app:

```
TaskCreate: subject="QA test <app> on port <port>", description="<test plan>"
```

Then spawn tester agents in parallel using the Agent tool. Each agent gets:

- `name`: `qa-<app>` (e.g., `qa-mail`)
- `team_name`: `qa`
- `mode`: `auto`
- The full tester prompt (see Tester Agent Prompt below)

### Step 6: Monitor and Alert

After spawning testers, wait for their idle notifications and task updates.

- When a tester completes its task, note its findings
- When a tester reports being blocked (missing credentials, app won't start, needs user input), relay that to the user via AskUserQuestion
- When all testers are done, proceed to Step 7

### Step 7: Compile Report and Shutdown

Gat

BLOCKgrade D · trust 69/100 Do not install this without reading the findings. The audit found something that could harm you or your machine.

What the source does

Filesystem

none-observed

Network

none-observed

Shell

none-observed

Dependencies

pinned

Secrets in source

none-found

Findings (6)

Read each app's `.env` file (if it exists) only to check whether required names

.agents/skills/an

.agents/skills/content-product-development

.agents/skills/design-exploration

.agents/skills/turn-into-app

.agents/skills/visual-edit

Gates applied: no_behavioural_pass.

[![ModelsAtlas audit: Qa](https://modelsatlas.com/skills/builderio/qa/badge.svg)](https://modelsatlas.com/skills/builderio/qa/)

Not your listing, or something out of date? Claim or update your listing.