Atlas / Skills / leoyeai / Testrail

TestrailCAUTION

skills/leoyeai/testrail

🧠 Curated collection of 1209+ best OpenClaw skills — weekly updated by MyClaw.ai

Verdict
CAUTION
Grade
B
Trust score
89 /100
Version
—
Hosts
1 documented
License
MIT
Stars
2,160
01

Overview

🧠 Curated collection of 1209+ best OpenClaw skills — weekly updated by MyClaw.ai

Read from source at commit 4f3b4a2a472eOBSERVED · 2026-10-08
02

Host compatibility

What the documentation claims. We have not run a compatibility test.

HostStatusNotes
openclawmentioned
03

What it tells the agent

The instruction file, verbatim from the audited commit — this is the text the model reads, and the surface the audit's instruction layer examines. Quoted here so you can judge it without cloning anything.

---
name: "testrail"
description: >-
  Sync tests with TestRail. Use when user mentions "testrail", "test management",
  "test cases", "test run", "sync test cases", "push results to testrail",
  or "import from testrail".
---

# TestRail Integration

Bidirectional sync between Playwright tests and TestRail test management.

## Prerequisites

Environment variables must be set:
- `TESTRAIL_URL` — e.g., `https://your-instance.testrail.io`
- `TESTRAIL_USER` — your email
- `TESTRAIL_API_KEY` — API key from TestRail

If not set, inform the user how to configure them and stop.

## Capabilities

### 1. Import Test Cases → Generate Playwright Tests

```
/pw:testrail import --project <id> --suite <id>
```

Steps:
1. Call `testrail_get_cases` MCP tool to fetch test cases
2. For each test case:
   - Read title, preconditions, steps, expected results
   - Map to a Playwright test using appropriate template
   - Include TestRail case ID as test annotation: `test.info().annotations.push({ type: 'testrail', description: 'C12345' })`
3. Generate test files grouped by section
4. Report: X cases imported, Y tests generated

### 2. Push Test Results → TestRail

```
/pw:testrail push --run <id>
```

Steps:
1. Run Playwright tests with JSON reporter:
   ```bash
   npx playwright test --reporter=json > test-results.json
   ```
2. Parse results: map each test to its TestRail case ID (from annotations)
3. Call `testrail_add_result` MCP tool for each test:
   - Pass → status_id: 1
   - Fail → status_id: 5, include error message
   - Skip → status_id: 2
4. Report: X results pushed, Y passed, Z failed

### 3. Create Test Run

```
/pw:testrail run --project <id> --name "Sprint 42 Regression"
```

Steps:
1. Call `testrail_add_run` MCP tool
2. Include all test case IDs found in Playwright test annotations
3. Return run ID for result pushing

### 4. Sync Status

```
/pw:testrail status --project <id>
```

Steps:
1. Fetch test cases from TestRail
2. Scan local Playwright tests for TestRail annotations
3. Report coverage:
   ```
   TestRail cases: 150
   Playwright tests with TestRail IDs: 120
   Unlinked TestRail cases: 30
   Playwright tests without TestRail IDs: 15
   ```

### 5. Update Test Cases in TestRail

```
/pw:testrail update --case <id>
```

Steps:
1. Read the Playwright test for this case ID
2. Extract steps and expected results from test code
3. Call `testrail_update_case` MCP tool to update steps

## MCP Tools Used

| Tool | When |
|---|---|
| `testrail_get_projects` | List available projects |
| `testrail_get_suites` | List suites in project |
| `testrail_get_cases` | Read test cases |
| `testrail_add_case` | Create new test case |
| `testrail_update_case` | Update existing case |
| `testrail_add_run` | Create test run |
| `testrail_add_result` | Push individual result |
| `testrail_get_results` | Read historical results |

## Test Annotation Format

All Playwright tests linked to TestRail include:

```typescript
test('should login successfully', async ({ page }) => {
  test.info().annotations.push({
    type: 'testrail',
    description: 'C12345',
  });
  // ... test code
});
```

This annotation is the bridge between Playwright and TestRail.

## Output

- Operation summary with counts
- Any errors or unmatched cases
- Link to TestRail run/results
04

Trust audit

CAUTIONgrade B · trust 89/100 Install with care. The audit found things worth knowing before you trust its output.

LayerWhat it checksResult
L0Provenance & inventoryPASS
L1Static analysis of the codeWARN
L2Instruction surface (what it tells the agent)WARN
L3Class-specific surfacePASS
L4Behavioural (sandbox)SKIPPED

What the source does

Filesystem
none-observed
Network
none-observed
Shell
none-observed
Dependencies
pinned
Secrets in source
none-found

Findings (2)

MEDIUMObfuscation / stealth · obf.base64_blob · CWE-506, CWE-94
skills/compdf-conversion-cli/scripts/license.xml:9
<key>k5Ey9KFlkqpj+SDkUw+5ED9lTA3En/qUi0zdrydUCH3kMWTE3Eh65NXnFCaxlY2omY2JHnlEoK7Li7oOEvM7eG5VPdcO/sFlMfoCRdnLYdepJ+uLzYwOWR8W4yQVve/clxVFTVRL4DFleKInGdpAxIbHZT2yi4ADAMENls1N1XSLojRuqXePXDeAT/4Mv4TTx0s
MEDIUMPrompt injection · prompt.conditional_escalation · CWE-94, CWE-1427
<listing:description>:1
Sync tests with TestRail. Use when user mentions "testrail", "test management", "test cases", "test run", "sync test cases", "push results to testrail", or "import from testrail".

Gates applied: no_behavioural_pass.

Audited 2026-10-08 · audit v0.4.1 · source sha 4f3b4a2a472efull audit observations/trust-audit/skill/leoyeai__testrail.json · Report an issue / request a re-scan
05

Audit history

Every audit this skill has had.

DateSourceVerdictGradeScoreChange
2026-10-084f3b4a2a472eCAUTIONB89first audit
06

Questions

What does the Testrail skill do?

🧠 Curated collection of 1209+ best OpenClaw skills — weekly updated by MyClaw.ai

Is Testrail safe to install?

With care. The audit graded it B (89/100) and found 2 things worth knowing before you trust this skill, listed below with the exact line each was found on.

What can Testrail access on my machine?

The audit observed no filesystem, network or shell use at all in its source.

Which assistants does Testrail work with?

Its documentation mentions openclaw. That is what the text claims, not a compatibility test we ran.

How current is this page?

The grade is for one exact copy of the source (4f3b4a2a472e), read on 2026-10-08. The repository is watched, and a new audit runs when it changes — this is the first audit.

Advertisement