Live survey304 models · 14 providers · 23,592 auditsAudited implement · SAFE →What changedHow we verifyllms.txt

Agent skill · Audited from source

eval-harness

skill · affaan-m/everything-claude-codeMIT licenceAudited 14 Nov 2026 · source 22362fa06126

★ 302,535pushed 4 days ago
✓SAFEGrade B · 89/100 · 14 Nov 2026 · 22362fa
302,535repo stars · whole affaan-m/everything-claude-code

Is it safe? Yes — SAFE, grade B (89/100). We read this skill's source at commit 22362fa06126 and found no filesystem, network or shell access and no hard-coded secrets. What it does: Eval-driven development (EDD) framework for AI coding sessions — define capability and regression evals before coding, grade with code-based, model-based, rule, or human graders, and track pass@k and pass^k reliability.…

Ask an AI about this pageGChatGPTCClaudePPerplexityGeGeminiXGrok
More
Cite

Cite this page

ModelsAtlas. “eval-harness (affaan-m/everything-claude-code)”. Retrieved 14 Nov 2026. https://modelsatlas.com/skills/affaan-m/eval-harness

2 skills are called “eval-harness”. You are viewing affaan-m/everything-claude-code.

Trust audit

Audit 0.4.1 · source read 14 Nov 2026

Trust score89/100✓SAFE B
0 critical0 high0 medium0 low0 info

SAFE means nothing in the source contradicts what the skill says it does. Grade B is shared by 17,406 of 18,389 audited skills.

What the source does

  • FilesystemNone observedNo reads or writes in the source
  • NetworkNone observedNo outbound calls in the source
  • ShellNone observedNo commands run in the source
  • DependenciesPinnedNothing floats
  • SecretsNone foundNo keys in source

Five audit layers

LayerWhat we checkResultFindings
L0 ProvenanceRepo owner, licence, archive status, activityPASS0
L1 Static sourceSecrets, path traversal, supply chain, hidden codeNA0
L2 InstructionsPrompt injection, authority framing, hidden directivesPASS0
L3 InventoryThe files and declarations match the sourcePASS0
L4 SandboxRun in isolation and watch behaviourSKIPPED why0

Gates applied: no behavioural pass.

Findings (0)

No findings at any severity. We re-audit when the source changes.

Audit 0.4.1 · source sha 22362fa06126 · 14 Nov 2026Report an issue or request a re-scan

Provenance

Who published it, and what the verdict is pinned to

  • Publisheraffaan-m GitHub userOwner of the repository this skill is read from
  • Source folderaffaan-m/everything-claude-code/skills/eval-harnessThe exact folder we audited
  • LicenceMITAs the repository states it
  • Commit pinned for this verdict22362fa06126A new audit runs when the source changes
  • Repository statusNot archivedLast push 4 days ago
  • Sandboxed runNot runSkills are read, not executed; layer L4 is skipped

What it tells the agent

Full source on GitHub ↗

Its description: Eval-driven development (EDD) framework for AI coding sessions — define capability and regression evals before coding, grade with code-based, model-based, rule, or human graders, and track pass@k and pass^k reliability.…

---
name: eval-harness
description: Eval-driven development (EDD) framework for AI coding sessions — define capability and regression evals before coding, grade with code-based, model-based, rule, or human graders, and track pass@k and pass^k reliability. Use when defining pass/fail criteria for agent tasks, measuring agent reliability, building regression suites for prompt or agent changes, or benchmarking across model versions.
metadata:
  origin: ECC
tools: Read, Write, Edit, Bash, Grep, Glob
---

# Eval Harness Skill

A formal evaluation framework for Claude Code sessions, implementing eval-driven development (EDD) principles.

## When to Activate

- Setting up eval-driven development (EDD) for AI-assisted workflows
- Defining pass/fail criteria for Claude Code task completion
- Measuring agent reliability with pass@k metrics
- Creating regression test suites for prompt or agent changes
- Benchmarking agent performance across model versions

## Philosophy

Eval-Driven Development treats evals as the "unit tests of AI development":
- Define expected behavior BEFORE implementation
- Run evals continuously during development
- Track regressions with each change
- Use pass@k metrics for reliability measurement

## Eval Types

### Capability Evals
Test if Claude can do something it couldn't before:
```markdown
[CAPABILITY EVAL: feature-name]
Task: Description of what Claude should accomplish
Success Criteria:
  - [ ] Criterion 1
  - [ ] Criterion 2
  - [ ] Criterion 3
Expected Output: Description of expected result
```

### Regression Evals
Ensure changes don't break existing functionality:
```markdown
[REGRESSION EVAL: feature-name]
Baseline: SHA or checkpoint name
Tests:
  - existing-test-1: PASS/FAIL
  - existing-test-2: PASS/FAIL
  - existing-test-3: PASS/FAIL
Result: X/Y passed (previously Y/Y)
```

## Grader Types

### 1. Code-Based Grader
Deterministic checks using code:
```bash
# Check if file contains expected pattern
grep -q "export function handleAuth" src/auth.ts && echo "PASS" || echo "FAIL"

# Check if tests pass
npm test -- --testPathPattern="auth" && echo "PASS" || echo "FAIL"

# Check if build succeeds
npm run build && echo "PASS" || echo "FAIL"
```

### 2. Model-Based Grader
Use Claude to evaluate open-ended outputs:
```markdown
[MODEL GRADER PROMPT]
Evaluate the following code change:
1. Does it solve the stated problem?
2. Is it well-structured?
3. Are edge cases handled?
4. Is error handling appropriate?

Score: 1-5 (1=poor, 5=excellent)
Reasoning: [explanation]
```

### 3. Human Grader
Flag for manual review:
```markdown
[HUMAN REVIEW REQUIRED]
Change: Description of what changed
Reason: Why human review is needed
Risk Level: LOW/MEDIUM/HIGH
```

## Metrics

### pass@k
"At least one success in k attempts"
- pass@1: First attempt success rate
- pass@3: Success within 3 attempts
- Typical target: pass@3 > 90%

### pass^k
"All k trials succeed"
- Higher bar for reliability
- pass^3: 3 consecutive successes
- Use for critical paths

## Eval Workflow

### 1. Define (Before Coding)
```markdown
## EVAL DEFINITION: feature-xyz

### Capability Evals
1. Can create new user account
2. Can validate email format
3. Can hash password securely

### Regression Evals
1. Existing login still works
2. Session management unchanged
3. Logout flow intact

### Success Metrics
- pass@3 > 90% for capability evals
- pass^3 = 100% for regression evals
```

### 2. Implement
Write code to pass the defined evals.

### 3. Evaluate
```bash
# Run capability evals
[Run each capability eval, record PASS/FAIL]

# Run regression evals
npm test -- --testPathPattern="existing"

# Generate report
```

### 4. Report
```markdown
EVAL REPORT: feature-xyz
========================

Capability Evals:
  create-user:     PASS (pass@1)
  validate-email:  PASS (pass@2)
  hash-password:   PASS (pass@1)
  Overall:         3/3 passed

Regression Evals:
  login-flow:      PASS
  session-mgmt:    PASS
  logout-flow:     PASS
  Overall:         3/3 passed

Metrics:
  pass@1: 67% (2/3)
  pass@3: 100% (3/3)

Status: READY FOR REVIEW
```

## Integration Patterns

### Pre-Implementation
```
/eval define feature-name
```
Creates eval definition file at `.claude/evals/feature-name.md`

### During Implementation
```
/eval check feature-name
```
Runs current evals and reports status

### Post-Implementation
```
/eval report feature-name
```
Generates full eval report

## Eval Storage

Store evals in project:
```
.claude/
  evals/
    feature-xyz.md      # Eval definition
    feature-xyz.log     # Eval run history
    baseline.json       # Regression baselines
```

## Best Practices

1. **Define evals BEFORE coding** - Forces clear thinking about success criteria
2. **Run evals frequently** - Catch regressions early
3. **Track pass@k over time** - Monitor reliability trends
4. **Use code graders when possible** - Deterministic > probabilistic
5. **Human review for security** - Never fully automate security checks
6. **Keep evals fast** - Slow evals don't get run
7. **Version evals with code** - Evals are first-class artifacts

## Example: Adding Authentication

```markdown
## EVAL: add-authentication

### Phase 1: Define (10 min)
Capability Evals:
- [ ] User can register with email/password
- [ ] User can login with valid credentials
- [ ] Invalid credentials rejected with proper error
- [ ] Sessions persist across page reloads
- [ ] Logout clears session

Regression Evals:
- [ ] Public routes still accessible
- [ ] API responses unchanged
- [ ] Database schema compatible

### Phase 2: Implement (varies)
[Write code]

### Phase 3: Evaluate
Run: /eval check add-authentication

### Phase 4: Report
EVAL REPORT: add-authentication
==============================
Capability: 5/5 passed (pass@3: 100%)
Regression: 3/3 passed (pass^3: 100%)
Status: SHIP IT
```

## Local Framework Utilities

The mechanical utilities ship in `scripts/lib/eval-harness/`:

```sh
node scripts/eval-harness.js example
```

- Capsule: hash-linked journal with five

Excerpt, first 6,000 characters of SKILL.md, verbatim from the audited commit. It is text from someone else’s repository, quoted so you can judge it before your agent reads it.

Install

Pinned to the audited commit 22362fa

Download folder ↗
git clone https://github.com/affaan-m/everything-claude-code
git -C everything-claude-code checkout 22362fa06126

The author documents no install command, so this uses the repository directly. Our verdict covers commit 22362fa06126 only.

Works in which agents?

What its own text says, not a test

AgentStatusMore
Claude CodeMentioned in its text, not testedSkills for Claude Code · About Claude Code
“Mentioned” means the skill’s text names the agent. We have not run it there.

Audit history

Every audit, every source change

  1. ✓SAFE B 89 / 100

    first audit · source 22362fa06126

README badge

For maintainers · shows the latest audit

ModelsAtlas: SAFE BMODELSATLASSAFE · B
[![ModelsAtlas: SAFE B](https://modelsatlas.com/skills/affaan-m/eval-harness/badge.svg)](https://modelsatlas.com/skills/affaan-m/eval-harness/)

The badge shows our verdict and grade for this skill, and links back to this page. It changes when the audit does.

ecc-recipes✓SAFE

affaan-m/everything-claude-code

Map a described workflow to the right ECC command group with run-order and stop condition, or browse all command-group recipe families read live from the commands directory. Advisory only — never executes. Use when…

★ 302kGrade B
dynamic-workflow-mode✓SAFE

affaan-m/everything-claude-code

Design task-local harnesses, eval gates, and reusable skill extraction for Claude dynamic workflow mode and other adaptive agent harnesses. Use when building a task-local harness, adding eval gates, or extracting a…

★ 302kGrade B
django-verification!CAUTION

affaan-m/everything-claude-code

Run the full Django verification loop — environment check, mypy/ruff/black linting, migration safety, pytest with coverage targets, pip-audit and bandit security scans, settings and logging review, and diff review —…

★ 294kGrade B
django-patterns✓SAFE

affaan-m/everything-claude-code

Django architecture patterns, REST API design with DRF, ORM best practices, caching, signals, middleware, and production-grade Django apps. Use when building or reviewing Django apps, DRF APIs, ORM queries, or caching.

★ 294kGrade B
django-security✓SAFE

affaan-m/everything-claude-code

Django security best practices, authentication, authorization, CSRF protection, SQL injection prevention, XSS prevention, and secure deployment configurations. Use when reviewing Django authentication, authorization,…

★ 294kGrade B
contract-first✓SAFE

affaan-m/everything-claude-code

Coordinate frontend/backend or service-to-service work through one authoritative machine-checkable contract (OpenAPI, AsyncAPI, Protocol Buffers, or JSON Schema), with generated consumer types and contract-verified…

★ 293kGrade B

Add this badge to your README

[![ModelsAtlas: SAFE B](https://modelsatlas.com/skills/affaan-m/eval-harness/badge.svg)](https://modelsatlas.com/skills/affaan-m/eval-harness/)

Questions about eval-harness

What does the eval-harness skill do?
The agent harness performance optimization system. Skills, instincts, memory, security, and research-first development for Claude Code, Codex, Opencode, Cursor and beyond.
Is eval-harness safe to install?
The audit found nothing in the source that contradicts what it says it does, and graded it B (89/100). Grade A is held back for packages that have also passed a sandboxed behavioural run, which is why a clean skill reads B.
What can eval-harness access on my machine?
The audit observed no filesystem, network or shell use at all in its source.
Which assistants does eval-harness work with?
Its documentation mentions claude-code. That is what the text claims, not a compatibility test we ran.
How current is this page?
The grade is for one exact copy of the source (22362fa06126), read on 2026-11-14. The repository is watched, and a new audit runs when it changes — this is the first audit.

Sources

Spotted something wrong? Report a correction · Methodology

Keep going in your AI

Opens with this page as the source. We add nothing else to the prompt.