Agent skill · Audited from source
eval-harness
skill · affaan-m/everything-claude-codeMIT licenceAudited 14 Nov 2026 · source 22362fa06126
Is it safe? Yes — SAFE, grade B (89/100). We read this skill's source at commit 22362fa06126 and found no filesystem, network or shell access and no hard-coded secrets. What it does: Eval-driven development (EDD) framework for AI coding sessions — define capability and regression evals before coding, grade with code-based, model-based, rule, or human graders, and track pass@k and pass^k reliability.…
Cite
Cite this page
ModelsAtlas. “eval-harness (affaan-m/everything-claude-code)”. Retrieved 14 Nov 2026. https://modelsatlas.com/skills/affaan-m/eval-harness
2 skills are called “eval-harness”. You are viewing affaan-m/everything-claude-code.
- affaan-m/everything-claude-code★ 303k✓SAFE
- sangrokjung/claude-forge★ 855✓SAFE
Trust audit
Audit 0.4.1 · source read 14 Nov 2026
SAFE means nothing in the source contradicts what the skill says it does. Grade B is shared by 17,406 of 18,389 audited skills.
What the source does
- FilesystemNone observedNo reads or writes in the source
- NetworkNone observedNo outbound calls in the source
- ShellNone observedNo commands run in the source
- DependenciesPinnedNothing floats
- SecretsNone foundNo keys in source
Five audit layers
| Layer | What we check | Result | Findings |
|---|---|---|---|
L0 Provenance | Repo owner, licence, archive status, activity | PASS | 0 |
L1 Static source | Secrets, path traversal, supply chain, hidden code | NA | 0 |
L2 Instructions | Prompt injection, authority framing, hidden directives | PASS | 0 |
L3 Inventory | The files and declarations match the source | PASS | 0 |
L4 Sandbox | Run in isolation and watch behaviour | SKIPPED why | 0 |
Gates applied: no behavioural pass.
Findings (0)
No findings at any severity. We re-audit when the source changes.
Audit 0.4.1 · source sha 22362fa06126 · 14 Nov 2026Report an issue or request a re-scan
Provenance
Who published it, and what the verdict is pinned to
- Publisheraffaan-m GitHub userOwner of the repository this skill is read from
- Source folderaffaan-m/everything-claude-code/skills/eval-harnessThe exact folder we audited
- LicenceMITAs the repository states it
- Commit pinned for this verdict22362fa06126A new audit runs when the source changes
- Repository statusNot archivedLast push 4 days ago
- Sandboxed runNot runSkills are read, not executed; layer L4 is skipped
What it tells the agent
Full source on GitHub ↗Its description: Eval-driven development (EDD) framework for AI coding sessions — define capability and regression evals before coding, grade with code-based, model-based, rule, or human graders, and track pass@k and pass^k reliability.…
---
name: eval-harness
description: Eval-driven development (EDD) framework for AI coding sessions — define capability and regression evals before coding, grade with code-based, model-based, rule, or human graders, and track pass@k and pass^k reliability. Use when defining pass/fail criteria for agent tasks, measuring agent reliability, building regression suites for prompt or agent changes, or benchmarking across model versions.
metadata:
origin: ECC
tools: Read, Write, Edit, Bash, Grep, Glob
---
# Eval Harness Skill
A formal evaluation framework for Claude Code sessions, implementing eval-driven development (EDD) principles.
## When to Activate
- Setting up eval-driven development (EDD) for AI-assisted workflows
- Defining pass/fail criteria for Claude Code task completion
- Measuring agent reliability with pass@k metrics
- Creating regression test suites for prompt or agent changes
- Benchmarking agent performance across model versions
## Philosophy
Eval-Driven Development treats evals as the "unit tests of AI development":
- Define expected behavior BEFORE implementation
- Run evals continuously during development
- Track regressions with each change
- Use pass@k metrics for reliability measurement
## Eval Types
### Capability Evals
Test if Claude can do something it couldn't before:
```markdown
[CAPABILITY EVAL: feature-name]
Task: Description of what Claude should accomplish
Success Criteria:
- [ ] Criterion 1
- [ ] Criterion 2
- [ ] Criterion 3
Expected Output: Description of expected result
```
### Regression Evals
Ensure changes don't break existing functionality:
```markdown
[REGRESSION EVAL: feature-name]
Baseline: SHA or checkpoint name
Tests:
- existing-test-1: PASS/FAIL
- existing-test-2: PASS/FAIL
- existing-test-3: PASS/FAIL
Result: X/Y passed (previously Y/Y)
```
## Grader Types
### 1. Code-Based Grader
Deterministic checks using code:
```bash
# Check if file contains expected pattern
grep -q "export function handleAuth" src/auth.ts && echo "PASS" || echo "FAIL"
# Check if tests pass
npm test -- --testPathPattern="auth" && echo "PASS" || echo "FAIL"
# Check if build succeeds
npm run build && echo "PASS" || echo "FAIL"
```
### 2. Model-Based Grader
Use Claude to evaluate open-ended outputs:
```markdown
[MODEL GRADER PROMPT]
Evaluate the following code change:
1. Does it solve the stated problem?
2. Is it well-structured?
3. Are edge cases handled?
4. Is error handling appropriate?
Score: 1-5 (1=poor, 5=excellent)
Reasoning: [explanation]
```
### 3. Human Grader
Flag for manual review:
```markdown
[HUMAN REVIEW REQUIRED]
Change: Description of what changed
Reason: Why human review is needed
Risk Level: LOW/MEDIUM/HIGH
```
## Metrics
### pass@k
"At least one success in k attempts"
- pass@1: First attempt success rate
- pass@3: Success within 3 attempts
- Typical target: pass@3 > 90%
### pass^k
"All k trials succeed"
- Higher bar for reliability
- pass^3: 3 consecutive successes
- Use for critical paths
## Eval Workflow
### 1. Define (Before Coding)
```markdown
## EVAL DEFINITION: feature-xyz
### Capability Evals
1. Can create new user account
2. Can validate email format
3. Can hash password securely
### Regression Evals
1. Existing login still works
2. Session management unchanged
3. Logout flow intact
### Success Metrics
- pass@3 > 90% for capability evals
- pass^3 = 100% for regression evals
```
### 2. Implement
Write code to pass the defined evals.
### 3. Evaluate
```bash
# Run capability evals
[Run each capability eval, record PASS/FAIL]
# Run regression evals
npm test -- --testPathPattern="existing"
# Generate report
```
### 4. Report
```markdown
EVAL REPORT: feature-xyz
========================
Capability Evals:
create-user: PASS (pass@1)
validate-email: PASS (pass@2)
hash-password: PASS (pass@1)
Overall: 3/3 passed
Regression Evals:
login-flow: PASS
session-mgmt: PASS
logout-flow: PASS
Overall: 3/3 passed
Metrics:
pass@1: 67% (2/3)
pass@3: 100% (3/3)
Status: READY FOR REVIEW
```
## Integration Patterns
### Pre-Implementation
```
/eval define feature-name
```
Creates eval definition file at `.claude/evals/feature-name.md`
### During Implementation
```
/eval check feature-name
```
Runs current evals and reports status
### Post-Implementation
```
/eval report feature-name
```
Generates full eval report
## Eval Storage
Store evals in project:
```
.claude/
evals/
feature-xyz.md # Eval definition
feature-xyz.log # Eval run history
baseline.json # Regression baselines
```
## Best Practices
1. **Define evals BEFORE coding** - Forces clear thinking about success criteria
2. **Run evals frequently** - Catch regressions early
3. **Track pass@k over time** - Monitor reliability trends
4. **Use code graders when possible** - Deterministic > probabilistic
5. **Human review for security** - Never fully automate security checks
6. **Keep evals fast** - Slow evals don't get run
7. **Version evals with code** - Evals are first-class artifacts
## Example: Adding Authentication
```markdown
## EVAL: add-authentication
### Phase 1: Define (10 min)
Capability Evals:
- [ ] User can register with email/password
- [ ] User can login with valid credentials
- [ ] Invalid credentials rejected with proper error
- [ ] Sessions persist across page reloads
- [ ] Logout clears session
Regression Evals:
- [ ] Public routes still accessible
- [ ] API responses unchanged
- [ ] Database schema compatible
### Phase 2: Implement (varies)
[Write code]
### Phase 3: Evaluate
Run: /eval check add-authentication
### Phase 4: Report
EVAL REPORT: add-authentication
==============================
Capability: 5/5 passed (pass@3: 100%)
Regression: 3/3 passed (pass^3: 100%)
Status: SHIP IT
```
## Local Framework Utilities
The mechanical utilities ship in `scripts/lib/eval-harness/`:
```sh
node scripts/eval-harness.js example
```
- Capsule: hash-linked journal with fiveExcerpt, first 6,000 characters of SKILL.md, verbatim from the audited commit. It is text from someone else’s repository, quoted so you can judge it before your agent reads it.
Install
Pinned to the audited commit 22362fa
git clone https://github.com/affaan-m/everything-claude-code git -C everything-claude-code checkout 22362fa06126
The author documents no install command, so this uses the repository directly. Our verdict covers commit 22362fa06126 only.
Works in which agents?
What its own text says, not a test
| Agent | Status | More |
|---|---|---|
| Claude Code | Mentioned in its text, not tested | Skills for Claude Code · About Claude Code |
Audit history
Every audit, every source change
- ✓SAFE B 89 / 100
first audit · source
22362fa06126
README badge
For maintainers · shows the latest audit
[](https://modelsatlas.com/skills/affaan-m/eval-harness/)
The badge shows our verdict and grade for this skill, and links back to this page. It changes when the audit does.
Similar skills
All coding & review skills →affaan-m/everything-claude-code
Map a described workflow to the right ECC command group with run-order and stop condition, or browse all command-group recipe families read live from the commands directory. Advisory only — never executes. Use when…
affaan-m/everything-claude-code
Design task-local harnesses, eval gates, and reusable skill extraction for Claude dynamic workflow mode and other adaptive agent harnesses. Use when building a task-local harness, adding eval gates, or extracting a…
affaan-m/everything-claude-code
Run the full Django verification loop — environment check, mypy/ruff/black linting, migration safety, pytest with coverage targets, pip-audit and bandit security scans, settings and logging review, and diff review —…
affaan-m/everything-claude-code
Django architecture patterns, REST API design with DRF, ORM best practices, caching, signals, middleware, and production-grade Django apps. Use when building or reviewing Django apps, DRF APIs, ORM queries, or caching.
affaan-m/everything-claude-code
Django security best practices, authentication, authorization, CSRF protection, SQL injection prevention, XSS prevention, and secure deployment configurations. Use when reviewing Django authentication, authorization,…
affaan-m/everything-claude-code
Coordinate frontend/backend or service-to-service work through one authoritative machine-checkable contract (OpenAPI, AsyncAPI, Protocol Buffers, or JSON Schema), with generated consumer types and contract-verified…
Add this badge to your README
[](https://modelsatlas.com/skills/affaan-m/eval-harness/)
<a href="https://modelsatlas.com/skills/affaan-m/eval-harness/"><img src="https://modelsatlas.com/skills/affaan-m/eval-harness/badge.svg" alt="ModelsAtlas: SAFE B"></a>
Questions about eval-harness
What does the eval-harness skill do?
Is eval-harness safe to install?
What can eval-harness access on my machine?
Which assistants does eval-harness work with?
How current is this page?
Sources
- github.com/affaan-m/everything-claude-code/tree/main/skills/eval — the audited source at commit 22362fa06126 and every finding aboveObserved · 14 Nov 2026
Spotted something wrong? Report a correction · Methodology
Keep going in your AI
Opens with this page as the source. We add nothing else to the prompt.
- What does the eval-harness skill do?
- Is eval-harness safe to install?
- What can eval-harness access on my machine?