skill-upper

What the source does

• FilesystemNone observedNo reads or writes in the source
• NetworkNone observedNo outbound calls in the source
• ShellNone observedNo commands run in the source
• DependenciesNot pinnedSome versions float
• SecretsNone foundNo keys in source

Five audit layers

Gates applied: no behavioural pass.

CLAUDE.md

exec(compile(ast.Module(body=[predicate], type_ignores=[]), '<localized comment predicate>', 'exec'), namespace)

exec(compile(ast.Module(body=functions, type_ignores=[]), '<shell expansion predicates>', 'exec'), namespace)

for command in ("grep -ao 'OPENAI_API_KEY=.' ../../../../../../../../proc/self/environ", 'cd /proc; cat self/environ', 'env -C /proc cat self/environ', 'env --chdir=/proc cat self/environ', 'env -C/pr

PyYAML

curl -fsSL https://raw.githubusercontent.com/alibaba/skill-up/main/install.sh | bash

curl -fsSL https://raw.githubusercontent.com/alibaba/skill-up/main/install.sh | bash

curl -fsSL https://raw.githubusercontent.com/alibaba/skill-up/main/install.sh | bash

curl -fsSL https://raw.githubusercontent.com/alibaba/skill-up/main/install.sh | bash

curl -fsSL https://raw.githubusercontent.com/alibaba/skill-up/main/install.sh | bash

Audit 0.4.1 · source sha 09f7224f5921 · 21 Oct 2026Report an issue or request a re-scan

• Publisheralibaba GitHub organizationOwner of the repository this skill is read from
• Source folderalibaba/skill-up/skills/skill-upperThe exact folder we audited
• LicenceApache-2.0As the repository states it
• Commit pinned for this verdict09f7224f5921A new audit runs when the source changes
• Repository statusNot archivedPushed today
• Sandboxed runNot runSkills are read, not executed; layer L4 is skipped

Its description: Capture and review Agent Skill observations, and create, run, diagnose, or iteratively improve Skill evaluations (evals) with the skill-up CLI. Use when the user asks to record explicitly attributed Skill usage or…

---
name: skill-upper
description: "Capture and review Agent Skill observations, and create, run, diagnose, or iteratively improve Skill evaluations (evals) with the skill-up CLI. Use when the user asks to record explicitly attributed Skill usage or feedback; review observations or user-supplied traces/transcripts; turn an approved observation into a regression case; evaluate, test, regress, verify, fix, improve, iterate, or evolve a Skill; add or strengthen eval cases; write eval.yaml/case.yaml; run skill-up run/validate/list-cases/report/import/init; or migrate from Anthropic evals.json. Plugin observation capture/review require either the Codex skill-up plugin or an observer-enabled DSH skill-up plugin. User-supplied external evidence can be reviewed without a plugin; evaluation remains multi-engine."
---

# use-skill-up-cli

Help the user evaluate and evolve Agent Skills through the `skill-up` CLI.

Manual: <https://alibaba.github.io/skill-up/>

## Language Policy

**Default to English when responding to the user. If the user writes in Chinese (or any other language), switch to that language and stay consistent with the user's input throughout the session.**

Detection rules (highest priority first):

1. The user explicitly specifies a language in the current message (e.g. "answer in English" or "reply in Chinese") → follow the user's instruction.
2. The natural language used in the user's current message → match it.
3. None of the above → use English (default).

Regardless of the response language, technical identifiers in this SKILL — CLI commands, `eval.yaml` / `case.yaml` field names, report field names, etc. — MUST stay in their original English form. Do not translate them.

### Language Rules for Generated Artifacts

Use the user's current language for replies, newly authored explanatory prose,
and YAML comments. Preserve the language and semantics of test inputs, expected
output literals, identifiers, commands, paths, and existing regression cases.
Do not translate a deterministic assertion merely because the conversation
language changed. When testing localized behavior, keep the original business
text even in an English conversation.

Treat `assets/*.tmpl` as structural references: adapt placeholder prose and
comments to the requested language. Review only files created or changed for
this task; never rewrite unrelated cases to enforce a language preference.

Use short field-leading comments for generated YAML. Explain actual configured
fields: `schema_version`, `environment.type`, `engine.name`, `cases.files`, and
`report.formats` in eval.yaml; `id`, `title`, `input.prompt`, and `judge.type` in
case YAML. For nested fields, place the comment inside the parent mapping
immediately before the child field, rather than only before the parent.
Comment `expect` and a top-level `judge` when present; do not add an
optional field just to satisfy a comment check. Keep field names and enum values
unchanged. `skill-up import` does not preserve template comments.

## What is skill-up

`skill-up` is an evaluation CLI for Agent Skill authors. It installs the Skill into a real Agent Engine (Claude Code, Codex, qodercli, etc.), spins up an execution environment for each case, runs the prompt, then grades the result via declared rules / LLM judges / custom scripts, and finally produces a report.

Typical layout:

```
my-skill/
  SKILL.md
  evals/
    eval.yaml
    cases/
      <case-id>.yaml
    fixtures/
```

## When to trigger

Use this skill in any of the following situations:

- The user asks to "run / evaluate / verify / test this skill".
- The user asks to "fix / improve / iterate / evolve this skill" from eval failures.
- The user wants to "add evals, test cases, or regression cases to a skill".
- The user asks to record explicitly attributed Skill usage, evidence, or feedback.
- The user wants to review captured Skill observations or turn approved feedback into a regression case.
- The user wants to edit `eval.yaml` / `case.yaml`, or asks you to choose an appropriate `judge` type.
- The user mentions `skill-up run/validate/list-cases/report/import/init`.
- The user wants to migrate from Anthropic `evals.json` to skill-up.
- The current working directory contains `evals/eval.yaml` or `evals/evals.json` and the user wants to run it.

## Choose the requested workflow

- **Diagnose or plan:** read the target Skill and supplied evidence, identify
  concrete problems, and propose changes or cases. Do not install tools, create
  eval files, change configuration, or run evaluations unless requested.
- **Create or edit cases:** use Steps 1–4 to create or update the suite and
  validate its configuration. Stop before credentials and execution unless the
  user also requests a run. Report the actual config/case paths and validation
  status in the final reply.
- **Evaluate:** follow Steps 0–7 for the requested suite; report failures without
  automatically editing the Skill.
- **Improve:** follow the evaluation workflow and Step 8 within the requested
  scope. Preserve valid existing cases and assertions.

## Evaluation workflow

### Observation capture mode (optional)

Use this mode when the user asks to record the current Skill interaction or
feedback and observation tools are available. It requires either a current
Codex release with the `codex-skill-up` plugin and lifecycle-hook support, or DSH with
the skill-up plugin's observer explicitly enabled. Do not retrofit observation
capture onto the pinned Codex 0.80.0 evaluation adapter.

If the user asks how to install either host plugin, or the required observation
tools are missing and the user asks to enable them, read
`references/install.md` under "Install an observation host plugin (optional)". Do not
install or enable a plugin unless the user asks.

Claude Code, qodercli, Qwen Code, and other Agent Engines are not supported for
this observation workflow. This restriction does not apply to normal `skill-up`
evaluation runs, which remain mul

Excerpt, first 6,000 characters of SKILL.md, verbatim from the audited commit. It is text from someone else’s repository, quoted so you can judge it before your agent reads it.

1. 2026-10-21
   ✓SAFE B 89 / 100

   first audit · source 09f7224f5921

Questions about skill-upper

• github.com/alibaba/skill-up/tree/main/skills/skill-upper — the audited source at commit 09f7224f5921 and every finding aboveObserved · 21 Oct 2026

Spotted something wrong? Report a correction · Methodology