Live survey304 models · 14 providers · 23,592 auditsAudited implement · SAFE →What changedHow we verifyllms.txt

Agent skill · Audited from source

PaperOrchestra

skill · Ar9av/PaperOrchestraNOASSERTION licenceAudited 12 Nov 2026 · source eb2a731c706f

★ 701pushed 8 days ago
!CAUTIONGrade C · 79/100 · 7 findings · eb2a731
701repo stars · whole Ar9av/PaperOrchestra

Is it safe? With care — CAUTION, grade C (79/100). We read this skill's source at commit eb2a731c706f and observed filesystem and network access and no hard-coded secrets. The audit raised 7 findings, 1 critical or high. What it does: Pre-pipeline aggregator that scans AI agent cache directories (.claude, .cursor, .antigravity, .openclaw) or any user-specified directory for experimentation logs, extracts insights and numeric results, and formats them…

Ask an AI about this pageGChatGPTCClaudePPerplexityGeGeminiXGrok
More
Cite

Cite this page

ModelsAtlas. “PaperOrchestra (Ar9av/PaperOrchestra)”. Retrieved 12 Nov 2026. https://modelsatlas.com/skills/ar9av/paperorchestra

Trust audit

Audit 0.4.1 · source read 12 Nov 2026

Trust score79/100!CAUTION C
0 critical1 high0 medium5 low1 info

CAUTION means the audit found things worth knowing before you trust this skill. Grade C is shared by 105 of 18,389 audited skills.

What the source does

  • FilesystemDeclared1 observation
  • NetworkDeclared5 observations
  • ShellNone observedNo commands run in the source
  • DependenciesNot pinnedSome versions float
  • SecretsNone foundNo keys in source

Five audit layers

LayerWhat we checkResultFindings
L0 ProvenanceRepo owner, licence, archive status, activityPASS1
L1 Static sourceSecrets, path traversal, supply chain, hidden codePASS2
L2 InstructionsPrompt injection, authority framing, hidden directivesFAIL4
L3 InventoryThe files and declarations match the sourcePASS0
L4 SandboxRun in isolation and watch behaviourSKIPPED why0

Gates applied: instruction override, no behavioural pass.

Findings (7)

HIGHPrompt injectionprompt.override · examples/agentic-security-report/inputs/idea.md:25
F-01 Prompt injectionCWE-94 · CWE-1427Layer L2
A signed advisory feed (`immunity-feed.json`, 217 advisories as of March 2026) with Ed25519 integrity verification provides runtime correlation. The feed is generated by an automated NVD-fetch pipelin
Why
asks the agent to drop prior instructions or safety
Fix
remove the instruction
LOWObfuscation / stealthobf.decode_call · skills/plotting-agent/scripts/paperbanana_render.py:225
F-07 Obfuscation / stealthCWE-506 · CWE-94Layer L1
img_bytes = base64.b64decode(b64_jpeg)
Why
No further detail recorded for this rule.
Fix
—
LOWSupply chainsupply.unpinned · requirements.txt
F-08 Supply chainCWE-829 · CWE-1357Layer L1
jsonschema, python-Levenshtein, reportlab, matplotlib, pypdf
Why
5 requirement(s) not pinned with ==
Fix
pin exact versions
LOWPrompt injectionprompt.authority_framing · skills/outline-agent/references/prompt.md:12
F-01 Prompt injectionCWE-94 · CWE-1427Layer L2
You are a senior AI researcher drafting a paper for a top-tier conference
Why
No further detail recorded for this rule.
Fix
—
LOWPrompt injectionprompt.authority_framing · skills/paper-writing-bench/references/dense-idea-prompt.md:11
F-01 Prompt injectionCWE-94 · CWE-1427Layer L2
You are a Lead Research Scientist planning a new project. Your task is to
Why
No further detail recorded for this rule.
Fix
—
LOWPrompt injectionprompt.credential_read · examples/agentic-security-report/inputs/idea.md:3
F-01 Prompt injectionCWE-94 · CWE-1427Layer L2
AI coding agents (Claude Code, Cursor, Windsurf, GitHub Copilot) have become the dominant way software is written in 2025-2026. These systems are granted broad filesystem access, execute arbitrary she
Why
asks the agent to read credentials
Fix
—
INFOInventory / provenanceinv.oversize · examples/agentic-security-report/workspace/final/paper.pdf
F-13 Inventory / provenanceCWE-1104Layer L0
examples/agentic-security-report/workspace/final/paper.pdf
Why
1502113 bytes not read
Fix
—

Audit 0.4.1 · source sha eb2a731c706f · 12 Nov 2026Report an issue or request a re-scan

Provenance

Who published it, and what the verdict is pinned to

  • PublisherAr9av GitHub userOwner of the repository this skill is read from
  • Source folderAr9av/PaperOrchestraThe exact folder we audited
  • LicenceNOASSERTIONAs the repository states it
  • Commit pinned for this verdicteb2a731c706fA new audit runs when the source changes
  • Repository statusNot archivedLast push 8 days ago
  • Sandboxed runNot runSkills are read, not executed; layer L4 is skipped

What it tells the agent

Full source on GitHub ↗

Its description: Pre-pipeline aggregator that scans AI agent cache directories (.claude, .cursor, .antigravity, .openclaw) or any user-specified directory for experimentation logs, extracts insights and numeric results, and formats them…

---
name: agent-research-aggregator
description: Pre-pipeline aggregator that scans AI agent cache directories (.claude, .cursor, .antigravity, .openclaw) or any user-specified directory for experimentation logs, extracts insights and numeric results, and formats them as PaperOrchestra-ready inputs (idea.md + experimental_log.md). TRIGGER when the user says "aggregate my agent logs for paper writing", "extract experiments from my coding agent history", "prepare PaperOrchestra inputs from my cache", "turn my agent logs into a paper", mentions a folder or directory they want to use as the basis for a paper, or wants to run PaperOrchestra but only has scattered agent experiment histories rather than structured inputs. Run this BEFORE paper-orchestra. Also called automatically by paper-orchestra when workspace/inputs/idea.md or workspace/inputs/experimental_log.md are missing.
---

# agent-research-aggregator

---

## Should I run? (decision gate)

Before starting Phase 1, check whether aggregation is actually needed:

| Situation | Action |
|---|---|
| `workspace/inputs/idea.md` **and** `workspace/inputs/experimental_log.md` both exist and are non-empty | **Skip this skill entirely.** Proceed directly to `paper-orchestra`. |
| Either file is missing or empty, **and** the user provided a directory path | **Run this skill** with that directory as `--search-roots`. |
| Either file is missing or empty, **and** no directory was provided | Scan cwd and `~` by default; show the discovery summary to the user before continuing. |
| The inputs exist but look thin (e.g. idea.md has < 5 lines, no numeric data in experimental_log.md) | **Ask the user** whether to supplement with aggregation or proceed as-is. |

The skill is intentionally a pre-pass — it is cheap to skip and should only run when the structured inputs don't already exist.

---

A pre-processing skill for PaperOrchestra (arXiv:2604.05018). Reads scattered
experimentation artifacts from AI coding-agent cache directories and synthesizes
them into the structured `(I, E)` input pair the PaperOrchestra pipeline expects.

```
[.claude/]  [.cursor/]  [.antigravity/]  [.openclaw/]
      │            │              │               │
      └────────────┴──────────────┴───────────────┘
                          │
                    Phase 1: Discovery
                  (discover_logs.py)
                          │
                    discovered_logs.json
                          │
                    Phase 2: Extraction
                  (LLM call per log batch)
                          │
                    raw_experiments.json
                          │
                    Phase 3: Synthesis
                  (LLM call — consolidate)
                          │
                    synthesis.json
                          │
                    Phase 4: Formatting
                  (format_po_inputs.py)
                          │
             ┌────────────┴────────────┐
      workspace/inputs/         workspace/ara/
        idea.md                   aggregation_report.md
        experimental_log.md       discovered_logs.json
                                  raw_experiments.json
                                  synthesis.json
```

The output drops directly into `workspace/inputs/` so the user can immediately
run `paper-orchestra` on the same workspace.

---

## Inputs

| Parameter | Required | Default | Description |
|---|---|---|---|
| `--search-roots` | no | cwd, `~` | Comma-separated directories to scan for agent caches |
| `--agents` | no | all | Comma-separated subset: `claude,cursor,antigravity,openclaw` |
| `--workspace` | no | `./workspace` | PaperOrchestra workspace root |
| `--depth` | no | 4 | Max directory scan depth (prevents runaway scans on large home dirs) |
| `--since` | no | none | Only include logs modified after this date (ISO 8601: `2025-01-01`) |

The user specifies these when invoking the skill, or you may ask them for
`--search-roots` if the current directory has no detectable agent caches.

---

## Phase 1 — Discovery (deterministic)

Run the discovery script to catalog every relevant log file:

```bash
python skills/agent-research-aggregator/scripts/discover_logs.py \
    --search-roots <roots> \
    --agents <agents> \
    --depth <depth> \
    --since <since> \
    --out workspace/ara/discovered_logs.json
```

The script exits with code **2** when no `--project` filter is set (this is
expected on the first run). It prints a **"Projects found"** list to stdout —
show it to the user immediately.

**If no logs are found at all:** stop and ask the user to specify
`--search-roots` or point you at a directory that contains agent cache folders.

---

## Phase 1.5 — Project Selection (mandatory)

**A paper can only be written from a single project. You must ask the user
which project to use before any LLM processing begins.**

1. Display the numbered project list from the discovery summary, e.g.:
   ```
   Projects found:
     [1] /home/alice/projects/my-rl-experiment  (42 files)
     [2] /home/alice/projects/llm-eval-suite    (17 files)
     [3] /home/alice/projects/old-demo          (3 files)
   ```
2. Ask: *"Which project should this paper be based on? Please choose a number
   or paste the project path."*
3. **Do not proceed to Phase 2 until the user has answered.**
4. Re-run discovery with the chosen project to filter the manifest:

```bash
python skills/agent-research-aggregator/scripts/discover_logs.py \
    --search-roots <roots> \
    --agents <agents> \
    --depth <depth> \
    --since <since> \
    --project "<chosen project path>" \
    --out workspace/ara/discovered_logs.json
```

This overwrites `discovered_logs.json` so only the selected project's files
remain. The script exits 0 on success.

**If the discovery finds only one project:** skip the question and inform the
user: *"Only one project found: `<path>`. Using it for the paper."* — then
re-run with `--project` automatically.

**If the discovery summary 

Excerpt, first 6,000 characters of SKILL.md, verbatim from the audited commit. It is text from someone else’s repository, quoted so you can judge it before your agent reads it.

Install

Pinned to the audited commit eb2a731

Download folder ↗
git clone <this repo> ~/paper-orchestra
pip install -r requirements.txt   # deterministic helpers only
git clone https://github.com/dwzhu-pku/PaperBanana
pip install -r requirements.txt
git clone https://github.com/<you>/paper-orchestra ~/paper-orchestra

Commands as the repository documents them: shown, never run by us. Our verdict covers commit eb2a731c706f only.

Works in which agents?

What its own text says, not a test

AgentStatusMore
Claude CodeMentioned in its text, not testedSkills for Claude Code · About Claude Code
CodexMentioned in its text, not testedSkills for Codex · About Codex
GitHub CopilotMentioned in its text, not testedSkills for GitHub Copilot · About GitHub Copilot
CursorMentioned in its text, not testedSkills for Cursor · About Cursor
OpenClawMentioned in its text, not testedSkills for OpenClaw · About OpenClaw
WindsurfMentioned in its text, not testedSkills for Windsurf · About Windsurf
“Mentioned” means the skill’s text names the agent. We have not run it there.

Audit history

Every audit, every source change

  1. !CAUTION C 79 / 100

    first audit · source eb2a731c706f

README badge

For maintainers · shows the latest audit

ModelsAtlas: CAUTION CMODELSATLASCAUTION · C
[![ModelsAtlas: CAUTION C](https://modelsatlas.com/skills/ar9av/paperorchestra/badge.svg)](https://modelsatlas.com/skills/ar9av/paperorchestra/)

The badge shows our verdict and grade for this skill, and links back to this page. It changes when the audit does.

Agent Research Aggregator✓SAFE

Ar9av/PaperOrchestra

Pre-pipeline aggregator that scans AI agent cache directories (.claude, .cursor, .antigravity, .openclaw) or any user-specified directory for experimentation logs, extracts insights and numeric results, and formats them…

★ 682Grade B
Content Refinement Agent✓SAFE

Ar9av/PaperOrchestra

Step 5 of the PaperOrchestra pipeline (arXiv:2604.05018). Iteratively refine drafts/paper.tex by simulating peer review and applying targeted revisions, with strict accept/revert halt rules, deterministic 0-100 decision…

★ 682Grade B
Literature Review Agent✓SAFE

Ar9av/PaperOrchestra

Step 3 of the PaperOrchestra pipeline (arXiv:2604.05018). Execute the literature search strategy from outline.json — discover candidate papers via web search, verify them through Semantic Scholar (Levenshtein > 70 fuzzy…

★ 682Grade B
Outline Agent✓SAFE

Ar9av/PaperOrchestra

Step 1 of the PaperOrchestra pipeline (arXiv:2604.05018). Convert (idea.md, experimental_log.md, template.tex, conference_guidelines.md) into a strict JSON outline containing a plotting plan, literature search plan…

★ 682Grade B
Paper Autoraters✓SAFE

Ar9av/PaperOrchestra

Run the four paper-quality autoraters from PaperOrchestra (arXiv:2604.05018, App. F.3) — Citation F1 (P0/P1 partition + Precision/Recall/F1), Literature Review Quality (6-axis 0-100 with anti-inflation rules), SxS…

★ 682Grade B
Paper Orchestra✓SAFE

Ar9av/PaperOrchestra

Orchestrate the full PaperOrchestra (Song et al., 2026, arXiv:2604.05018) five-agent pipeline to turn unstructured research materials (idea, experimental log, LaTeX template, conference guidelines, optional figures)…

★ 682Grade B

Add this badge to your README

[![ModelsAtlas: CAUTION C](https://modelsatlas.com/skills/ar9av/paperorchestra/badge.svg)](https://modelsatlas.com/skills/ar9av/paperorchestra/)

Questions about PaperOrchestra

What does the PaperOrchestra skill do?
An automated AI research-paper writer based off Google's PaperOrchestra paper's implementation through a skills - benchmark + autoraters using any coding agent (Claude Code, Cursor, Antigravity, Cline, Aider). No API keys, no LLM SDKs.
Is PaperOrchestra safe to install?
With care. The audit graded it C (79/100) and found 7 things worth knowing before you trust this skill, listed below with the exact line each was found on.
What can PaperOrchestra access on my machine?
The audit observed that it reaches the network and reads or writes files. Each of those is consistent with what it says it does. Secrets in the source: none found.
Which assistants does PaperOrchestra work with?
Its documentation mentions claude-code, codex, copilot, cursor, openclaw and windsurf. That is what the text claims, not a compatibility test we ran.
How current is this page?
The grade is for one exact copy of the source (eb2a731c706f), read on 2026-11-12. The repository is watched, and a new audit runs when it changes — this is the first audit.

Sources

Spotted something wrong? Report a correction · Methodology

Keep going in your AI

Opens with this page as the source. We add nothing else to the prompt.