Dedupe Rank

What the source does

• FilesystemNone observedNo reads or writes in the source
• NetworkNone observedNo outbound calls in the source
• ShellNone observedNo commands run in the source
• DependenciesPinnedNothing floats
• SecretsNone foundNo keys in source

Five audit layers

Gates applied: no behavioural pass, no license.

.claude/skills

Audit 0.4.1 · source sha 81994516a430 · 19 Oct 2026Report an issue or request a re-scan

• PublisherWILLOSCAR GitHub userOwner of the repository this skill is read from
• Source folderWILLOSCAR/research-units-pipeline-skills/.codex/skills/dedupe-rankThe exact folder we audited
• LicenceNot declaredAs the repository states it
• Commit pinned for this verdict81994516a430A new audit runs when the source changes
• Repository statusNot archivedLast push 14 days ago
• Sandboxed runNot runSkills are read, not executed; layer L4 is skipped

Its description: Use when a broad paper candidate pool needs deterministic deduplication and a stable core set.

---
name: dedupe-rank
description: "Use when a broad paper candidate pool needs deterministic deduplication and a stable core set."
---

# Dedupe + Rank

## Triggers & routing

- **Trigger**: dedupe, rank, core set, 去重, 排序, 精选论文, 核心集合.
- **Use when**: 检索后需要把广覆盖集合收敛成可管理的 core set(用于 taxonomy/outline/mapping)。
- **Skip if**: 已经有人手工整理了稳定的 `papers/core_set.csv`(无需再次 churn)。
- **Network**: none.
- **Guardrail**: 偏 deterministic;输出应可重复(稳定 paper_id、字段规范)。


Turns a raw candidate pool into a deduped pool and a stable core set.

## Input

- `papers/papers_raw.jsonl`

## Outputs

- `papers/papers_dedup.jsonl`
- `papers/core_set.csv`

## Script boundary

`scripts/run.py` should own only:
- title/year deduplication
- deterministic ranking
- stable `paper_id` generation

Use shared domain packs or pipeline contract metadata for topic-specific or product-specific behavior.

## Contract-driven behavior

The script should prefer pipeline contract metadata over profile-name branching.

Current important field:
- `quality_contract.candidate_pool_policy.keep_full_deduped_pool`
- `quality_contract.candidate_pool_policy.include_domain_pins`
- `quality_contract.candidate_pool_policy.minimum_domain_surveys`
- `quality_contract.candidate_pool_policy.survey_title_bonus`

If true, the script keeps the full deduped pool in `papers/core_set.csv` unless the user explicitly overrides core size.
Focused briefing Workflows can disable unconditional domain pins, reserve a
smaller survey floor, and remove the generic survey-title bonus without
changing broad Survey defaults.

## Acceptance

- deduped JSONL exists
- core-set CSV exists
- reruns are stable for the same inputs

## Non-goals

- retrieval
- screening
- manual topic authoring inside the script

Excerpt, first 1,750 characters of SKILL.md, verbatim from the audited commit. It is text from someone else’s repository, quoted so you can judge it before your agent reads it.

git clone https://github.com/WILLOSCAR/research-units-pipeline-skills
git -C research-units-pipeline-skills checkout 81994516a430

Works in

1. 2026-10-19
   ✓SAFE B 89 / 100

   first audit · source 81994516a430

Questions about Dedupe Rank

• github.com/WILLOSCAR/research-units-pipeline-skills/tree/main/.c — the audited source at commit 81994516a430 and every finding aboveObserved · 19 Oct 2026

Spotted something wrong? Report a correction · Methodology