Idea EvaluatorSAFE

将博导十年科研经验炼化为可直接调用的 AI 技能。从 Idea 构思到论文投稿,你的 AI 科研副导师。

The instruction file, verbatim from the audited commit — this is the text the model reads, and the surface the audit's instruction layer examines. Quoted here so you can judge it without cloning anything.

---
name: idea-evaluator
description: >-
  Evaluates a preliminary research idea against a five-dimension framework
  (Higher, Faster, Stronger, Cheaper, Broader) plus idea-lifecycle and
  student-capability matching, paradigm-shift probing, and a fatal-flaws
  audit. Returns a reviewer-style verdict; non-STEM ideas route to
  substitute frameworks. Use when the user has a draft research idea and
  asks whether it is worth pursuing, asks to 'evaluate this idea', 'score
  this idea', 'assess feasibility', 'novelty check', 'is this a good
  research direction', or before committing to a paper scope.
license: CC-BY-4.0
---

# Idea Evaluator

## Overview

This skill evaluates a preliminary research idea from the combined
perspective of a top-venue reviewer and an experienced advisor. It
scores the idea against five improvement dimensions from the
idea-generation guide (Higher, Faster, Stronger, Cheaper,
Broader), matches the idea's lifecycle against the user's actual
capability and available hours per week, probes whether the idea has
paradigm-shift potential, flags fatal flaws, and returns one of three
verdicts: Strong Accept, Accept with Revisions, or Reject and Pivot.

The goal is to kill weak ideas before the student invests months, and to
shape promising-but-underdeveloped ideas into stronger forms before
writing begins.

## When to use this skill

- The user has a draft idea and asks whether it is worth pursuing.
- The user asks for novelty check, feasibility assessment, or scoring.
- Before the user commits to a paper scope or starts implementation.
- The user is comparing two or three candidate ideas and needs a
  structured trade-off.
- The user suspects scope creep and wants an external check.
- The user mentions 'evaluate this idea', 'score this idea', 'assess
  feasibility', or 'is this a good research direction'.

## When NOT to use this skill

- The user has already implemented the idea and is writing the paper.
  Use `intro-drafter`, `tech-paper-template`, or
  `benchmark-paper-template` (separate plugin) instead.
- The user explicitly wants brainstorming of new ideas from scratch.
  Use plain conversation; see handbook 2.3 for a disruptive-innovation
  playbook.
- The user asks for review of an existing manuscript. Use
  `pre-submission-reviewer`.
- The user asks to evaluate a benchmark contribution specifically.
  Use `benchmark-paper-template` (separate plugin) in targeted mode.

## Core procedure

### Step 1: First impression and paper-type positioning

Read the user's idea description. In one paragraph, state whether the
idea reads as Novel Problem, Novel Method, or New Setting. Is the
story compelling in one sentence? If you cannot write that sentence,
the idea itself is probably not yet clear enough for evaluation; ask
the user to restate.

While restating, classify the research paradigm **by method, not by
department name**: experiments, benchmarks, ablations, or model
architectures mean STEM (continue with the five dimensions and fatal
flaws below); text analysis, archives, or conceptual argument mean
humanities; surveys, interviews, statistics, or fieldwork mean
empirical social science; regressions, IV, DID, RDD, or panel data
mean finance or economics; statutes, cases, or doctrine mean law.

See: references/domain-evaluation-frameworks.md for the substitute
five-dimension frameworks and fatal-flaw substitutions used by the
non-STEM paradigms. When the paradigm is unclear, ask one question:
"is the core method experiments, surveys, text analysis, or
theoretical derivation?"

### Step 2: Fatal-flaws audit (early gate)

See: references/fatal-flaws.md for the ten canonical fatal flaws,
each with a detection rule and a defense strategy.

Run the fatal-flaws audit **before** the scoring steps rather than
after them. Identify at most two fatal flaws. For each, state the
flaw, cite the detection rule, and recommend a concrete defense.

**Ground the novelty flaw (F1) in real retrieval whenever the
environment has a literature-search capability** (a scholarly search
tool, web search over scholarly indexes, or shell access to public
APIs). Extract two or three keyword groups from the idea (core method
plus domain; mechanism plus task; technique plus benchmark), search,
and name the three to five closest published works with title, authors,
and year. For each, state which axis actually differs: the object acted
on, the mechanism, the input granularity, or the problem setting. A
similar title alone never establishes duplication; duplication requires
failing to find even one differing axis. Retrieval results support
metadata-level judgments only (who did what, where); never quote
numbers or method details from search snippets. And "not found" does
not prove novelty: report it as "no directly overlapping work retrieved
under these keywords". With no retrieval capability, label the novelty
judgment "unverified; literature check required".

**A data-refuted core mechanism is an automatic CRITICAL.** If the
user's own reported data or attachments already show the core mechanism
matched or beaten by a baseline or simple control, lock the verdict to
Reject and Pivot; write no defense and invent no optimistic threshold.
See: references/fatal-flaws.md, the data-refuted section, for the exact
boundary between "refuted by data" and "merely untested".

**Short-circuit rule.** If any fatal flaw is tagged CRITICAL in the
severity taxonomy (single-handedly causes rejection, unfixable
within the lifecycle), stop here and emit the verdict directly:

- Verdict: Reject and Pivot.
- Output sections 1 (First impression), 2 (Fatal flaws with the
  CRITICAL flaw), and 7 (Verdict with the flaw-driven rationale)
  only.
- Do **not** run the five-dimension scoring, paradigm-shift probe,
  feasibility check, or integrity gate. Those would be decoration
  on a rejection.

If no CRITICAL flaw is found, continue to Step 3.

### Step 3: Lifecycle and capability matching

See: references/lifecycle-capability-

SAFEgrade B · trust 89/100 Nothing in the source contradicts what it says it does. Grade A is reserved for packages that have also passed the behavioural sandbox.

What the source does

Filesystem

none-observed

Network

none-observed

Shell

none-observed

Dependencies

pinned

Secrets in source

none-found

Findings (0)

No findings outside the package's declared scope.

Gates applied: no_behavioural_pass.

[![ModelsAtlas audit: Idea Evaluator](https://modelsatlas.com/skills/hkustdial/idea-evaluator/badge.svg)](https://modelsatlas.com/skills/hkustdial/idea-evaluator/)

Not your listing, or something out of date? Claim or update your listing.