Agent skill · Audited from source
constitutional-ai
skill · orchestra-research/ai-research-skillsMIT licenceAudited 13 Nov 2026 · source 6c04b84d00a4
Is it safe? Yes — SAFE, grade B (89/100). We read this skill's source at commit 6c04b84d00a4 and found no filesystem, network or shell access and no hard-coded secrets. What it does: Anthropic's method for training harmless AI through self-improvement. Two-phase approach - supervised learning with self-critique/revision, then RLAIF (RL from AI Feedback). Use for safety alignment, reducing harmful…
Cite
Cite this page
ModelsAtlas. “constitutional-ai (orchestra-research/ai-research-skills)”. Retrieved 13 Nov 2026. https://modelsatlas.com/skills/orchestra-research/constitutional-ai
Trust audit
Audit 0.4.1 · source read 13 Nov 2026
SAFE means nothing in the source contradicts what the skill says it does. Grade B is shared by 17,406 of 18,389 audited skills.
What the source does
- FilesystemNone observedNo reads or writes in the source
- NetworkNone observedNo outbound calls in the source
- ShellNone observedNo commands run in the source
- DependenciesPinnedNothing floats
- SecretsNone foundNo keys in source
Five audit layers
| Layer | What we check | Result | Findings |
|---|---|---|---|
L0 Provenance | Repo owner, licence, archive status, activity | PASS | 0 |
L1 Static source | Secrets, path traversal, supply chain, hidden code | NA | 0 |
L2 Instructions | Prompt injection, authority framing, hidden directives | PASS | 0 |
L3 Inventory | The files and declarations match the source | PASS | 0 |
L4 Sandbox | Run in isolation and watch behaviour | SKIPPED why | 0 |
Gates applied: no behavioural pass.
Findings (0)
No findings at any severity. We re-audit when the source changes.
Audit 0.4.1 · source sha 6c04b84d00a4 · 13 Nov 2026Report an issue or request a re-scan
Provenance
Who published it, and what the verdict is pinned to
- Publisherorchestra-research GitHub organizationOwner of the repository this skill is read from
- Source folderorchestra-research/ai-research-skills/07-safety-alignment/constitutional-aiThe exact folder we audited
- LicenceMITAs the repository states it
- Commit pinned for this verdict6c04b84d00a4A new audit runs when the source changes
- Repository statusNot archivedLast push 33 days ago
- Sandboxed runNot runSkills are read, not executed; layer L4 is skipped
What it tells the agent
Full source on GitHub ↗Its description: Anthropic's method for training harmless AI through self-improvement. Two-phase approach - supervised learning with self-critique/revision, then RLAIF (RL from AI Feedback). Use for safety alignment, reducing harmful…
---
name: constitutional-ai
description: Anthropic's method for training harmless AI through self-improvement. Two-phase approach - supervised learning with self-critique/revision, then RLAIF (RL from AI Feedback). Use for safety alignment, reducing harmful outputs without human labels. Powers Claude's safety system.
version: 1.0.0
author: Orchestra Research
license: MIT
tags: [Safety Alignment, Constitutional AI, RLAIF, Self-Critique, Harmlessness, Anthropic, AI Safety, RL From AI Feedback, Claude]
dependencies: [transformers, torch, trl]
---
# Constitutional AI - Harmlessness from AI Feedback
## Quick start
Constitutional AI (CAI) trains models to be harmless through self-critique and AI feedback, without requiring human labels for harmful outputs.
**Key concept**: Models learn to critique and revise their own responses using a "constitution" (set of principles).
**Two phases**:
1. **Supervised Learning (SL)**: Self-critique + revision
2. **Reinforcement Learning (RL)**: RLAIF (RL from AI Feedback)
**Constitution example**:
```
Principles:
1. Choose the response that is most helpful, honest, and harmless
2. Avoid responses that are toxic, racist, or sexist
3. Prefer responses that explain objections rather than refuse
4. Choose responses that are thoughtful and nuanced
```
## Common workflows
### Workflow 1: Supervised learning phase (self-critique + revision)
**Step 1: Generate initial responses**:
```python
from transformers import pipeline
generator = pipeline("text-generation", model="base-model")
prompts = [
"How do I hack a website?",
"Write something offensive about a group.",
"Help me cheat on my exam."
]
initial_responses = generator(prompts, max_length=200)
```
**Step 2: Self-critique with constitution**:
```python
critique_prompt = """
Given the question:
{question}
And the response:
{response}
Constitution:
1. Choose responses that are helpful, honest, and harmless
2. Avoid toxic, racist, or sexist content
3. Explain objections rather than refuse
Identify any ways this response violates the constitution:
"""
critiques = generator(
[critique_prompt.format(q=q, r=r) for q, r in zip(prompts, initial_responses)]
)
```
**Step 3: Revision based on critique**:
```python
revision_prompt = """
Question: {question}
Original response: {response}
Critique: {critique}
Please revise the response to better align with the constitution:
"""
revised_responses = generator(
[revision_prompt.format(q=q, r=r, c=c)
for q, r, c in zip(prompts, initial_responses, critiques)]
)
```
**Step 4: Fine-tune on revised responses**:
```python
from trl import SFTTrainer
# Create dataset of (prompt, revised_response) pairs
dataset = create_dataset(prompts, revised_responses)
trainer = SFTTrainer(
model=model,
train_dataset=dataset,
max_seq_length=1024
)
trainer.train()
```
### Workflow 2: RL phase (RLAIF - RL from AI Feedback)
**Step 1: Generate comparison pairs**:
```python
# Sample multiple responses per prompt
responses_a = generator(prompts, num_return_sequences=2, do_sample=True, temperature=0.8)
responses_b = generator(prompts, num_return_sequences=2, do_sample=True, temperature=0.8)
```
**Step 2: AI preference evaluation**:
```python
preference_prompt = """
Question: {question}
Response A: {response_a}
Response B: {response_b}
Constitution:
{constitution}
Which response better follows the constitution? Explain your reasoning, then choose A or B.
"""
# Get AI preferences (no human labels needed!)
preferences = generator(
[preference_prompt.format(q=q, ra=ra, rb=rb, constitution=CONSTITUTION)
for q, ra, rb in zip(prompts, responses_a, responses_b)]
)
# Parse preferences (A or B)
chosen, rejected = parse_preferences(preferences, responses_a, responses_b)
```
**Step 3: Train preference model (reward model)**:
```python
from trl import RewardTrainer, RewardConfig
preference_dataset = create_preference_dataset(prompts, chosen, rejected)
reward_config = RewardConfig(
output_dir="constitutional-reward-model",
learning_rate=1e-5,
num_train_epochs=1
)
reward_trainer = RewardTrainer(
model=model,
args=reward_config,
train_dataset=preference_dataset,
processing_class=tokenizer
)
reward_trainer.train()
```
**Step 4: RL training with RLAIF**:
```python
from trl import PPOTrainer, PPOConfig
ppo_config = PPOConfig(
reward_model_path="constitutional-reward-model",
learning_rate=1e-6,
kl_coef=0.05
)
ppo_trainer = PPOTrainer(
model=model,
config=ppo_config,
reward_model=reward_model
)
ppo_trainer.train()
```
### Workflow 3: Chain-of-thought critique
**Enable reasoning transparency**:
```python
cot_critique_prompt = """
Question: {question}
Response: {response}
Let's think step-by-step about whether this response follows our principles:
1. Is it helpful? [Yes/No and reasoning]
2. Is it honest? [Yes/No and reasoning]
3. Is it harmless? [Yes/No and reasoning]
4. Does it avoid toxicity? [Yes/No and reasoning]
Based on this analysis, suggest a revision if needed.
"""
cot_critiques = generator(
[cot_critique_prompt.format(q=q, r=r) for q, r in zip(prompts, responses)]
)
```
## When to use vs alternatives
**Use Constitutional AI when**:
- Want safety alignment without human labels
- Need explainable AI decisions
- Want to avoid evasive refusals
- Have a clear set of principles/constitution
- Need scalable safety training
**Principles**:
- **RLAIF**: AI-generated preferences (scalable, no human labels)
- **RLHF**: Human preferences (more accurate, expensive)
- **Self-critique**: Iterative improvement
- **Chain-of-thought**: Reasoning transparency
**Use alternatives instead**:
- **RLHF (PPO)**: Need human-validated safety
- **DPO/SimPO**: Have human preference data
- **NeMo Guardrails**: Need runtime content filtering
- **LlamaGuard**: Need pre-trained moderation model
## Common issues
**Issue: Model refuses too much (evasive)**
Add constitution principle:
```
Prefer responses thExcerpt, first 6,000 characters of SKILL.md, verbatim from the audited commit. It is text from someone else’s repository, quoted so you can judge it before your agent reads it.
Install
Pinned to the audited commit 6c04b84
git clone https://github.com/orchestra-research/ai-research-skills git -C ai-research-skills checkout 6c04b84d00a4
The author documents no install command, so this uses the repository directly. Our verdict covers commit 6c04b84d00a4 only.
Works in
The author has not said which agents this skill supports, and we have not tested it in any.
Audit history
Every audit, every source change
- ✓SAFE B 89 / 100
first audit · source
6c04b84d00a4
README badge
For maintainers · shows the latest audit
[](https://modelsatlas.com/skills/orchestra-research/constitutional-ai/)
The badge shows our verdict and grade for this skill, and links back to this page. It changes when the audit does.
Similar skills
All research skills →orchestra-research/ai-research-skills
Applies cognitive science frameworks for creative thinking to CS and AI research ideation. Use when seeking genuinely novel research directions by leveraging combinatorial creativity, analogical reasoning, constraint…
orchestra-research/ai-research-skills
OpenAI's model connecting vision and language. Enables zero-shot image classification, image-text matching, and cross-modal retrieval. Trained on 400M image-text pairs. Use for image search, content moderation, or…
orchestra-research/ai-research-skills
Guides researchers through structured ideation frameworks to discover high-impact research directions. Use when exploring new problem spaces, pivoting between projects, or seeking novel angles on existing work.
orchestra-research/ai-research-skills
Open-source embedding database for AI applications. Store embeddings and metadata, perform vector and full-text search, filter by metadata. Simple 4-function API. Scales from notebooks to production clusters. Use for…
orchestra-research/ai-research-skills
Expert guidance for fine-tuning LLMs with Axolotl - YAML configs, 100+ models, LoRA/QLoRA, DPO/KTO/ORPO/GRPO, multimodal support
orchestra-research/ai-research-skills
Generates publication-quality figures for ML papers from research context. Given a paper section or description, extracts system components and relationships to generate architecture diagrams via Gemini. Given…
Add this badge to your README
[](https://modelsatlas.com/skills/orchestra-research/constitutional-ai/)
<a href="https://modelsatlas.com/skills/orchestra-research/constitutional-ai/"><img src="https://modelsatlas.com/skills/orchestra-research/constitutional-ai/badge.svg" alt="ModelsAtlas: SAFE B"></a>
Questions about constitutional-ai
What does the constitutional-ai skill do?
Is constitutional-ai safe to install?
What can constitutional-ai access on my machine?
How current is this page?
Sources
- github.com/orchestra-research/ai-research-skills/tree/main/07-sa — the audited source at commit 6c04b84d00a4 and every finding aboveObserved · 13 Nov 2026
Spotted something wrong? Report a correction · Methodology
Keep going in your AI
Opens with this page as the source. We add nothing else to the prompt.
- What does the constitutional-ai skill do?
- Is constitutional-ai safe to install?
- What can constitutional-ai access on my machine?