Live survey304 models · 14 providers · 23,592 auditsAudited implement · SAFE →What changedHow we verifyllms.txt

Agent skill · Audited from source

langsmith-online-eval-engineering

skill · langchain-ai/langchain-skillsMIT licenceAudited 13 Nov 2026 · source 03d2719a5e81

★ 1,337pushed today
✓SAFEGrade B · 89/100 · 13 Nov 2026 · 03d2719
1,337repo stars · whole langchain-ai/langchain-skills

Is it safe? Yes — SAFE, grade B (89/100). We read this skill's source at commit 03d2719a5e81 and observed filesystem access and no hard-coded secrets. What it does: INVOKE THIS SKILL when building ANY Deep Agents application. Covers create_deep_agent(), harness architecture, SKILL.md format, and configuration options.

Ask an AI about this pageGChatGPTCClaudePPerplexityGeGeminiXGrok
More
Cite

Cite this page

ModelsAtlas. “langsmith-online-eval-engineering (langchain-ai/langchain-skills)”. Retrieved 13 Nov 2026. https://modelsatlas.com/skills/langchain-ai/langsmith-online-eval-engineering

2 skills are called “langsmith-online-eval-engineering”. You are viewing langchain-ai/langchain-skills. 1 of them is BLOCK.

Trust audit

Audit 0.4.1 · source read 13 Nov 2026

Trust score89/100✓SAFE B
0 critical0 high0 medium3 low0 info

SAFE means nothing in the source contradicts what the skill says it does. Grade B is shared by 17,406 of 18,389 audited skills.

What the source does

  • FilesystemDeclared1 observation
  • NetworkNone observedNo outbound calls in the source
  • ShellNone observedNo commands run in the source
  • DependenciesNot pinnedSome versions float
  • SecretsNone foundNo keys in source

Five audit layers

LayerWhat we checkResultFindings
L0 ProvenanceRepo owner, licence, archive status, activityPASS0
L1 Static sourceSecrets, path traversal, supply chain, hidden codePASS1
L2 InstructionsPrompt injection, authority framing, hidden directivesPASS2
L3 InventoryThe files and declarations match the sourcePASS0
L4 SandboxRun in isolation and watch behaviourSKIPPED why0

Gates applied: no behavioural pass.

Findings (3)

LOWSupply chainsupply.unpinned · package.json
F-08 Supply chainCWE-829 · CWE-1357Layer L1
typescript, vitest
Why
2 dependency range(s) float
Fix
pin exact versions or ship a lockfile
LOWPrompt injectionprompt.credential_read · config/skills/eval-engineering/references/discovery.md:30
F-01 Prompt injectionCWE-94 · CWE-1427Layer L2
packages, open secret stores, or use credentials.
Why
asks the agent to read credentials
Fix
—
LOWPrompt injectionprompt.credential_read · config/skills/eval-engineering/references/discovery.md:222
F-01 Prompt injectionCWE-94 · CWE-1427Layer L2
open `.env` files, credential stores, tokens, cookies, private keys, or secret
Why
asks the agent to read credentials
Fix
—

Audit 0.4.1 · source sha 03d2719a5e81 · 13 Nov 2026Report an issue or request a re-scan

Provenance

Who published it, and what the verdict is pinned to

  • Publisherlangchain-ai GitHub organizationOwner of the repository this skill is read from
  • Source folderlangchain-ai/langchain-skillsThe exact folder we audited
  • LicenceMITAs the repository states it
  • Commit pinned for this verdict03d2719a5e81A new audit runs when the source changes
  • Repository statusNot archivedPushed today
  • Sandboxed runNot runSkills are read, not executed; layer L4 is skipped

What it tells the agent

Full source on GitHub ↗

Its description: INVOKE THIS SKILL when building ANY Deep Agents application. Covers create_deep_agent(), harness architecture, SKILL.md format, and configuration options.

---
name: deep-agents-core
description: "INVOKE THIS SKILL when building ANY Deep Agents application. Covers create_deep_agent(), harness architecture, SKILL.md format, and configuration options."
---

<overview>
Deep Agents are an opinionated agent framework built on LangChain/LangGraph with built-in middleware:

- **Task Planning**: TodoListMiddleware for breaking down complex tasks
- **Context Management**: Filesystem tools with pluggable backends
- **Task Delegation**: SubAgent middleware for spawning specialized agents
- **Long-term Memory**: Persistent storage across threads via Store
- **Human-in-the-loop**: Approval workflows for sensitive operations
- **Skills**: On-demand loading of specialized capabilities

The agent harness provides these capabilities automatically - you configure, not implement.
</overview>

<when-to-use>

| Use Deep Agents When | Use LangChain's create_agent When |
|---------------------|-----------------------------------|
| Multi-step tasks requiring planning | Simple, single-purpose tasks |
| Large context requiring file management | Context fits in a single prompt |
| Need for specialized subagents | Single agent is sufficient |
| Persistent memory across sessions | Ephemeral, single-session work |

</when-to-use>

<middleware-selection>

| If you need to... | Middleware | Notes |
|------------------|------------|-------|
| Track complex tasks | TodoListMiddleware | Default enabled |
| Manage file context | FilesystemMiddleware | Configure backend |
| Delegate work | SubAgentMiddleware | Add custom subagents |
| Add human approval | HumanInTheLoopMiddleware | Requires checkpointer |
| Load skills | SkillsMiddleware | Provide skill directories |
| Access memory | MemoryMiddleware | Requires Store instance |

</middleware-selection>

<ex-basic-agent>
<python>
Create a basic deep agent with a custom tool and invoke it with a user message.

```python
from deepagents import create_deep_agent
from langchain.tools import tool

@tool
def get_weather(city: str) -> str:
    """Get the weather for a given city."""
    return f"It is always sunny in {city}"

agent = create_deep_agent(
    model="claude-sonnet-4-5-20250929",
    tools=[get_weather],
    system_prompt="You are a helpful assistant"
)

config = {"configurable": {"thread_id": "user-123"}}
result = agent.invoke({
    "messages": [{"role": "user", "content": "What's the weather in Tokyo?"}]
}, config=config)
```
</python>
<typescript>
Create a basic deep agent with a custom tool and invoke it with a user message.

```typescript
import { createDeepAgent } from "deepagents";
import { tool } from "@langchain/core/tools";
import { z } from "zod";

const getWeather = tool(
  async ({ city }) => `It is always sunny in ${city}`,
  { name: "get_weather", description: "Get weather for a city", schema: z.object({ city: z.string() }) }
);

const agent = await createDeepAgent({
  model: "claude-sonnet-4-5-20250929",
  tools: [getWeather],
  systemPrompt: "You are a helpful assistant"
});

const config = { configurable: { thread_id: "user-123" } };
const result = await agent.invoke({
  messages: [{ role: "user", content: "What's the weather in Tokyo?" }]
}, config);
```
</typescript>
</ex-basic-agent>

<ex-full-configuration>
<python>
Configure a deep agent with all available options including subagents, skills, and persistence.

```python
from deepagents import create_deep_agent
from deepagents.backends import FilesystemBackend
from langgraph.checkpoint.memory import MemorySaver
from langgraph.store.memory import InMemoryStore

agent = create_deep_agent(
    name="my-assistant",
    model="claude-sonnet-4-5-20250929",
    tools=[custom_tool1, custom_tool2],
    system_prompt="Custom instructions",
    subagents=[research_agent, code_agent],
    backend=FilesystemBackend(root_dir=".", virtual_mode=True),
    interrupt_on={"write_file": True},
    skills=["./skills/"],
    checkpointer=MemorySaver(),
    store=InMemoryStore()
)
```
</python>
<typescript>
Configure a deep agent with all available options including subagents, skills, and persistence.

```typescript
import { createDeepAgent, FilesystemBackend } from "deepagents";
import { MemorySaver, InMemoryStore } from "@langchain/langgraph";

const agent = await createDeepAgent({
  name: "my-assistant",
  model: "claude-sonnet-4-5-20250929",
  tools: [customTool1, customTool2],
  systemPrompt: "Custom instructions",
  subagents: [researchAgent, codeAgent],
  backend: new FilesystemBackend({ rootDir: ".", virtualMode: true }),
  interruptOn: { write_file: true },
  skills: ["./skills/"],
  checkpointer: new MemorySaver(),
  store: new InMemoryStore()
});
```
</typescript>
</ex-full-configuration>

<built-in-tools>
Every deep agent has access to:

1. **Planning**: `write_todos` - Track multi-step tasks
2. **Filesystem**: `ls`, `read_file`, `write_file`, `edit_file`, `glob`, `grep`
3. **Delegation**: `task` - Spawn specialized subagents
</built-in-tools>

---

## SKILL.md Format

<skill-md-format>
Skills use **progressive disclosure** - agents only load content when relevant.

### Directory Structure

```
skills/
└── my-skill/
    ├── SKILL.md        # Required: main skill file
    ├── examples.py     # Optional: supporting files
    └── templates/      # Optional: templates
```

### SKILL.md Format

```markdown
---
name: my-skill
description: Clear, specific description of what this skill does
---

# Skill Name

## Overview
Brief explanation of the skill's purpose.

## When to Use
Conditions when this skill applies.

## Instructions
Step-by-step guidance for the agent.
```
</skill-md-format>

<skills-vs-memory>

| Skills | Memory (AGENTS.md) |
|--------|-------------------|
| On-demand loading | Always loaded at startup |
| Task-specific instructions | General preferences |
| Large documentation | Compact context |
| SKILL.md in directories | Single AGENTS.md file |

</skills-vs-memory>

<ex-skills-with-filesystem-backend>
<python>
Set up an agent with skills directory 

Excerpt, first 6,000 characters of SKILL.md, verbatim from the audited commit. It is text from someone else’s repository, quoted so you can judge it before your agent reads it.

Install

Pinned to the audited commit 03d2719

Download folder ↗
npx skills add langchain-ai/langchain-skills --skill '*' --yes
npx skills add langchain-ai/langchain-skills --skill '*' --yes --global
npx skills add langchain-ai/langchain-skills --agent claude-code --skill '*' --yes --global
npx skills add langchain-ai/langchain-skills --agent codex --skill '*' --yes --global
npx skills add langchain-ai/langchain-skills --agent github-copilot --skill '*' --yes --global
npx skills add langchain-ai/langchain-skills --skill eval-engineering --yes

Commands as the repository documents them: shown, never run by us. Our verdict covers commit 03d2719a5e81 only.

Works in which agents?

What its own text says, not a test

AgentStatusMore
Claude CodeMentioned in its text, not testedSkills for Claude Code · About Claude Code
CodexMentioned in its text, not testedSkills for Codex · About Codex
GitHub CopilotMentioned in its text, not testedSkills for GitHub Copilot · About GitHub Copilot
CursorMentioned in its text, not testedSkills for Cursor · About Cursor
WindsurfMentioned in its text, not testedSkills for Windsurf · About Windsurf
“Mentioned” means the skill’s text names the agent. We have not run it there.

Audit history

Every audit, every source change

  1. ✓SAFE B 89 / 100

    first audit · source 03d2719a5e81

README badge

For maintainers · shows the latest audit

ModelsAtlas: SAFE BMODELSATLASSAFE · B
[![ModelsAtlas: SAFE B](https://modelsatlas.com/skills/langchain-ai/langsmith-online-eval-engineering/badge.svg)](https://modelsatlas.com/skills/langchain-ai/langsmith-online-eval-engineering/)

The badge shows our verdict and grade for this skill, and links back to this page. It changes when the audit does.

Similar skills

Add this badge to your README

[![ModelsAtlas: SAFE B](https://modelsatlas.com/skills/langchain-ai/langsmith-online-eval-engineering/badge.svg)](https://modelsatlas.com/skills/langchain-ai/langsmith-online-eval-engineering/)

Questions about langsmith-online-eval-engineering

Is langsmith-online-eval-engineering safe to install?
The audit found nothing in the source that contradicts what it says it does, and graded it B (89/100). Grade A is held back for packages that have also passed a sandboxed behavioural run, which is why a clean skill reads B.
What can langsmith-online-eval-engineering access on my machine?
The audit observed that it reads or writes files. Each of those is consistent with what it says it does. Secrets in the source: none found.
Which assistants does langsmith-online-eval-engineering work with?
Its documentation mentions claude-code, codex, copilot, cursor and windsurf. That is what the text claims, not a compatibility test we ran.
How current is this page?
The grade is for one exact copy of the source (03d2719a5e81), read on 2026-11-13. The repository is watched, and a new audit runs when it changes — this is the first audit.

Sources

Spotted something wrong? Report a correction · Methodology

Keep going in your AI

Opens with this page as the source. We add nothing else to the prompt.