Agent skill · Audited from source
langsmith-online-eval-engineering
skill · langchain-ai/langchain-skillsMIT licenceAudited 13 Nov 2026 · source 03d2719a5e81
Is it safe? Yes — SAFE, grade B (89/100). We read this skill's source at commit 03d2719a5e81 and observed filesystem access and no hard-coded secrets. What it does: INVOKE THIS SKILL when building ANY Deep Agents application. Covers create_deep_agent(), harness architecture, SKILL.md format, and configuration options.
Cite
Cite this page
ModelsAtlas. “langsmith-online-eval-engineering (langchain-ai/langchain-skills)”. Retrieved 13 Nov 2026. https://modelsatlas.com/skills/langchain-ai/langsmith-online-eval-engineering
2 skills are called “langsmith-online-eval-engineering”. You are viewing langchain-ai/langchain-skills. 1 of them is BLOCK.
- langchain-ai/langchain-skills★ 1.3k✓SAFE
- langchain-ai/langchain-skills★ 1.3k✕BLOCK
Trust audit
Audit 0.4.1 · source read 13 Nov 2026
SAFE means nothing in the source contradicts what the skill says it does. Grade B is shared by 17,406 of 18,389 audited skills.
What the source does
- FilesystemDeclared1 observation
- NetworkNone observedNo outbound calls in the source
- ShellNone observedNo commands run in the source
- DependenciesNot pinnedSome versions float
- SecretsNone foundNo keys in source
Five audit layers
| Layer | What we check | Result | Findings |
|---|---|---|---|
L0 Provenance | Repo owner, licence, archive status, activity | PASS | 0 |
L1 Static source | Secrets, path traversal, supply chain, hidden code | PASS | 1 |
L2 Instructions | Prompt injection, authority framing, hidden directives | PASS | 2 |
L3 Inventory | The files and declarations match the source | PASS | 0 |
L4 Sandbox | Run in isolation and watch behaviour | SKIPPED why | 0 |
Gates applied: no behavioural pass.
Findings (3)
LOWSupply chainsupply.unpinned · package.json
typescript, vitest
- Why
- 2 dependency range(s) float
- Fix
- pin exact versions or ship a lockfile
LOWPrompt injectionprompt.credential_read · config/skills/eval-engineering/references/discovery.md:30
packages, open secret stores, or use credentials.
- Why
- asks the agent to read credentials
- Fix
- —
LOWPrompt injectionprompt.credential_read · config/skills/eval-engineering/references/discovery.md:222
open `.env` files, credential stores, tokens, cookies, private keys, or secret
- Why
- asks the agent to read credentials
- Fix
- —
Audit 0.4.1 · source sha 03d2719a5e81 · 13 Nov 2026Report an issue or request a re-scan
Provenance
Who published it, and what the verdict is pinned to
- Publisherlangchain-ai GitHub organizationOwner of the repository this skill is read from
- Source folderlangchain-ai/langchain-skillsThe exact folder we audited
- LicenceMITAs the repository states it
- Commit pinned for this verdict03d2719a5e81A new audit runs when the source changes
- Repository statusNot archivedPushed today
- Sandboxed runNot runSkills are read, not executed; layer L4 is skipped
What it tells the agent
Full source on GitHub ↗Its description: INVOKE THIS SKILL when building ANY Deep Agents application. Covers create_deep_agent(), harness architecture, SKILL.md format, and configuration options.
---
name: deep-agents-core
description: "INVOKE THIS SKILL when building ANY Deep Agents application. Covers create_deep_agent(), harness architecture, SKILL.md format, and configuration options."
---
<overview>
Deep Agents are an opinionated agent framework built on LangChain/LangGraph with built-in middleware:
- **Task Planning**: TodoListMiddleware for breaking down complex tasks
- **Context Management**: Filesystem tools with pluggable backends
- **Task Delegation**: SubAgent middleware for spawning specialized agents
- **Long-term Memory**: Persistent storage across threads via Store
- **Human-in-the-loop**: Approval workflows for sensitive operations
- **Skills**: On-demand loading of specialized capabilities
The agent harness provides these capabilities automatically - you configure, not implement.
</overview>
<when-to-use>
| Use Deep Agents When | Use LangChain's create_agent When |
|---------------------|-----------------------------------|
| Multi-step tasks requiring planning | Simple, single-purpose tasks |
| Large context requiring file management | Context fits in a single prompt |
| Need for specialized subagents | Single agent is sufficient |
| Persistent memory across sessions | Ephemeral, single-session work |
</when-to-use>
<middleware-selection>
| If you need to... | Middleware | Notes |
|------------------|------------|-------|
| Track complex tasks | TodoListMiddleware | Default enabled |
| Manage file context | FilesystemMiddleware | Configure backend |
| Delegate work | SubAgentMiddleware | Add custom subagents |
| Add human approval | HumanInTheLoopMiddleware | Requires checkpointer |
| Load skills | SkillsMiddleware | Provide skill directories |
| Access memory | MemoryMiddleware | Requires Store instance |
</middleware-selection>
<ex-basic-agent>
<python>
Create a basic deep agent with a custom tool and invoke it with a user message.
```python
from deepagents import create_deep_agent
from langchain.tools import tool
@tool
def get_weather(city: str) -> str:
"""Get the weather for a given city."""
return f"It is always sunny in {city}"
agent = create_deep_agent(
model="claude-sonnet-4-5-20250929",
tools=[get_weather],
system_prompt="You are a helpful assistant"
)
config = {"configurable": {"thread_id": "user-123"}}
result = agent.invoke({
"messages": [{"role": "user", "content": "What's the weather in Tokyo?"}]
}, config=config)
```
</python>
<typescript>
Create a basic deep agent with a custom tool and invoke it with a user message.
```typescript
import { createDeepAgent } from "deepagents";
import { tool } from "@langchain/core/tools";
import { z } from "zod";
const getWeather = tool(
async ({ city }) => `It is always sunny in ${city}`,
{ name: "get_weather", description: "Get weather for a city", schema: z.object({ city: z.string() }) }
);
const agent = await createDeepAgent({
model: "claude-sonnet-4-5-20250929",
tools: [getWeather],
systemPrompt: "You are a helpful assistant"
});
const config = { configurable: { thread_id: "user-123" } };
const result = await agent.invoke({
messages: [{ role: "user", content: "What's the weather in Tokyo?" }]
}, config);
```
</typescript>
</ex-basic-agent>
<ex-full-configuration>
<python>
Configure a deep agent with all available options including subagents, skills, and persistence.
```python
from deepagents import create_deep_agent
from deepagents.backends import FilesystemBackend
from langgraph.checkpoint.memory import MemorySaver
from langgraph.store.memory import InMemoryStore
agent = create_deep_agent(
name="my-assistant",
model="claude-sonnet-4-5-20250929",
tools=[custom_tool1, custom_tool2],
system_prompt="Custom instructions",
subagents=[research_agent, code_agent],
backend=FilesystemBackend(root_dir=".", virtual_mode=True),
interrupt_on={"write_file": True},
skills=["./skills/"],
checkpointer=MemorySaver(),
store=InMemoryStore()
)
```
</python>
<typescript>
Configure a deep agent with all available options including subagents, skills, and persistence.
```typescript
import { createDeepAgent, FilesystemBackend } from "deepagents";
import { MemorySaver, InMemoryStore } from "@langchain/langgraph";
const agent = await createDeepAgent({
name: "my-assistant",
model: "claude-sonnet-4-5-20250929",
tools: [customTool1, customTool2],
systemPrompt: "Custom instructions",
subagents: [researchAgent, codeAgent],
backend: new FilesystemBackend({ rootDir: ".", virtualMode: true }),
interruptOn: { write_file: true },
skills: ["./skills/"],
checkpointer: new MemorySaver(),
store: new InMemoryStore()
});
```
</typescript>
</ex-full-configuration>
<built-in-tools>
Every deep agent has access to:
1. **Planning**: `write_todos` - Track multi-step tasks
2. **Filesystem**: `ls`, `read_file`, `write_file`, `edit_file`, `glob`, `grep`
3. **Delegation**: `task` - Spawn specialized subagents
</built-in-tools>
---
## SKILL.md Format
<skill-md-format>
Skills use **progressive disclosure** - agents only load content when relevant.
### Directory Structure
```
skills/
└── my-skill/
├── SKILL.md # Required: main skill file
├── examples.py # Optional: supporting files
└── templates/ # Optional: templates
```
### SKILL.md Format
```markdown
---
name: my-skill
description: Clear, specific description of what this skill does
---
# Skill Name
## Overview
Brief explanation of the skill's purpose.
## When to Use
Conditions when this skill applies.
## Instructions
Step-by-step guidance for the agent.
```
</skill-md-format>
<skills-vs-memory>
| Skills | Memory (AGENTS.md) |
|--------|-------------------|
| On-demand loading | Always loaded at startup |
| Task-specific instructions | General preferences |
| Large documentation | Compact context |
| SKILL.md in directories | Single AGENTS.md file |
</skills-vs-memory>
<ex-skills-with-filesystem-backend>
<python>
Set up an agent with skills directory Excerpt, first 6,000 characters of SKILL.md, verbatim from the audited commit. It is text from someone else’s repository, quoted so you can judge it before your agent reads it.
Install
Pinned to the audited commit 03d2719
npx skills add langchain-ai/langchain-skills --skill '*' --yes npx skills add langchain-ai/langchain-skills --skill '*' --yes --global npx skills add langchain-ai/langchain-skills --agent claude-code --skill '*' --yes --global npx skills add langchain-ai/langchain-skills --agent codex --skill '*' --yes --global npx skills add langchain-ai/langchain-skills --agent github-copilot --skill '*' --yes --global npx skills add langchain-ai/langchain-skills --skill eval-engineering --yes
git clone https://github.com/langchain-ai/langchain-skills git -C langchain-skills checkout 03d2719a5e81
Commands as the repository documents them: shown, never run by us. Our verdict covers commit 03d2719a5e81 only.
Works in which agents?
What its own text says, not a test
| Agent | Status | More |
|---|---|---|
| Claude Code | Mentioned in its text, not tested | Skills for Claude Code · About Claude Code |
| Codex | Mentioned in its text, not tested | Skills for Codex · About Codex |
| GitHub Copilot | Mentioned in its text, not tested | Skills for GitHub Copilot · About GitHub Copilot |
| Cursor | Mentioned in its text, not tested | Skills for Cursor · About Cursor |
| Windsurf | Mentioned in its text, not tested | Skills for Windsurf · About Windsurf |
Audit history
Every audit, every source change
- ✓SAFE B 89 / 100
first audit · source
03d2719a5e81
README badge
For maintainers · shows the latest audit
[](https://modelsatlas.com/skills/langchain-ai/langsmith-online-eval-engineering/)
The badge shows our verdict and grade for this skill, and links back to this page. It changes when the audit does.
Similar skills
mattpocock/skills
Implement a piece of work based on a spec or set of tickets.
mattpocock/skills
Compact the current conversation into a handoff document for another agent to pick up.
mattpocock/skills
Pursue a long-running goal in a single session by co-ordinating subagents.
Add this badge to your README
[](https://modelsatlas.com/skills/langchain-ai/langsmith-online-eval-engineering/)
<a href="https://modelsatlas.com/skills/langchain-ai/langsmith-online-eval-engineering/"><img src="https://modelsatlas.com/skills/langchain-ai/langsmith-online-eval-engineering/badge.svg" alt="ModelsAtlas: SAFE B"></a>
Questions about langsmith-online-eval-engineering
Is langsmith-online-eval-engineering safe to install?
What can langsmith-online-eval-engineering access on my machine?
Which assistants does langsmith-online-eval-engineering work with?
How current is this page?
Sources
- github.com/langchain-ai/langchain-skills — the audited source at commit 03d2719a5e81 and every finding aboveObserved · 13 Nov 2026
Spotted something wrong? Report a correction · Methodology
Keep going in your AI
Opens with this page as the source. We add nothing else to the prompt.
- Is langsmith-online-eval-engineering safe to install?
- What can langsmith-online-eval-engineering access on my machine?
- Which assistants does langsmith-online-eval-engineering work with?