omneumBLOCK
An open-source context evaluator for AI agents.
Overview
From the repository's own README, as read at the audited commit. Badges and raw HTML are left out.
An open-source context evaluator for AI agents.
Agents routinely assemble context from many different sources — a RAG pipeline, web search, APIs, databases, files, memory systems, other agents — but stuffing dozens of raw chunks into a single context window can leave the model working through a mess of information that was never organized for it.
Most agents focus on retrieving more context. Omneum is focused on making sure the context reaching the model is clean and reliable.
Why Omneum and how it works
Modern agent pipelines use several different techniques to decide what context should reach a model:
Semantic similarity (Vector DBs). The piece of text that looks closest to the prompt wins.
Semantic reranking. Retrieved context is reordered based on how relevant it is to the query.
LLM-as-a-Judge. An LLM reads the retrieved context and is instructed to evaluate it against a set of instructions.
These techniques can be highly effective at selecting context, but relevance is only one dimension of context quality.
Chunks can still hold conflicting or stale information. Multiple retrieved chunks might not be deduplicated or appear independent despite sharing an upstream source. This contributes to context bloat: low-q
f86cd5da3378OBSERVED · 2026-10-13Connect
Built from this server's own package name, version and transport as found in its source — not copied from anyone's documentation, so it cannot drift against a page we do not control. Replace the environment placeholders with a token scoped to the least it needs.
claude mcp add omneum --env ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY} --env COHERE_API_KEY=${COHERE_API_KEY} --env TEST_NETWORK_KEY=${TEST_NETWORK_KEY} -- uvx omneum{
"mcpServers": {
"omneum": {
"command": "uvx",
"args": [
"omneum"
],
"env": {
"ANTHROPIC_API_KEY": "${ANTHROPIC_API_KEY}",
"COHERE_API_KEY": "${COHERE_API_KEY}",
"TEST_NETWORK_KEY": "${TEST_NETWORK_KEY}"
}
}
}
}Exposed tools (4)
4 read · 0 write · 0 destructive.
| Tool | Risk | Description |
|---|---|---|
async_tool | read | await context.report_progress(50, 100) |
evaluate_observations | read | Evaluate new assertion observations locally; contribute only if explicitly enabled. |
my_tool | read | return str(x) |
tool_with_context | read | ctx.info(f |
Trust audit
BLOCKgrade D · trust 63/100 Do not install this without reading the findings. The audit found something that could harm you or your machine.
| Layer | What it checks | Result |
|---|---|---|
| L0 | Provenance & inventory | WARN |
| L1 | Static analysis of the code | FAIL |
| L2 | Instruction surface (what it tells the agent) | PASS |
| L3 | Class-specific surface | PASS |
| L4 | Behavioural (sandbox) | SKIPPED |
What the source does
- Filesystem
- declared (8 observation(s))
- Network
- declared (1 observation(s))
- Shell
- declared (2 observation(s))
- Dependencies
- not all pinned
- Secrets in source
- found
Findings (25)
fromhex( ... subprocess.
01ec548baccbe69625b54206ef7100f5ed03719f
0ea3ccd8ee098ae87f36056cd5528af8294f11aa
328ba6be30a1eb88c49c234fab16d961ad5b9812
7f074be19b945e77563f1b65907f43287406ea1e
a42779b09629bce81b76ef626a57a0b40f2ad827
server_module = importlib.import_module(module_name)
allowed_origins=["http://127.0.0.1:*", "http://localhost:*", "http://[::1]:*"],
api_key='unexpected-private-key')],
user='omneum_ingestion', password='private-test-password', sslmode='verify-full')
exec(
value = eval(compile(ast.Expression(assignment.value), '<prepared-public-pins>', 'eval'), vars(module))
@pytest.mark.parametrize('name', ['../escape', '/tmp/escape', 'launchd/../../escape', './omneumd', 'launchd//evil', 'C:\\evil'])('.venv/../../escape', tarfile.REGTYPE), ('other', tarfile.REGTYPE),write('/etc/nginx/conf.d/disposable-provider.conf',f"server {{ listen 443 ssl; server_name identity.fixture-audit.net; ssl_certificate {c['tls_certificate']}; ssl_certificate_key {c['tls_private_key']assert 'proxy_ssl_verify on' in nginx and 'https://127.0.0.1:8444' in nginx
assert 'proxy_pass https://127.0.0.1:8444;' in site
open http://127.0.0.1:8765
if LIBC.ptrace(16, pid, None, None) == -1: # PTRACE_ATTACH
raise OSError(ctypes.get_errno(), 'ptrace denied')
assert LIBC.ptrace(17, pid, None, None) == 0 # PTRACE_DETACH
result['ptrace'] = attempt(lambda: attach(pid))
for operation in ('environ','maps','mem_open','fds','ptrace','signal','kill_zero','pidfd_signal'):return tuple(json.loads(bytes.fromhex(claim.entity[len(prefix):]).decode("ascii")))wrong_key = bytes.fromhex(
Gates applied: critical_finding, no_behavioural_pass.
f86cd5da3378full audit observations/trust-audit/mcp-server/cole-h06__omneum.json · Report an issue / request a re-scanAudit history
Every audit this server has had. A grade with a past is a grade somebody is still checking.
| Date | Source | Verdict | Grade | Score | Change |
|---|---|---|---|---|---|
| 2026-10-13 | f86cd5da3378 | BLOCK | D | 63 | first audit |
Questions
What is the omneum MCP server?
An open-source context evaluator for AI agents.
What tools does omneum expose?
4 in total: 4 read-only, 0 that write, and 0 that can delete or overwrite. Every one is listed on this page with its risk.
Is omneum safe to connect to an agent?
No — not without reading the findings first. The audit graded it D (63/100) and found 1 critical or high issue in the source. Each one is listed on this page with the file and line it is on.
What credentials does omneum need?
It reads ANTHROPIC_API_KEY, COHERE_API_KEY and TEST_NETWORK_KEY from the environment. Give it a token scoped to the least it needs — an agent that can be talked into calling a tool can be talked into calling it with your credentials.
How does omneum run?
It speaks stdio and streamable-http, so it runs as a local process your client starts. It is published on PyPI as omneum.
How current is this page?
The grade is for one exact copy of the source (f86cd5da3378), read on 2026-10-13. The repository is watched and re-audited when it changes.