OpenGEOSAFE
An open, reproducible, causal benchmark for Generative Engine Optimization: does changing a page actually change whether an AI answer engine cites it?
Overview
From the repository's own README, as read at the audited commit. Badges and raw HTML are left out.
Fictional pages written to test the opengeo skill. Any resemblance to a real business is coincidental.
current.htmlandedited.html: a Tucson plumber's water-heater page, and the same page
with prices added. comp1-4.html are four competitor pages. edited_length_block.html deliberately changes the length by more than three words, to test how the skill handles the tool's length check.
emergency.htmlandem_comp1-3.html: a local, "near me" case, the kind of query the
benchmark never covered.
running-store/: a running-shoe store's fitting page, the same page with one sentence
swapped for specific replacement guidance, and five competitor pages.
results-running-shoes/: the real output ofopengeo test0.2.0 onrunning-store/,
for the question "How often should I replace my running shoes?", run 2026-09-30 for $0.29. Every file is as the tool wrote it, including runs.jsonl. The report was regenerated from those same runs, with no new calls, after the tool's interval method changed (design/opengeo-test.md, 0.2.0).
Every page has a ``, as real pages do; the tool shows each page's title to the engines with its excerpt.
48c94134b92fOBSERVED · 2026-10-13Install
Commands as the repository documents them. They are shown, not run.
pip install numpy
What it tells the agent
The instruction file, verbatim from the audited commit — this is the text the model reads, and the surface the audit's instruction layer examines. Quoted here so you can judge it without cloning anything.
---
name: opengeo
description: Run a controlled experiment to test whether a change to a web page makes AI answer engines — ChatGPT, Gemini, Claude, DeepSeek, Grok — more likely to cite it, against the user's real competitors, using the OpenGEO benchmark's method. Use this whenever someone wants to know if editing their page, landing page, product page, docs or blog post will help it get cited, mentioned or recommended in AI answers, or asks whether a specific content change "works" for AI search. Trigger on phrasings like AI visibility, getting cited by ChatGPT, GEO, generative engine optimization, AEO, answer engine optimization, LLM SEO, "will AI recommend my business", or "does adding X to my page help with AI" — even if they never mention testing. Also use it when someone asks what actually works for getting cited by AI, because it carries the published evidence with its confidence levels. It measures what a change did; it does not audit pages or prescribe changes.
---
# OpenGEO: test whether a change gets a page cited more by AI
This skill runs `opengeo test`, which puts the user's page — as it is, and with their
change — among their competitors' pages, asks five AI engines the user's question 24 times
each, and measures whether the change moved how often the page gets cited. It's the same
paired, controlled design as every round OpenGEO has published, applied to one person's
content.
## The one line this skill doesn't cross
It measures what the user's change did. It never decides what the change should be.
That line is the whole point of the project. The AI-visibility market is full of tools that
score a page and sell a list of fixes; their conclusions can't be checked, and they profit
from the fixes. OpenGEO's value is that it is neutral and its numbers are reproducible.
The moment this skill starts telling people what their page should say, it becomes one more
vendor with an opinion, and its measurements stop being trustworthy.
So in practice:
- **Do** help the user test *their* idea, and turn it into a clean edit they approve.
- **Do** relay what the published evidence found, with how strong it is, and help them pick
which of their ideas is worth testing first — framed as something to test, not a verdict.
See `references/evidence.md`.
- **Don't** hand them a list of changes their page "needs", or present an untested tactic as
proven.
- **Don't** invent facts to put in an edit. If they want to add their prices, get the real
prices from them. A test built on made-up numbers measures whether engines reward
fabricated specificity — and if they then publish it, they've published something false.
## Setup
Run these checks before gathering anything, because each one fails in a way that wastes the
user's time if it's discovered later.
1. **Find the tool.** `opengeo.py` lives at the root of the OpenGEO repository. If the
current directory isn't that repo, ask the user where their copy is.
2. **Python with numpy.** The analysis needs numpy, and the tool refuses to run, dry run
included, without it. Check with `python3 -c "import numpy"`. If that fails, try other
interpreters on the machine (`python3.13`, `python3.12`), or suggest
`pip install numpy`. Use whichever works for every command after this, including any
command you hand the user to run themselves.
3. **The API key**, which must be in the environment as `OPENROUTER_API_KEY`. Check it
without revealing it: `python3 -c "import os; print('set' if os.environ.get('OPENROUTER_API_KEY') else 'not set')"`.
If it's missing, ask the user to add `export OPENROUTER_API_KEY=sk-or-...` to their
shell profile and restart the session. **Never ask them to paste the key into chat, and
never put it inside a command you run.** Commands can be saved: in this very project an
approved command carrying a key inline ended up written into
`.claude/settings.local.json`. The key belongs in their environment, nowhere else.
## The workflow
### 1. Pin down what's being tested
Get clear on the change the user has in mind and the question it's meant to help with.
"Add our prices" is a change; "how much does water heater replacement cost?" is the
question. If they only have a vague goal ("get cited more"), that's the moment to read
`references/evidence.md` and help them pick something specific to test.
### 2. Gather the four inputs
- **The question, or three to five of them.** Real questions their customers ask, in the
customer's words. One question is fine to start, but the result then describes that
question only — say so now rather than after they've paid. Two kinds of question are
worth adding on purpose:
- **One the old wording answered**, if the edit removes or rewrites anything. An edit can
win the question it was written for and lose another; without such a question, a loss
can't show up.
- **Local questions, made explicit.** Engines queried through an API don't know where the
asker is, so "near me" has to become a named place ("emergency plumber in Tucson"). If
a question names a neighbourhood, a page that lists it has a built-in keyword advantage:
a real effect, but one that applies to people who type that neighbourhood.
- **Their page as it is now:** a URL or a saved file.
- **Their edited page.** If they already have one, use it. If they want help making it, see
step 3.
- **Two to five competitor pages** that someone asking this question might also be shown.
Five is best: every published round had five other documents, and engines cite several
pages per answer, so in a smaller field the user's current page is often cited almost
every time already. The test then stops halfway, at half the cost, saying there's no room
to show a change; tell them that up front. If they don't know who competes, suggest they
try the question in an AI assistant or search engine and note who comes up. The tool
deliberately doesn't discover competitors itself, because that wTrust audit
SAFEgrade B · trust 89/100 Nothing in the source contradicts what it says it does. Grade A is reserved for packages that have also passed the behavioural sandbox.
| Layer | What it checks | Result |
|---|---|---|
| L0 | Provenance & inventory | WARN |
| L1 | Static analysis of the code | NA |
| L2 | Instruction surface (what it tells the agent) | PASS |
| L3 | Class-specific surface | PASS |
| L4 | Behavioural (sandbox) | SKIPPED |
What the source does
- Filesystem
- none-observed
- Network
- none-observed
- Shell
- none-observed
- Dependencies
- pinned
- Secrets in source
- none-found
Findings (2)
.agents/skills/opengeo
AGENTS.md
Gates applied: no_behavioural_pass.
48c94134b92ffull audit observations/trust-audit/skill/dcefferty__opengeo.json · Report an issue / request a re-scanAudit history
Every audit this skill has had.
| Date | Source | Verdict | Grade | Score | Change |
|---|---|---|---|---|---|
| 2026-10-13 | 48c94134b92f | SAFE | B | 89 | first audit |
Questions
What does the OpenGEO skill do?
An open, reproducible, causal benchmark for Generative Engine Optimization: does changing a page actually change whether an AI answer engine cites it?
Is OpenGEO safe to install?
The audit found nothing in the source that contradicts what it says it does, and graded it B (89/100). Grade A is held back for packages that have also passed a sandboxed behavioural run, which is why a clean skill reads B.
What can OpenGEO access on my machine?
The audit observed no filesystem, network or shell use at all in its source.
How current is this page?
The grade is for one exact copy of the source (48c94134b92f), read on 2026-10-13. The repository is watched, and a new audit runs when it changes — this is the first audit.