Baba Is EvalSAFE
Claude et al. play the brilliant puzzle title "Baba is You"
Overview
From the repository's own README, as read at the audited commit. Badges and raw HTML are left out.
https://github.com/user-attachments/assets/cb6d0d49-b583-46a2-b262-1cf38279a531
We evaluate language models' meta-level reasoning in an interactive setting, like the upcoming ARC-AGI-3, except we use the brilliant puzzle game "Baba Is You". In short, the player manipulates the game rules by pushing word blocks around to form or destroy rules. We provide an MCP server to interact with the game in text format. The project is currently an alpha and not stable. Contributions are welcome, and brave devs with model credits to spare are invited to give it a try.
Setup 🔩
- Buy "Baba Is You" somewhere, like on Steam: https://store.steampowered.com/app/736260/BabaIsYou
- Clone this repo in the
Datafolder - macOS + Steam:
/Users/[username]/Library/Application Support/Steam/steamapps/common/Baba Is You/Baba Is You.app/Contents/Resources/Data - Windows + Steam (?):
C:\Program Files (x86)\Steam\steamapps\common\Baba Is You\Data - Install prerequisites (or use
uv)
pip install mcp fastmcp pyautogui configparser
- Run the setup script (after inspecting its contents) (also required before restarting the game, and for changes to
io.luawhich in turn require restarting the game)
cd baba_is_eval chmod +x setup.sh ./setup.sh
- Open the game, preferably in a command line
- macOS + Steam:
/Users/[username]/Library/Application Support/Steam/steamapps/common/Baba Is You/Baba Is You.app/Contents/MacOS/Chowdren - (Currently we use clicking as a workaround for focusing, you might need to move the window to (800, 500) pixels on monitor one)
- Start the MCP Server
mcp dev baba_is_eval/game_mcp.py
On top of this MCP client agnostic setup, you can use a client like Claude Desktop to have a model interact with the server and play the game.
Available MCP Tools
The server provides these tools for interacting with the game:
- `enter_level(level: str)` - Enter a specific level (e.g., "1", "2", "3")
- **`get_
2fc3a2da1805OBSERVED · 2026-10-07Exposed tools (7)
5 read · 2 write · 0 destructive.
| Tool | Risk | Description |
|---|---|---|
enter_level | read | Enter a level. |
execute_commands | write | Execute a sequence of commands. |
game_rules | read | Get help on Baba Is You game rules. Use |
get_game_state | read | Prints the game state as a matrix, with each cell showing the game entity |
leave_level | read | Exits the current level. |
restart_level | write | Restarts the current level. |
undo_multiple | read | Undo the last n moves by creating a control file with n undo() commands. |
Trust audit
SAFEgrade B · trust 89/100 Nothing in the source contradicts what it says it does. Grade A is reserved for packages that have also passed the behavioural sandbox.
| Layer | What it checks | Result |
|---|---|---|
| L0 | Provenance & inventory | PASS |
| L1 | Static analysis of the code | PASS |
| L2 | Instruction surface (what it tells the agent) | PASS |
| L3 | Class-specific surface | PASS |
| L4 | Behavioural (sandbox) | SKIPPED |
What the source does
- Filesystem
- none-observed
- Network
- none-observed
- Shell
- none-observed
- Dependencies
- pinned
- Secrets in source
- none-found
Findings (1)
Gates applied: no_behavioural_pass, no_license.
2fc3a2da1805full audit observations/trust-audit/mcp-server/lennart-finke__baba-is-eval.json · Report an issue / request a re-scanAudit history
Every audit this server has had. A grade with a past is a grade somebody is still checking.
| Date | Source | Verdict | Grade | Score | Change |
|---|---|---|---|---|---|
| 2026-10-07 | 2fc3a2da1805 | SAFE | B | 89 | first audit |
Questions
What is the Baba Is Eval MCP server?
Claude et al. play the brilliant puzzle title "Baba is You"
What tools does Baba Is Eval expose?
7 in total: 5 read-only, 2 that write, and 0 that can delete or overwrite. Every one is listed on this page with its risk.
Is Baba Is Eval safe to connect to an agent?
The audit found nothing in the source that contradicts what it says it does, and graded it B (89/100). Grade A is held back for packages that have also passed a sandboxed behavioural run, which is why a clean server reads B.
What credentials does Baba Is Eval need?
No credential environment variables were found in its source, so it appears to need none.
How current is this page?
The grade is for one exact copy of the source (2fc3a2da1805), read on 2026-10-07. The repository is watched and re-audited when it changes.