Ui.Vision - Browser and Desktop AutomationBLOCK
Browser and desktop automation with an MCP server for AI assistants. Record macros or write JavaScript to automate websites, extract data, and control desktop apps with OCR and image recognition.
Overview
From the repository's own README, as read at the audited commit. Badges and raw HTML are left out.
Automate websites and desktop applications with macros, JavaScript, or an AI assistant. Ui.Vision combines browser automation, OCR and image recognition, with an MCP server that lets AI agents create, edit and run reusable macros in your browser.
Install the extension · Connect an AI assistant with MCP · API reference · User forum
What can you automate?
- Browser tasks: fill forms, extract web data, download reports and repeat workflows in your existing browser session.
- Website testing: record and replay interactions, use Selenium IDE commands, and run checks with different input data.
- Visual tasks: find on-screen text with OCR and locate controls by image when HTML selectors are unavailable.
- Desktop workflows: automate applications and remote desktop interfaces using visual recognition and mouse and keyboard input. Desktop automation requires the additional Ui.Vision desktop components (XModules).
- AI-assisted automation: ask the built-in AI assistant or an external MCP client to write and run macros, then inspect the resulting script, logs and screenshots.
Use the macro recorder and command table, write JavaScript with the uiv.* API, or let an AI assistant author the macro. The resulting automation can be saved and run again.
Get started
Install the browser extension; you do not need to build this repository:
Open Ui.Vision and record a browser task, or start with a macro from the extension's demos. For desktop input and additional native capabilities,
6788d8b4b8bdOBSERVED · 2026-09-22Connect
Built from this server's own package name, version and transport as found in its source — not copied from anyone's documentation, so it cannot drift against a page we do not control. Replace the environment placeholders with a token scoped to the least it needs.
claude mcp add uivision-mcp-bridge -- npx -y [email protected]
Exposed tools (19)
11 read · 7 write · 1 destructive. Blast radius: 1 tool can delete or overwrite — an agent that can be talked into calling a tool can be talked into calling this one.
| Tool | Risk | Description |
|---|---|---|
browser_snapshot | read | Returns the structure of the browser tab Ui.Vision plays in. Default mode |
click_at | read | EXPLORATION ONLY: click the point (x, y) read off the MOST RECENT screenshot — the extension converts the picture\ |
computer | read | Control the current browser tab. Take a screenshot to see the page, then click, move the mouse, type text or press keys. |
create_macro | write | Create a NEW macro from the given Ui.Vision JSON and save it in the |
delete_macro | destructive | Delete a macro from the |
get_ai_settings | read | Reports the ACTIVE AI provider/model as the chat will resolve it, which providers have keys stored (booleans only — keys never cross the bridge), the per-provider model settings, computer-vision scope and OCR engine. Call before set_ai_settings to plan a change. |
get_chat | write | Returns the in-panel AI Chat transcript and whether a run is in progress (running:true/false). Poll this after send_chat; the transcript is text-only (inline images are omitted). |
get_macro | read | Returns the macro currently loaded in the Ui.Vision editor, as Ui.Vision JSON. A JS script macro comes back with a |
get_page | read | Alias of browser_snapshot (the older Ui.Vision name) — identical behaviour. |
list_macros | read | Lists all macros stored in Ui.Vision (name and id). Use open_macro to load one into the editor before reading or running it. |
open_macro | read | Opens the macro with the given name in the Ui.Vision editor and returns its JSON. Subsequent get_macro/set_macro/run_macro calls operate on it. Note: unsaved editor changes may be discarded. |
run_macro | write | Run a macro against the browser tab and wait for it to finish (can take minutes). Three forms: pass |
save_element_image | write | Crop a rectangle from the MOST RECENT screenshot and save it as a Ui.Vision vision image, for image-based commands like |
save_relative_image | write | Create a green/pink RELATIVE vision image from the MOST RECENT screenshot, for commands like |
screenshot | read | Returns a screenshot of the visible part of the browser tab Ui.Vision plays in. Coordinates passed to save_element_image / save_relative_image are absolute pixels in this image. |
select_browser | read | Switches which connected browser the other tools talk to. Several browsers (or several instances of one browser) can hold a bridge connection at the same time; bridge_status lists their labels. Pass a full label ( |
send_chat | write | Sends a message to the REAL in-panel AI Chat — the configured chat model runs it with its own agent tools, exactly as if a user had typed it. Returns immediately; a chat run takes minutes, so poll get_chat until it reports running:false. For self-testing the chat agent end-to-end. |
set_macro | write | Apply changes to the macro in the Ui.Vision editor — pass the complete Ui.Vision JSON. PREFER the JS script form { |
type_at | read | EXPLORATION ONLY: click the point (x, y) of the MOST RECENT screenshot and type text there (key names like ${KEY_ENTER}, ${KEY_TAB}, ${KEY_CTRL+a} are honoured). Same conversion and same caveat as click_at. Returns what changed on the page afterwards. |
Trust audit
BLOCKgrade F · trust 42/100 Do not install this without reading the findings. The audit found something that could harm you or your machine.
| Layer | What it checks | Result |
|---|---|---|
| L0 | Provenance & inventory | WARN |
| L1 | Static analysis of the code | FAIL |
| L2 | Instruction surface (what it tells the agent) | PASS |
| L3 | Class-specific surface | WARN |
| L4 | Behavioural (sandbox) | SKIPPED |
What the source does
- Filesystem
- declared (6 observation(s))
- Network
- declared (11 observation(s))
- Shell
- declared (10 observation(s))
- Dependencies
- not all pinned
- Secrets in source
- found
Findings (25)
code = eval('`' + raw + '`')var ast = thisInterpreter.parse_('(function(' + argsStr + ') {' + code + '})',eval(code: string): Promise<unknown>
? 'function (' + n.params.map(p => source.slice(p.start, p.end)).join(',') + ') ' + source.slice(n.body.start, n.body.end)return { ok: false, error: `${cspError}\n→ This site's CSP blocks page-world eval (uiv.evaluate / executeScript) — not a macro bug, do not retry. Read the DOM through the content script instead, whichvision2.wasm
._.DS_Store
.DS_Store
if (wr && wr.source === 'beacon') {uiv.log('Wayland: screen->viewport origin taken from the window-rect beacon: ' + originX + ',' + originY + ' (event.screenX is not trustworthy under this compositor)', 'blue');} catch (e) { /* no beacon measurement - the event-learned origin stands */ }if (wr && wr.source === 'beacon') {uiv.log('Wayland: screen->viewport origin taken from the window-rect beacon: ' + originX + ',' + originY + ' (event.screenX is not trustworthy under this compositor)', 'blue');placeholder="eg. http://0.0.0.0:1234"
background: url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQAgMAAABinRfyAAAADFBMVEUAAABaWlrMzMz////nPAkwAAAAAWJLR0QAiAUdSAAAAAlwSFlzAAAOxAAADsQBlSsOGwAAAAd0SU1FB+IDGRUHMxeV5KYAAAAXSURBVAjXY1
background: url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQAgMAAABinRfyAAAADFBMVEUAAABaWlrMzMz////nPAkwAAAAAWJLR0QAiAUdSAAAAAlwSFlzAAAOxAAADsQBlSsOGwAAAAd0SU1FB+IDGRUHMxeV5KYAAAAXSURBVAjXY1
$checkerboard-image: "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQAgMAAABinRfyAAAADFBMVEUAAABaWlrMzMz////nPAkwAAAAAWJLR0QAiAUdSAAAAAlwSFlzAAAOxAAADsQBlSsOGwAAAAd0SU1FB+IDGRUHMxeV5KYAAAAXSURB
import { Button, message, Modal } from 'antd'const v = JSON.parse(text.replace(/^/, ''))
const token = 'client-identity-test-only'
delete_macro
.babelrc
.eslintignore
.eslintrc.js
._.DS_Store
Gates applied: no_behavioural_pass.
6788d8b4b8bdfull audit observations/trust-audit/mcp-server/a9t9__ui-vision.json · Report an issue / request a re-scanAudit history
Every audit this server has had. A grade with a past is a grade somebody is still checking.
| Date | Source | Verdict | Grade | Score | Change |
|---|---|---|---|---|---|
| 2026-09-22 | 6788d8b4b8bd | BLOCK | F | 42 | first audit |
Questions
What is the Ui.Vision MCP server?
Browser and desktop automation with an MCP server for AI assistants. Record macros or write JavaScript to automate websites, extract data, and control desktop apps with OCR and image recognition.
What tools does Ui.Vision expose?
19 in total: 11 read-only, 7 that write, and 1 that can delete or overwrite (delete_macro). Every one is listed on this page with its risk.
Is Ui.Vision safe to connect to an agent?
No — not without reading the findings first. The audit graded it F (42/100) and found 5 critical or high issues in the source. Each one is listed on this page with the file and line it is on. Separately from the audit: 1 of its tools can destroy data, so scope the token you give it to what you actually need.
What credentials does Ui.Vision need?
It reads UIVISION_MCP_TOKEN from the environment. Give it a token scoped to the least it needs — an agent that can be talked into calling a tool can be talked into calling it with your credentials.
How does Ui.Vision run?
It speaks stdio, so it runs as a local process your client starts. It is published on npm as webextension-imagesearch at 1.0.1.
How current is this page?
The grade is for one exact copy of the source (6788d8b4b8bd), read on 2026-09-22. The repository is watched and re-audited when it changes.