geohotstan/dsh-computer-use
30Last commit Aug 14, 2026
dsh-computer-use DSH plugin
Cloned from OpenAI's Codex Computer Use window API, this plugin enables AI models to control macOS applications. It uses an accessibility-tree-first approach, returning element indexes for model actions, with window-relative coordinates as fallback. The package includes the local Swift daemon, tools, approval policy, and an MCP server.
How to install the dsh-computer-use DSH plugin
dsh plugin --profile <name> add @zibokapi/dsh-codex-computer-useThis source command needs manual review. Copying does not run it.
dsh-computer-use DSH plugin data source
dsh-computer-use DSH plugin snapshot date: Aug 16, 2026
discovered
What the dsh-computer-use DSH plugin can do
- App listing and key-window capture as screenshot with model-readable accessibility tree
- Subsequent captures of the same app return diffs instead of the full tree
- Synthesized mouse and keyboard input via background-first execution (SkyLight path with CGEvent.postToPid fallback) so the user's foreground never changes
- Per-app approval policy with Codex-style tier guidance and computer_use_list_granted_applications tool
- Standalone MCP stdio server exposing the same tools for external MCP clients (Codex CLI, Claude Code, etc.)
Where the dsh-computer-use DSH plugin fits
- AI agents performing desktop automation tasks on macOS
- Automated UI testing and interaction via accessibility tree
- Remote desktop control by language models through the MCP server
- Accessibility testing by inspecting element tree diffs
Who the dsh-computer-use DSH plugin is for
- Developers building AI-powered desktop automation workflows
- Researchers experimenting with computer use models and agentic desktop control
dsh-computer-use DSH plugin limitations
- macOS only – requires Accessibility and Screen Recording TCC permissions
- Requires Xcode Command Line Tools (`xcode-select --install`) to build the Swift daemon
- Early stage; feature parity against OpenAI's Codex is documented in docs/codex-parity.md
- TCC grants persist only with a stable code signing identity (ad-hoc signature invalidates on rebuild)
dsh-computer-use DSH plugin: from the repository README
Quoted from the geohotstan/dsh-computer-use README, the upstream source of the dsh-computer-use DSH plugin. Copyright remains with the original authors.
English | [中文](README.zh.md) A standalone plugin for the [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness): desktop computer use on macOS — app listing, key-window capture as a screenshot plus model-readable accessibility tree, and synthesized mouse and keyboard input — cloned from OpenAI's Codex Computer Use window API. Input delivery is background-first like Codex's own execution model: semantic AX actions need no focus, and raw events ride the private SkyLight path (`SLEventPostToPid` with the keyboard authentication envelope, field stamps, and focus-without-raise records, ported from the MIT-licensed [trycua/cua](https://github.com/trycua/cua) project) with a public `CGEvent.postToPid` fallback per symbol, so the user's foreground never changes. It needs no OpenAI components. The design is accessibility-tree-first: `computer_use_get_app_state` returns a numbered, tab-indented element tree; the model acts on element indexes, with window-relative coordinates as a fallback, and later captures of the same app return diffs instead of the full tree. Everything ships in this one package: the seam, the local Swift-daemon provider, the `computer_use_*` tools, the ap
Read the full READMERepository license: MIT
dsh-computer-use DSH plugin questions
How do I install this plugin?
Run `dsh plugin --profile <name> add @zibokapi/dsh-codex-computer-use` to add the plugin to your profile. Then execute `npx @zibokapi/dsh-codex-computer-use` to build the Swift daemon and grant the required Accessibility and Screen Recording permissions. Finally, restart the profile with `dsh --profile <name> web` to load the computer_use_* tools.
Does it work on Windows or Linux?
No, it is macOS-only. The daemon is a Swift binary that depends on macOS-specific APIs (Accessibility and Screen Recording). The Node.js part also throws a platform gate on non-darwin hosts.
What permissions does the daemon need?
It needs Accessibility (to read the accessibility tree and target input) and Screen Recording (to capture window screenshots). Both are TCC permissions requested at startup via system dialogs. If a grant is missing or revoked, the plugin refuses to activate until the permission is granted.
Can I use the MCP server with other LLM clients?
Yes, the plugin includes a standalone MCP stdio server under `@zibokapi/dsh-codex-computer-use/computer-mcp`. It exposes the same ten Codex Computer Use window tools plus request_access and event_stream_* tools. You can run it with `node lib/mcp.js` and connect it to Codex CLI, Claude Code, or any MCP client.
How do I update the plugin?
Use `dsh plugin --profile <name> update @zibokapi/dsh-codex-computer-use` to update the plugin package. The daemon is installed at a fixed path (`~/.dsh/computer-use/`) to preserve TCC grants across updates. After updating, you may need to re-run `npx @zibokapi/dsh-codex-computer-use` if the daemon binary changed.