Anionex/dsh-computer-use
213Last commit Aug 14, 2026
dsh-computer-use DSH plugin
DSH Computer Use provides a native action layer that lets an Agent observe and control macOS applications through Accessibility APIs. It routes pointer and keyboard input to the selected process without moving the system cursor or activating the target app by default, offering a non-interfering mode for background operations. The Bundle includes a deterministic fixture for testing and a Skill that exposes focused execution tools.
How to install the dsh-computer-use DSH plugin
dsh plugin --profile web add @anionex/dsh-computer-useCopying does not run this command. Review the repository and version before installing the dsh-computer-use DSH plugin.
dsh-computer-use DSH plugin data source
dsh-computer-use DSH plugin snapshot date: Aug 16, 2026
discovered
What the dsh-computer-use DSH plugin can do
- Observe the Accessibility tree of a selected application, returning indexed elements, metadata, and an optional screenshot artifact.
- Prefer semantic input (AXPress, set value, selected text) before falling back to target-process pointer or keyboard input.
- Route pointer and keyboard input to a specific process and window, keeping the system cursor untouched and using a separate agent cursor for feedback.
- Return fresh observations after each action, with resolution details (mode, confidence, target changed) and stale-state rejection.
- Scope application access with separate read and control leases, and require one-use confirmation for high-impact actions.
Where the dsh-computer-use DSH plugin fits
- Let an Agent operate a background macOS application while the user continues working in the foreground.
- Automate native macOS app workflows that require precise UI element targeting, such as filling forms or clicking buttons.
- Test or verify the state of a macOS application by observing its UI and performing actions without activating the app.
- Integrate with other DSH plugins (e.g., dsh-vision-toolkit) for visual grounding or OCR, then use Computer Use to execute the action.
Who the dsh-computer-use DSH plugin is for
- DeepSeek Harness users who need to control macOS applications programmatically through an Agent.
- Developers building automation or testing tools for macOS desktop apps.
- Power users who want to delegate UI tasks to an AI agent without losing control of their current workspace.
dsh-computer-use DSH plugin limitations
- Requires macOS 14 or newer and full macOS Accessibility permission (Screen Recording permission only when screenshots are requested).
- Not suitable for browser tasks; browser automation and CDP state are recommended instead.
- Does not include OCR, visual grounding, or pixel interpretation; those capabilities are handled by the separate dsh-vision-toolkit plugin.
- Depends on DeepSeek Harness with a Web or Headless Profile and the Skill Tool mounted; the agent must load the Computer Use Skill to access execution tools.
dsh-computer-use DSH plugin: from the repository README
Quoted from the Anionex/dsh-computer-use README, the upstream source of the dsh-computer-use DSH plugin. Copyright remains with the original authors.
**Native macOS control for DeepSeek Harness that keeps your real cursor and foreground application alone by default; the Bundle may bring the target app forward before keyboard input for reliable typing.** DSH Computer Use gives an Agent fresh Accessibility observations, exact process/window targeting, stale-state rejection, scoped application access, and verified post-action state. Semantic Accessibility comes first; mouse, drag, wheel, and keyboard fallback are routed to the selected process instead of the global desktop. English | [中文](README.zh.md) ## Why it is different Accessibility permission lets a process inspect and operate macOS UI elements, but the permission itself does not prevent focus stealing or cursor movement. Those behaviors depend on the input route. The default DSH Computer Use route is deliberately non-interfering: - **No system-cursor movement:** the helper contains no cursor-warp path. - **No global pointer injection:** click, scroll, and drag fallback use a pid/window-targeted SkyLight route, not the global HID event stream. - **No pointer-triggered activation:** semantic Accessibility, process-targeted pointer input, and `keyboardPolicy: preserve` r
Read the full READMERepository license: MIT
dsh-computer-use DSH plugin questions
How do I install DSH Computer Use?
Run `dsh plugin --profile web add @anionex/dsh-computer-use` for the Web profile, or replace `web` with `headless`. After installation, restart the `dsh web` host and start a new Session. Then load the Skill with `/computer-use`.
Does DSH Computer Use require Accessibility permissions?
Yes, it requires macOS Accessibility permission for observation and native actions. If you want to capture screenshots, you also need Screen Recording permission. You can grant these in System Settings > Privacy & Security > Accessibility and Screen Recording.
Can I use this plugin for browser automation?
No, DSH Computer Use is designed for native macOS applications. For browser tasks, the README recommends using browser automation tools and DOM/CDP state instead, as they are more reliable and efficient.
How does the agent cursor work?
The agent cursor is a separate, non-interactive, click-through software cursor displayed during click, scroll, and drag actions. It does not move the system cursor. It is visible by default and stays at the action position until the bound window changes or a hide command. You can configure `cursorAutoHideMs` for timed auto-hide.
What is the difference between `focusPolicy: preserve` and `keyboardPolicy: activate`?
`focusPolicy: preserve` (default) keeps the foreground application unchanged; the agent operates in the background. `keyboardPolicy: activate` (Bundle default) brings the target app forward before keyboard input to ensure reliable typing. The broader `focusPolicy: activate` also activates the target before pointer input. These policies are configurable in the interaction policy YAML.