deepseekplugins

DeepSeek Harness plugin

dsh-computer-use-haifeiwu

DSH plugin: local macOS computer use — screenshot/mouse/keyboard/browser control + OCR fallback for dsh (DeepSeek Harness)

Models & Providers
Jump to install

Source facts

Repository
haifeiWu/dsh-computer-use
Latest update
Aug 15, 2026
Category
Models & Providers
GitHub stars
0
Format
plugin
Catalog evidence
Upstream dsh.bundle evidence
Evidence path
package.json#dsh.bundle
Checked against
0.1.0-rc.8
Upstream check date
2026-08-20

This evidence comes from the upstream catalog. This site has not installed, run, or security-reviewed the plugin.

Install

Start with a prompt that asks an agent to review the GitHub repository and source. Switch to the command if you want to install it yourself.

Copy this prompt into DSH, Codex, or another agent and ask it to review the GitHub repository and source first.

Do not install or run any commands yet. Read this plugin's GitHub repository, README, and relevant source code. Then answer the questions below clearly and directly so I can decide whether it fits my needs:

1. What is this plugin, and what problem does it solve?
2. Who is it for, and what are its typical use cases?
3. How is it used after installation? Include one minimal example.
4. What known limitations or privacy, security, compatibility, or maintenance risks does it have?
5. Give a clear recommendation: recommend, conditionally recommend, or do not recommend, with reasons.

Distinguish statements documented by the repository, inferences from source code, and unknowns. If evidence is insufficient, say so explicitly. Do not guess or simply repeat the README.

GitHub: https://github.com/haifeiWu/dsh-computer-use
Plugin: dsh-computer-use-haifeiwu
Author: haifeiWu

Check the source files

Read the README and other files from this plugin directory before installing.

File explorer2 files
README.mdSource · read only

dsh-computer-use

dsh-computer-use is a plugin for DSH (DeepSeek Harness) that gives your agents local macOS computer use capabilities — see the screen (screenshot), drive mouse/keyboard/scroll, and control the browser at DOM level, with an OCR fallback for models without vision.

  • Tools: screenshot / screen_ocr (macOS Vision) / desktop_mouse_move|click|drag|scroll (cliclick) / keyboard_type|key / browser_open|click|type|scroll|extract|close (Playwright-core + system Chrome)
  • Enforces an act → verify loop: screenshot before and after every operation; text-only models read the screen via screen_ocr / browser_extract
  • Safety by design: on-screen coordinate guards, keyboard shortcut whitelist, TCC permission boundaries (never bypassed)
  • Mounts via profile cordis.patch.yml; does not modify the official dsh repository

---

以下为中文文档 / Chinese documentation below:

为 dsh（DeepSeek Harness） 提供本地 macOS computer use 能力：模型可以看屏幕（截图）、操作屏幕（鼠标/键盘/滚动）、控制浏览器（DOM 级），并在模型无视觉能力时用 OCR 文本兜底。

独立插件包，通过 profile cordis.patch.yml 挂载，不改动 dsh 官方仓库。

能力总览

场景工具实现
看屏幕screenshot（全屏/区域/窗口 → image block）/usr/sbin/screencapture + attachment 服务
看屏幕（无视觉模型）screen_ocr（文本 + 像素坐标）macOS Vision framework（swift）
桌面键鼠desktop_mouse_move/click/drag/scrollcliclick
键盘keyboard_type（含中文）/ keyboard_key（白名单快捷键）cliclick + osascript
浏览器（主）browser_open/click/type/scroll/extract/closeplaywright-core + 系统 Chrome

操作纪律（模型侧）：每次操作前 screenshot 确认状态，操作后 screenshot 验证（act → verify 闭环）。无视觉模型用 screen_ocr / browser_extract 读屏。

前置条件

依赖状态安装/授权
screencapturemacOS 自带—
swift（Vision OCR）macOS 自带—
Google Chrome / Edge已安装浏览器工具必需
cliclick需安装brew install cliclick
屏幕录制权限需授权系统设置 → 隐私与安全性 → 屏幕录制（勾选运行 dsh 的终端应用）
辅助功能权限需授权系统设置 → 隐私与安全性 → 辅助功能（键鼠模拟必需）

权限/依赖缺失时工具会返回带引导的错误信息，不会静默失败。

安装与挂载

cd ~/work/opensource/dsh-computer-use
pnpm install
pnpm build   # 编译到 lib/（package main 指向 lib/index.js，file: 方式安装前必须构建）

# 方式一：一次性 overlay
dsh web --patch "$PWD/cordis.patch.yml"

# 方式二：持久化到 profile（推荐）
# 把 cordis.patch.yml 中的 insert 段合并进 $DSH_HOME/profiles/<profile>/cordis.patch.yml

插件运行时按 npm 包名 dsh-computer-use 解析：开发阶段可在 profile 的 package.json 中通过 file: 依赖或 pnpm workspace 链接到本目录，或将包发布后安装。

配置项

config:
  enableDesktop: true   # 桌面键鼠 + 键盘工具
  enableBrowser: true   # 浏览器工具
  ocrFallback: true     # screen_ocr 兜底工具

模型视觉能力自适应

模型能力行为
支持图像输入screenshot 返回 image block，完整 Computer Use 体验
不支持图像输入screenshot 拒绝并提示；改用 screen_ocr（桌面）+ browser_extract（浏览器）读屏 → 计算坐标 → 操作

安全说明

  • 坐标护栏：所有坐标/区域参数严格限制在主屏边界内，越界拒绝（防盲点屏外）
  • 键盘白名单：keyboard_key 仅接受白名单按键与快捷键，防任意键注入
  • 操作前看屏：工具描述强制引导 act → verify 闭环
  • 权限即边界：TCC 缺失即拒绝并给授权引导，不做任何绕过
  • 本插件不新增审批层（信任模型 + dsh 全局策略），键鼠操作即时生效

开发与测试

pnpm test        # vitest 单测（118 用例：参数校验/护栏/命令构造/浏览器管理/OCR 解析/坐标换算）
pnpm typecheck   # tsc --noEmit

测试全部 mock 子进程与浏览器，不触碰真实屏幕，CI 安全。

设计文档

  • 设计：docs/superpowers/specs/2026-08-14-dsh-computer-use-design.md
  • 计划：docs/superpowers/plans/2026-08-14-dsh-computer-use.md
Back to searchOpen GitHub repository

More related plugins

View category