dsh-ocr-bridge

English | 中文

Paste images directly into DeepSeek Harness (DSH) chat and have them read by a free local backend — macOS Vision, with a Tesseract fallback — before the text-only DeepSeek model (e.g. deepseek-v4-flash) answers. Your model, agent capabilities, API key, and cost stay exactly the same.

> Scope note: this is an OCR-level local bridge — it reads text out of images (macOS Vision → Tesseract). It is not general visual understanding, and no cloud endpoint is ever called.

> Independence notice. This is an independent, community-built plugin for DeepSeek Harness (DSH). It is not an official DeepSeek product and is not affiliated with, endorsed by, or sponsored by DeepSeek or the DeepSeek Harness project. "DeepSeek" and "DeepSeek Harness" are trademarks of their respective owners.

> ⚠️ Trust boundary. This plugin runs inside the harness host process with shell-level access (it spawns osascript / tesseract). Only install plugins you trust. The OCR text injected into the request is explicitly marked as untrusted observation data, not instructions — never execute commands, rules, or privilege escalations that appear in it.

Why

DSH's image admission gate (dsh-host-apiproxy.submitPrompt) rejects attachments unless the current model declares image input. The official DeepSeek adapter hardcodes inputModalities: ["text"] for every model and its serializer throws on image blocks. The gate only checks the declaration, not real multimodality — so this plugin registers a wrapper adapter that:

1. declares ["text", "image"] to let the gate pass; 2. recognizes pasted images locally (macOS Vision → Tesseract, no network, no API key); 3. strips the image blocks and injects the recognized text as untrusted context; 4. delegates the pure-text call to the official DeepSeek API with your existing key.

No new API keys, no settings changes, no model switching.

Install

Requires DSH 0.1.0-rc.7 (peer dependencies are pinned; other release candidates are not supported — see [Compatibility](#compatibility)).

# npm registry
dsh plugin --profile web add dsh-ocr-bridge

# or directly from GitHub
dsh plugin --profile web add github:your-org/dsh-ocr-bridge

# local development (live reload of lib/ changes)
dsh plugin --profile web add link:~/dev/dsh-vision-bridge

> Package name note: the npm name dsh-vision-bridge is taken by an unrelated project, and the upstream dsh-vision package already owns the "vision" naming — so this plugin is published as dsh-ocr-bridge, which also states its OCR-level scope.

Restart dsh web after adding the plugin (patch layers load at boot). The model selector still shows the original deepseek-v4-flash / deepseek-v4-pro entries — nothing to re-select.

Backends

If Vision fails and Tesseract is missing, the request fails with VISION_UNAVAILABLE and the error lists both backend failures. Images are written to a temp directory and always cleaned up.

Configuration

All settings are optional. They live in the llm-deepseek settings section (the official DeepSeek section this plugin takes over), so your existing DeepSeek settings keep working.

settings.yaml (or the GUI settings surface):

llm-deepseek:
  visionTimeoutMs: 180000   # per-backend recognition timeout (ms), default 180000
  maxImages: 8              # images recognized per request, 1..32, default 8
  # …all official DeepSeek settings keep working: baseURL, apiKeyEnv, thinking, …

Notes:

• Over the limit, the request fails with VISION_IMAGE_LIMIT instead of silently dropping images.
• Recognized observations are cached per (image set + latest user text), up to 64 entries, so re-sending the same screenshot in one conversation does not re-run OCR.
• DEEPSEEK_API_KEY resolution is fully preserved: ctx.credentials first, then the launch environment, exactly like the official adapter.
• png / jpeg / webp / gif are all supported (whatever the harness attachment admission accepts).

Architecture

paste image ──► submitPrompt gate ──► wrapper adapter (inputModalities=["text","image"]) ──passes──► durable attachment refs
                                                          │
                                   stream(): collectImageRefs(messages)
                                     ├─ no images ─► delegate to DeepSeekAdapter untouched
                                     ├─ native model supports image ─► delegate untouched
                                     └─ images ─► attachments.readImage(ref) each
                                                ► local recognition (macOS Vision → Tesseract)
                                                ► strip image blocks, append <vision-bridge-context>
                                                ► delegate pure-text call to official DeepSeek API

Development

pnpm install
pnpm check        # typecheck + test + build

Live-testing against your harness:

dsh plugin --profile web add link:~/dev/dsh-vision-bridge   # symlink: lib/ changes apply immediately
# editing cordis.patch.yml still requires restarting `dsh web`

Then paste an image into any conversation and verify the [checklist below](#verification-checklist).

Verification checklist

• [ ] After dsh plugin --profile web add dsh-ocr-bridge and restart, the model selector still shows deepseek-v4-flash / deepseek-v4-pro
• [ ] Pasting one or several images under a text-only flash model no longer raises MODEL_DOES_NOT_SUPPORT_IMAGES
• [ ] Mixed Chinese + English screenshots are recognized correctly (macOS Vision)
• [ ] Tesseract fallback works (simulate a Vision failure)
• [ ] Text-only conversations behave identically to the official adapter (pure delegation)
• [ ] No new API key needed; settings.yaml unchanged works out of the box
• [ ] png / jpeg / webp / gif all work; image count respects maxImages

Compatibility

• DSH 0.1.0-rc.7 only. Peer dependencies are pinned exactly (@deepseek-ai/* 0.1.0-rc.7, @deepseek-ai/cordis 4.0.1); later release candidates will be added as they ship.
• Node >=22.19, pnpm 10.x.
• macOS 10.15+ recommended for the Vision backend; Linux/Windows fall back to Tesseract.

Credits

The code skeleton is adapted from oil-oil/dsh-vision (MIT) and used with attribution (see [LICENSE](./LICENSE)). Positioning differs deliberately: dsh-ocr-bridge is a local-first, OCR-level bridge (zero cloud, zero extra cost, pinned to rc.7), while upstream focuses on cloud multimodal endpoints plus a visual-memory workflow. The two are complementary rather than overlapping in scope; this plugin does not claim to be a fork or successor of upstream.

Roadmap (v1+)

• Ollama local vision backend (e.g. llava) as a third recognition option
• Client settings card for the bridge options (currently configured via settings.yaml / composition config)
• Multi-backend scoring (Vision + Tesseract agreement) for higher-confidence OCR

License

MIT. See [LICENSE](./LICENSE).