dsh-maclens 🍎🔍

> Apple's on-device Vision framework, bridged into DeepSeek Harness (dsh) as local tools: OCR, image classification, face detection, document layout, and a combined read — 100% offline, no API key, no daemon.

---

👤 For humans — quick start (30 seconds)

# 1. Install into your dsh profile
dsh plugin --profile desktop add dsh-maclens

# 2. Restart DSH. Then just ask the model:
#    "OCR this screenshot: /Users/me/Desktop/shot.png"

The five tools the model can call:

When to pick maclens vs a VLM: maclens is a CV toolkit — it transcribes, classifies, detects but does not narrate "what this image is about". Need open-ended understanding? Pair it with a VLM bridge (e.g. modlens + qwen-vl). Need fast, free, private OCR/detection? maclens.

---

🤖 For agents — precise contract

TL;DR

Plugin:     dsh-maclens (npm), installs with dsh plugin add
Runtime:    macOS 14+ with Xcode Command Line Tools (Swift 6+)
Tools:      maclens_ocr | maclens_classify | maclens_faces | maclens_document | maclens_describe
Input:      absolute local image path (string) — required on every tool
Output:     one JSON object on stdout; {"error": "..."} + exit 1 on failure
Binary:     bin/maclens in the package (auto-chmod'd), else MACLENS_BIN, else PATH
No network: the CLI makes zero network requests

Install (exact commands)

# From npm — includes the prebuilt binary, no build step:
dsh plugin --profile desktop add dsh-maclens

# From a git checkout — build the Swift bridge first:
cd dsh-maclens && bash scripts/build.sh        # produces bin/maclens
dsh plugin --profile desktop add ./dsh-maclens

Binary resolution order: bin/maclens in the package → $MACLENS_BIN → swift/.build/release/MaclensBridge → maclens on PATH. The plugin chmods the found binary to 0755 at resolve time (npm tarballs drop the exec bit).

Tool schemas

All five tools take path (required, string). OCR-family tools additionally accept:

Output contract

maclens_ocr returns:

{
  "task": "ocr",
  "language": ["zh-Hans", "en-US"],
  "full_text": "跨境增长研究室\n从一个问题，抵达一个决定。",
  "lines": [
    {
      "text": "跨境增长研究室",
      "confidence": 1.0,
      "bbox": { "x": 0.055, "y": 0.519, "width": 0.517, "height": 0.144 }
    }
  ],
  "line_count": 2,
  "truncated": false,
  "sliced": false
}

• bbox is normalized (0–1), origin top-left (converted from Vision's bottom-left so it is intuitive).
• With slice: true, tall images are split into overlapping strips, each strip OCR'd, results stitched back to whole-image coordinates, and duplicate lines in the overlap band de-duplicated. Output adds "sliced": true and "slice_count": N.
• maclens_document = ocr + layout.columns (left/right) + layout.image_dimensions.
• maclens_describe = ocr + classification.observations + faces + layout.
• maclens_faces → faces[] + face_count; maclens_classify → observations[] (identifier, confidence).

Error contract

Raw CLI (for testing outside dsh)

bin/maclens ocr --image /path/to/img.png
bin/maclens classify --image /path/to/img.png --top 3
bin/maclens faces --image /path/to/img.png
bin/maclens document --image /path/to/img.png --slice
bin/maclens describe --image /path/to/img.png --slice --top 2

---

🏗️ How it works

dsh (text-only model)
  └─ maclens_* tools (lib/index.js)
       └─ bin/maclens (Swift CLI, spawned per call — no daemon, no ports)
            └─ Apple Vision: VNRecognizeTextRequest / VNClassifyImageRequest /
               VNDetectFaceRectanglesRequest        ← on-device, offline

🧪 Development

cd swift && swift build -c release
swift test --package-path swift      # 6 behavioral tests

CI (GitHub Actions): Swift release build + smoke tests on macOS, plugin-load + pack-contents check on Ubuntu. All green on main.

📄 Docs

• [AGENTS.md](AGENTS.md) — the agent-facing quick reference (mirrors this section).
• [CHANGELOG.md](CHANGELOG.md) — version history.
• [SECURITY.md](SECURITY.md) — security model & reporting.

License

MIT — see [LICENSE](LICENSE).