dsh-tool-vision

中文文档

A DeepSeek Harness (DSH) plugin that gives a text-only model route (e.g. deepseek-v4-flash) an image-reading capability by routing the image to a user-configured, vision-capable LLM route.

When the main model needs to see an image, it calls the plugin's read_image_vision tool. The plugin reads the image file and sends the image + a task-specific prompt (written by the main model per task) to the configured vision model, then returns the textual description as the tool result. No image block is ever injected into the session, so the text-only main route keeps working.

Features

• Zero build step: plain ESM JavaScript, lib/ committed — installs from GitHub, npm, or a tarball with no build scripts, no allowBuilds permission needed.
• Uses DSH's own LLM infrastructure: providers, credentials, retry, and proxies configured through the normal DSH model settings (e.g. llm-pi-ai.providers.<name>).
• Prompt authored by the main model: the plugin has no fixed prompt; the main model decides what to ask about the image per task.
• Capability-checked: the tool refuses to run unless the configured model declares image input, before any bytes leave the machine.
• Attachment-safe: image bytes are validated by the attachment service (magic-byte check, size caps) and never enter the conversation history.

How it works

main model (no vision) ──read_image_vision(file_path, prompt)──▶ dsh-tool-vision
                                                                    │
                                                                    │ read file → attachments.saveImage (validate/persist)
                                                                    │ llm.stream({provider, model, messages:[image+prompt]})
                                                                    ▼
                                                     configured vision model (e.g. xiaomi/mimo-v2.5)
                                                                    │
                                                                    ▼ text description
main model ◀──────────── tool result (path / provider / model / description) ──┘

The vision model sees only the image and the prompt — it has no access to the conversation, so the main model must put everything needed to answer into the prompt argument.

Requirements

• DeepSeek Harness installed (dsh CLI on PATH)
• A registered LLM route whose model declares image input. Built-in example: the xiaomi route with mimo-v2.5 (configured via the Web Models page or settings.yaml → llm-pi-ai.providers.xiaomi, with its apiKeyEnv credential set, e.g. XIAOMI_API_KEY).

Installation

Choose one — the profile commands below use web as an example; pass --profile <name> for any other profile.

# From npm
dsh plugin --profile web add @pzqian123/dsh-tool-vision

# From GitHub (no build step, so no allowBuilds is required)
dsh plugin --profile web add github:pzqian123/dsh-tool-vision

# From a local tarball
npm pack
dsh plugin --profile web add .\pzqian123-dsh-tool-vision-<version>.tgz

> Restart dsh web (or the profile's process) after installing — the bundle list is read at startup.

Configuration

Configuration lives in three layers, all in your own files — the package ships no defaults and refuses to call the tool until the vision route is configured.

Layer 1 — the vision route (endpoint + models): settings.yaml

Routes are registered under llm-pi-ai.providers.<name> in ~/.dsh/settings.yaml (or via the Web Models page, see below).

Catalog route — if pi-ai ships a catalog for the provider (e.g. xiaomi, openai, anthropic), only the credential reference is needed; endpoint and models come from the built-in catalog:

llm-pi-ai:
  providers:
    xiaomi:
      apiKeyEnv: XIAOMI_API_KEY

Custom OpenAI-compatible endpoint — hand-declare the route, its protocol, endpoint, and model list. The vision model's input must include image, or the tool's capability check refuses the call before any bytes leave the machine:

llm-pi-ai:
  providers:
    my-vision:
      displayName: My Vision Gateway
      apiKeyEnv: MY_VISION_API_KEY
      api: openai-completions
      baseURL: https://gateway.example.com/v1
      models:
        - id: vision-1
          name: Vision Model 1
          contextWindow: 131072
          maxTokens: 8192
          input: [text, image]      # ← required for vision

Layer 2 — the API key: ~/.dsh/.credentials.yaml (or an environment variable)

Keys are stored by the credentials service, never in settings.yaml. The apiKeyEnv reference from Layer 1 names the key:

MY_VISION_API_KEY: sk-your-key-here

…or export the same name as an environment variable instead of writing the file.

Layer 3 — point the plugin at the route: profile cordis.patch.yml

Add the vision route to the tool-vision row in your profile's patch layer (~/.dsh/profiles/web/cordis.patch.yml):

- id: tool-vision
  config:
    provider: my-vision        # route name from Layer 1
    model: vision-1            # model id from Layer 1
    # maxTokens: 1024          # optional: vision model max output tokens (default 1024)
    # timeoutMs: 120000        # optional: total timeout per vision call in ms (default 120000)

Config changes take effect via HMR — no restart needed.

Recommended: configure via the Web Models page

Settings → Models covers Layers 1 and 2 with a form instead of YAML:

• Pick a catalog provider and type only the API key — the page stores it write-only through the credentials service and records the apiKeyEnv reference (derived <ROUTE>_API_KEY) for you.
• Add a custom provider card: enter the Provider ID (must start with a lowercase letter), the endpoint, the protocol, and at least one model — or click Fetch available models to pull the list from the endpoint directly.
• Note: the page does not edit a model's input field; for a hand-declared custom vision model, add input: [text, image] to settings.yaml afterwards.

Tips

• Let the agent configure it for you — you never have to edit YAML by hand. In any DSH session, just say: "Install dsh-tool-vision and configure it to use xiaomi/mimo-v2.5" (or any provider/model you have). The agent will run dsh plugin add, write the tool-vision row, and store the API key through the credentials service. Mention the key in the same message; it goes into ~/.dsh/.credentials.yaml, not into chat history.
• Installation is agent-doable too — ask the agent to run the install command above. The only step it cannot do for you is restarting dsh web (it runs inside that process); a restart is needed after install/uninstall.
• Config changes are hot-reloaded — editing the tool-vision row needs no restart; only install/uninstall does.
• Just say "read this image" — you never call the tool yourself; the main model picks read_image_vision whenever it needs to see an image.
• Start small — test with a small image first (attachment cap defaults to 5 MB). If the vision model's answer is truncated, ask for a narrower prompt or raise maxTokens.
• Vision-capable main route? No needless forwarding — when your main model can already receive images, read_image_vision refuses and points you to read_image, so the image is never redundantly sent to the configured vision provider.
• Keys never go into settings.yaml — keys live in ~/.dsh/.credentials.yaml (or an environment variable); the credential name is referenced by apiKeyEnv in the provider profile.

Usage

Just ask the main model to read an image:

> Read D:\images\screenshot.png and tell me, character by character, what text is in it.

The model will call read_image_vision with a task-specific prompt. The harness's built-in read_image tool coexists in the tool catalog; the system prompt and the tools' own behavior keep the two straight.

Which tool is used, per route

• Text-only main route (e.g. deepseek-v4-flash): the built-in read_image refuses (the route cannot carry images); the model uses read_image_vision, which forwards the image to the configured vision model and returns a text description.
• Vision-capable main route: the model reads the image directly with the built-in read_image. In this case read_image_vision refuses — forwarding would be redundant and would ship the image to another provider — with an error pointing back to read_image.

Tool contract

Privacy

The image bytes are sent to the configured vision provider (the same provider/credential you configure in DSH model settings). Nothing else from the session is sent. The description is stored in the session as plain text.

Troubleshooting

Development

dsh-tool-vision/
├── package.json       # declares dsh.bundle (bundle manifest)
├── cordis.patch.yml   # the patch layer that inserts the tool-vision row
├── lib/index.js       # the plugin implementation (plain ESM, no build step)
├── README.md
└── README.zh-CN.md

Iterate locally:

npm pack
dsh plugin --profile web remove @pzqian123/dsh-tool-vision
dsh plugin --profile web add .\pzqian123-dsh-tool-vision-<version>.tgz

> Note: pnpm symlinks local-directory dependencies, which breaks runtime resolution — always install from the tarball (or GitHub/npm) rather than a local path.

Config schema

License

MIT