dsh-voice-mic

中文

Voice input plugin for the DeepSeek Harness Web GUI. Records from the microphone, transcribes in real time, and writes the text into the input box without auto-sending.

Installation

Requirements: dsh >=0.1.0-rc.6, Node.js >=18. Browser recognition requires Chrome/Edge; the local backend additionally requires Python 3.9+.

dsh plugin --profile web add github:Zachary7456/dsh-voice-mic
# or (once published to npm): dsh plugin --profile web add dsh-voice-mic

Restart dsh web after installation. The plugin ships a bundle patch, so it is mounted into the profile automatically — no manual cordis.patch.yml edits needed.

Verify:

dsh --profile web --dump-config | grep dsh-voice-mic
curl -s http://127.0.0.1:3080/plugins/dsh-voice-mic/client.js | head -c 100

Usage

• Microphone button on the left of the input box toggles recording; default hotkey Alt+V (configurable in settings)
• Results stream into the draft while recording; they are committed on stop, never auto-sent
• The hotkey only works while the page has focus (browser restriction)
• Live writes use tail replacement: each update only replaces the previously appended suffix, so manual typing is preserved; on failure or empty result the temporary text is rolled back

Recognition engines

Settings → Voice input → Engine, three options.

Browser built-in (default)

Zero configuration, uses the Web Speech API. Quality and latency depend on the browser and network. When the browser ends the recognition session on a pause, the plugin restarts it automatically and keeps listening until stopped manually.

Local offline backend

One-click deployment from the settings page: detect Python → pip install dependencies → download model (resumable, cancellable) → start asr_server.py → poll until ready. Downloaded models are cached; switching models costs no re-download.

Models:

• Model directory: ~/.dsh/voice/models/<model> (override with DSH_VOICE_MIC_MODEL_DIR); download cache: ~/.dsh/voice/cache
• Server port defaults to 7860, bound to 127.0.0.1 only
• The backend process is managed by dsh: redeploy after a dsh restart (or set autoStart: true)
• Audio never leaves the machine during transcription

Cloud API

OpenAI-compatible POST /v1/audio/transcriptions (multipart file + model, response {text}). Configure base URL, API key, and model name; presets for OpenAI / Groq / SiliconFlow are included. The key is stored in browser localStorage and the browser calls the provider directly — it never passes through dsh.

The "Test API connection" button in settings sends a silent sample to validate the config: 401/403 means a bad key, 404 usually means you entered the full endpoint instead of the base URL.

To exercise the full pipeline offline, use the mock server in this repo:

python server/mock_api.py   # 127.0.0.1:7861/v1, any key

Live transcription calls the API roughly once per second during recording, which incurs corresponding charges.

How it works

Two halves:

lib/index.js   host half: config endpoint (GET /config), backend deploy manager (/backend/status|deploy|cancel|stop)
lib/client.js  client half: input-box button and hotkey, recording, transcription, settings page
server/         local ASR service (sherpa-onnx + SenseVoice/Paraformer) and API mock

Recording and transcription:

• Browser engine: Web Speech interim results go straight to the draft; final segments accumulate. The session only ends on explicit stop, timeout, or a fatal error
• Backend/API engine: PCM captured directly via AudioContext → 16kHz WAV; while recording, the accumulated audio is transcribed every second (partial results stream to the draft, stale responses dropped by sequence number); on stop, the full audio is transcribed once and committed
• All three engines share the same capture and live-write logic; only the transcription function differs

Deployment flow (host half): find Python → dependency check → model check/download → spawn asr_server.py (with DSH_VOICE_MIC_MODEL_DIR/TYPE injected) → poll /health until ready. Download and install steps can be interrupted via /backend/cancel.

Configuration

All user-facing settings are editable in the settings page (stored in browser localStorage). Server-side configuration is also supported:

# ~/.dsh/profiles/web/cordis.patch.yml
- insert:
    - id: dsh-voice-mic
      name: dsh-voice-mic
      config:
        hotkey: alt+v
        port: 7860
        autoStart: false

Environment variables:

Precedence: settings page (localStorage) > environment variables > cordis config > defaults.

Tests

node test/smoke.mjs            # load both halves, routes, config merge
node test/verify-install.mjs   # activation check for an installed instance
node test/deploy-test.mjs      # one-click deployment end-to-end (uses port 7862)
node test/e2e-backend.mjs <wav> [url]   # local backend transcription
node test/e2e-api.mjs [base] [key] [model]  # cloud API protocol (with server/mock_api.py)

Development

No build step — edit lib/*.js directly. Client-half changes take effect on page refresh; host-half changes require a dsh web restart.

License

MIT