MC5lan/dsh-multimodal
51Last commit Aug 15, 2026
dsh-multimodal DSH plugin
This plugin adds vision and image generation to DeepSeek Harness. Users can paste images into conversations, which are transcribed by a configured vision provider, then DeepSeek continues solving the problem. It also supports image generation via pluggable backends, with automatic failover and a rich UI.
How to install the dsh-multimodal DSH plugin
dsh plugin --profile web add https://github.com/MC5lan/dsh-multimodalCopying does not run this command. Review the repository and version before installing the dsh-multimodal DSH plugin.
dsh-multimodal DSH plugin data source
dsh-multimodal DSH plugin snapshot date: Aug 16, 2026
discovered
What the dsh-multimodal DSH plugin can do
- Paste images into chat and get verbatim transcription of error messages, code, UI text.
- Generate images on demand using configured backends, with failover to next backend if active one fails.
- Auto-detect API key platform via paste-key auto-connect (key fingerprint → endpoint probing).
- Extract text from images (OCR) via explicit `extract_text` function usable in any session.
- Configure built-in transcription modes: error-fix, chart-sql, design-code.
Where the dsh-multimodal DSH plugin fits
- Debugging: paste an error screenshot, get transcription and fix suggestions from DeepSeek.
- Design: paste a UI mockup, ask for HTML/CSS code generation.
- Data analysis: paste a chart, get SQL/Pandas code for analysis.
- General: ask DeepSeek to generate an image, get a picture in the conversation.
Who the dsh-multimodal DSH plugin is for
- Developers using DeepSeek Harness for coding assistance, debugging, and documentation.
- Users who need to combine image understanding (screenshots, diagrams) with DeepSeek's reasoning.
dsh-multimodal DSH plugin limitations
- Requires Node.js 18+ and DeepSeek Harness 0.1.0-rc.6+.
- No built-in models or providers; users must configure their own vision endpoints and image backends.
- Vision transcription quality depends on the configured third-party API; latency and accuracy vary.
- Plugin is still in early development (0.2.x versions); configuration and API may change.
dsh-multimodal DSH plugin: from the repository README
Quoted from the MC5lan/dsh-multimodal README, the upstream source of the dsh-multimodal DSH plugin. Copyright remains with the original authors.
English | [简体中文](README.zh.md) Give DeepSeek a pair of eyes and a paintbrush: paste a screenshot or image into the conversation and the vision provider **you configured** first transcribes it verbatim (error messages, code, UI text preserved), then **DeepSeek keeps solving your problem** — all in the same turn, no extra steps. When an illustration is needed, DeepSeek automatically calls the image backend and the generated pictures appear right in the conversation. > **Blank slate by design**: this plugin ships **no built-in models, providers, or backends**. Vision endpoints, image backends, and models are all declared by *you* — plug in whatever API you already have (DeepSeek, Zhipu, Aliyun, SiliconFlow, ModelScope, Xfyun, Qianfan, local Ollama, …). Nothing is preloaded, nothing is assumed. > **Compatibility**: built for [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) `0.1.0-rc.6` (Web and headless). See [CHANGELOG](CHANGELOG.md). ## Features | Scenario | Behavior | |---|---| | Plain-text chat | Straight to the DeepSeek API (unchanged) | | Image + question (e.g. error screenshot) | Your configured vision provider "looks" first → transcribes to text → **Deep
Read the full READMERepository license: MIT
dsh-multimodal DSH plugin questions
How do I install dsh-multimodal?
You can install it via `dsh plugin --profile web add https://github.com/MC5lan/dsh-multimodal` or clone the repo and add it locally. The plugin requires DeepSeek Harness 0.1.0-rc.6+ and Node.js 18+. After installation, restart DSH and the Multimodal settings page appears.
What vision providers are supported?
Any OpenAI-compatible vision endpoint can be used. The plugin offers preset cards for Zhipu, Bailian, Xfyun, ModelScope, SiliconFlow, Qianfan, and local Ollama. You can also add custom endpoints via `extraProviders`. The vision provider must be configured in the settings before transcription works.
Do I need to send images to DeepSeek?
No. Images are never sent to DeepSeek's API. They are transcribed by your configured vision provider, and only the transcription text is passed to DeepSeek. This keeps your image data private and avoids DeepSeek's image limitations.
How does the paste-key auto-connect work?
Paste an API key into the quick-add box on the Multimodal settings page. The plugin tries to identify the platform by matching the key's fingerprint, then probes known endpoints with `/models` to confirm. Once recognized, it automatically configures the endpoint, allow-list, credential, model list, and enables vision and image generation in one step.
Can I use local models for vision?
Yes. The plugin includes a one-click Ollama preset for local vision models. This keeps sensitive images on your machine and does not require external API keys. You can also add any local endpoint as a custom provider by putting it in the `trustedBaseUrls` list.