qwen-mm-plugins-api

name

qwen-mm-plugins-api

description

Cloud MCP tools for understanding media, by model family. VL model: vision_chat (caption/VQA), ocr, grounding (detect/locate objects). Omni model (reads frames + audio together): timestamped captioning, ASR (plain / controllable / multi-speaker diarized), temporal grounding, event counting, music captioning. Plus transcribe_audio (ASR) and segmentation (SAM3). Use when a question about an image/video/audio needs an external model, not just local reading.

Qwen-MM-Plugins API

You have qwen-mm-plugins-api MCP tools available. They call external models/services to understand media, grouped by model family:

• VL model (Qwen-VL, OpenAI-compatible endpoint): vision_chat, ocr, grounding.
• Omni model (Qwen-Omni — reads video frames and the embedded audio track together, so one call reasons over both): omni_asr, omni_asr_timestamped, omni_multi_speaker_asr, omni_av_caption, omni_av_grounding, omni_av_counting, omni_music_caption.
• Other services: transcribe_audio (Qwen3-ASR), segmentation (a SAM3 server).

Prefer these over manual ffmpeg/ffprobe scripting. Check the qwen-mm-plugins-api tools in your tool list for full schemas and parameters.

When to Use Which Tool

VL model (single images/videos, spatial reasoning):

• Ask a VLM about images/videos (caption, VQA, free-form) → vision_chat
• Extract text from an image → ocr