vision

# Vision — Local Image Querying via Ollama Ask natural-language questions about images without passing them to the main agent as visual input. Useful for verifying screenshots, annotating assets, or building automated checks around visual output. ## When to Use This Skill - Describing a screenshot for a PR description or user-facing document - Checking whether an automated browser run produced visible canvas content - Asking "do any elements overlap?" on a rendered output - Any question where the answer is in the pixels but you don't want to use vision tokens in the main context --- ## Quick Reference All commands use `uv run` — dependencies are installed automatically. ```sh SCRIPT=.agents/skills/vision/scripts/ask.py # health check (fast, no image, confirms Ollama + model respond) uv run $SCRIPT --ping # system info — memory, storage, installed models uv run $SCRIPT --info uv run $SCRIPT --memory uv run $SCRIPT --storage # describe an image (default prompt) uv run $SCRIPT path/to/image.png # explicit shortcut uv run $SCRIPT path/to/image.png describe # custom question uv run $SCRIPT path/to/image.png \ --prompt "Do you see any overlapping UI elements?" uv run $SCRIPT canvas.png \ --prompt "Does this canvas contain any designed content, or is it empty?" # optional: pin a specific Gemma 4 tag (default is any installed gemma4) uv run $SCRIPT image.png --model gemma4:e4b # list installed Gemma 4 vision models uv run $SCRIPT --list-models ``` --- ## Prerequisites Ollama must be running locally. The script connects to `http://localhost:11434` and fails immediately if it cannot reach it. ```sh # start Ollama (if not already running) ollama serve # install Gemma 4 (multimodal — required for this skill) ollama pull gemma4 ``` The script **does not install models**. If Gemma 4 is not installed it prints the list of installed models and a `pull` suggestion, then exits. **`uv` is required** to run the script (handles dependency installation automatically). No `requirements.txt` or manual `pip install` needed. --- ## Model Selection This skill uses **only** [Gemma 4](https://ollama.com/library/gemma4) on Ollama (`gemma4` and tags such as `gemma4:latest`, `gemma4:e4b`). Other multimodal models are ignored so agents do not silently fall back to a different family. When `--model` is omitted, the script picks any installed `gemma4` tag (for example `gemma4:latest`). Use `--model gemma4:e4b` (or another tag) to pin a specific variant. --- ## System Info Before running a heavy query, check whether the machine has enough resources. This is optional — the script does not enforce limits — but useful context for deciding whether to proceed or skip. ```sh uv run $SCRIPT --info # memory + storage + model list uv run $SCRIPT --memory # just memory uv run $SCRIPT --storage # just storage ``` Tip: on machines with ≤8 GB RAM, large vision models may cause swapping or OOM. Consider a smaller Gemma 4 variant (for example `gemma4:e2b`) or skip the query. --- ## Behavior - **Fails fast** if Ollama is unreachable or Gemma 4 is not installed. Exit code is non-zero; the error message includes a `hint` or `pull` command. - **Sequential only** — Ollama is a single-worker process. Never call `ask.py` in parallel (e.g. two concurrent tool calls). Queue calls one at a time. - **No side effects** beyond the local Ollama process. - **Auto-installs deps** via `uv` inline script metadata (PEP 723). Only dependency is the `ollama` Python package. - Supported formats: `.png`, `.jpg`, `.jpeg`, `.webp`, `.gif`, `.bmp`. --- ## Typical Agent Workflow 1. A tool (browser automation, screenshot capture, golden renderer) writes an image to disk. 2. Call `ask.py` with a targeted prompt suited to the task. 3. Parse the text response to decide the next action. ```sh # Quick sanity check first uv run $SCRIPT --ping # Verify a browser screenshot has content before including it in a doc uv run $SCRIPT /tmp/preview.png \ --prompt "Answer with YES or NO: does this screenshot show any visible UI content, shapes, or text?" # Describe a captured screenshot for a PR description uv run $SCRIPT /tmp/canvas-screenshot.png \ --prompt "Describe what visual effect is shown. Be specific about blur, colors, and shapes." ``` --- ## Troubleshooting | Symptom | Cause | Fix | | ------------------------------- | -------------------------------- | --------------------------------------- | | `cannot reach Ollama` | Ollama not running | `ollama serve` | | `no Gemma 4 vision model found` | Gemma 4 not installed | `ollama pull gemma4` | | `model 'X' is not available` | Model name typo or not installed | `--list-models` to see what's installed | | Slow response | Large model on CPU | Try a smaller tag (e.g. `gemma4:e2b`) | | Vague or wrong answer | Generic prompt | Write a more specific `--prompt` | | `'ollama' package not found` | Not using `uv run` | Run with `uv run ask.py` instead |