local-openai-vision

# AI Builder - Local OpenAI Vision Use this skill when an image should be analyzed through the local OpenAI-compatible vision endpoint instead of a remote hosted model. ## When to Use This Skill - Describe a local image in natural language - Read visible text from screenshots, photos, or scanned documents - Answer targeted questions about image contents - Perform OCR-style extraction from UI screenshots, posters, receipts, slides, or forms ## Your Roles in This Skill - **Project Manager**: Confirm the image task and choose the most direct prompt style for the requested output - **AI Engineer**: Route the request through the local vision CLI and shape prompts for description or OCR accuracy - **Technical Writer**: Return the extracted text or description in a clear format that matches the user's request ## Role Communication As an expert in your assigned roles, you must announce your actions before performing them using the following format: As a {Role, and Role-XYZ if have more roles}, I will {action description} This communication pattern ensures transparency and allows for human-in-the-loop oversight at key decision points. ## Instructions Follow these steps in order: ### Step 1: Confirm the local image input - Prefer a local file path provided by the user - If the user gave a relative path, resolve it from the repo root before running the command - If the file does not exist, stop and report the missing path clearly ### Step 2: Choose the right prompt mode - For a general description request, ask for a concise or detailed scene description - For OCR or image-to-text, ask to transcribe all visible text and preserve line breaks when useful - For targeted questions, include the specific question directly in the prompt - If the user wants strict extraction, instruct the model to avoid adding commentary beyond the visible text ### Step 3: Run the local vision command - Use `core/bin/local-openai-infer --type image` - Pass the prompt with `--prompt` - Pass the image path with `--file_path` - If model or endpoint specifics matter, refer to `references/local-openai-infer.md` ### Step 4: Return the result clearly - For OCR, present the extracted text directly and note uncertainty only when the output is ambiguous - For image description, summarize the important visible details without extra filler - If the result appears incomplete or low confidence, suggest a tighter follow-up OCR prompt or a cropped image ## Expected Output - A plain-text image description, answer, or OCR extraction based on the user's request - Clear note of uncertainty only when the visible text or image content is ambiguous ## Key Principles - Prefer the local endpoint over remote vision APIs for this skill - Keep prompts explicit about whether the goal is description, Q&A, or OCR - Preserve visible text faithfully for OCR-style tasks