codex-image-analysis

name	codex-image-analysis
description	Use Codex/ChatGPT image analysis for one or many images with a custom prompt and structured output. Use when the user asks to analyze screenshots, inspect visuals, compare images, find UI moments, label extracted frames, or batch-process images with Codex. Automatically resizes images to Codex-friendly dimensions before analysis and supports one-image or many-image workflows.

Codex Image Analysis

Analyze one or many images with Codex using existing local Codex/ChatGPT auth. This skill is for reusable image understanding workflows where the agent should:

• take one or many images
• accept a custom prompt
• optionally enforce structured JSON output
• automatically resize images before sending them to Codex
• save per-image and aggregate results

When to use this skill

Use this skill when the user asks things like:

• "analyze this screenshot"
• "look at these frames and tell me what is happening"
• "compare these images"
• "find the image where X appears"
• "label all these extracted frames"
• "run Codex on these screenshots with this prompt"

Why this skill exists

We verified in this environment that:

• codex exec supports image input with -i
• structured JSON output works well with --output-schema
• gpt-5.4-mini works reliably with current Codex auth
• one-image-per-request is the most reliable way to get clean JSON for batch jobs

Default model

Prefer:

• gpt-5.4-mini

Escalate only if needed for key frames or dense UIs.

Resizing strategy

Before sending images to Codex, resize them to a practical analysis size.

Default target:

• longest edge: 1568 px
• preserve aspect ratio
• never upscale smaller images
• JPEG quality: 90 for converted JPEG outputs

Why:

• large enough for UI/text-heavy screenshots
• smaller than raw desktop screenshots, reducing upload cost/latency
• good default for Codex image analysis in this setup

Use smaller targets for bulk batches if needed:

• 1024 px longest edge for cheaper high-volume passes
• 768 px longest edge for coarse first-pass labeling

Main workflow

1. Gather image paths.
2. Normalize/resize them with scripts/prepare_images.py.
3. If structured output is needed, write a JSON Schema file.
4. Run one Codex request per image with scripts/run_codex_image_batch.py.
5. Save:
   • prepared images
   • raw stdout/stderr
   • per-image JSON
   • aggregate JSON
   • CSV export
6. Return the best answer or shortlist.

Single-image workflow

Use this when the user has one image and wants a custom analysis.

python3 scripts/prepare_images.py image.png --output-dir /tmp/codex-image-prep

Then run Codex:

codex exec \
  --skip-git-repo-check \
  --sandbox read-only \
  --model gpt-5.4-mini \
  -i /tmp/codex-image-prep/image.jpg \
  --output-schema schema.json \
  --output-last-message out.json \
  "Analyze this image and return concise JSON."

Batch workflow

Use batch mode when the user has many images or extracted frames.

python3 scripts/run_codex_image_batch.py \
  --images "/path/to/frame_*.jpg" \
  --prompt-file prompt.txt \
  --schema-file schema.json \
  --output-dir /tmp/codex-batch \
  --model gpt-5.4-mini

What the batch script does:

• expands globs
• prepares resized images in prepared/
• runs one Codex request per image
• writes per-image JSON files
• writes all_results.json
• writes all_results.csv
• writes raw stdout/stderr logs for debugging

Custom prompt guidance

When the user gives a vague request, rewrite it into a good vision prompt.

Good prompts should specify:

• what to look for
• whether to stay literal vs infer context
• desired fields
• whether to extract visible text
• how concise to be

Example:

Analyze this image and describe only what is visibly on screen.
Return JSON with:
- filename
- short_visual_description
- likely_context
- visible_text_or_brands
- notable_change
- importance
- confidence
Do not invent unreadable text.

Find-this-visual workflow

If the user asks to find a visual across many frames:

1. Search existing all_results.json first.
2. Match on:
   • descriptions
   • context
   • visible text/brands
   • notable changes
3. Return likely candidates with confidence.
4. If weak, rerun a tighter prompt on top candidates.
5. If still weak, do denser local sampling near the candidate range.

Recommended schema for frame labeling

{
  "type": "object",
  "properties": {
    "filename": {"type": "string"},
    "short_visual_description": {"type": "string"},
    "likely_context": {"type": "string"},
    "visible_text_or_brands": {
      "type": "array",
      "items": {"type": "string"}
    },
    "notable_change": {"type": "string"},
    "importance": {"type": "string", "enum": ["low", "medium", "high"]},
    "confidence": {"type": "number"}
  },
  "required": [
    "filename",
    "short_visual_description",
    "likely_context",
    "visible_text_or_brands",
    "notable_change",
    "importance",
    "confidence"
  ],
  "additionalProperties": false
}

Best practices

• Prefer one image per Codex request for stable JSON.
• Use gpt-5.4-mini by default.
• Resize before analysis instead of sending giant raw screenshots.
• Save raw stdout/stderr for debugging.
• Search previous outputs before re-running expensive jobs.
• Escalate to stronger models only on key images.

Failure handling

If Codex fails:

• verify codex exec works interactively
• verify current auth still supports the selected model
• inspect stderr logs
• retry on a single image first
• if schema enforcement is too brittle, remove schema and parse looser JSON/text

Files

• scripts/prepare_images.py — resize/normalize images for Codex
• scripts/run_codex_image_batch.py — batch Codex runner with aggregation
• examples/frame_schema.json — starter schema for frame labeling