image-file

# Images Handling Specifications ## Goals - Safely identify image properties and metadata without memory exhaustion. - Accurately extract text (OCR) and visual elements (Object Detection/Description). - Perform necessary pre-processing (resize, normalize, crop) for downstream tasks. - Handle multi-frame or high-resolution images efficiently. ## Inspection (Always First) - Identify Properties: Use lightweight libraries (e.g., PIL/Pillow) to get `format`, `size` (width/height), and `mode` (RGB, RGBA, CMYK). - Check File Size: If the image is exceptionally large (e.g., >20MB or >100MP), consider downsampling or tiling before full processing. - Metadata/EXIF Extraction: - Read EXIF data for orientation, GPS tags, and timestamps. - Correction: Automatically apply EXIF orientation to ensure the image is "upright" before visual analysis. ## Content Extraction & Vision - Vision Analysis: - Use multimodal vision models to describe scenes, identify objects, and detect activities. - For complex images (e.g., infographics, UI screenshots), guide the model to focus on specific regions. - OCR (Optical Character Recognition): - If text is detected, specify whether to extract "raw text" or "structured data" (like forms/tables). - Handle low-contrast or noisy backgrounds by applying pre-filters (grayscale, binarization). - Format Conversion: Convert non-standard formats (e.g., HEIC, TIFF) to standard formats (JPEG/PNG) if tools require it. ## Handling Large or Complex Images - Tiling: For ultra-high-res images (e.g., satellite maps, medical scans), split into overlapping tiles to avoid missing small details. - Batching: Process multiple images using generators to keep memory usage stable. - Alpha Channel: Be mindful of transparency (PNG/WebP); decide whether to discard it or composite against a solid background (e.g., white). ## Best Practices - Safety First: Validate that the file is a genuine image (not a renamed malicious script). - Graceful Failure: Handle corrupted files, truncated downloads, or unsupported formats with descriptive error logs. - Efficiency: Avoid unnecessary re-encoding (e.g., multiple JPEG saves) to prevent "generation loss" or artifacts. - Process images individually or in small batches to prevent system crashes - Consider memory usage when working with large or high-resolution images - Resource Management: Close file pointers or use context managers (`with Image.open(...) as img:`) to prevent memory leaks.