viewport-highlighter

# Viewport Highlighter ## Purpose Automate viewport segmentation of construction drawing sheets. Each sheet typically contains 1-12+ distinct views (floor plans, sections, elevations, details, schedules). This skill identifies all views, determines their boundaries, extracts metadata (title, detail number, scale, type), and creates viewport highlights — producing the same result as a user manually drawing and labeling each viewport in the UI. Does NOT: modify existing viewports, delete viewports, or change element data. Only creates new viewport highlights and populates their metadata. --- ## Step 0: Verify AgentCM Mode This skill **requires AgentCM**. It writes viewport overlays through the AgentCM REST API and has no standalone output path. **Check for `.construction/` directory at the project root.** If `.construction/` is absent, **stop immediately** and tell the user: > "viewport-highlighter requires an AgentCM project — `.construction/` directory not found. This skill submits viewport overlays through the AgentCM API and cannot operate without it. Open this project in AgentCM first, then re-run." If `.construction/` exists: - Read `.construction/CLAUDE.md` for project context - Read `.construction/database.yaml` for `query_command`, `project_id`, `api_url` - Read `.construction/index/sheet_index.yaml` for sheet inventory - Sheet images at `.construction/rasters/{sheet_number}.png` - OCR data queryable via `extracted_items` table in PostgreSQL - Write viewports via REST API ## Step 1: User Scopes the Task User provides sheet scope: - Specific sheets: "A2.01, A2.02, A5.01" - By discipline: "all architectural sheets" - All sheets: "every sheet in the set" Optional: user can specify which view types to look for (e.g., "only floor plans and sections"). Default: identify ALL views on each sheet. **Before proceeding:** Confirm the sheet list and any filters with the user. Show the count of sheets to process. ## Step 2: Vision — Identify View Boundaries For each sheet in scope, rasterize to PNG (if not already available) and examine with vision. ```bash # Rasterize on demand if the pre-rendered PNG is missing ${CLAUDE_SKILL_DIR}/../../bin/construction-python \ ${CLAUDE_SKILL_DIR}/../../scripts/pdf/rasterize_page.py \ "{pdf_path}" {page_index} --dpi 200 --output /tmp/{sheet_number}.png ``` **Vision task:** Examine the full sheet image. Identify every distinct view (drawing area) on the sheet. For each view, extract: | Field | What to look for | Example | |-------|-----------------|---------| | **Title** | View title bar text (usually centered below the view) | "FIRST FLOOR PLAN" | | **Detail Number** | Number in the detail bubble or title bar | "1", "A2.01", "3/A5" | | **Scale** | Scale annotation near the title bar | `1/8" = 1'-0"`, `1/4" = 1'-0"`, "NTS" | | **View Type** | Classification of the drawing content | plan, section, elevation, detail, schedule | | **Bounding Region** | Approximate rectangle enclosing the entire view (normalized 0-1) | `{x: 0.02, y: 0.05, w: 0.48, h: 0.65}` | ### View boundary detection signals Use these visual cues to determine where one view ends and another begins: 1. **Heavy border lines** — thick lines separating drawing areas 2. **Title bars** — horizontal bars with view name, scale, detail number 3. **Detail bubbles** — circles or hexagons with detail/sheet reference 4. **Whitespace gaps** — clear separations between drawing content 5. **Grid systems** — column/row grids define the extents of a plan view 6. **Section cut lines** — long dash-dot lines with directional arrows 7. **Match lines** — indicate where a plan continues on another sheet ### View type classification | View Type | `extractionScope` value | Signals | |-----------|------------------------|---------| | Floor plan | `plan` | Grid lines, room names/numbers, dimension strings, north arrow | | Enlarged plan | `plan` | "ENLARGED" in title, larger scale than base plan, room detail | | Section | `section` | Section cut reference (e.g., "SECTION A-A"), vertical layers, material hatching | | Elevation | `elevation` | "ELEVATION" in title, facade view, material callouts, floor lines | | Detail | `detail` | Detail bubble reference, large scale (3"=1'-0"), construction assembly closeup | | Schedule | `schedule` | Tabular grid, column headers, row data (door schedule, finish schedule) | | Diagram | `other` | Riser diagrams, single-line diagrams, flow diagrams | ### Bounding region estimation from vision When estimating the bounding region as normalized 0-1 coordinates: - **x** = left edge of view content / sheet width (0.0 = left edge) - **y** = top edge of view content / sheet height (0.0 = top edge) - **width** = view content width / sheet width - **height** = view content height / sheet height (include title bar) **Include** the title bar and any associated notes/legends within the view. **Exclude** the sheet title block (typically bottom-right corner). **Margins:** Add ~1% padding on each side to avoid clipping content. ### Multi-view sheet layout patterns Common construction sheet layouts to expect: - **Single view:** One large plan fills most of the sheet - **2-up vertical:** Two views stacked (e.g., floor plan top, reflected ceiling bottom) - **2-up horizontal:** Two views side by side (e.g., two elevations) - **Grid layout:** 4-6 details arranged in a 2x3 or 3x2 grid - **Mixed:** Large plan on left, 2-3 sections/details stacked on right - **Schedule + details:** Schedule table in upper portion, details below ## Step 3: OCR Anchor Verification For each vision-identified view, verify and refine metadata using OCR data. ### Title verification Search for the view title text in `extracted_items`: ```bash {query_command} -c "SELECT id, text, x_min, y_min, x_max, y_max FROM extracted_items WHERE sheet_id = '{sheet_id}' AND text ILIKE '%{distinctive_title_word}%' ORDER BY y_max DESC" ``` The title bar is typically near the **bottom** of the view content area. Use the title's y-coordinate to refine the view's bottom boundary. ### Scale verification Search for scale text near the title: ```bash {query_command} -c "SELECT id, text, x_min, y_min, x_max, y_max FROM extracted_items WHERE sheet_id = '{sheet_id}' AND text ILIKE '%SCALE%' AND y_min BETWEEN {title_y - 0.02} AND {title_y + 0.02} AND x_min BETWEEN {title_x - 0.15} AND {title_x + 0.15}" ``` Common scale text patterns: - `SCALE: 1/8" = 1'-0"` - `1/4" = 1'-0"` - `SCALE: NTS` (not to scale) - `3" = 1'-0"` ### Detail number verification Search for detail number in title bar area or detail bubbles: ```bash {query_command} -c "SELECT id, text, x_min, y_min, x_max, y_max FROM extracted_items WHERE sheet_id = '{sheet_id}' AND y_min BETWEEN {title_y - 0.02} AND {title_y + 0.02} AND x_min BETWEEN {title_x - 0.10} AND {title_x + 0.10} AND text ~ '^[0-9A-Z]'" ``` Detail number formats: `1`, `2`, `A`, `A2.01`, `3/A5`, `1/A5.01`. ## Step 4: Bounding Region Refinement Refine vision-estimated boundaries using OCR element positions. Query all extracted items within and near the estimated viewport area: ```bash {query_command} -c "SELECT x_min, y_min, x_max, y_max FROM extracted_items WHERE sheet_id = '{sheet_id}' AND (x_min + x_max) / 2 BETWEEN {est_x - 0.02} AND {est_x + est_w + 0.02} AND (y_min + y_max) / 2 BETWEEN {est_y - 0.02} AND {est_y + est_h + 0.02}" ``` Use the extremes of contained items + 1% padding to set the final bounds. ### Overlap prevention After refining all viewports on a sheet, check for overlaps: - If two viewports overlap, shrink the boundary of the one with fewer contained elements at the overlapping edge - Adjacent viewports should have a gap of 0.5-2% of sheet dimension ### Validation checklist Before creating each viewport, verify: - [ ] `x >= 0.0` and `x + width <= 1.0` - [ ] `y >= 0.0` and `y + height <= 1.0` - [ ] `width > 0.02` (at least 2% of sheet width) - [ ] `height > 0.02` (at least 2% of sheet height) - [ ] Title is non-empty - [ ] View type is one of: plan, section, elevation, detail, schedule, other - [ ] No significant overlap with other viewports on same sheet ## Step 5: Submit Viewports for Review ### Submit viewport suggestions Submit all viewports for a sheet as **pending suggestions** via the viewport suggestion ingest endpoint. The user reviews and approves them in the Group Review Gallery before they become live viewports. ```bash curl -s --fail-with-body -X POST "{api_url}/projects/{project_id}/viewport-suggestions/ingest" \ -H "Content-Type: application/json" \ -d '{ "sheet_id": "{sheet_id}", "viewports": [ { "title": "FIRST FLOOR PLAN", "detail_number": "1", "scale_text": "1/8\" = 1'"'"'-0\"", "extraction_scope": "plan", "bounding_region": {"x": 0.02, "y": 0.05, "width": 0.48, "height": 0.65}, "confidence": 0.85, "element_ids": [] }, { "title": "BUILDING SECTION A-A", "detail_number": "A", "scale_text": "1/4\" = 1'"'"'-0\"", "extraction_scope": "section", "bounding_region": {"x": 0.52, "y": 0.05, "width": 0.46, "height": 0.45}, "confidence": 0.80, "element_ids": [] } ] }' ``` **Response:** Returns `suggestion_ids` for the created pending cards. **Processing order:** Submit all viewports for one sheet in a single POST. The system creates `group_suggestions` rows with `status: pending` and `proposedType: viewport_highlight`. Crop images are generated when the user opens the Group Review Gallery. **Confidence scoring:** - 0.90+ = title, scale, and detail number all OCR-verified - 0.70–0.89 = vision-identified with partial OCR verification - 0.50–0.69 = vision-only, no OCR anchors found ### Review flow Viewport suggestions appear in the **Group Review Gallery** as cards with cropped raster previews. The user can: - Edit title, detail number, scale text, and view type - Adjust bounding region (future: via canvas interaction) - Accept → promotes to a live viewport in `graph_views` with containment rebuild and crop generation - Reject → suggestion archived, no viewport created ### Title formatting conventions When setting viewport titles, follow these conventions: - Use the exact title text from the drawing (preserve case) - If the title includes the sheet number reference (e.g., "1/A5.01"), put only the detail number portion in `detail_number` and the full title in `title` - Common abbreviations to preserve: "FLR", "CLG", "ELEV", "TYP" ## Step 6: Verification & Markup ### Sheet markup (REQUIRED for all modes) Mark up each processed sheet with viewport boundary rectangles to show exactly what was identified. ```bash # Build items JSON with rectangles for each viewport # Convert normalized 0-1 coordinates to pixel coordinates using image dims ${CLAUDE_SKILL_DIR}/../../bin/construction-python \ ${CLAUDE_SKILL_DIR}/scripts/markup_viewports.py \ --base "{sheet_image_path}" \ --items "{items_json_path}" \ --output "{output_path}" \ --color "amber" \ --label-style "titled" ``` Items JSON format (pixel coordinates): ```json [ { "x": 100, "y": 200, "width": 4000, "height": 3200, "shape": "rect", "label": "1 - FIRST FLOOR PLAN (plan)" }, { "x": 4200, "y": 200, "width": 3800, "height": 1500, "shape": "rect", "label": "A - BUILDING SECTION (section)" } ] ``` ### Verification report After processing all sheets, produce a summary: ``` VIEWPORT HIGHLIGHTING SUMMARY ============================== Sheets processed: 12 Total viewports created: 47 Sheet | Views | Types -----------|-------|------------------ A2.01 | 3 | plan, section, section A2.02 | 2 | plan, plan A5.01 | 8 | detail (x8) A5.02 | 6 | detail (x4), section (x2) ... Element containment: A2.01 / FIRST FLOOR PLAN: 342 elements