fine-grained-widget-grounding
Use this skill when the user wants tiny, crowded, or easy-to-misclick GUI targets. Trigger it for requests like “make the answer depend on a small icon,” “use repeated buttons so the agent has to pick the right one,” “test tiny checkboxes or reply icons,” or “make it fail unless it clicks the exact small widget.” This skill is for GUI tasks where precision, counting, and local disambiguation matter more than broad page understanding.
소스 정보
- 저장소
- Dingxingdi/paper_fast_search_backup
- 최근 소스 활동
- 2026년 4월 8일 15:14
- 감지된 SKILL.md 언어
- 영어
- 스타
- 0
- 포크
- 0
설치 방법
기본적으로 소스를 먼저 확인하는 Prompt가 선택됩니다. 직접 명령으로 전환하거나 로컬 사본을 다운로드할 수도 있습니다.
소스 파일 검토
설치 여부를 결정하기 전에 SKILL.md와 SkillsMP에 표시된 보조 파일을 읽어 보세요.
파일 탐색기
4 개 파일SKILL.md 표시 중
SKILL.md
소스 지침 · 읽기 전용 미리보기- name
- fine-grained-widget-grounding
- description
- Use this skill when the user wants tiny, crowded, or easy-to-misclick GUI targets. Trigger it for requests like “make the answer depend on a small icon,” “use repeated buttons so the agent has to pick the right one,” “test tiny checkboxes or reply icons,” or “make it fail unless it clicks the exact small widget.” This skill is for GUI tasks where precision, counting, and local disambiguation matter more than broad page understanding.
# Skill: fine-grained-widget-grounding
## 1. Capability Definition & Real Case
* **Professional Definition**: The capability to localize and act on a precise GUI widget in dense, repetitive, or small-scale layouts, especially when the target belongs to a long-tail element type such as icons, toggles, radio buttons, or small action affordances. The agent must separate the true target from visually similar distractors and execute a pixel-accurate action on the right element.
* **Dimension Hierarchy**: GUI Perception and Environment Modeling->Element Grounding->fine-grained-widget-grounding
### Real Case
**[Case 1]**
* **Initial Environment**: An email client shows an inbox with several messages arranged in a vertical list. Each row contains multiple compact icons, and the target action is the small reply icon associated specifically with the second email rather than the first or third.
* **Real Question**: Click the reply icon near the second email.
* **Real Trajectory**: Count the email rows from top to bottom, inspect the action cluster attached to the second row, distinguish the reply icon from adjacent icons such as archive or more-actions, and click that precise icon.
* **Real Answer**: The reply composer for the second email is opened.
* **Why this demonstrates the capability**: This case stresses fine-grained localization because multiple nearly identical icons are present and the instruction depends on ordinal position. The agent must jointly solve row counting, icon classification, and precise pointing under a dense layout. That makes the task a direct probe of high-resolution widget grounding rather than generic page comprehension.
## Pipeline Execution Instructions
To synthesize data for this capability, you must strictly follow a 3-phase pipeline. **Do not hallucinate steps.** Read the corresponding reference file for each phase sequentially:
1. **Phase 1: Environment Exploration**
Read the exploration guidelines to discover raw knowledge seeds:
`references/EXPLORATION.md`
2. **Phase 2: Trajectory Selection**
Once Phase 1 is complete, read the selection criteria to evaluate the trajectory:
`references/SELECTION.md`
3. **Phase 3: Data Synthesis**
Once a trajectory passes Phase 2, read the synthesis instructions to generate the final data:
`references/SYNTHESIS.md`
GitHub에서 보기