open-world-object-goal-navigation

# Skill: open-world-object-goal-navigation ## 1. Capability Definition & Real Case * **Professional Definition**: This capability measures whether an embodied agent can interpret a compact semantic goal description and convert it into a physically grounded navigation strategy that reaches a target object in an open world. It requires semantic grounding, environment-aware search, obstacle-aware movement, and correct stopping behavior near the target. * **Dimension Hierarchy**: Goal-Directed Exploration->Semantic Goal Navigation->open-world-object-goal-navigation ### Real Case **[Case 1]** * **Initial Environment**: Aerial observations begin over a city park containing benches, trees, a playground, and walking paths. The agent has four synchronized views: front, left, right, and down, plus depth for each view. * **Real Question**: Help me search for a small human. The target is a child wearing a pale green shirt and dark pants, seated in a relaxed posture. * **Real Trajectory**: The agent first scans the park region instead of nearby road objects, descends slightly to improve visual detail, orients toward the playground seating area, and stops once it is within target distance of the matching child. * **Real Answer**: The target is the seated child near the playground seating area. * **Why this demonstrates the capability**: The instruction includes category, scale, and appearance cues but no route. The agent must reason about where such a target is likely to appear and then search with egocentric perception only. This is a direct test of open-world object-goal navigation. --- **[Case 2]** * **Initial Environment**: The agent starts near a waterfront zone with roads behind it, boats by the shore, and several parked bicycles farther inland. The environment is cluttered and no global map is available. * **Real Question**: Find this target: name = boat; size = medium; description = a dark boat moored close to the waterline. * **Real Trajectory**: The agent rejects inland search directions, follows the water boundary, maintains obstacle-safe altitude, narrows to medium-sized boats, and stops near the dark moored instance. * **Real Answer**: The dark medium-sized boat at the waterline. * **Why this demonstrates the capability**: The environment contains semantically plausible distractors such as vehicles and outdoor furniture. Correct behavior requires object-scene co-occurrence reasoning and safe navigation toward the appropriate region. That is the core benchmark difficulty. --- **[Case 3]** * **Initial Environment**: The embodied agent begins above a mixed-use town scene with a road, a bus stop, trees, and scattered urban furniture. The target lies within the local search radius but is not visible from the first forward view. * **Real Question**: Find this target: name = bicycle; size = medium; description = a black bicycle resting near the roadside barrier. * **Real Trajectory**: The agent rotates to gather additional multi-view evidence, shifts laterally along the road, avoids colliding with signs and trees, and then issues Stop only after entering the success radius around the matching bicycle. * **Real Answer**: The black bicycle next to the roadside barrier. * **Why this demonstrates the capability**: The agent must combine semantic instruction parsing, active view gathering, and grounded stopping. A near miss or a visually similar bicycle in a different context should not count. This captures the physical and semantic demands of object-goal navigation. ## Pipeline Execution Instructions To synthesize data for this capability, you must strictly follow a 3-phase pipeline. **Do not hallucinate steps.** Read the corresponding reference file for each phase sequentially: 1. **Phase 1: Environment Exploration** Read the exploration guidelines to discover raw knowledge seeds: `references/EXPLORATION.md` 2. **Phase 2: Trajectory Selection** Once Phase 1 is complete, read the selection criteria to evaluate the trajectory: `references/SELECTION.md` 3. **Phase 3: Data Synthesis** Once a trajectory passes Phase 2, read the synthesis instructions to generate the final data: `references/SYNTHESIS.md`