| gap | {"allowed_tools":["robot.get_observation","grounding-dino.detect","vlm.query","vlm.query_yes_no","sam3.segment_box","sam3.segment_text","geometry.mask_to_world_points","geometry.exclude_robot_points","geometry.filter_and_compute_obb"],"exit_conditions":{"found":"A graspable target item distinct from the container was localized; target_obb/mask and container_obb/mask/cloud bound in the subgraph outputs. Route to a grasp skill.","none":"No target item remains — only the container is left, so every item is packed. This is the clean loop terminator; route to done (NOT abort — it is a success).","perception_failed":"Perception raised an actual error (empty detection is NOT this — that is the normal `none` path). Route to abort. Named so it is never confused with \"nothing left\"; do NOT route it to done."},"produces_outputs":{"target_obb":"OrientedBoundingBox","target_mask":"Mask","container_obb":"OrientedBoundingBox","container_mask":"Mask","container_cloud":"PointCloud"},"hard_rules":["perception_pipeline_invariants.md#emit-both-obb-and-mask","geometry_calling_conventions.md#obb-field-binding","The item perception MUST pass an `object_description` that names the grocery item categories AND explicitly excludes the container (e.g. \"a packaged grocery product such as a can, box, carton, jar, or bottle. Never the wicker basket or storage container.\"). This exclusion is what makes the tournament return \"no item\" once only the basket remains — it is the whole basis of the clean `none` loop-exit. Do NOT rely on geometry to reject the basket.\n","This subgraph declares NO inputs (`inputs: {}`) and is fully self-contained. Every value that tells the perceive nodes WHAT to look for is a LITERAL string hardcoded in the node — use exactly the node input names `object_name` and `object_description` from examples/canonical_subgraph.json (do not rename them to `item_name`, `container_name`, `item_description`, etc.). DO NOT lift ANY of them to a subgraph input and DO NOT write `Ref(\"in.<anything>\")` for them: there is no upstream producer, so it fails validation (W8: no upstream producer). The basket and item names are known at authoring time — write them in.\n","The `decide` router (scripts/<sg>/decide_next_item.py) sits BETWEEN `perceive_item` and `filter_obb_item`, and maps `found -> filter_obb_item`, `none -> none`. Declare `success_values: [\"found\", \"none\"]` so BOTH are clean exits; only genuine perception errors fall through to `on_error: perception_failed`.\n","TOP-LEVEL ROUTING (do not get this wrong): map this subgraph's `found -> grasp`, `none -> done` (all items packed — a SUCCESS), and `perception_failed -> abort`. `none` is the loop terminator, NOT `perception_failed`; never route `perception_failed` to done.\n","Perceive the item from the raw `observe.cameras`, but perceive the container through `exterior_view` (drop the wrist cam) — the angled eye-in-hand view bloats the container OBB by fusing neighbouring points.\n","WIRE A SINGLE CHAIN, never parallel branches: observe → exterior_view → perceive_container → filter_obb_container → perceive_item → decide. Do NOT add a second edge from `observe` straight to `perceive_item` to \"parallelize\" the two perceptions: the runtime scheduler has no join barrier, so `decide` (which consumes `filter_obb_container.obb`) can be scheduled by `perceive_item`'s completion before `filter_obb_container` has run — the $ref fails at runtime and the subgraph aborts on every trial (validator rule S12 rejects this shape).\n","RESTRICTED ITEM SETS: when the task's target set is restricted — a SUBSET (\"pack the milk and the tomato sauce…\") OR an EXCLUSION (\"pack everything except the milk…\") — i.e. whenever some visible object must NOT be packed, the `perceive_item` node MUST also pass `reject_unverified: true` (a boolean node input, next to object_name/object_description), and its `object_name`/ `object_description` literals MUST spell out the allowed set and every exclusion (excluded items AND the container). Why: the default perceive favors recall — an unverified best-guess pick is kept so a pack-ALL loop never stops early — but under a restricted query that same fallback returns EXCLUDED items: a subset loop packs things the task forbade, and an exclusion loop ends by grasping at the excluded item and aborting instead of exiting `none`. `reject_unverified: true` flips the gate to precision: no verified allowed-item ⇒ `found=false` ⇒ the loop exits `none` exactly when only excluded objects remain. Set it ONLY for restricted sets — for true pack-all tasks (\"all objects\", \"every item\") leave it unset; recall mode is what keeps those loops from terminating early. The allowed set must be a CONCRETE CATEGORY, never a tautology: write \"a packaged grocery product such as a can, box, carton, jar, or bottle — never the milk carton, and never the wicker basket\", NOT \"any object except the milk\" — \"any object\" is satisfied by every spurious detection (table patch, shadow, robot part), so the precision gate can never reject the leftovers and the loop cannot exit.\n"],"canonical_scripts":[{"perceive_dino_vlm":"scripts/perceive_dino_vlm.py"},{"exterior_view":"scripts/exterior_view.py"},{"decide_next_item":"scripts/decide_next_item.py"}],"prompts":{"vlm_pairwise":"prompts/vlm_pairwise.md"},"references":[{"title":"Perception pipeline invariants (emit obb + mask + cloud)","path":"references/perception_pipeline_invariants.md"},{"title":"Geometry tool calling conventions (output field binding)","path":"references/geometry_calling_conventions.md"}],"examples":[{"title":"Canonical loop-head subgraph (observe → container + item → decide)","path":"examples/canonical_subgraph.json"}],"streaming":false} |
