メニュー
Add a robot arm to XRoboToolkit MuJoCo teleop in blupe-evals — the recipe plus the EE tool-frame gotcha that cost us a day on YAM. Use when integrating a new arm or debugging "gizmo/wrist orientation is wrong" teleop.
Codex または Claude でインストール この Prompt をコピーして Codex、Claude、または他のアシスタントに貼り付けると、Skill ページを確認してインストールできます。
SOC 職業分類に基づく
| name | adding-a-new-arm |
| description | Add a robot arm to XRoboToolkit MuJoCo teleop in blupe-evals — the recipe plus the EE tool-frame gotcha that cost us a day on YAM. Use when integrating a new arm or debugging "gizmo/wrist orientation is wrong" teleop. |
teleop_details.md).xml, sim) and a placo URDF (.urdf, IK) for the same robot:
identical link & joint names, identical kinematics. Generate the URDF from the MJCF
so they can't drift (scripts/gen_yam_urdf.py).scene.xml (vis_target) — the goal gizmo.link_name, pose_source, control_trigger, vis_target, control_mode="pose".assets/<arm>/. Ground truth = the MJCF the vendor's
stack actually drives the real arm with. (i2rt loads only the .xml; its .urdf is stale.)home keyframe, real mass on any
massless EE link (else NaN), armature/damping, and visual-only geoms
(contype=0 conaffinity=0) so the home pose can't self-collide and jam a joint.grasp/tool0 body whose
+Z = the gripper approach axis.vis_target mocap to scene.xml.link_name=<tool frame>, control_mode="pose").scripts/test_yam_ik.py): consistency (placo EE == MuJoCo
EE ≈ 0), hold-test (goal=current ⇒ no motion = no jump on clutch), track-test.Symptom: teleop works for translation but the goal gizmo's blue (Z) axis points the wrong way (e.g. down) while the gripper points forward, and wrist rotation feels wrong / flails.
Cause: we pointed the IK at the raw last body link6 from the standalone yam.xml. That
link is a placeholder (pos="0 0 0" quat="1 0 0 0", joint6 axis="0 0 1"). i2rt never uses
it — combine_arm_and_gripper_xml overrides link6's pos/quat/joint-axis from the gripper mount
config when attaching the gripper. The real link6 is
pos="0 -0.042 0.0405" quat="0.5 -0.5 -0.5 -0.5", joint6 axis="0 0 -1". So the EE frame's +Z
pointed down, not along the gripper.
Why it was sneaky: the URDF was generated from the same bad MJCF, so sim and IK were perfectly self-consistent — every consistency/hold/track test passed. It was "consistent with a wrong model." Self-consistency ≠ correct; the model must match the real arm.
Fix:
link6 (pos/quat/joint6 axis) from i2rt's combined model:
combine_arm_and_gripper_xml(ArmType.YAM, GripperType.LINEAR_4310).grasp EE body matching i2rt's grasp_site — rel link6 pos="0 0 -0.1347" quat="0 1 0 0",
+Z = approach, 13.5 cm out.link_name="grasp", regenerate URDF.Result: blue ∥ gripper, natural 6-DOF wrist, and the model matches the real arm (same URDF works on hardware).
General rule: the EE/IK frame must be the tool/grasp frame from the COMBINED arm+gripper
model, never the raw last link of a standalone arm xml. UR5e worked out of the box because
mujoco_menagerie ships a proper tool0 frame; YAM's vendored xml did not.
from i2rt.robots.utils import combine_arm_and_gripper_xml, ArmType, GripperType
xml = combine_arm_and_gripper_xml(ArmType.YAM, GripperType.LINEAR_4310) # writes /tmp/i2rt_*.xml
# load in mujoco, read grasp_site site_xmat/site_xpos, express relative to link6:
# R_rel = R_link6.T @ R_site ; p_rel = R_link6.T @ (p_site - p_link6)
Needs python-can click pydantic rich tabulate pyyaml in the env.
assets/yam/yam.xml — MJCF, real link6 + grasp tool frame, actuators, home keyframe.assets/yam/yam.urdf — generated from the MJCF (scripts/gen_yam_urdf.py).assets/yam/scene.xml — floor + right_target mocap gizmo.scripts/teleop_yam_mujoco.py — stock full-pose config, link_name="grasp".scripts/test_yam_ik.py — consistency / hold / track checks.Operate and debug the distributed teleop infrastructure (Mac+Quest operator node, GCP cloud relay, Orin+YAM robot node, and remote GPU model services). Use when: something is down or stale (video, joints, input, policy, tracker); choosing/provisioning hardware for inference; restarting/deploying any tier; remote-managing a GPU or the Orin over ssh; or when a new infra issue gets root-caused — append it to the Issue log here so it is never re-debugged from scratch.
Logging conventions + latency instrumentation for the teleop stack. Use when: adding log lines or latency probes to any tier; diagnosing teleop lag from [lat] lines; hunting for the right log file on the Mac/Orin/relay; or when a new logging convention is adopted — record it here so every tier stays greppable the same way.
Read the docs BEFORE writing code whenever we start using a new external project, library, framework, tool, or file format — or try a new part of one we already use. Vendor the authoritative docs into docs/refs/<project>/ as a persistent, searchable reference, then do an in-depth search of those docs + the project's own examples for OUR specific use case to see whether guidance already exists, and only then implement. Use when: adding/integrating a new dependency or format; doing something new with an existing one; or when files/APIs "don't add up" and we're trial-and-erroring against an external tool. The project's docs are the source of truth — not assumptions, not training-data memory, not "it looks like a one-liner."
Quick symptom -> fix lookup for known teleop papercuts (no re-debugging). Use when: a familiar small failure appears (dead joystick/menu, stale saved IP, NO SIGNAL flash, B-button surprises, headset doze fallout) — check here FIRST before investigating; or when a small error gets a confirmed quick fix — add it here. Deep root-caused outages belong in the infra skill's Issue log instead; link entries there when one exists.
Project coding and deployment hygiene for blupe-evals/LeLab/Jetson work. Use when Codex is modifying blupe-evals, patching installed LeLab, changing Jetson setup, adding scripts, running services, or deciding where code/configuration should live. Enforces that durable code lives in the repository, environment changes are reproducible from repo-owned scripts or patches, and Jetson state can be wiped and rebuilt without losing work.
Logging conventions + latency instrumentation for the teleop stack. Use when: adding log lines or latency probes to any tier; diagnosing teleop lag from [lat] lines; hunting for the right log file on the Mac/Orin/relay; or when a new logging convention is adopted — record it here so every tier stays greppable the same way.