check-inference

name	check-inference
description	Probe a model inference WebSocket server (e.g. `serve_policy`) and validate the response — using the `geniesim benchmark check-inference` CLI verb, which wraps the benchmark package's `check_inference.py`. Trigger: When the user asks to "check inference", "校验模型推理", "test inference server", "verify policy server", "ping the model", or provides an IP/port and wants to confirm a serve_policy / WebSocket inference server is working before running benchmarks.
license	MPL-2.0
metadata	{"author":"genie-sim","version":"2.0"}
prerequisites	["geniesim_cli:fresh-machine-setup"]
inputs	[{"name":"payload","desc":"Path to a corobot `.pkl` payload (defaults to the bundled corobot_payload.pkl)","required":false},{"name":"infer_host","desc":"IP:PORT of the inference WebSocket server","required":true},{"name":"iters","desc":"Number of consecutive requests to send (catch flakiness)","required":false,"default":"1"}]
outputs	[{"desc":"Pass/fail verdict with per-dim min/max/mean/std/NaN/Inf checks and round-trip latency; non-zero exit on failure"}]

When to Use

• Sanity-check whether a running inference server actually accepts requests and returns valid actions, before launching a full task.
• User provides ip:port and a payload, and asks to verify connectivity / output validity.
• Quick smoke test in CI / pre-deploy.
• Diagnosing "the benchmark hangs / outputs garbage" — this probe surfaces protocol mismatches and NaN/Inf in actions before you sink time into a full simulator launch.

Do not use for:

• Running the benchmark itself → use the run-benchmark skill.
• Submitting jobs to the Challenge platform → challenge-submit-job.

What This Skill Does

Sends a saved corobot .pkl payload to ws://<HOST>:<PORT> and validates the reply:

1. Loads the payload.
2. Connects to the WebSocket server using msgpack-numpy.
3. Sends one request, receives one action chunk.
4. Validates: schema (key presence), per-dim min/max/mean/std, NaN/Inf flags, out-of-range checks (kind-aware: JOINT_ABS in radians, EEF_ABS in meters+quat, gripper in [0,1]), large jumps from the input state.

The payload is a corobot JSON-RPC envelope — {"method": "infer", "params": {...}} — and the server replies with {"result": {"left_arm": …, "right_arm": …, …}} (or {"error": …}).

Required Input

The user must provide:

• HOST — server IP (e.g. 127.0.0.1).
• PORT — server port (e.g. 8999).

PAYLOAD is optional — it defaults to the bundled corobot_payload.pkl. Pass a path only to override it (see Generating a payload below). If host/port are missing, ask the user before running.

How to Run

# Bundled payload — just point it at the server
geniesim benchmark check-inference --infer-host=<HOST>:<PORT>

# Override with your own payload (positional)
geniesim benchmark check-inference debug_preview/debug_0001.pkl \
    --host <HOST> --port <PORT>

If geniesim isn't on $PATH (the launcher wasn't installed), substitute python3 -m geniesim_cli benchmark check-inference … — same args, same behaviour.