i4h-workflow-e2e

name	i4h-workflow-e2e
version	0.6.0
description	Run the full end-to-end agentic pipeline (record → mimic → annotate → replay → convert → visualize → finetune → validate). Use when asked to run the whole pipeline or do an e2e, smoke, or demo run.
license	Apache-2.0
metadata	{"author":"Isaac for Healthcare Team <[e-mail]>","tags":["isaac-for-healthcare","i4h","agentic-workflow","end-to-end","pipeline"]}

i4h Workflow — End-to-End

Purpose

Run the full end-to-end agentic pipeline (record, mimic, annotate/filter, replay, convert, visualize, finetune, validate). Use when the user asks to run the full pipeline, smoke the whole workflow, demo the workflow, or do an e2e run.

Base Code

These steps drive the i4h-workflows base code (the workflows/agentic/ tree). To reuse an existing checkout, set I4H_WORKFLOWS to its path (no clone happens). Otherwise this resolves the current repo, or clones to ~/i4h-workflows — pick that default without prompting. Run every command below from the resolved root:

# Resolve the i4h-workflows base code (provides workflows/agentic/).
ROOT="${I4H_WORKFLOWS:-$(git rev-parse --show-toplevel 2>/dev/null)}"
if [ ! -d "$ROOT/workflows/agentic" ]; then
  ROOT="${I4H_WORKFLOWS:-$HOME/i4h-workflows}"
  [ -d "$ROOT/workflows/agentic" ] || git clone https://github.com/isaac-for-healthcare/i4h-workflows "$ROOT"
fi
export I4H_WORKFLOWS="$ROOT"; cd "$ROOT"

Basics

• Env config (source of truth): workflows/agentic/config/environments/<env>.yaml — drives every stage for <env> (robot, task, policy, cameras, arena.max_timesteps, dataset.* mappings).
• Use the e2e script for full pipeline runs.
• For per-stage work, use the corresponding dataset/finetune/validate skills.
• assemble_trocar is inference-only; the e2e script skips finetune and checkpoint validation for it.

Dry Run

REPO_ROOT="${I4H_WORKFLOWS:-$(git rev-parse --show-toplevel 2>/dev/null)}"; [ -d "$REPO_ROOT/workflows/agentic" ] || REPO_ROOT="$HOME/i4h-workflows"
"${REPO_ROOT}/workflows/agentic/scripts/e2e/run.sh" --dry-run --env <env>

Run

"${REPO_ROOT}/workflows/agentic/scripts/e2e/run.sh" --env <env>

Flags

• --skip-mimic, --skip-annotate, --skip-replay, --skip-viz
• --from-stage <stage> --run-dir <existing-run> resumes from a prior run.

Stages: setup record mimic annotate replay convert viz finetune validate summary.

Outputs

The script prints RUN_DIR and symlinks it to runs/.latest. Subdirs:

• logs/ — per-stage logs, workflow.log (full teed output), and logs/SUMMARY.txt (the final summary report)
• data/
• lerobot/
• checkpoint/ (trainable envs only)

Monitor

run.sh runs every stage in the foreground and returns only when the whole pipeline ends, so track a long run from a separate shell (do not expect to query it from the shell that launched it):

tail -f "${REPO_ROOT}/workflows/agentic/runs/.latest/logs/workflow.log"   # live per-stage progress
cat    "${REPO_ROOT}/workflows/agentic/runs/.latest/logs/SUMMARY.txt"     # final report (once DONE)

Stop

"${REPO_ROOT}/workflows/agentic/stop.sh" all --env <env>

Prerequisites

• Workflow set up via [[i4h-workflow-setup]] (the .venv must exist); setup is also the first pipeline stage.
• A valid --env name to drive the run.
• For per-stage work, use the corresponding dataset/finetune/validate skills instead.

Limitations

• assemble_trocar is inference-only; the e2e script skips finetune and checkpoint validation for it.
• checkpoint/ outputs are produced for trainable envs only.
• Resuming requires both --from-stage <stage> and --run-dir <existing-run>.

Troubleshooting

• Error: .venv not found / module import fails - Cause: workflow not set up. Fix: run [[i4h-workflow-setup]] first.
• Error: env not recognized - Cause: wrong --env name. Fix: pass a valid env name; dry-run first with --dry-run --env <env>.
• Error: resume fails to find prior outputs - Cause: --from-stage used without a matching --run-dir. Fix: pass --from-stage <stage> --run-dir <existing-run>.
• Error: stale processes block a rerun - Cause: a previous pipeline session is still running. Fix: run stop.sh all --env <env> before retrying.

Final Response

Report env, run dir, skipped stages, per-stage success/failure, key artifact paths.