harbor-hf

name	harbor-hf
description	Plan and profile Harbor benchmark runs, then launch and monitor them. Repair and verify results before publishing through the hosted Harbor-HF control service and Hugging Face infrastructure.

Harbor-HF operations

Use this skill for Harbor benchmark work on Hugging Face Jobs, Inference Providers, and Inference Endpoints.

The steady-state service has two persistent resources: one publicly reachable, application-protected control Space and one private <artifact-bucket> Bucket. The Space runs the TypeScript API, reconciler, disposable SQLite projection, and React console. The Bucket stores immutable control records, profiles, evidence, normalized results, and catalogs. Anonymous callers can reach static, login, callback, health, and the official GET /api/v1/leaderboard snapshot. Control access requires an access-listed identity or a short-lived worker capability.

Do not create a run-specific repository, Space, Bucket, Dataset, schedule, lease store, status store, backup store, or result service. A new persistent resource needs an explicit failure-domain or access reason and operator approval.

The control Space has two operator-managed persistent secrets. HF_TOKEN is the control credential; keep it private and never forward it to a Job, benchmark agent, model server, browser, log, action payload, or evidence object. HF_INFERENCE_TOKEN is a separate inference-only credential. Pass it only to a reviewed worker whose locked deployment profile marks inference as required, then isolate it in the root-owned inference bridge. Workers also receive a short-lived signed capability scoped to their Run, launch action, and tasks, and they never receive a writable mount of the canonical control Bucket.

Read before operating

Read the relevant complete documents:

• docs/CONTROL_SERVICE.md
• docs/architecture.md
• docs/run-spec.md
• docs/harbor-integration-contract.md
• docs/trial-evidence-bundle.md
• references/planning-and-capacity.md
• references/launch-and-monitoring.md
• references/recovery.md
• references/evidence-and-publication.md
• references/provider-agents-and-security.md
• references/operator-checklists.md

Use the paid-compute-launch skill before launching, scaling, retrying, or automatically continuing paid accelerator work.

Keep one run path

Harbor-HF must treat benchmark and model names as data. The same rule applies to harness names. Reject any run design that adds name-based branches, per-benchmark or per-model scripts, source parsers, profile generators, workers, API routes, or schema fields to the core service.

Use the pinned Harbor version in an isolated preparation Job without persistent secrets. Harbor resolves the requested job and returns its exact lock.json. The control service validates and stores that lock before execution. Every retry and recovery action uses the same lock and never resolves the benchmark source again.

A supported benchmark or model must use the existing run path through configuration and immutable records. The same rule applies to supported harnesses. New harness implementation code belongs in a Harbor agent plugin behind the common agent interface. If work needs a missing feature, add a general capability at the correct Harbor, agent, provider, or Hugging Face adapter boundary, or report it as unsupported.

Before implementation or launch, confirm that another compatible benchmark or model could use the same path without core code changes or a new package script. Apply the same check to harness support.

Required workflow

Inspect state

1. Confirm the repository state and current commit.
2. Confirm the control service is ready.
3. Inspect promoted profiles and every run or Job. Check the Endpoint and result views plus the audit view.
4. Check for an existing run or physical action before creating anything.
5. Classify the request as a new run, infrastructure repair, audit, publication recovery, or migration.

export HARBOR_HF_CONTROL_URL=https://<control-space>.hf.space
uv run harbor-hf status
uv run harbor-hf profiles
uv run harbor-hf capacity
uv run harbor-hf run list
uv run harbor-hf jobs
uv run harbor-hf endpoints

Resolve immutable inputs

Resolve and record:

• benchmark source revision and exact task input digests
• model ID and revision
• harness name and version plus image and configuration, prompt, tool, or skill revisions
• worker image digest and reviewed command
• deployment hardware and route, timeout, plus credential boundary
• launch policy with its physical-attempt limit and reservation plus ceiling and publication role

Aliases are only submission conveniences. The run lock must contain exact resolved profile identities and task digests.

Apply the cost and capacity gates

Use representative wall time and throughput measurements. Include failed Jobs, replacements, provider calls, and endpoint active time in the cost range. Compare the result with the remaining cumulative authorization.

Stop when cost, hardware, model, route, method, or checkpoint assumptions differ from the approved launch.

Launch once

uv run harbor-hf run submit \
  --benchmark <benchmark-profile> \
  --model <model-profile> \
  --harness <harness-profile> \
  --deployment <deployment-profile> \
  --launch-policy <launch-policy-profile> \
  --ceiling-microusd <approved-ceiling> \
  --idempotency-key <stable-request-key> \
  --yes

Preserve the returned run ID and action ID. Repeating the same actor and idempotency key must adopt the existing request.

Monitor logical and physical state

uv run harbor-hf run status <run-id>
uv run harbor-hf jobs
uv run harbor-hf endpoints
uv run harbor-hf results
uv run harbor-hf audit

A Job log or remote terminal state is not authoritative. The Bucket must contain the selected attempt receipt with its terminal selection, recorded cost, and evidence digest. Before posting that receipt, a worker uses its short-lived capability to upload content-addressed evidence chunks and a canonical manifest. The control service verifies the manifest and every chunk during acceptance and replay. Jobs never receive HF_TOKEN or a writable canonical Bucket mount.

Do not run benchmark tasks, model servers, or provider agents on the operator machine.

Repair only infrastructure failures

uv run harbor-hf run retry-infrastructure <run-id> \
  --task <task-id> \
  --reason "<infrastructure reason>" \
  --yes

uv run harbor-hf run retry-infrastructure <run-id> \
  --all-eligible \
  --reason "<infrastructure reason>" \
  --yes

A retry is valid when the latest attempt is an eligible infrastructure failure and the physical-attempt limit remains. An infrastructure seal is replaceable. Semantic outcomes, refusals, verifier failures, and benchmark failures are terminal. The same is true for cancellations and benchmark timeouts.

Never rerun a valid logical task. Never turn publication recovery into inference.

Verify cleanup and publication

A run is complete only when:

• every logical task is sealed
• no control action is pending
• every owned endpoint is paused with zero ready replicas
• spend remains within the ceiling
• publication receipts and normalized objects are durable
• catalog provenance validates
• rebuilding SQLite from Bucket records produces the same state

Stop conditions

Stop immediately for:

• credential, cookie, header, or private route exposure
• input digest or source revision mismatch
• a deterministic shared worker defect
• unsupported backend, fallback, emulation, or runtime mismatch
• spend above the approved ceiling
• duplicate logical execution
• endpoint cleanup that cannot be verified
• a projection or immutable-object integrity error

Record the durable evidence and ask for an operator decision. Do not bypass a stop with a new resource, credential, compatibility path, or unreviewed runtime.