| name | nemo-experiments-upload |
| description | End-to-end guide for getting evaluation data into NeMo Platform Intake so it shows up in the Experiments leaderboard. Create an Experiment, create an Evaluation, then log traces and evaluator results via ATIF (Harbor), chat-completions, or OTLP and view the rollups in Studio. Use when a user wants to create named evaluation runs, publish evaluation results, or compare runs in NeMo Experiments. |
| triggers | ["log evaluation traces to experiments","upload experiment results","ingest evaluation data","how do I log to experiments","send evaluation traces to nemo experiments","publish evaluation results","log to experiments","upload harbor / atif results","get my eval data into nemo"] |
| not-for | ["nemo-intake (use for general instrumentation, telemetry ingestion, trace queries, or evaluator results outside an Experiments leaderboard)","nemo-evaluator (use to AUTHOR and RUN evaluations/metrics; this skill UPLOADS results)","nemo-status (use for a read-only platform health dashboard)","nemo-skill-selection (use for dispatch when intent is unclear)"] |
| preconditions | ["nemo_setup_complete","workspace_exists","clickhouse_ready"] |
| compatibility | nemo-platform >= 0.1.0; needs a reachable local or remote intake service (with auth and entities) backed by ClickHouse for rollups/results; talks HTTP to /apis/intake/v2 (curl only, no Docker); Experiments viewing in Studio is behind the VITE_FF_EXPERIMENT feature flag. |
| maturity | beta |
| license | Apache-2.0 |
| user-invocable | true |
| allowed-tools | ["Bash","Read","Write"] |
Log evaluation data to NeMo Intake
Get evaluation runs into the platform end-to-end: create an Experiment → create an Evaluation → log traces + scores to an ingest endpoint → see the rollups. The API calls the parent (the leaderboard) an Experiment and each row an Evaluation; the whole feature is called Experiments.
Everything below uses ${NMP_BASE_URL} (default http://localhost:8080) and a ${WORKSPACE}
(default default). Point NMP_BASE_URL at the local platform or a remote HTTPS origin. Reject
non-loopback http:// targets, and never send authentication across an HTTP redirect. All routes
are under /apis/intake/v2/workspaces/${WORKSPACE}.
Pre-flight
Confirm the target platform is reachable before doing anything. If this fails, report the target as
unreachable and stop; route to setup/nemo-status only for a local platform.
set -euo pipefail
: "${NMP_BASE_URL:=http://localhost:8080}"
: "${WORKSPACE:=default}"
nmp_authority=${NMP_BASE_URL#*://}
nmp_authority=${nmp_authority%%/*}
case "${nmp_authority}" in
*@*) echo "NMP_BASE_URL must not contain userinfo" >&2; exit 1 ;;
esac
case "${NMP_BASE_URL}" in
https://*) ;;
http://*)
case "${nmp_authority}" in
localhost|127.0.0.1) ;;
localhost:*|127.0.0.1:*)
nmp_port=${nmp_authority#*:}
case "${nmp_port}" in
|*[!0-9]*) >&2; 1 ;;
;;
*) >&2; 1 ;;
;;
*) >&2; 1 ;;
curl -sf >/dev/null;
>&2
1
