| name | nemo-relay-tune-adaptive-hints |
| description | Consume NeMo Relay adaptive hints, predictions, latency sensitivity, ACG diagnostics, or tool-parallelism guidance safely in application logic after the adaptive plugin is already configured |
| author | NVIDIA Corporation and Affiliates |
| license | Apache-2.0 |
Use Adaptive Hints And Guidance
Use This When
Use this skill when the adaptive plugin is already configured and the
application wants to consume adaptive outputs such as model-request hints,
parallelism guidance, or ACG diagnostics.
Do Not Use This When
Do not use this skill to design the first adaptive rollout or to configure the
plugin from scratch. Use nemo-relay-tune-performance or
nemo-relay-tune-adaptive-config first.
Focus Areas
adaptive_hints metadata injected into outgoing model requests.- Latency sensitivity and request-local guidance.
- Parallel groups or tool-parallelism guidance.
- Adaptive Cache Governor reports and diagnostics during rollout.
Embedded Hint Semantics
- Adaptive hints are request-intercept behavior. They can inject metadata into a
configured header or body path; the default body path is
nvext.agent_hints.
- Hint configuration includes
priority, break_chain, inject_header, and
inject_body_path. Lower priority values run earlier.
- Tool parallelism can run in
observe_only, inject_hints, or schedule.
Use observe_only until tool idempotency and race behavior are understood.
- ACG output is provider-specific prompt-cache planning guidance. Use more
samples, raise stability thresholds, or switch to
passthrough when cache
planning is unstable.
set_latency_sensitivity(...) is a request-local execution hint, not
persistent adaptive configuration.- Normal adaptive runtime behavior should come from explicit config objects, not
environment variables.
NEMO_RELAY_ACG_DEBUG is for cache-governor diagnostics
and NEMO_RELAY_RUN_REDIS_TESTS is for Redis-backed tests.
Default Path
- Confirm adaptive telemetry and config validation are working.
- Identify where the hint is injected or surfaced in the chosen binding.
- Keep hints advisory unless the consuming API defines stronger semantics.
- Test behavior when no prediction or hint is available.
- Compare application output against the baseline after enabling hints.
- Escalate from hints to scheduling only after idempotency and race behavior
are understood.
Failure Modes To Avoid
- Do not make application correctness depend on a prediction being present.
- Do not hide behavior changes behind adaptive hints without runtime evidence.
- Do not enable scheduling for non-idempotent or order-sensitive tools.
- Do not adjust intercept priority without checking other request intercepts.
Related Skills
nemo-relay-tune-performancenemo-relay-tune-adaptive-confignemo-relay-debug-runtime-integration
