nim-dry-run-integration

name	nim-dry-run-integration
description	Implement real LLM provider request-building with dry-run mode that validates payloads, prompts, and secrets without making network calls
source	auto-skill
extracted_at	2026-05-30T05:45:30.593Z

NIM Dry-Run Integration

When integrating real LLM providers (NVIDIA NIM, OpenAI, etc.) into an agent simulation, implement a dry-run mode that builds and validates the exact outbound payload without sending network calls. This allows prompt review, payload inspection, and secret validation before any live API usage.

Provider Mode Progression

mock → nim-dry-run → nim-live-disabled → nim-live (future)

Mode	Network Calls	Use Case
mock	None	Development, testing
nim-dry-run	None	Payload validation, prompt review
nim-live-disabled	None	Explicit safety lock
nim-live	Yes	Production (future)

Procedure

1. Create the NIM provider with dry-run mode

class NvidiaNimProvider(BaseProvider):
    def __init__(self, name, api_key_env, model, base_url, mode="nim-dry-run"):
        self._api_key_env = api_key_env
        self._model = model
        self._base_url = base_url
        self._mode = mode  # mock, nim-dry-run, nim-live-disabled
        self._last_dry_run_payload = None

    def generate(self, prompt, agent, tick):
        api_key = os.environ.get(self._api_key_env, "")

        if self._mode == "nim-dry-run":
            return self._dry_run(prompt, agent, tick, api_key)
        elif self._mode == "nim-live-disabled":
            return self._live_disabled(prompt, agent, tick)
        else:
            return self._mock_fallback(prompt, agent, tick)

2. Dry-run builds the exact outbound payload

def _dry_run(self, prompt, agent, tick, api_key, start):
    payload = {
        "model": self._model,
        "messages": [{"role": "user", "content": prompt}],
        "temperature": 0.7,
        "max_tokens": 500,
        "response_format": {"type": "json_object"},
    }

    # Validate payload structure
    errors = self._validate_payload(payload)
    self._last_dry_run_payload = payload  # Store for inspection

    # Log redacted summary
    payload_json = json.dumps(payload)
    logger.info(
        "NIM dry-run: agent=%s model=%s payload_chars=%d estimated_tokens=%d "
        "key_present=%s key_preview=%s endpoint=%s validation_errors=%d",
        agent, self._model, len(payload_json), len(payload_json) // 4,
        bool(api_key), redact_key_preview(api_key), self._base_url, len(errors),
    )

    # Return mock response — NO network call
    return json.dumps({
        "thought": f"[nim-dry-run:{agent}:{tick}] Payload built but not sent.",
        "action": f"{agent} performs a dry-run action for tick {tick}.",
    })

3. Redact secrets in all logging

def redact_key_preview(key: str) -> str:
    if not key:
        return "(not set)"
    if len(key) <= 4:
        return "(set, short)"
    return f"...{key[-4:]}"  # Never show more than last 4 chars

Never log: full API keys, auth headers, raw request bodies.

4. Build structured prompts with contracts

Every agent prompt must include:

SOURCE_BOUNDARY_WARNING = (
    "\n\nIMPORTANT SOURCE BOUNDARY:\n"
    "You are a simulation character inside a Bible-inspired world.\n"
    "You may act, think, remember, and respond.\n"
    "You may NOT declare new scripture.\n"
    "You may NOT claim your generated actions are canon.\n"
    "When uncertain, your output will be marked as SIMULATION_EVENT.\n"
    "CANON_ANCHOR is only valid when you reference a specific scripture passage.\n"
)

NIM_OUTPUT_SCHEMA = {
    "type": "object",
    "required": ["thought", "action"],
    "properties": {
        "thought": {"type": "string"},
        "action": {"type": "string"},
        "label": {"type": "string", "enum": ["CANON_ANCHOR", "INTERPRETIVE_BRIDGE", "SIMULATION_EVENT"]},
        "source_anchor": {"type": "string"},
        "canon_claim": {"type": "string"},
        "simulation_note": {"type": "string"},
    },
}

def build_agent_prompt(agent_name, role, world_snapshot, memory_context, traits=None):
    prompt = f"You are {agent_name}. Role: {role}.\n\n## World State\n"
    for key, value in world_snapshot.items():
        prompt += f"- {key}: {value}\n"
    prompt += f"\n## Recent Memories\n{memory_context}\n"
    if traits:
        prompt += f"\n## Current Traits\n"
        for k, v in traits.items():
            prompt += f"- {k}: {v}\n"
    prompt += f"\n## Output Format\nRespond with JSON ONLY.\nSchema: {json.dumps(NIM_OUTPUT_SCHEMA)}\n"
    prompt += SOURCE_BOUNDARY_WARNING
    prompt += f"\n\nWhat do you think and do? Return ONLY valid JSON."
    return prompt

5. Per-agent configuration

# Config
adam_nim_model: str = "meta/llama-3.1-8b-instruct"
eve_nim_model: str = "mistralai/mixtral-8x7b-instruct-v0.1"
nvidia_nim_key_adam_env: str = "NVIDIA_NIM_KEY_ADAM"
nvidia_nim_key_eve_env: str = "NVIDIA_NIM_KEY_EVE"

# Setup
adam.set_provider(NvidiaNimProvider(
    name="adam_nim",
    api_key_env=config.nvidia_nim_key_adam_env,
    model=config.adam_nim_model,
    mode=adam_cfg.provider,  # "nim-dry-run"
))

6. Payload validation

def _validate_payload(self, payload):
    errors = []
    if not payload.get("model"):
        errors.append("Missing or empty 'model'")
    if not payload.get("messages"):
        errors.append("Missing or empty 'messages'")
    for i, msg in enumerate(payload.get("messages", [])):
        if "role" not in msg:
            errors.append(f"Message {i}: missing 'role'")
        if not msg.get("content"):
            errors.append(f"Message {i}: missing 'content'")
    t = payload.get("temperature")
    if t is not None and (not isinstance(t, (int, float)) or t < 0 or t > 2):
        errors.append(f"Invalid temperature: {t}")
    return errors

7. API endpoint for dry-run inspection

@app.get("/api/providers")
def get_providers():
    payload_preview = None
    if provider.last_dry_run_payload:
        p = provider.last_dry_run_payload
        payload_preview = {
            "model": p.get("model"),
            "message_count": len(p.get("messages", [])),
            "payload_chars": len(json.dumps(p)),
            "estimated_tokens": len(json.dumps(p)) // 4,
        }
    return {
        "config": {"provider_mode": config.provider_mode, "is_dry_run": config.is_dry_run()},
        "agents": {
            "Adam": {"provider": ..., "model": ..., "key_present": ..., "dry_run_payload": payload_preview},
        },
        "call_log": call_log.summary(),
    }

8. Dry-run tests

def test_dry_run_builds_payload_no_network():
    provider = NvidiaNimProvider(mode="nim-dry-run", api_key_env="TEST_KEY")
    os.environ["TEST_KEY"] = "sk-test-fake"
    response = provider.generate("test", "TestAgent", 1)

    assert provider.last_dry_run_payload is not None  # Payload built
    assert "model" in provider.last_dry_run_payload   # Valid structure
    assert "dry-run" in json.loads(response)["thought"].lower()  # Marker present
    del os.environ["TEST_KEY"]

Key Rules

• Dry-run builds the exact payload — same structure as a real call, just not sent
• No network calls in dry-run mode — verified by test suite
• Secrets always redacted — redact_key_preview() shows at most last 4 chars
• Prompt includes source-boundary warning — every agent prompt
• Prompt includes output schema — JSON-only, required fields specified
• Per-agent model config — Adam and Eve can use different models
• Payload stored for inspection — last_dry_run_payload property
• Validation errors logged — but don't block dry-run (they're informational)
• Dry-run marker in response — so downstream code knows it's not real

Anti-Patterns

• ❌ Sending network calls in dry-run mode
• ❌ Logging full API keys anywhere
• ❌ Building prompts without source-boundary warning
• ❌ Not including output schema in prompts
• ❌ Sharing a single model for all agents (use per-agent config)
• ✅ Always verify dry-run with tests before enabling live mode
• ✅ Use /api/providers endpoint to inspect dry-run payloads
• ✅ Review prompt content and payload structure before going live