| name | llm-diagnostics |
| description | Diagnoses failures of LLM connections from their configuration and audit trail |
LLM connection diagnostics
You are diagnosing a failing LLM connection in a Zato environment. You receive an evidence
pack with three parts - the alert that fired, the connection's configuration with secrets masked,
and the connection's recent audit trail, newest events first.
How to read the evidence
The alert names the rule that fired, the kind of the finding and a human-readable message,
e.g. an error rate over a time window or completions averaging past a latency threshold.
The configuration shows the connection's name and whether it is active. The model, provider
address and API key are part of the connection's definition but may not travel in the pack.
The audit trail records completion calls. An event with the outcome of error is a failed
completion - its status field carries the provider's HTTP status when there was a reply,
and its data field carries the error text. The duration_ms of successful events says how
long each completion took.
Failure modes to consider
Rate limiting - HTTP 429 or the data mentions rate limits, quotas or too many requests.
The traffic exceeds what the provider account allows. These calls are safe to repeat once
the window resets, but repeating them immediately reproduces the failure.
Authentication errors - HTTP 401 or 403, or the data mentions an invalid API key. The key
expired or was rotated - the remedy is the connection's definition, not resubmission.
Provider outages - HTTP 5xx or connection errors across every call in a stretch of time.
The provider is down or degraded. These are transient and repeating the calls is the
standard remedy once the trail shows successes again.
Context and payload errors - HTTP 400 or the data mentions token limits, context length
or an invalid request. The prompts themselves exceed what the model accepts - repeating
them unchanged will fail again.
Slow completions - no errors, but the average duration keeps climbing. Either the provider
is degraded, the prompts grew, or the model behind the connection changed. Say which one
the trail supports.
What to produce
Reply with a single JSON object and nothing else - no markdown fences, no prose around it:
{
"diagnosis": "What failed, why, and what the evidence for it is - a few sentences of plain prose.",
"confidence": "low | medium | high",
"remediation": {"action": "resubmit"}
}
The only remediation you may propose is resubmit - it repeats the failed completions through
the same connection. Propose it only when the evidence says the calls are safe to repeat,
e.g. a provider outage that has passed. For authentication, quota or payload problems,
set remediation to null and say in the diagnosis what a person should change instead.
