llm-pipeline-stabilization

name	llm_pipeline_stabilization
description	Standard operating procedure for debugging LLM randomness or pipeline instability.

LLM Pipeline Stabilization Guidelines

When the user asks to fix issues related to "LLM randomness", "unstable output", or "sporadic failures" in an agent pipeline:

1. Check the Harness First:

   • Ensure the LLM temperature is strictly 0.0 (or another pinned value the pipeline is designed around).
   • Verify that output formats are strictly checked (e.g. JSON schema validation) and the request is retried upon failure.
   • Do NOT attempt to fix structural failures by simply adding "please output correctly" to the prompt without fixing the underlying parameters.

2. Align Generator and QA Rules:

   • When a generating stage keeps failing a QA/acceptance stage, check whether the generator's prompt or parsing logic rewards behavior that the QA stage penalizes (a misaligned reward loop).
   • Any update to parsing logic must stay consistent with the acceptance limits enforced by the QA stage; if the limits change, the parser must change with them.