| name | run-the-requested-analysis |
| description | Use when the supplied data looks synthetic, degraded, incomplete or wrong, and whenever you are tempted to reframe the study around what you found about the inputs, the harness or the evaluation. Covers what to do with a real data problem without losing the study. |
A problem with the inputs is a finding to add, not a study to replace
You will often be right that the supplied data is a thin surrogate, that a file
is mislabelled, that the corpus cannot support the claim it was cut from. Noticing
that is good research. What follows it decides whether the run is worth anything.
The failure mode is consistent: the run investigates the inputs, finds something
real, and reorganises the whole study around that finding. The requested analysis
is then never run at all — not because it was impossible, but because the run
stopped being about it. From the inside this feels like following the evidence.
From the outside every requested result is missing.
Do both, in this order
- Run the requested analysis on the supplied data anyway, exactly as asked,
and report it as the primary result. If the data is a surrogate, the numbers
are what they are; caveat them in a sentence.
- Then add what you found, as a section of its own: what is wrong with the
inputs, how you established it, what it means for the numbers above.
- If you fetched better data, report that as an additional arm beside the
requested one, never instead of it.
The caveated result plus the audit is strictly more than the audit. It is also
more useful: a reader can see both what the specified protocol yields and why
they should discount it.
What never to substitute
Do not turn the study into an analysis of the harness, the evaluation, the
scoring, or the scaffold you are running inside. That is a different paper, it
answers none of the task's questions, and no amount of insight in it counts
toward the study you were asked to do.
The check before you commit
Look at your section list against the task statement. If a requested analysis
has no section, you have replaced the study rather than extended it.
