| name | smr-empirical-illustration |
| description | Use when building the real-data empirical illustration for a Sociological Methods & Research (SMR) paper — a demonstration that the method changes a substantive conclusion, not a decorative example. Designs the illustration; does not derive properties or design the Monte Carlo. |
SMR Empirical Illustration
Use this to make the real-data section earn its place. SMR expects a methods paper to show that the
method matters substantively — that using it instead of the incumbent leads to a different,
better-justified conclusion about the social world. A throwaway "we also applied it to some data"
section is a reviewer flag.
The "it changes the answer" standard
The illustration's job is to demonstrate consequence:
- Run the incumbent and the new method on the same data, and show where they diverge. The payoff
sentence is "the standard approach would have concluded X; our method shows Y, and Y is the
defensible answer because [reason tied to the method's properties]."
- Tie the divergence to the mechanism established in
smr-derivation-and-properties and the regime
identified in smr-simulation-studies: the data should sit in the regime where the incumbent is
known to fail.
- State the substantive stake: who would have made a wrong inference, and about what, if they had
used the old method? The stake makes the method consequential, not just correct.
Choosing the dataset
- Pick data that lives in the failure regime (e.g., few clusters, non-invariance across groups,
informative missingness, network dependence) so the method has something to do.
- Prefer public or depositable data — SMR's availability policy expects the data and code behind
the illustration to be accessible (see
smr-software-and-reproducibility). If data are restricted,
plan the availability statement now.
- A familiar, recognizable dataset lets readers judge the result against intuition; an exotic one
forces them to trust you on both the data and the method.
What to report
| Element | Purpose |
|---|
| Side-by-side incumbent vs. new method | Show the divergence concretely |
| The substantive conclusion under each | Make the stake visible |
| A diagnostic that the data are in the failure regime | Justify why the new method is needed here |
| Uncertainty for both methods | Avoid replacing one overconfident answer with another |
| Link to released code/data | Satisfy reproducibility expectations |
Keep it an illustration, not a substantive paper
The danger runs both ways. Too thin and it is decorative; too thick and the paper becomes a
substantive study that belongs in ASR/AJS (the failure flagged in smr-topic-selection). Calibrate:
the illustration should be deep enough to show the method changes the answer, and no deeper. The unit
of analysis is the method's behavior on real data, not a full substantive argument with its own
literature.
Checklist
Anti-patterns
- Decorative application: the method is run, but it would not change any conclusion.
- Regime mismatch: data where the incumbent is fine, so the new method has nothing to prove.
- Substantive creep: the illustration grows into an ASR/AJS-style paper and loses methods focus.
- One-method reporting: showing only the new method's result, hiding what the incumbent would say.
- Inaccessible data with no plan: an illustration readers can never reproduce.
Output format
[Illustration status] consequential / decorative / not ready
[Dataset + regime] <data : why it sits in the failure regime>
[Divergence] <incumbent conclusion vs. new-method conclusion>
[Substantive stake] <who would have been wrong, about what>
[Reproducibility] data/code accessible? restricted-data plan?
[Next SMR skill] smr-tables-figures
