iterate-model-types

---

name: iterate-model-types description: Use when the user wants to sweep model types or group keys against a fixed history run — keeping versions.osm_data pinned while producing multiple fitted models. Triggers: "try a different model type", "rerun the model for <group_key>", "sweep model variants", "compare random_by_type vs constant", "fit a new group_key on the same observations".

Iterate model types for a pinned run date

Rerun just the modeling step (step 3 of the full pipeline) for a fixed source-data version.

Prerequisites

• osm_observations.parquet already generated by scripts/osm_data/format_tabular.py at the pinned versions.osm_data. Each row is one (POI version, shared_label) pair — POIs mapping to multiple taxonomy categories are exploded into multiple rows.
• You know which model variant you want next.

Steps

1. Pin versions.osm_data in config.yaml. Do not change it — that's the whole point.

2. Bump versions.model_output using the convention:

   • {date}_by_shared_label for the unified random-effects model (one intercept per taxonomy category)
   • {date}_by_{group_key} for ad-hoc groupings on other columns
   • {date}_constant for the single-rate baseline

3. Edit osm_turnover_model in config.yaml:

   • model_type: one of constant, random_by_type (registry at src/openpois/models/osm_models.py)
   • group_key: column to group by. Default shared_label (unified taxonomy). Null for constant. Other observation columns (raw OSM keys like shop, amenity, ...) are still accepted.
   • group_values: restrict to specific values, or null for all
   • min_value_count: drop groups below this count

4. Fit:

   python scripts/models/osm_turnover.py
   

5. Re-rate the snapshot (optional) — if the rated snapshot should use this fit, update osm_data.apply_model.model_stub and rerun the matching rater:

   make rate                                       # random_effects (per-cell) — the default
   # or, for a legacy constant / random_by_type fit:
   python scripts/osm_snapshot/apply_model.py      # per-group lookup
   

   make rate (apply_model_random_effects.py) rates each POI from its own (shared_label, MSA, urban_rural) cell — use it for any random_effects fit. apply_model.py is per-group and only valid for constant/random_by_type fits.

Comparing variants

• fitted_params.csv across directories — diff λ and σ.
• param_draws.csv — uncertainty bounds for each group.
• predictions.csv — head-to-head on specific POIs.

Pitfalls

• Forgetting to change versions.model_output overwrites the previous variant's outputs.
• group_key must exist as a column in the observations CSV. shared_label is populated by format_tabular.py from the conflation taxonomy crosswalk; if you change the crosswalk, rerun format_tabular.py.
• min_value_count filters groups silently — check fitted_params.csv row count vs. expected group count.