| name | dataset-provenance |
| description | Acquire and retain public building datasets reproducibly, with source hashes, safe extraction, and a strict raw-versus-derived boundary. |
Public dataset provenance
Goal
Acquire public building datasets reproducibly without committing huge archives or losing source provenance.
Procedure
- Record canonical landing page, DOI, release/version date, publisher and license.
- Prefer a publisher API or stable archive endpoint over scraping.
- Download to ignored
data/raw/ using a .part file and atomic rename.
- Reject ZIP members that resolve outside the destination directory.
- SHA-256 hash source archives and important extracted metadata.
- Write a machine-readable acquisition manifest with URL, DOI, bytes, hash and timestamp.
- Keep raw source files immutable. Derived tables belong in
data/processed/ with their own provenance.
- Never commit multi-GB telemetry for convenience.
Failure posture
Unexpected package layout, hash mismatch, ambiguous release or unsafe archive paths fail closed. Never silently swap in another dataset release.
