phoenix-sqlean

# phoenix-sqlean Vendored fork of [nalgeon/sqlean.py](https://github.com/nalgeon/sqlean.py). ## Bumping a bundled version Pins live in `packages/phoenix-sqlean/Makefile`. Each has a partner that must move with it: | Pin | Moves with | |---|---| | `SQLITE_VERSION` | `SQLITE_RELEASE_YEAR` (the URL embeds both, so a mismatch 404s) and `SQLITE_SHA256` | | `SQLEAN_COMMIT` | `SQLEAN_VERSION` in the `Makefile` *and* in `setup.py` — the commit is what gets fetched, the strings only describe it | | `SQLEAN_VERSION` | `setup.py`'s copy, compiled in as what `sqlean_version()` returns. Suffix `+<short sha>` iff `SQLEAN_COMMIT` sits ahead of the named tag | | `XXHASH_TAG` | `XXHASH_COMMIT` + `XXHASH_SHA256`, and the tag sqlean's own `Makefile` fetches | `check-sqlean-pin`, a prerequisite of `download-sqlean`, enforces the `SQLEAN_*` rows for one `ls-remote`: suffix prefixes the commit, base is a real tag, suffix present iff ahead, `setup.py` agrees. It cannot tell you the base names the *wrong* release — the single-commit fetch carries no ancestry. `SQLITE_*` and `XXHASH_*` get no equivalent cross-check — only their hashes, at fetch. sqlean is fetched over git by commit: GitHub's source archives are generated on demand and not byte-stable, so they cannot be checksum-pinned, while git hashes every object it receives. Hence `download-sqlean` needs `git` on PATH and no checksum. The other two arrive over plain HTTP, where curl verifies nothing — `XXHASH_COMMIT` fixes which bytes are correct and `XXHASH_SHA256` proves they are those. `src/test_windirent.h` needs neither; it is committed to this repo. ### With the script `scripts/check-upstream.sh` resolves and rewrites every pin, hashes included: ```bash cd packages/phoenix-sqlean ./scripts/check-upstream.sh sqlite # detection only, no downloads ./scripts/check-upstream.sh sqlite --apply # rewrite pins, print a Markdown summary ./scripts/check-upstream.sh sqlean [--apply] # also moves XXHASH_* if sqlean moved its own ``` ### By hand Resolve a tag to a commit with `git ls-remote <repo> 'refs/tags/<version>*'`, taking the peeled `^{}` line when the tag is annotated. sqlean also carries a non-version tag (`incubator`), so filter by shape before picking a latest. For `SQLITE_SHA256`, confirm the download against the SHA3-256 in the `PRODUCT` line on <https://sqlite.org/download.html> *before* recording the SHA-256 (`shasum` cannot do SHA3). Skipping that makes the pin trust-on-first-use. ### Verify ```bash cd packages/phoenix-sqlean make prepare-src download-sqlite download-sqlean # always the full chain: download-sqlean # APPENDS init.c, so alone it double-appends python setup.py build_ext -i python -m tests ``` ## Staying current Dependabot cannot see these pins — no custom-regex manager — and there is no Renovate here. `phoenix-sqlean-upstream.yml` fills the gap: weekly, one draft PR per component, running `check-upstream.sh --apply`, no model involved. **It builds and tests nothing.** CI covers that on the PR; verifying inside the bump job would re-run a subset of it, on the one platform where the likeliest failure — `patch(1)` on the MSVC patch — cannot reproduce. A failure of the script itself is different: upstream changed shape, so it exits before writing and the job goes red with no PR, because there is no computed bump to review. `XXHASH_*` is deliberately unwatched. sqlean drives it, and `download-sqlean` already fails when sqlean moves its own pin. ## Gotchas - **`src/test_windirent.h` is pinned at 3.50.4 — do not re-sync it.** SQLite removed it in 3.51.0 and no newer tag has it. Vendored deliberately, not downloaded. - **sqlean >= 0.28 needs `crypto/xxhash.impl.h`, which sqlean does not ship.** Any new third-party source needs a `THIRD_PARTY_LICENSES` section *and* its SPDX identifier in `license=`. - **`patch(1)` failing on `src/sqlean-time-msvc.patch` is the intended tripwire**, not a breakage. Regenerate it against the new sqlean sources. - **"sqlean fetches xxHash X, but XXHASH_TAG is Y" is the other one.** Move all three `XXHASH_*` pins, and the version named in `THIRD_PARTY_LICENSES`. - **The `sqlean_version()` test is a tautology on its own** — it compares the compiled macro to `setup.py`'s own constant, proving only that the binary is not stale. `check-sqlean-pin` is what ties that constant back to `SQLEAN_COMMIT`. - **Changing `SQLEAN_VERSION` alone does not trigger a recompile.** It only moves a `-D` flag, and `build_ext` keys on source mtime. `touch src/sqlean.c` after editing, or the version test fails against a merely stale build. - **Keep the package version plain SemVer.** release-please's version regex is unanchored, so `1.2.3.1` and `1.2.3.post1` truncate to `1.2.3` and collide with an existing tag, silently. For the same reason `VERSION` must stay above `SQLEAN_VERSION` in `setup.py`. The package version is intentionally independent of the bundled SQLite version. ## Wheels and publishing - Settings are in `cibuildwheel.toml`; runners and steps are in `phoenix-sqlean-build.yml`, a `workflow_call` workflow that `build-sqlean` in `publish.yaml` and CI's `wheels` job both invoke with no overrides. A PR runs the exact build a release runs — keep it that way rather than narrowing CI's copy. - The wheel matrix is every CPython `python_requires` admits × the toml's `build` platforms × the build workflow's runners, minus the toml's `skip`, plus an sdist of the C sources. Read it there rather than restating it. - **Every wheel is tested by cibuildwheel, which needs one runner per Linux/Windows arch.** Those archs are `native`; cibuildwheel skips testing a cross-compiled `win_arm64`, and `auto` on Windows AMD64 adds 32-bit x86. macOS builds both arches on one arm64 runner and tests x86_64 under Rosetta. Free-threaded builds are excluded by the toml's `skip`; cibuildwheel would otherwise build them by default. - **`scripts/check-wheelhouse.py` gates the merged set** in the build workflow's `check` job, so both CI and publish run it: exactly one wheel per CPython that `python_requires` admits on each of its `PLATFORMS`, plus one sdist. Adding or dropping a platform means editing `PLATFORMS` alongside the runners and `archs`. - **The supported Python range is `python_requires` in `setup.py`, and nothing else.** cibuildwheel reads it to pick CPythons, and the wheelhouse check expects exactly its versions. It must stay a string literal keyword in the `setuptools.setup(...)` call under `if __name__ == "__main__"` — cibuildwheel parses that statically, and silently builds every CPython it knows if it cannot find one (the wheelhouse check would then fail). - **Admitting a new CPython can fail two ways, both loudly.** cibuildwheel may still gate it as a prerelease, so its wheels are missing and the wheelhouse check fails — bump cibuildwheel. Or it may have removed private C API the driver uses, which fails that version's wheel builds. - **Source preparation is the build workflow's first job**, on Ubuntu because Windows runners lack make/curl/unzip. Its `ref` input selects what to build: CI leaves it empty (the PR's commit), publish passes the release tag. Callers pass nothing else. - **`test-sources` resolves against cibuildwheel's working directory, not `package-dir`** — hence the build legs unpack the sources at the workspace root. - Publishing is gated on the tag `arize-phoenix-sqlean-v<manifest version>`. Until release-please creates it, `sqlean-release` skips and nothing builds. - **sqlean publishes until PyPI holds every distribution, not just the version.** The generic "version on PyPI" gate in `list-python-packages` skips sqlean; `sqlean-release` runs on every release and runs `check-wheelhouse.py --missing-only` on PyPI's file list for that version. A release interrupted mid-upload is finished by re-running publish, which rebuilds from the tag and uploads with `skip-existing`. - **`check-wheelhouse.py` always comes from the commit running the workflow; `python_requires` comes from the sources being built.** `PLATFORMS` must match the runners, which also come from that commit, and older release tags predate the script. - PyPI uses trusted publishing with **no** GitHub environment; the publisher's Environment field must stay blank to match.