| name | versioned-sample-uses |
| description | When publishing a new elix-db version or making version changes, run versioned sample use cases, collect memory/time/latency, compare to the previous version report, and update reports. Remember this workflow for releases. |
Versioned Sample Uses and Release Benchmark
Apply when publishing a new version, cutting a release, or making version-related changes to elix-db.
When to Use
- Before or after publishing a new elix-db version (e.g. v0.2.0, v0.3.0)
- When the user says "publish", "release", "version change", or "new version"
- When adding or changing versioned sample use cases
What to Remember
Sample uses are organized by version: sample_uses/v0.1.0/, sample_uses/v0.2.0/, etc. Inside each version folder are the same use cases: 01_simple_search, 02_semantic_faq, 03_similar_items, 04_persistence. Each use case pins elix_db to that version in its mix.exs.
On a new version release:
- Ensure
sample_uses has a version folder for the new version (e.g. v0.2.0) with the same four use cases and elix_db pinned to that version.
- Run the benchmark runner for the new version (e.g. from repo root: run the script that executes each use case in
sample_uses/<version>/, collects metrics).
- Compare to the previous version: load the previous version’s report from
sample_uses/reports/, generate a comparison report (how the new version is better or worse on memory, time, latency, throughput).
- Update or add report files under
sample_uses/reports/ (e.g. <version>.json, <version>_vs_<prev>.md).
- Optionally update CHANGELOG or README with a one-line pointer to the comparison report.
Metrics that matter for the vector DB: memory (e.g. :erlang.memory()), wall time per use case, latency (mean/p50/p99 for search and upsert), throughput (inserts/s, searches/s). The comparison report should answer: how is this version compared to the previous one?
Workflow Summary
- Add or update version folder under
sample_uses/<vX.Y.Z>/ with use cases pointing to that elix_db version.
- Run benchmark for the new version; write
sample_uses/reports/<version>.json (and optional .md).
- Load previous version report; produce comparison report (e.g.
reports/<version>_vs_<prev>.md).
- Remember: this workflow should be run whenever we publish or make version changes so we track memory, time, latency, and other important factors across versions.
Related
- debug-verify-benchmark – for day-to-day changes to vector/store/API (tests, IEx, industry comparison).
- versioned-sample-uses – for release/version workflow (sample use cases, version comparison, reports).
