원클릭으로 Manus에서 모든 스킬 실행

vector-db-storage

스타4

포크0

업데이트2026년 1월 31일 04:37

Storage backends and indexing for vector databases in Elixir. Use when choosing persistence, implementing exact vs approximate k-NN, or integrating pgvector, file, or ETS for vector storage.

설치

Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.

Manus에서 실행

출처

8dazo

8dazo/elix-db

GitHub 저장소 열기 Creator 저장소 보기

다운로드

Manus에서 실행

Vector DB Storage and Indexing

Storage Backends

Backend	Use case	Persistence	Scale
ETS	Prototype, small index, low latency	No	< ~100k–500k vectors in memory
File	Simple persistence, no DB	Yes (binary/term)	Same as ETS, plus disk
PostgreSQL + pgvector	Production, SQL + vectors	Yes	Large; use IVFFlat/HNSW indexes
Qdrant (client)	Dedicated vector engine	Yes (external)	Very large

In-Memory (ETS)

:set table, key = vector id.
Value: {id, vector, metadata}; vector as list or binary.
No persistence: load from file on GenServer init if needed (see File below).
Exact k-NN: iterate all rows, compute distance, sort, take k. O(n) per query.

File Persistence

On shutdown: GenServer terminate/2 or :gen_server.cast(pid, :persist) → write ETS contents to file (e.g. :erlang.term_to_binary(entries) or custom binary format).
On start: read file, recreate ETS and insert all entries.
Keep dimension and metadata format in header or config so reload is consistent.

PostgreSQL + pgvector

Use pgvector and Ecto for persistent, indexed storage.

Schema: table with id, embedding (pgvector type), metadata (jsonb/map).
Index: create HNSW or IVFFlat index on embedding for approximate k-NN.
Queries: use pgvector’s distance operators (<=>, <->) in raw SQL or Ecto fragments; limit k.

# Example: nearest neighbors (cosine distance)
# fragment("embedding <=> ?::vector", ^Nx.to_flat_list(query))

Handles large datasets; persistence and backups via Postgres.

Exact vs Approximate k-NN

Method	Accuracy	Speed	When
Exact (scan all)	Exact	O(n) per query	Small n (e.g. < 100k), correctness critical
IVFFlat (pgvector)	Approximate	Fast	Medium/large, good recall with tuning
HNSW (pgvector)	Approximate	Very fast	Large, high recall

For in-memory ETS, start with exact k-NN; move to pgvector (or Qdrant) when n grows or persistence is required.

Design Checklist

Persistence requirement clear: none, file, or Postgres
If file: define format (term vs binary) and load/save on start/stop
If pgvector: add migration for vector column and chosen index (HNSW/IVFFlat)
Dimension and distance metric match across insert and search

이 저장소의 다른 Skills

같은 저장소

versioned-sample-uses

8dazo/elix-db

When publishing a new elix-db version or making version changes, run versioned sample use cases, collect memory/time/latency, compare to the previous version report, and update reports. Remember this workflow for releases.

2026-01-314

debug-verify-benchmark

8dazo/elix-db

Debug, verify, and compare elix-db to industry after each plan step. Use after implementing any plan step or changing vector/store/API logic; run tests, IEx checks, and document efficiency vs Qdrant/Milvus/pgvector.

2026-01-314

elixir-vector-patterns

8dazo/elix-db

Elixir patterns for vector storage and similarity search. Use when implementing GenServer-based vector store, ETS index, Nx/Scholar distance math, or supervision for a vector DB in Elixir.

2026-01-314

vector-db-elixir

8dazo/elix-db

Design and implement a vector database in Elixir. Use when building embedding storage, similarity search, k-NN retrieval, or when the user mentions vector DB, embeddings, or semantic search in Elixir.

2026-01-314

name	vector-db-storage
description	Storage backends and indexing for vector databases in Elixir. Use when choosing persistence, implementing exact vs approximate k-NN, or integrating pgvector, file, or ETS for vector storage.

Vector DB Storage and Indexing

Storage Backends

Backend	Use case	Persistence	Scale
ETS	Prototype, small index, low latency	No	< ~100k–500k vectors in memory
File	Simple persistence, no DB	Yes (binary/term)	Same as ETS, plus disk
PostgreSQL + pgvector	Production, SQL + vectors	Yes	Large; use IVFFlat/HNSW indexes
Qdrant (client)	Dedicated vector engine	Yes (external)	Very large

In-Memory (ETS)

:set table, key = vector id.
Value: {id, vector, metadata}; vector as list or binary.
No persistence: load from file on GenServer init if needed (see File below).
Exact k-NN: iterate all rows, compute distance, sort, take k. O(n) per query.

File Persistence

On shutdown: GenServer terminate/2 or :gen_server.cast(pid, :persist) → write ETS contents to file (e.g. :erlang.term_to_binary(entries) or custom binary format).
On start: read file, recreate ETS and insert all entries.
Keep dimension and metadata format in header or config so reload is consistent.

PostgreSQL + pgvector

Use pgvector and Ecto for persistent, indexed storage.

Schema: table with id, embedding (pgvector type), metadata (jsonb/map).
Index: create HNSW or IVFFlat index on embedding for approximate k-NN.
Queries: use pgvector’s distance operators (<=>, <->) in raw SQL or Ecto fragments; limit k.

# Example: nearest neighbors (cosine distance)
# fragment("embedding <=> ?::vector", ^Nx.to_flat_list(query))

Handles large datasets; persistence and backups via Postgres.

Exact vs Approximate k-NN

Method	Accuracy	Speed	When
Exact (scan all)	Exact	O(n) per query	Small n (e.g. < 100k), correctness critical
IVFFlat (pgvector)	Approximate	Fast	Medium/large, good recall with tuning
HNSW (pgvector)	Approximate	Very fast	Large, high recall

For in-memory ETS, start with exact k-NN; move to pgvector (or Qdrant) when n grows or persistence is required.

Design Checklist

Persistence requirement clear: none, file, or Postgres
If file: define format (term vs binary) and load/save on start/stop
If pgvector: add migration for vector column and chosen index (HNSW/IVFFlat)
Dimension and distance metric match across insert and search