embeddinggemma-qdrant

name	embeddinggemma-qdrant
description	EmbeddingGemma ONNX — CPU-only local embedding (onnxruntime+tokenizers, NO optimum/transformers). Model setup, embed function, Qdrant connection. Use embeddinggemma-knowledge or embeddinggemma-novel for specific use cases.
version	2.0.0
author	Kai
license	MIT
metadata	{"hermes":{"tags":["EmbeddingGemma","ONNX","Qdrant","Embedding","Vector-Database","Base-Skill"]}}

EmbeddingGemma ONNX — Base Skill

Base skill for local embedding pipeline. Use the specific skill for your use case:

• embeddinggemma-knowledge — hermes-knowledge collection (newsletter, riset, digest, blog)
• embeddinggemma-novel — novel chapters collection

Architecture

Text → EmbeddingGemma ONNX (768d) → embed_utils.py (rate control) → Qdrant (cosine similarity)

Property	Value
Model	onnx-community/embeddinggemma-300m-ONNX (308M params, Q4)
Engine	ONNX Runtime CPU (188 MB)
Dimensions	768 (Matryoshka — truncatable to 512/256/128)
Model path	${EMBEDDINGGEMMA_MODEL_DIR}/onnx/model_q4.onnx
Rate control	${EMBED_UTILS_PATH}
Vector DB	Qdrant http://${QDRANT_URL} (HTTP only, check_compatibility=False)
Perf	~2s cold load, ~18ms/single, ~0.03s/batch

Quick Start

import sys; sys.path.insert(0, "${SCRIPTS_DIR}")
import embeddinggemma

vec = embeddinggemma.embed_one("Hello world", dimensions=768)
vectors = embeddinggemma.embed_texts(["Hello", "World"], dimensions=512)

Packages: onnxruntime + tokenizers only. NO optimum/transformers/torch.

Rate Limits (MANDATORY)

Server: 4 cores, Qdrant on same machine. ALWAYS use embed_utils.py:

from embed_utils import embed_with_control, embed_one_with_control, acquire_embed_lock, release_embed_lock

# Single
vec = embed_one_with_control("text", dimensions=768)

# Batch with lock
fd = acquire_embed_lock()
try:
    vectors = embed_with_control(texts, dimensions=768)
finally:
    release_embed_lock(fd)

Rule	Limit
Max batch	10 (embed_utils.BATCH_SIZE)
Concurrent jobs	NEVER (file lock required)
Load check	Every batch via check_load()
Target load	~2.0 (server auto-restarts >2.5)

Collections

Collection	Dimensions	Purpose
hermes-knowledge	768	Skills, entities, agent memory
outline-docs	768	Outline docs
hermes-sessions	512	Session recall (real-time)
midjourney	512	Image references

⛔ Match dimension to collection — mismatch → 400 crash.

Critical Gotchas

• Server crash = embedding too fast. Always use embed_utils.py, never inline load checks.
• No concurrent jobs — combine check_load() + acquire_embed_lock().
• Qdrant URL must use ${QDRANT_URL} — localhost:6333 doesn't work from inside Docker.
• Point IDs must be UUID strings (use str(uuid.uuid4())). Hex truncation like uuid.uuid4().hex[:16] → 400 error.
• Qdrant client https=False required when connecting to local Qdrant from within Docker. Without it, SSL handshake fails: record layer failure. Also suppress warning: UserWarning: Api key is used with an insecure connection.
• embeddinggemma is a local module at ${EMBEDDINGGEMMA_MODULE}, not a pip package.
• Mean pooling: NEVER use keepdims=True — produces wrong shape (batch,1,768).

References

• Setup & Configuration — Model storage, packages, Qdrant config, collections, stress test
• Code Examples — Loading, embed functions, Qdrant client/REST, embed_utils, migration
• Pitfalls — All gotchas and critical warnings