Use when integrating Hugging Face Transformers.js (Xenova/transformers) for in-browser or in-Node inference, debugging quantized model loading, building bi-encoder / cross-encoder / classification pipelines, configuring model cache directories, or bypassing high-level pipelines to read raw logits. Triggers: "model failed to load", cross-encoder scores all 1.0 (softmax-over-1 trap), env.allowLocalModels confusion, cacheDir overrides, ONNX runtime mismatch, ESM vs CJS pipeline imports, browser vs node feature gaps. NOT for full transformers Python (different SDK), TensorFlow.js, ONNX Runtime Web directly without Transformers.js, or model training.
Use when integrating Hugging Face Transformers.js (Xenova/transformers) for in-browser or in-Node inference, debugging quantized model loading, building bi-encoder / cross-encoder / classification pipelines, configuring model cache directories, or bypassing high-level pipelines to read raw logits. Triggers: "model failed to load", cross-encoder scores all 1.0 (softmax-over-1 trap), env.allowLocalModels confusion, cacheDir overrides, ONNX runtime mismatch, ESM vs CJS pipeline imports, browser vs node feature gaps. NOT for full transformers Python (different SDK), TensorFlow.js, ONNX Runtime Web directly without Transformers.js, or model training.
metadata
{"category":"AI & Machine Learning","tags":["transformers-js","onnx","embeddings","cross-encoder","inference","huggingface"],"provenance":{"kind":"first-party","owners":["port-daddy"]},"pairs-with":[{"skill":"llm-router","reason":"When a retrieval or classification cascade must decide between local ONNX inference and a hosted model call, llm-router owns that routing decision."},{"skill":"cost-optimizer","reason":"Replacing hosted embedding/classification API calls with a local quantized model is a cost lever cost-optimizer tracks across a running budget."}],"io-contract":{"kind":"deliverable","consumes":["[Truncated]","[Truncated]"],"produces":["[Truncated]","[Truncated]"]}}
Transformers.js ONNX Pipelines
Transformers.js runs Hugging Face models on ONNX Runtime in-browser or in-Node. The high-level pipeline() is convenient until it isn't — most non-trivial models need direct tokenizer + model use because the pipeline's task wrapping makes assumptions that don't hold (e.g., the cross-encoder softmax-over-1 trap).
When to use
Local embeddings without an API (cosine search, RAG, dedup).
Cross-encoder reranking for retrieval cascades.
Browser-side inference (PII redaction, classification) without a server roundtrip.
Replacing OpenAI embedding calls with a quantized local model to control cost.
Integrating with a Cloudflare Worker via Workers AI (different surface, but ONNX know-how transfers).
functioncosine(a: Float32Array, b: Float32Array) {
let dot = 0;
for (let i = 0; i < a.length; i++) dot += a[i] * b[i];
return dot;
}
Cross-encoder reranking — bypass the pipeline
The MS MARCO MiniLM rerankers publish a single regression head (num_labels=1). Transformers.js's text-classification pipeline applies softmax to the logits. Softmax over a single value collapses to 1.0 — every score becomes 1.0.
Be honest about the download cost — show a progress bar.
Anti-patterns
Cross-encoder via text-classification pipeline
Symptom: Every candidate scores 1.0; ranking is degenerate.
Diagnosis: Pipeline softmaxes a single-logit head; softmax-over-1 = 1.
Fix: Use AutoTokenizer + AutoModelForSequenceClassification directly. Read outputs.logits.data as raw scores.
env.allowLocalModels = true in CI
Symptom: Model downloads succeed locally, fail in CI: "model not found".
Diagnosis: With allowLocalModels = true the SDK looks for files on disk first; in CI, no disk model = fail.
Fix:env.allowLocalModels = false (default). Combine with a CI cache for the download.
Importing the pipeline at module load
Symptom: Server startup blocks for 5s on first deploy.
Diagnosis: Top-level await pipeline(...) runs at import time.
Fix: Lazy-load inside a function. Cache the Promise so concurrent first calls share one load.
Forgetting normalize: true for cosine
Symptom: Scores look right but order is subtly off.
Diagnosis: Vectors aren't unit-normalized; raw dot products bias toward longer strings.
Fix: Always { pooling: 'mean', normalize: true }. Pre-normalize stored vectors.
Float32Array reuse across calls
Symptom: Stored vectors mysteriously change; cache looks corrupted.
Diagnosis: Some pipeline implementations return a view into a reused tensor buffer; the next call overwrites.
Fix: Copy on capture: new Float32Array(out.data).
Reranker on too-long documents
Symptom:RangeError: Tokenized inputs exceeded 512 tokens.
Diagnosis: Cross-encoders truncate at 512 tokens jointly across (query, candidate). Long candidates lose context.
Fix: Truncate candidate text to ~400 tokens. Description + name only; skip body.
Quality gates
First-load Promise cached so concurrent calls don't double-download.
env.cacheDir set to a CI-cacheable path.
Cosine similarity uses normalized vectors.
Cross-encoder scores read from outputs.logits.data directly, not via pipeline.
Embeddings persisted as Float32Array for compactness.
Model versions pinned in code; never "latest".
Quantized variants (quantized: true) unless full precision is critical.
Smoke test runs the model on 3 known queries on every CI build.
Deterministic Audit
Before wiring (or reviewing) a Transformers.js integration, write the decisions as a JSON
plan matching schemas/transformers-js-onnx-pipelines-plan.schema.json and run the
deterministic auditor:
auditTransformersJsOnnxPipelines(plan) (in scripts/transformers_js_onnx_pipelines_audit.mjs)
turns this skill's anti-patterns and Quality Gates into machine-checkable rules over
structured fields: a cross-encoder routed through the text-classification pipeline
instead of raw logits (the softmax-over-1 trap), un-normalized bi-encoder vectors fed to
cosine, allowLocalModels: true in CI, a missing CI model cache, an uncached first-load
Promise, persisted vectors that were never copied out of the reused tensor buffer,
unpinned model versions, and reranker inputs past the 512-token joint limit. It returns
{ pass, score, findings, recommendations }. examples/sample-input.json is a correctly
bypassed cross-encoder rerank plan (pass: true). Changes are tracked in CHANGELOG.md.
NOT for
Python transformers — different SDK; don't transfer assumptions.
TensorFlow.js — different runtime, different model formats.
ONNX Runtime Web directly without Transformers.js — lower-level; use only if you need custom ops.
Workers AI / Vertex AI — managed inference; use the platform SDK.
Model training — Transformers.js is inference-only.
