helixml/ramjet

Última actividad de origen registrada: 27 de agosto de 2026 a las 15:29

Catálogo de SkillsMP actualizado: 2 de noviembre de 2026 a las 18:02

skills recopiladas: 4

Estrellas en GitHub: 8
Forks en GitHub: 3

Instalar con un asistente de IA

Copia este prompt en el asistente de IA que estés usando.

Follow https://skillsmp.com/skill-install/prompt.md to install Agent Skills from https://github.com/helixml/ramjet.

README del repositorio

Archivo de origen del README: README.md

Leer el README original

Mostrar README completo

<div align="center"> <h1>ramjet</h1> <h3>Warm intake. Balanced burn.</h3> A compact Rust router that puts each OpenAI-compatible inference request on the healthy GPU replica where it can do the least repeated work.  <a href="rust-toolchain.toml"><img alt="Rust 1.95 or newer" src="https://img.shields.io/badge/Rust-1.95%2B-f06a35?style=flat-square"></a> </div> <img src="docs/assets/deployment.svg" alt="An incoming prompt is scored by ramjet and routed to the GPU replica with the best combination of reusable prefix and available capacity" width="1100"> ramjet sits between your clients and replicated model servers. It keeps conversations and shared system prompts near warm cache state, then lets live load override affinity before one replica becomes a hotspot. Clients keep the same OpenAI API; engines need no ramjet-specific integration. ## What we wrote about each model How ramjet and the engines behind it were tuned for each model, written up on the Helix blog. GLM-5.3 ran on an 8× H200 server; the others on our 8× RTX PRO 6000 server: | GLM-5.3 (8× H200) | GLM-5.3-Flash | Qwen3.8-Flash-Next | Qwen3.8-27B | DeepSeek V4 | | --- | --- | --- | --- | --- | | [48 coding agents from one server](https://helix.ml/blog/glm53-on-8x-h200) (29 Sep) | [Part 1: getting day-zero serving to work](https://helix.ml/blog/glm53-flash-on-rtx-pro-6000-part-1) (27 Aug) | [On eight GPUs: what actually helped](https://helix.ml/blog/qwen38-flash-next-on-rtx-pro-6000) (27 Aug) | [Chasing a 454 tok/s tweet](https://helix.ml/blog/chasing-454-toks-qwen38-rtx-pro-6000) (22 Aug) | [SGLang vs DwarfStar vs vLLM+DSpark](https://helix.ml/blog/running-ds4-on-rtx-pro-6000) (14 Aug) | | [Ramjet vs NVIDIA Dynamo](https://helix.ml/blog/ramjet-vs-nvidia-dynamo) (30 Sep) | [Running on 2, 4 or 8 GPUs](https://helix.ml/blog/glm53-flash-tp2-rtx-pro-6000) (14 Sep) | [One model, two speeds: smart routing](https://helix.ml/blog/smarter-qwen-routing-with-ramjet) (28 Aug) | [Doubling throughput by reading a log line](https://helix.ml/blog/the-ceiling-was-a-state-cache) (23 Aug) | [V4.1 Flash: encoder, Engram and KV cache](https://helix.ml/blog/deepseek-v41-flash-explained) (10 Sep) | | | [Ran out of cache snapshots, not cache tokens](https://helix.ml/blog/glm53-flash-hybrid-attention-prefix-cache) (26 Sep) | [Swift 1.5 cut thinking tokens](https://helix.ml/blog/swift-flash-next-four-gpu-evaluation) (27 Sep) | [A better lm_head, tested and shipped](https://helix.ml/blog/qwen38-bf16-lm-head-rollout) (25 Aug) | | ## Why it exists | Reuse more | Queue less | Fail cleanly | | --- | --- | --- | | Bounded prefix fingerprints find the replica most likely to reuse prior work. | Size-weighted reservations spread cold prefills and concurrent decodes. | Active health probes, retryable failover, and immediate disconnect cancellation keep capacity honest. | The ordinary router is stateless, privacy-bounded, and deliberately useful without raw KV-cache events. Optional DSpark enforcement persists only opaque quarantine commitments so an LB restart cannot forget a bad EngineCore. The production path remains the proxy plus your existing OpenAI-compatible engines. ## Built-in dashboard System overview to see how your node is doing: <img width="1441" height="1221" alt="image" src="https://github.com/user-attachments/assets/05deee62-4fcf-4220-ac77-bb318b2ce8ba" /> And specific serving tab: <img width="1419" height="1214" alt="image" src="https://github.com/user-attachments/assets/6703a7a9-53b4-4d9a-ab07-7389a96fc684" /> You can also just plug it into prometheus, `/metrics` API is available. ## Measured on real hardware Each metric has its own column, so green always means better: throughput and cache hits up, latency down. Red marks a measured cost, grey a change within noise, and a dash a metric that run did not record. | Result | Throughput | Latency | Cache hits | | :-- | --: | --: | --: | | **DeepSeek-V4-Flash**  2× TP4 · 8× RTX PRO 6000 | | | | | 12 same-app sessions, load-blind hash router → ramjet | 298 → **469** tok/s <img src="docs/assets/delta-up.svg" alt="▲" width="10" height="9"> 57% | batch wall time 7.5 → **4.5** s <img src="docs/assets/delta-down-good.svg" alt="▼" width="10" height="9"> 40% | no locality loss <img src="docs/assets/delta-flat.svg" alt="–" width="10" height="9"> tie | | 2–3 apps × 4 sessions × 2–3 turns, hash router → ramjet | — | — | 82.9% → **82.9%** <img src="docs/assets/delta-flat.svg" alt="–" width="10" height="9"> tie | | Whole-box deterministic code, c24/max256 | **1,820–1,844** tok/s | TTFT p50 **948–960** ms p95 1,270–1,319 ms | — | | **Qwen3.8-Flash-Next**  2× TP4 · 8× RTX PRO 6000 | | | | | Load-only → prefix routing, returning request beside a long one | long request 100% → **93.6%** <img src="docs/assets/delta-down-bad.svg" alt="▼" width="10" height="9"> 6.4% | TTFT p50 1,119 → **908** ms <img src="docs/assets/delta-down-good.svg" alt="▼" width="10" height="9"> 19% | 23.9% → **35.8%** <img src="docs/assets/delta-up.svg" alt="▲" width="10" height="9"> 12 pts | | Prefix routing → phase-aware load release, same probe | long request 100% → **99.1%** <img src="docs/assets/delta-down-bad.svg" alt="▼" width="10" height="9"> 0.9% | TTFT 2,496 → **287** ms <img src="docs/assets/delta-down-good.svg" alt="▼" width="10" height="9"> 88.5% | — | | Phase-aware load release under c32 load | 2,639.9 → **2,711.8** tok/s <img src="docs/assets/delta-up.svg" alt="▲" width="10" height="9"> 2.7% | TTFT p95 <img src="docs/assets/delta-up-bad.svg" alt="▲" width="10" height="9"> 7.3% | — | | Direct vLLM → same engine through ramjet | c1 −0.03% c16 −0.24% <img src="docs/assets/delta-flat.svg" alt="–" width="10" height="9"> ≈ 0 | TTFT slightly lower <img src="docs/assets/delta-flat.svg" alt="–" width="10" height="9"> ≈ 0 | — | | **GLM-5.3-Flash**  2× TP4 · 8× H200 | | | | | Coding-agent swarm, prefix routing → `marginal` affinity basis | 194–197 → **243–249** turns/min <img src="docs/assets/delta-up.svg" alt="▲" width="10" height="9"> 23–28% | TTFT p90 5.3–5.5 → **3.2–3.3** s <img src="docs/assets/delta-down-good.svg" alt="▼" width="10" height="9"> 38–42% | 85.5–86.0% → **92.0–92.4%** <img src="docs/assets/delta-up.svg" alt="▲" width="10" height="9"> 6–7 pts | | **DeepSeek-V4.1-Flash**  2× TP4 · 8× H200 | | | | | Coding-agent swarm, 64 agents, default `absolute` → `marginal` affinity basis | 222.7 → **263.8** turns/min <img src="docs/assets/delta-up.svg" alt="▲" width="10" height="9"> 18% | turn e2e p90 36.6 → **28.3** s <img src="docs/assets/delta-down-good.svg" alt="▼" width="10" height="9"> 23% TTFT p90 <img src="docs/assets/delta-up-bad.svg" alt="▲" width="10" height="9"> 8% | 88.5% → **91.6%** <img src="docs/assets/delta-up.svg" alt="▲" width="10" height="9"> 3 pts | | Replica answers `/health` but cannot generate, `RJ_UPSTREAM_RANK_PROBE` `on` → `all`, 12 requests | 6 → **12** of 12 served <img src="docs/assets/delta-up.svg" alt="▲" width="10" height="9"> 100% | batch wall time 182 → **2** s <img src="docs/assets/delta-down-good.svg" alt="▼" width="10" height="9"> 99% | — | | **GLM-5.3**  DP8 attention · 8× H200 | | | | | NVIDIA Dynamo 1.5.0 → ramjet, same engine, 32 and 48 agents | 73.0 → **88.9** 77.3 → **99.5** turns/min <img src="docs/assets/delta-up.svg" alt="▲" width="10" height="9"> 22%, 29% | TTFT p50 2.06 → **1.53** s 2.92 → **2.88** s <img src="docs/assets/delta-down-good.svg" alt="▼" width="10" height="9"> 26%, 1% p90 <img src="docs/assets/delta-up-bad.svg" alt="▲" width="10" height="9"> 13%, 2.5% | 85.0% → **92.4%** 84.7% → **91.7%** <img src="docs/assets/delta-up.svg" alt="▲" width="10" height="9"> 7 pts | These are workload results, not theoretical peaks. Reproduce the DeepSeek rows from [RESULTS.md](RESULTS.md); inspect every accepted and rejected experiment in [EXPERIMENTS.md](EXPERIMENTS.md). ### Models with a validated stack All measured on node06 — 8× RTX PRO 6000 Blackwell; the engine topology is listed per row, and each model name links to its Compose stack. The full-box column reports the best qualified saturation point recorded for that stack, not a shared concurrency level. | Model | Engine · measured shape | Decode @ c1 | Full-box peak | | :-- | :-- | --: | --: | | [**DeepSeek-V4-Flash**](deploy/dspark_0731/docker-compose.yaml) `deepseek-v4-flash` sparse MoE | vLLM + DSpark 2× TP4 · c24/max256 | **245.1** tok/s | **1,891.2** tok/s | | [**Qwen3.8-27B FP8**](deploy/qwen38_27b/docker-compose.yaml) `qwen3.8-27b` dense | vLLM 2× TP4 · c256/max256 · MTP off | 77 → **121** tok/s <img src="docs/assets/delta-up.svg" alt="▲" width="10" height="9"> 57% with MTP | **7,890.9** tok/s | | [**Qwen3.8-27B NVFP4 + BF16 head**](deploy/qwen38_27b/docker-compose.yaml) `qwen3.8-27b` dense | SGLang + DFlash2 8× TP1 · 208 slots · bf16 SSM | **153.3** tok/s greedy median <img src="docs/assets/delta-up.svg" alt="▲" width="10" height="9"> 7.5% vs Inferact | not yet requalified Inferact target: 7,882.6 tok/s | | [**Qwen3.8-Flash-Next FP8**](deploy/qwen38_flash_next/docker-compose.yaml) `qwen3.8-flash-next` sparse MoE | vLLM 2× TP4+EP · c64 · MTP3 on both | 113 → **202** tok/s <img src="docs/assets/delta-up.svg" alt="▲" width="10" height="9"> 79% with MTP3 | **3,340.5** tok/s | | [**GLM-5.3-Flash W4A16, FP8 experts**](deploy/glm53_flash_sm120/docker-compose.yaml) `glm-5.3-flash` sparse MoE | SGLang + EAGLE TP2 · c4/max256 | **164.8** tok/s | **388.2** tok/s per 2-GPU replica; box not yet saturated | No model — and neither Qwen3.8-27B stack — is simply better. Single-stream decode is what an interactive user feels; the full-box figure is a capacity landmark for a saturated agent fleet. These maxima come from separate model-specific workloads, so they are not a matched head-to-head benchmark. The vLLM row's saturation result has MTP off because speculation improves low-concurrency latency but wastes rejected drafts once the batch saturates the GPU. The SGLang row uses RadixArk's immutable BF16-`lm_head` checkpoint. Its matched one-engine canary measured 153.3 tok/s against 142.6 for the former Inferact target (+7.5%), with the same 7/8 objective answers and 20/25 deterministic agent-protocol cases. The smaller target exposes 26 running slots and 582,246 KV tokens per engine: 208 slots across the fleet. Full-box saturation has not yet been requalified on these weights; the former Inferact target reached 7,882.6 tok/s, within 0.1% of the vLLM reference. On that earlier SGLang stack, bf16 SSM state reduced c128 TTFT p95 from 3.99s to 0.221s. The same 3-app × 4-session × 2-turn locality run measured **87.3% cached prompt tokens**, and 12 concurrent same-app requests spread across 7 of 8 engines at 714 tok/s. Its cost is cold long-context prefill: a 196K-token first turn pays ~57s of TTFT on one GPU, with prefix-cached follow-ups at 2–4s. Qwen3.8-Flash-Next shows the same speculation trade-off: on 256-token outputs MTP3 adds 79% at c1 but only 7.5% at c32. The qualified pair therefore runs MTP3 on one engine and standard decoding on the other, and ramjet uses the requested output length to pick between them only once cache and load tie. Its full-box figure predates that split, with MTP3 on both engines. GLM-5.3-Flash runs on two GPUs per replica; its prefix cache is bounded by saved linear-attention states rather than KV tokens. Keeping two states per path instead of four, plus a 4 GB host tier, took a probe of 12 cyclic 20k-token sessions from 0/12 to 12/12 cached. The [Kev stack](deploy/qwen38_glm53_kev/README.md) adds a 0.8B decision model to the same server, sharing one Qwen GPU behind a second API profile: 77 ms p50 per short three-question request at c1 and 20.3 requests/s at c4, measured beside live traffic rather than saturated. [Model profiles](docs/models.md) covers the sizing, sharding, and speculative-decoding trade-offs behind these numbers. ### Full GLM-5.3 on one 8× H200 server [`deploy/glm53_h200`](deploy/glm53_h200/README.md) runs the 753B FP8 checkpoint as one SGLang engine with eight data-parallel attention ranks, and ramjet lists each rank as its own upstream (`RJ_UPSTREAM_DP_RANKS`). On a simulated team of continuously working coding agents: | Result | Throughput | Latency | Cache hits | | :-- | --: | --: | --: | | 16 agents: SGLang rank placement → ramjet per-rank routing | 43.0 → **72.7** turns/min <img src="docs/assets/delta-up.svg" alt="▲" width="10" height="9"> 69% | TTFT p50 2.6 → **0.76** s p90 5.0 → **1.7** s <img src="docs/assets/delta-down-good.svg" alt="▼" width="10" height="9"> 66–71% | 65.6% → **92.8%** <img src="docs/assets/delta-up.svg" alt="▲" width="10" height="9"> 27 pts | | 64 agents on ramjet per-rank routing: adding a 32 GB host KV tier per rank | 45.9 → **95.1** turns/min <img src="docs/assets/delta-up.svg" alt="▲" width="10" height="9"> 107% | TTFT p90 67 → **46** s <img src="docs/assets/delta-down-good.svg" alt="▼" width="10" height="9"> 31% | 71.8% → **91.8%** <img src="docs/assets/delta-up.svg" alt="▲" width="10" height="9"> 20 pts | | Capacity per server: 32–48 agents | **98–109** turns/min | TTFT p50 **1.3–2.8** s | — | The [blog post](https://helix.ml/blog/glm53-on-8x-h200) walks through each step, and [Ramjet vs NVIDIA Dynamo](https://helix.ml/blog/ramjet-vs-nvidia-dynamo) compares the router against Dynamo 1.5.0's KV router on the same engine; the raw cells are in [EXPERIMENTS.md](EXPERIMENTS.md) (2026-09-29 and 2026-09-30). ## Start in one minute For existing engines, the upstream list is normally the only setting you need: ```yaml services: ramjet: image: ghcr.io/helixml/ramjet:v0.10.0@sha256:4b3ced81f657e87f4f7597ef49728a08a584ee2c1aa0034a863880db7dd77f31 restart: unless-stopped ports: - "8000:8000" # OpenAI API + /health - "9090:9090" # Prometheus environment: RJ_UPSTREAM: http://model-server-1:8000,http://model-server-2:8000 # RJ_UPSTREAM_TOKEN: ${MODEL_SERVER_API_KEY} # if required ``` ```bash docker compose up -d curl --fail http://localhost:8000/health ``` The example pins a released image by immutable digest; see [`CHANGELOG.md`](CHANGELOG.md) for what each version contains. Safe defaults enable locality/load routing and keep tokenizer, raw KV-event, exact-placement, and snapshot paths off. See the complete [configuration table](docs/configuration.md), or start from the [eight-replica Compose stack](deploy/qwen38_27b/docker-compose.yaml) currently running in production. The [two-replica DeepSeek-V4-Flash stack](deploy/dspark_0731/docker-compose.yaml) is the previous deployment, kept as a reviewed alternative and rollback record. > **Backend compatibility:** `model-server-1` and `model-server-2` are example > Docker DNS names—replace them with your backends. The default router is not > tied to vLLM: it forwards OpenAI-compatible APIs and health-checks each server > with `GET /v1/models`. The opt-in `/tokenize`, KV-event, exact-routing, and > snapshot research paths are currently designed for vLLM/DSpark. ## The routing rule ```text score(replica) = min(prefix overlap, affinity cap) − α × live load ``` ramjet fingerprints only a bounded prefix, scores every healthy replica, and reserves load before forwarding. Warm state wins when it is valuable; idle capacity wins when reuse no longer pays for the queue. Score ties prefer the deeper raw overlap. ## Production surface - OpenAI-compatible chat/completions, streaming, reasoning, and tool calls. - `ok`, `degraded`, and `unhealthy` readiness at `GET /health`. - Optional SHA-pinned model/template compatibility admission for engines that expose the atomic identity contract, with fail-closed per-replica recovery; the node06 guide includes an opt-in, no-extra-hop vLLM middleware candidate. - Optional DSpark reliability observation and sticky per-replica quarantine when active K5 acceptance collapses to zero across multiple complete metric windows; enforcement fsyncs an opaque EngineCore commitment and only a different compatibility-attested EngineCore can durably rearm it. A precommitted dirty marker keeps unresolved replicas fenced after an unclean LB exit or failed state mutation. - Stable `ramjet_*` Prometheus metrics on port `9090`. - Opaque `X-Ramjet-Upstream` route correlation without leaking hosts. - Bounded memory, request sanitization, model metadata rewriting, and upstream cancellation when the client disappears. Exact tokenization, fenced KV indexes, authenticated snapshot companions, exact-placement canaries, and session-affinity shadow telemetry remain opt-in research surfaces. The session path cannot change placement. These paths fail closed and are not dependencies of ordinary serving. > **Naming:** the project was renamed from ramjet to ramjet. Settings now > use the `RJ_*` prefix and responses carry `X-Ramjet-*` headers; the retired > `MD_*` prefix is refused at startup rather than silently ignored, so a stale > overlay fails loudly instead of running a differently tuned proxy. The > `ramjet_*` metric names are deliberately unchanged so existing Grafana > history keeps resolving. ## Operate it | Task | Start here | | --- | --- | | Deploy or roll back | [Docker Compose operator guide](deploy/dspark_0731/README.md) | | Configure the router | [Environment reference](docs/configuration.md) | | Serve a different model | [Model profiles](docs/models.md) | | Understand the design | [Architecture and routing model](DESIGN.md) | | Inspect current work | [Roadmap](ROADMAP.md) | Codex-compatible repo skills are included for repeatable node operations: [`$deploy-ramjet`](.agents/skills/deploy-ramjet/SKILL.md), [`$optimize-ramjet-node`](.agents/skills/optimize-ramjet-node/SKILL.md), [`$load-test-ramjet-node`](.agents/skills/load-test-ramjet-node/SKILL.md), and [`$troubleshoot-ramjet-node`](.agents/skills/troubleshoot-ramjet-node/SKILL.md). ## Develop ```bash cargo fmt --check cargo test --locked cargo clippy --locked --all-targets --all-features -- -D warnings ``` <details> <summary>Privacy-safe production-shape replay</summary> For privacy-safe production-shape validation, `bench/agent_trace.py` accepts only numeric/enumerated trace shapes and synthesizes all request content. A bounded `/tokenize` preflight adjusts for the active chat-template overhead; authoritative response usage still enforces the token-density gate. See the [sovereign trace replay contract](bench/agent_cases/README.md#sovereign-trace-shape-replay). </details> See [AGENTS.md](AGENTS.md) for the GPU-free inner loop, full release gate, and node06 benchmark contract. ## Resources Everything we have written about serving on the [Helix blog](https://helix.ml/blog), grouped by topic and newest first. The [per-model table](#what-we-wrote-about-each-model) above picks from the same posts. **Routing with ramjet** - [Ramjet vs NVIDIA Dynamo: Which Router for Coding-Agent Traffic?](https://helix.ml/blog/ramjet-vs-nvidia-dynamo) (30 Sep) - [Serving Full GLM-5.3 to 48 Coding Agents From One 8×H200 Server](https://helix.ml/blog/glm53-on-8x-h200) (29 Sep) - [Self-Hosting Kev on an RTX PRO 6000 With Ramjet](https://helix.ml/blog/one-ramjet-two-apis-kev-systemone) (22 Sep) - [One Qwen Model, Two Speeds: What Smart Routing Bought Us](https://helix.ml/blog/smarter-qwen-routing-with-ramjet) (28 Aug) **GLM-5.3-Flash** - [GLM-5.3-Flash Ran Out of Cache Snapshots, Not Cache Tokens](https://helix.ml/blog/glm53-flash-hybrid-attention-prefix-cache) (26 Sep) - [Running GLM-5.3-Flash on 2, 4 or 8 RTX PRO 6000 GPUs](https://helix.ml/blog/glm53-flash-tp2-rtx-pro-6000) (14 Sep) - [GLM-5.3-Flash on RTX PRO 6000, Part 1: Getting Day-Zero Serving to Work](https://helix.ml/blog/glm53-flash-on-rtx-pro-6000-part-1) (27 Aug) **Qwen3.8** - [Swift 1.5 Flash-Next Cut Qwen3.8's Thinking Tokens in Our Pilot](https://helix.ml/blog/swift-flash-next-four-gpu-evaluation) (27 Sep) - [Qwen3.8-Flash-Next on Eight GPUs: What Actually Helped](https://helix.ml/blog/qwen38-flash-next-on-rtx-pro-6000) (27 Aug) - [A Better lm_head for Qwen3.8-27B: How We Tested and Shipped It](https://helix.ml/blog/qwen38-bf16-lm-head-rollout) (25 Aug) - [We Doubled Our Inference Throughput by Reading a Log Line](https://helix.ml/blog/the-ceiling-was-a-state-cache) (23 Aug) - [Chasing a 454 tok/s tweet: a day of tuning Qwen3.8-27B on the RTX PRO 6000](https://helix.ml/blog/chasing-454-toks-qwen38-rtx-pro-6000) (22 Aug) **DeepSeek** - [DeepSeek V4.1 Flash: Why Its Encoder, Engram and KV Cache Matter](https://helix.ml/blog/deepseek-v41-flash-explained) (10 Sep) - [SGLang vs DwarfStar vs vLLM+DSpark: Running DeepSeek 4 on the RTX Pro 6000](https://helix.ml/blog/running-ds4-on-rtx-pro-6000) (14 Aug) **Hardware** - [What's Actually in the Sovereign Server](https://helix.ml/blog/whats-inside-the-sovereign-server) (14 Aug) ## License [Apache-2.0](LICENSE).

Cargando el README del repositorio…

Skills en este repositorio

Administradores de redes y sistemas informáticos Analistas de garantía de calidad de software y probadores Científicos de datos

3 categorías ocupacionales · 100% clasificado

skill

ocupación

descripción

actualizado

load-test-ramjet-node

Analistas de garantía de calidad de software y probadores

Execute guarded, reproducible load tests against a ramjet GPU inference deployment, especially the node06 8x RTX PRO 6000 stack. Use when asked to benchmark TPS, TTFT, TPOT, concurrency, long context, cache locality, agent/tool correctness, full-box capacity,…

Idioma del texto original: inglés

2026-08-27

deploy-ramjet

Administradores de redes y sistemas informáticos

Deploy or upgrade ramjet and OpenAI-compatible GPU engines on a Linux node with Docker Compose. Use for first-time node setup, creating a Compose stack, validating GPU/container prerequisites, promoting an immutable image, performing an LB-only rollout, or…

Idioma del texto original: inglés

2026-08-25

optimize-ramjet-node

Científicos de datos

Measure and tune a ramjet GPU inference node using controlled, reproducible experiments. Use for GPU/NUMA placement, vLLM or DSpark scheduler settings, routing weights, cache locality, throughput, latency, concurrency, context-size, or image A/B optimization.

Idioma del texto original: inglés

2026-08-25

troubleshoot-ramjet-node

Administradores de redes y sistemas informáticos

Diagnose a ramjet GPU inference node by checking NVIDIA GPUs, Docker Compose services, OpenAI endpoints, router health, Prometheus metrics, logs, networking, and basic synthetic requests. Use for outages, degraded health, low throughput, cache misses, stuck…

Idioma del texto original: inglés

2026-08-25

Mostrando 4 de 4 skills recopiladas.