<div align="center">
<h1>ramjet</h1>
<h3>Warm intake. Balanced burn.</h3>
<p>A compact Rust router that puts each OpenAI-compatible inference request on<br>the healthy GPU replica where it can do the least repeated work.</p>
<p>
<!-- The release and license badges were API-backed and rendered "repo not
found" because this repository is private: shields.io queries the GitHub
API anonymously. The release and LICENSE themselves are fine. If the
repository is ever made public, they can be restored as:
https://img.shields.io/github/v/release/helixml/ramjet
https://img.shields.io/github/license/helixml/ramjet -->
<a href="rust-toolchain.toml"><img alt="Rust 1.95 or newer" src="https://img.shields.io/badge/Rust-1.95%2B-f06a35?style=flat-square"></a>
</p>
</div>
<p align="center">
<img src="docs/assets/deployment.svg" alt="An incoming prompt is scored by ramjet and routed to the GPU replica with the best combination of reusable prefix and available capacity" width="1100">
</p>
ramjet sits between your clients and replicated model servers. It keeps
conversations and shared system prompts near warm cache state, then lets live
load override affinity before one replica becomes a hotspot. Clients keep the
same OpenAI API; engines need no ramjet-specific integration.
## What we wrote about each model
How ramjet and the engines behind it were tuned for each model, written up
on the Helix blog. GLM-5.3 ran on an 8× H200 server; the others on our
8× RTX PRO 6000 server:
| GLM-5.3 (8× H200) | GLM-5.3-Flash | Qwen3.8-Flash-Next | Qwen3.8-27B | DeepSeek V4 |
| --- | --- | --- | --- | --- |
| [48 coding agents from one server](https://helix.ml/blog/glm53-on-8x-h200) (29 Sep) | [Part 1: getting day-zero serving to work](https://helix.ml/blog/glm53-flash-on-rtx-pro-6000-part-1) (27 Aug) | [On eight GPUs: what actually helped](https://helix.ml/blog/qwen38-flash-next-on-rtx-pro-6000) (27 Aug) | [Chasing a 454 tok/s tweet](https://helix.ml/blog/chasing-454-toks-qwen38-rtx-pro-6000) (22 Aug) | [SGLang vs DwarfStar vs vLLM+DSpark](https://helix.ml/blog/running-ds4-on-rtx-pro-6000) (14 Aug) |
| [Ramjet vs NVIDIA Dynamo](https://helix.ml/blog/ramjet-vs-nvidia-dynamo) (30 Sep) | [Running on 2, 4 or 8 GPUs](https://helix.ml/blog/glm53-flash-tp2-rtx-pro-6000) (14 Sep) | [One model, two speeds: smart routing](https://helix.ml/blog/smarter-qwen-routing-with-ramjet) (28 Aug) | [Doubling throughput by reading a log line](https://helix.ml/blog/the-ceiling-was-a-state-cache) (23 Aug) | [V4.1 Flash: encoder, Engram and KV cache](https://helix.ml/blog/deepseek-v41-flash-explained) (10 Sep) |
| | [Ran out of cache snapshots, not cache tokens](https://helix.ml/blog/glm53-flash-hybrid-attention-prefix-cache) (26 Sep) | [Swift 1.5 cut thinking tokens](https://helix.ml/blog/swift-flash-next-four-gpu-evaluation) (27 Sep) | [A better lm_head, tested and shipped](https://helix.ml/blog/qwen38-bf16-lm-head-rollout) (25 Aug) | |
## Why it exists
| Reuse more | Queue less | Fail cleanly |
| --- | --- | --- |
| Bounded prefix fingerprints find the replica most likely to reuse prior work. | Size-weighted reservations spread cold prefills and concurrent decodes. | Active health probes, retryable failover, and immediate disconnect cancellation keep capacity honest. |
The ordinary router is stateless, privacy-bounded, and deliberately useful
without raw KV-cache events. Optional DSpark enforcement persists only opaque
quarantine commitments so an LB restart cannot forget a bad EngineCore. The
production path remains the proxy plus your existing OpenAI-compatible engines.
## Built-in dashboard
System overview to see how your node is doing:
<img width="1441" height="1221" alt="image" src="https://github.com/user-attachments/assets/05deee62-4fcf-4220-ac77-bb318b2ce8ba" />
And specific serving tab:
<img width="1419" height="1214" alt="image" src="https://github.com/user-attachments/assets/6703a7a9-53b4-4d9a-ab07-7389a96fc684" />
You can also just plug it into prometheus, `/metrics` API is available.
## Measured on real hardware
Each metric has its own column, so green always means better: throughput
and cache hits up, latency down. Red marks a measured cost, grey a change
within noise, and a dash a metric that run did not record.
| Result | Throughput | Latency | Cache hits |
| :-- | --: | --: | --: |
| **DeepSeek-V4-Flash** <sub>2× TP4 · 8× RTX PRO 6000</sub> | | | |
| <sub>12 same-app sessions, load-blind hash router → ramjet</sub> | <sub>298 → **469** tok/s<br><img src="docs/assets/delta-up.svg" alt="▲" width="10" height="9"> 57%</sub> | <sub>batch wall time<br>7.5 → **4.5** s<br><img src="docs/assets/delta-down-good.svg" alt="▼" width="10" height="9"> 40%</sub> | <sub>no locality loss<br><img src="docs/assets/delta-flat.svg" alt="–" width="10" height="9"> tie</sub> |
| <sub>2–3 apps × 4 sessions × 2–3 turns, hash router → ramjet</sub> | <sub>—</sub> | <sub>—</sub> | <sub>82.9% → **82.9%**<br><img src="docs/assets/delta-flat.svg" alt="–" width="10" height="9"> tie</sub> |
| <sub>Whole-box deterministic code, c24/max256</sub> | <sub>**1,820–1,844** tok/s</sub> | <sub>TTFT p50 **948–960** ms<br>p95 1,270–1,319 ms</sub> | <sub>—</sub> |
| **Qwen3.8-Flash-Next** <sub>2× TP4 · 8× RTX PRO 6000</sub> | | | |
| <sub>Load-only → prefix routing, returning request beside a long one</sub> | <sub>long request<br>100% → **93.6%**<br><img src="docs/assets/delta-down-bad.svg" alt="▼" width="10" height="9"> 6.4%</sub> | <sub>TTFT p50 1,119 → **908** ms<br><img src="docs/assets/delta-down-good.svg" alt="▼" width="10" height="9"> 19%</sub> | <sub>23.9% → **35.8%**<br><img src="docs/assets/delta-up.svg" alt="▲" width="10" height="9"> 12 pts</sub> |
| <sub>Prefix routing → phase-aware load release, same probe</sub> | <sub>long request<br>100% → **99.1%**<br><img src="docs/assets/delta-down-bad.svg" alt="▼" width="10" height="9"> 0.9%</sub> | <sub>TTFT 2,496 → **287** ms<br><img src="docs/assets/delta-down-good.svg" alt="▼" width="10" height="9"> 88.5%</sub> | <sub>—</sub> |
| <sub>Phase-aware load release under c32 load</sub> | <sub>2,639.9 → **2,711.8** tok/s<br><img src="docs/assets/delta-up.svg" alt="▲" width="10" height="9"> 2.7%</sub> | <sub>TTFT p95<br><img src="docs/assets/delta-up-bad.svg" alt="▲" width="10" height="9"> 7.3%</sub> | <sub>—</sub> |
| <sub>Direct vLLM → same engine through ramjet</sub> | <sub>c1 −0.03%<br>c16 −0.24%<br><img src="docs/assets/delta-flat.svg" alt="–" width="10" height="9"> ≈ 0</sub> | <sub>TTFT slightly lower<br><img src="docs/assets/delta-flat.svg" alt="–" width="10" height="9"> ≈ 0</sub> | <sub>—</sub> |
| **GLM-5.3-Flash** <sub>2× TP4 · 8× H200</sub> | | | |
| <sub>Coding-agent swarm, prefix routing → `marginal` affinity basis</sub> | <sub>194–197 → **243–249**<br>turns/min<br><img src="docs/assets/delta-up.svg" alt="▲" width="10" height="9"> 23–28%</sub> | <sub>TTFT p90 5.3–5.5 →<br>**3.2–3.3** s<br><img src="docs/assets/delta-down-good.svg" alt="▼" width="10" height="9"> 38–42%</sub> | <sub>85.5–86.0% →<br>**92.0–92.4%**<br><img src="docs/assets/delta-up.svg" alt="▲" width="10" height="9"> 6–7 pts</sub> |
| **DeepSeek-V4.1-Flash** <sub>2× TP4 · 8× H200</sub> | | | |
| <sub>Coding-agent swarm, 64 agents, default `absolute` → `marginal` affinity basis</sub> | <sub>222.7 → **263.8**<br>turns/min<br><img src="docs/assets/delta-up.svg" alt="▲" width="10" height="9"> 18%</sub> | <sub>turn e2e p90 36.6 → **28.3** s<br><img src="docs/assets/delta-down-good.svg" alt="▼" width="10" height="9"> 23%<br>TTFT p90 <img src="docs/assets/delta-up-bad.svg" alt="▲" width="10" height="9"> 8%</sub> | <sub>88.5% → **91.6%**<br><img src="docs/assets/delta-up.svg" alt="▲" width="10" height="9"> 3 pts</sub> |
| <sub>Replica answers `/health` but cannot generate, `RJ_UPSTREAM_RANK_PROBE` `on` → `all`, 12 requests</sub> | <sub>6 → **12** of 12 served<br><img src="docs/assets/delta-up.svg" alt="▲" width="10" height="9"> 100%</sub> | <sub>batch wall time<br>182 → **2** s<br><img src="docs/assets/delta-down-good.svg" alt="▼" width="10" height="9"> 99%</sub> | <sub>—</sub> |
| **GLM-5.3** <sub>DP8 attention · 8× H200</sub> | | | |
| <sub>NVIDIA Dynamo 1.5.0 → ramjet, same engine, 32 and 48 agents</sub> | <sub>73.0 → **88.9**<br>77.3 → **99.5**<br>turns/min<br><img src="docs/assets/delta-up.svg" alt="▲" width="10" height="9"> 22%, 29%</sub> | <sub>TTFT p50 2.06 → **1.53** s<br>2.92 → **2.88** s<br><img src="docs/assets/delta-down-good.svg" alt="▼" width="10" height="9"> 26%, 1%<br>p90 <img src="docs/assets/delta-up-bad.svg" alt="▲" width="10" height="9"> 13%, 2.5%</sub> | <sub>85.0% → **92.4%**<br>84.7% → **91.7%**<br><img src="docs/assets/delta-up.svg" alt="▲" width="10" height="9"> 7 pts</sub> |
These are workload results, not theoretical peaks. Reproduce the DeepSeek rows
from [RESULTS.md](RESULTS.md); inspect every accepted and rejected experiment in
[EXPERIMENTS.md](EXPERIMENTS.md).
### Models with a validated stack
All measured on node06 — 8× RTX PRO 6000 Blackwell; the engine topology is
listed per row, and each model name links to its Compose stack. The full-box
column reports the best qualified saturation point recorded for that stack,
not a shared concurrency level.
| Model | Engine · measured shape | Decode @ c1 | Full-box peak |
| :-- | :-- | --: | --: |
| <sub>[**DeepSeek-V4-Flash**](deploy/dspark_0731/docker-compose.yaml)<br>`deepseek-v4-flash`<br>sparse MoE</sub> | <sub>vLLM + DSpark<br>2× TP4 · c24/max256</sub> | <sub>**245.1** tok/s</sub> | <sub>**1,891.2** tok/s</sub> |
| <sub>[**Qwen3.8-27B FP8**](deploy/qwen38_27b/docker-compose.yaml)<br>`qwen3.8-27b`<br>dense</sub> | <sub>vLLM<br>2× TP4 · c256/max256 · MTP off</sub> | <sub>77 → **121** tok/s<br><img src="docs/assets/delta-up.svg" alt="▲" width="10" height="9"> 57% with MTP</sub> | <sub>**7,890.9** tok/s</sub> |
| <sub>[**Qwen3.8-27B NVFP4 + BF16 head**](deploy/qwen38_27b/docker-compose.yaml)<br>`qwen3.8-27b`<br>dense</sub> | <sub>SGLang + DFlash2<br>8× TP1 · 208 slots · bf16 SSM</sub> | <sub>**153.3** tok/s<br>greedy median<br><img src="docs/assets/delta-up.svg" alt="▲" width="10" height="9"> 7.5% vs Inferact</sub> | <sub>not yet requalified<br>Inferact target:<br>7,882.6 tok/s</sub> |
| <sub>[**Qwen3.8-Flash-Next FP8**](deploy/qwen38_flash_next/docker-compose.yaml)<br>`qwen3.8-flash-next`<br>sparse MoE</sub> | <sub>vLLM<br>2× TP4+EP · c64 · MTP3 on both</sub> | <sub>113 → **202** tok/s<br><img src="docs/assets/delta-up.svg" alt="▲" width="10" height="9"> 79% with MTP3</sub> | <sub>**3,340.5** tok/s</sub> |
| <sub>[**GLM-5.3-Flash W4A16, FP8 experts**](deploy/glm53_flash_sm120/docker-compose.yaml)<br>`glm-5.3-flash`<br>sparse MoE</sub> | <sub>SGLang + EAGLE<br>TP2 · c4/max256</sub> | <sub>**164.8** tok/s</sub> | <sub>**388.2** tok/s<br>per 2-GPU replica;<br>box not yet saturated</sub> |
No model — and neither Qwen3.8-27B stack — is simply better. Single-stream
decode is what an interactive user feels; the full-box figure is a capacity
landmark for a saturated agent fleet. These maxima come from separate
model-specific workloads, so they are not a matched head-to-head benchmark.
The vLLM row's saturation result has MTP off because speculation improves
low-concurrency latency but wastes rejected drafts once the batch saturates
the GPU. The SGLang row uses RadixArk's immutable
BF16-`lm_head` checkpoint. Its matched one-engine canary measured 153.3 tok/s
against 142.6 for the former Inferact target (+7.5%), with the same 7/8
objective answers and 20/25 deterministic agent-protocol cases. The smaller
target exposes 26 running slots and 582,246 KV tokens per engine: 208 slots
across the fleet. Full-box saturation has not yet been requalified on these
weights; the former Inferact target reached 7,882.6 tok/s, within 0.1% of the
vLLM reference. On that earlier SGLang stack, bf16 SSM state reduced c128 TTFT
p95 from 3.99s to 0.221s. The same
3-app × 4-session × 2-turn locality run measured **87.3% cached prompt
tokens**, and 12 concurrent same-app requests spread across 7 of 8 engines
at 714 tok/s. Its cost is cold long-context prefill: a 196K-token first
turn pays ~57s of TTFT on one GPU, with prefix-cached follow-ups at 2–4s.
Qwen3.8-Flash-Next shows the same speculation trade-off: on 256-token outputs
MTP3 adds 79% at c1 but only 7.5% at c32. The qualified pair therefore runs
MTP3 on one engine and standard decoding on the other, and ramjet uses the
requested output length to pick between them only once cache and load tie. Its
full-box figure predates that split, with MTP3 on both engines. GLM-5.3-Flash
runs on two GPUs per replica; its prefix cache is bounded by saved
linear-attention states rather than KV tokens. Keeping two states per path
instead of four, plus a 4 GB host tier, took a probe of 12 cyclic 20k-token
sessions from 0/12 to 12/12 cached. The [Kev
stack](deploy/qwen38_glm53_kev/README.md) adds a 0.8B decision model to the
same server, sharing one Qwen GPU behind a second API profile: 77 ms p50 per
short three-question request at c1 and 20.3 requests/s at c4, measured beside
live traffic rather than saturated.
[Model profiles](docs/models.md) covers the sizing, sharding, and
speculative-decoding trade-offs behind these numbers.
### Full GLM-5.3 on one 8× H200 server
[`deploy/glm53_h200`](deploy/glm53_h200/README.md) runs the 753B FP8
checkpoint as one SGLang engine with eight data-parallel attention ranks, and
ramjet lists each rank as its own upstream (`RJ_UPSTREAM_DP_RANKS`). On a
simulated team of continuously working coding agents:
| Result | Throughput | Latency | Cache hits |
| :-- | --: | --: | --: |
| <sub>16 agents: SGLang rank placement → ramjet per-rank routing</sub> | <sub>43.0 → **72.7**<br>turns/min<br><img src="docs/assets/delta-up.svg" alt="▲" width="10" height="9"> 69%</sub> | <sub>TTFT p50 2.6 → **0.76** s<br>p90 5.0 → **1.7** s<br><img src="docs/assets/delta-down-good.svg" alt="▼" width="10" height="9"> 66–71%</sub> | <sub>65.6% → **92.8%**<br><img src="docs/assets/delta-up.svg" alt="▲" width="10" height="9"> 27 pts</sub> |
| <sub>64 agents on ramjet per-rank routing: adding a 32 GB host KV tier per rank</sub> | <sub>45.9 → **95.1**<br>turns/min<br><img src="docs/assets/delta-up.svg" alt="▲" width="10" height="9"> 107%</sub> | <sub>TTFT p90 67 → **46** s<br><img src="docs/assets/delta-down-good.svg" alt="▼" width="10" height="9"> 31%</sub> | <sub>71.8% → **91.8%**<br><img src="docs/assets/delta-up.svg" alt="▲" width="10" height="9"> 20 pts</sub> |
| <sub>Capacity per server: 32–48 agents</sub> | <sub>**98–109**<br>turns/min</sub> | <sub>TTFT p50 **1.3–2.8** s</sub> | <sub>—</sub> |
The [blog post](https://helix.ml/blog/glm53-on-8x-h200) walks through each
step, and [Ramjet vs NVIDIA Dynamo](https://helix.ml/blog/ramjet-vs-nvidia-dynamo)
compares the router against Dynamo 1.5.0's KV router on the same engine; the
raw cells are in [EXPERIMENTS.md](EXPERIMENTS.md) (2026-09-29 and 2026-09-30).
## Start in one minute
For existing engines, the upstream list is normally the only setting you need:
```yaml
services:
ramjet:
image: ghcr.io/helixml/ramjet:v0.8.0@sha256:fe432bbca183d2a457a7713fb150ea5ee36aba7a13f92280ef3ec7195ec23673
restart: unless-stopped
ports:
- "8000:8000" # OpenAI API + /health
- "9090:9090" # Prometheus
environment:
RJ_UPSTREAM: http://model-server-1:8000,http://model-server-2:8000
# RJ_UPSTREAM_TOKEN: ${MODEL_SERVER_API_KEY} # if required
```
```bash
docker compose up -d
curl --fail http://localhost:8000/health
```
The example pins a released image by immutable digest; see
[`CHANGELOG.md`](CHANGELOG.md) for what each version contains.
Safe defaults enable locality/load routing and keep tokenizer, raw KV-event,
exact-placement, and snapshot paths off. See the complete
[configuration table](docs/configuration.md), or start from the
[eight-replica Compose stack](deploy/qwen38_27b/docker-compose.yaml) currently
running in production. The
[two-replica DeepSeek-V4-Flash stack](deploy/dspark_0731/docker-compose.yaml)
is the previous deployment, kept as a reviewed alternative and rollback
record.
> **Backend compatibility:** `model-server-1` and `model-server-2` are example
> Docker DNS names—replace them with your backends. The default router is not
> tied to vLLM: it forwards OpenAI-compatible APIs and health-checks each server
> with `GET /v1/models`. The opt-in `/tokenize`, KV-event, exact-routing, and
> snapshot research paths are currently designed for vLLM/DSpark.
## The routing rule
```text
score(replica) = min(prefix overlap, affinity cap) − α × live load
```
ramjet fingerprints only a bounded prefix, scores every healthy replica,
and reserves load before forwarding. Warm state wins when it is valuable; idle
capacity wins when reuse no longer pays for the queue. Score ties prefer the
deeper raw overlap.
## Production surface
- OpenAI-compatible chat/completions, streaming, reasoning, and tool calls.
- `ok`, `degraded`, and `unhealthy` readiness at `GET /health`.
- Optional SHA-pinned model/template compatibility admission for engines that
expose the atomic identity contract, with fail-closed per-replica recovery;
the node06 guide includes an opt-in, no-extra-hop vLLM middleware candidate.
- Optional DSpark reliability observation and sticky per-replica quarantine
when active K5 acceptance collapses to zero across multiple complete metric
windows; enforcement fsyncs an opaque EngineCore commitment and only a
different compatibility-attested EngineCore can durably rearm it. A
precommitted dirty marker keeps unresolved replicas fenced after an unclean
LB exit or failed state mutation.
- Stable `ramjet_*` Prometheus metrics on port `9090`.
- Opaque `X-Ramjet-Upstream` route correlation without leaking hosts.
- Bounded memory, request sanitization, model metadata rewriting, and upstream
cancellation when the client disappears.
Exact tokenization, fenced KV indexes, authenticated snapshot companions,
exact-placement canaries, and session-affinity shadow telemetry remain opt-in
research surfaces. The session path cannot change placement. These paths fail
closed and are not dependencies of ordinary serving.
> **Naming:** the project was renamed from ramjet to ramjet. Settings now
> use the `RJ_*` prefix and responses carry `X-Ramjet-*` headers; the retired
> `MD_*` prefix is refused at startup rather than silently ignored, so a stale
> overlay fails loudly instead of running a differently tuned proxy. The
> `ramjet_*` metric names are deliberately unchanged so existing Grafana
> history keeps resolving.
## Operate it
| Task | Start here |
| --- | --- |
| Deploy or roll back | [Docker Compose operator guide](deploy/dspark_0731/README.md) |
| Configure the router | [Environment reference](docs/configuration.md) |
| Serve a different model | [Model profiles](docs/models.md) |
| Understand the design | [Architecture and routing model](DESIGN.md) |
| Inspect current work | [Roadmap](ROADMAP.md) |
Codex-compatible repo skills are included for repeatable node operations:
[`$deploy-ramjet`](.agents/skills/deploy-ramjet/SKILL.md),
[`$optimize-ramjet-node`](.agents/skills/optimize-ramjet-node/SKILL.md),
[`$load-test-ramjet-node`](.agents/skills/load-test-ramjet-node/SKILL.md),
and
[`$troubleshoot-ramjet-node`](.agents/skills/troubleshoot-ramjet-node/SKILL.md).
## Develop
```bash
cargo fmt --check
cargo test --locked
cargo clippy --locked --all-targets --all-features -- -D warnings
```
<details>
<summary>Privacy-safe production-shape replay</summary>
For privacy-safe production-shape validation, `bench/agent_trace.py` accepts
only numeric/enumerated trace shapes and synthesizes all request content. A
bounded `/tokenize` preflight adjusts for the active chat-template overhead;
authoritative response usage still enforces the token-density gate. See the
[sovereign trace replay contract](bench/agent_cases/README.md#sovereign-trace-shape-replay).
</details>
See [AGENTS.md](AGENTS.md) for the GPU-free inner loop, full release gate, and
node06 benchmark contract.
## Resources
Everything we have written about serving on the
[Helix blog](https://helix.ml/blog), grouped by topic and newest first. The
[per-model table](#what-we-wrote-about-each-model) above picks from the same
posts.
**Routing with ramjet**
- [Ramjet vs NVIDIA Dynamo: Which Router for Coding-Agent Traffic?](https://helix.ml/blog/ramjet-vs-nvidia-dynamo) (30 Sep)
- [Serving Full GLM-5.3 to 48 Coding Agents From One 8×H200 Server](https://helix.ml/blog/glm53-on-8x-h200) (29 Sep)
- [Self-Hosting Kev on an RTX PRO 6000 With Ramjet](https://helix.ml/blog/one-ramjet-two-apis-kev-systemone) (22 Sep)
- [One Qwen Model, Two Speeds: What Smart Routing Bought Us](https://helix.ml/blog/smarter-qwen-routing-with-ramjet) (28 Aug)
**GLM-5.3-Flash**
- [GLM-5.3-Flash Ran Out of Cache Snapshots, Not Cache Tokens](https://helix.ml/blog/glm53-flash-hybrid-attention-prefix-cache) (26 Sep)
- [Running GLM-5.3-Flash on 2, 4 or 8 RTX PRO 6000 GPUs](https://helix.ml/blog/glm53-flash-tp2-rtx-pro-6000) (14 Sep)
- [GLM-5.3-Flash on RTX PRO 6000, Part 1: Getting Day-Zero Serving to Work](https://helix.ml/blog/glm53-flash-on-rtx-pro-6000-part-1) (27 Aug)
**Qwen3.8**
- [Swift 1.5 Flash-Next Cut Qwen3.8's Thinking Tokens in Our Pilot](https://helix.ml/blog/swift-flash-next-four-gpu-evaluation) (27 Sep)
- [Qwen3.8-Flash-Next on Eight GPUs: What Actually Helped](https://helix.ml/blog/qwen38-flash-next-on-rtx-pro-6000) (27 Aug)
- [A Better lm_head for Qwen3.8-27B: How We Tested and Shipped It](https://helix.ml/blog/qwen38-bf16-lm-head-rollout) (25 Aug)
- [We Doubled Our Inference Throughput by Reading a Log Line](https://helix.ml/blog/the-ceiling-was-a-state-cache) (23 Aug)
- [Chasing a 454 tok/s tweet: a day of tuning Qwen3.8-27B on the RTX PRO 6000](https://helix.ml/blog/chasing-454-toks-qwen38-rtx-pro-6000) (22 Aug)
**DeepSeek**
- [DeepSeek V4.1 Flash: Why Its Encoder, Engram and KV Cache Matter](https://helix.ml/blog/deepseek-v41-flash-explained) (10 Sep)
- [SGLang vs DwarfStar vs vLLM+DSpark: Running DeepSeek 4 on the RTX Pro 6000](https://helix.ml/blog/running-ds4-on-rtx-pro-6000) (14 Aug)
**Hardware**
- [What's Actually in the Sovereign Server](https://helix.ml/blog/whats-inside-the-sovereign-server) (14 Aug)
## License
[Apache-2.0](LICENSE).