MetaInfer

Designs prefix-aware KV cache and radix-tree caching for LLM inference engines. Use when implementing or refactoring block tables, paged KV, radix prefix cache, cache hit/miss, eviction, or when the user mentions nano-vllm BlockManager, nano-sglang/mini-sglang RadixCache, or Agent-generated inference frameworks.

Pre-allocates GPU KV buffers and implements slot/page allocation with optional reference counting. Use when implementing ReqToTokenPool, TokenToKVPool, MHAKVCache, nano-vllm allocate_kv_cache, profiling GPU memory for KV capacity, or Agent-generated inference frameworks.