| name | qdrant-memory-usage-optimization |
| description | Diagnoses and reduces Qdrant memory usage. Use when someone reports 'memory too high', 'RAM keeps growing', 'node crashed', 'out of memory', 'memory leak', or asks 'why is memory usage so high?', 'how to reduce RAM?'. Also use when memory doesn't match calculations, quantization didn't help, or nodes crash during recovery. |
Understanding memory usage
Qdrant operates with 2 types of RAM memory:
-
Resident memory (aka RSSAnon) - memory used for internal data structures like ID tracker, and some components forced to stay in RAM, quantized vectors if always_ram=true, payload indexes, e.t.c.
-
OS page cache - memory used for caching disk reads, which can be released when needed. Original vectors are normally stored in page cahce, so the service won't crash if RAM is full, but performance may degrade.
It is normal of OS page cache to occupy whole available RAM, but if resident memory is above 80% of total, it's a sign of a problem.
Memory usage monitoring
- Qdrant exposes memory usage with
/metrics endpoint see more in Monitoring docs.
How much memory is needed for Qdrant?
Optimal memory usage depends on the use case.
For detailed breakdown of memory usage on a large scale exmaple see Large scale memory usage example.
Payload indexes and HNSW graph also require memory, along with vectors themselves, so it's important to consider them in calculations.
Additionally, qdrant requires some extra memory for optimizations. During optimization, optimized segments are fully loaded into RAM, so it is importat to have some headroom for that.
Large the max_segment_size, more headroom is needed.
When to put HNSW index on disk
Putting frequently used components (such as HNSW index) on disk might cause significant performance degradation.
There are some scenarios, however, when it can be a good option:
- Deployments with low latency disks - local NVMe or similar.
- Multi-tenant deployments, where only a subset of tenants is frequently accessed, so that only a fraction of data & index is loaded in RAM at a time.
- For deployments with inline storage enabled.
How to minimize memory footprint
The main challenge is to put on disk those parts of data, which are rarely accessed.
Here are the main techniques to achieve that:
-
Use quantization to store only compressed vectors in ram Quantization docs
-
Use float16 or int8 datatypes to reduce memory usage of vectors by 2x or 4x respectively, with some tradeoff in precision. Read more about vector datatypes in documentation
-
Leverage Matryoshka Representation Learning (MRL) to store only small vectors in RAM, while keeping large vectors on disk. Examples of how to use MRL with qdrant cloud inferece: MRL docs
-
For multi-tenant deployments with small tenants, vectors might be stored on-disk, as same tenant data is stored together Multitenancy docs
-
For deployments with fast local storage and relatively low requirements for search throughput, it may be possible to store all components of vector store on disk. Read more about the performance implications of on-disk storage in the article
-
For low RAM environments, consider async_scorer config, which enables support of io_uring for parallel disk access, which can significantly improve performance of on-disk storage. Read more about async_scorer in the article (only available on Linux with kernel 5.11+)
-
Consider storing Sparse Vectors and text payload on disk, as they are usually more disk-friently compared to dense vectors.
- Configuring payload indexes to be stored on disk docs
- Configuring sparse vectors to be stored on disk docs
