litellm-rs

LiteLLM-RS Routing Architecture. Covers 7 routing strategies, lock-free design with DashMap, health-aware selection, fallback chains, and load balancing.

LiteLLM-RS A2A Protocol Architecture. Covers Agent-to-Agent communication, JSON-RPC 2.0 messaging, multi-provider orchestration, agent registry, and task state management.

LiteLLM-RS Authentication Architecture. Covers JWT + API Key + RBAC multi-method auth, rate limiting with DashMap, middleware pipeline, and secure credential management.

LiteLLM-RS Caching Architecture. Covers Redis caching, vector database semantic caching, multi-tier cache strategy, TTL management, and cache invalidation patterns.

LiteLLM-RS Configuration Architecture. Covers YAML loading, environment variable override, validation patterns, type-safe config models, and hot reloading.

LiteLLM-RS Error Handling Architecture. Covers two-tier error hierarchy, ProviderError factory methods, HTTP status mapping, retry logic, and error context preservation.

LiteLLM-RS MCP Gateway Architecture. Covers Model Context Protocol, JSON-RPC 2.0 implementation, multi-transport support (HTTP, SSE, WebSocket, Stdio), and permission system.

LiteLLM-RS Observability Architecture. Covers Prometheus metrics, OpenTelemetry tracing, structured logging, health checks, and alerting integration.

LiteLLM-RS Provider Architecture Guide. Covers 66+ provider integration, trait object design, unified error handling, connection pooling, and LLMProvider trait implementation patterns.

LiteLLM-RS Provider 开发与架构指南。用于添加新 provider、迁移错误处理、维护 66+ provider 的一致性。包含架构对比分析和最佳实践。

LiteLLM-RS Streaming Architecture. Covers UnifiedSSEParser, SSETransformer trait, VecDeque buffering, provider-specific transformers, and real-time event handling.