vllm-setup

name	vllm-setup
description	Install vLLM, configure environment, resolve dependency issues
triggers	["When user wants to install vLLM","When user encounters installation errors","When user needs to configure GPU drivers or CUDA","When user wants to install vLLM on specific hardware (ROCm, NPU, XPU, CPU)"]

vllm-setup

Overview

vLLM installation varies by hardware backend and use case. This skill covers installation methods, environment configuration, and troubleshooting common setup issues.

Prerequisites

• Python 3.9 - 3.12
• CUDA 11.8+ or ROCm 5.0+ (for GPU inference)
• 8GB+ available disk space
• For CUDA: NVIDIA driver 450.80.02+

Main Workflow

Step 1: Choose Installation Method

Option A: pip (Recommended for most users)

# CUDA 12.1
pip install vllm

# CUDA 11.8
pip install vllm --extra-index-url https://download.pytorch.org/whl/cu118

Option B: uv (Fastest, for development)

pip install uv
uv pip install vllm

Option C: Source (For development or latest features)

git clone https://github.com/vllm-project/vllm.git
cd vllm
pip install -e .

Option D: Docker

# CUDA 12.1
docker run --runtime nvidia --gpus all \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -p 8000:8000 \
  vllm/vllm-openai:latest

# With specific model
docker run --runtime nvidia --gpus all \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -p 8000:8000 \
  vllm/vllm-openai:latest \
  --model meta-llama/Llama-2-7b-hf

Step 2: Verify Installation

# Check vLLM version
python -c "import vllm; print(vllm.__version__)"

# Verify GPU access
python -c "import torch; print(f'CUDA available: {torch.cuda.is_available()}')"

# Run quick test
python -c "from vllm import LLM; print('vLLM imported successfully')"

Step 3: Configure Hugging Face (for gated models)

# Login to Hugging Face
pip install huggingface_hub
huggingface-cli login

# Or set token via environment variable
export HF_TOKEN=your_token_here

Hardware-Specific Installation

AMD ROCm

# Install ROCm-compatible vLLM
pip install vllm --extra-index-url https://download.pytorch.org/whl/rocm6.1

# Verify ROCm
rocm-smi

Intel XPU

# Install Intel extension
pip install vllm-xpu

# Set environment
export VLLM_TARGET_DEVICE=xpu

Intel CPU

# CPU-only installation
pip install vllm --extra-index-url https://download.pytorch.org/whl/cpu

# Or from source with CPU target
export VLLM_TARGET_DEVICE=cpu
pip install -e .

Neural Processing Unit (NPU)

# Ascend NPU
export VLLM_TARGET_DEVICE=npu
pip install vllm

Common Patterns

Pattern 1: Virtual Environment Setup

# Create isolated environment
python -m venv vllm-env
source vllm-env/bin/activate  # Linux/Mac
# or: vllm-env\Scripts\activate  # Windows

# Install vLLM
pip install vllm

Pattern 2: Requirements File Setup

# requirements.txt
vllm>=0.5.0
transformers>=4.40.0
accelerate>=0.25.0

pip install -r requirements.txt

Pattern 3: Development Setup

# Clone and setup for development
git clone https://github.com/vllm-project/vllm.git
cd vllm

# Install in editable mode with dev dependencies
pip install -e ".[dev]"

# Install pre-commit hooks
pre-commit install

Troubleshooting

Problem: CUDA Out of Memory during installation

Solution:

Install without building CUDA extensions (CPU-only first):

export VLLM_SKIP_CUDA_BUILD=1
pip install vllm --no-build-isolation

Problem: ImportError: cannot import name '_C'

Solution:

The CUDA extensions failed to build. Reinstall:

pip uninstall vllm -y
rm -rf ~/.cache/pip/wheels/vllm*
pip install vllm

Problem: Version conflict with PyTorch

Solution:

# Check current PyTorch version
python -c "import torch; print(torch.__version__)"

# Install matching vLLM
# For PyTorch 2.3.0 + CUDA 12.1
pip install vllm==0.5.0

# Or reinstall PyTorch to match vLLM
pip install torch --upgrade --index-url https://download.pytorch.org/whl/cu121

Problem: Permission denied errors

Solution:

# Use --user flag
pip install --user vllm

# Or use virtual environment (recommended)
python -m venv vllm-env
source vllm-env/bin/activate
pip install vllm

Problem: Slow model download

Solution:

# Use mirror endpoint (China)
export HF_ENDPOINT=https://hf-mirror.com

# Or configure cache location
export HF_HOME=/path/to/large/disk

References

• GPU Compatibility - GPU and CUDA version compatibility matrix
• Hardware Backends - Detailed configuration for each hardware backend