apptainer

# Apptainer - HPC Container Management ## Overview The `apptainer` command manages Apptainer (formerly Singularity) containers for HPC-compatible workloads. It provides SIF image management with automatic GPU detection. **Key Concept:** Apptainer is the HPC standard. Unlike Docker/Podman, containers run as the user (no root). SIF files are single-file images. ## Quick Reference | Action | Command | Description | |--------|---------|-------------| | Build | `ujust apptainer build DEF` | Build SIF from definition file | | Cache | `ujust apptainer cache [clean\|status]` | Manage Apptainer cache | | Exec | `ujust apptainer exec IMAGE CMD` | Execute specific command in container | | Inspect | `ujust apptainer inspect IMAGE` | Show SIF file metadata | | Pull | `ujust apptainer pull IMAGE` | Download container image to SIF file | | Run | `ujust apptainer run IMAGE` | Run container with default command | | Shell | `ujust apptainer shell [-- CMD]` | Open interactive shell in container | ## Parameters | Parameter | Long Flag | Short | Default | Description | |-----------|-----------|-------|---------|-------------| | action | (positional) | - | required | Action: pull, run, shell, exec, build, inspect, gpu, cache | | image | `--image` | `-i` | `""` | SIF file path, image name, or DEF file | | tag | `--tag` | `-t` | `""` | Image tag, output file, or cache subaction | | cmd | (variadic) | - | `""` | Command to execute (use `--` separator) | ## Pull Images ### bazzite-ai Pod Images ```bash # Pull nvidia-python (long form) ujust apptainer pull --image=nvidia-python # Pull with tag (long form) ujust apptainer pull --image=nvidia-python --tag=testing # Pull nvidia-python (short form) ujust apptainer pull -i nvidia-python # Pull with tag (short form) ujust apptainer pull -i nvidia-python -t testing # Pull jupyter ujust apptainer pull --image=jupyter --tag=stable ``` ### External Images ```bash # Docker Hub ujust apptainer pull --image=docker://ubuntu:22.04 # NVIDIA NGC ujust apptainer pull --image=docker://nvcr.io/nvidia/pytorch:latest # Sylabs Cloud ujust apptainer pull --image=library://sylabsed/examples/lolcow ``` ### Pull Output Images are saved as SIF files: ``` ~/.local/share/apptainer/bazzite-ai-pod-nvidia-python.sif ``` ## Run Containers ### Run with Default Command ```bash # Run nvidia-python (long form) ujust apptainer run --image=nvidia-python # Run nvidia-python (short form) ujust apptainer run -i nvidia-python # Run specific SIF file ujust apptainer run --image=./my-container.sif ``` ### Run with Command ```bash # Run Python in container (use -- separator for commands) ujust apptainer run --image=nvidia-python -- python # Run script ujust apptainer run --image=nvidia-python -- python script.py # Short form ujust apptainer run -i nvidia-python -- python train.py ``` ### GPU Auto-Detection GPU flags are auto-detected: - NVIDIA: Adds `--nv` - AMD: Adds `--rocm` ```bash # GPU is automatically enabled ujust apptainer run --image=nvidia-python -- python -c "import torch; print(torch.cuda.is_available())" ``` ## Interactive Shell ```bash # Shell into container (long form) ujust apptainer shell --image=nvidia-python # Shell into container (short form) ujust apptainer shell -i nvidia-python # Now inside container python --version nvidia-smi exit ``` ## Execute Commands ```bash # Execute single command (use -- separator) ujust apptainer exec --image=nvidia-python -- pip list # Execute Python one-liner ujust apptainer exec -i nvidia-python -- python -c 'print(1+1)' ``` ## Build from Definition ### Definition File Example ```def Bootstrap: docker From: ubuntu:22.04 %post apt-get update apt-get install -y python3 python3-pip %runscript python3 "$@" ``` ### Build ```bash # Build SIF from definition (image=DEF, tag=OUTPUT) ujust apptainer build --image=mydef.def --tag=myimage.sif # Build to default location ujust apptainer build --image=mydef.def # Short form ujust apptainer build -i mydef.def -t myimage.sif ``` ## GPU Support ### Test GPU ```bash # Detect and test GPU ujust apptainer gpu ``` ### GPU Flags | GPU | Flag | Auto-Detection | |-----|------|----------------| | NVIDIA | `--nv` | Yes | | AMD | `--rocm` | Yes | | Intel | (none yet) | No | ### Manual GPU Override ```bash # Direct apptainer command with GPU apptainer run --nv nvidia-python.sif nvidia-smi ``` ## Cache Management ### List Cache ```bash # Long form ujust apptainer cache --tag=list # Or ujust apptainer cache list ``` ### Clean Cache ```bash # Long form ujust apptainer cache --tag=clean # Or ujust apptainer cache clean ``` Cache is stored in `~/.apptainer/cache/`. ## Common Workflows ### HPC Development ```bash # Pull HPC-ready image ujust apptainer pull --image=nvidia-python # Test GPU ujust apptainer gpu # Development shell ujust apptainer shell --image=nvidia-python # Run production workload ujust apptainer run --image=nvidia-python -- python train.py ``` ### Use NGC Images ```bash # Pull NVIDIA PyTorch ujust apptainer pull --image=docker://nvcr.io/nvidia/pytorch:23.10-py3 # Run training ujust apptainer run --image=pytorch_23.10-py3.sif -- python train.py ``` ### Build Custom Image ```bash # Create definition file cat > myenv.def << 'EOF' Bootstrap: docker From: python:3.11 %post pip install numpy pandas scikit-learn %runscript python "$@" EOF # Build ujust apptainer build --image=myenv.def --tag=myenv.sif # Test ujust apptainer run --image=myenv.sif -- python -c "import numpy; print(numpy.__version__)" ``` ## Apptainer vs Docker/Podman | Feature | Apptainer | Docker/Podman | |---------|-----------|---------------| | Root required | No | Sometimes | | Single file | Yes (SIF) | No (layers) | | HPC compatible | Yes | Limited | | GPU support | --nv, --rocm | nvidia-docker | | Security model | User namespace | Container namespace | **Use Apptainer when:** - Running on HPC clusters - Need single-file portability - Can't run as root - Need reproducibility ## Troubleshooting ### Pull Failed **Check:** ```bash # Test network curl -I https://ghcr.io # Check registry auth apptainer remote list ``` **Fix:** ```bash # Login to registry apptainer remote login docker://ghcr.io ``` ### GPU Not Available **Check:** ```bash ujust apptainer gpu nvidia-smi # or rocm-smi ``` **Fix:** ```bash # Ensure drivers installed # For NVIDIA: nvidia-smi # For AMD: rocm-smi ``` ### SIF File Corrupted **Fix:** ```bash # Remove and re-pull rm ~/.local/share/apptainer/*.sif ujust apptainer pull --image=nvidia-python ``` ### Cache Too Large **Check:** ```bash du -sh ~/.apptainer/cache/ ``` **Fix:** ```bash ujust apptainer cache --tag=clean ``` ## Cross-References - **Related Skills:** `pod` (build OCI images), `jupyter` (uses containers) - **GPU Setup:** `ujust config gpu setup` - **Apptainer Docs:** <https://apptainer.org/docs/> ## When to Use This Skill Use when the user asks about: - "apptainer", "singularity", "HPC container" - "SIF file", "pull image", "build container" - "apptainer GPU", "run with GPU" - "HPC workload", "cluster container"