Cinder provides persistent block storage volumes that attach to Nova compute instances. Unlike ephemeral storage (which disappears when an instance is deleted), Cinder volumes persist independently of the instance lifecycle. They can be detached from one instance and reattached to another, snapshotted for point-in-time copies, and backed up for disaster recovery.
Architecture
Cinder uses a backend driver architecture that abstracts the underlying storage technology. The storage backend handles the actual block device operations while Cinder provides a uniform API.
LVM/iSCSI backend: The default for single-node deployments. Cinder creates logical volumes in an LVM volume group and exports them over iSCSI to the compute host. Simple, well-understood, and requires no external storage infrastructure.
Ceph (RBD) backend: The standard for multi-node production deployments. Cinder creates RBD images in a Ceph pool. Provides replication, snapshots, and thin provisioning natively.
NFS backend: Mounts NFS shares and creates volume files on them. Useful for environments with existing NFS infrastructure.
Service components:
cinder-api: Receives REST API requests and routes them to the scheduler.
cinder-scheduler: Selects the appropriate backend and volume node for each operation using configurable filters and weighers.
cinder-volume: Manages the actual volume lifecycle on the backend. One instance per backend.
cinder-backup: Handles volume backup and restore operations to a backup target (Swift, NFS, Ceph).
Deploy
Kolla-Ansible Configuration
Key settings in globals.yml:
# Enable Cinderenable_cinder:"yes"# Enable backup service (optional but recommended)enable_cinder_backup:"yes"# LVM backend configurationcinder_volume_group:"cinder-volumes"# Backup target (Swift is the default when Swift is enabled)# cinder_backup_driver: "swift"# cinder_backup_driver: "nfs"
LVM Volume Group Setup
Before deploying Cinder with the LVM backend, create the volume group:
# Identify available disk or partition (e.g., /dev/sdb)
lsblk
# Create a physical volume
pvcreate /dev/sdb
# Create the volume group (name must match cinder_volume_group)
vgcreate cinder-volumes /dev/sdb
# Verify
vgs
# Should show cinder-volumes with available space
If using a loop device for testing (not recommended for production):
# List Cinder containers
docker ps --format '{{.Names}}' | grep cinder
# Expected containers:# cinder_api, cinder_scheduler, cinder_volume, cinder_backup (if enabled)# Check volume service status
openstack volume service list
# All services should show "up" and "enabled"
Configure
Volume Types and Extra Specs
Volume types define storage classes with specific capabilities. Extra specs control backend behavior.
# Create a volume type for the LVM backend
openstack volume type create lvm-standard
openstack volume typeset lvm-standard --property volume_backend_name=lvm-1
# Create a high-performance type with IOPS limits
openstack volume type create lvm-performance
openstack volume typeset lvm-performance \
--property volume_backend_name=lvm-1 \
--property provisioning:type=thick
# Set default volume type
openstack volume typeset lvm-standard --property is_default=true
QoS Policies
# Create a QoS spec
openstack volume qos create standard-qos \
--consumer front-end \
--property read_iops_sec=1000 \
--property write_iops_sec=500 \
--property total_bytes_sec=104857600
# Associate QoS with a volume type
openstack volume qos associate standard-qos lvm-standard
Volume Encryption
Cinder supports volume encryption using LUKS (dm-crypt). Requires Nova to have the os-brick encryption connector configured.
# Create an encryption type for a volume type
openstack volume type create encrypted-volumes
openstack volume typeset encrypted-volumes --encryption-provider luks \
--encryption-cipher aes-xts-plain64 \
--encryption-key-size 256 \
--encryption-control-location front-end
Note: Volume encryption requires Barbican (key management) or a static key in cinder.conf. For single-node lab deployments, a static key is simpler but less secure.
Backend Configuration
The LVM backend is configured through Kolla-Ansible's cinder-volume.conf override:
Control how much virtual capacity Cinder can allocate beyond physical capacity:
# In cinder.conf or volume backend sectionmax_over_subscription_ratio = 1.0# No oversubscription (safe default)# max_over_subscription_ratio = 2.0 # Allow 2x oversubscription for thin provisioning
Operate
Volume CRUD
# Create a volume
openstack volume create --size 10 my-volume
# Create from an image (bootable volume)
openstack volume create --size 20 --image cirros --bootable boot-volume
# List volumes
openstack volume list
# Show volume details
openstack volume show my-volume
# Delete a volume (must be detached and not in use)
openstack volume delete my-volume
Attach and Detach
# Attach a volume to an instance
openstack server add volume my-instance my-volume
# Check attachment
openstack volume show my-volume -c attachments
# Detach
openstack server remove volume my-instance my-volume
Snapshot Management
# Create a snapshot (volume can be in-use if force is specified)
openstack volume snapshot create --volume my-volume my-snapshot
# Create a volume from a snapshot
openstack volume create --snapshot my-snapshot --size 10 restored-volume
# List and delete snapshots
openstack volume snapshot list
openstack volume snapshot delete my-snapshot
Volume Backup and Restore
# Create a backup (requires cinder-backup service)
openstack volume backup create my-volume --name my-backup
# Restore a backup to a new volume
openstack volume backup restore my-backup restored-volume
# List backups
openstack volume backup list
Extending Volumes
# Extend a volume (can be done while attached in some backends)
openstack volume set --size 20 my-volume
# Note: The filesystem inside the instance must be resized separately# For ext4: resize2fs /dev/vdb# For XFS: xfs_growfs /mountpoint
Transfer Volumes Between Projects
# Initiate transfer (source project)
openstack volume transfer request create my-volume --name transfer-1
# Note the transfer ID and auth key# Accept transfer (destination project)
openstack volume transfer request accept <transfer-id> <auth-key>
Force-Detach Stuck Volumes
# Reset volume state if stuck in "attaching" or "detaching"
openstack volume set --state available my-volume
# Force-detach from all instances
cinder reset-state --state available --attach-status detached my-volume
Troubleshoot
Volume Stuck in Creating/Deleting State
Symptoms: Volume shows "creating" or "deleting" status indefinitely.
Diagnostic sequence:
Check cinder-volume logs:docker logs cinder_volume 2>&1 | tail -50. Look for errors about LVM commands or iSCSI target creation failures.
Check volume group space:vgs cinder-volumes. If VFree is 0, no space to create volumes.
Check LVM directly:lvs cinder-volumes. Look for the logical volume corresponding to the Cinder volume UUID.
Reset state if needed:openstack volume set --state error <volume> then delete, or openstack volume set --state available <volume> if the underlying LV exists.
Check iSCSI target:targetcli ls to see if the iSCSI target was created but Cinder lost track of it.
Volume Attach Failures
Symptoms:openstack server add volume fails or times out; instance does not see the block device.
Diagnostic sequence:
Check iSCSI initiator (compute host):iscsiadm -m session to list active iSCSI sessions. If the volume target is not connected, check iscsid service.
Check target availability:targetcli ls on the volume host. The target should be listed with the correct LUN.
Check Nova compute logs:docker logs nova_compute 2>&1 | grep <volume-id>. Look for os-brick connector errors.
Check multipath (if configured):multipath -ll. Misconfigured multipath can cause attach failures or device conflicts.
Check device busy: If a previous detach did not clean up, the device may still be in use. Check lsblk and dmsetup ls for stale mappings.
Snapshot Failures
Symptoms: Snapshot creation fails or produces a zero-size snapshot.
Diagnostic sequence:
Check VG space: Snapshots (LVM COW) consume space from the volume group. vgs cinder-volumes to check free space.
COW overflow: If the volume changes faster than the snapshot can track, the COW snapshot becomes invalid. Check lvs for snapshot status (should show active, not "invalid").
Volume in use: Some backends require the volume to be detached for consistent snapshots. Check openstack volume show <volume> -c status.
Symptoms: cinder-volume fails to start or reports "VG cinder-volumes not found."
Diagnostic sequence:
Check VG exists:vgs on the host. If cinder-volumes is not listed, it was never created or the disk failed.
Check VG name match: Compare cinder_volume_group in globals.yml with the actual VG name. They must match exactly.
Check physical volume:pvs to verify the underlying disk/partition is still a PV. Disk failure or partition table changes can remove PV metadata.
Loop device (if used): Check losetup -l to verify the loop device is still attached. Loop devices do not persist across reboots without explicit configuration.
Recreate if needed:pvcreate /dev/<disk> && vgcreate cinder-volumes /dev/<disk>. Warning: this destroys any existing data on the device.
Integration Points
Keystone: All Cinder API calls require Keystone authentication. Cinder registers volumev3 service and endpoint in the Keystone catalog. Service user cinder authenticates for internal operations.
Nova: Nova uses os-brick to connect Cinder volumes to instances. When openstack server add volume is called, Nova coordinates with Cinder to export the volume (iSCSI target) and connect it to the hypervisor (iSCSI initiator). The nova-compute service must have access to the iSCSI network.
Glance: Cinder can create bootable volumes directly from Glance images (openstack volume create --image). Glance can also use Cinder as a backend store for image data.
Swift: When cinder_backup_driver: swift is configured, Cinder stores volume backups as objects in Swift containers. This provides object-level redundancy for backup data.
NASA SE Cross-References
SE Phase
Cinder Activity
Reference
Phase B (Preliminary Design)
Design storage backend: LVM vs Ceph trade study. Plan volume group sizing. Define volume types and QoS requirements. Design backup strategy.
