fabric-storage

name	fabric-storage
description	Configure NVMe storage, persistent volumes, and local disk on FABRIC nodes
allowed-tools	["Read","Grep","Glob","Write","Edit","Bash"]

Instructions

When invoked, generate code to configure storage on FABRIC nodes. There are three storage types:

1. Local Disk — Default VM disk, sized via disk parameter in add_node()
2. NVMe — High-performance Intel P4510 NVMe SSD added as a component
3. Persistent Storage — Project-level volumes that persist across slices

Help the user choose the right storage type and generate the full setup code including formatting and mounting.

API Reference

Local Disk (Default)

Configured at node creation time:

node = slice.add_node(name="node1", disk=100)  # 100 GB local disk

NVMe Component

# Add NVMe to a node
node.add_component(model="NVME_P4510", name="nvme1")

# After slice submit, get storage object
storage = node.get_storage("nvme1")

# Get device name (e.g., /dev/vdb)
device_name = storage.get_device_name()

# Auto-configure NVMe (partition, format, mount)
storage.configure_nvme(mount_point="")  # Default: /mnt/{device_name}

Finding sites with NVMe:

fablib.get_random_site(
    filter_function=lambda x: x["nvme_available"] > 0
)

Persistent Storage

# Attach a pre-created persistent storage volume
node.add_storage(
    name: str,                # Name of the storage volume (must be pre-created in project)
    auto_mount: bool = False, # Auto-mount at /mnt/<name>
) -> Component

# After submit, get storage object and device name
storage = node.get_storage("volume-name")
device_name = storage.get_device_name()

Persistent storage volumes are created at the project level via the FABRIC portal, not through FABlib API.

Patterns

NVMe Storage Setup (Using configure_nvme)

from fabrictestbed_extensions.fablib.fablib import FablibManager

fablib = FablibManager()

# Find a site with NVMe
site = fablib.get_random_site(
    filter_function=lambda x: x["nvme_available"] > 0
)

slice = fablib.new_slice(name="nvme-experiment")

node = slice.add_node(
    name="storage-node",
    site=site,
    cores=4,
    ram=16,
    disk=50,
    image="default_ubuntu_24",
)
node.add_component(model="NVME_P4510", name="nvme1")

slice.submit()

# Use the built-in configure_nvme helper
node = slice.get_node(name="storage-node")
storage = node.get_storage("nvme1")

# This partitions, formats, and mounts the NVMe drive
storage.configure_nvme()

# Verify
stdout, _ = node.execute("df -h")
print(stdout)

NVMe Manual Setup

node = slice.get_node(name="storage-node")
storage = node.get_storage("nvme1")
device_name = storage.get_device_name()

# Format the drive
node.execute(f"sudo mkfs.ext4 {device_name}")

# Mount the drive
node.execute(f"sudo mkdir -p /mnt/nvme")
node.execute(f"sudo mount {device_name} /mnt/nvme")

# Make writable by user
node.execute("sudo chown $(whoami):$(whoami) /mnt/nvme")

# Verify
stdout, _ = node.execute("df -h /mnt/nvme")
print(stdout)

NVMe with Persistent Mount (survives reboot)

storage = node.get_storage("nvme1")
device_name = storage.get_device_name()

commands = [
    f"sudo mkfs.ext4 {device_name}",
    "sudo mkdir -p /mnt/nvme",
    f"echo '{device_name} /mnt/nvme ext4 defaults 0 0' | sudo tee -a /etc/fstab",
    "sudo mount -a",
    "sudo chown $(whoami):$(whoami) /mnt/nvme",
]
for cmd in commands:
    node.execute(cmd)

Persistent Storage (Pre-created Volume)

site = "STAR"
storage_name = "my-project-data"

slice = fablib.new_slice(name="persistent-storage")

# Node must be at the same site as the storage volume
node = slice.add_node(name="node1", site=site, cores=4, ram=16, disk=50)

# Attach pre-created persistent storage volume
node.add_storage(name=storage_name)

slice.submit()

# Get storage device name
node = slice.get_node(name="node1")
storage = node.get_storage(storage_name)
device_name = storage.get_device_name()

# First use ONLY: format the volume (WARNING: erases all data!)
# node.execute(f"sudo mkfs.ext4 {device_name}")

# Mount the volume (works for first and subsequent uses)
node.execute(f"sudo mkdir -p /mnt/fabric_storage")
node.execute(f"sudo mount {device_name} /mnt/fabric_storage")

# Verify
stdout, _ = node.execute("df -h /mnt/fabric_storage")
print(stdout)

# Data persists even after slice deletion!

Large Local Disk

slice = fablib.new_slice(name="large-disk")

# Request a larger local disk (up to site limits)
node = slice.add_node(
    name="data-node",
    cores=4,
    ram=16,
    disk=500,  # 500 GB local disk
)

slice.submit()

# Check disk space
stdout, _ = node.execute("df -h /")
print(stdout)

Multiple NVMe Drives

site = fablib.get_random_site(
    filter_function=lambda x: x["nvme_available"] >= 2
)

slice = fablib.new_slice(name="multi-nvme")
node = slice.add_node(name="node1", site=site, cores=8, ram=32, disk=50)
node.add_component(model="NVME_P4510", name="nvme1")
node.add_component(model="NVME_P4510", name="nvme2")

slice.submit()

# Configure both NVMe drives
node.get_storage("nvme1").configure_nvme()
node.get_storage("nvme2").configure_nvme()

# Verify
stdout, _ = node.execute("df -h")
print(stdout)

Important Notes

• Use get_device_name(): Don't hardcode device names like /dev/nvme0n1 — use storage.get_device_name() instead
• Use configure_nvme(): This helper partitions, formats, and mounts automatically
• First use vs subsequent use: Only format (mkfs) persistent storage on first use! Formatting erases all data.
• Persistent storage volumes are created in the FABRIC portal under your project, not via the API
• Same-site requirement: Persistent storage must be attached to a node at the same site as the volume
• Auto-mount: When auto_mount=True, the volume is mounted automatically