coreweave-data-handling

name	coreweave-data-handling
description	Handle training data and model artifacts on CoreWeave persistent storage. Use when managing large datasets, configuring storage classes, or implementing data pipelines for GPU workloads. Trigger with phrases like "coreweave data", "coreweave storage", "coreweave pvc", "coreweave dataset management".
allowed-tools	Read, Write, Edit, Bash(kubectl:*), Grep
version	1.11.0
license	MIT
author	Jeremy Longshore <[e-mail]>
tags	["saas","gpu-cloud","kubernetes","inference","coreweave"]
compatibility	Designed for Claude Code

CoreWeave Data Handling

Community-contributed. Not affiliated with, endorsed by, or sponsored by CoreWeave, Inc. CoreWeave is a registered trademark of CoreWeave, Inc.

Overview

CoreWeave GPU cloud workloads involve large-scale data artifacts: model weights (multi-GB safetensors/GGUF), training datasets (parquet, TFRecord, WebDataset), checkpoint snapshots, and inference cache volumes. Data flows through Kubernetes PersistentVolumeClaims backed by region-specific storage classes. Compliance requires encryption at rest via the storage driver, namespace-scoped RBAC for volume access, and audit logging for any data egress from GPU nodes.

Data Classification

Data Type	Sensitivity	Retention	Encryption
Model weights	Medium	Until deprecated	AES-256 at rest
Training datasets	High (may contain PII)	Per data license	AES-256 + TLS in transit
Checkpoint snapshots	Medium	30 days post-training	AES-256 at rest
Inference cache	Low	Session/TTL	Volume-level encryption
HuggingFace tokens	Critical	Rotate quarterly	K8s Secret + KMS

Data Import

import { KubeConfig, BatchV1Api } from '@kubernetes/client-node';

async function importDataset(pvcName: string, sourceUrl: string, namespace: string) {
  const kc = new KubeConfig();
  kc.loadFromDefault();
  const batch = kc.makeApiClient();
   job = {
    : { : , namespace },
    : { : { : {
      : ,
      : [{ : , : ,
        : [, , ],
        : [{ : , :  }],
      }],
      : [{ : , : { : pvcName } }],
    }}}
  };
   batch.(namespace, { : job });
}