datasets

Work with geospatial data using GDAL/OGR and DuckDB, especially reading remote files via virtual filesystems (/vsicurl/, /vsis3/), inspecting multi-layer sources, converting between formats, and handling geometry edge cases. Covers the relationship between GDAL, DuckDB spatial, and Parquet/GeoParquet — including which tool can read what and common pitfalls with driver availability. Use when working with geospatial file formats (GDB, GPKG, Shapefile, GeoParquet, GeoTIFF, PMTiles), converting between them, or reading remote geospatial data.

Run batch processing jobs on the NRP (National Research Platform) Nautilus Kubernetes cluster. Covers the mandatory requirements for CPU jobs: opportunistic priority class, resource requests/limits, and GPU node avoidance. Use when creating or managing Kubernetes jobs on the NRP Nautilus cluster, or when the user mentions NRP, Nautilus, or needs to run batch workloads on a shared academic cluster.

Manage S3 object storage on the NRP (National Research Platform) Nautilus cluster, which uses Ceph S3 (not AWS). Covers public vs internal endpoints, rclone usage, bucket creation, public-read policies, CORS configuration, DuckDB S3 access, credential handling, and syncing to external S3 providers like source.coop. Use when working with S3 buckets on NRP Nautilus, uploading or downloading data, configuring bucket access, reading data from Ceph S3 storage, or syncing datasets to source.coop or other AWS S3 destinations.