xarray brings the power of pandas-style labeled indexing to N-dimensional arrays.
Combined with Dask for lazy out-of-core computation and Zarr for cloud-native storage,
it forms the backbone of modern climate, oceanography, and remote-sensing workflows.
When to Use This Skill
Use this skill when you need to:
Open, inspect, and process NetCDF4, HDF5, or GRIB files without loading all data to RAM.
Work with coordinates (lat, lon, time, level) attached to array dimensions.
Run climatological analyses: seasonal means, anomalies, rolling statistics.
Rechunk large datasets for parallel processing with Dask workers.
Convert legacy .nc files to chunked Zarr stores for fast cloud access.
Attach CF-compliant metadata (units, standard_name, calendar) to output files.
Merge, align, or interpolate multiple datasets with different grids.
Do not use this skill for purely tabular data (use pandas) or for small arrays
that fit comfortably in memory without any need for coordinate labels (use NumPy).
Background & Key Concepts
xarray Data Structures
Class
Description
xr.DataArray
Single N-D variable with named dims, coords, and attrs
xr.Dataset
Dict-like container of DataArrays sharing dims/coords
A DataArray records dimensions (e.g., "time", "lat", "lon"),
coordinates (1-D arrays of tick values for each dim), and attributes (metadata dict).
NetCDF & CF Conventions
NetCDF-4 is a hierarchical binary format built on HDF5.
The CF (Climate and Forecast) Metadata Conventions standardise:
Coordinate names (latitude, longitude, time, pressure)
units string format ("degrees_east", "K", "days since 1900-01-01")
xarray decodes CF metadata automatically when opening NetCDF files.
Lazy Loading with Dask
When chunks={} or chunks="auto" is passed to xr.open_dataset, each variable
becomes a dask.array — a graph of deferred operations. Nothing is loaded until
.compute() or .load() is called, or data is written to disk.
Choosing good chunk sizes:
Target ~100 MB per chunk.
Chunk along the dimensions you will iterate over (usually time).
Avoid tiny chunks (overhead) and huge chunks (memory spikes).
Zarr
Zarr stores each chunk as a separate compressed file (in a directory or cloud bucket).
This enables concurrent reads by many Dask workers, far outperforming NetCDF for
parallel workloads. The xarray.Dataset.to_zarr() method handles the conversion.
from dask.distributed import Client
client = Client(n_workers=4, threads_per_worker=2, memory_limit="4GB")
print(client.dashboard_link) # open in browser for task graph visualization
Environment variables
# Path to your local data archiveexport NC_DATA_DIR="/data/netcdf"# Optional: fsspec token for cloud access (GCS, S3)export FSSPEC_S3_KEY="<paste-your-key>"export FSSPEC_S3_SECRET="<paste-your-secret>"
Area-weighted global mean accounts for the fact that grid cells near the poles
are smaller than those at the equator.
import xarray as xr
import numpy as np
import pandas as pd
rng = np.random.default_rng(1)
lats = np.linspace(-90, 90, 73)
lons = np.linspace(0, 357.5, 144)
times = pd.date_range("2020-01-01", periods=12, freq="ME")
data = (280 + 20 * np.cos(np.deg2rad(lats))[:, None]
+ rng.standard_normal((73, 12, 144))).transpose(1, 0, 2)
da = xr.DataArray(data, dims=["time", "lat", "lon"],
coords={"time": times, "lat": lats, "lon": lons},
name="tas")
# Cosine-latitude weights
weights = np.cos(np.deg2rad(da.lat))
weights.name = "weights"# Weighted mean over lat and lon
global_mean = da.weighted(weights).mean(dim=["lat", "lon"])
print("Global mean time series:", global_mean.values)
print("Annual mean:", float(global_mean.mean()))
Working with CF Time Calendars
import xarray as xr
import cftime
import numpy as np
# 360-day calendar (used in many climate models)
times_360 = xr.cftime_range(
start="1850-01-01", periods=120, freq="MS", calendar="360_day"
)
data_360 = np.random.rand(120, 10, 20).astype("float32")
da_360 = xr.DataArray(
data_360,
dims=["time", "lat", "lon"],
coords={
"time": times_360,
"lat": np.linspace(-90, 90, 10),
"lon": np.linspace(0, 342, 20),
},
)
print("360-day calendar time range:", da_360.time.values[[0, -1]])
print("Number of time steps:", len(da_360.time))
# Select a decade
decade = da_360.sel(time=slice("1900-01-01", "1909-12-30"))
print("Decade shape:", decade.shape)
# Convert to standard Gregorian calendar by resampling is not trivial;# the recommended path is to convert units after writing back to NetCDF.
Parallel Processing with Dask Distributed
import xarray as xr
import numpy as np
import pandas as pd
from dask.distributed import Client, LocalCluster
import os
DATA_DIR = os.getenv("NC_DATA_DIR", "./data")
defrun_parallel_analysis():
cluster = LocalCluster(n_workers=2, threads_per_worker=2)
client = Client(cluster)
print("Dashboard:", client.dashboard_link)
# Open a large (synthetic) dataset with dask
times = pd.date_range("1950-01-01", periods=840, freq="ME")
lats = np.linspace(-90, 90, 73)
lons = np.linspace(0, 357.5, 144)
rng = np.random.default_rng(7)
data = rng.standard_normal((840, 73, 144)).astype("float32") + 280
ds = xr.Dataset(
{"tas": (["time", "lat", "lon"], data)},
coords={"time": times, "lat": lats, "lon": lons},
).chunk({"time": 60, "lat": -1, "lon": -1})
# Compute global mean in parallel
weights = np.cos(np.deg2rad(ds["tas"].lat))
global_mean = ds["tas"].weighted(weights).mean(["lat", "lon"])
result = global_mean.compute()
print("Global mean std (70 years):", float(result.std()))
client.close(); cluster.close()
return result
# result = run_parallel_analysis()
Troubleshooting
File opens but all variables show NaN
Cause: _FillValue is being decoded but the compression codec does not match
the version of netCDF4 used to write the file.
Fix: Open with mask_and_scale=False to inspect raw values first.
ds_raw = xr.open_dataset("file.nc", mask_and_scale=False)
print(ds_raw["tas"].values[:5]) # inspect without NaN substitution
