| name | portaljs-migrate |
| description | Migrate (harvest) datasets between open-data platforms. Reads CKAN, a DCAT-US /data.json catalog (DKAN, ArcGIS Hub, data.gov), a DCAT / DCAT-AP RDF feed (JSON-LD, Turtle, or RDF/XML), Socrata, OpenDataSoft, or an ArcGIS FeatureServer, and writes them to a static PortalJS catalog or pushes them into a CKAN instance over its API. Use when moving datasets from an external open-data platform into a PortalJS portal, or bridging one CKAN instance to another. |
| allowed-tools | Read, Write, Edit, Bash(npm:*), Bash(npx:*), Bash(node:*), Bash(curl:*), Bash(git:*), WebFetch |
| version | 1.0.0 |
| author | Datopian <hello@datopian.com> |
| license | MIT |
| compatibility | Claude Code with PortalJS portals (Next.js 14, React 18, Node 18+). Runs from any project via the plugin, a personal ~/.claude/commands install, or a portaljs clone. |
| tags | ["portaljs","data-portal","migration","harvest","dcat","ckan"] |
PortalJS — Migrate
Overview
Harvest datasets from an external open-data platform into a PortalJS portal, or
push them into a CKAN instance. Every source reads into one canonical shape —
{ slug, namespace, name, description, resources[] } — then writes to the target
from that shape, so adding a source covers every target automatically.
Supported sources: CKAN, a DCAT-US /data.json catalog (DKAN, ArcGIS Hub,
data.gov), a DCAT / DCAT-AP RDF feed, Socrata, OpenDataSoft, or an ArcGIS
FeatureServer/MapServer. Supported targets: a static PortalJS catalog
(datasets.json, resources linked by URL or downloaded into Cloudflare R2) or a
CKAN instance over its write API.
This is the copy-into-the-portal path, the inverse of /portaljs-connect-ckan
(which reads the source live at build time): a one-time, re-runnable snapshot so
the portal stands alone and needs no backend.
Prerequisites
- An existing PortalJS portal for the static target:
datasets.json,
package.json, and pages/[owner]/[slug].tsx must exist (the portaljs-catalog
template).
- Node 18+ and npm, with portal dependencies installed (
npm install).
- Network access to the source platform's API or feed URL.
- For a CKAN target: a write API key in
CKAN_API_KEY — never hardcoded.
- For
download copy mode: git-lfs and a PortalJS Arc account (or a
self-hosted Giftless endpoint) to push resource files to R2.
- For
dcat-rdf sources: lib/metadata/dcat-harvest.ts and dcat-profiles.ts
(copy from examples/portaljs-catalog/lib/metadata/ if missing), run via npx tsx.
Instructions
Full step-by-step workflow:
.claude/commands/portaljs-migrate.md.
- Gather input — source type, source URL, target (
static/ckan), portal
directory, copy mode (link/download), filters. Interview instead of erroring.
- Validate the target — confirm the static catalog files exist, or that the CKAN
URL and API key authenticate.
- Detect the source type from the URL (or
--source) and verify reachability.
- Read the source into canonical dataset entries, per the source's field table in
the reference doc.
