portaljs-migrate

name	portaljs-migrate
description	Migrate (harvest) datasets between open-data platforms. Reads CKAN, a DCAT-US /data.json catalog (DKAN, ArcGIS Hub, data.gov), a DCAT / DCAT-AP RDF feed (JSON-LD, Turtle, or RDF/XML), Socrata, OpenDataSoft, or an ArcGIS FeatureServer, and writes them to a static PortalJS catalog or pushes them into a CKAN instance over its API. Use when moving datasets from an external open-data platform into a PortalJS portal, or bridging one CKAN instance to another.
allowed-tools	Read, Write, Edit, Bash(npm:*), Bash(npx:*), Bash(node:*), Bash(curl:*), Bash(git:*), WebFetch
version	1.0.0
author	Datopian <[e-mail]>
license	MIT
compatibility	Claude Code with PortalJS portals (Next.js 14, React 18, Node 18+). Runs from any project via the plugin, a personal ~/.claude/commands install, or a portaljs clone.
tags	["portaljs","data-portal","migration","harvest","dcat","ckan"]

PortalJS — Migrate

Overview

Harvest datasets from an external open-data platform into a PortalJS portal, or push them into a CKAN instance. Every source reads into one canonical shape — { slug, namespace, name, description, resources[] } — then writes to the target from that shape, so adding a source covers every target automatically.

Supported sources: CKAN, a DCAT-US /data.json catalog (DKAN, ArcGIS Hub, data.gov), a DCAT / DCAT-AP RDF feed, Socrata, OpenDataSoft, or an ArcGIS FeatureServer/MapServer. Supported targets: a static PortalJS catalog (datasets.json, resources linked by URL or downloaded into Cloudflare R2) or a CKAN instance over its write API.

This is the copy-into-the-portal path, the inverse of /portaljs-connect-ckan (which reads the source live at build time): a one-time, re-runnable snapshot so the portal stands alone and needs no backend.

Prerequisites

• An existing PortalJS portal for the static target: datasets.json, package.json, and pages/[owner]/[slug].tsx must exist (the portaljs-catalog template).
• Node 18+ and npm, with portal dependencies installed (npm install).
• Network access to the source platform's API or feed URL.
• For a CKAN target: a write API key in CKAN_API_KEY — never hardcoded.
• For download copy mode: git-lfs and a PortalJS Arc account (or a self-hosted Giftless endpoint) to push resource files to R2.
• For dcat-rdf sources: lib/metadata/dcat-harvest.ts and dcat-profiles.ts (copy from examples/portaljs-catalog/lib/metadata/ if missing), run via npx tsx.

Instructions

Full step-by-step workflow: .claude/commands/portaljs-migrate.md.

1. Gather input — source type, source URL, target (static/ckan), portal directory, copy mode (link/download), filters. Interview instead of erroring.
2. Validate the target — confirm the static catalog files exist, or that the CKAN URL and API key authenticate.
3. Detect the source type from the URL (or --source) and verify reachability.
4. Read the source into canonical dataset entries, per the source's field table in the reference doc, applying the metadata hygiene rules (license sentinel — never fabricate CC-BY; description sanitizer + fallback chain; source dates preserved with separate; display-name cleanup with slugs from the raw title; from a meaningful theme).