gaiadr3-aip-de-adql

name	gaiadr3-aip-de-adql
description	Query the Gaia DR3 PostgreSQL database at gaia.aip.de via its Daiquiri REST API. Includes CSRF handling, async job submission, queue names, and result fetching. Access 1.8 billion sources with ~153 columns.
version	1.0.0
author	AstroAgent / AIP
license	MIT
metadata	{"hermes":{"tags":["astronomy","gaia-dr3","postgresql","adql","daiquiri","rest-api","gaia-aip"],"category":"astronomy","related_skills":["rave-dr6","shboost24-cmd","starhorse-access","gaiadr3-aip-de-adql"]}}

Gaia DR3 @AIP — Daiquiri REST API

When to Use

Use this skill when querying the Gaia DR3 catalog at https://gaia.aip.de/ programmatically — count rows, sample data, filtered selections. For browser-based queries use the web UI; for scripted, large, or reproducible queries use this REST API.

Prerequisites

• curl and Python 3 stdlib only — no API keys, no installation needed.

Procedure

1. Get CSRF token and session cookie

curl -s -c /tmp/gaia_cookies.txt "https://gaia.aip.de/query//sql/" > /dev/null
CSRF=$(grep csrftoken /tmp/gaia_cookies.txt | awk '{print $7}')

2. Submit an async query job

curl -s -b /tmp/gaia_cookies.txt -c /tmp/gaia_cookies.txt \
    -X POST \
    -H "Referer: https://gaia.aip.de/query//sql/" \
    -H "Content-Type: application/json" \
    -H "X-CSRFToken: $CSRF" \
    -H "X-Requested-With: XMLHttpRequest" \
    -d '{"query": "SELECT COUNT(*) FROM gaiadr3.gaia_source", "query_language": "postgresql", "queue": "5m"}' \
    "https://gaia.aip.de/query/api/jobs/"

Returns: {"id": "<uuid>", "phase": "QUEUED", ...}

3. Poll for completion

JOB_ID="<uuid_from_step2>"
for i in $(seq 1 20); do
    sleep 5
    PHASE=$(curl -s -b /tmp/gaia_cookies.txt -c /tmp/gaia_cookies.txt \
        -H "Referer: https://gaia.aip.de/query/$JOB_ID/" \
        -H "X-CSRFToken: $CSRF" \
        -H "X-Requested-With: XMLHttpRequest" \
        "https://gaia.aip.de/query/api/jobs/$JOB_ID/" | \
        python3 -c "import sys,json; print(json.load(sys.stdin).get('phase'))")
    echo "[$((i*5))s] Phase: $PHASE"
    [ "$PHASE" = "COMPLETED" ] && break
    [ "$PHASE" = "ERROR" ] && break
done

4. Fetch results

curl -s -b /tmp/gaia_cookies.txt -c /tmp/gaia_cookies.txt \
    -H "Referer: https://gaia.aip.de/query/$JOB_ID/" \
    -H "X-CSRFToken: $CSRF" \
    -H "X-Requested-With: XMLHttpRequest" \
    "https://gaia.aip.de/query/api/jobs/$JOB_ID/rows/?limit=10&offset=0"

5. Get table metadata (column list)

curl -s "https://gaia.aip.de/metadata/gaiadr3/gaia_source/"

Queue Names

UI label	Queue param	Timeout
"30 Seconds"	omit (default)	30s — times out on large queries
"5 Minutes"	"5m"	5 min
"2 Hours"	"2h"	2 hours

Always use "5m" or "2h" for COUNT or large scans.

Query Language

Always use "postgresql" — not adql, even though the web UI defaults to ADQL.

Key Endpoints

Method	Endpoint	Purpose
POST	/query/api/jobs/	Submit async query job
GET	/query/api/jobs/{id}/	Poll job status (phase)
GET	/query/api/jobs/{id}/rows/	Fetch JSON results
GET	/metadata/gaiadr3/gaia_source/	Full column list with types/units

Known Limitations

• /query/tap/sync/ and /query/tap/async/ return HTML — do NOT use for programmatic queries.
• /query/sync/ returns 405 Method Not Allowed.
• Default 30s queue times out on COUNT over gaiadr3.gaia_source (~1.8B rows) — always use "5m".
• CSRF token must be extracted from the cookie jar (csrftoken cookie) and sent via X-CSRFToken header.
• Required headers: X-Requested-With: XMLHttpRequest, Referer: https://gaia.aip.de/query//sql/.

Database Stats

• Table: gaiadr3.gaia_source
• Row count: ~1,811,709,771 sources (as of 2026)
• Columns: 153 (solution_id, designation, source_id, ra, dec, parallax, parallax_error, pm, pmra, pmdec, phot_g_mean_mag, phot_rp_mean_mag, phot_bp_mean_mag, bp_rp, radial_velocity, l, b, ecl_lon, ecl_lat, and many more)
• Distance estimation: distance_gspphot, or 1000/parallax for simple estimates

Canonical Routing

This is a specialized or legacy example skill. For new work, start with astro-data-access-umbrella and route through:

• gaia-aip-data-access

Keep this skill for dataset-specific examples, but prefer the canonical skills for new implementations, live probes, REANA execution, and plotting/cache conventions.

Pitfalls

• Do NOT use TAP endpoints — they return HTML, not JSON.
• Do NOT omit the queue parameter for large scans — the default 30s timeout will truncate results.
• CSRF token expires — always refresh it by re-requesting the base URL before submitting a new job.
• Do NOT assume the UI queue names match the API — use "5m", "2h".

Verification

• CSRF extraction returns a non-empty token.
• Job submission returns a valid UUID.
• Polling eventually reaches COMPLETED (not ERROR).
• /metadata/ endpoint returns a JSON list of 153 columns.