forgetmenot

Tidy a messy SPARQL query into the house style WITHOUT changing what it means, and prove the meaning is unchanged. Use whenever you are cleaning up, reformatting, commenting, or de-duplicating a SPARQL query (e.g. the queries embedded in the api.parliament.uk procedure-browser pages) and need a repeatable methodology plus a checker that confirms the before/after pair is equivalent. Bundles the cleanup rules, a worked before/after example, and `bin/sparql-equiv.sh` — an equivalence checker using Apache Jena ARQ (canonical algebra) plus optional live-endpoint result-set comparison.

Bulk-export the UK Parliament Thesaurus — the SKOS concept scheme used to subject-tag debates, papers, written questions, and other parliamentary records. Crawls the legacy Linked Data API at lda.data.parliament.uk/terms (no SPARQL endpoint exposed for it) into a single N-Quads file split across topical named graphs. Use when a question needs the thesaurus as RDF (e.g. for joining into a local SPARQL store, building a faceted UI, or exporting subject hierarchies). The thesaurus is the canonical subject vocabulary at parliament.uk; pair with `linked-data-api` for the document-level metadata that references these terms.

Research dossier + method for reconciling UK Bills across every surface that records them — the Bills REST API, the UK Parliament Thesaurus (LEG terms), the DDP SPARQL store (ActOfParliament / StatutoryInstrumentPaper), legislation.gov.uk, Wikidata (via QLever), modern Hansard and Historic Hansard (1803–2005), and the Journals of the House. Use when you need to know WHERE a bill (especially a pre-2015 historical one) is recorded, how complete each source is, whether matching them is a SPARQL/ETL job or an AI/fuzzy one, and how to build a bill-identity crosswalk graph (billId ↔ LEG term ↔ Hansard stages ↔ resulting Act ↔ legislation.gov.uk ↔ Wikidata QID). Complements the `bills` skill (which just wraps the live Bills API).

Query the forgetmenot RSS/Atom feed catalogue — a concrete, metadata-rich index of every syndication feed across the UK Parliament web estate (House of Commons Library and House of Lords Library WordPress archives, exposed as <archive>/feed/, plus the first-party Bills API RSS), built by this repo and served on the fpkg demo app at https://fpkg.fly.dev/kgx/feeds. Use when the question is "what RSS/Atom feeds exist for Parliament", to enumerate per-topic / per-type / per-tag / per-author Library feeds with chamber + subject metadata, to track new Library briefings or Bills, or to cross-walk Library subject feeds against a controlled vocabulary (EuroVoc / GEMET) via the skosdex skill. Distinct from `library-feeds` (which wraps the official api.parliament.uk/library-feeds Rails aggregator); this is our own derived catalogue with a richer taxonomy and a live reader UI.

Search and query skosdex (skosdex.fly.dev) — a skills-first index of ~648 openly-licensed SKOS concept schemes (controlled vocabularies / thesauri) from around the world: AGROVOC, EuroVoc, GEMET, LCSH, Getty AAT/ULAN, IPTC NewsCodes, Loterre, Finto/YSO and many more, ~2.5M concepts with cross-vocabulary skos:exactMatch / closeMatch mappings. Two surfaces: a Solr full-text index (the fast, populated search surface) and a SPARQL 1.1 endpoint (Oxigraph) where each scheme is its OWN NAMED GRAPH — the default graph is empty, so SPARQL patterns must use GRAPH ?g { … }. Third-party, NOT UK Parliament. Use when you need to look up a subject term / concept, walk a broader/narrower thesaurus hierarchy, or bridge a term across controlled vocabularies (e.g. tag parliamentary material against EuroVoc or GEMET).

kgx node-flow lib — the canonical implementation of the daisychain / pivcab / studio op-node API. Use when wiring a new chain, adding an op kind, exposing a new SPARQL endpoint, writing tests against the library chain spec, generating a TriG manifest from a chain, or debugging why a saved chain isn't replaying. Read this before touching `demos/parliament-live/web/kgx/lib/*.mjs`, `demos/parliament-live/web/kgx/daisychain/index.html` op-table or starter-list, `bin/kgx.mjs` SPARQL routing, or the LIBRARY iteration tests.

Fetch, parse and recurse XML sitemaps (the Sitemaps 0.9 protocol — sitemap.xml / sitemapindex.xml) for the UK Parliament web presence, enumerate every public page URL (~1.09M across the estate), and track updates via the RSS/Atom feeds. Use when you need the inventory of public web pages rather than structured API records — especially the many pages with NO JSON/REST API (corporate "About"/living-heritage, visiting, get-involved, site-information, Library landing pages, guidance), to know what the wrapped Members/Bills/Hansard/SPARQL facilities do NOT cover. Entry: www.parliament.uk/sitemapindex.xml; publications via www.publications.parliament.uk/sitemap_index.xml (21 child sitemaps, ~1.04M URLs). A cached snapshot, a concrete 918-feed catalogue, a liveness survey, a D3 web-estate map (/kgx/sitemap-tree) and a feed reader (/kgx/feeds) are all committed — see docs/data-state.md. Parses offline from a saved file too. Generic — works on any site's sitemap.

Notes, gotchas and utility recipes for the cloud-platform / container infrastructure this repo uses — fly.io (the fpkg demo app), Docker multi-stage builds bundling RDF / Oxigraph stores, GitHub Actions workflow patterns (dispatch + path triggers + flyctl), and the tiny mobile-only operational reality the repo runs in. Use when you're touching a Dockerfile, a fly.toml, a GHA workflow, or hitting an "it should be easy but…" cloud-tooling surprise.

Drive a deployed (or local) web app from this rootful Ubuntu container via the chrome-devtools MCP — to verify interactive UX (modal a11y, focus management, keyboard shortcuts, streamed renders, shadow-DOM state) that source-grep + unit tests can't see. Use whenever someone asks "does Escape actually close the modal", "does focus restore", "is the tile grid streaming smoothly", or "does the live deploy actually have my fix" and you need to drive a real Chromium rather than reason about it. Covers the install + shim recipe needed to make Chrome work as root in this container, the worked verification pattern (snapshot → click → evaluate → assert), and a real-bug case study (shadow-DOM focus restoration) that proves source-grep is not enough.

Use the UK Parliament parameterised query browser (api.parliament.uk/query/) — a catalogue of pre-canned SPARQL queries that return JSON without you having to write the SPARQL yourself. Each template has a fixed name and a fixed parameter signature, e.g. person_by_id?person_id=… or constituency_lookup_by_postcode?postcode=…. Use when you want a specific, pre-vetted answer about a Parliament resource and do not want to author SPARQL.

Query the UK Parliament public SPARQL 1.1 endpoint at api.parliament.uk/sparql. Integrated graph (DDP) covering people, parties, governments, houses, constituencies, parliamentary periods, incumbencies, formal-body memberships, the procedural meta-layer (procedures / steps / routes / work packages), treaties, statutory instruments, written questions and answers, ePetitions, and historical Acts. Use whenever a question joins across more than one of those topics — SPARQL is the only place where everything is linked by URI. Not in DDP: Bills, Hansard contributions, Divisions / votes, Elections / candidates / results, Early Day Motions, the Register of Members' Financial Interests, Erskine May, committee evidence / inquiries — those require their dedicated REST APIs.

Cross-source identity reconciliation for every UK MP and peer in this repository, as a single RDF N-Quads file. Bridges Members API id ↔ MNIS id ↔ DDP person LocalId ↔ scraped per-MP website ↔ APPG officer roles ↔ GOV.UK people factoids ↔ Wikidata QIDs, each in its own named graph. Use whenever a question needs to join data across more than one of those sources by person — most commonly to confirm that "the gov.uk page for X", "the psephology winning candidacy for X" and "the X currently sitting in the Lords as Lord Y" are the same human. The `parl:memberId` literal is the stable join key; it stays the same across MP-then-peer career transitions (e.g. Hague MNIS 379 = MP for Richmond + Lord Hague of Richmond; Cameron MNIS 1467 = MP for Witney + Lord Cameron of Chipping Norton 2023; Darling MNIS 596 = MP for Edinburgh SW + Lord Darling of Roulanish 2015-20).

Spin up a local SPARQL endpoint over any RDF file in this repository — N-Quads, Turtle, N-Triples, RDF/XML, TriG, gzipped or plain. Three backends covered: `rdflib-endpoint` (pure-Python, easiest), Apache Jena Fuseki (Java, web UI, SPARQL Update), and Oxigraph (Rust via Docker, persistent). Use whenever you want to query one of the repo's N-Quads dumps (psephology, govuk-orgchart, fcdo-treaties, identity-graph, tna-legislation) with real SPARQL instead of grep, or to host an endpoint for downstream tooling. Generic — not tied to any one corpus.

Data-quality discipline for the corpora in this repo. Use whenever building a new crawler/extractor, when an extraction "feels done", when a question hits the data and would benefit from explicit confidence, or when you find yourself reaching for a hybrid query that papers over a single-source gap. Prescribes anchor cases, cross-extractor checks, upstream-bug reporting, and identifier hygiene (Wikidata QIDs, gov.uk content-id, parliament.uk member id) rather than URLs or names.

Authoritative election-results database for UK Commons elections, maintained by the House of Commons Library and distributed as daily PostgreSQL dumps at github.com/ukparliament/psephology. Covers candidacies, certifications by political parties, vote counts/shares, electorates, constituency boundary sets, general elections and by-elections. **SCOPE — earliest real polling-day data is 2010-05-06**; the 2005 GE is present only as 591 notional re-allocations onto post-2010 boundaries (every row `is_notional=true`); by-elections run 2010 → present. Commons only — Lords members never appear. Use when a question needs structured per-candidate / per-constituency vote totals, vote shares, majorities or boundary-set history at any GE from 2010 onwards, or for any by-election since 2010. For pre-2010 questions (Brown 2009 cabinet, Blair-era contests, peer-vs-Commons-at-the-time joins) route via the live `api.parliament.uk/sparql` (DDP) — see `docs/sparql-endpoints.md`.

Use the `parl` Node CLI (and the matching `lib/facilities/*.mjs` JS library) to query every public UK Parliament API and dataset — Members, Bills, Hansard, Committees, Commons / Lords divisions, oral and written questions, Early Day Motions, statutory instruments, treaties, the Register of Members' Financial Interests, Erskine May, the live annunciator, e-petitions, the SPARQL endpoint, OData, the parameterised-query browser, the Linked Data API, historic Hansard, MNIS, the explore.data.parliament.uk dataset catalogue, the All-Party Parliamentary Groups Register, the What's On calendar (sittings / sessions / procedural dates), the MPs' Guide to Procedure, the Bill Papers CSV catalogue, and the Library Feeds RSS aggregator. Use whenever a question references UK Parliament data — this is the single entry point that every per-facility skill in this repo delegates to.

Crawl and query FCDO UK Treaties Online (treaties.fcdo.gov.uk) -- the canonical UK government register of treaties and international agreements, ~21,957 records back to the early 19th century. Catalogue metadata only (title, parties, signed date / place, treaty series references, subject); signatory NAMES are NOT in the public-anonymous record, despite the prior design doc's claim. Use whenever the question is about UK treaties beyond the ~323-record Parliament Treaties API CRaG window, or treaties pre-2010.

Crawl, extract and query the GOV.UK government org chart -- ministerial roles, the people who hold them, the organisations they serve, and past office-holders. Use whenever a question is about who holds which UK ministerial role, which department they sit in, past holders of an office, or cross-joining with the Parliament Members API or Wikidata. Loads its query capability via a bundled script (no MCP server, no boot-time tokens).

Query FixMyStreet (mySociety) — the per-council issue-reporting service used for potholes, fly-tipping, broken street furniture, and other street-level issues across UK local authorities. Surface is RSS only (no public JSON API on fixmystreet.com — the /reports.json and /api paths return 404; the open-source FixMyStreet platform's API spec is only exposed on cobrand instances that opt in). Wraps the all-reports feed, per-area feeds, and per-postcode / lat-lon proximity feeds. Use when the question is about street-level issues reported in a constituency, a council's response patterns, or as constituency-level civic-engagement signal.

Query the Northern Ireland Assembly Open Data API at data.niassembly.gov.uk — Members of the Legislative Assembly (MLAs, ~90 current), parties, constituencies, Hansard, written questions, departments, organisations, plenary divisions, motions and business diary. Wraps 35+ JSON operations across six ASMX services (members, hansard, committees, questions, organisations, plenary). Free, no auth. First-party to the NI Assembly; tier-3 from a Westminster perspective. Use when the question is about an MLA, an NI Assembly committee, a Stormont division, a question to an NI department, or NI electoral geography.

STUB — Query Senedd Cymru (Welsh Parliament) Open Data. The official endpoint at business.senedd.wales/mgWebService.asmx is a Microsoft ModernGov SOAP service. We have not yet built a SOAP client — operations return HTTP 500 to plain GET requests. This stub exposes WSDL fetching for discovery; a full wrap needs proper SOAP envelope POSTs (see WSDL for operation list). First-party to Senedd Cymru; tier-3 from a Westminster perspective. Pairs with sp (Scottish Parliament) and nia (NI Assembly) for cross-devolved research.

Query the EUR-Lex CELLAR SPARQL endpoint at publications.europa.eu/webapi/rdf/sparql — EU law and case-law as Linked Data, with extensive cross-references to national-law transpositions. Of particular UK Parliament interest for Retained EU Law Act 2023 research — every UK statute or SI that retains, modifies, or revokes EU-origin law has a CELEX-number reference that resolves here. Pair with tna-legislation: legislation.gov.uk uses the same FRBR model and includes CELLAR links on EU-origin instruments. Free SPARQL 1.1, no auth.

Query the Scottish Government statistics SPARQL endpoint at statistics.gov.scot — the official Scottish counterpart to ONS data, published as RDF DataCube vocabulary. Use when the question is about Scotland-specific statistics that join cleanly to the sp (Scottish Parliament) facility: economic indicators, demographics, public-sector finance, housing, transport, environment per Scottish council / SPC / Intermediate Zone. Free SPARQL 1.1, no auth. Endpoint sometimes 503s under load; retry with backoff.

Query the Environment Agency Real-Time Flood Monitoring API — active flood warnings and alerts, ~5,000 monitoring stations measuring river levels, flows, rainfall, tide and groundwater, and the historic archive. Free, no auth. Use when the question is about a current flood warning in a particular area (constituency / council / county), a river-level reading, rainfall recordings, or flood-area polygons. Coverage: England only (Scotland uses SEPA, Wales uses NRW, NI uses DfI Rivers). Pairs with mapit / ons-geo to ask 'is this constituency at flood risk right now'.

Query the Food Standards Agency Food Hygiene Rating Scheme API — every food business in England, Wales and Northern Ireland scored 0-5 for hygiene by their council (the Scotland equivalent FHIS uses Pass / Improvement Required). ~660,000 establishments, each with the inspecting council, inspection date, and hygiene + structural + management-confidence sub-scores. Use when the question is about food hygiene ratings for a postcode, area, business, or council, when comparing inspection performance across local authorities, or when food-safety debates in Parliament reference specific outbreaks or rating distributions. Free, no auth, but the API requires the header `x-api-version: 2` (library injects it automatically).

Query the gov.uk Content API and Search API — every page on www.gov.uk as structured JSON, plus full-text + facet search across them. Use when you want the structured form of a gov.uk page (e.g. /rubbish-collection-day, /check-uk-vat-number, /foreign-travel-advice/france), when you want to enumerate every lookup-shaped page of a given document_type (local_transaction, smart_answer, place, transaction, licence_transaction, finder, hmrc_contact, …), or when you need bank-holiday dates as JSON. Sister to gov-data which wraps the CKAN dataset catalogue at data.gov.uk; this wraps the *content* store. Free, no auth, OGL v3.0.

Query the data.gov.uk CKAN catalogue — the UK government's central index of ~58,000 published datasets aggregated from every central department, most local authorities, and statutory bodies. Wraps CKAN's full-text search, dataset detail, publishing organisations, tags, and topic groups. Use when the question is about whether a department has published structured data on a topic, who publishes a given dataset, or what data is newly available. The catalogue is OGL v3.0; individual dataset licences vary and should be checked per-record.

Query the Scottish Parliament Open Data API at data.parliament.scot/api — Members of the Scottish Parliament (MSPs, current and historical), constituencies, regions, parties, committees, and committee membership. JSON endpoints with PascalCase resource names; the API REQUIRES an Accept: application/json header (default content negotiation returns 405). First-party to the Scottish Parliament; tier-3 from a Westminster perspective. Use when the question is about an MSP, a Holyrood committee, or Scottish electoral geography (constituencies / regions for the Additional Member System).

Query the Information Commissioner's Office — the UK's independent data-protection and FOI / EIR regulator. Wraps the HTML index of 'Action we've taken' (enforcement notices, Monetary Penalty Notices, decision notices, reprimands, undertakings, audits) and a generic page fetcher. No JSON API; ICO publishes regulatory action only as HTML. Use when the question is about ICO enforcement against a government department or other body, an FOI decision notice on a public authority, or the regulatory consequences of a data breach. Pairs with `bills` (Data Protection Bills) and `committees` (DSIT / Public Administration committee scrutiny).

Query the National Audit Office — Parliament's external auditor of UK central government. Wraps the NAO WordPress REST API and reports RSS feed for published value-for-money reports, financial audits, investigations, and good-practice guides. Use when the question is about an NAO report, audit findings on a department or programme, or what reports the Public Accounts Committee is likely to take evidence on next. NAO reports feed PAC; pair with the `committees` facility for the resulting parliamentary scrutiny.

Query the Office for Budget Responsibility — the UK's independent fiscal watchdog. Wraps OBR's RSS feeds (all publications + per-topic) and HTML page fetcher for Economic and Fiscal Outlooks (EFO), Fiscal Risks and Sustainability reports (FSR), Welfare Trends, Forecast Evaluation Reports (FER), and policy costings of Budget / Spring Statement measures. Use when the question is about OBR forecasts, fiscal headroom, or scrutiny of HM Treasury policy. Treasury Committee material; pair with the `committees` facility for the resulting parliamentary scrutiny.

Query the Office for Statistics Regulation — the independent regulatory arm of the UK Statistics Authority that polices the production and use of official statistics. Wraps OSR's RSS feed and HTML page fetcher for formal censures, case studies, review reports, and guidance. Use when the question is about whether a Minister or MP has been censured for misusing statistics, accreditation status of National Statistics, or compliance casework. OSR is the body that publicly rebukes politicians for citing dodgy numbers; pair with `hansard` for what was said and `interests` / `members` for who said it.

Search and fetch records from TNA's Discovery catalogue — ~37 million record descriptions covering The National Archives' own holdings plus ~2,500 partner archives across the UK. Use when the question is about a specific archival file (treaty ratifications in FO 94, Cabinet papers in CAB, Foreign Office political correspondence in FO 371, Home Office records in HO, Prime Minister's Office in PREM), about a record series, about where a record is held, or about hierarchical browsing (department → series → piece). Pairs with fcdo-treaties / tna-legislation for archival depth behind contemporary citations.

Query DemocracyClub Candidates — the canonical list of UK electoral candidates (past, present, declared) with stable IDs, party affiliations, and sources. Use when the question is who stood (or is standing) for a seat at a given election, what the candidate's history is across seats, or to enumerate every candidate at a given general election. Joins to Parliament via Westminster constituency name + general-election date. Free JSON API; no auth.

Query DemocracyClub EveryElection — the canonical list of every UK election (Westminster, devolved, mayoral, local, PCCs, parishes) with stable election IDs and the organisations / divisions they're held in. Use when the question is about which elections are scheduled or have happened on a date, or to look up the canonical id (e.g. `parl.2024-07-04`) for joining to other DC datasets. Pairs with dc-candidates.

Query the Electoral Commission's open search API for donations, campaign spending returns, loans to political parties, registered campaigners (referendum / election), and the Registers of political parties / non-party campaigners / accounting units. Use when the question is about who donated to which party, how much was spent on a campaign, or who's registered to spend on a referendum. Joins to the Parliament Register of Members' Financial Interests indirectly via the recipient name (party / MP / candidate).

Query TheyWorkForYou (mySociety) — UK Parliament debates, written answers, written ministerial statements, MP / peer profiles, voting summaries, and user comments. Useful when you want MP-level data with mySociety's analyses on top (rebellion detection, attendance, contribution patterns) that Parliament's own APIs don't publish, or when the question is about a specific debate / written answer / WMS by date or by speaker. Requires a free API key from theyworkforyou.com/api/key — pass --api-key or set TWFY_API_KEY.

Find Case Law — The National Archives' open service for UK court and tribunal judgments at caselaw.nationalarchives.gov.uk. Use when the question is about an approved judgment from the Supreme Court, Court of Appeal (Civil or Criminal), High Court (King's Bench / Chancery / Family / Administrative / Commercial / TCC), Upper Tribunal, Employment Appeal Tribunal, or a growing set of First-tier Tribunals. Distributed as LegalDocML (Akoma Ntoso) XML with Atom feeds and a public search. Pair with tna-legislation — judgments cite legislation; legislation Acts are interpreted by judgments.

Query Wikidata's SPARQL endpoint at query.wikidata.org/sparql, plus a quick `search` shortcut against the Wikidata web API for label → QID lookups, and an `entity` shortcut for the full Linked-Data JSON of a QID. Use when you need to cross-reference Parliament records (people, parties, constituencies) to identifiers in other ecosystems, when joining two datasets via Wikidata QIDs as a common bridge, or for free-form SPARQL over the world's largest open knowledge graph. CC0; the only ask is a polite User-Agent.

Query Nomis — ONS's free Census 2021 + labour-market data service — at constituency / LAD / MSOA / LSOA / OA granularity. Use when the question is about the population, employment, claimant count, occupation, qualifications, ethnicity, age, health, or any other Census 2021 / labour-market table for a specific Westminster constituency, council area, or ward, or for by-constituency comparisons across the UK. Coverage spans ~1,600 datasets including the full Census 2021 (TS-series) and the ONS labour-market series (claimant count, BRES, ASHE, APS). Anonymous SDMX-JSON; no auth.