umbraco-reference-instance

name

umbraco-reference-instance

description

Boot the committed reference Umbraco site and prove that a skill's produced code actually runs. Use this whenever you want to run/validate an implementation or content-modelling skill against a real, running Umbraco instance — e.g. "test the sitemap skill in Umbraco", "does this skill's code actually work", "boot the reference site", "run the instance", "validate <skill> end-to-end". This is the runtime counterpart to `umbraco-skill-evaluator`: the evaluator grades whether Claude *writes* the right code; this skill proves the code *builds and serves* in Umbraco. Authoring/maintainer tooling — not shipped in any plugin.

Umbraco reference instance

Umbraco-CMS.Skills/ (at the repo root, with Umbraco-CMS.Skills.sln) is a real Umbraco 17 web project used only for validating skills. It:

• targets net10.0, Umbraco.Cms 17.5.3 (scaffolded via the psw CLI; versions live in Umbraco-CMS.Skills/Directory.Packages.props) — matching the skills' "Umbraco 17+" target;
• installs unattended on first boot into a SQLite DB ([e-mail] / 1234567890);
• ships the Clean starter kit, so there is real content (Document Types, templates, published pages) for skill output to run against — /sitemap.xml has URLs to emit, a missing page has a site to render a 404 into, the Delivery API returns real nodes.

The runtime DB and build output are .gitignored — only the project scaffolding is committed, and Clean re-installs on first boot. Never commit a populated App_Data SQLite DB, bin/, obj/, the Umbraco.Skills.Sandbox/ scratch project, or .local-nuget-feed/.

Environment conventions

Same variable names as the backoffice test-runner, so tooling is interchangeable:

Variable	Default	Meaning
UMBRACO_URL	https://localhost:44372	Base URL the instance binds to (also http://localhost:60372).
UMBRACO_USER_LOGIN	[e-mail]	Unattended admin email.
UMBRACO_USER_PASSWORD	1234567890	Unattended admin password.

The scripts read these from the environment and fall back to the defaults above.

Just boot the site

.claude/skills/umbraco-reference-instance/scripts/instance.sh boot    # start + wait until ready
.claude/skills/umbraco-reference-instance/scripts/instance.sh status  # is it up?
.claude/skills/umbraco-reference-instance/scripts/instance.sh stop     # stop only if this script started it

boot is idempotent: if the instance is already answering on UMBRACO_URL it reuses it and does not start a second one. First boot is slow (unattended install + Clean import) — the script polls for up to ~4 minutes.

Log in to the backoffice at <UMBRACO_URL>/umbraco with the credentials above. For clicking through the backoffice, use the umbraco-chrome-navigation skill.

Find out what content the instance actually has

Do this before writing an example or its assertions. Skill assets routinely navigate by Document Type alias (FirstChildOfType("errorPage"), "first child of the site root"), and such code only proves something if the alias it looks for exists in a shape it can reach — guessing produces either a mysteriously null lookup or, worse, a test that passes for the wrong reason.

Boot the instance and inspect it with the Umbraco Developer MCP's CLI, which reads the content tree and Document Types over the versioned Management API. Use --call, not the MCP tools through a chat session: same data, but a command you can re-run and paste into a commit message. It needs the API user from above; credentials come from .mcp.json.

MCP="npx @umbraco-cms/mcp-dev@lts-17 --umbraco-base-url https://localhost:44372 --umbraco-readonly"
export NODE_TLS_REJECT_UNAUTHORIZED=0 UMBRACO_CLIENT_ID=umbraco-back-office-mcp UMBRACO_CLIENT_SECRET=1234567890

$MCP --list-tools                                             # what's available
$MCP --call get-document-root      --call-args '{}'            # site root + published state
$MCP --call get-document-children  --call-args '{"parentId":"<root-id>"}'
$MCP --call get-document-type-by-id --call-args '{"id":"<type-id>"}'   # ...and its ALIAS

--umbraco-readonly keeps an inspection from mutating the instance. Note the last call is not optional: the API identifies types by key and display name, not alias, so the document and document-type listings give you a GUID and a name but never the alias your C# navigates by. Don't infer it from the name — Clean's "XML Sitemap" type has the alias xMLSitemap.

For a quick look with no auth at all, the Delivery API answers the same question for published content and needs no API user:

curl -sk "https://localhost:44372/umbraco/delivery/api/v2/content?fetch=children:/&take=100"

That returns each child's contentType as its alias directly, which is why the precondition tests use it (see below). None of this is needed to run the dotnet test gate.

The MCP server is configured in .mcp.json at the repo root, pinned to the v17 line (@umbraco-cms/mcp-dev@lts-17) to match the instance. It authenticates as an API user, which the instance doesn't have until you create one:

.claude/skills/umbraco-reference-instance/scripts/instance.sh boot
.claude/skills/umbraco-reference-instance/scripts/instance.sh api-user

api-user wraps scripts/create-api-user.mjs: it logs in as the unattended admin, gets a token via the Swagger OAuth client, and creates an API user holding the client credentials .mcp.json expects. Idempotent — re-running it when the credentials already authenticate just exits. The script is vendored from the MCP repo's v17 branch because it ships in git only, not in the npm package; its header explains why the branch matters. The credentials are fixed local dev values for a throwaway instance, so never point this config at a real site.

Both the MCP and this script talk HTTPS to the ASP.NET dev certificate, which Node won't trust — hence NODE_TLS_REJECT_UNAUTHORIZED=0 in .mcp.json, and the same exemption inside api-user (scoped to localhost only). Claude Code reads .mcp.json at startup and asks you to approve the server, so a new session is needed before the Umbraco tools appear.

Don't reach into the SQLite database for this. It works, but it couples to Umbraco's internal schema (nodeObjectType GUIDs, cmsDocumentType, the umbracoContent/cmsContentType join) — private, unversioned, and liable to change between majors, failing confusingly rather than loudly when it does.

Validate a skill deterministically (the CI gate)

Runtime validation is a dotnet test gate — no LLM, reproducible pass/fail. Each validated approach ships an examples/<approach>/ project; the skill's assets/ are projected into it at build time with placeholders substituted, a reference host references it, and that host's test project boots in-process (WebApplicationFactory) and asserts the skill's endpoints over HTTP.

There are two hosts — Umbraco-CMS.Skills (Clean) for approaches written as C# that registers into DI, and Umbraco-CMS.Skills.Blank (no starter kit) for approaches written as Document Types + templates + config. They are tested as separate processes, because Umbraco's StaticServiceProvider is process-wide static state.

dotnet build Umbraco-CMS.Skills.sln
dotnet test Umbraco-CMS.Skills.TestHost/Umbraco-CMS.Skills.TestHost.csproj --no-build
dotnet test Umbraco-CMS.Skills.TestHost.Blank/Umbraco-CMS.Skills.TestHost.Blank.csproj --no-build
python3 scripts/generate-examples.py --lint  # every placeholder an asset carries is declared

Adding a skill to the gate is an authoring task, so it's documented where authors work: umbraco-skill-author's runtime-validation reference covers the examples/<approach>/ layout, how .generate.json substitutes placeholders and declares content preconditions, how to write a fixture against the shared host, and how to prove the fixture can actually fail. It is the single source of truth for those mechanics — this file covers running and debugging the instance instead.

assets/ stay the single source of truth, and nothing generated is committed: each example is projected into its own obj/ during the build, so what compiles and serves IS what the skill ships and there is no second copy to drift. Skills whose assets/ aren't on the current branch are skipped, so a build is safe pre-merge.

When a skill's code doesn't compile

Expect this. Umbraco's published-content API changed substantially across v13 → v15 → v17, and skills written from older docs or tutorials carry APIs that no longer exist. Catching that is the main thing this gate buys, so treat a red build as the gate working, not as a problem with the example. Two failure modes cost real time if you don't know them:

Fix declaration errors before believing the error count. Roslyn resolves declarations first and won't report method-body diagnostics while any remain, so an unresolved type (CS0246 on a missing using) hides every bad call in the file. umbraco-custom-error-pages reported 2 errors; adding the missing usings turned that into 8, and only then did the actually-interesting problems appear. If a line you're sure is wrong reports nothing, you have an earlier error to clear — don't conclude the call is fine.

Check API shape against the version you're compiling, not the source you have open. The Umbraco-CMS working directory (see CLAUDE.md) is on whatever branch it happens to be on, which may be a different major than Directory.Packages.props pins — confirm with git -C <umbraco-src> rev-parse --abbrev-ref HEAD. Then let the compiler answer rather than grep: add a temporary probe file that calls the API with deliberately wrong arguments, and read which overload it binds to. That is how the FirstChild(alias) bug was pinned down — v17 has no alias overload, so the alias was silently binding to the culture parameter and matching the first child of any type. Anything that compiles but binds a string to the wrong parameter is invisible to a compile check and will only be caught by an assertion on real content, which is why the gate asserts behaviour and not just build success.

Where an asset genuinely targets an older floor than the instance (e.g. a skill claiming 16.1+ while the instance is 17), the gate can only prove the 17 end. Compiling the assets in a throwaway project pinned to the floor version (Umbraco.Cms.Web.Website 16.1.*, net9.0) checks the claim, but compile-only — it proves the APIs bind, not that behaviour matches.

Explore interactively (manual boot)

For poking at a skill by hand, or for the parts a dotnet test can't cover (backoffice setup — e.g. sitemap Approach B's Document Type + content node, driven via umbraco-chrome-navigation), use the manual harness:

scripts/instance.sh try plugins/implementation/skills/umbraco-sitemap  # materialize assets → sidecar → reference
scripts/instance.sh boot
curl -sk https://localhost:44372/sitemap.xml
scripts/instance.sh reset                                              # restore the committed instance

try copies every assets/*.cs into a sidecar library (namespace-substituted) — so for mutually-exclusive assets, point it at a pruned copy. This path is for exploration; the dotnet test gate above is the source of truth for whether a skill works.

Relationship to other skills

• umbraco-skill-evaluator — grades whether Claude produces correct skill output (LLM grading). Run it for quality/regression scoring; run this skill to prove the output compiles and serves.
• umbraco-chrome-navigation (backoffice plugin) — drives the running backoffice for the browser half of validation.