sap-testing

name	sap-testing
description	Entry point for the SAP UI test-automation environment. Explains project layout, the configured test folder, available tools, phase workflows, and bounded agent delegation. Use when the user asks about SAP UI testing, Playwright for SAP, writing tests for any ABAP report/transaction/process, generating test evidence docs, or mentions a SAP Testing folder.

SAP Testing — Overview

Non-negotiable execution gate

Every required phase step and artifact is a prerequisite for execution. The abapfs_run_playwright_tests tool verifies the full workflow state and will reject the run if any required item was skipped, deferred, left missing, stale, or unverified. Complete every requirement at the point specified by the relevant phase workflow; you cannot defer it and repair the omission at run time.

Why

This is production SAP for a real business. Every bug that ships costs money — wrong invoice, wrong stock movement, wrong customer charge, blown financial close. Test suites here are the last gate before code touches accounts, orders, or inventory. The goal is NOT to finish; the goal is to catch every bug before it ships. Cutting corners because the code is long, the sandbox is thin, or the task is tedious is exactly how a million-dollar defect reaches production. Speed is worthless if you miss the bug.

The available SAP Testing tools drive Playwright against SAP WebGUI. Specs use the bundled SapSession/SapArtifacts runtime through @sap-testing/runtime, and runs produce .docx evidence. There is no terminal, no npm, no .env — every action below is either a tool call or a normal file edit inside the configured test folder.

Where things live — two different places, on purpose

Lives in	Contains	Editable by you?
The test folder (abapfs.testing.folder setting — ask the user to run "ABAP FS: Enable SAP UI Testing Features" if unset)	tests/<PROGRAM>/test-cases/*.md, test-scripts/*.spec.ts, test-results/**, one managed tsconfig.json	Yes — this is a normal, git-trackable project folder. Read/write it with your normal file tools.
Bundled SAP Testing runtime (available through @sap-testing/runtime and the tools below; not a workspace folder)	SapSession, SapArtifacts, resolveTestData, buildFixture, and the 10 tools below	No — its implementation is not available to file tools. See helpers-reference when a capability is missing.

Everything for one program lives together under tests/<PROGRAM>/ inside the test folder:

<test folder>/
├── tsconfig.json                (managed automatically by SAP Testing — never hand-edit)
├── recordings/                  User-assisted Playwright reference recordings; never runnable specs
└── tests/
    └── <PROGRAM_NAME>/          One folder per ABAP object under test
        ├── test-cases/          Phases 1–4 output — _flow.md + _units.md + _findings.md (P1) + _screens.md (P2) + TC-XXX.md + _index.md + _index.docx (P3) + TC-XXX.data.md (P4)
        ├── test-scripts/        Phase 6 output — TC-XXX.spec.ts (real @playwright/test specs)
        └── test-results/        Phase 5 (data.json) + Phase 7 (evidence) output — segregated per connectionId
            ├── <CONNECTION-ID>/              (e.g. DEV-100)
            │   └── <TC-ID>/                 per TC: screenshots, manifest.json, data.json, fixtures/
            └── <PROGRAM>-<CONNECTION-ID>-report.docx   ONE aggregated docx per (program, connectionId)

Rules:

• All artifacts for a program are siblings under tests/<PROGRAM>/. Never scatter TC-001.md, TC-001.spec.ts, and its screenshots.
• Raw user recordings live under recordings/*.recording.ts, outside tests/. They are temporary reference evidence, not phase outputs or runnable specs.
• Program folder name = the name the user asked for (report / tcode / class), uppercase, no spaces. When the user names a TRANSACTION, keep the folder as the tcode — do NOT rename it to the program the tcode runs. Phase 1 resolves the tcode to its executable object and records BOTH in _flow.md frontmatter (target: = the tcode/name, resolvedProgram: = the report/class actually analysed), so the mapping is explicit and no later phase re-derives it.
• The per-connection results folder is the connectionId in UPPERCASE (test-results/<CONNECTION-ID>/, e.g. DEV-100). The framework derives this folder by uppercasing the connectionId at run time, so any hand-written cache/results path MUST use the uppercase form — a lowercase folder is silently not found on a case-sensitive filesystem (Linux/macOS).
• Every .spec.ts you write imports from the fixed specifier @sap-testing/runtime — never a relative path (there is no helpers/ folder in the test folder to point at). See build-scripts and helpers-reference for the full import convention and method reference.

Skills, tools, and agents are three different things — know which is which

Backticked names in these docs refer to one of three kinds of thing, and you interact with each differently. When a doc names one, it should say the kind ("the X skill/tool/agent"); if it doesn't, use this list to tell them apart:

• Skills (you LOAD/READ them — they are operating procedures): analyze-and-plan, explore-ui, design-cases, define-data, prepare-data, build-scripts, run-scripts, sap-webgui, sap-webgui-recording, helpers-reference, anst-guide. "follow X" / "load X" always means a skill.
• Tools (you CALL them — they take inputs and return a result): the 10 SAP Testing tools listed below (abapfs_get_test_folder, abapfs_get_sap_webgui_url, abapfs_run_playwright_tests, abapfs_build_test_index, abapfs_build_test_index_docx, abapfs_split_test_cases, abapfs_verify_test_data_usage, abapfs_check_test_data, abapfs_build_evidence_report, abapfs_analyze_anst_enhancements) plus the ABAP research tools (abapfs_search_objects, abapfs_get_object_source, abapfs_run_sql_query, abapfs_get_sql_syntax, abapfs_get_connected_systems, abapfs_download_object, …).
• Agents (you DELEGATE to them — they run as separate subagents with their own context): sap-source-download, sap-code-grep, sap-enhancement-research, sap-findings-reviewer, sap-screens-reviewer, sap-test-plan-reviewer, sap-data-scout, sap-task-helper, anst-enhancement-analyser.

If a tool you need isn't in your active toolset, search for it by name before giving up (see "Tools can be hidden" below).

The 10 available SAP Testing tools

Tool	Use for
abapfs_get_sap_webgui_url	Given an abapfs connectionId and optionally a transaction, returns a ready-to-open SAP WebGUI URL — already signed in when the connection allows auto-login. Call this whenever you need to open a browser for live exploration. Open the URL exactly as returned; an auto-login URL is single-use, so never append ~transaction to it yourself.
abapfs_get_test_folder	Get the configured test folder path. Call this first before reading or writing any test artifact. If the folder is not open in the workspace, tell the user to open it (File > Add Folder to Workspace) before proceeding.
abapfs_analyze_anst_enhancements	Classify an ANST Customer Code xlsx and write its Markdown work list beside the workbook; no Python or openpyxl required.
abapfs_run_playwright_tests	Run exact tcIds (array) or every spec in one program. Replaces terminal npx playwright test. Use runInParallel only for cases confirmed independent from their TC/data/automation notes; batch dependency chains sequentially. Up to maxTasks workers after one login (default 3, cap 5). maxFailures defaults 3, caps 10. GATED by readiness and SE16N proof. NEVER use terminal Playwright or generic runTests.
abapfs_build_test_index	Validate cases and rebuild _index.md plus the printable, bordered _index.docx; requires the analyze-and-plan sourceSnapshot, records analyzedOn automatically, and is GATED on a reviewerConfirmation certifying sap-test-plan-reviewer returned PASS. A missing .data.md for a dataRequired: yes case is only a warning (authored later in Phase 4).
abapfs_build_test_index_docx	Refresh _index.docx from the current _index.md, including preserved Notes, without rebuilding or validating the Markdown index.
abapfs_split_test_cases	Validate and split a aggregate into one per tagged case. Does not run .

Everything else — reading ABAP source, live SAP UI exploration, writing markdown/TypeScript files — uses tools you already have (your built-in file tools, and your built-in browser tool for live exploration; see analyze-and-plan for source reading and explore-ui for live exploration).

The 7 phases (one artifact each, in dependency order)

Test creation is deliberately split into seven standalone phases so no single step is rushed. Each phase produces ONE kind of artifact and hands off to the next:

#	Phase skill	Produces
1	analyze-and-plan	_flow.md + _units.md + _findings.md — the full code picture
2	explore-ui	_screens.md — the live web-UI control map
3	design-cases	TC-XXX.md + _index.md/_index.docx
4	define-data	TC-XXX.data.md — data requirement specs
5	prepare-data	test-results/<conn>/TC-XXX/data.json
6	build-scripts	test-scripts/TC-XXX.spec.ts
7	run-scripts	test-results/** evidence + report docx

Phases → skills

User intent	Load this skill
"Analyze this report", "Read the code", "What does this program do"	analyze-and-plan (Phase 1)
"Explore the UI", "Map the screens", "Produce _screens.md"	explore-ui (Phase 2)
"Design test cases", "What should we test", "Write the test plan"	design-cases (Phase 3)
"Define the data specs", "What data does each case need"	define-data (Phase 4)
"Figure out data for TC-005", "How do I get a valid material for this test"	prepare-data (Phase 5)
"Write the Playwright script for TC-005", "Convert TC-005 to code"	build-scripts (Phase 6)
"Run TC-003", "Generate evidence doc", "Why did the test fail"	run-scripts (Phase 7)
"Find all enhancements for ", "What user exits run in ", "ANST"	anst-guide (guides user to collect xlsx) → then delegate to anst-enhancement-analyser agent
"How does setField work", "What do I do if a helper is missing"	helpers-reference
"This process crosses systems", "Stage 2 uses another connection", "Pass a PO to another system"	multi-system-workflows alongside the active phase skill
"How should WebGUI controls be identified", "Are these locators stable"	sap-webgui
"Record this WebGUI flow", "I cannot explore this complex SAP control"	sap-webgui-recording

These are separate skills, not sub-agents — they activate automatically when relevant. You (the active Copilot agent) do the work directly, following whichever skill applies. Delegate only specific, bounded jobs to dedicated agents instead of doing them yourself — each is deliberately small and narrowly scoped:

• sap-code-grep — mechanical grep/count over the program's own source (message counts, branch counts, AUTHORITY-CHECK, flow-control exits), using an ACTUAL Grep over the snapshot (real line numbers, no eyeballing). See analyze-and-plan Step 5.2.
• sap-source-download — synchronously discovers recursive includes and downloads one complete source snapshot. See analyze-and-plan Step 1.
• sap-enhancement-research — finds and thoroughly explains customer enhancements on the standard call surface. See analyze-and-plan Step 5.2; launch it and sap-code-grep together in one parallel batch.
• sap-findings-reviewer — the adversarial Phase 1 gate: reads the source itself and re-checks _findings.md/_flow.md/_units.md for fabricated line numbers, missed MESSAGE/branch/auth statements, un-analysed value-transformation/default logic, wrong date/number formats, and an under-counted target minimum. See analyze-and-plan Step 6.1. Phase 1 does not hand off until it returns PASS.
• sap-screens-reviewer — the adversarial Phase 2 gate: statically checks _screens.md describes the live web-GUI (accessible names, initial states, dialogs, ALV) and is NOT an ABAP selection-screen/source description (no ABAP names, MODIF IDs, source snippets, message classes). Catches a _screens.md derived from source instead of observed. See explore-ui Step 9.1. Phase 2 does not hand off until it returns PASS.
• anst-enhancement-analyser — consumes an ANST xlsx after anst-guide, calls abapfs_analyze_anst_enhancements, and researches every classified object.
• sap-test-plan-reviewer — the adversarial check that your test-case count and category coverage actually meet the minimum _findings.md derives, before you build the index. See design-cases Step 3. abapfs_build_test_index will not run until this agent has returned PASS.
• sap-data-scout — resolves ad-hoc data requirements from a live SAP system (find 5 articles listed at both sites, find open POs in company code 1000, etc.). Invoke in parallel, one instance per distinct requirement. Handles + + spot-validation internally. See Step 2.

Use these dedicated agents for their bounded tasks so large enumerations remain complete and verifiable instead of being shortened when context grows.

Every subagent is ephemeral and one-shot. A delegated agent runs with its own fresh context, cannot see your conversation, cannot ask you a follow-up, and returns exactly ONE response — there is no back-and-forth. Two consequences: (1) give it everything it needs UP FRONT (all inputs its contract lists); a missing input means it must reject and you re-invoke, costing a full round. (2) Trust its single response to be complete — a good gate agent reports every gap at once, and a good worker reports every value plus any deviation it made, precisely because it knows it gets no second message. If an agent's rejection is vague ("input invalid"), that's an agent bug to raise with the user, because the ephemeral contract requires it to name the exact missing/invalid thing and the fix.

Delegation discipline — pass inputs, not methods. When you delegate to a subagent, pass ONLY the inputs its contract specifies (IDs, paths, the objective, e.g. for sap-source-download: <TEST_FOLDER>, <PROGRAM>, connectionId, object name, object type). Do NOT tell it HOW to do its job — which tools to use, to read the source with abapfs_get_object_source, to "create the files," to skip a step. Every subagent already knows its procedure and has its own guardrails, and over-instructing can OVERRIDE those guardrails and make it do the wrong thing: telling sap-source-download to "read the code and write the files" makes it fabricate the compliance snapshot by hand instead of downloading it with abapfs_download_object — the exact opposite of its purpose. Give it the objective and the inputs; let it choose the method. If you believe a subagent's method is wrong, that's a skill bug to raise with the user — never something to "fix" by adding how-to at the call site.

One phase per chat is supported

Each phase skill is a standalone operating procedure. A fresh chat must be able to continue from artifacts on disk without the prior chat's context. Starting a new chat for each phase is therefore valid and often preferable:

1. analyze-and-plan reads the source and writes _findings.md.
2. explore-ui explores the live UI and writes _screens.md.
3. design-cases writes the reviewed TC-XXX.md files and the index.
4. define-data writes each TC-XXX.data.md requirement spec.
5. prepare-data resolves those specs into per-system data.json caches.
6. build-scripts converts approved cases to TC-XXX.spec.ts.
7. run-scripts validates prerequisites, executes, and produces evidence.

The user should only need to name the phase plus the program or TC-ID. Do not ask them to repeat facts already recorded in _findings.md, _index.md, _screens.md, TC frontmatter, or per-system result folders. Do not rely on conversation memory either: disk artifacts are the handoff contract.

The phases are mostly linear, with ONE expected loop. A seeded data requirement (Phase 4) points at a spec that only exists after Phase 6, so it cannot be resolved on the first Phase 5 pass — Phase 5 marks it deferred-until-phase-6, Phase 6 writes the seeding spec, then Phase 5 runs a SECOND time to resolve just the deferred seeded keys before Phase 7. So the order for programs with seeded preconditions is 4 → 5 → 6 → 5 → 7. This is expected, not rework; each phase's handoff names any deferred seeded key so the next chat knows a second prepare pass is owed.

Tools can be hidden. The editor may not surface every SAP Testing tool until it is searched for, and smaller models often fail to find them and improvise instead. At the start of any phase, if a tool that phase needs (always abapfs_get_test_folder; also abapfs_run_playwright_tests for Phase 7, abapfs_build_test_index/abapfs_split_test_cases for Phase 3–4, etc.) is not available, SEARCH your available tools for it by name before proceeding. Never substitute a terminal command or a fabricated result for a tool you couldn't find — say which tool is missing.

Universal rules

1. ALWAYS call abapfs_get_test_folder before any other SAP Testing action. Treat its returned absolute path as <TEST_FOLDER> for the whole chat. Never infer it from the workspace, current directory, a prior chat, or a path mentioned in an old artifact. If unset, STOP and ask the user to run "ABAP FS: Enable SAP UI Testing Features". If it is not open in the workspace, ask the user to add it before using normal file tools.
2. Then establish the target system. Call abapfs_get_connected_systems and pick the target connectionId. If ambiguous, ASK. The connectionId determines the WebGUI URL, ABAP tool calls, and test-results/<connectionId>/.
3. Rediscover context from disk in every chat. Confirm the program and TC-ID from the user's request and artifacts under <TEST_FOLDER>/tests/. If the request is ambiguous and multiple candidates exist, ask; never silently pick one or depend on prior-chat memory.
4. Every phase enforces its input gate before work. Missing or inconsistent upstream artifacts are blockers. Stop the current workflow and follow analyze-and-plan for test-plan problems, prepare-data for data problems, build-scripts for spec problems, or run-scripts for execution problems. Never create plausible replacements from memory.
5. No login logic in a spec, ever. abapfs_run_playwright_tests signs the browser in itself, before any spec runs, using a SAP reentrance ticket minted from the ABAP FS connection — so a spec never needs to. Never write credentials, a logon step, or an auth workaround into a .spec.ts, .md, or .data.md. If a run still lands on a logon screen, that is a real failure to report (see run-scripts), not something to work around.
6. Never bypass the runtime. All specs use SapSession/SapArtifacts from @sap-testing/runtime. A missing capability is a real limitation — see helpers-reference for what to do (you cannot add a helper method yourself).
7. No CSS-class selectors, no ref numbers, no positional guessing. Only role + accessible name, scoped to a container — enforced inside the runtime; keep it in mind for any sap.raw() escape-hatch code.
8. English only. SAP language must be en in every configured system URL.