Create a Kobiton test run from a test case or suite, then offer to monitor it. When the user gives only partial details (or just a test case id), fill the rest with sensible defaults that match the createTestRun schema, show a summary of what will run, and ask to proceed or customize before creating. After the run is created, offer monitoring in a single prompt — monitor + auto-open live remediation (only when the org's live-remediation flag is ON), monitor only, or don't monitor — and hand off to the monitor-test-run skill if chosen. Use when the user asks to "create / kick off / start / run a test run", "run test case X on N devices", or similar. Wraps the createTestRun MCP tool (and getOrgSettings / listDevices for defaults); delegates the watch to monitor-test-run.
Instrucciones de origen · Vista previa de solo lectura
name
create-test-run
description
Create a Kobiton test run from a test case or suite, then offer to monitor it. When the user gives only partial details (or just a test case id), fill the rest with sensible defaults that match the createTestRun schema, show a summary of what will run, and ask to proceed or customize before creating. After the run is created, offer monitoring in a single prompt — monitor + auto-open live remediation (only when the org's live-remediation flag is ON), monitor only, or don't monitor — and hand off to the monitor-test-run skill if chosen. Use when the user asks to "create / kick off / start / run a test run", "run test case X on N devices", or similar. Wraps the createTestRun MCP tool (and getOrgSettings / listDevices for defaults); delegates the watch to monitor-test-run.
allowed-tools
Read, Skill
version
1.0.0
author
Kobiton Inc.
license
MIT
compatibility
Runs on any MCP-aware host, including hosts with no local filesystem — this is the plugin's only pure-MCP skill, needing no local file, binary, or shell. Uses the Kobiton MCP tools createTestRun, getTestCase/getTestSuite, listDevices, and getOrgSettings; requires an authenticated Kobiton MCP connection. Delegates monitoring to the monitor-test-run skill (same plugin), which does need a local poller — so on a filesystem-less host, create the run and report its id rather than offering to watch it.
Needs only an authenticated Kobiton MCP connection — no local filesystem, no credentials file, no
binary, no shell. It is the plugin's only pure-MCP skill, so it is the only one that works where the
host supplies nothing else. The monitoring hand-off in Step 5 is the exception: monitor-test-run
runs a local poller, so where you can't run a local command at all, create the run and report its id
instead of offering to watch it (Step 5 carries the wording). Where you can, hand off and let that
skill check its own credentials and streaming options — it degrades rather than refusing. See the Skill
compatibility matrix in CLAUDE.md.
Overview
Turn a "run this" request into a created test run with as little friction as the
user wants, then offer to watch it. Two phases:
Build + confirm the run. Resolve what to run (test case or suite), on which
devices, with what app — filling any unspecified field with a documented
default — then show a one-screen summary and create on confirmation.
Offer monitoring once. After creation, present the monitor choice in a
single prompt and delegate to monitor-test-run if the user wants it.
Tool naming. Kobiton MCP tools are referenced by bare name (createTestRun,
getOrgSettings, listDevices, getTestCase, getTestSuite). The host
resolves the prefix (mcp__plugin_automate_kobiton__createTestRun,
mcp__kobiton__createTestRun, etc.).
Inputs
Input
Required
Notes
test case id or test suite id
one of
A test case id → TEST_CASE selection; a suite id → TEST_SUITE. If the user named neither, ask for it (it's the one thing with no sensible default).
device count / specific devices
no
Default: 1 device matching the test case/suite platform. The user may say "3 devices", name models, or give UDIDs.
run name / app version / allocation
no
All defaulted (see Step 2).
Steps
1. Resolve the target and its platform
A test case id → fetch it () to learn its platform and the app under test. Selection
will be with (default to the latest version
if the user didn't pin one).
getTestCase
TEST_CASE
testCaseSelections: [{ testCaseId, version }]
A test suite id → fetch it (getTestSuite) for platform + member test cases. Selection will be
TEST_SUITE with testSuiteId.
If the user gave neither a case nor a suite id, ask for one — there's no sensible default for what to
run. Everything else can be defaulted.
2. Fill defaults for anything unspecified
Apply these defaults so a bare request ("run test case X") becomes a complete, valid createTestRun
payload. Use the exact enum values below — they are upper-case and case-sensitive; the API rejects
lower-case variants (test_case, specific_devices, …).
Field
Default
Notes
testSelection.type
TEST_CASE (case) / TEST_SUITE (suite)
per Step 1
test case version
latest version of the case
from getTestCase
deviceSelection.type
INDIVIDUAL_DEVICES
prefer explicit devices over a bundle (avoids stale-bundle surprises)
devices
1 available device matching the target's platform
call listDevices(platform=<platform>, available=true) and pick online, unbooked, non-cloud devices; if the user asked for N, pick N distinct ones. Pass each as { udid, isCloud }.
appSelections
the test case's app under test, latest version
derive appPackage + appVersionId from getTestCase (the case records its app); omit only if the target carries no app.
deviceAllocationStrategy
CROSS_DEVICE
"All Permutations" — run each test case on each device (see label map below)
Allocation-strategy labels (show the human label to the user, send the enum to the API). Mirror the
Portal's "Device Allocation Strategy" dropdown wording:
API enum
Show to the user
CROSS_DEVICE
All Permutations — run each test case on each device
SINGLE_DEVICE
Random Allocation — run each test case once, randomly chosen from the selected devices
Never surface the bare enum (CROSS_DEVICE) in the summary or prompts — it's meaningless to the user.
3. Show the summary and confirm (skip the prompt only if the user already gave full, explicit details)
Post a compact summary of exactly what will be created, e.g.:
About to create this test run:
Test case : API Demos (id 019ef40b…, version 2, Android)
Devices : 3 × Android (Pixel 8 · Galaxy S24 Ultra · Pixel 4)
App : io.appium.android.apis (version 73202)
Allocation: All Permutations — run each test case on each device
Name : "API Demos — 3 devices — 2026-06-24 11:20"
(Allocation shows the human label, not the CROSS_DEVICE enum.)
Then ask to proceed or customize in one prompt — offer "Proceed", "Change devices", "Change
allocation", "Change name", and let the user free-type any other tweak. When the user wants to change
allocation, present the two human-labeled choices (All Permutations / Random Allocation) and map their
pick back to the enum. Re-summarize and re-ask only if they change something. If the user's original
request was already fully explicit (every field named), you may create directly and just state what you
created.
4. Create the run
Call createTestRun with the assembled payload (camelCase keys as in the tool schema; the exact enum
values from Step 2). On success it returns the test run id + queued sessions. Report the id and a
one-line confirmation.
If createTestRun returns a validation error, fix it from the error text and the Step-2 enum reference
rather than guessing — do not retry the same body.
5. Offer monitoring — one prompt, then delegate
First, decide whether monitoring is even possible here.monitor-test-run runs a bundled Node
poller, so it needs a local filesystem, a shell, and ~/.kobiton/.credentials. You can't probe for
those from this skill (it has no shell of its own), so judge from what your host has given you: if
you have no way to run a local command, monitoring is out — don't present the choices below, because
offering a watch you can't perform is the trap the matrix in CLAUDE.md warns about. Close out instead:
Run created (<testRunId>) — . I can't watch it from here (monitoring needs to run a
local poller, which isn't available in this environment), so follow it in the portal — or ask me again
from a host with a shell and I'll monitor it for you.
Then stop. If you can run local commands, continue below and let monitor-test-run handle the rest:
it checks for the credentials file itself and reports the /automate:setup remedy, and it degrades to a
foreground loop on hosts without a streamed-output affordance rather than refusing. Don't pre-empt
either of those decisions here.
Read the live-remediation flag first: call getOrgSettings once and note live_remediation_enabled
(flagOn). This decides whether the auto-open option is meaningful.
Then offer monitoring in a single message (don't fire multiple separate questions — that's the
annoying part). Present the choices inline and let the user pick one:
Monitor + auto-open live remediation — only list this option when flagOn = true. Watches the
run and, when an execution blocks, automatically opens the live-remediation window for the device.
Monitor only — watches the run and surfaces blockers / the final summary, but doesn't auto-open
windows (you print the URL instead).
Don't monitor — stop here; give the user the test run id (and the portal link if handy) so they can
watch it themselves later.
Phrase it as one prompt, e.g. (flag ON):
Run created (<testRunId>). Want me to monitor it? (a) monitor + auto-open the live-remediation
window when a blocker hits, (b) monitor only (I'll surface blockers + the URL), or (c) don't
monitor. Reply a / b / c.
(flag OFF — drop option a, since there's no live window to open):
Run created (<testRunId>). Want me to monitor it? (b) monitor only (I'll surface blockers + the
portal URL), or (c) don't monitor. Reply b / c.
On the user's answer:
(a) or (b) → invoke the monitor-test-run skill for <testRunId>. Pass along the auto-open
intent: for (a), the user has already opted into auto-open, so tell monitor-test-run to skip its own
up-front auto-open question and treat autoOpen = yes; for (b), autoOpen = no. monitor-test-run
owns the watch loop (the bundled poller, blocker surfacing, post-mortem) from here.
(c) → done. Report the test run id and how to watch later (monitor-test-run <testRunId>), then
stop.
Errors
Condition
Handling
No test case/suite id given
Ask for one — it's the only field with no default.
Correct from the error + the Step-2 enum reference; don't blind-retry.
No available devices for the platform
Tell the user; offer to widen (cloud devices) or wait. Don't create a run that can't dispatch.
getOrgSettings fails before the monitor offer
Assume flagOn = false (drop the auto-open option); still offer monitor-only / don't-monitor.
Notes
This skill creates the run and hands off the watch; it does not implement monitoring itself —
that's monitor-test-run (same plugin), which runs the bundled emit-on-change poller.
Default to the smallest run that satisfies the request (1 device unless asked for more) — creating real
device sessions consumes minutes/concurrency.
Prefer INDIVIDUAL_DEVICES with explicit { udid, isCloud } over a device bundle, so the exact devices
are known and a stale/oversized bundle can't surprise the run.
