autonomous-experiment

name	autonomous-experiment
description	Design and run GP-driven adaptive experiments via Tsuchinoko

Autonomous Experiments

When the user asks for a smart/adaptive scan, peak finding, parameter optimisation, or any other GP-driven experiment, follow this workflow. Do not improvise around it — each step relies on the previous one.

1. Design

Load gpCAM's experiment-designer skill (via the Skill tool) and follow its conversation flow. If you cannot see that skill, gpCAM is not installed in Lightfall's environment — tell the user:

"I can't see the gpCAM design skills. They ship inside gpCAM itself, but the PyPI wheel does not include them — install gpCAM from source, e.g. pip install "gpcam @ git+https://github.com/lbl-camera/gpcam", in the Lightfall environment and restart Lightfall, then ask me again."

…and stop. Do not proceed with tsuchinoko_* tools before the user confirms.

Sibling skills are available for lazy load via the Skill tool when the design needs them: acquisition-functions, kernel-designer, prior-mean-functions, noise-functions, cost-functions, gp2scale-advanced, multi-task-advanced.

Tsuchinoko also ships skills for live engine state. Load parameter-tree (from Tsuchinoko) when the design needs to set hyperparameter bounds/initial values, change the training-schedule milestones, or read/write any other field not exposed by tsuchinoko.experiment.configure.

2. Discover Tsuchinoko

Call tsuchinoko_discover(). If the list is empty, tell the user:

"No Tsuchinoko instance is responding on the bus. Start one (tsuchinoko run) and tell me when it's ready."

…and stop.

3. Reset stale state

If Tsuchinoko's state is anything other than Inactive (check with tsuchinoko_status()), a previous experiment left residual state. You must stop it before configuring:

tsuchinoko_stop()
# poll tsuchinoko_status() until state == "Inactive"

configure only updates GP parameters — it does not reset the iteration counter, accumulated data, or run state. The full reset happens during the stop → Inactive transition (clears data) and the subsequent Starting transition (resets the GP model). Skipping this step causes the new plan to connect to a stale engine that serves few or no targets, leading to a near-empty run.

4. Upload custom callables (if needed)

If the design includes a user-authored acquisition function, kernel, prior mean, or noise function, upload each one before configure:

tsuchinoko_upload_design_code(
    name="my_ucb", kind="acquisition", code="<python source>"
)

kind is one of acquisition, kernel, prior_mean, noise. The tool returns a ref string of the form "user:<name>"; use it in configure.

5. Configure

tsuchinoko_configure(payload) — payload is a dict with these fields (omit any that should keep the engine default):

• parameter_bounds: list of [lo, hi] per axis (required)
• dimensionality: optional int
• kernel: "matern_3_2" | "matern_1_2" | "matern_5_2" | "se" | "periodic" | "user:<name>"
• acquisition_function: "variance" | "ucb" | "ei" | "user:<name>"
• prior_mean: null or "user:<name>"
• noise_function: null or "user:<name>"
• noise_variances: null, float, or list of floats
• initial_points: int (default 10)
• training_method: "global" | "local" | "mcmc" | "adam" | "hgdl"
• hyperparameters: optional list of floats (initial values)
• x_out: optional, for fvGP multi-task

Unknown keys are an error. The configure tool will surface that verbatim — fix it before retrying.

6. Pre-flight: verify motor positions are within bounds

Before launching the plan, read each motor's current position with lightfall_read_device(name=<motor>) and verify it lies inside the corresponding [lo, hi] from parameter_bounds. If any motor is outside its bound, do not start the plan. Tell the user which motor is where, then either:

1. Ask permission to move the motor into range, or
2. Ask the user to widen parameter_bounds and re-run configure (remember: configure requires tsuchinoko_stop() → Inactive first — see step 3).

Tsuchinoko/gpCAM initialise the GP from the supplied bounds. A motor sitting outside its axis bound rejects the first measurement as out-of-domain and the run dies before producing any data.

7. Run

Use the existing plan tool:

lightfall_run_plan(
    plan_name="adaptive_experiment",
    params={
        "detectors": [<detector names>],
        "motors": [<motor names, in the same order as parameter_bounds>],
        "timeout": 300.0,
    },
)

The plan opens a single Bluesky run, hands off Tiled credentials to Tsuchinoko via bind_run, and drives the move-and-measure loop.

8. Monitor

Tell the user to open the Visualization Panel — the AdaptiveHeatmapVisualization (posterior mean / variance / acquisition) and AdaptiveHyperparameterPlot widgets will populate live as iterations land in the Tiled adaptive stream.

For textual progress, call tsuchinoko_status().

9. Control

tsuchinoko_pause(), tsuchinoko_resume(), tsuchinoko_stop() — each takes no arguments. Use tsuchinoko_stop() to finalise the experiment from the Tsuchinoko side; the Lightfall plan exits cleanly when targets stop arriving (configurable timeout).

Constraints

• Always stop before reconfiguring. tsuchinoko_configure only updates GP parameters — it does not reset state, data, or the iteration counter. You must call tsuchinoko_stop() and wait for Inactive before configuring a new experiment, whether the previous one is running, paused, or finished.
• Do not start a new adaptive_experiment before stopping the current one — the bind_run handshake is single-occupant.
• motors order in lightfall_run_plan must match the axes order in parameter_bounds. Disagreement silently produces nonsense.
• Verify motor positions before running. A motor sitting outside its parameter_bounds axis crashes gpCAM at the first iteration — the plan opens a Bluesky run, then dies with no data. Always read each motor with lightfall_read_device after configure and before lightfall_run_plan; the plan itself does not validate this.