multi-recording-processing

name	multi-recording-processing
description	Orchestrates multi-recording neural imaging batch processing via the cindra MCP server, dispatching to configuration and results skills as needed. Use when the user asks to run multi-recording (cross-day ROI tracking) processing, process multiple recordings as a tracked dataset, monitor multi-recording batch jobs, re-run a multi-recording processing phase, or when invoking /multi-recording-processing.
user-invocable	true

Multi-recording processing

Orchestrates the multi-recording batch processing workflow: verify prerequisites, organize recordings by dataset, prepare execution manifests, dispatch jobs, monitor progress, and hand off to downstream skills for output verification.

---

Scope

Covers:

• Batch processing workflow: prerequisite verification, dataset organization, preparation, execution, monitoring, and completion
• The MCP preparation, execution, management, and supporting tools listed in the Available tools section
• Dataset name resolution via resolve_dataset_name_tool
• Resource management and CPU allocation guidance
• Status formatting and progress monitoring
• Error routing to appropriate upstream skills

Does not cover:

• Configuration parameters, tuning guidance, or config file creation (see /multi-recording-configuration)
• Output data formats, array shapes, dtypes, file references, or data interpretation (see /multi-recording-results)
• Single-recording processing workflow or prerequisites (see /single-recording-processing)
• Input data format, TIFF requirements, or acquisition parameters (see /acquisition-data-preparation)
• MCP server connectivity or environment issues (see /cindra-mcp-environment-setup)
• Visual inspection of results (see /visualization)

Handoff rules: If the user asks about specific output files, array shapes, data interpretation, registration arrays, tracking templates, or processing result verification, invoke /multi-recording-results. If the user asks about parameter tuning, registration/tracking configuration, or ROI selection criteria, invoke /multi-recording-configuration.

---

Agent requirements

You MUST use the cindra MCP tools for all processing operations. Do not import cindra Python functions directly or run processing via scripts or CLI commands. If MCP tools are not available, invoke /cindra-mcp-environment-setup to diagnose and resolve connectivity issues.

---

Prerequisites

An incomplete recording routes to the earliest missing step in the chain /acquisition-data-preparation → /single-recording-configuration → /single-recording-processing. Workflow step 1 states the check itself.

---

Available tools

Preparation tools

Tool	Purpose
get_pipeline_job_universe_tool	Reports every job a dataset declares and which can run right now
size_pipeline_jobs_tool	Reports the cores and memory every job holds, before dispatching anything
check_threading_runtime_tool	Reports whether the numeric threading layer this host needs is loadable
prepare_multi_recording_batch_tool	Prepares execution manifest without starting execution (idempotent)
execute_full_pipeline_tool	Convenience: prepares and executes all phases with automatic sequencing

Execution tools

Tool	Purpose
execute_processing_jobs_tool	Dispatches prepared jobs for background execution
get_processing_jobs_status_tool	Returns per-job status of active execution session
get_active_execution_timing_tool	Returns per-job timing and session-level throughput
cancel_processing_jobs_tool	Cancels active execution, clears pending queues

Management tools

Tool	Purpose
get_batch_status_overview_tool	Bird's-eye view of all processing status under a root directory
reset_processing_phases_tool	Selectively reset completed phases for re-runs
clean_processing_output_tool	Delete output files for specific phases to reclaim disk space

Supporting tools

/multi-recording-configuration owns the first four. Invoke that skill for their parameters and usage guidance.

Tool	Purpose
discover_recordings_tool	Discovers single and multi-recording candidates under a root dir
generate_config_file_tool	Generates default multi-recording configuration YAML
resolve_dataset_name_tool	Constructs qualified dataset names from base name + specifier
set_config_values_tool	Writes new values into an existing configuration file
get_recording_status_tool	Checks single and multi-recording processing status

---

Pipeline architecture

Two-phase pipeline per dataset:

Phase 1: DISCOVER (phase name: discovery, CPU bound, parallel by dataset)
├── Selects/filters ROIs from each recording's single-recording outputs
├── Registers all recordings to common reference frame
├── Clusters ROI masks across recordings
├── Generates template masks for tracked ROIs
├── Projects template masks back to each recording's coordinate system
└── One job per dataset, workers from the discovery resource class (2 per job, see Resource management)

Phase 2: EXTRACT (phase name: extraction, CPU bound, parallel by recording)
├── Applies template masks to extract fluorescence
├── Computes neuropil signals, spike deconvolution
└── One job per recording, workers from the extraction resource class (16 per job, see Resource management)

---

Dataset name resolution

Each dataset in a batch needs a unique dataset_name for output directories and batch tracking. The resolve_dataset_name_tool constructs qualified names by combining a shared base name with a batch-specific specifier:

resolve_dataset_name_tool(
    dataset_name="learning_task",           # shared analysis name from user
    output_roots=["/data/animal_A/rec1", "/data/animal_A/rec2"],
    specifier=""                            # auto-derived from common parent → "animal_a"
)
→ { "dataset_name": "animal_a_learning_task", "specifier": "animal_a", "base_name": "learning_task" }

output_roots holds the pipeline output roots of the completed single-recording runs, each the parent of that recording's cindra/ directory. The tool is a pure string computation that writes no file. The name it returns reaches a configuration only when you pass it as a dataset's dataset_name to prepare_multi_recording_batch_tool, which writes the lowercased name into recording_io.dataset_name of the resolved per-dataset copy. To set the field on a template directly, use set_config_values_tool, owned by /multi-recording-configuration.

Specifier derivation strategies:

• Auto (default): Derived from the deepest common parent directory of the output roots.
• Explicit: The user or agent provides a specifier directly (e.g., brain region, session group).
• Semantic: The agent determines the specifier by analyzing recording directory names or paths.

This enables batch bootstrapping: discover all recordings under a project directory, group them by common parent, and call resolve_dataset_name_tool once per group to generate unique dataset names.

---

Processing workflow

Execution model

The processing workflow uses a prepare-then-execute model:

1. Prepare creates an execution manifest (tracker files, per-dataset configurations, job lists) without starting any computation. This step is idempotent, so calling it again on the same datasets returns the existing manifest.

2. Execute dispatches jobs from the manifest with prerequisite validation, resource allocation, and automatic phase sequencing. Only one execution session can be active at a time.

For simple cases, execute_full_pipeline_tool combines both steps into a single call with automatic phase advancement. For fine-grained control (e.g., running only specific phases, custom resource allocation, or selective re-runs), use prepare_multi_recording_batch_tool followed by execute_processing_jobs_tool.

STOP: Steps 1 through 5 below are the entry conditions for dispatch. Complete every one of them, and confirm each against the verification checklist at the end of this skill, before calling any execute tool.

Workflow steps

1. Verify prerequisites. Use discover_recordings_tool, documented in /multi-recording-configuration, to find eligible recordings and get_recording_status_tool to confirm each has single-recording status completed. Each multi_recording_candidates entry is an object, so read candidate["output_root"] for both calls and carry that value through the rest of this workflow. If any recording is incomplete, invoke /single-recording-processing (or upstream skills as needed).

2. Organize into datasets. Group output roots by common parent directory, user-provided grouping, or semantic analysis of their paths. Each group becomes one dataset in the batch.

3. Resolve dataset names. Ask the user for a shared base dataset name (e.g., "learning_task"). For each group, call resolve_dataset_name_tool with the base name and the group's output_roots to generate a unique qualified name. The specifier is derived automatically from the common parent directory, or the user can provide one explicitly.

4. Configure. Ask the user if they have an existing template configuration file. If not, invoke /multi-recording-configuration to create one. Template configs are reusable across datasets and live at user-chosen locations (e.g., /Data/CA1_GCaMP6f_MD.yaml). The template's dataset_name only needs to be a non-empty, filesystem-safe string to pass validation, because the prepare tool overwrites it per dataset as the Dataset name resolution section states. Do NOT create per-dataset config copies. The prepare tool automatically saves resolved copies as multi_recording_configuration.yaml inside each dataset's output directory, preserving the original template. Pass the same template path for multiple datasets that share parameters.

5. Confirm the machine budget. The engine resolves the per-class worker counts and concurrency caps itself, so do not ask the user to choose them and do not build an allocation table before dispatching. Ask only how much of the machine this run may take, because that is the one allocation question the engine cannot answer: it claims every core but two and the memory available when the session starts. Confirm that the host is free for the run, or take an explicit ceiling from the user and pass it as max_parallel_jobs. That ceiling caps the concurrent jobs of each resource class separately rather than the session as a whole, so the classes still sum above it. After dispatch, report the resolved allocation from the mapping the execute tool returns, which for 2 datasets of 15 recordings on a 128-core host reads:

Output-root path rule

get_recording_status_tool, verify_multi_recording_output_tool, and clean_processing_output_tool all name their argument output_root and take the pipeline output root (the parent of the cindra/ folder), which equals the output_roots entries passed to the prepare tools. It is not the raw_data_path holding the recording's TIFF files, and the distinction matters on a separate-output layout where the two roots differ:

• get_recording_status_tool and clean_processing_output_tool resolve cindra/ directly under the given path with NO fallback. Feeding the raw-data path makes them report not_started or "output root not found", a silent false negative.
• verify_multi_recording_output_tool also recursively searches for configuration.yaml, so it may still pass via that fallback even when fed the wrong root. The two then disagree.

Always reuse the output root that single-recording processing used, its output_root entry in the prepare_single_recording_batch_tool manifest, for status, verify, and clean. The multi-recording prepare manifest exposes no output_root field: each dataset entry holds only configuration_path, tracker_path, dataset_name, pipeline_type, discover_job, and extract_jobs. Every job entry additionally carries executor_id when the dataset's tracker already existed.

Re-running specific phases

1. Use reset_processing_phases_tool with tracker_path, phases, and pipeline_type="multi-recording" to reset the target phases to SCHEDULED status. Downstream phases are automatically reset (e.g., resetting discovery also resets extraction).
2. Optionally modify the configuration file with set_config_values_tool before re-execution.
3. Use clean_processing_output_tool to delete output files from the reset phases (requires output_root, pipeline_type="multi-recording", and the lowercased dataset name).
4. Call execute_processing_jobs_tool with the reset jobs from the manifest.

Discovery is not idempotent under a reset alone. A reset returns the phase to SCHEDULED and deletes nothing, so each substage re-reads the output the previous run left and returns early.

Sub-stage	Skips while	Forced by
ROI selection	Channel 1 selections exist, and channel 2 when it has data	repeat_selection
Registration	Every recording reports itself registered	repeat_registration
Tracking	tracking_template_masks.npz exists in the first output path	repeat_registration

Such a re-run reports SUCCEEDED while writing byte-identical output, so the new parameter never takes effect. Clean the discovery phase, which deletes the template masks, or set repeat_selection in recording_io and repeat_registration in diffeomorphic_registration with set_config_values_tool, all owned by /multi-recording-configuration.

reset_processing_phases_tool detects the ROI-selection case itself. It returns a warnings list naming recording_io.repeat_selection whenever it resets discovery while that flag is false and the dataset already carries selected ROIs. Act on every warning before dispatching the reset phase, because the key is absent when nothing would skip and its presence is the only signal that a reset alone accomplishes nothing.

---

Resource management

Discovery and extraction run under separate resource classes, each carrying its own measured per-job worker count. The session CPU budget is cpu_count - 2, with 2 cores reserved for system operations. The dispatcher holds the sum of the cores committed by every class inside that budget, so the two classes interleave rather than each claiming the whole budget.

Phase	Resource class	Cores per job	Concurrency cap
DISCOVER	discovery	2	Session CPU budget
EXTRACT	extraction	16	Session CPU budget

Discovery's 2 covers the deformation pool alone. The stage has no parallel critical path, so quadrupling the allocation shortens a twenty-recording dataset by two percent. Extraction's 16 follows the concurrency a compute node sustains rather than a measured plateau, leaving room for the six to eight datasets a node extracts at once while still reaching a sevenfold single-job speedup.

Each class caps its own concurrency at min(max(1, budget // cores_per_job), max(1, job_count)). The discover phase contributes one job per dataset, and the extract phase contributes one job per recording. On a 128-core host, with a budget of 126, discovery's budget-derived bound is 63 jobs, so its dataset count binds first in any realistic batch, and extraction runs at most 7.

No cap here is a fixed number the host outgrows. Both derive from the session core budget, so a wider host raises them on its own, and the dispatcher then admits against the live core and memory budgets rather than against the cap alone. The engine saturates the host it is given, so leave both parameters as None unless the user asks for an override.

Extraction holds its 16 workers on a small host too. A 16-core machine still asks for 16 against a budget of 14 and dispatches one job regardless, because the dispatcher always admits a single job even when the budget cannot cover it.

Memory bounds dispatch separately from both class caps. Each job is estimated from the dataset it will process, and the dispatcher holds the sum of the running jobs' estimates inside the session memory budget, reported as memory_budget_mb. That budget is the host's available memory sampled once when the session starts and never re-read. Every job runs in its own spawned process, so a job the host kills for exhausting memory takes down its own worker rather than the batch, and the engine records a terminal outcome for every job the failure strands.

Both workers_per_job and max_parallel_jobs default to None and can be overridden in execute_processing_jobs_tool or execute_full_pipeline_tool. A positive value of either is used exactly. Setting workers_per_job to -1 gives every job the whole session core budget, while setting max_parallel_jobs to -1 lifts the derived cap so that only the job count bounds concurrency. An override is a single scalar applied to every non-fixed class alike, so passing workers_per_job=20 sets discovery and extraction to 20 both. max_parallel_jobs is a per-class cap rather than a session ceiling, so max_parallel_jobs=4 permits 4 discovery jobs and 4 extraction jobs at once, and the session CPU and memory budgets remain the only terms bounding the classes in aggregate. Both execute tools return the session-level cpu_budget and memory_budget_mb, and a resource_classes mapping keyed by class name, with discovery and extraction entries carrying workers_per_job, max_parallel_jobs and job_count. get_processing_jobs_status_tool returns the same mapping with pending and active in place of job_count, and adds a session-level awaiting_prerequisites count of the jobs still held in the admission pool.

Planning before dispatch

get_pipeline_job_universe_tool and size_pipeline_jobs_tool both need a resolved per-dataset configuration rather than a template. Each loads the file through the loader the pipeline uses, which rejects a configuration naming fewer than two recording_io.recording_directories or holding an empty recording_io.dataset_name. A generated template holds neither, so a template makes both return success: false carrying the loader's message rather than a plan. Run them against the multi_recording_configuration.yaml that prepare_multi_recording_batch_tool writes into a dataset's output directory, or fill both fields on a template first with set_config_values_tool. Preparation is idempotent and starts no computation, so planning a multi-recording batch means preparing it first and dispatching second.

get_pipeline_job_universe_tool answers which jobs can run right now. It reads the inventory the output roots already hold rather than any tracker. The discovery job is ready once every recording carries its single-recording output, and an extraction job once discovery has written the template masks its recording projects. The recording identifiers come from the configured directory paths rather than from what those directories hold, so a wholly unprocessed dataset still reports resolved: true with the full universe and ready: false on every job. Every configuration the tool accepts names at least two directories, so resolved is true whenever the call succeeds and carries no information. Gate on ready alone to decide what to dispatch. Use it to plan a selective re-run, and get_recording_status_tool to read recorded outcomes once a batch has been prepared.

size_pipeline_jobs_tool reports the cores and memory every job of a dataset holds, reading the completed single-recording output the dataset runs on. Pass the dataset's configuration path and pipeline_type="multi-recording". The response lists each job's name, specifier, cores, and memory_mb, plus peak_memory_mb for the single largest job and total_memory_mb for every job at once. Read peak_memory_mb rather than assuming which stage dominates, because discovery's clustering term grows with the square of the region count while extraction's trace arrays grow with the frame count, so either stage leads depending on the dataset.

check_threading_runtime_tool reports whether the numeric threading layer this host needs is loadable, which is OpenMP on macOS and TBB elsewhere. Gate a batch on its ready flag. A macOS host that is not ready aborts every job at the pipeline entry point before any stage runs, while a non-macOS host missing TBB fails at the job's first parallelized call. Neither outcome returns a tool error, so both surface only as per-job tracker failures.

---

Status formatting

When presenting batch status to the user, format as a table:

**Multi-Recording Batch Processing Status**

Current Phase: EXTRACT
Summary: 1/2 datasets complete | 2/4 recordings extracted | 0 failed

| Dataset                | Discover | Extract Progress | Status     |
|------------------------|----------|------------------|------------|
| animal_a_learning_task | done     | 2/2              | COMPLETED  |
| animal_b_learning_task | done     | 0/2              | EXTRACTING |

---

Error routing

Preparation errors

Error Message	Resolution
"At least one dataset configuration is required"	Provide dataset configurations
"Configuration not found"	Invoke /multi-recording-configuration
"output_roots must be a list"	Pass output_roots as a list of strings
"Need at least 2 recordings"	Provide at least 2 output roots
"Invalid recordings"	Verify output roots exist and are dirs
"Empty dataset_name"	Resolve a name with the name tool first

Partially accepted batches

A batch that rejects some of its input still returns success: true, so you MUST read the rejection lists rather than treat the absence of an error as full acceptance. Report every rejected dataset to the user by name before proceeding, because the batch runs without it.

Key	Returned by	Meaning
invalid_configurations	both prepare and full-pipeline tools	A dataset entry was rejected, with its reason
path_conflicts	both prepare and full-pipeline tools	The dataset runs against stored roots, not the ones passed
unsizable_datasets	execute_full_pipeline_tool	The sizing models cannot size the dataset, so it is omitted
invalid_jobs	execute_processing_jobs_tool	A job failed validation or sizing and was not dispatched

path_conflicts is the one entry that names a dataset the batch still runs. Preparation never reinitializes an existing tracker, so a dataset prepared earlier keeps using its stored recording_io.recording_directories and the entry reports the dataset, the stored value, the passed value, and the directory to remove to prepare it again. Report it before dispatching, because the run otherwise covers recordings the user did not ask for.

A dataset the sizing pass cannot measure is excluded from the batch rather than aborting it, so a run that reports started: true may still cover fewer datasets than you submitted. execute_full_pipeline_tool returns no dataset count of its own, so name every entry of invalid_configurations and unsizable_datasets to the user rather than looking for a total to compare. Only the prepare tool returns total_datasets.

These lists do not share one element shape. See tool-responses.md for the element shapes, the full return-key reference of every processing tool, and the terminal messages the engine writes to a tracker.

Execution errors

Error Message	Resolution
"An execution session is already active"	Wait for current session or cancel first
"Job ID not found in tracker"	Re-prepare the batch to regenerate manifests
"Prerequisite ... has not succeeded"	Execute prerequisite phases first

Prerequisite failures are returned inside the invalid_jobs list with a reason field (for example, "Unable to execute job {job_id}. Its prerequisite 'discovery' job {prerequisite_id} has not succeeded and is not part of this submission."), not as a top-level error.

Processing failure routing

When processing fails for some datasets/recordings, read the error messages and route:

Error pattern	Skill to invoke
Missing cindra output, incomplete single-recording	/single-recording-processing
Plane binary left marked by an interrupted write	/single-recording-processing
Missing raw data, no cindra_parameters.json	/acquisition-data-preparation
Configuration parameter issues, bad dataset name	/multi-recording-configuration
Registration tuning needed (too much/little drift)	/multi-recording-configuration
No trackable ROIs found	/multi-recording-configuration
MCP tools unavailable, server connection errors	/cindra-mcp-environment-setup

Wait for the current execution session to complete before starting retries. cancel_processing_jobs_tool empties the admission pool and every class queue but never stops a job already dispatched, so poll get_recording_status_tool until the previously RUNNING jobs leave RUNNING before starting a new session.

---

Related skills

Skill	Relationship
/cindra-pipeline	Overview: end-to-end phases, handoffs, and the single-vs-multi entry point
/cindra-mcp-environment-setup	Prerequisite: MCP server connectivity
/cli-reference	Reference: cindra run, the manual counterpart this workflow replaces
/acquisition-data-preparation	Upstream: raw data preparation
/single-recording-configuration	Prerequisite chain: configure recordings before single-recording processing
/single-recording-processing	Prerequisite: all recordings must be single-recording complete
/multi-recording-configuration	Configuration: parameter reference and file creation
/multi-recording-results	Output: verify and explain processing results
/visualization	Downstream: visual inspection of results

---

Verification checklist

Multi-Recording Processing Workflow:
- [ ] MCP server connected (if not, invoke `/cindra-mcp-environment-setup`)
- [ ] All recordings confirmed as single-recording complete (status: completed)
- [ ] Recordings grouped into datasets
- [ ] Dataset names resolved via `resolve_dataset_name_tool`
- [ ] Template configuration confirmed or created via `/multi-recording-configuration` (reusable across datasets)
- [ ] Share of the machine to dedicate to processing confirmed with user
- [ ] Batch prepared or full pipeline executed
- [ ] `path_conflicts` read from the prepare response and every named dataset reported to the user
- [ ] Every `warnings` entry from `reset_processing_phases_tool` acted on before dispatching the reset phase
- [ ] Parameter-change re-run cleaned the `discovery` phase or set the matching repeat flag
- [ ] Status monitored until all datasets complete or fail
- [ ] Failed datasets routed to appropriate skill (see Error routing)
- [ ] Successful datasets verified via `/multi-recording-results`