| name | single-recording-processing |
| description | Orchestrates single-recording neural imaging batch processing via the cindra MCP server, dispatching to acquisition-preparation, configuration, and results skills as needed. Use when the user asks to process single-recording imaging data, run the single-recording batch pipeline, monitor single-recording batch jobs, re-run a single-recording processing phase, or when invoking /single-recording-processing. |
| user-invocable | true |
Single-recording processing
Orchestrates the single-recording batch processing workflow: discover recordings, validate prerequisites,
prepare execution manifests, dispatch jobs, monitor progress, and hand off to downstream skills for output
verification.
Scope
Covers:
- Batch processing workflow: discovery, validation, preparation, execution, monitoring, and completion
- MCP preparation tools (
prepare_single_recording_batch_tool, execute_full_pipeline_tool)
- MCP execution tools (
execute_processing_jobs_tool, get_processing_jobs_status_tool,
get_active_execution_timing_tool, cancel_processing_jobs_tool)
- MCP management tools (
get_batch_status_overview_tool, reset_processing_phases_tool,
clean_processing_output_tool)
- Supporting tools for validation and status checking (recording discovery owned by
/single-recording-configuration)
- Resource management and CPU allocation guidance
- Status formatting and progress monitoring
- Error routing to appropriate upstream skills
Does not cover:
- Configuration parameters, tuning guidance, or config file creation (see
/single-recording-configuration)
- Output data formats, array shapes, dtypes, file references, or data interpretation (see
/single-recording-results)
- Input data format, TIFF requirements, or acquisition parameters (see
/acquisition-data-preparation)
- Multi-recording processing workflow (see
/multi-recording-processing)
- MCP server connectivity or environment issues (see
/cindra-mcp-environment-setup)
- Visual inspection of results (see
/visualization)
Handoff rules: If the user asks about specific output files, array shapes, data interpretation, or processing
result verification, invoke /single-recording-results. If the user asks about parameter tuning or configuration
options, invoke /single-recording-configuration. This skill owns the processing workflow only — not the data
it produces or the parameters it consumes.
Agent requirements
You MUST use the cindra MCP tools for all processing operations. Do not import cindra Python functions
directly or run processing via scripts or CLI commands. If MCP tools are not available, invoke
/cindra-mcp-environment-setup to diagnose and resolve connectivity issues.
Available tools
Preparation tools
| Tool | Purpose |
|---|
prepare_single_recording_batch_tool | Prepares execution manifest without starting execution (idempotent) |
execute_full_pipeline_tool | Convenience: prepares and executes all phases with automatic sequencing |
Execution tools
| Tool | Purpose |
|---|
execute_processing_jobs_tool | Dispatches prepared jobs for background execution |
get_processing_jobs_status_tool | Returns per-job status of active execution session |
get_active_execution_timing_tool | Returns per-job timing and session-level throughput |
cancel_processing_jobs_tool | Cancels active execution, clears pending queues |
Management tools
| Tool | Purpose |
|---|
get_batch_status_overview_tool | Bird's-eye view of all processing status under a root directory |
reset_processing_phases_tool | Selectively reset completed phases for re-runs |
clean_processing_output_tool | Delete output files for specific phases to reclaim disk space |
Supporting tools (used during workflow)
| Tool | Purpose |
|---|
discover_recordings_tool | Discovers single and multi-recording candidates |
validate_recording_readiness_tool | Validates raw data and parameters before processing |
get_recording_status_tool | Checks single and multi-recording processing status |
Pipeline architecture
Three-phase sequential pipeline per recording:
Phase 1: BINARIZE (phase name: binarization; I/O bound, up to 4 parallel)
├── Converts raw TIFFs to binary format
└── Determines plane count
Phase 2: PROCESS (phase name: processing; CPU bound, parallel by plane)
├── Motion correction, ROI detection, signal extraction
└── Workers per plane via saturating allocation (see Resource management)
Phase 3: COMBINE (phase name: combination; I/O bound, up to 4 parallel)
└── Merges all plane results into a unified combined_metadata.npz dataset
Batch processing across multiple recordings:
BINARIZE: Up to 4 concurrent recordings (I/O bound, fixed concurrency)
PROCESS: Parallel (recording-plane pairs up to core limit)
COMBINE: Up to 4 concurrent recordings (I/O bound, fixed concurrency)
Processing workflow
Execution model
The processing workflow uses a prepare-then-execute model:
-
Prepare creates an execution manifest (tracker files, per-recording configurations, job lists) without
starting any computation. This step is idempotent — calling it again on the same recordings returns the
existing manifest.
-
Execute dispatches jobs from the manifest with prerequisite validation, resource allocation, and automatic
phase sequencing. Only one execution session can be active at a time.
For simple cases, execute_full_pipeline_tool combines both steps into a single call with automatic phase
advancement. For fine-grained control (e.g., running only specific phases, custom resource allocation, or
selective re-runs), use prepare_single_recording_batch_tool followed by execute_processing_jobs_tool.
Pre-processing checklist
- [ ] Recordings discovered or paths provided
- [ ] Raw data validated (or existing binaries confirmed via get_recording_status_tool)
- [ ] Template configuration confirmed or created (one template can serve multiple recordings)
- [ ] Output directory confirmed with user
- [ ] CPU core allocation confirmed with user
- [ ] Recordings to process confirmed
STOP: If any checkbox is incomplete, do not proceed. Complete the missing steps first.
Workflow steps
-
Discover recordings — Use discover_recordings_tool (check the single_recording_candidates list) or
accept explicit paths from user.
-
Validate raw data — Use validate_recording_readiness_tool on each recording. Skip for recordings where
get_recording_status_tool shows status binarizing, processing, combining, or completed. If validation
fails, invoke /acquisition-data-preparation to resolve issues before continuing.
-
Configure — Ask the user if they have an existing template configuration file. If not,
invoke /single-recording-configuration to create one. Template configs are reusable across
recordings and live at user-chosen locations (e.g., /Data/CA1_GCaMP6f_SD.yaml). Do NOT create
per-recording config copies — the prepare tool automatically saves resolved copies as
cindra/configuration.yaml inside each recording's output directory, preserving the original
template. Pass the same template path for all recordings that share parameters.
-
Confirm output directory — Ask the user where processed data should be written. Each
recording requires an explicit output path; the pipeline does not auto-resolve output locations.
Common patterns include writing output alongside the raw data (producing a cindra/ subdirectory
inside each recording) or writing to a separate root by mirroring the recording directory
structure. recording_output_paths is a required parameter for both prepare_single_recording_batch_tool
and execute_full_pipeline_tool.
-
Confirm CPU allocation — Present the resource allocation model and ask the user how many cores
to use (see Resource management section).
-
Execute — Choose one of two approaches:
Simple (recommended for straightforward runs):
Call execute_full_pipeline_tool with pipeline_type="single-recording", the confirmed recording
paths, configuration path, output paths, and worker settings. This prepares and executes all
phases automatically.
Fine-grained (for selective execution or re-runs):
a. Call prepare_single_recording_batch_tool with recording paths, configuration path, and
output paths. This returns a manifest with job IDs and statuses.
b. Select the jobs to execute from the manifest (e.g., only SCHEDULED jobs, only specific phases).
c. Call execute_processing_jobs_tool with the selected job descriptors and worker settings. Each
job descriptor needs configuration_path, tracker_path, job_id, and pipeline_type from
the manifest.
-
Monitor — Use get_processing_jobs_status_tool to check progress. Optionally use
get_active_execution_timing_tool for per-job timing and session throughput. These two tools
reflect only the active in-process execution session and return active: false with empty jobs
when no session is running. This drained state happens not only after an MCP server restart, a
reconnect, or a batch dispatched by a prior process, but also after NORMAL completion: the
manager clears session state on success AND on failure. So an all-zero, inactive status can mean
"finished," not "nothing ran." Do not read it as failure. For final per-job outcomes, read
persisted on-disk tracker state via get_batch_status_overview_tool for a whole-tree view,
get_recording_status_tool per recording, or verify_single_recording_output_tool (all using
the output directory, see the Output-directory path rule). Present status as a formatted table
(see Status formatting section).
-
Handle completion — When all recordings finish, check for failures. A success: true return
only means a tool ran, not that work is ready or done: gate decisions on the domain flag, not on
success. For verify_single_recording_output_tool, gate on complete (false whenever missing
is non-empty); for validate tools, gate on valid; for execute_full_pipeline_tool, gate on
started (it returns started: false with a next_step when all phases are already complete).
Checking success alone can advance on an unready or already-complete state. Route errors to the
appropriate skill (see Error routing section). On success, invoke /single-recording-results
to verify outputs, then /visualization for visual inspection.
Output-directory path rule
get_recording_status_tool, verify_single_recording_output_tool, and clean_processing_output_tool
all take the recording OUTPUT directory (the parent of the cindra/ folder), which equals the
recording_output_paths / per-entry output_path the prepare tool returns — NOT the raw-data root.
This matters on a separate-output layout where output and raw-data roots differ:
get_recording_status_tool and clean_processing_output_tool resolve cindra/ directly under the
given path with NO fallback. Feeding the raw-data root makes them report not_started or
"directory not found" — a silent false negative.verify_single_recording_output_tool also recursively searches for configuration.yaml, so it may
still pass via that fallback even when fed the wrong root. The two then disagree.
Always reuse the output_path captured from the prepare manifest for status, verify, and clean.
Re-running specific phases
To re-run specific phases (e.g., after changing ROI detection parameters):
- Use
reset_processing_phases_tool to reset the target phases to SCHEDULED status. Downstream
phases are automatically reset (e.g., resetting processing also resets combination).
- Optionally modify the configuration file before re-execution.
- Optionally use
clean_processing_output_tool to delete output files from the reset phases.
- Call
execute_processing_jobs_tool with the reset jobs from the manifest.
Both reset_processing_phases_tool and clean_processing_output_tool require pipeline_type="single-recording"
and a phases list drawn from the valid single-recording phase names: binarization, processing,
combination. reset_processing_phases_tool also requires tracker_path; clean_processing_output_tool
requires recording_path.
Resource management
The system uses saturating core allocation to distribute CPU cores across parallel compute-bound jobs.
I/O-bound jobs (binarize, combine) always use a fixed concurrency of 4 regardless of resource settings.
When both workers_per_job and max_parallel_jobs are set to -1 (automatic), the allocator
runs the following algorithm for compute-bound jobs:
- Budget:
cpu_count - 2 (2 cores reserved for system operations)
- Max parallel jobs:
min(total_jobs, max(1, budget // 30)) (targets ~30 workers per job, with a floor of 1)
- Raw workers per job:
budget // max_parallel_jobs
- Round down to the nearest multiple of 5
- Saturate: If workers per job falls below 10 and parallelism > 1, reduce parallelism and
recalculate until each job has at least 10 workers
| CPU Cores | Budget | Planes | Workers/Plane | Max Parallel | Total Utilized |
|---|
| 128 | 126 | 4 | 30 | 4 | 120 |
| 64 | 62 | 4 | 30 | 2 | 60 |
| 32 | 30 | 4 | 30 | 1 | 30 |
| 16 | 14 | 4 | 14 (→ 10) | 1 | 10 |
Either or both values can be overridden explicitly via workers_per_job and max_parallel_jobs
parameters in execute_processing_jobs_tool or execute_full_pipeline_tool.
Status formatting
When presenting batch status to the user, format as a table:
**Single-Recording Batch Processing Status**
Current Phase: PROCESS
Summary: 10/30 recordings complete | 2 processing | 18 queued | 0 failed
| Recording | Binarize | Process | Combine | Status |
|----------------------------|----------|---------|---------|------------|
| 2024-01-15-10-30-00-123456 | done | 2/4 | pending | PROCESSING |
| 2024-01-15-11-45-00-234567 | done | 4/4 | running | PROCESSING |
| 2024-01-16-09-00-00-111111 | done | 4/4 | done | SUCCEEDED |
| 2024-01-16-10-15-00-222222 | pending | 0/0 | pending | QUEUED |
Error routing
Preparation errors
| Error Message | Resolution |
|---|
| "At least one recording path is required" | Provide recording paths |
| "Configuration file not found" | Invoke /single-recording-configuration |
| "No valid recording paths provided" | Inspect invalid_paths in the response |
Execution errors
| Error Message | Resolution |
|---|
| "An execution session is already active" | Wait for current session or cancel first |
| "Job ID not found in tracker" | Re-prepare the batch to regenerate manifests |
| "Prerequisite ... has not succeeded" | Execute prerequisite phases first |
Prerequisite failures are returned inside the invalid_jobs list with a reason field (for example,
"Prerequisite BINARIZE job X has not succeeded."), not as a top-level error.
Processing failure routing
When processing fails for some recordings, read the error messages and route to the appropriate skill:
| Error pattern | Skill to invoke |
|---|
Missing cindra_parameters.json, TIFF read error | /acquisition-data-preparation |
| Invalid parameter values, wrong plane/channel | /acquisition-data-preparation |
| Configuration parameter issues | /single-recording-configuration |
| MCP tools unavailable, server connection errors | /cindra-mcp-environment-setup |
Wait for the current execution session to complete before starting retries.
Related skills
| Skill | Relationship |
|---|
/cindra-pipeline | Overview: end-to-end phases, handoffs, and the single-vs-multi entry point |
/cindra-mcp-environment-setup | Prerequisite: MCP server connectivity |
/acquisition-data-preparation | Input: raw data preparation and validation |
/single-recording-configuration | Configuration: parameter reference and file creation |
/single-recording-results | Output: verify and explain processing results |
/multi-recording-configuration | Downstream: configure cross-recording tracking |
/multi-recording-processing | Downstream: cross-recording ROI tracking |
/visualization | Downstream: visual inspection of results |
Verification checklist
Single-Recording Processing Workflow:
- [ ] MCP server connected (if not, invoke `/cindra-mcp-environment-setup`)
- [ ] Recordings discovered or explicit paths provided
- [ ] Raw data validated via `validate_recording_readiness_tool` (or existing binaries confirmed)
- [ ] Configuration file confirmed or created via `/single-recording-configuration`
- [ ] Output directory confirmed with user (required, no default)
- [ ] CPU core allocation confirmed with user
- [ ] Batch prepared or full pipeline executed
- [ ] Status monitored until all recordings complete or fail
- [ ] Failed recordings routed to appropriate skill (see Error routing)
- [ ] Successful recordings verified via `/single-recording-results`
