la-bench-procedure-generator

name

la-bench-procedure-generator

description

This skill should be used when generating detailed experimental procedures from LA-Bench format JSONL files. It orchestrates multiple subagents to parse input data, fetch reference materials, generate procedures, validate outputs, refine results, and produce final formatted outputs. Triggered by requests to process LA-Bench data or generate experimental protocols from data/public_test.jsonl or data/private_test_input.jsonl files.

LA-Bench Procedure Generator

Overview

This skill orchestrates a multi-agent workflow to generate detailed experimental procedures from LA-Bench format JSONL input files. Instead of generating procedures in a single step, it coordinates specialized subagents for parsing, reference fetching, generation, validation, refinement, and final output creation.

When to Use This Skill

• When the user requests to generate experimental procedures from LA-Bench data
• When processing files like data/public_test.jsonl or data/private_test_input.jsonl
• When the user asks to "process LA-Bench format" or "generate detailed experimental protocols"

Core Workflow

This skill follows a workflow-based orchestration pattern with six distinct phases:

Phase 0: Initialize

1. Create TODO list using TodoWrite tool to track all phases
2. Verify input/output paths:
   • Input: data/public_test.jsonl or data/private_test_input.jsonl
   • Output: outputs/runs/generated_YYYYMMDD_HHMMSS.jsonl
3. Set up workspace for intermediate results if needed

Phase 1: Data Acquisition (Parallel Execution)

Launch multiple Task tools in parallel to maximize efficiency:

Task 1: JSONL Parser Agent

Prompt: "Parse the JSONL file at [path] and extract all entries.
Return a list of all entries with their id, input, and output fields."

• Input: File path to JSONL
• Output: List of all entries
• See: references/agent_specs.md#parser for detailed specifications

Task 2: Reference Fetcher Agent (uses web-reference-fetcher skill)

Prompt: "Use the web-reference-fetcher skill to fetch content from
all reference URLs found in the JSONL entries."

• Uses existing web-reference-fetcher skill
• Input: List of reference URLs from all entries
• Output: Fetched reference content for each URL
• See: references/agent_specs.md#reference-fetcher

Task 3: Procedure Generator Agent (one per entry or batched)

Prompt: "Generate detailed procedure_steps for entry [id] using:
- instruction
- mandatory_objects
- source_protocol_steps
- fetched reference content
Output format: List of {id: int, text: str} objects"

• Input: Single entry data + fetched references
• Output: Generated procedure_steps
• See: references/agent_specs.md#generator

Phase 2: Quality Validation

Task 4: Checker Agent

Prompt: "Validate the generated procedures against quality criteria
in references/quality_criteria.md. Check:
- Output format compliance
- Logical consistency
- Completeness
Report any issues found."

• Input: All generated procedure_steps
• Output: Validation report with issues (if any)
• See: references/agent_specs.md#checker

Phase 3: Refinement (Conditional)

If validation finds issues:

Task 5: Refiner Agent

Prompt: "Address the following validation issues: [issues].
Regenerate or fix the affected procedure_steps."

• Input: Validation issues + original data
• Output: Corrected procedure_steps
• See: references/agent_specs.md#refiner

Phase 4: Final Output

Task 6: Output Generator Agent

Prompt: "Format all validated procedure_steps into LA-Bench output format
and save to outputs/runs/generated_[timestamp].jsonl.
Each line should be: {id: string, output: {procedure_steps: [...]}}
Use assets/output_schema.json as reference."

• Input: All validated/refined procedure_steps
• Output: Final JSONL file
• See: references/agent_specs.md#output

Important Notes

Data Flow

• All entries in the JSONL are processed (loop through all IDs)
• Data passes between agents through shared workspace or direct handoff
• See references/data_flow.md for detailed inter-agent communication patterns

TODO Management

• Update TODO status after each phase completion
• Mark agents as in_progress when launching
• Mark as completed only when phase is fully done

Parallel vs Sequential

• Phase 1 agents run in parallel (use single message with multiple Task calls)
• Phases 2-4 run sequentially (each depends on previous completion)

Error Handling

• If any agent fails, document the failure and retry with adjusted prompt
• If persistent failures occur, consult references/agent_specs.md for troubleshooting

Example Session

See references/example_session.md for a complete walkthrough of a typical execution.

Resources

references/

Documentation loaded into context as needed:

• agent_specs.md: Detailed specifications for each subagent (prompts, inputs, outputs, implementation guidelines)
• data_flow.md: How data passes between agents, workspace structure, and file formats
• example_session.md: Real example of a complete workflow execution with agent interactions

assets/

Files used in final output:

• output_schema.json: JSON schema for the final output format, ensures compliance with LA-Bench expected format