This skill covers the principles for identifying tasks suited to LLM processing, designing effective project architectures, and iterating rapidly using agent-assisted development.
Installer avec Codex ou Claude Copiez ce prompt, collez-le dans Codex, Claude ou un autre assistant, puis laissez-le vérifier la page du skill et l'installer pour vous.
Une commande directe contourne le prompt de vérification. Examinez la source avant de l'exécuter.
This skill covers the principles for identifying tasks suited to LLM processing, designing effective project architectures, and iterating rapidly using agent-assisted development.
type
skill
created
2026-02-27T00:00:00.000Z
domain
ai-ml
category
llm-agents
risk
unknown
source
community
tags
["skill","ai-ml","llm-agents","project"]
Project Development Methodology
This skill covers the principles for identifying tasks suited to LLM processing, designing effective project architectures, and iterating rapidly using agent-assisted development. The methodology applies whether building a batch processing pipeline, a multi-agent research system, or an interactive agent application.
When to Use
Activate this skill when:
Starting a new project that might benefit from LLM processing
Evaluating whether a task is well-suited for agents versus traditional code
Designing the architecture for an LLM-powered application
Planning a batch processing pipeline with structured outputs
Choosing between single-agent and multi-agent approaches
Estimating costs and timelines for LLM-heavy projects
Core Concepts
Task-Model Fit Recognition
Not every problem benefits from LLM processing. The first step in any project is evaluating whether the task characteristics align with LLM strengths. This evaluation should happen before writing any code.
LLM-suited tasks share these characteristics:
Characteristic
Why It Fits
Synthesis across sources
LLMs excel at combining information from multiple inputs
Subjective judgment with rubrics
LLMs handle grading, evaluation, and classification with criteria
Natural language output
When the goal is human-readable text, not structured data
Error tolerance
Individual failures do not break the overall system
Batch processing
No conversational state required between items
Domain knowledge in training
The model already has relevant context
LLM-unsuited tasks share these characteristics:
Characteristic
Why It Fails
Precise computation
Math, counting, and exact algorithms are unreliable
Real-time requirements
LLM latency is too high for sub-second responses
Perfect accuracy requirements
Hallucination risk makes 100% accuracy impossible
Proprietary data dependence
The model lacks necessary context
Sequential dependencies
Each step depends heavily on the previous result
Deterministic output requirements
Same input must produce identical output
The evaluation should happen through manual prototyping: take one representative example and test it directly with the target model before building any automation.
The Manual Prototype Step
Before investing in automation, validate task-model fit with a manual test. Copy one representative input into the model interface. Evaluate the output quality. This takes minutes and prevents hours of wasted development.
This validation answers critical questions:
Does the model have the knowledge required for this task?
Can the model produce output in the format you need?
What level of quality should you expect at scale?
Are there obvious failure modes to address?
If the manual prototype fails, the automated system will fail. If it succeeds, you have a baseline for comparison and a template for prompt design.
Pipeline Architecture
LLM projects benefit from staged pipeline architectures where each stage is:
Discrete: Clear boundaries between stages
Idempotent: Re-running produces the same result
Cacheable: Intermediate results persist to disk
Independent: Each stage can run separately
The canonical pipeline structure:
acquire → prepare → process → parse → render
Acquire: Fetch raw data from sources (APIs, files, databases)
Prepare: Transform data into prompt format
Process: Execute LLM calls (the expensive, non-deterministic step)
Parse: Extract structured data from LLM outputs
Render: Generate final outputs (reports, files, visualizations)
Stages 1, 2, 4, and 5 are deterministic. Stage 3 is non-deterministic and expensive. This separation allows re-running the expensive LLM stage only when necessary, while iterating quickly on parsing and rendering.
File System as State Machine
Use the file system to track pipeline state rather than databases or in-memory structures. Each processing unit gets a directory. Each stage completion is marked by file existence.
To check if an item needs processing: check if the output file exists. To re-run a stage: delete its output file and downstream files. To debug: read the intermediate files directly.
This pattern provides:
Natural idempotency (file existence gates execution)
Easy debugging (all state is human-readable)
Simple parallelization (each directory is independent)
Trivial caching (files persist across runs)
Structured Output Design
When LLM outputs must be parsed programmatically, prompt design directly determines parsing reliability. The prompt must specify exact format requirements with examples.
Effective structure specification includes:
Section markers: Explicit headers or prefixes for parsing
Format examples: Show exactly what output should look like
Rationale disclosure: "I will be parsing this programmatically"
Analyze the following and provide your response in exactly this format:
## Summary
[Your summary here]
## Score
Rating: [1-10]
## Details
- Key point 1
- Key point 2
Follow this format exactly because I will be parsing it programmatically.
The parsing code must handle variations gracefully. LLMs do not follow instructions perfectly. Build parsers that:
Use regex patterns flexible enough to handle minor formatting variations
Provide sensible defaults when sections are missing
Log parsing failures for later review rather than crashing
Agent-Assisted Development
Modern agent-capable models can accelerate development significantly. The pattern is:
Describe the project goal and constraints
Let the agent generate initial implementation
Test and iterate on specific failures
Refine prompts and architecture based on results
This is about rapid iteration: generate, test, fix, repeat. The agent handles boilerplate and initial structure while you focus on domain-specific requirements and edge cases.
Key practices for effective agent-assisted development:
Provide clear, specific requirements upfront
Break large projects into discrete components
Test each component before moving to the next
Keep the agent focused on one task at a time
Cost and Scale Estimation
LLM processing has predictable costs that should be estimated before starting. The formula:
Total cost = (items × tokens_per_item × price_per_token) + API overhead
For batch processing:
Estimate input tokens per item (prompt + context)
Estimate output tokens per item (typical response length)
Multiply by item count
Add 20-30% buffer for retries and failures
Track actual costs during development. If costs exceed estimates significantly, re-evaluate the approach. Consider:
Reducing context length through truncation
Using smaller models for simpler items
Caching and reusing partial results
Parallel processing to reduce wall-clock time (not token cost)
Detailed Topics
Choosing Single vs Multi-Agent Architecture
Single-agent pipelines work for:
Batch processing with independent items
Tasks where items do not interact
Simpler cost and complexity management
Multi-agent architectures work for:
Parallel exploration of different aspects
Tasks exceeding single context window capacity
When specialized sub-agents improve quality
The primary reason for multi-agent is context isolation, not role anthropomorphization. Sub-agents get fresh context windows for focused subtasks. This prevents context degradation on long-running tasks.
See multi-agent-patterns skill for detailed architecture guidance.
Architectural Reduction
Start with minimal architecture. Add complexity only when proven necessary. Production evidence shows that removing specialized tools often improves performance.
Vercel's d0 agent achieved 100% success rate (up from 80%) by reducing from 17 specialized tools to 2 primitives: bash command execution and SQL. The file system agent pattern uses standard Unix utilities (grep, cat, find, ls) instead of custom exploration tools.
When reduction outperforms complexity:
Your data layer is well-documented and consistently structured
The model has sufficient reasoning capability
Your specialized tools were constraining rather than enabling
You are spending more time maintaining scaffolding than improving outcomes
When complexity is necessary:
Your underlying data is messy, inconsistent, or poorly documented
The domain requires specialized knowledge the model lacks
Operations are truly complex and benefit from structured workflows
See tool-design skill for detailed tool architecture guidance.
Iteration and Refactoring
Expect to refactor. Production agent systems at scale require multiple architectural iterations. Manus refactored their agent framework five times since launch. The Bitter Lesson suggests that structures added for current model limitations become constraints as models improve.
Build for change:
Keep architecture simple and unopinionated
Test across model strengths to verify your harness is not limiting performance
Design systems that benefit from model improvements rather than locking in limitations
Practical Guidance
Project Planning Template
Task Analysis
What is the input? What is the desired output?
Is this synthesis, generation, classification, or analysis?
What error rate is acceptable?
What is the value per successful completion?
Manual Validation
Test one example with target model
Evaluate output quality and format
Identify failure modes
Estimate tokens per item
Architecture Selection
Single pipeline vs multi-agent
Required tools and data sources
Storage and caching strategy
Parallelization approach
Cost Estimation
Items × tokens × price
Development time
Infrastructure requirements
Ongoing operational costs
Development Plan
Stage-by-stage implementation
Testing strategy per stage
Iteration milestones
Deployment approach
Anti-Patterns to Avoid
Skipping manual validation: Building automation before verifying the model can do the task wastes significant time when the approach is fundamentally flawed.
Monolithic pipelines: Combining all stages into one script makes debugging and iteration difficult. Separate stages with persistent intermediate outputs.
Over-constraining the model: Adding guardrails, pre-filtering, and validation logic that the model could handle on its own. Test whether your scaffolding helps or hurts.
Ignoring costs until production: Token costs compound quickly at scale. Estimate and track from the beginning.
Perfect parsing requirements: Expecting LLMs to follow format instructions perfectly. Build robust parsers that handle variations.
Premature optimization: Adding caching, parallelization, and optimization before the basic pipeline works correctly.
Examples
Example 1: Batch Analysis Pipeline (Karpathy's HN Time Capsule)
Task: Analyze 930 HN discussions from 10 years ago with hindsight grading.
