| name | evaluation |
| description | Systematic evaluation and comparison of technologies, tools, frameworks, and approaches using weighted criteria and scoring matrices. Use when choosing between multiple technologies, vendor/tool selection, framework comparison, library selection, or architecture pattern selection. |
Evaluation Skill
Core Principle
Evaluate with criteria, decide with data. Document the trade-offs.
Good evaluation:
- Starts with clear criteria — Define what matters before evaluating
- Uses consistent scoring — Apply the same standards to all options
- Weighs criteria by importance — Not all factors are equal
- Documents trade-offs — Every choice has costs and benefits
- Leads to defensible decisions — Data-driven, not opinion-driven
When to Use This Skill
Use this skill when:
- Choosing between multiple technologies — React vs Vue vs Angular
- Vendor/tool selection — Cloud providers, monitoring tools, CI/CD platforms
- Framework comparison — Express vs Fastify, Django vs Flask
- Library selection — State management, testing frameworks, HTTP clients
- Architecture pattern selection — Microservices vs Monolith, REST vs GraphQL
Evaluation Framework
┌────────────────────────────────────────────────────┐
│ EVALUATION PROCESS │
└────────────────────────────────────────────────────┘
1. Define Evaluation Criteria
↓
2. Assign Weights to Criteria
↓
3. Research Each Option
↓
4. Score Against Criteria
↓
5. Hands-On Testing (PoCs)
↓
6. Calculate Weighted Scores
↓
7. Document Trade-offs
↓
8. Make Recommendation
Step 1: Define Evaluation Criteria
Categories of Criteria
Technical Criteria
Features:
- Does it have required functionality?
- Are critical features mature and stable?
- Does it support our use cases?
- Are there deal-breaker missing features?
Performance:
- Throughput (requests/sec, transactions/sec)
- Latency (response times, p95, p99)
- Resource usage (CPU, memory, disk)
- Benchmark results for our workload
Scalability:
- Horizontal scaling capabilities
- Vertical scaling limits
- Load handling characteristics
- Data volume limits
Security:
- Vulnerability history
- Security update frequency
- Authentication/authorization support
- Compliance certifications (SOC2, ISO, HIPAA)
- Data encryption (at rest, in transit)
Operational Criteria
Reliability:
- Uptime history (SLA guarantees)
- Error handling capabilities
- Fault tolerance features
- Disaster recovery support
Monitoring:
- Built-in observability
- Metrics export (Prometheus, etc.)
- Logging capabilities
- Tracing support (OpenTelemetry)
Maintenance:
- Update frequency and process
- Breaking change policy
- Backward compatibility
- Upgrade complexity
Team Criteria
Learning Curve:
- Time to productivity for new developers
- Similarity to existing tools
- Conceptual complexity
- Available training materials
Documentation:
- Quality of official docs
- API reference completeness
- Tutorial and guide availability
- Example code quality
Community:
- Community size and activity
- GitHub stars/forks/activity
- Stack Overflow questions
- Forum/Discord/Slack activity
- Response time for issues
Business Criteria
Cost:
- Licensing fees (per user, per server, per transaction)
- Infrastructure costs
- Training costs
- Migration costs
- Opportunity costs
Licensing:
- License type (MIT, Apache, GPL, proprietary)
- Compatibility with our license
- Commercial use restrictions
- Attribution requirements
Vendor Lock-in:
- Portability to alternatives
- Data export capabilities
- Proprietary features vs standards
- Switching costs
Support:
- Commercial support availability
- Support response times
- Support quality
- Community vs paid support
Strategic Criteria
Ecosystem:
- Integration with other tools
- Plugin/extension marketplace
- Third-party tool support
- Industry adoption
Roadmap:
- Future feature plans
- Product vision alignment
- Investment in development
- Feature request process
Adoption Trends:
- Growing vs declining usage
- Industry momentum
- Job market demand
- Analyst recommendations
Step 2: Assign Weights
Not all criteria are equally important. Weight them based on your context.
Weighting Scale
| Weight | Priority | Meaning |
|---|
| HIGH | Critical | Deal-breaker if inadequate |
| MEDIUM | Important | Significant impact on decision |
| LOW | Nice-to-have | Minor impact on decision |
Example: Choosing a Database
| Criterion | Weight | Rationale |
|---|
| ACID guarantees | HIGH | Financial data requires consistency |
| Query performance | HIGH | Core to user experience |
| Horizontal scaling | MEDIUM | Will need in 12-18 months |
| Document flexibility | MEDIUM | Schema will evolve |
| Managed service available | MEDIUM | Small ops team |
| SQL compatibility | LOW | Team can learn new query language |
| Graph query support | LOW | Not a current requirement |
Step 3: Create Scoring Matrix
Scoring Matrix Template
| Criterion | Weight | Option A | Option B | Option C |
|---|
| Category 1 | | | | |
| Criterion 1.1 | HIGH | 8 | 6 | 9 |
| Criterion 1.2 | MEDIUM | 7 | 9 | 5 |
| Category 2 | | | | |
| Criterion 2.1 | HIGH | 9 | 7 | 8 |
| Criterion 2.2 | LOW | 6 | 8 | 7 |
| Weighted Score | | X.X | X.X | X.X |
Scoring Scale
0-10 Scoring:
- 9-10 — Excellent, exceeds requirements
- 7-8 — Good, meets requirements well
- 5-6 — Acceptable, meets minimum requirements
- 3-4 — Poor, below requirements
- 0-2 — Inadequate, deal-breaker
Visual Indicators:
- ✅ Excellent (8-10)
- ⚠️ Acceptable (5-7)
- ❌ Poor (0-4)
Step 4: Research Each Option
Research Sources
Trusted:
- ✅ Official documentation
- ✅ GitHub repository (code, issues, PRs)
- ✅ Release notes and changelog
- ✅ Production case studies
- ✅ Performance benchmarks (reputable sources)
- ✅ Security advisories and CVE database
- ✅ Team experience and feedback
Skeptical:
- ⚠️ Vendor marketing materials
- ⚠️ Paid comparison sites
- ⚠️ Outdated blog posts (>2 years old)
- ⚠️ Hype-driven articles
- ⚠️ Benchmarks by vendors
Technology Evaluation Checklist
For each option, evaluate:
Documentation Quality:
Community Health:
GitHub Activity (if open source):
Production Usage:
Integration Complexity:
Performance Characteristics:
Security:
License:
Cost:
Step 5: Hands-On Testing
Build Proof of Concepts (PoCs)
For finalists (typically 2-3 options), build focused PoCs.
PoC Scope:
- Time-boxed (1-3 days per option)
- Tests critical features only
- Validates key assumptions
- Measures actual performance
- Assesses developer experience
What to Test:
- Integration with critical systems
- Performance with realistic data
- Learning curve (time to hello world)
- Key features that matter most
- Migration complexity (if replacing existing tool)
PoC Documentation:
# PoC: [Option Name]
## Objective
[What we're validating]
## Setup Time
[How long to get running]
## Key Features Tested
- [ ] Feature 1: [Result]
- [ ] Feature 2: [Result]
- [ ] Feature 3: [Result]
## Performance Results
- Throughput: [X requests/sec]
- Latency p95: [X ms]
- Resource usage: [CPU/Memory]
## Developer Experience
- Ease of use: [Rating 1-10]
- Documentation quality: [Rating 1-10]
- Time to productivity: [X hours/days]
## Issues Encountered
- [Issue 1 and resolution]
- [Issue 2 and resolution]
## Recommendation
[Go/No-go with rationale]
See the proof-of-concept skill for detailed PoC guidance.
Step 6: Document Trade-offs
Every choice involves trade-offs. Document them explicitly.
Trade-off Template
Option A vs Option B:
| Dimension | Option A | Option B | Winner |
|---|
| Performance | 100k req/sec | 50k req/sec | A |
| Learning curve | 2 weeks | 2 days | B |
| Community size | 10k stars | 100k stars | B |
| Flexibility | High | Low | A |
| Maintenance burden | High | Low | B |
Analysis:
- Option A is 2x faster but requires 10x more learning time
- Option B has 10x larger community but less flexibility
- Trade-off: Do we optimize for performance or team velocity?
Complete Evaluation Example
Scenario: Frontend Framework Selection
Context:
- Team: 5 developers (React experience)
- Project: New e-commerce platform
- Requirements: High performance, SEO support, large ecosystem
- Timeline: 6 months to MVP
Evaluation Criteria with Weights
| Criterion | Weight | Rationale |
|---|
| Technical | | |
| Component model | HIGH | Core to development experience |
| Performance | HIGH | Critical for e-commerce |
| SSR/SSG support | HIGH | Required for SEO |
| TypeScript support | MEDIUM | Team prefers strong typing |
| Mobile support | MEDIUM | Future React Native consideration |
| Operational | | |
| Bundle size | MEDIUM | Affects page load time |
| Build tooling | LOW | All options have good tooling |
| Team | | |
| Learning curve | HIGH | Team has React experience |
| Documentation | HIGH | Need to onboard new devs |
| Community | HIGH | Need plugins and support |
| Business | | |
| Ecosystem | HIGH | Need rich plugin ecosystem |
| Job market | MEDIUM | Hiring considerations |
| Long-term viability | HIGH | 5+ year commitment |
Scoring Matrix
| Criterion | Weight | React | Vue | Svelte | Angular |
|---|
| Technical | | | | | |
| Component model | HIGH | ✅ 9 | ✅ 9 | ✅ 10 | ⚠️ 7 |
| Performance | HIGH | ✅ 8 | ✅ 8 | ✅ 10 | ⚠️ 7 |
| SSR/SSG support | HIGH | ✅ 10 | ✅ 9 | ✅ 8 | ✅ 9 |
| TypeScript | MEDIUM | ✅ 9 | ⚠️ 7 | ✅ 9 | ✅ 10 |
| Mobile support | MEDIUM | ✅ 10 | ⚠️ 6 | ❌ 3 | ⚠️ 7 |
| Operational | | | | | |
| Bundle size | MEDIUM | ⚠️ 6 | ✅ 8 | ✅ 10 | ❌ 4 |
| Build tooling | LOW | ✅ 9 | ✅ 8 | ✅ 8 | ✅ 9 |
| Team | | | | | |
| Learning curve | HIGH | ✅ 10 | ✅ 8 | ⚠️ 7 | ⚠️ 6 |
| Documentation | HIGH | ✅ 10 | ✅ 9 | ⚠️ 7 | ✅ 9 |
| Community | HIGH | ✅ 10 | ✅ 8 | ⚠️ 6 | ✅ 9 |
| Business | | | | | |
| Ecosystem | HIGH | ✅ 10 | ✅ 8 | ⚠️ 5 | ✅ 9 |
| Job market | MEDIUM | ✅ 10 | ⚠️ 7 | ⚠️ 5 | ✅ 8 |
| Long-term | HIGH | ✅ 10 | ✅ 8 | ⚠️ 6 | ✅ 9 |
| Weighted Score | | 9.2 | 7.9 | 7.0 | 7.8 |
Legend:
- ✅ Excellent (8-10)
- ⚠️ Acceptable (5-7)
- ❌ Poor (0-4)
Detailed Analysis
React (Score: 9.2):
Strengths:
- Team already knows React (minimal learning curve)
- Largest ecosystem (components, tools, libraries)
- Best SSR/SSG story (Next.js is industry standard)
- React Native path for mobile app
- Massive community and job market
- Proven long-term viability
Weaknesses:
- Larger bundle size than Vue/Svelte
- Performance slightly behind Svelte
- More boilerplate than Svelte
PoC Findings:
- Setup time: 30 minutes with Create React App
- First component: 1 hour
- Full CRUD page: 4 hours
- Next.js SSG working out of the box
- Found excellent UI library (shadcn/ui)
Vue (Score: 7.9):
Strengths:
- Excellent documentation
- Great developer experience
- Good SSR support (Nuxt.js)
- Smaller bundle size than React
- Strong community
Weaknesses:
- Team has no Vue experience
- Smaller ecosystem than React
- TypeScript support improving but not as mature
- React Native vs Vue Native comparison
PoC Findings:
- Setup time: 45 minutes
- Learning curve steeper than expected for React developers
- Composition API similar to React hooks
- Found most needed plugins
Svelte (Score: 7.0):
Strengths:
- Best performance (compiles to vanilla JS)
- Smallest bundle size
- Elegant syntax, minimal boilerplate
- Growing momentum
Weaknesses:
- Smallest ecosystem
- Team has no experience
- SSR story (SvelteKit) less mature than Next.js
- No mobile story
- Smaller job market
PoC Findings:
- Setup time: 1 hour
- Syntax very different from React
- Impressed by performance
- Harder to find UI components
- SvelteKit less documented than Next.js
Angular (Score: 7.8):
Strengths:
- Complete framework (batteries included)
- Enterprise backing (Google)
- Excellent TypeScript support
- Strong patterns and structure
Weaknesses:
- Steep learning curve (RxJS, dependency injection)
- Largest bundle size
- Team has no Angular experience
- Less flexibility than React/Vue
PoC Not Pursued:
- Eliminated after research phase
- Learning curve too steep for timeline
- Team preference for React-like model
Trade-offs Summary
React vs Vue:
- React: Better ecosystem, team knowledge, mobile path
- Vue: Slightly better performance, smaller bundle
- Winner: React (team velocity and ecosystem outweigh minor performance gains)
React vs Svelte:
- Svelte: Best performance, cleanest code
- React: Mature ecosystem, team knowledge, proven at scale
- Winner: React (maturity and team velocity critical for 6-month timeline)
Recommendation
Recommended: React with Next.js
Rationale:
- Zero learning curve — Team already knows React
- Best ecosystem — Largest selection of components and tools
- Proven SSR/SSG — Next.js is production-ready and well-documented
- Future mobile app — React Native provides clear path
- Hiring advantage — Largest React job market
- Long-term safety — React is here to stay
Risks & Mitigations:
- Risk: Larger bundle size than alternatives
- Mitigation: Use Next.js code splitting, lazy loading, bundle analysis
- Risk: Team could become React-only
- Mitigation: Encourage learning Vue/Svelte for side projects
Implementation Plan:
- Setup Next.js project with TypeScript (Week 1)
- Choose and configure UI library (Week 1)
- Setup testing framework (Week 1-2)
- Build design system (Week 2-3)
- Begin feature development (Week 3+)
Estimated Effort: 1 week setup, team productive immediately
Common Evaluation Scenarios
1. Database Selection
Key Criteria:
- Data model fit (relational, document, graph, time-series)
- ACID vs eventual consistency requirements
- Query patterns and performance
- Horizontal scaling needs
- Managed service availability
- Team expertise
Example Comparison:
- PostgreSQL: ACID, complex queries, good scaling, mature
- MongoDB: Flexible schema, horizontal scaling, eventual consistency
- DynamoDB: Fully managed, massive scale, limited query flexibility
- Cassandra: Write-heavy, linear scaling, eventual consistency
2. Cloud Provider Selection
Key Criteria:
- Service offerings for your use case
- Regional availability and latency
- Pricing model and total cost
- Team expertise and learning curve
- Vendor lock-in concerns
- Compliance certifications
- Support quality
Example Comparison:
- AWS: Most complete offering, most complex, largest ecosystem
- Azure: Best for Microsoft shops, good enterprise features
- GCP: Excellent for data/ML, strong Kubernetes support, competitive pricing
- DigitalOcean: Simplest, lower cost, fewer features
3. Testing Framework
Key Criteria:
- Language/framework support
- Test types supported (unit, integration, e2e)
- Speed of test execution
- Mocking capabilities
- Assertion library quality
- IDE integration
- Learning curve
Example Comparison (JavaScript):
- Jest: Complete solution, React-friendly, good DX
- Vitest: Faster than Jest, Vite integration, modern
- Mocha: Flexible, mature, requires more setup
- Playwright: Best for e2e, multi-browser support
4. CI/CD Platform
Key Criteria:
- Integration with version control
- Pipeline flexibility
- Performance (build speed)
- Cost (minutes, concurrent jobs)
- Self-hosted vs cloud
- Secret management
- Marketplace/plugins
Example Comparison:
- GitHub Actions: Tight GitHub integration, generous free tier
- GitLab CI: All-in-one platform, self-hosted option
- Jenkins: Fully self-hosted, maximum flexibility, maintenance burden
- CircleCI: Fast, good DX, credit-based pricing
Decision Documentation
Link to ADR
For significant decisions, create an Architecture Decision Record (ADR).
See the adr skill for ADR templates and workflow.
ADR Quick Template
# ADR-XXX: [Decision Title]
## Status
Accepted
## Context
[Why we needed to evaluate options]
[Constraints and requirements]
[Timeline and stakeholders]
## Options Considered
1. Option A
2. Option B
3. Option C
## Evaluation
[Link to full evaluation document or scoring matrix]
## Decision
We chose [Option X] because [key reasons].
## Consequences
**Positive:**
- [Benefit 1]
- [Benefit 2]
**Negative:**
- [Trade-off 1] — Mitigated by [mitigation]
- [Trade-off 2] — Accepted because [rationale]
## Follow-up Actions
- [ ] Action 1
- [ ] Action 2
Evaluation Document Template
# Technology Evaluation: [Topic]
**Date:** YYYY-MM-DD
**Author:** [Your name]
**Reviewers:** [Team members who reviewed]
**Status:** [Draft | Under Review | Approved]
---
## Executive Summary
**Decision needed:** [One sentence]
**Recommendation:** [Option X]
**Key reason:** [Primary rationale in one sentence]
---
## Context
**Problem:**
[What problem are we solving?]
**Requirements:**
- [Requirement 1]
- [Requirement 2]
- [Requirement 3]
**Constraints:**
- [Constraint 1: e.g., Budget]
- [Constraint 2: e.g., Timeline]
- [Constraint 3: e.g., Team size]
**Stakeholders:**
- [Stakeholder 1: Interest]
- [Stakeholder 2: Interest]
**Decision deadline:** [Date]
---
## Evaluation Criteria
| Criterion | Weight | Rationale |
|-----------|--------|-----------|
| [Criterion 1] | HIGH | [Why critical] |
| [Criterion 2] | MEDIUM | [Why important] |
| [Criterion 3] | LOW | [Why nice-to-have] |
---
## Options Evaluated
### Option A: [Name]
**Description:** [Brief overview]
**Pros:**
- [Pro 1]
- [Pro 2]
- [Pro 3]
**Cons:**
- [Con 1]
- [Con 2]
- [Con 3]
**Score:** X.X / 10
**PoC findings:** [Link or summary]
---
### Option B: [Name]
[Same structure as Option A]
---
### Option C: [Name]
[Same structure as Option A]
---
## Scoring Matrix
[Include full scoring matrix from evaluation]
---
## Trade-offs Analysis
[Document key trade-offs between top options]
---
## Recommendation
**Recommended:** [Option X]
**Rationale:**
1. [Key reason 1]
2. [Key reason 2]
3. [Key reason 3]
**Risks & Mitigations:**
- **Risk:** [Potential issue]
- **Mitigation:** [How we'll handle it]
- **Risk:** [Another concern]
- **Mitigation:** [Response plan]
---
## Implementation Plan
### Phase 1: Setup (Timeline)
- [ ] Task 1
- [ ] Task 2
### Phase 2: Migration (Timeline)
- [ ] Task 1
- [ ] Task 2
### Phase 3: Validation (Timeline)
- [ ] Task 1
- [ ] Task 2
**Total estimated effort:** [Time]
---
## References
- [Official documentation]
- [Benchmark source]
- [PoC repository]
- [Case study]
- [Related ADR]
---
## Appendix: Detailed Scoring
[Include research notes, detailed scores, calculations]
Evaluation Anti-Patterns
1. Resume-Driven Evaluation
❌ Anti-pattern: Choosing technology because it's trendy or looks good on resume
Signs:
- "Everyone's talking about [new tech]"
- Ignoring that current solution works well
- Chasing hype cycles
- Complexity for complexity's sake
Fix: Evaluate based on actual needs, not trends
2. Analysis Paralysis
❌ Anti-pattern: Evaluating forever, never deciding
Signs:
- Evaluating 10+ options
- Adding criteria endlessly
- "Let's research more" when data is sufficient
- Perfectionism blocking decision
Fix: Set decision deadline, limit options to 3-5, accept "good enough"
3. Confirmation Bias
❌ Anti-pattern: Deciding first, evaluating to confirm
Signs:
- Only researching preferred option deeply
- Ignoring negative evidence
- Scoring subjectively to favor preference
- Cherry-picking benchmarks
Fix: Score all options before forming opinion, blind evaluation where possible
4. Sunk Cost Fallacy
❌ Anti-pattern: Continuing with poor choice due to past investment
Signs:
- "We've already invested 6 months"
- Ignoring better alternatives
- Escalating commitment despite evidence
- Defending past decision
Fix: Evaluate current state objectively, past investment is sunk
5. Feature Checklist Evaluation
❌ Anti-pattern: Choosing option with most features
Signs:
- Counting features without weighting importance
- Ignoring that 80% of features won't be used
- Missing quality vs quantity distinction
- Complexity without benefit
Fix: Weight features by actual need, value simplicity
Integration with Other Skills
With Proof-of-Concept
Evaluation identifies options, PoC validates top candidates.
Workflow:
- Evaluate → Narrow to 2-3 finalists
- PoC → Test finalists hands-on
- Update scores → Based on PoC findings
- Decide → With data from both evaluation and PoC
See proof-of-concept skill.
With Technical Research
Research gathers information, evaluation structures decision-making.
Workflow:
- Research → Understand the landscape
- Evaluate → Score options systematically
- Document → Create decision record
See technical-research skill.
With Architecture Review
Evaluation informs architecture decisions, review validates them.
Workflow:
- Evaluate → Choose technology/pattern
- Design → Create architecture
- Review → Validate architectural decisions
See architecture-review skill.
With ADR
Evaluation provides the data, ADR documents the decision.
Workflow:
- Evaluate → Create scoring matrix
- Decide → Make recommendation
- ADR → Record decision with context
See adr skill.
With Technical Lead Role
Tech leads use evaluation to make informed decisions.
Workflow:
- Tech lead identifies need for evaluation
- Team collaborates on criteria and weights
- Tech lead facilitates evaluation process
- Team reviews findings
- Tech lead makes final decision
See technical-lead-role skill.
Quick Reference
Evaluation Process Checklist
Planning:
Research:
Scoring:
Validation:
Decision:
Evaluation Criteria Quick List
Technical: Features, performance, scalability, security
Operational: Reliability, monitoring, maintenance
Team: Learning curve, documentation, community
Business: Cost, licensing, vendor lock-in, support
Strategic: Ecosystem, roadmap, adoption trends
Scoring Guidelines
- 9-10: Excellent, exceeds requirements
- 7-8: Good, meets requirements well
- 5-6: Acceptable, meets minimum
- 3-4: Poor, below requirements
- 0-2: Inadequate, deal-breaker
When to Re-evaluate
- New major version released
- Significant change in requirements
- Team size or expertise changes
- Better alternatives emerge
- Performance degrades
- Security issues discovered
- Vendor acquired or direction changes
- Every 1-2 years for critical dependencies
Remember: Good evaluation leads to good decisions. Define criteria before scoring, weight by importance, validate with hands-on testing. Document trade-offs transparently so future you (and your team) understands why the decision was made. Every choice has costs—make them explicit.
