القائمة
Framework for assessing evidence quality in decision-making
التثبيت باستخدام Codex أو Claude انسخ هذا Prompt والصقه في Codex أو Claude أو مساعد آخر ليراجع صفحة Skill ويثبّتها لك.
استنادا إلى تصنيف SOC المهني
Introduction to the Deliberate Decisions framework - use when helping humans make high-stakes decisions
Gate 4 - Consolidate Facts vs Assumptions, update Knowns/Unknowns, create assumption inventory
Gate 5 - Pre-mortem, steel-man opposition, surface biases, second-order effects
Gate 7 - Document final decision, rationale, and commit to artifact
Gate 3 - Web search, document analysis, reference class, gather evidence
Gate 2 - Surface Decision Points, Intersections, alternatives, and initial Knowns/Unknowns
| name | evidence-evaluation |
| description | Framework for assessing evidence quality in decision-making |
| version | 1.0.0 |
| type | supporting |
| dependencies | [] |
| triggers | ["evaluating claims or assertions","assessing data quality","reviewing research or reports","fact-checking assumptions","building arguments with evidence"] |
A systematic framework for assessing the quality and reliability of evidence used in decision-making.
Evidence-based decisions require more than just having data - they require understanding how trustworthy that data is. This skill provides tools to distinguish facts from assumptions, evaluate source quality, and document evidence systematically.
Understanding the difference between facts and assumptions is fundamental to evidence evaluation.
Definition: Verifiable statements supported by direct observation, measurement, or documentation.
Characteristics:
Examples:
| Statement | Why It's a Fact |
|---|---|
| "Server response time averaged 245ms over 1000 requests" | Measured and logged |
| "The contract expires on March 15, 2026" | Documented in signed agreement |
| "Three customers reported this bug in the last week" | Tracked in support system |
| "The test suite has 847 passing tests" | Verified by running tests |
Definition: Beliefs or expectations accepted as true without direct verification.
Characteristics:
Examples:
| Statement | Why It's an Assumption |
|---|---|
| "Users prefer the new design" | No user research conducted |
| "Performance will improve with more servers" | Not tested in this context |
| "The competitor will respond with a price cut" | Prediction, not fact |
| "The team can deliver this in two sprints" | Estimate, not commitment |
When an assumption is critical to a decision, consider how to verify it:
Assumption: "Most users access the app on mobile"
-> Verification: Check analytics data for device breakdown
-> Result: "68% of sessions are from mobile devices" (fact)
Assumption: "The API can handle 10x current load"
-> Verification: Run load tests
-> Result: "API failed at 7x load with memory exhaustion" (fact)
Assumption: "Customers want feature X"
-> Verification: Conduct user interviews or surveys
-> Result: "23 of 30 interviewed customers expressed need for X" (fact)
Evaluate evidence sources across four dimensions, each rated High/Medium/Low.
Assesses the trustworthiness of the information source.
| Rating | Criteria | Examples |
|---|---|---|
| High | Expert in relevant domain; established track record; no conflicts of interest; peer-reviewed or audited | Academic research, official documentation, recognized industry experts, audited financial statements |
| Medium | Knowledgeable but not expert; some track record; minor potential conflicts; reviewed but not rigorously | Industry reports, experienced practitioners, trade publications, internal analysis |
| Low | Unknown expertise; no track record; significant conflicts of interest; unreviewed | Anonymous sources, marketing materials, unverified claims, self-reported without validation |
Questions to Ask:
Assesses how the information was gathered or conclusions were reached.
| Rating | Criteria | Examples |
|---|---|---|
| High | Rigorous, documented methodology; appropriate sample size; controlled conditions; reproducible | Scientific studies, comprehensive testing, statistical analysis with proper controls |
| Medium | Reasonable methodology with some gaps; moderate sample size; some documentation | Surveys with decent response rates, pilot tests, case studies with multiple examples |
| Low | Unclear or flawed methodology; small or biased samples; anecdotal; not reproducible | Single anecdotes, informal polls, untested hypotheses, "common knowledge" |
Questions to Ask:
Assesses how current the information is relative to the decision context.
| Rating | Criteria | Examples |
|---|---|---|
| High | Current or real-time; collected within relevant timeframe; accounts for recent changes | Live metrics, recent market research, current documentation, data from this quarter |
| Medium | Reasonably recent; may not reflect latest changes; still largely applicable | Last year's data, recent but not current reports, slightly outdated documentation |
| Low | Outdated; from before significant changes; may no longer apply | Multi-year-old research, pre-pivot data, documentation from deprecated systems |
Questions to Ask:
Assesses how directly the information applies to the specific decision.
| Rating | Criteria | Examples |
|---|---|---|
| High | Directly addresses the question; same context and conditions; specific to this situation | Data from this system, research on this exact problem, feedback from target users |
| Medium | Related but not exact; similar context; requires some extrapolation | Industry benchmarks, analogous case studies, research on similar problems |
| Low | Tangentially related; different context; significant extrapolation required | General statistics, research from different domains, opinions on different problems |
Questions to Ask:
Combine the four dimension ratings into an overall quality assessment.
Assign points to each rating:
Calculate total (max 12 points):
| Total Score | Overall Quality | Guidance |
|---|---|---|
| 10-12 | Strong Evidence | Can rely on for important decisions |
| 7-9 | Moderate Evidence | Useful but seek corroboration |
| 4-6 | Weak Evidence | Use cautiously; actively seek better sources |
| 1-3 | Poor Evidence | Do not rely on; treat as unverified assumption |
Evidence: "Industry report says 40% of companies are adopting this technology"
| Dimension | Rating | Rationale |
|---|---|---|
| Credibility | Medium | Reputable firm but funded by vendor |
| Methodology | Medium | Survey of 500 companies; self-reported |
| Recency | High | Published this quarter |
| Relevance | Medium | Includes our industry but broader scope |
Score: 2 + 2 + 3 + 2 = 9 (Moderate Evidence)
Conclusion: Useful directional indicator but seek additional sources before major commitments.
Warning signs that evidence may be unreliable:
Use this template to document and evaluate evidence systematically.
## Evidence Record
### Claim
[What assertion does this evidence support?]
### Source
- **Origin**: [Where did this evidence come from?]
- **Author/Creator**: [Who produced it?]
- **Date**: [When was it created/published?]
- **URL/Reference**: [How to find the original]
### Quality Assessment
| Dimension | Rating | Rationale |
|-----------|--------|-----------|
| Credibility (Who) | High/Medium/Low | [Why?] |
| Methodology (How) | High/Medium/Low | [Why?] |
| Recency (When) | High/Medium/Low | [Why?] |
| Relevance (What) | High/Medium/Low | [Why?] |
**Overall Score**: [X/12] - [Strong/Moderate/Weak/Poor]
### Red Flags Identified
- [ ] Conflict of interest
- [ ] Cherry-picking
- [ ] Missing methodology
- [ ] Survivorship bias
- [ ] Correlation as causation
- [ ] Appeal to authority
- [ ] Outdated context
- [ ] Sample issues
- [ ] Other: [specify]
### Notes
[Additional context, caveats, or observations]
### Recommendation
[How should this evidence be used in the decision?]