mindit-confidence

name

mindit-confidence

description

Analyze the evidence standing behind a design decision — research, data, intuition, or assumption. Use this whenever the user asks "how do we know this is right," wants a research review, asks about user research, A/B test results, or interviews. Also trigger when the user says "I think users want X" or "we believe Y" — the skill surfaces what the team actually knows vs. what they're assuming. Produces a structured decision artifact with score, criteria, and anti-pattern matches.

mindit-confidence

The seventh of the eight forces. Run this when the question is "how do we actually know."

The force

Every design decision is backed by something. Sometimes that something is real research — interviews, observations, data, prior experiments. Sometimes it is convention — "this is how everyone does it." Sometimes it is intuition — "I think users would want this." Sometimes it is assumption — "we believe users will."

These are not equal. Confidence is highest with triangulated, fresh, broad, well-mechanized evidence. Confidence is lowest with single-source, stale, narrow, weakly-mechanized claims, or with no claims at all.

Confidence asks: where on this spectrum does this decision actually live?

The goal is not to require evidence for every decision. The goal is to know which decisions you are flying blind on, so you can decide whether to invest in research or move forward on conviction.

When to run this

• The user states a belief about users ("users want X," "users will not Y," "users prefer Z").
• The user mentions research, interviews, A/B tests, surveys, analytics, or "what the data says."
• The user has just shipped or is about to ship a hypothesis-driven design.
• The user uses phrases like "we know," "we believe," "I think," "users tend to."

How to analyze

1. Surface the claims. Walk through the design and extract every claim about users it implicitly or explicitly makes. ("Users want fewer steps," "users skim before reading," "users will trust this badge.")

2. For each claim, ask three questions:

   • What evidence backs this? Research, data, convention, intuition, assumption.
   • How fresh is the evidence? When was it produced? Has the user base changed since?
   • How broad is the evidence? One user, ten, thousands? One segment or all segments?

3. Score the criteria below. Each criterion measures one dimension of evidence quality.

4. Distinguish "thin evidence" from "no evidence." Both are valid findings but they get different fixes. Thin evidence can be strengthened. No evidence requires deciding whether to invest in research or proceed on conviction.

5. Check anti-patterns.yaml.

6. Write the artifact.

Rubric

Criterion	Weight	What you are scoring
Evidence type	0.25	Research-backed (interviews, observed behavior, data) → strong. Convention → medium. Intuition → low. Assumption stated as fact → very low.
Evidence freshness	0.20	When was the evidence produced relative to the current user base? Research more than 18 months old in a fast-changing product is stale.
Evidence breadth	0.20	One participant or many? One segment or all? A single power user's preference is not the audience.
Mechanism of evidence	0.20	How was the evidence gathered? Observation > self-report. Behavioral data > stated preferences. Interview > survey.
Triangulation	0.15	Does multiple, independent evidence point the same way? Or is the decision resting on a single source?

Anti-patterns

See anti-patterns.yaml. Brief list: vibes-driven-design, old-research, n-of-one, borrowed-confidence, selection-bias, hippo-decision, post-rationalization, conflating-stated-with-observed.

Output format

Two files in .mindit/decisions/. Filename: confidence-<yyyymmdd>-<slug>.{md,json}.

JSON conforms to schemas/decision.schema.json with skill: "mindit-confidence" and force: "confidence". The evidence array should be populated whenever the user has cited research.

Example

Input: "We're going to make the signup flow three steps instead of one. Users said in interviews they want simpler signup."

Extracted claims:

• "Users want simpler signup."
• "Three steps is simpler than one step." (implicit)
• "The current one-step flow is failing because of complexity." (implicit)

Sample findings:

• Evidence type: interview-based, stated preference. Match: conflating-stated-with-observed. Users said "simpler," not "three steps." Severity high.
• Evidence freshness: interviews are six weeks old. Acceptable. Severity info.
• Evidence breadth: five participants. Single segment. Match: n-of-one if interpretation is broad. Severity medium.
• Mechanism: self-report only. No observation, no behavioral data, no funnel analysis on the current one-step flow. Severity high.
• Triangulation: zero. The current funnel data has not been checked. A/B test not designed. Severity high.

Sample weighted score: 33.

Suggested next action: pull the current one-step funnel before committing to a three-step redesign. The stated cause may be wrong.

What this skill does NOT do

• Does not require all decisions to have research. Many are correctly made on conviction. The skill surfaces which is which.
• Does not validate or invalidate specific research methods. It scores how well the evidence supports the specific claim being made.
• Does not block the design. It tells the team what they are betting on.