| name | AI Feedback Loops |
| description | Design feedback mechanisms that help AI systems learn from users - thumbs up/down, preference ranking, corrections, and human-in-the-loop escalation. Use when: RLHF UX, user feedback for AI, thumbs up down design, AI correction flow, human in the loop, feedback signal design, AI improvement loops. |
AI Feedback Loops
Design feedback mechanisms that simultaneously improve the AI model AND improve the user's experience of giving feedback. The SIGNAL framework ensures feedback is low-friction, high-quality, and genuinely acted upon.
Core Principle
Feedback is a transaction. Users invest effort (reporting an error, rating an output, explaining what's wrong). If they never see a return on that investment, they stop giving feedback. Design feedback loops that close - where users can see that their input made a difference.
The SIGNAL Framework
| Letter | Principle | Design Question |
|---|
| S | Surface the Moment | Is feedback offered at the exact moment the user has an opinion? |
| I | Incentivize Honestly | Does the feedback design encourage genuine assessment, not just positive ratings? |
| G | Graduate the Effort | Can users give 1-second feedback OR 1-minute feedback, depending on their willingness? |
| N | Narrate the Impact | Can users see how their feedback improved the system? |
| A | Aggregate Intelligently | Is individual feedback combined into actionable patterns, not just counted? |
| L | Loop the Learning | Does the improved AI behavior visibly reflect the feedback it received? |
The Feedback Pyramid
Design feedback collection in layers - most users will only reach the first layer, and that's fine.
| Layer | Effort | Signal Quality | Collection Rate | Mechanism |
|---|
| L1: Implicit | Zero effort | Low but high-volume | 100% of users | Usage patterns: regeneration rate, copy rate, session length, abandonment points |
| L2: Binary | 1 second | Medium | 15-30% of users | Thumbs up/down, helpful/not helpful |
| L3: Categorical | 5 seconds | Medium-High | 5-15% of users | "What was wrong?" dropdown: Inaccurate / Irrelevant / Incomplete / Offensive / Other |
| L4: Textual | 30 seconds | High | 2-5% of users | Free-text explanation of what was wrong and what would be better |
| L5: Comparative | 60 seconds | Very High | 1-3% of users | Side-by-side preference ranking: "Which response is better? A or B" |
Design rule: Never require L3+ feedback to complete a task. Deeper feedback should always be optional and offered after the primary interaction is complete.
Implicit Feedback Signals
These require zero user effort but reveal powerful behavioral data:
| Signal | What It Indicates | Measurement |
|---|
| Regeneration | User rejected the output | Count of "regenerate" clicks per session |
| Copy/use rate | User accepted the output | Percentage of outputs that are copied, saved, or acted upon |
| Edit distance | User partially accepted | How much the user modified the AI output before using it |
| Abandonment point | AI failed to deliver value | Where in the flow users leave |
| Scroll depth | Output was too long or too short | How far users scroll before acting |
| Time-to-next-action | User needed time to evaluate | Long pause = uncertainty; immediate action = confidence |
| Return frequency | Product is valuable | How often users come back after first use |
| Prompt refinement rate | AI didn't understand the first time | How often users rephrase their query |
Binary Feedback Design
The thumbs up/down is the most common AI feedback mechanism - and the most commonly botched.
Thumbs Up/Down Best Practices
| Design Decision | Recommended Approach | Anti-Pattern |
|---|
| Placement | Below every AI response, right-aligned | Hidden in a menu, or only on some responses |
| Initial state | Both unselected (neutral) | Pre-selected thumbs up (biases positive) |
| After thumbs down | Immediately offer L3 dropdown: "What was wrong?" | Just log the downvote with no follow-up |
| After thumbs up | Optional: "What did you like?" (but don't insist) | Pop-up survey that interrupts the flow |
| Visual feedback | Button fills/highlights to confirm the vote registered | No visual change (user doesn't know if it worked) |
| Changeability | User can change their vote anytime | Vote is locked after submission |
The Thumbs Down Expansion
When a user clicks thumbs down, expand into a lightweight triage:
What was wrong?
[ ] Inaccurate information [ ] Not what I asked for
[ ] Too long / too short [ ] Offensive or inappropriate
[ ] Outdated information [ ] Other: ___________
[Optional] What would a better answer look like?
[_______________________________________________]
[Submit] [Skip]
Design rule: "Skip" must be prominent. Forcing explanation creates resentment and low-quality data.
Comparative Feedback (A/B Preference)
The highest-quality feedback signal: showing two AI outputs and asking which is better.
When to Use Comparative Feedback
| Scenario | Comparative Feedback? | Why |
|---|
| General chat responses | Occasionally (10-20% of responses) | High-quality signal but adds friction |
| Creative generation (writing, images) | Yes - great fit | Subjective quality is hard to measure otherwise |
| Factual answers | Rarely | Usually one answer is objectively correct - binary feedback is sufficient |
| Code generation | Sometimes | "Which code is cleaner?" is valuable but requires expertise to judge |
Comparison UI Patterns
| Pattern | How It Works | Best For |
|---|
| Side-by-side | Two responses shown simultaneously | Short responses (< 200 words each) |
| Sequential with memory | Show response A, then B, then ask preference | Longer responses where side-by-side is unwieldy |
| Inline replacement | "I have an alternative answer. Want to see it?" | Minimal disruption to the main flow |
| Batch review | End-of-session: "Help us improve - which answers were best today?" | Users willing to spend 2 minutes helping |
Closing the Feedback Loop
Users who give feedback and never see impact stop giving feedback. Design visible closure.
Feedback Impact Patterns
| Pattern | Implementation | Example |
|---|
| Immediate acknowledgment | Thank the user and explain what happens next | "Thanks - this helps us improve. We review flagged responses daily." |
| Aggregate impact | Show users how feedback from all users improved the system | "Users flagged 500 inaccurate responses last month. 340 were corrected." |
| Personal impact | Show the user how THEIR feedback changed behavior | "Based on your feedback, I now handle [specific task] differently." |
| Release notes | Tie model improvements to user feedback themes | "v2.3 improvements: Better code generation (informed by user feedback)" |
Human-in-the-Loop Escalation
When AI confidence is low, feedback becomes real-time human decision-making.
Escalation Triggers
| Trigger | Condition | Escalation Type |
|---|
| Confidence below threshold | Model confidence < 40% on a high-stakes output | Route to human reviewer before showing to user |
| User reports critical error | Thumbs down + "Inaccurate" + free-text with urgency signals | Flag for immediate human review |
| Sensitive content detected | Output touches medical, legal, financial, or safety topics | Add human verification badge or mandatory disclaimer |
| Novel situation | Query is outside the model's training distribution | Transparent handoff: "This is outside my expertise. Let me connect you with..." |
| Repeated failures | Same user gives thumbs down 3+ times in a session | Proactive human intervention: "Would you like to speak with a specialist?" |
Anti-Patterns
| Pattern | Why It Fails |
|---|
| Feedback widget appears before the user has read the response | Users can't rate what they haven't consumed |
| Only collecting positive feedback (e.g., "Was this helpful? Yes / No" with only "Yes" prominent) | Survivorship bias - you only hear from satisfied users |
| Pop-up NPS survey mid-task | Interrupts flow; generates resentful 0-scores |
| "Thanks for your feedback!" with no evidence of action | Teaches users that feedback is performative, not functional |
| Requiring login to give feedback | Reduces feedback volume by 80%+ |
| Treating all thumbs-down equally | "Inaccurate" and "too long" require completely different responses |
Quick Reference
| Task | Framework Element | Key Deliverable |
|---|
| Add feedback to AI product | Feedback Pyramid (L1-L5) | Layered feedback system design |
| Improve feedback quality | SIGNAL framework | Audit of current feedback loops + gaps |
| Design thumbs up/down experience | Binary Feedback Best Practices | Thumbs down expansion flow |
| Build RLHF training pipeline UX | Comparative Feedback patterns | A/B preference UI + sampling strategy |
| Close the feedback loop | Feedback Impact Patterns | User-visible impact communication plan |
Integration
Works with: ai-error-resilience (error reports as feedback), ai-trust-transparency (feedback builds trust when acted upon), ai-agent-ux (rating agent actions), ai-conversation-architect (in-conversation feedback moments).
