| name | ai-feedback-design-principles |
| description | Audit and redesign AI-generated feedback for pedagogical quality, timing, and learning impact. Use when building or reviewing automated feedback in digital learning tools. |
| disable-model-invocation | false |
| user-invocable | true |
| effort | medium |
| skill_id | ai-learning-science/ai-feedback-design-principles |
| skill_name | AI Feedback Design Principles |
| domain | ai-learning-science |
| version | 1.0 |
| evidence_strength | strong |
| evidence_sources | ["Shute (2008) — Focus on formative feedback (comprehensive review)","Narciss (2008) — Feedback strategies for interactive learning tasks (informative tutoring feedback model)","Hattie & Timperley (2007) — The power of feedback (meta-analysis, effect size 0.73)","Dai et al. (2023) — Can large language models provide useful feedback on research papers? A large-scale empirical analysis","Kluger & DeNisi (1996) — The effects of feedback interventions on performance: A historical review and a meta-analysis"] |
| input_schema | {"required":[{"field":"feedback_scenario","type":"string","description":"The specific context in which AI will deliver feedback — what the student has done and what kind of feedback is needed"},{"field":"current_feedback_design","type":"string","description":"The current or proposed AI feedback approach — what the system currently says or plans to say in response to student work"}],"optional":[{"field":"student_level","type":"string","description":"Age/year group and proficiency level"},{"field":"subject_area","type":"string","description":"The curriculum subject"},{"field":"feedback_goals","type":"string","description":"What the feedback should achieve — error correction, motivation, deeper thinking, self-regulation, or something else"},{"field":"system_constraints","type":"string","description":"Technical or practical constraints on the feedback — character limits, timing requirements, or format restrictions"}]} |
| output_schema | {"type":"object","fields":[{"field":"feedback_evaluation","type":"object","description":"Analysis of the current feedback design against research criteria — what works and what doesn't"},{"field":"improved_feedback","type":"object","description":"A redesigned version of the feedback that addresses identified weaknesses"},{"field":"feedback_type_analysis","type":"object","description":"Classification of the feedback by type (verification, elaboration, strategic) with recommendations for the optimal mix"},{"field":"implementation_guidance","type":"object","description":"Practical advice for deploying the improved feedback in the target system"}]} |
| chains_well_with | ["adaptive-hint-sequence-designer","formative-assessment-loop-designer","intelligent-tutoring-dialogue-designer","self-explanation-prompt-designer","technological-pedagogical-content-knowledge-developer"] |
| teacher_time | 4 minutes |
| tags | ["feedback","AI-feedback","formative","Shute","Narciss","Hattie","LLM","automated-feedback"] |
AI Feedback Design Principles
What This Skill Does
Evaluates a proposed AI feedback design against research criteria for effective automated feedback and suggests specific improvements. This skill takes a feedback scenario (what the student did) and the current or proposed AI response (what the system says), then analyses the feedback against principles from Shute (2008), Narciss (2008), Hattie & Timperley (2007), and emerging LLM feedback research (Dai et al., 2023). The output includes a diagnosis of what's working and what isn't, a redesigned version of the feedback, and practical implementation guidance. The core challenge is that most AI feedback falls into one of two failure modes: it's either too vague to be actionable ("Good effort! Try to improve your argument.") or too specific and does the thinking for the student ("Your thesis should be: Climate change is the defining challenge of our generation because…"). Effective feedback lives in the narrow space between these extremes — specific enough that the student knows what to do, but not so specific that it bypasses the cognitive work that produces learning. AI is specifically valuable here because it can generate feedback at scale, but this makes the design principles even more critical: bad feedback at scale is worse than no feedback at all.
Evidence Foundation
Hattie & Timperley (2007) conducted a meta-analysis finding that feedback has an average effect size of 0.73 — making it one of the most powerful influences on learning. However, they found enormous variation: some feedback interventions produced large positive effects while others had zero or even NEGATIVE effects. The critical variable was not whether feedback was given, but WHAT KIND of feedback was given. They proposed a model with four levels: task feedback (is the answer correct?), process feedback (what strategies can improve the work?), self-regulation feedback (how can you monitor your own learning?), and self feedback (you're a great student!). Task and process feedback were most effective; self feedback ("Good job!") was least effective and sometimes harmful because it directs attention to the self rather than the task. Shute (2008) reviewed formative feedback research and identified key principles: effective feedback is specific, timely, non-threatening, and focused on the task rather than the learner. She distinguished between verification feedback (correct/incorrect), elaborated feedback (why it's correct/incorrect and what to do next), and various combinations. She found that elaborated feedback generally outperforms simple verification, BUT that overly detailed feedback can overwhelm novice learners — creating a feedback paradox where more information sometimes produces less learning. Narciss (2008) developed the Informative Tutoring Feedback (ITF) model, which specifies that effective feedback should include: knowledge of result (correct or not), knowledge of the correct response (if wrong), and elaboration on the error (why it's wrong and what misconception it reveals). Critically, Narciss found that the optimal feedback depends on the error type: conceptual errors benefit from elaborated feedback, while careless slips benefit from simple verification. Kluger & DeNisi (1996) found in their meta-analysis that feedback that directs attention to the self (rather than the task) can DECREASE performance — a finding with direct implications for AI systems that generate encouraging but empty praise. Dai et al. (2023) evaluated LLM-generated feedback and found that while LLMs can produce fluent, well-structured feedback, they tend toward a specific pattern: excessive positivity, vague suggestions, and a reluctance to identify specific errors — precisely the pattern that research identifies as least effective.
