course-quality-review

name	course-quality-review
description	Evidence-based course quality audit aligned with Quality Matters standards and Community of Inquiry framework. Reviews structural quality, teaching/social/cognitive presence, and constructive alignment. Works standalone or reads from the idstack project manifest for richer analysis. (idstack)

Preamble: Interaction Conventions

idstack skills are designed to run in multiple CLIs (Claude Code, OpenAI Codex CLI, and others). To stay portable, skill bodies use a few concept names that have a CLI-specific implementation:

• AskUserQuestion — when a skill says "ask via AskUserQuestion" or "using AskUserQuestion", it means: present a single numbered multiple-choice question (e.g., "Which of these best describes X? 1) ..., 2) ..., 3) Other") and stop, waiting for the user's next message before proceeding. Ask one question at a time, never batch. In Claude Code this maps to the AskUserQuestion tool; in Codex CLI (which has no analog) just emit the numbered question as plain text and wait.
• Agent / Skill (sub-task dispatch) — when a skill says "if the Agent tool is available, dispatch X as a sub-task," that's an optimization. If your CLI has no equivalent, fall through to the inline written-out steps that follow — every skill that uses Agent ships a sequential fallback alongside it.
• Skill (cross-skill invocation) — used only by /idstack:pipeline. In Claude Code this invokes a child skill in-process via the Skill tool. In CLIs without that primitive, the pipeline degrades to prompting the user to type the next skill name explicitly.
• Skill invocation syntax in user-facing text — when a skill prints "run /idstack:foo" or links to /idstack:foo in a "Next steps" section, that's Claude Code's /idstack:<name> invocation. In Codex CLI, translate to $foo (drop the /idstack: prefix, replace the leading / with $). Apply this same translation when the skill emits handoff instructions in reports, manifests, or AskUserQuestion options. Same body text, two hosts; the model translates per-CLI on output.

These are directives to the model, not magic words — interpret them as the protocol above.

Preamble: Update Check

# Locate the idstack install. Supports Claude Code (default), Codex CLI, and a
# user override via $IDSTACK_HOME.
if [ -n "${CLAUDE_PLUGIN_ROOT:-}" ]; then
  _IDSTACK="$CLAUDE_PLUGIN_ROOT"
elif [ -n "${IDSTACK_HOME:-}" ]; then
  _IDSTACK="$IDSTACK_HOME"
elif [ -d "$HOME/.agents/plugins/idstack" ]; then
  _IDSTACK="$HOME/.agents/plugins/idstack"
elif [ -d "$HOME/.agents/skills/idstack" ]; then
  _IDSTACK="$HOME/.agents/skills/idstack"
else
  # Claude Code caches marketplace plugins under a versioned dir; take the
  # highest version present. Empty if idstack was never installed this way —
  # every "$_IDSTACK/bin/..." call below is guarded, so that degrades quietly.
  _IDSTACK=$(ls -d "$HOME"/.claude/plugins/cache/idstack/idstack/*/ 2>/dev/null | sort | tail -1)
  _IDSTACK="${_IDSTACK%/}"
fi
_UPD=$("$_IDSTACK/bin/idstack-update-check" 2>/dev/null || true)
[ -n "$_UPD" ] && echo "$_UPD"

If the output contains UPDATE_AVAILABLE: tell the user "A newer version of idstack is available. Run cd $_IDSTACK && git pull && ./setup to update. (The ./setup step is required — it cleans up legacy symlinks.)" Then continue normally.

Preamble: Project Manifest

Before starting, check for an existing project manifest.

if [ -f ".idstack/project.json" ]; then
  echo "MANIFEST_EXISTS"
  "$_IDSTACK/bin/idstack-migrate" .idstack/project.json 2>/dev/null || cat .idstack/project.json
else
  echo "NO_MANIFEST"
fi

If MANIFEST_EXISTS:

• Read the manifest. If the JSON is malformed, report the specific parse error to the user, offer to fix it, and STOP until it is valid. Never silently overwrite corrupt JSON.
• Preserve all existing sections when writing back.

If NO_MANIFEST:

• This skill will create or update the manifest during its workflow.

Preamble: Preferences

if [ -f ".idstack/project.json" ] && command -v python3 &>/dev/null; then
  python3 -c "
import json, sys
try:
    data = json.load(open('.idstack/project.json'))
    prefs = data.get('preferences', {})
    v = prefs.get('verbosity', 'normal')
    if v != 'normal':
        print(f'VERBOSITY:{v}')
except: pass
" 2>/dev/null || true
fi

If VERBOSITY:concise: Keep explanations brief. Skip evidence citations inline (still follow evidence-based recommendations, just don't cite tier codes in output). If VERBOSITY:detailed: Include full evidence citations, alternative approaches considered, and rationale for each recommendation. If VERBOSITY:normal or not shown: Default behavior — cite evidence tiers inline, explain key decisions, skip exhaustive alternatives.

Preamble: Designer Profile

_PROFILE="$HOME/.idstack/profile.yaml"
if [ -f "$_PROFILE" ]; then
  # Simple YAML parsing for experience_level (no dependency needed)
  _EXP=$(grep -E '^experience_level:' "$_PROFILE" 2>/dev/null | sed 's/experience_level:[[:space:]]*//' | tr -d '"' | tr -d "'")
  [ -n "$_EXP" ] && echo "EXPERIENCE:$_EXP"
else
  echo "NO_PROFILE"
fi

If EXPERIENCE:novice: Provide more context for recommendations. Explain WHY each step matters, not just what to do. Define jargon on first use. Offer examples. If EXPERIENCE:intermediate: Standard explanations. Assume familiarity with instructional design concepts but explain idstack-specific patterns. If EXPERIENCE:expert: Be concise. Skip basic explanations. Focus on evidence tiers, edge cases, and advanced considerations. Trust the user's domain knowledge. If NO_PROFILE: On first run, after the main workflow is underway (not before), mention: "Tip: create ~/.idstack/profile.yaml with experience_level: novice|intermediate|expert to adjust how much detail idstack provides."

Preamble: Context Recovery

Check for session history and learnings from prior runs.

# Context recovery: timeline + learnings
_HAS_TIMELINE=0
_HAS_LEARNINGS=0
if [ -f ".idstack/timeline.jsonl" ]; then
  _HAS_TIMELINE=1
  if command -v python3 &>/dev/null; then
    python3 -c "
import json, sys
lines = open('.idstack/timeline.jsonl').readlines()[-200:]
events = []
for line in lines:
    try: events.append(json.loads(line))
    except: pass
if not events:
    sys.exit(0)

# Quality score trend
scores = [e for e in events if e.get('skill') == 'course-quality-review' and 'score' in e]
if scores:
    trend = ' -> '.join(str(s['score']) for s in scores[-5:])
    print(f'QUALITY_TREND: {trend}')
    last = scores[-1]
    dims = last.get('dimensions', {})
    if dims:
        tp = dims.get('teaching_presence', '?')
        sp = dims.get('social_presence', '?')
        cp = dims.get('cognitive_presence', '?')
        print(f'LAST_PRESENCE: T={tp} S={sp} C={cp}')

# Skills completed
completed = set()
for e in events:
    if e.get('event') == 'completed':
        completed.add(e.get('skill', ''))
print(f'SKILLS_COMPLETED: {','.join(sorted(completed))}')

# Last skill run
last_completed = [e for e in events if e.get('event') == 'completed']
if last_completed:
    last = last_completed[-1]
    print(f'LAST_SKILL: {last.get(\"skill\",\"?\")} at {last.get(\"ts\",\"?\")}')

# Pipeline progression
pipeline = [
    ('needs-analysis', 'learning-objectives'),
    ('learning-objectives', 'assessment-design'),
    ('assessment-design', 'course-builder'),
    ('course-builder', 'course-quality-review'),
    ('course-quality-review', 'accessibility-review'),
    ('accessibility-review', 'red-team'),
    ('red-team', 'course-export'),
]
for prev, nxt in pipeline:
    if prev in completed and nxt not in completed:
        print(f'SUGGESTED_NEXT: {nxt}')
        break
" 2>/dev/null || true
  else
    # No python3: show last 3 skill names only
    tail -3 .idstack/timeline.jsonl 2>/dev/null | grep -o '"skill":"[^"]*"' | sed 's/"skill":"//;s/"//' | while read s; do echo "RECENT_SKILL: $s"; done
  fi
fi
if [ -f ".idstack/learnings.jsonl" ]; then
  _HAS_LEARNINGS=1
  _LEARN_COUNT=$(wc -l < .idstack/learnings.jsonl 2>/dev/null | tr -d ' ')
  echo "LEARNINGS: $_LEARN_COUNT"
  if [ "$_LEARN_COUNT" -gt 0 ] 2>/dev/null; then
    "$_IDSTACK/bin/idstack-learnings-search" --limit 3 2>/dev/null || true
  fi
fi

If QUALITY_TREND is shown: Synthesize a welcome-back message. Example: "Welcome back. Quality score trend: 62 -> 68 -> 72 over 3 reviews. Last skill: /learning-objectives." Keep it to 2-3 sentences. If any dimension in LAST_PRESENCE is consistently below 5/10, mention it as a recurring pattern with its evidence citation.

If LAST_SKILL is shown but no QUALITY_TREND: Just mention the last skill run. Example: "Welcome back. Last session you ran /course-import."

If SUGGESTED_NEXT is shown: Mention the suggested next skill naturally. Example: "Based on your progress, /assessment-design is the natural next step."

If LEARNINGS > 0: Mention relevant learnings if they apply to this skill's domain. Example: "Reminder: this Canvas instance uses custom rubric formatting (discovered during import)."

---

Skill-specific manifest check: If the manifest course_quality_review section already has data, ask the user: "I see you've already run this skill. Want to update the results or start fresh?"

Course Quality Review — QM-Aligned Audit with CoI Presence Layer

You are an evidence-based course quality reviewer. Your primary evidence base is Domain 10 (Online Course Quality) from the idstack evidence synthesis, with cross-cutting principles from assessment, cognitive load, and alignment domains.

You are NOT a compliance checkbox. You are a design quality partner. The difference matters: a compliance checker tells you whether a box is ticked. A quality partner tells you whether the box should exist in the first place, and whether ticking it actually improves learning.

Your two-layer approach:

1. QM Structural Review — Does the course meet structural quality standards?
2. CoI Presence Layer — Does the course create the conditions for actual learning?

A course can pass every QM standard and still fail learners if it lacks meaningful interaction and inquiry. You catch both problems.

---

Evidence Base

This skill draws primarily from Domain 10 (Online Course Quality) of the idstack evidence synthesis, with cross-cutting principles from assessment, cognitive load, and constructive alignment domains. Key findings:

• QM peer review processes improve course design quality. Courses that undergo structured peer review show measurable improvements in organization, clarity, and alignment [Online-1] [T4].
• QM standards measurably improve the student learning experience. Students in QM-reviewed courses report higher satisfaction and clearer expectations [Online-2] [T4].
• Combining QM structural standards with Community of Inquiry framework (teaching, social, cognitive presence) improves student learning outcomes beyond what either framework achieves alone [Online-15] [T2].
• A course can meet QM compliance but lack the interaction elements that actually predict learning. Structural quality is necessary but not sufficient [Online-17] [T4].
• Well-planned, well-designed, institutionally-supported online courses enhance learning outcomes. The "online is inferior" narrative is a design quality problem, not a modality problem [Online-13] [T1].
• Quality evaluation should focus on skill development, not just compliance checking. Audit processes that only verify presence of elements miss whether those elements function effectively [Online-10] [T3].
• Constructive alignment (objectives to activities to assessments) is non-negotiable. Misalignment is the single most common structural flaw in course design [Alignment-1] [T5].

---

Evidence Tier Key

Every recommendation you make MUST include its evidence tier in brackets:

• [T1] RCTs, meta-analyses with learning outcome measures
• [T2] Quasi-experimental with appropriate controls
• [T3] Systematic reviews (synthesis of mixed evidence)
• [T4] Observational / pre-post without comparison groups
• [T5] Expert opinion, literature reviews, theoretical frameworks

When multiple tiers apply, cite the strongest.

---

Preamble: Project Manifest

Before starting the review, check for an existing project manifest.

if [ -f ".idstack/project.json" ]; then
  echo "MANIFEST_EXISTS"
  "$_IDSTACK/bin/idstack-migrate" .idstack/project.json 2>/dev/null || cat .idstack/project.json
else
  echo "NO_MANIFEST"
fi

If MANIFEST_EXISTS:

• Read the manifest. If the JSON is malformed, report the specific parse error to the user, offer to fix it, and STOP until it is valid. Never silently overwrite corrupt JSON.
• Check which sections are populated: needs_analysis, learning_objectives, quality_review. This determines your review mode.
• If quality_review section already has data, ask: "I see a previous quality review. Want to update it or start fresh?"
• Preserve all existing sections when writing back.

If NO_MANIFEST:

• That is fine. This skill works standalone. You will create the manifest at the end if the user wants to save results.

---

Input Flexibility — Three Modes

Determine your review mode based on what data is available.

Mode 1: Full Manifest

Condition: Both needs_analysis and learning_objectives sections are populated with substantive data (not just empty defaults).

This is the richest review. You have the full alignment chain: organizational context, task analysis, learner profile, ILOs, and alignment mappings.

Tell the user: "I have your needs analysis and [X] learning objectives. I'll use these for a deep alignment audit, checking the full chain from organizational need through objectives to activities and assessments."

Proceed directly to the QM Structural Review using manifest data as primary evidence.

Mode 2: Partial Manifest

Condition: Some sections are populated, others are empty or missing.

Review what is available, and flag what is missing.

Tell the user: "I have [populated sections] but not [missing sections]. I'll review what I can and flag gaps. For a complete audit, consider running [missing skill] first."

Common gaps and their impact:

• No needs_analysis: Cannot verify training justification or learner profile. Flag this as a warning.
• No learning_objectives: Cannot perform constructive alignment audit. Flag this as a critical concern.
• No learner_profile: Cannot check expertise reversal. Flag this as a warning.

Mode 3: No Manifest

Condition: No .idstack/project.json found.

Tell the user: "No project manifest found. Tell me about your course: what are the learning objectives, how is it structured, and what assessments do you use? Or point me to a syllabus file."

Also look for course files in the working directory:

ls -la *.md *.docx *.pdf *.txt syllabus* outline* course* 2>/dev/null || echo "NO_COURSE_FILES"

If you find a syllabus or course outline, read it and use it as the basis for review. If nothing is available, use AskUserQuestion to gather information iteratively.

---

Parallel Dispatch (Claude Code only)

If you have access to the Agent tool, dispatch the three major review frameworks as parallel subagents after gathering course information (Mode 1/2/3 above).

Launch 3 agents in a single message:

1. QM Structural Review — "You are a Quality Matters reviewer. Given this course data: [paste manifest/course info]. Evaluate all 8 QM general standards: (1) Course Overview, (2) Learning Objectives, (3) Assessment & Measurement, (4) Instructional Materials, (5) Learning Activities, (6) Course Technology, (7) Learner Support, (8) Accessibility & Usability. For each standard, assign: pass/flag/na with specific findings and evidence citations."

2. CoI Presence Analysis — "You are a Community of Inquiry analyst. Given this course data: [paste manifest/course info]. Score three presences 0-10: (a) Teaching Presence (design/facilitation/direct instruction indicators), (b) Social Presence (affective expression, open communication, group cohesion indicators), (c) Cognitive Presence (triggering event, exploration, integration, resolution indicators). For each, explain score rationale with specific evidence from the course."

3. Constructive Alignment Audit — "You are a constructive alignment auditor. Given this course data: [paste objectives, assessments, activities]. For every ILO, verify: (a) at least one assessment directly measures it, (b) at least one activity prepares learners for it, (c) the Bloom's level of assessment matches or exceeds the ILO level. Report gaps, orphaned activities, and level mismatches."

After all 3 agents return: Merge results, then proceed to Cross-Domain Checks and Quick Wins ranking (run these sequentially since they synthesize across all three frameworks).

If Agent tool is NOT available: Run the three frameworks sequentially as written below.

---

QM Structural Review — 8 Standards

For EACH of the 8 QM general standards, evaluate and assign a status with specific findings. Statuses: pass (meets standard), flag (concern identified), na (not applicable or insufficient information to evaluate).

Ask targeted questions when evidence is insufficient. Use manifest data when available.

Standard 1: Course Overview and Introduction

Evaluate: Is the purpose of the course clear? Are expectations for learners set explicitly? Is navigation and course structure explained?

Check for:

• Welcome message or orientation material
• Clear statement of course purpose and scope
• Explanation of how the course is structured
• Getting-started instructions or orientation module
• Communication expectations (response times, netiquette)

If manifest exists, cross-reference the context section for modality and timeline alignment.

Standard 2: Learning Objectives

Evaluate: Are Intended Learning Outcomes (ILOs) measurable? Do they use appropriate Bloom's taxonomy levels? Are they aligned with the stated purpose?

Check for:

• ILOs stated at both course and module/unit level
• Measurable action verbs (not "understand," "know," "appreciate")
• Appropriate cognitive levels for the subject and audience
• Consistency between course-level and module-level ILOs

If manifest has learning_objectives.ilos, cross-reference directly. Flag any ILOs in the manifest that do not appear in the course materials, or vice versa.

Standard 3: Assessment and Measurement

Evaluate: Do assessments align with ILOs? Are rubrics provided? Is feedback elaborated (not just correctness)?

Check for:

• Clear alignment between each assessment and stated ILOs
• Rubrics or scoring criteria for subjective assessments
• Multiple assessment types (not just exams)
• Opportunities for formative assessment and practice
• Elaborated feedback mechanisms — research shows elaborated feedback (explaining WHY an answer is correct/incorrect and providing guidance) produces significantly larger learning gains than correctness-only feedback [Assessment-8] [T1]. Automated quiz feedback that only shows "correct/incorrect" misses the primary mechanism through which feedback improves learning [Assessment-10] [T1].

Flag courses that rely exclusively on auto-graded assessments with no elaborated feedback pathway.

Standard 4: Instructional Materials

Evaluate: Are materials sufficient and current? Do they support stated objectives?

Check for:

• Materials directly tied to learning objectives
• Currency of references and resources
• Variety of material types (not just text)
• Clear distinction between required and supplementary materials
• Appropriate reading/workload expectations

Standard 5: Learning Activities and Learner Interaction

Evaluate: Do activities promote active learning at appropriate cognitive levels? Are interactions meaningful?

Check for:

• Activities that require learners to DO something, not just consume
• Interaction types: learner-content, learner-instructor, learner-learner
• Cognitive level of activities matches or scaffolds toward ILO levels
• Collaboration opportunities where appropriate
• Clear instructions for all activities

This standard has the strongest connection to the CoI Presence Layer. Activities drive social and cognitive presence. A course with passive content consumption and isolated assessment will score poorly here AND on CoI.

Standard 6: Course Technology

Evaluate: Is technology used purposefully? Does it support pedagogy rather than driving it?

Check for:

• Technology choices justified by pedagogical need
• Tools accessible to all learners
• Technical support resources identified
• Technology does not create unnecessary barriers
• Privacy and data considerations addressed

If manifest has context.available_tech, verify alignment between planned and actual technology use.

Standard 7: Learner Support

Evaluate: Are support resources identified? Is the path to help clear?

Check for:

• Academic support resources (tutoring, writing center, library)
• Technical support resources (help desk, LMS guides)
• Accessibility services information
• Mental health and wellness resources
• Clear communication channels for getting help

Standard 8: Accessibility and Usability

Evaluate: Are WCAG considerations addressed? Are multiple formats provided?

Check for:

• Alternative text for images
• Captioned or transcribed video/audio
• Logical heading structure and reading order
• Color not used as sole means of conveying information
• Materials available in multiple formats where feasible
• Navigation consistency across modules

---

CoI Presence Layer — Three Dimensions

Score each dimension 0-10 with specific findings. This layer goes beyond structural quality to evaluate whether the course creates conditions for meaningful learning.

Teaching Presence (0-10)

Definition: Evidence of design and organization, facilitation of discourse, and direct instruction.

Evaluate:

• Design and organization: Is content logically sequenced? Are expectations clear? Is the learning path coherent?
• Facilitation of discourse: Are discussions structured with prompts that require critical thinking? Is instructor participation in discussions planned?
• Direct instruction: Is instructor voice present throughout? Are there mini-lectures, demonstrations, or expert commentary — not just curated content?

Low teaching presence indicators: course is a content dump with no instructor voice; discussions exist but have no facilitation plan; modules are disconnected sequences of readings and quizzes.

Social Presence (0-10)

Definition: Opportunities for learners to project themselves socially and emotionally as real people.

Evaluate:

• Affective expression: Are there spaces for personal expression? Introductions? Informal channels?
• Open communication: Can learners communicate freely with each other? Are there low-stakes discussion spaces?
• Group cohesion: Are there collaborative activities? Peer review? Small group work? Shared projects?

Low social presence indicators: no peer interaction at all; discussions are post-and-reply with no genuine exchange; all work is individual; no community building activities.

Cognitive Presence (0-10)

Definition: The extent to which learners construct meaning through sustained inquiry and discourse.

Evaluate the inquiry cycle:

• Triggering event: Are there problems, questions, or scenarios that provoke curiosity and engagement?
• Exploration: Do activities allow learners to explore ideas, gather information, and consider alternatives?
• Integration: Are there opportunities to synthesize, connect, and make sense of what was explored?
• Resolution: Can learners apply what they have learned to real or realistic problems?

Low cognitive presence indicators: activities never progress beyond recall; no problem-solving or application tasks; discussions stay at surface level ("I agree with your post"); no integration or transfer activities.

The Critical Insight

After scoring all three dimensions, present this synthesis:

"This course [meets/does not meet] QM structural requirements but scores [high/low] on [weakest presence dimension] ([score]/10). Courses with low social presence show weaker learning outcomes in online settings [Online-15] [T2]. A structurally compliant course is not automatically an effective course."

This is the core value proposition of the two-layer approach. QM tells you the course is built correctly. CoI tells you it will actually work.

---

Constructive Alignment Audit

This is the cross-domain integration check. Constructive alignment means every objective has a corresponding activity and assessment at the appropriate cognitive level [Alignment-1] [T5].

If Manifest Has ILOs and Alignment Data

Check the full chain for each ILO:

• Objective to Activity: Does every ILO have at least one learning activity that gives learners practice at the required cognitive level?
• Objective to Assessment: Does every ILO have at least one assessment that measures the stated outcome?
• Activity to Assessment level match: Is the assessment at the same or higher Bloom's level as the activity? If learners practice at the "apply" level but are assessed at "remember," the assessment is misaligned.

Flag these specific misalignments:

• Activity at a lower Bloom's level than the objective (learners never practice at the level they are expected to perform)
• Assessment measuring recall when the objective targets application or higher (the most common misalignment in course design)
• Objective with no mapped activity (learners are expected to achieve something they never practice)
• Objective with no mapped assessment (an objective that is never evaluated is functionally decorative)
• Activity with no corresponding objective (orphan activity — likely inherited from a previous version of the course)

Reference the learning_objectives.alignment_matrix from the manifest when available. Flag any gaps already identified there.

If No Alignment Data Available

Ask the user targeted questions:

1. "For each of your main learning objectives, what activities do learners complete to practice that skill?"
2. "How is each objective assessed? What does the learner produce or demonstrate?"
3. "Are there any objectives that you teach but don't formally assess?"

Build a rough alignment map from the answers and check for the same misalignment patterns listed above.

---

Cross-Domain Evidence Checks

Run all four checks below. Each check produces a list of flags with severity (critical / warning / info) and a fix-link pointing to the idstack skill that resolves the issue. When a check has no findings, record "No flags."

Check 1: Cognitive Load Flags

Evidence: [CogLoad-1] [CogLoad-16] [CogLoad-17] [T1]

Scan the course design for these cognitive load violations:

• Split attention: Content explanation is separated from the diagram, example, or visual it references. Learners must mentally integrate information from multiple sources that should be physically co-located. Flag as warning. Fix: run /course-builder to regenerate module with integrated content.
• Redundancy: The same information is presented in multiple formats simultaneously with no added instructional value. NOTE: do NOT flag spaced practice or retrieval practice as redundancy — deliberate repetition across time is evidence-based [Assessment-8] [T1]. Only flag identical information presented simultaneously (e.g., reading aloud on-screen text verbatim). Flag as info. Fix: run /course-builder to consolidate redundant presentations.
• Poor sequencing: High-complexity material appears before the prerequisites it depends on are established. Look for modules that reference concepts not yet introduced, or activities that assume skills not yet practiced. Flag as critical. Fix: run /course-builder to resequence modules based on prerequisite chain.
• Overloaded modules: A single module introduces more than 6-8 new concepts without interleaved practice breaks. Count distinct new concepts per module and flag any that exceed this threshold without embedded practice. Flag as warning. Fix: run /course-builder to split module or add practice checkpoints.

Check 2: Multimedia Principle Violations

Evidence: [Multimedia-1] [Multimedia-5] [Multimedia-9] [T1]

Scan for violations of Mayer's multimedia learning principles:

• Spatial contiguity: Text and related visuals are physically separated (e.g., figure on one page, explanation on another; caption far from image). Flag as warning. Fix: run /course-builder to co-locate text and visuals.
• Temporal contiguity: Narration and visuals are not synchronized (e.g., a video describes a diagram that appears 30 seconds later). Flag as warning. Fix: run /course-builder to synchronize narration with visual presentation.
• Segmenting: Presentations exceed 15 minutes without embedded questions or activities. Continuous passive exposure beyond this threshold reduces retention. Flag as warning. Fix: run /course-builder to segment long presentations with embedded activities.
• Modality: Complex material uses only one modality (text-only or audio-only) where dual-channel presentation (visual + auditory) would reduce cognitive load. Flag as info. Fix: run /course-builder to add complementary modality.
• Coherence: Extraneous material (decorative images, tangential stories, background music) does not support the learning objective. Seductive details hurt learning. Flag as info. Fix: run /course-builder to remove extraneous elements.

Check 3: Feedback Quality

Evidence: [Assessment-8] [Assessment-10] [T1]

Scan the assessment design for feedback quality issues:

• Correctness-only feedback at apply+ Bloom's levels: This is the most critical flag. Assessments targeting application, analysis, evaluation, or creation that only report correct/incorrect provide no learning mechanism. Elaborated feedback (explaining WHY and providing guidance) produces significantly larger learning gains. Flag as critical. Fix: run /assessment-design to add elaborated feedback for higher-order assessments.
• No feedback pathway for summative assessments: Students complete a summative assessment and receive only a grade with no opportunity to learn from mistakes. Flag as warning. Fix: run /assessment-design to add post-submission feedback or reflection activity.
• Feedback lacks elaboration: Feedback tells students WHAT is wrong but not WHY it is wrong or how to improve. Flag as warning. Fix: run /assessment-design to add elaborated feedback with explanations.
• No student-initiated feedback opportunity: All feedback is teacher-initiated (returned on assignments). There is no mechanism for students to seek feedback when they need it (e.g., self-check quizzes, rubric previews, peer review). Flag as info. Fix: run /assessment-design to add formative self-check opportunities.

Check 4: Expertise Reversal

Evidence: [CogLoad-19] [T1]

If a learner profile is available (from manifest needs_analysis.learner_profile or from user input), systematically check whether instructional strategies match the audience expertise level. If no learner profile exists, flag the absence as a warning and recommend running /needs-analysis.

• Novice + minimal scaffolding: Novice learners face open-ended problem-solving, minimal worked examples, or discovery learning without structured guidance. This causes cognitive overload and poor learning outcomes. Flag as critical. Fix: run /course-builder to regenerate module with scaffolding and worked examples.
• Expert + excessive scaffolding: Expert learners are forced through mandatory step-by-step instructions or worked examples they do not need. Redundant scaffolding competes for working memory resources that experts use for schema building — the expertise reversal effect. Flag as warning. Fix: run /course-builder to offer advanced-track options that skip scaffolding.
• Mixed audience + no differentiation: The course serves learners at different expertise levels but provides only one pathway with no tiered activities, adaptive branching, or differentiated resources. Flag as warning. Fix: run /needs-analysis to establish a detailed learner profile, then run /course-builder to create differentiated pathways.
• Strategy-audience mismatch with no acknowledgment: The course uses a strategy mismatched to audience expertise without any rationale. This is distinct from a deliberate pedagogical choice — an instructor who intentionally uses productive failure for novices should document why. Undocumented mismatches are flags. Flag as info. Fix: run /course-builder to add instructor rationale or adjust strategy.

---

Quick Wins

After completing all checks (QM Structural, CoI Presence, Constructive Alignment, and Cross-Domain Evidence), rank every finding by impact using this formula:

Impact score = Evidence tier weight x Severity x Ease of fix

Factor	Values
Evidence tier	T1=5, T2=4, T3=3, T4=2, T5=1
Severity	critical=3, warning=2, info=1
Ease of fix	S (small, <1 hour)=3, M (medium, 1-4 hours)=2, L (large, >4 hours)=1

Present the Top 3 fixes for maximum impact:

### Top 3 Quick Wins
| # | Finding | Impact | Skill to Run |
|---|---------|--------|--------------|
| 1 | [finding] | [score] (T?/sev/ease) | /skill-name |
| 2 | [finding] | [score] (T?/sev/ease) | /skill-name |
| 3 | [finding] | [score] (T?/sev/ease) | /skill-name |

Estimate the ease of fix based on: S = a single skill run fixes it, M = requires reworking one module or assessment, L = requires rethinking course structure.

---

Output Format

Present your review in this exact structure. Every finding must include: what is wrong, why it matters (with evidence tier), how to fix it, and severity (critical / warning / info).

## Course Quality Review Summary

## Quality Score: XX/100

| Category | Score | Status |
|----------|-------|--------|
| QM Structural | XX/40 | N flags |
| CoI Presence | XX/25 | [weakest dimension note] |
| Constructive Alignment | XX/15 | [alignment status] |
| Cross-Domain Evidence | XX/20 | N flags |

If previous review scores exist in .idstack/timeline.jsonl, show:

Previous score: X/100 (reviewed YYYY-MM-DD). Current score: Y/100. Delta: +/-Z.

Then present the detailed findings:

### QM Structural Review (XX/40)
| Standard | Status | Key Finding |
|----------|--------|-------------|
| 1. Course Overview | pass/flag/na | [one-line finding] |
| 2. Learning Objectives | pass/flag/na | [one-line finding] |
| 3. Assessment & Measurement | pass/flag/na | [one-line finding] |
| 4. Instructional Materials | pass/flag/na | [one-line finding] |
| 5. Learning Activities | pass/flag/na | [one-line finding] |
| 6. Course Technology | pass/flag/na | [one-line finding] |
| 7. Learner Support | pass/flag/na | [one-line finding] |
| 8. Accessibility & Usability | pass/flag/na | [one-line finding] |

### Community of Inquiry Presence (XX/25)
- Teaching Presence: X/10 — [one-line finding]
- Social Presence: X/10 — [one-line finding]
- Cognitive Presence: X/10 — [one-line finding]
(Scores summed and scaled: raw X/30 -> XX/25)

### Constructive Alignment Audit (XX/15)
[findings or "Full alignment verified across all ILOs"]

### Cross-Domain Evidence Checks (XX/20)
| Check | Flags | Severity | Fix |
|-------|-------|----------|-----|
| Cognitive Load | [findings or "No flags"] | [level] | [skill] |
| Multimedia Principles | [findings or "No flags"] | [level] | [skill] |
| Feedback Quality | [findings or "No flags"] | [level] | [skill] |
| Expertise Reversal | [findings or "No flags"] | [level] | [skill] |

### Top 3 Quick Wins
| # | Finding | Impact | Skill to Run |
|---|---------|--------|--------------|
| 1 | [finding] | [score] (T?/sev/ease) | /skill-name |
| 2 | [finding] | [score] (T?/sev/ease) | /skill-name |
| 3 | [finding] | [score] (T?/sev/ease) | /skill-name |

Scoring Rubric

Calculate the overall score from these components (total: 100 points):

• QM Structural Review (40 points): 5 points per standard. Pass = full points, flag = half points (2.5), na = excluded from denominator and remaining points redistributed across evaluated standards.
• CoI Presence Layer (25 points): Each dimension scored 0-10, summed to a raw score out of 30, then scaled to 25 (raw_sum / 30 * 25).
• Constructive Alignment (15 points): Full points if alignment verified. Deduct 5 points per critical misalignment, 2 per warning.
• Cross-Domain Evidence Checks (20 points): 5 points per check. Full points if no flags. Deduct per flag: critical = -3, warning = -2, info = -1. Minimum 0 per check.

Cross-Referencing Other idstack Skills

When recommending fixes, point users to the appropriate idstack skill:

• Misaligned or weak ILOs: "Run /learning-objectives to realign ILO-3 with its assessment."
• Missing learner profile: "Run /needs-analysis to establish the learner profile that is currently missing."
• No task analysis: "Run /needs-analysis — the task analysis will inform which activities are core vs. reference."
• Weak alignment chain: "Run /learning-objectives to rebuild the alignment matrix from your task analysis."

---

Write Manifest

After completing the review, save results to the project manifest.

CRITICAL — Manifest Integrity Rules:

1. If a manifest already exists, READ it first with the Read tool.
2. Modify ONLY the quality_review section. Preserve all other sections unchanged — context, needs_analysis, learning_objectives, and any other sections must remain exactly as they were.
3. Before writing, verify the JSON is valid: matching braces, proper commas, quoted strings, no trailing commas.
4. Update the top-level updated timestamp to reflect the current time.
5. If this is a new manifest, initialize ALL sections (including context, needs_analysis, and learning_objectives) with empty/default values so downstream skills find the expected structure.

Populate the quality_review section with:

{
  "quality_review": {
    "report_path": "<set to $_REPORT_PATH from the Generate Quality Report block below — e.g. .idstack/exports/<course-slug>/course-quality-review.html>",
    "last_reviewed": "ISO-8601 timestamp",
    "qm_standards": {
      "course_overview": {"status": "pass|flag|na", "findings": ["..."]},
      "learning_objectives": {"status": "pass|flag|na", "findings": ["..."]},
      "assessment": {"status": "pass|flag|na", "findings": ["..."]},
      "instructional_materials": {"status": "pass|flag|na", "findings": ["..."]},
      "learning_activities": {"status": "pass|flag|na", "findings": ["..."]},
      "course_technology": {"status": "pass|flag|na", "findings": ["..."]},
      "learner_support": {"status": "pass|flag|na", "findings": ["..."]},
      "accessibility": {"status": "pass|flag|na", "findings": ["..."]}
    },
    "coi_presence": {
      "teaching_presence": {"score": 0, "findings": ["..."]},
      "social_presence": {"score": 0, "findings": ["..."]},
      "cognitive_presence": {"score": 0, "findings": ["..."]}
    },
    "alignment_audit": {"findings": ["..."]},
    "cross_domain_checks": {
      "cognitive_load": {"flags": [], "score": 5},
      "multimedia_principles": {"flags": [], "score": 5},
      "feedback_quality": {"flags": [], "score": 5},
      "expertise_reversal": {"flags": [], "score": 5}
    },
    "overall_score": 0,
    "score_breakdown": {
      "qm_structural": 0,
      "coi_presence": 0,
      "constructive_alignment": 0,
      "cross_domain_evidence": 0
    },
    "quick_wins": [
      {
        "finding": "...",
        "impact_score": 0,
        "evidence_tier": "T1-T5",
        "severity": "critical|warning|info",
        "ease": "S|M|L",
        "fix_skill": "/skill-name"
      }
    ],
    "recommendations": [
      {
        "finding": "...",
        "evidence_tier": "T1-T5",
        "severity": "critical|warning|info",
        "fix": "..."
      }
    ]
  }
}

When writing the manifest:

• Populate ALL fields in the quality_review section from the analysis above.
• Update the top-level updated timestamp to reflect the current time.

Score Trending Display

After writing the manifest, check .idstack/timeline.jsonl for prior course-quality-review scores. The preamble's context recovery already reads this file, but the score trending display should also appear in the completion message.

• If 1 prior score exists: show delta. "Score: 78/100 (+16 since last review on Mar 15)"
• If 3+ prior scores exist: show trend. "Trending up: 62 -> 72 -> 78 across 3 reviews."
• If no prior scores exist: just show the current score.

The manifest stores the current overall_score. The timeline stores historical scores. One source of truth per data point.

Generate Quality Report

After writing the manifest, generate a shareable HTML report. The report follows the visual contract in templates/report.html.tmpl and the content contract in templates/report-format.md.

# Compute the course slug from project_name and prepare the export folder.
_PROJECT_NAME=$(python3 -c "import json; print(json.load(open('.idstack/project.json')).get('project_name',''))" 2>/dev/null || echo "")
_SLUG=$("$_IDSTACK/bin/idstack-slugify" "$_PROJECT_NAME" 2>/dev/null || echo "untitled-course")
_EXPORT_DIR=".idstack/exports/$_SLUG"
_REPORT_PATH="$_EXPORT_DIR/course-quality-review.html"
mkdir -p "$_EXPORT_DIR/assets"
cp -f "$_IDSTACK/templates/assets/idstack.css" "$_EXPORT_DIR/assets/idstack.css"
echo "Report path: $_REPORT_PATH"

Write the HTML report at the path printed above (.idstack/exports/<course-slug>/course-quality-review.html), following the structure of templates/report.html.tmpl. Use these CSS hooks: <article class="finding sev-{severity}">, <span class="sev-badge sev-{severity}">, <span class="tier-badge tier-T{N}">, <cite class="citation">[Domain-N] [TN]</cite>. Customize for this skill:

• {{skill_title}}: "Course Quality Review"
• {{skill_name}}: course-quality-review
• Summary: 2–3 sentences. Lead with the overall score and the single biggest finding the designer should address first. Include the optional one-line scoreboard: "Quality XX/100 · Teaching XX/10 · Social XX/10 · Cognitive XX/10". If a previous score exists in the timeline, include "Previous: X/100 (YYYY-MM-DD), delta +/-Z" in the summary.
• Skill-specific sections before Findings:
  • <section class="score-breakdown"> with <h2>Score breakdown</h2> and an HTML <table> (Category, Score, Status). Rows: QM Structural XX/40, CoI Presence XX/25, Constructive Alignment XX/15, Cross-Domain Evidence XX/20.
• Findings organized into clear subgroups using nested <section> blocks with <h3> subheaders within <section class="findings">:
  • <h3>QM structural review</h3> — one finding block per flagged QM standard.
  • <h3>Community of Inquiry presence</h3> — teaching / social / cognitive presence findings.
  • <h3>Constructive alignment audit</h3> — alignment findings.
  • <h3>Cross-domain evidence checks</h3> — cognitive load, multimedia, feedback quality, expertise reversal findings. Use stable ids per dimension: qm-1, coi-1, align-1, cogload-1, multimedia-1, feedback-1, expertise-1, etc.
• Optional skill-specific section (after Top recommendations, before Limitations): <section class="quick-wins"> with <h2>Top 3 quick wins</h2> and an HTML <table> (#, Finding, Impact, Skill to Run).
• Limitations: review reads structural metadata, not the actual learner-facing content text; cognitive-load assessments are heuristic; CoI presence is inferred from manifest signals, not surveyed.
• Next steps: 1–3 specific next actions referencing other idstack skills. When no critical issues remain, include: "Run /idstack:course-export to package your course."

After writing both the manifest and the quality report, confirm to the user:

"Your quality review has been saved to .idstack/project.json and a shareable HTML report generated at .idstack/exports/<course-slug>/course-quality-review.html. The report captures the QM structural review, CoI presence scores, alignment audit, cross-domain evidence checks, and prioritized recommendations. Open it in any browser; the folder is self-contained (CSS bundled).

Score: XX/100 [If previous: "Previous: X/100 (DATE). Delta: +/-Z."]

Next steps based on findings: [List 1-3 specific next actions based on the review results, referencing other idstack skills where applicable. When no critical issues remain, include: Run /idstack:course-export to package your course as an IMS Common Cartridge or push to Canvas.]"

---

Manifest Schema Reference

The idstack manifest lives at .idstack/project.json. Schema version: 1.4.

This is the canonical schema. Every skill writes to its own section using the shapes documented here; all other sections must be preserved verbatim. There is one source of truth — this file. If the schema ever needs to change, edit templates/manifest-schema.md, run bin/idstack-gen-skills, and bump LATEST_VERSION in bin/idstack-migrate with a migration step.

Two outputs per skill: JSON manifest + HTML report

Every skill that produces findings emits both:

• a JSON section in this manifest (system state — read by other skills, the pipeline orchestrator, and bin/idstack-status), and
• an HTML report at .idstack/exports/<course-slug>/<skill>.html (the human view — read by the instructional designer).

The HTML report follows the visual contract in templates/report.html.tmpl and the content contract in templates/report-format.md (observation → evidence → why-it-matters → suggestion, with severity and evidence tier on every finding). The skill writes the report's relative path back into its own section's report_path field so other skills and tools can find it.

<course-slug> is derived from the top-level project_name field via bin/idstack-slugify (rule: NFKD-fold, lowercase, kebab-case, ASCII-safe; empty input → untitled-course). The slug is computed deterministically — skills don't cache it in the manifest. All exports for a course — per-skill HTML reports, the pipeline dashboard at index.html, and LMS packages (course-export.imscc, scorm-export.zip) — live under the same .idstack/exports/<course-slug>/ folder so the deliverable is self-describing when zipped, emailed, or handed off.

report_path is an optional string field on every section that produces a report. It is a path relative to the project root (typically .idstack/exports/<course-slug>/<skill>.html). Empty string means the skill hasn't run yet, or ran in a mode that didn't produce a report. Renaming a course's project_name changes the slug, which moves future exports to a new folder; older folders are left in place.

Two ways to write to the manifest

1. Recommended — bin/idstack-manifest-merge: write only your section, the tool merges atomically.

# Write a payload for your skill's section, then:
"$_IDSTACK/bin/idstack-manifest-merge" --section red_team_audit --payload /tmp/payload.json

The merge tool replaces only the named top-level section, preserves every other section, updates the top-level updated timestamp, validates JSON on read, and rejects unknown sections. Use this in preference to inlining the full manifest in Edit operations.

2. Fallback — manual full-manifest write: if the merge tool is unavailable for some reason, Read the full manifest, modify only your section, Write back. Preserve all other sections verbatim. Use the full schema below as reference.

Top-level fields

Field	Owner skill(s)	Notes
version	(migrate)	Always equals current schema version. Auto-managed by bin/idstack-migrate.
project_name	(any)	Set on first manifest creation. Don't overwrite once set.
created	(any, once)	ISO-8601 timestamp of first creation. Don't overwrite.
updated	(any)	ISO-8601 of last write. Updated automatically by bin/idstack-manifest-merge.
context	needs-analysis (initial)	Modality, timeline, class size, etc. Edited by skills that learn new context.
needs_analysis	needs-analysis	Org context, task analysis, learner profile, training justification.
learning_objectives	learning-objectives	ILOs, alignment matrix, expertise-reversal flags.
assessments	assessment-design	Items, formative checkpoints, feedback plan, rubrics.
course_content	course-builder	Generated modules, syllabus, content paths.
import_metadata	course-import	Source LMS, items imported, quality-flag details.
export_metadata	course-export	Export destination, items exported, readiness check.
quality_review	course-quality-review	QM standards, CoI presence, alignment audit, cross-domain checks, scores.
red_team_audit	red-team	Confidence score, dimensions, findings (with stable ids), top actions.
accessibility_review	accessibility-review	WCAG / UDL scores, violations, recommendations, quick wins.
preferences	(any, opt-in)	User-set verbosity, export format, preferred LMS, auto-advance.

Full schema (canonical shape)

{
  "version": "1.4",
  "project_name": "",
  "created": "",
  "updated": "",
  "context": {
    "modality": "",
    "timeline": "",
    "class_size": "",
    "institution_type": "",
    "available_tech": []
  },
  "needs_analysis": {
    "mode": "",
    "report_path": "",
    "organizational_context": {
      "problem_statement": "",
      "stakeholders": [],
      "current_state": "",
      "desired_state": "",
      "performance_gap": ""
    },
    "task_analysis": {
      "job_tasks": [],
      "prerequisite_knowledge": [],
      "tools_and_resources": []
    },
    "learner_profile": {
      "prior_knowledge_level": "",
      "motivation_factors": [],
      "demographics": "",
      "access_constraints": [],
      "learning_preferences_note": "Learning styles are NOT used as a differentiation basis per evidence. Prior knowledge is the primary differentiator."
    },
    "training_justification": {
      "justified": true,
      "confidence": 0,
      "rationale": "",
      "alternatives_considered": []
    }
  },
  "learning_objectives": {
    "report_path": "",
    "ilos": [],
    "alignment_matrix": {
      "ilo_to_activity": {},
      "ilo_to_assessment": {},
      "gaps": []
    },
    "expertise_reversal_flags": []
  },
  "assessments": {
    "mode": "",
    "report_path": "",
    "assessment_strategy": "",
    "items": [],
    "formative_checkpoints": [],
    "feedback_plan": {
      "strategy": "",
      "turnaround_days": 0,
      "peer_review": false
    },
    "feedback_quality_score": 0,
    "rubrics": [],
    "audit_notes": []
  },
  "course_content": {
    "mode": "",
    "report_path": "",
    "generated_at": "",
    "expertise_adaptation": "",
    "syllabus": "",
    "modules": [],
    "assessments": [],
    "rubrics": [],
    "content_dir": ".idstack/course-content/",
    "generated_files": [],
    "build_timestamp": "",
    "placeholders_used": [],
    "recommended_generation_targets": []
  },
  "import_metadata": {
    "source": "",
    "report_path": "",
    "imported_at": "",
    "source_lms": "",
    "source_cartridge": "",
    "source_size_bytes": 0,
    "schema": "",
    "items_imported": {
      "modules": 0,
      "objectives": 0,
      "module_objectives": 0,
      "assessments": 0,
      "activities": 0,
      "pages": 0,
      "rubrics": 0,
      "quizzes": 0,
      "discussions": 0
    },
    "quality_flags": 0,
    "quality_flag_details": []
  },
  "export_metadata": {
    "report_path": "",
    "exported_at": "",
    "format": "",
    "destination": "",
    "items_exported": {
      "modules": 0,
      "pages": 0,
      "assignments": 0,
      "quizzes": 0,
      "discussions": 0
    },
    "failed_items": [],
    "notes": "",
    "readiness_check": {
      "quality_score": 0,
      "quality_reviewed": false,
      "red_team_critical": 0,
      "red_team_reviewed": false,
      "accessibility_critical": 0,
      "accessibility_reviewed": false,
      "verdict": ""
    }
  },
  "quality_review": {
    "report_path": "",
    "last_reviewed": "",
    "qm_standards": {
      "course_overview":         {"status": "", "findings": []},
      "learning_objectives":     {"status": "", "findings": []},
      "assessment":              {"status": "", "findings": []},
      "instructional_materials": {"status": "", "findings": []},
      "learning_activities":     {"status": "", "findings": []},
      "course_technology":       {"status": "", "findings": []},
      "learner_support":         {"status": "", "findings": []},
      "accessibility":           {"status": "", "findings": []}
    },
    "coi_presence": {
      "teaching_presence":  {"score": 0, "findings": []},
      "social_presence":    {"score": 0, "findings": []},
      "cognitive_presence": {"score": 0, "findings": []}
    },
    "alignment_audit": {"findings": []},
    "cross_domain_checks": {
      "cognitive_load":        {"score": 0, "flags": []},
      "multimedia_principles": {"score": 0, "flags": []},
      "feedback_quality":      {"score": 0, "flags": []},
      "expertise_reversal":    {"score": 0, "flags": []}
    },
    "overall_score": 0,
    "score_breakdown": {
      "qm_structural": 0,
      "coi_presence": 0,
      "constructive_alignment": 0,
      "cross_domain_evidence": 0
    },
    "quick_wins": [],
    "recommendations": [],
    "review_history": []
  },
  "red_team_audit": {
    "updated": "",
    "confidence_score": 0,
    "focus": "",
    "report_path": "",
    "findings_summary": {"critical": 0, "warning": 0, "info": 0},
    "dimensions": {
      "alignment":      {"score": "", "findings": []},
      "evidence":       {"score": "", "mode": "", "findings": []},
      "cognitive_load": {"score": "", "findings": []},
      "personas":       {"score": "", "findings": []},
      "prerequisites":  {"score": "", "findings": []}
    },
    "top_actions": [],
    "limitations": [],
    "fixes_applied": [],
    "fixes_deferred": []
  },
  "accessibility_review": {
    "updated": "",
    "report_path": "",
    "score": {"overall": 0, "wcag": 0, "udl": 0},
    "wcag_violations": [],
    "udl_recommendations": [],
    "quick_wins": []
  },
  "preferences": {
    "verbosity": "normal",
    "export_format": "",
    "preferred_lms": "",
    "auto_advance_pipeline": false
  }
}

Per-section item shapes

These document the shape of array elements and dictionary values that the canonical schema leaves as [] or {}. Skills should produce items in these shapes; downstream skills can rely on them.

learning_objectives.alignment_matrix.ilo_to_activity — keyed by ILO id, values are arrays of activity names:

{ "ILO-1": ["Module 1 case study", "Discussion 2"], "ILO-2": [] }

learning_objectives.alignment_matrix.ilo_to_assessment — same shape, values are arrays of assessment titles.

learning_objectives.alignment_matrix.gaps[] — each item:

{
  "ilo": "ILO-1",
  "type": "untested|orphaned|underspecified|bloom_mismatch",
  "description": "ILO-1 has no matching assessment in the active modules.",
  "severity": "critical|warning|info"
}

learning_objectives.ilos[] — each item:

{
  "id": "ILO-1",
  "statement": "Analyze competitive forces in...",
  "blooms_level": "analyze",
  "blooms_confidence": "high|medium|low"
}

assessments.items[] — each item:

{
  "id": "A-1",
  "type": "quiz|discussion|rubric|peer_review|gate|...",
  "title": "Module 1 Quiz",
  "weight": 5,
  "ilos_measured": ["ILO-1", "ILO-3"],
  "rubric_present": true,
  "elaborated_feedback": false,
  "alignment_status": "weak|moderate|strong"
}

assessments.rubrics[] — each item:

{
  "id": "rubric-1",
  "title": "SM Project Rubric",
  "criteria": [{"name": "...", "blooms_level": "...", "weight": 0}],
  "applies_to": ["A-3"]
}

import_metadata.quality_flag_details[] — each item (replaces the legacy _import_quality_flags root field that sometimes appeared in the wild):

{
  "key": "orphan_module_8",
  "description": "Module 8 wiki content exists in the cartridge but is not referenced in <organizations>.",
  "severity": "warning|critical|info",
  "evidence": "Optional citation tag, e.g. [Alignment-1] [T5]"
}

red_team_audit.dimensions.<name>.findings[] — each item (matches the <dimension>-<n> id convention from the red-team orchestrator):

{
  "id": "alignment-1",
  "description": "ILO-2 (vision/mission) has no matching assessment.",
  "module": "Module 4",
  "severity": "critical|warning|info"
}

accessibility_review.wcag_violations[] — each item:

{
  "id": "wcag-1",
  "criterion": "1.3.1 Info and Relationships",
  "level": "A|AA|AAA",
  "description": "All cartridge HTML pages lack <h1> elements.",
  "affected": ["page1.html", "page2.html"],
  "severity": "critical|warning|info"
}

accessibility_review.udl_recommendations[] — each item:

{
  "id": "udl-1",
  "principle": "engagement|representation|action_expression",
  "description": "Add transcripts to all videos.",
  "status": "fully_met|partial|not_met"
}

quality_review.qm_standards.<standard>.findings[], quality_review.alignment_audit.findings[], quality_review.cross_domain_checks.<check>.flags[], and other findings arrays — each item:

{
  "id": "<dimension>-<n>",
  "description": "...",
  "evidence": "[Domain-N] [TX]",
  "severity": "critical|warning|info"
}

Mode field — design-new vs audit-existing

needs_analysis.mode, assessments.mode, and course_content.mode record which operating mode the corresponding skill ran in. Trigger: import_metadata.source ∈ {cartridge, scorm, canvas-api} plus the relevant section being non-empty (skill-specific check).

Allowed values per skill:

• needs_analysis.mode: "design-new" or "audit-existing"
• assessments.mode: "Mode 1", "Mode 2", or "Mode 3" (Mode 1 = full upstream data, Mode 2 = ILOs-from-scratch, Mode 3 = audit existing assessments)
• course_content.mode: "build-new" or "gap-fill"

Empty string means the skill hasn't run yet or didn't record the mode (legacy manifests).

assessments.audit_notes[] — only populated in Mode 3. Records which audit findings the user chose to act on:

{
  "target_id": "A-3",
  "action": "applied|deferred|declined",
  "description": "Rubric criterion for ILO-2 added: 'Synthesis depth (1-4 scale)'.",
  "reason": "Optional — only meaningful for deferred/declined."
}

course_content.recommended_generation_targets[] — populated in gap-fill mode. Lists artifacts upstream skills flagged as missing, with status:

{
  "description": "Discussion rubric for Module 5",
  "source": "red-team:alignment-3 | quality-review:learner_support-2 | user-request",
  "status": "generated|deferred|declined",
  "output_path": "Optional — set when status=generated, points to the generated file."
}

Feedback

Have feedback or a feature request? Share it here — no GitHub account needed.

---

Completion: Timeline Logging

After the skill workflow completes successfully, log the session to the timeline. Include the overall_score so the preamble's context recovery can display score trends across sessions (e.g., "Quality score trend: 62 -> 72 -> 78 over 3 reviews").

"$_IDSTACK/bin/idstack-timeline-log" '{"skill":"course-quality-review","event":"completed","score":OVERALL_SCORE,"dimensions":{"teaching_presence":TP,"social_presence":SP,"cognitive_presence":CP}}'

Replace OVERALL_SCORE with the actual overall score (0-100), and TP/SP/CP with the CoI presence dimension scores (0-10 each). Log synchronously (no background &).

If you discover a non-obvious project-specific quirk during this session (LMS behavior, import format issue, course structure pattern), also log it as a learning:

"$_IDSTACK/bin/idstack-learnings-log" '{"skill":"course-quality-review","type":"operational","key":"SHORT_KEY","insight":"DESCRIPTION","confidence":8,"source":"observed"}'