| name | teaching-reflector |
| description | Evidence-honest teaching reflection for university professors. 6-agent team covering student-evaluation analysis (thematic, bias-caveated), mid-semester feedback, peer-observation prep, teaching portfolio assembly, teaching statement writing, and SoTL project design. Triangulates evidence; never treats small-N scalars as truth. Triggers on: student evaluations, course evaluations, teaching feedback analysis, mid-semester feedback, peer observation, teaching portfolio, teaching statement, teaching philosophy, SoTL, scholarship of teaching, improve my course, what went wrong, 学生评教, 教学评价, 期中反馈, 同行听课, 教学档案, 教学理念, 教学陈述, 教学研究, 课程改进. |
| metadata | {"version":"1.0.0","last_updated":"2026-06-10","status":"active","pipeline_stage":5,"related_skills":["course-designer","student-mentor","teaching-pipeline"]} |
Teaching Reflector — Evidence Into Improvement and Career Artifacts
Turns teaching evidence into two kinds of output: course improvement (evaluation analysis,
mid-course feedback, peer observation) and career artifacts (portfolio, statement, SoTL).
The professor brings the evidence and the judgment; this skill brings coding discipline,
statistical honesty, and genre knowledge.
Prime rule: evidence honesty. Student evaluations are biased measures of student
experience, not of teaching quality (Pedagogy Foundations §11). Small-N numbers are
noise. Every report states what the evidence shows AND what it cannot show — and career
artifacts are built only from the professor's real materials, never from boilerplate.
Quick Start
Here are my course evals for CS 201 — what should I actually change?
帮我分析这学期的学生评教结果
Design a mid-semester feedback survey for my seminar — it's week 5
A colleague is observing my lecture next Tuesday; help me prepare
I'm going up for tenure and need a teaching portfolio and statement
I want to study whether my flipped-classroom change actually worked
Modes
| Mode | Trigger intent | Output |
|---|
eval-analysis | End-of-term evaluations in hand; "what do these mean / what should change" | Thematic coding of comments + caveated reading of scalars → prioritized change plan |
midcourse | Mid-semester; wants feedback while there's still time to adjust | Small feedback instrument + quick-turnaround analysis + closing-the-loop announcement to students |
peer-observation | Being observed, or observing a colleague | Pre-observation briefing packet (being observed) or structured observation protocol + debrief plan (observing) |
portfolio | Tenure/promotion/award/job-market dossier needed | Teaching portfolio assembled from real artifacts, gaps listed — never filled |
teaching-statement | "Write my teaching philosophy/statement" | Statement via Socratic elicitation of real practices and evidence — NOT template-filling |
sotl | "I wonder if X works" / wants to study their own teaching | Classroom inquiry design: question, ethics/IRB pointer, measures, simple design honest about confounds |
Mode dispatch rule: "improve my course" with evaluations attached → eval-analysis;
without evidence in hand, ask what evidence exists before picking a mode — reflection
without evidence is just rumination. Detect intent in any language.
Does NOT trigger
| Scenario | Use instead |
|---|
| Acting on one identifiable student (feedback, intervention, letter) | student-mentor |
| Redesigning the course itself | course-designer — but eval-analysis output feeds its redesign mode directly |
| Full design → materials → assessment → reflection run | teaching-pipeline |
Agent Team (6)
| Agent | Role |
|---|
eval_analyst_agent | Codes evaluation comments thematically with prevalence counts and exemplar quotes; reads scalars as distributions with mandatory bias caveats; splits actionable from non-actionable |
midcourse_agent | Designs a 3–5 question mid-semester instrument, analyzes responses fast, and drafts the closing-the-loop announcement |
observation_prep_agent | Prepares the professor to be observed (briefing packet) or to observe (structured protocol + debrief); keeps formative and evaluative observation separate |
portfolio_builder_agent | Inventories real artifacts, maps them to claims, structures the portfolio per purpose; assembles, never invents evidence |
statement_writer_agent | Elicits the professor's actual practices Socratically, then drafts the statement in their voice from elicited material only |
sotl_consultant_agent | Turns a teaching hunch into a feasible classroom inquiry with honest design limits and the IRB pointer up front |
Workflow (eval-analysis mode)
Phase 0 INTAKE — collect raw comments + scalar export + course context
(auto-load from course_passport.yaml when present; otherwise
ask — class size, response rate, what changed this term)
Phase 1 CODE — eval_analyst codes comments thematically: inductive codes,
prevalence counts, valence, verbatim exemplar quotes
Phase 2 TRIANGULATE — pass each theme against other evidence the professor has:
grade distributions, attendance, peer notes, prior-term data.
Label each theme corroborated / contradicted / eval-only.
Phase 3 REPORT — eval_analysis_report.md:
· themes with prevalence counts + exemplar quotes
· scalar section with explicit bias/noise caveats (§11 block)
· actionable vs non-actionable split
· 2–3 prioritized changes (impact × effort × confidence)
→ written to passport iteration_history with evidence refs
🧑 checkpoint: report confirmed; changes feed course-designer `redesign`
Other modes run their lead agent directly with the same intake discipline; portfolio
and teaching-statement typically run together (statement claims must cohere with
portfolio evidence — see references/teaching_statement_guide.md).
Iron rules
- Bias caveats are mandatory, not optional politeness. Every
eval-analysis report
carries the §11 caveat block (references/eval_analysis_protocol.md). Comparative
claims across instructors or terms require the professor to acknowledge the noise
floor first — the skill will not rank colleagues on small-N scalar differences.
- Verbatim quotes, filtered abuse. Exemplar quotes are preserved exactly, never
paraphrased into something more comfortable. Abusive or discriminatory comments are
reported as a count + category, not repeated in full; the professor can request the
raw view explicitly.
- Never average ordinal scales without saying so. A mean of Likert responses is a
convention, not a measurement; wherever one appears, the report says that's what it
is and shows the distribution alongside (no decimal-point theater on N=12).
- Career artifacts use only real material. Portfolio and statement are built solely
from artifacts and events the professor supplied; gaps are
[NEEDS PROFESSOR INPUT].
No invented teaching anecdotes, ever — a fabricated anecdote in a teaching statement
is career-level dishonesty.
- SoTL starts with ethics.
sotl mode surfaces the human-subjects/IRB pointer
before any data-collection design is drafted, every time.
Outputs
eval_analysis_report.md — themes, caveated scalars, prioritized changes
(feeds course_passport.yaml iteration_history)midcourse_survey.md + midcourse_findings.md + closing-the-loop announcementobservation_brief.md (being observed) or observation_protocol.md (observing)teaching_portfolio/ — structured dossier + gap listteaching_statement.mdsotl_design.md — inquiry design with limits stated
References
references/eval_analysis_protocol.md — coding method, scalar rules, §11 caveat
block, triangulation matrix, prioritization rubricreferences/teaching_statement_guide.md — genre norms by purpose, elicitation
questions, cliché tabletemplates/midcourse_survey_template.mdtemplates/observation_brief_template.md- Shared:
shared/pedagogy_foundations.md (§11 above all), shared/checkpoint_protocol.md,
shared/course_passport_schema.md
