| name | review-reply-suggestion-feedback |
| description | Daily scheduled skill that reviews yesterday's #feed-mentions and #community-mentions activity, compares Buzz's reply suggestions to the team's emoji reactions and thread feedback, and opens one PR to improve draft-brand-reply if patterns emerge.
|
Review Reply Suggestion Feedback
You are a daily self-improvement loop for Buzz's community reply quality. Each morning you review yesterday's #feed-mentions and #community-mentions activity, compare Buzz's suggestions to what the team actually did, and — if there are durable patterns — update the draft-brand-reply skill and open one PR.
Inputs
Your prompt may optionally contain:
- Date to review — a specific date (YYYY-MM-DD) to analyze
If no date is provided, default to yesterday (UTC).
Channels
Review both channels in one run:
#feed-mentions: FEED_MENTIONS_CHANNEL_ID#community-mentions: COMMUNITY_MENTIONS_CHANNEL_ID
Use $BUZZ_SLACK_TOKEN, $FEED_MENTIONS_CHANNEL_ID, and $COMMUNITY_MENTIONS_CHANNEL_ID via os.environ for Slack reads and status updates. Never echo, log, or hardcode it.
Process
1. Collect raw data from Slack
Write and run a Python script that gathers yesterday's parent messages, reactions, and thread replies for both channels. The script collects raw data only — it does not try to parse Buzz's suggestion or source message body from the message format. You'll interpret that yourself in Step 2.
Use this shape for the collection script:
import json, os, urllib.parse, urllib.request
from datetime import datetime, timezone, timedelta
TOKEN = os.environ["BUZZ_SLACK_TOKEN"]
REVIEW_DATE = os.environ.get("REVIEW_DATE") or (datetime.now(timezone.utc) - timedelta(days=1)).strftime("%Y-%m-%d")
CHANNELS = [
{"name": "#feed-mentions", "id": os.environ.get("FEED_MENTIONS_CHANNEL_ID", "")},
{"name": "#community-mentions", "id": os.environ.get("COMMUNITY_MENTIONS_CHANNEL_ID", "")},
]
CHANNELS = [channel for channel in CHANNELS if channel["id"]]
def slack_api(method, params=None):
url = f"https://slack.com/api/{method}"
if params:
url += "?" + urllib.parse.urlencode(params)
req = urllib.request.Request(url, headers={"Authorization": f"Bearer {TOKEN}"})
return json.loads(urllib.request.urlopen(req).read())
day_start = datetime.strptime(REVIEW_DATE, "%Y-%m-%d").replace(tzinfo=timezone.utc)
day_end = day_start + timedelta(days=1)
oldest = str(int(day_start.timestamp()))
latest = str(int(day_end.timestamp()))
results = []
for channel in CHANNELS:
history = slack_api("conversations.history", {
"channel": channel["id"], "oldest": oldest, "latest": latest, "limit": "200"
})
for msg in history.get("messages", []):
ts = msg["ts"]
msg_text = msg.get("text", "")
if "⏭️" in msg_text and "Skipped mentions" in msg_text:
continue
if msg_text.startswith("+") and "more unanswered community messages" in msg_text:
continue
reactions_resp = slack_api("reactions.get", {"channel": channel["id"], "timestamp": ts})
reaction_list = reactions_resp.get("message", {}).get("reactions", [])
reactions = {r["name"]: r["count"] for r in reaction_list}
team_decision = None
if "white_check_mark" in reactions:
team_decision = "reply"
elif "heart" in reactions:
team_decision = "like"
elif "x" in reactions:
team_decision = "skip"
thread = slack_api("conversations.replies", {
"channel": channel["id"], "ts": ts, "limit": "50"
})
replies = thread.get("messages", [])[1:]
human_feedback = []
for reply in replies:
if reply.get("bot_id") or reply.get("subtype") == "bot_message":
continue
if reply.get("ts") == ts:
continue
human_feedback.append(reply.get("text", ""))
if team_decision is None and not human_feedback:
continue
results.append({
"channel_name": channel["name"],
"channel_id": channel["id"],
"ts": ts,
"message_text": msg_text,
"blocks": msg.get("blocks", []),
"team_decision": team_decision,
"human_feedback": human_feedback,
})
print(json.dumps(results, indent=2))
If the prompt specifies a date, set REVIEW_DATE as an env var before running, or let it default to yesterday.
2. Interpret the collected data
For each collected message, read message_text and blocks to determine:
- Channel — whether this came from
#feed-mentions or #community-mentions.
- Buzz's suggestion — look for the triage indicator in the message (for example ✅ suggested reply, ❤️ suggestion: like, ❌ suggestion: skip). Community cards usually only have suggested replies.
- Buzz's draft — if a reply was suggested, extract the draft text from the message.
- Source text — the quoted social/community message excerpt from the card.
- Mismatch — whether Buzz's suggestion differs from the team's reaction (
team_decision).
Don't rely on exact formatting — use your judgment to identify these elements from message content and blocks. The format may evolve over time.
3. Identify cases worth learning from
From your interpretation in Step 2, filter to cases that have learning value:
- Reaction mismatches — Buzz suggested X, team reacted Y. If the reason for the mismatch is obvious from context, include it. If it's ambiguous and there's no thread feedback, ignore it.
- Thread feedback — any thread where a human posted a message. Include these regardless of whether the reaction matched.
- No Buzz suggestion — Buzz didn't post a suggestion but the team did react or leave feedback. The team's reaction tells us the right call even without Buzz's take.
- No signal — no reaction and no feedback. Skip entirely.
If there are zero cases worth learning from across both channels, skip to step 6 (status update).
4. Learn from the cases
Use the reply-learning skill to analyze the combined batch of cases from both channels. Present them clearly:
For each case, provide:
- The channel
- The original social/community text
- Buzz's suggestion (or "no suggestion") and draft if applicable
- The team's reaction
- Any human feedback messages
- Whether it was a match, mismatch, or missing suggestion
The reply-learning skill will determine whether the cases reveal durable patterns worth codifying in draft-brand-reply/SKILL.md. Follow its process — it may conclude that no changes are needed even with mismatches (one-offs, ambiguous cases, etc.).
5. Open a PR if changes were made
If draft-brand-reply/SKILL.md was edited:
- Create a branch:
buzz/reply-learning-YYYY-MM-DD
- Commit the changes with a message explaining what was learned
- Open a ready-for-review PR (not a draft) using
gh pr create — do NOT pass --draft:
- Title:
draft-brand-reply: learnings from YYYY-MM-DD feedback
- Body: summary of what was learned, the specific cases that drove the changes, and the diff
- CODEOWNERS will auto-assign reviewers
If no edits were warranted, skip the PR.
6. Post status updates to Slack
Post one status message to each reviewed channel using BUZZ_SLACK_TOKEN.
If a PR was opened:
📝 Daily reply review (YYYY-MM-DD)
Reviewed N suggestions in this channel. Found X mismatches and Y feedback threads.
Opened a PR with improvements: <PR_URL|link>
Brief summary of what changed.
If no PR was opened:
📝 Daily reply review (YYYY-MM-DD)
Reviewed N suggestions in this channel. [Reason no PR was needed — e.g. "All suggestions matched team decisions" or "2 mismatches found but no durable patterns to codify"]
Use the Buzz bot identity. Include unfurl_links: False and unfurl_media: False. Keep the status concise; one top-level message per channel, no threads.
Constraints
- Do NOT fabricate feedback or reactions — only use what's actually in Slack
- Do NOT make changes to
draft-brand-reply unless the reply-learning process concludes they're warranted
- Do NOT include the Slack token in any output, log, or file
- Do NOT post the status update until all other steps are complete
- Keep the status update concise — one message, no threads
