| name | scrask-bot |
| version | 3.0.0 |
| description | When the user sends a screenshot via Telegram, parse it using Gemini (fast, default) with automatic Claude fallback when confidence is low. Saves results to Google Calendar (events) or Google Tasks (reminders and tasks). Saves silently if confidence is high; asks for confirmation if uncertain.
|
| author | your-name |
| metadata | {"openclaw":{"emoji":"🦞","primaryEnv":"GEMINI_API_KEY","requires":{"env":["GOOGLE_CREDENTIALS","GEMINI_API_KEY"],"bins":["python3"]},"config":{"vision_provider":{"type":"string","description":"Vision model provider. 'auto' = Gemini first, Claude fallback if any item confidence < fallback_threshold. 'gemini' = Gemini only. 'claude' = Claude only.\n","default":"auto"},"fallback_threshold":{"type":"number","description":"Confidence floor for auto mode. If any item is below this, Claude reruns the parse.","default":0.6},"timezone":{"type":"string","description":"User's IANA timezone. Used when none is detected in the screenshot.","default":"UTC"},"confidence_threshold":{"type":"number","description":"0.0–1.0. Items below this score ask for confirmation. Items above save silently.","default":0.75},"reminder_minutes_before":{"type":"integer","description":"Popup reminder lead time in minutes for Google Calendar events.","default":30}}}} |
Scrask Bot
Overview
This skill activates when the user sends a screenshot via Telegram.
It uses vision AI to extract actionable information from the image, then:
- High confidence (≥ 0.75): Saves immediately and replies with a brief confirmation.
- Low confidence (< 0.75): Shows a structured preview in Telegram and asks for confirmation before saving.
Provider behaviour (auto mode, default):
| Step | What happens |
|---|
| 1 | Gemini 2.0 Flash parses the screenshot (fast, cheap) |
| 2 | If any item confidence < 0.60, Claude Opus reruns the parse |
| 3 | Whichever provider scores higher average confidence wins |
| 4 | Output includes provider, fallback_triggered, and confidence delta |
Set vision_provider to "claude" or "gemini" to lock a specific provider.
Output destinations (AI-decided by content type):
| Detected type | Destination |
|---|
| Event (has date+time, venue, or invite link) | Google Calendar |
| Reminder (deadline, due date, personal action) | Google Tasks (with due date) |
| Task (no date, pure action item) | Google Tasks (no due date) |
Trigger Conditions
Activate when:
- The user sends a message in Telegram that contains an image attachment
- The image appears to be a screenshot — not a photo of a person, place, or physical object
- No other skill has already claimed the image
Do not activate for:
- Photos of people, places, food, scenery
- Screenshots of code, errors, or UI bugs (leave for other skills)
- Images the user explicitly asks to edit, describe, or analyze for another purpose
Step-by-Step Instructions
Step 1: Acknowledge Immediately
Reply in Telegram right away so the user knows the skill is working:
"📸 Got it — analyzing your screenshot..."
Do not make the user wait silently.
Step 2: Run the Parser
python3 ~/.openclaw/skills/scrask-bot/scripts/scrask_bot.py \
--image-path "<path-to-temp-image>" \
--provider "$CONFIG_VISION_PROVIDER" \
--timezone "$CONFIG_TIMEZONE" \
--google-credentials "$GOOGLE_CREDENTIALS"
The script auto-resolves the API key from ANTHROPIC_API_KEY or GEMINI_API_KEY
depending on the provider — no need to pass it explicitly.
The script returns a JSON object with:
success — whether parsing workedno_actionable_content — true if nothing foundresults[] — one entry per detected item, each with confidence, type, destination, needs_confirmation, action_takentelegram_reply — the pre-formatted message to send back to the user
Step 3: Handle the Output
If no_actionable_content is true:
Reply: "🤷 I couldn't find any event, reminder, or task info in that screenshot. Could you describe what you'd like to add?"
If success is true:
Send the telegram_reply value directly back to the user in Telegram. The script has already:
- Saved high-confidence items silently
- Formatted confirmation prompts for low-confidence items
Do not rephrase or reformat the telegram_reply — send it as-is.
Step 4: Handle Confirmation Responses
If the script returned items with needs_confirmation: true, wait for the user's reply.
"yes" or "save" or "add":
Re-run the script for that specific item with confirmed=true, or use the calendar_create / tasks_create tools directly with the extracted fields.
"edit":
Ask what to change, update the relevant field, then save.
"skip" or "no":
Reply: "Got it, skipped ✓"
Step 5: Confirm Saves
For items saved silently (high confidence), the telegram_reply from the script already contains the confirmation message. Examples of what the user will see:
📅 Added to Calendar: **Team Standup** — 2026-03-01 at 09:00🔔 Added to Tasks: **Pay electricity bill** (due 2026-02-28)✅ Added to Tasks: **Review PR for Arjun**
Edge Cases
| Scenario | Behavior |
|---|
| Screenshot is in Hindi, Tamil, or another language | Extract and translate silently; save title in English |
| Recurring event ("every Monday") | Set RRULE on the calendar event; mention it in the reply |
| Date has already passed | Flag in the reply: "⚠️ This date has already passed (Feb 10). Save anyway?" |
| Multiple items in one screenshot | Process each independently; confirm per item if needed |
| Screenshot of someone's calendar | Detect already_in_calendar_hint; reply: "Looks like this event is already in your calendar 🗓️" |
| Google API auth failure | Reply with the specific error and suggest re-checking GOOGLE_CREDENTIALS |
| Zoom/Meet link found | Add to Calendar as both location and description |
Configuration
{
"skills": {
"entries": {
"scrask-bot": {
"enabled": true,
"env": {
"GEMINI_API_KEY": "AIza-your-gemini-key",
"ANTHROPIC_API_KEY": "sk-ant-your-key-here",
"GOOGLE_CREDENTIALS": "/home/user/.openclaw/google-creds.json"
},
"config": {
"vision_provider": "auto",
"fallback_threshold": 0.60,
"timezone": "Asia/Kolkata",
"confidence_threshold": 0.75,
"reminder_minutes_before": 30
}
}
Permissions Required
image:read — to access the screenshot from Telegramnetwork:outbound — to call Anthropic API and Google APIstelegram:reply — to send confirmation messages back to the user
