sensing-track

# Sensing Event History ## Quick Start The primary data source is the **flow events JSONL** at `/root/local/flow_events_YYYY-MM-DD.jsonl`. Each file covers one calendar day (7-day retention, no size-rotation mid-day). Use Bash + `jq` to query it. > **Important:** Always use the absolute path `/root/local/` — the `read` tool cannot access files outside the workspace, so use `exec` (Bash) for all JSONL queries. Persistent camera snapshots are stored under `/var/lib/hal/snapshots/sensing_<prefix>/<ms>.jpg` (72h TTL, 50 MB cap) — one subdir per event kind: | Event type | Folder | |---|---| | `presence.enter`, `presence.leave` | `sensing_face/` | | `motion.activity` | `sensing_motion_activity/` | | `emotion.detected` | `sensing_emotion/` | Reference these when the user asks what happened visually. ## JSONL format Each line is a JSON object: ```json {"kind":"enter","node":"sensing_input","ts":1712345678.123,"seq":42,"trace_id":"run-abc","data":{"type":"presence.enter","message":"Person detected — new: friend (gray); faces in frame: 1 (gray)\n[snapshot: /var/lib/hal/snapshots/sensing_face/1712345678123.jpg]"},"version":"1.2.3"} {"kind":"exit","node":"sensing_input","ts":1712345678.456,"seq":43,"trace_id":"run-abc","duration_ms":332,"data":{"path":"agent","run_id":"run-abc"},"version":"1.2.3"} ``` Key fields: - `node` — filter on `"sensing_input"` for sensing events - `kind` — `"enter"` = event received, `"exit"` = event processed (with `duration_ms`) - `data.type` — event type: `presence.enter`, `presence.leave`, `motion`, `motion.activity`, `sound`, `light.level`, `voice`, `voice_command`, `emotion.detected`, `speech_emotion.detected` - `data.message` — natural-language description; may contain `[snapshot: /var/lib/hal/snapshots/sensing_<prefix>/<ms>.jpg]` - `data.path` — in `exit` records: `"agent"` (forwarded), `"local"` (handled locally), or has `"error"` key (failed/dropped) - `ts` — Unix timestamp (seconds with fractional ms) - `trace_id` — correlates enter/exit and links to agent turn ## Tools **Bash** — `jq`, `cat`, date arithmetic. No writes. --- ## Query recipes ### Timezone — always set before date arithmetic ```bash export TZ=$(cat /etc/timezone) ``` ### All sensing events in a time range ```bash export TZ=$(cat /etc/timezone) DATE="$(date +%Y-%m-%d)" FROM_TS=$(date -d "$DATE 22:00:00" +%s) TO_TS=$(date -d "$DATE 23:59:59" +%s) jq -c 'select(.node=="sensing_input" and .kind=="enter" and .ts >= '"$FROM_TS"' and .ts <= '"$TO_TS"')' \ "/root/local/flow_events_${DATE}.jsonl" ``` ### Events of a specific type in the last N hours Use `"motion"` for raw motion, `"motion.activity"` for activity analysis (HAL-categorised — bucket names `drink`/`break`/`celebrate` and raw Kinetics sedentary labels like `using computer`, `writing`, `reading`). Most queries want both: For any relative range, select every local calendar day's file intersecting that range, including yesterday when crossing midnight. Run this range setup and the chosen query in the same Bash call; do not assume variables survive between tool calls. GNU `date -d` is available on the device. ```bash export TZ=$(cat /etc/timezone) SINCE=$(date -d "1 hour ago" +%s) UNTIL=$(date +%s) first_day=$(date -d "@$SINCE" +%F) last_day=$(date -d "@$UNTIL" +%F) range_files=() range_day=$first_day while [[ "$range_day" < "$last_day" || "$range_day" == "$last_day" ]]; do range_file="/root/local/flow_events_${range_day}.jsonl" if [ -r "$range_file" ]; then range_files+=("$range_file") else printf 'History unavailable for %s\n' "$range_day" >&2 fi range_day=$(date -d "$range_day +1 day" +%F) done if [ "${#range_files[@]}" -gt 0 ]; then jq -c --argjson since "$SINCE" --argjson until "$UNTIL" \ 'select(.node=="sensing_input" and .kind=="enter" and .ts >= $since and .ts <= $until and (.data.type=="motion" or .data.type=="motion.activity"))' \ "${range_files[@]}" fi ``` ### Any activity in the last N minutes Use the same range setup above with `SINCE=$(date -d "30 minutes ago" +%s)` and remove the type predicate from the `jq` filter. Keep both time bounds and all `range_files`; even 30 minutes can span two local dates. ### Presence events only (who came by) Names in messages are lowercase (`friend (gray)`). Use `test()` with `"i"` flag for case-insensitive search: ```bash TODAY=$(date +%Y-%m-%d) # All presence events jq -c 'select(.node=="sensing_input" and .kind=="enter" and (.data.type=="presence.enter" or .data.type=="presence.leave"))' \ "/root/local/flow_events_${TODAY}.jsonl" # Search for a specific person (case-insensitive) jq -c 'select(.node=="sensing_input" and .kind=="enter" and .data.type=="presence.enter" and (.data.message | test("gray";"i")))' \ "/root/local/flow_events_${TODAY}.jsonl" ``` ### Events spanning multiple days ```bash export TZ=$(cat /etc/timezone) YESTERDAY=$(date -d "yesterday" +%Y-%m-%d) TODAY=$(date +%Y-%m-%d) cat "/root/local/flow_events_${YESTERDAY}.jsonl" "/root/local/flow_events_${TODAY}.jsonl" \ | jq -c 'select(.node=="sensing_input" and .kind=="enter" and .ts >= '"$FROM_TS"' and .ts <= '"$TO_TS"')' ``` ### Dropped events (agent was busy) ```bash TODAY=$(date +%Y-%m-%d) jq -c 'select(.node=="sensing_input" and .kind=="exit" and .data.error != null)' \ "/root/local/flow_events_${TODAY}.jsonl" ``` ### List snapshots (72h TTL — older files may be purged) Snapshots are bucketed into `sensing_face/` (presence), `sensing_motion_activity/`, `sensing_emotion/`. Recurse into subdirs: ```bash # Most-recent snapshots across all categories find /var/lib/hal/snapshots -type f -name '*.jpg' -printf '%T@ %p\n' | sort -rn | head -20 | cut -d' ' -f2- # Only a specific category ls -lt /var/lib/hal/snapshots/sensing_motion_activity/ | head -20 ``` ### Pose buckets (posture history) Posture snapshots are NOT in `/var/lib/hal/snapshots/` — they live in tmp under a per-window bucket layout at `/tmp/hal-sensing-snapshots/sensing_pose/buckets/<bucket_id>/`. A bucket only exists when a tumbling window closed with bad posture (`bad_ratio >= POSE_BAD_RATIO`). Kept buckets survive ~2 days (`POSE_BUCKET_KEEP_S`); windows that didn't fire a nudge are deleted immediately, so the buckets you can see are by definition "bad posture" sessions. Each kept bucket contains: - `<sample_ts>_<score>.jpg` — annotated frame per sample (skeleton overlay + RULA score) - `bucket.json` — manifest: - `bucket_id`, `window_start_ts`, `window_end_ts`, `kept: true` - `summary` — same shape as the `[posture_summary:]` block on `motion.activity` (`bad_ratio`, `dominant_region`, `samples`, …) - `samples[]` — `{ts, score, risk_level, filename, left, right}` (per-side RULA body_scores + angles) - `worst_snapshots[]` — pre-selected worst filenames (the ones the device auto-attaches to `/dm` on posture nudges) ```bash # List recent buckets (newest first) ls -lt /tmp/hal-sensing-snapshots/sensing_pose/buckets/ | head -10 # Read a specific bucket's manifest jq . /tmp/hal-sensing-snapshots/sensing_pose/buckets/1779259742/bucket.json # Buckets that closed in the last 2 hours find /tmp/hal-sensing-snapshots/sensing_pose/buckets -maxdepth 1 -type d -mmin -120 -name '[0-9]*' | sort # Worst-frame paths from the latest kept bucket LATEST=$(ls -t /tmp/hal-sensing-snapshots/sensing_pose/buckets/ | head -1) jq -r '.worst_snapshots[]' "/tmp/hal-sensing-snapshots/sensing_pose/buckets/${LATEST}/bucket.json" \ | sed "s|^|/tmp/hal-sensing-snapshots/sensing_pose/buckets/${LATEST}/|" # Today's bad-posture sessions — bucket id == window_start unix-seconds TODAY_START=$(date -d "today 00:00" +%s) for b in /tmp/hal-sensing-snapshots/sensing_pose/buckets/*/bucket.json; do jq --arg start "$TODAY_START" 'select((.window_start_ts | floor) >= ($start | tonumber)) | {bucket_id, dominant: .summary.dominant_region, bad_ratio: .summary.bad_ratio, started: .window_start_ts}' "$b" done ``` Note: `motion.activity` event messages contain `[pose_bucket: <id>]` and `[pose_worst: <fn1>,<fn2>,...]` markers — parse these out of `data.message` when you need to map a sensing_input record to its bucket. Markers are present whenever a posture nudge folded in. --- ## Fallback: system log For detailed debugging or when you need Go-side log context (errors, warnings, lifecycle details), fall back to `${OS_LOG:-/var/log/os-server.log}`: ```bash LOG="${OS_LOG:-/var/log/os-server.log}" sed 's/\x1b\[[0-9;]*m//g' "$LOG" | grep "sensing event received" ``` The system log uses lumberjack rotation (1 MB cap, 3 backups) — it may miss data during high traffic. Use it only when JSONL doesn't have enough detail, or when investigating bugs. --- ## Mood history A dedicated mood history log tracks **user mood** per user. Only the user's emotional state is logged — not system events or device emotions. Each user's mood data lives in their own directory. **Read API:** ```bash # Current user's mood history (auto-detects who's present) curl -s "http://127.0.0.1:5000/api/openclaw/mood-history?date=$(date +%Y-%m-%d)&last=100" # Specific user's mood history curl -s "http://127.0.0.1:5000/api/openclaw/mood-history?user=gray&date=$(date +%Y-%m-%d)&last=100" ``` **Write:** Follow the **Mood** skill to log user mood from camera or conversation. ```json {"ts":1776138500,"seq":1,"hour":10,"mood":"happy","source":"camera","trigger":"laughing"} {"ts":1776139200,"seq":2,"hour":10,"mood":"stressed","source":"conversation","trigger":"user said feeling overwhelmed"} ``` Storage: `/root/local/users/{name}/mood/YYYY-MM-DD.jsonl` (30-day retention). ---