edge-intake-backfill

name	edge-intake-backfill
description	Class-level workflow for triaging vadimgest edge/local source batches into sourced inbox notes and exact checkpoint commits.

Edge intake backfill

Use this skill when a pre-run batch of vadimgest edge/local source records needs to be compacted, routed, and checkpointed into the MyBrain inbox/state-log layer.

Core goal

Turn grouped raw records into durable, sourced summaries without writing one note per row.

Workflow

1. Read the batch as grouped conversations/activities, not as individual rows.
2. Compact each channel/thread group first.
3. Deduplicate repeated facts and repeated message windows inside the same group.
4. For every group, answer four questions:
   • What needs to be done?
   • How can I help now?
   • What facts changed?
   • What patterns/signals changed?
5. Route durable facts to the smallest correct home:
   • clear entity target -> update the entity note with sourced state/log facts
   • unclear or cross-entity target -> write one sourced Inbox backfill note for the batch
6. Include source provenance in every durable summary:
   • line ranges when available
   • representative vadimgest://... URIs for each group
7. Treat metadata-only rows, headers, participant lists, and synthetic smoke as noise unless they carry a real state change.
8. After triage/writeback, run the exact checkpoint snippet for the batch and commit only the listed source lines.

Routing rules

• Prefer one grouped note per thread/channel, not one note per row.
• If a thread is active but the target note is uncertain, use a sourced inbox note first instead of guessing.
• If a record contains business, deal, legal, research, relationship, commitment, or process facts, preserve it.
• If a source window is repeat-heavy, summarize the delta, not the entire transcript.
• GTasks backlog snapshots are evidence, not a prompt to duplicate tasks unless the batch introduces a genuinely new action item.
• Skip only truly unusable rows: empty headers, synthetic inventory, service codes, and stale notifications with no new fact.

Checkpointing

• Use the exact commit lines supplied by the batch.
• Do not commit sources not listed in the batch snippet.
• If any source cannot be fully processed, do not advance its checkpoint line.
• Verify the checkpoint write happened before reporting success.

Output shape

When reporting the batch, keep it compact and include:

• a short summary of the batch
• which notes/files were written
• whether the exact checkpoint snippet ran
• a ---DELTAS--- JSON array with the processed sources, line ranges, and note paths

Pitfalls

• Do not create one note entry per raw row when a conversation is repeated across many lines.
• Do not treat contact headers or call participant stubs as meaningful content unless a real message is present.
• Do not mix raw evidence with inference without marking the distinction.
• Do not commit early; triage/writeback comes first, checkpoint second.
• Do not broaden the write target when a sourced inbox note is enough.
• When one thread in a noisy batch has a durable entity delta, write only that delta into the entity note and keep the rest of the batch in one compact Inbox note instead of over-splitting the residue.
• When appending to an existing long Inbox note, re-read the exact tail or the full file before patching; partial reads can make patch context ambiguous.
• If a batch is being appended to a long note that already has prior batch headings, append a fresh dated section at EOF instead of trying to replace a repeated heading.
• Treat patch warnings about partial reads / duplicate headings as a hard prompt to refresh the tail before retrying.
• If the note was read with offset/limit pagination, treat the visible span as partial and re-read the exact tail before patching.
• Never paste read_file output verbatim into a note — strip the LINE_NUM| prefixes first; paginated reads include them in the tool output.
• If a patch cannot anchor cleanly because the same batch heading repeats, switch to a full-file append/write path instead of forcing a fuzzy replacement.
• After patching or appending, re-read the edited span to make sure the new batch section is clean and no stray markers survived.
• If a long inbox note already contains a stale or duplicate batch heading, prefer appending a fresh dated section at EOF and then verifying the top and tail, rather than trying to surgically replace a heading that may appear more than once.
• When a long inbox note is being updated across retries, verify the final rendered markdown is still valid; a bad patch can leave literal |-prefixed lines or duplicated heading markers that need cleanup before checkpointing.
• For mixed launch batches, split the durable writes by function: project note for the core launch state, person notes for strong inbound reactions/questions, and one compact Inbox addendum for residue/noise.
• If the checkpoint snippet is provided, run it only after writeback has been verified, then confirm the checkpoint file changed.
• See references/2026-06-30-edge-backfill-session-lesson.md for the concise tail-read → append-at-EOF → verify → checkpoint pattern from this session.
• See references/2026-06-30-edge-backfill-batch-lessons.md for the end-to-end pattern from this batch (group first, append at EOF, verify, checkpoint).
• See references/2026-06-30-batch-42-lessons.md for a compact recap of the exact routing/writeback/checkpoint pattern from batch 42.
• See references/2026-07-01-mixed-batch-routing.md for a concise note on keeping one mixed WhatsApp/Signal batch in a single Inbox note when the only durable signal is a few high-thesis thread groups.
• See references/2026-07-07-edge-backfill-mixed-telegram-signal-dayflow.md for a mixed Telegram / Signal / Dayflow example where the project note, person note, and batch Inbox addendum were split cleanly before the exact checkpoint commit.

Related skills

• source-backed-intake-writeback
• source-backed-intake-triage
• vadimgest-intake-routing

References

• See references/2026-06-30-edge-backfill-example.md for a worked example of grouped backfill, provenance formatting, and exact checkpointing.
• See references/2026-06-30-edge-backfill-batch-lessons.md for the mixed WhatsApp/Gmail/Signal/GitHub batch routing pattern from this session.
• See references/2026-06-30-edge-backfill-session-lesson.md for the tail-read / append-at-EOF / verify-before-checkpoint recovery pattern.
• See references/2026-06-30-batch-41-routing.md for the concrete pattern from this batch: one compact inbox note plus two clear entity-note updates, then checkpoint verification.