batch-processing-protocol

# Batch Processing Protocol ## When to Use Use this skill when a user describes any of these situations: - The user processes the same type of item repeatedly throughout the day or week and wants to consolidate those interruptions into dedicated sessions (examples: reviewing expense reports as they arrive, answering similar support requests one at a time, manually logging sales calls, tagging and filing photos after each shoot) - The user asks how to stop letting accumulated work pile up into an overwhelming backlog -- they want a structured system for clearing a queue efficiently rather than letting it become stressful - The user wants a repeatable, written protocol they can follow without re-inventing the approach each time (examples: weekly newsletter curation, monthly contract renewals, quarterly performance review drafts) - The user mentions that context switching between different task types is hurting their focus and they want to consolidate like-for-like work into a single window - The user has a predictable volume of incoming items (even if timing is irregular) and wants to design a cadence that matches that volume rather than reacting to each arrival - The user is onboarding someone else to do their recurring processing work and needs a documented protocol the other person can follow without supervision - The user describes a backlog situation -- they have let items accumulate without a system and now want to design a protocol that handles both the backlog and the ongoing flow **Do NOT use this skill when:** - The user needs a software data pipeline or ETL process (use a software architecture or data engineering skill instead -- the concepts of batch trigger, queue, and processing loop apply but the implementation is entirely technical) - The user wants to design a business-wide operations process involving multiple departments, approval chains, or ERP systems (use a business process design or operations workflow skill instead) - The user wants to reduce how often they do a single recurring task, not how they process accumulated items (use `recurring-task-optimizer` -- the distinction is: batch processing handles queues of similar items, recurring task optimization handles the cadence of standalone tasks) - The user is asking about personal productivity frameworks like time blocking, deep work, or calendar architecture without a specific item type to process (use a time management or scheduling skill instead) - The user needs to prioritize a backlog of dissimilar tasks (use a task prioritization skill -- batching only works when items are similar enough to follow the same processing sequence) - The user wants to automate the processing using software tools (use an automation or workflow tool skill instead -- this skill designs human-executed protocols, not software automations) - The user wants to track habits or streaks around processing cadence (use a habit design skill instead) --- ## Process ### Step 1: Diagnose the Batch Candidate Before designing anything, gather precise information about what the user is processing. The protocol quality depends entirely on how well you understand the item type. - Ask or infer: **What type of item accumulates?** Be specific. "Emails" is too vague -- "vendor invoice emails requiring three-way match before payment approval" is precise enough to design a protocol. The item type determines whether the per-item sequence can be standardized. - Ask: **What is the current arrival rate?** Get a number: items per day, per week, or per month. If the user says "varies a lot," establish a low end, a typical week, and a high end. You will use all three to calibrate the trigger. - Ask: **How long does one item take today, end-to-end?** This is the per-item processing time. Users often underestimate this. If they say "2 minutes," probe what that includes: does it include finding the item, opening the relevant system, making a decision, filing it, and logging it? Every step counts. - Ask: **What is the acceptable delay?** Different item types have different urgency tolerances. A customer complaint email may have a 4-hour tolerance; a vendor invoice may have a 2-day tolerance; a book highlight to review may have a 2-week tolerance. The acceptable delay determines the maximum trigger interval. - Ask: **What tools or systems does processing require?** Every system the user needs to open adds friction. Identifying these upfront allows you to build them into the pre-batch setup checklist. - Ask: **What does "done" look like for one item?** If the user cannot articulate a clear done state, the per-item sequence will be ambiguous. Press for specificity: "filed in X folder with status updated in Y system" is a done state. "Handled" is not. - Look for **natural groupings** within the item type. If some items require a decision and others are purely mechanical, those may need separate processing sequences or a triage step. --- ### Step 2: Select and Configure the Batch Trigger The trigger is the most operationally critical component of the protocol. A poorly chosen trigger causes batches to be too large (marathon sessions), too small (constant context switching), or too infrequent (backlog buildup). - **Time-based triggers** work best when item arrival is predictable and the acceptable delay aligns with a clean calendar rhythm. Common examples: daily at 4 PM, Tuesday and Thursday at 9 AM, every Friday morning. The risk is that a time trigger fires when the queue is nearly empty (wasted setup overhead) or after a flood (batch is too large for the window). - **Count-based triggers** work best when arrival is irregular but volume is predictable. Set the count threshold at the batch size that fills roughly 60-75% of the available processing window. Example: if you have 60 minutes and each item takes 8 minutes, a threshold of 6-7 items fills the window without risking overrun. Never set the count threshold at the mathematical maximum -- leave 15-20% buffer for items that take longer than average. - **Hybrid triggers** (count OR time, whichever fires first) are the most robust for most personal processing situations. They protect against both the flood scenario (count trigger caps the batch) and the drought scenario (time trigger ensures the queue gets cleared even when few items arrive). Always recommend hybrid triggers unless the user's item type has a very consistent arrival rate. - **Event-based triggers** work well when processing naturally follows another activity. Examples: process the week's voicemails immediately after Friday team standup; process the week's receipts every time you complete an expense report; process interview feedback forms within 24 hours of the last interview in a hiring cycle. Event triggers are easy to remember because they piggyback on an existing anchor. - **Set a maximum batch size in BOTH items and minutes.** These are two separate ceilings, not alternatives. Example: "No more than 12 items AND no more than 75 minutes, whichever is reached first." This prevents marathon sessions when items are more complex than average. - **Calculate the trigger math explicitly:** (count threshold) x (per-item average time) = expected batch duration. This duration should land between 30 minutes and 90 minutes for most protocols. Under 30 minutes means the trigger fires too often for the overhead to be worth it; over 90 minutes means the session is too long to sustain focus without fatigue degrading quality. - For new protocols, recommend starting with a **slightly more frequent trigger than seems necessary.** It is easier to extend the interval once the user sees the queue size than to undo a backlog that built up because the trigger was too infrequent. --- ### Step 3: Design the Pre-Batch Setup Phase The pre-batch phase eliminates the setup friction that causes users to procrastinate on processing sessions. Every item in this checklist should be completable in under 2 minutes total. If setup takes longer than 2 minutes, the protocol will be avoided. - **Identify every tool and system the user needs open before the first item begins.** List them explicitly. "Open Gmail, navigate to the Pending Review label, open the expense tracker in a second tab, pull up the vendor contract folder in Drive" is a real pre-batch checklist. "Get your materials ready" is not. - **Include a workspace clearing step.** Close unrelated browser tabs, silence phone notifications (or set to Do Not Disturb), and set the physical or digital environment to reduce interruption. This is not optional -- cognitive interruptions during a batch session increase per-item time by 15-40% due to re-orientation overhead. - **Include a triage or sort step for batches where complexity varies significantly.** Before processing begins, do a 60-second scan of all items and sort them into: Quick (under half the per-item average), Standard (within normal range), Complex (above 1.5x the per-item average), and Exceptions (items that cannot wait or cannot be batch-processed). Process Exceptions immediately after setup. Then decide: process Quick items first (builds momentum, clears volume) or process Standard items first (most consistent pace). Complex items typically go last unless they have a deadline. - **Require the user to count items and estimate session duration before starting.** Write the count down: "13 items x 8 min = 104 min. That exceeds my 90-minute limit, so I will process 10 today and set the remainder for next batch." This prevents the most common batch processing failure: starting a session that was always going to overrun the available time. - **Set a timer for the total session.** Not optional. A timer creates a commitment device that prevents individual items from ballooning. When the timer fires, the batch is over regardless of the queue state. - **If this is the first run of the protocol or the first run after a long gap,** add a "warm-up" note: review the per-item processing sequence and the expense category reference (or equivalent domain reference) before starting, not during. Reviewing mid-batch creates pauses that break rhythm. --- ### Step 4: Design the Per-Item Processing Sequence This is the core of the protocol. The per-item sequence must be granular enough that the user is making decisions about content, not decisions about process. Process decisions kill batch momentum. - **Break each item into 3-6 discrete steps with time allocations.** Each step should have a clear action verb: verify, code, approve, tag, draft, file, log, flag. Vague verbs like "handle," "review," or "deal with" are not actionable in a timed sequence. - **Time-allocate each step.** If step 1 (verify) takes 1 minute, step 2 (process) takes 4 minutes, and step 3 (file) takes 1 minute, write those times explicitly. Users who see time allocations develop realistic pacing expectations and are less likely to let one step expand to fill the whole item window. - **Include a decision tree for every branch point.** The typical item has 3-4 states: - **Standard:** Item matches the expected pattern, process normally - **Edge case:** Item has an unusual attribute (unexpected amount, missing information, unfamiliar source) -- define the exact handling for each known edge case - **Escalation:** Item requires a decision or input from someone else -- define the escalation action (send one specific email, add to a specific list) and immediately remove the item from the active queue - **Defer:** Item cannot be processed right now for a defined reason -- define what constitutes a valid defer reason, how the item is marked, where it goes, and when it will be revisited - **Discard:** Item turns out not to need processing (duplicate, canceled, irrelevant) -- define what qualifies for discard and how it is marked - **Set a per-item time limit.** When the timer for a single item expires, the user must choose one of three actions: complete it (if within 30 seconds of done), defer it (if more work is needed), or escalate it (if it requires external input). They may not continue processing the item. This rule is the single most important mechanism for keeping a batch on schedule. - **Define the done state with physical precision.** "Done" should be a state you can verify by looking: the email is in the "Processed" label, the row in the spreadsheet is marked "Approved," the file is in the "Completed Q4" folder. A done state that requires judgment ("looks done," "feels handled") will cause the user to second-guess and re-open items. - **Minimize tool-switching within the per-item sequence.** Each time the user switches between applications or systems during item processing, they incur a ~30-second reorientation cost. Design the sequence to front-load reading (gather all information from all sources) before switching to acting (update all systems). "Read-then-act" sequences are faster than "read-act-read-act" sequences. - **Use parallel processing where the item type allows.** If one step involves waiting (sending an email and waiting for a status to refresh, or an upload to complete), design the next item's first step to happen during that wait rather than staring at a progress bar. --- ### Step 5: Design the Exception and Escalation Path Every batch protocol needs a fast lane for items that cannot wait for the scheduled batch. - **Define the exact criteria for an item that must be processed immediately, outside the batch.** These should be narrow and objective, not vague. "Marked urgent by the sender" or "NET 3 payment terms" or "client-facing deadline within 24 hours" are specific. "Feels important" is not a valid exception criterion and will cause the exception path to swallow the batching discipline. - **Design the exception path to be as lightweight as possible.** The goal is to handle the urgent item quickly and return focus to the current task (or the batch). Exception handling should not require reading the full protocol. Consider a one-sentence rule posted somewhere visible: "Process immediately only if [condition]. Otherwise, add to queue." - **Create a holding mechanism for items that arrive during a batch session.** When a new item arrives while the user is mid-batch, they should have a single action to "park" it without breaking flow: drag to pending folder, add to list, drop in inbox tray. Do not re-triage new arrivals during a batch -- that breaks the rhythm and reintroduces context switching. - **Distinguish between escalation (needs someone else) and deferral (needs more time or information from the same person).** These are different states with different follow-up actions and different expected resolution times. Mixing them in a single "deferred" pile creates ambiguity about next steps. --- ### Step 6: Design the Post-Batch Wrap-Up Phase The post-batch phase serves two functions: closing out the current batch cleanly and generating data that improves the protocol over time. Without this phase, the batch ends in a mess of half-finished states and the user has no feedback loop. - **Require a count of processed, deferred, and escalated items.** This takes 30 seconds and creates the data for the batch log. The ratio of deferred to processed is the most important signal: a consistent defer rate above 20% indicates the trigger is set too infrequently, the per-item time limit is too aggressive, or the item type contains too many exceptions to batch efficiently. - **Handle deferred items immediately after the batch, not later.** "Later" becomes never. Each deferred item should receive one specific next action before the user closes the batch: send the clarification email, schedule a call, add to the right list with a specific date, or escalate to the right person. This should take no more than 2-3 minutes total for a normal batch. - **Log batch metrics in a running table.** Minimum fields: date, items processed, items deferred, total time, observations. Over time this table reveals patterns: batches that consistently run over time signal a trigger threshold that is too high; batches that consistently finish in half the allotted time signal a trigger that is too sensitive. - **Reset the workspace** so the next batch starts clean: close the relevant tabs, clear the staging area, archive or move the completed items out of the active view. A cluttered workspace at the start of a batch creates a false sense that work is already in progress and makes it harder to count items accurately. - **Write one protocol observation per batch** (optional but recommended). This is a single sentence about something that felt inefficient, confusing, or time-consuming during the session. Over four to six batches, these observations reveal which steps need refinement before the formal protocol review. --- ### Step 7: Set the Protocol Review Cadence Batch protocols decay. The item type changes, volume changes, tools change, and the per-item sequence becomes outdated. Build a review process into the protocol itself so it stays useful. - **Set a review trigger based on batch count, not calendar time.** After every 10 batches is a better cadence than "quarterly" because batches that happen daily need review sooner than batches that happen monthly. Ten batches provides enough log data to identify patterns. - **Define specific review criteria.** "Review the protocol" is too vague. The review should check: Is the average per-item time trending more than 20% above baseline? Is the defer rate consistently above 15%? Is the trigger firing more than twice before the count threshold is reached (meaning items arrive slower than expected)? Is the trigger not firing on the count threshold before the time trigger fires (meaning items arrive faster than expected)? - **Include a "sunset condition."** If the item type stops arriving (the vendor relationship ends, the role changes, the project concludes), the protocol should be archived rather than left as an active protocol that never fires. Define what would cause this protocol to be retired. - **Version the protocol document.** When changes are made during a review, note the version number and date. Users who have shared the protocol with colleagues (for delegation or coverage purposes) need to know which version they are running. --- ### Step 8: Compile the Protocol Document Assemble everything into a single self-contained reference card. The test of a good protocol document: can the user pick it up after a 3-week absence and execute a batch without asking themselves what to do next? - **Make the per-item sequence a numbered list with checkboxes**, not prose. Prose requires reading; a numbered checklist with checkboxes enables execution. - **Put the trigger conditions on the first page**, not buried in the document. The trigger is the entry point; it must be instantly findable. - **Include the "done" and "defer" state definitions on the same page as the per-item sequence.** The user should not have to flip pages to remember what "done" looks like. - **Print or pin the protocol somewhere accessible** -- pinned note, printed reference card, saved browser tab. A protocol that requires searching for is a protocol that will not be used. --- ## Output Format ``` ## Batch Processing Protocol: [Protocol Name] **Version:** 1.0 | **Created:** [Date] | **Last Reviewed:** [Date] --- ### Overview | Field | Value | |---|---| | Item type | [Precise description of what is being processed] | | Arrival rate | [X items per day / week / month -- low: Y, typical: Z, high: W] | | Per-item processing time | [X minutes (breakdown: Step 1: Xm, Step 2: Xm, Step 3: Xm)] | | Acceptable delay | [X hours / days before processing is overdue] | | Recommended batch frequency | [Daily / 2x per week / Weekly / etc.] | | Expected batch duration | [X items x Y min = Z minutes per session] | | Maximum batch size | [X items OR Y minutes, whichever comes first] | --- ### Trigger Conditions **Fire a batch session when ANY of the following is true:** - [ ] **Time trigger:** [Day and time -- e.g., "Every Tuesday and Thursday at 9:00 AM"] - [ ] **Count trigger:** [X items have accumulated in the queue] - [ ] **Event trigger:** [Specific preceding event -- e.g., "Within 2 hours of completing weekly team standup"] **Hybrid rule:** The time trigger fires on schedule regardless of queue size. The count trigger fires early if the threshold is reached before the scheduled time. Never wait for both to be true. **Exception -- process immediately (outside the batch) if:** - [Urgent condition 1: Specific, objective criterion] - [Urgent condition 2: Specific, objective criterion] **Items that arrive during an active batch session:** [Specific parking action -- e.g., "Drag to Pending folder; do not triage during batch"] --- ### Pre-Batch Setup Checklist *Target: Complete in under 2 minutes* - [ ] Open [Tool 1]: navigate to [specific location / folder / view] - [ ] Open [Tool 2]: navigate to [specific location / folder / view] - [ ] Open [Reference document]: [name and location] - [ ] Close all unrelated browser tabs - [ ] Set phone / notifications to Do Not Disturb - [ ] Count items in queue: ______ items - [ ] Calculate session time: ______ items x [Y] min = ______ minutes - [ ] If calculated time exceeds [maximum], limit this batch to [N] items and schedule remainder - [ ] Triage: sort items into Quick / Standard / Complex / Exception (60-second scan) - Quick (under [X] min each): ______ items - Standard ([X]-[Y] min each): ______ items - Complex (over [Y] min each): ______ items - Exceptions (process immediately): ______ items - [ ] Process any Exceptions now, before starting the batch timer - [ ] Set batch timer for ______ minutes - [ ] Begin --- ### Per-Item Processing Sequence **Processing order:** Exceptions (pre-batch) → Quick items → Standard items → Complex items **Per-item time limit:** [X] minutes. At the limit, complete within 30 seconds or defer immediately. --- **For each item:** **Step 1: [Initial Assessment / Verify] -- [X] minutes** - [Specific check 1 -- e.g., "Confirm sender matches known vendor list"] - [Specific check 2 -- e.g., "Confirm item is not a duplicate (check log)"] - IF [discard condition]: [Discard action] -- move to next item - IF [escalation condition]: [Escalation action] -- remove from queue -- move to next item **Step 2: [Core Processing Action] -- [X] minutes** - [Specific action 1] - [Specific action 2] - IF [standard condition A]: [Action A] - IF [standard condition B]: [Action B] - IF [edge case condition]: [Edge case action] - IF unclear, ambiguous, or approaching time limit: [Defer action] -- move to next item **Step 3: [Secondary Processing / Decision] -- [X] minutes** - [Specific action] - IF [threshold condition -- e.g., amount over $X, requires approval from Y]: [Escalation action] **Step 4: [Completion / Filing / Logging] -- [X] minutes** - [File / archive action with specific destination] - [System update action -- e.g., "Update row in tracking spreadsheet: status = Approved"] - [Log action -- e.g., "Add to monthly batch log: date, item ID, category, outcome"] --- **Done state:** [Precise, verifiable description -- e.g., "Email moved to 'Processed' label, row in tracker marked 'Approved,' entry added to monthly log"] **Defer state:** [Precise description -- e.g., "Email moved to 'Needs Attention' label, sticky note added with specific blocker, name added to Deferred list with date and reason"] **Escalation state:** [Precise description -- e.g., "Forwarded to [person/role] with [specific subject line], added to Escalated tab in tracker, date escalated noted"] **Discard state:** [Precise description -- e.g., "Archived with label 'Duplicate' or 'Not Applicable,' no log entry needed"] --- ### Post-Batch Wrap-Up Checklist *Target: Complete in under 5 minutes* - [ ] Stop timer. Record actual session time: ______ minutes - [ ] Count outcomes: - Items completed (Done): ______ - Items deferred: ______ - Items escalated: ______ - Items discarded: ______ - [ ] For each deferred item: assign one specific next action and date in [deferred list location] - [ ] Check deferred list: any item deferred for more than [X days]? Escalate immediately. - [ ] Log batch in Batch Log table (below) - [ ] Clear workspace: [specific reset actions] - [ ] Write one protocol observation (optional but recommended): ______________________ - [ ] Confirm next batch trigger: [next date/time or count threshold reminder] --- ### Batch Log | Date | Quick | Standard | Complex | Deferred | Escalated | Actual Time | Defer Rate | Notes |