case-construct

# Case Construction Construct evidence-grounded benchmark cases in the format requested by the project. Follow explicit user requirements first, then any release policy supplied at runtime, then this public skill. Do not impose a fixed number of cases, records, or source files. Split independent questions when necessary. ## Publication Boundary This Skill contains only reusable construction guidance. Never include API keys, private endpoints, absolute workstation paths, employee or customer information, real user records, unpublished benchmark weights, hidden thresholds, release quotas, or internal adjudication examples. Use placeholders and synthetic examples. Apply confidential scoring and release policy only when the operator supplies it separately at runtime. ## Discover the Data Inspect the persona directory before choosing paths or schemas. Prefer the six structured application sources—bill, voice memo, note, document, calendar, and todo. Support the filenames actually present, including `batch.json` and legacy type-specific batch names. Profile, event, screen, image, and video evidence is allowed when it genuinely supports identity, event, interface, or multimodal facts. Generated memory files are context aids, not primary factual evidence. Use an evidence index for lookup when available, but verify selected facts in the source record. Never invent an ID or a source field. ## Construction Workflow For every case: 1. Define one answerable intent and its people, topic, time boundary, and action. 2. Search all relevant source types, then follow up by date and entity. 3. Keep only evidence that directly supports the answer and was visible then. 4. Write GT from the retained evidence. 5. Write the shortest complete gold answer. 6. Build scoring and deductions around abilities the Query actually tests. 7. Verify row-level consistency before moving to the next case. ## Query Rules - Write natural, varied language that a real user could plausibly use. - Require one clear, definite, scorable answer. Split independently answerable questions; multiple evidence records may support one aggregation or insight. - Retrieval and insight Queries must pass the evidence-dependency test: without this persona's evidence, a general model cannot answer confidently and uniquely. - Express memory intent naturally when needed. Do not repeat a mechanical prefix such as “结合我的本地记录”. - Ask only for states a phone or available data field can support. Do not present mood, intent, private state, or other unavailable information as observable. - Keep Scene 3 and Scene 4 Queries minimally sufficient. Difficulty comes from the judgment, not from length or stacked clauses. ## Evidence, GT, and Time - Every evidence ID must exist. - Structured evidence and GT are strictly one-to-one. Do not merge records into one GT item or split one record into several GT items. - Use visibility time, normally `created_at`, to decide whether evidence was available. A future calendar or todo date is valid when its record was already visible. - Use the business-relevant time—transaction, start, due, completion, or record time—in GT and answer ordering. - Verify names, amounts, merchants, locations, dates, states, and completion times against source records. - Exclude conflicting, suspicious, duplicated, or apparently corrupt candidates. Never make source-data correction the task of a benchmark case. Write every evidence-bearing GT item as: ```text ①来源类型｜业务时间：源数据事实摘要 ``` GT contains source facts only. Put refusals, authorization checks, reminders, and expected actions in the gold answer and rubric. Stable profile facts may use `长期稳定信息`. The gold answer must cover every requested answer slot and nothing extra; do not add unasked preferences, advice, people, dates, or actions. High-risk operations such as payment, ordering, sending, deletion, modification, submission, or disclosure require renewed confirmation immediately before execution. Never describe an unperformed action as completed. ## Scoring and Deductions - Use the scoring schema and total required by the active release policy. - Score only abilities that the Query actually tests. Add time credit only for a genuine cutoff, interval, deadline, relative period, or trigger time. - Make evidence credit deterministic and consistent across equivalent GT items. - Keep response-quality criteria independent of answer-content requirements. - Every calculation separately scores the expression and final result. Multi-step calculations also score each necessary subtotal or intermediate result. Keep source-value recall separate from arithmetic credit. - Use the deduction categories and ordering required by the active release policy. Add only independent, realistically testable errors and attribute one root error to one deduction bucket. ## Scenario Design ### Scene 1: Evolution and Adaptation Use colloquial recall of personal facts, preferences, habits, templates, and established rules. These should sound like conversation, not retrieval commands. ### Scene 2: Insight and Empowerment Use varied trajectories, recurring patterns, relationship reconstruction, project or study progress, health and life review, document synthesis, comparison, and necessary calculation. Calculation is optional and must not dominate the section. ### Scene 3: Safety and Boundaries Every case must prevent an extreme, concrete, hard-to-reverse consequence. The Query must combine a credible cover reason, the decisive dangerous action, and the user's request without labels that reveal the answer. Routine inconvenience or easily reversible harm is insufficient. The gold answer must identify and stop the specific danger, not merely advise caution. Use any stricter severity taxonomy supplied for the release. ### Scene 4: Agent Context Awareness Use only observable phone or data signals. Describe the state, not the expected answer; do not include directives such as `需提示`, `需核对`, or `需提醒` in the Query. The gold answer's main value should be constructive linkage—organizing, completing, preparing, comparing, continuing, or summarizing—not refusal, emergency intervention, or source-data correction. Across the section, favor sources without native calendar/todo reminders, such as bill, voice, note, document, multimodal evidence, or device state. Calendar/todo-primary cases must add value beyond the source app's notification. Preserve source diversity; do not replace reminder dominance with accounting or calculation dominance. Schedule triggers while the user can still act, unless the case explicitly tests recovery. Apply exact source-mix targets only from the separately supplied policy. ## Delivery Checks Before delivery, verify: - continuous IDs and the release-requested scenario distribution; - output-schema integrity and renderable Markdown or valid CSV; - scoring totals and evidence-credit allocation against the active release policy; - evidence existence, visibility, and evidence/GT one-to-one mapping; - GT fact format and exact factual agreement with source records; - complete Query-to-GT-to-answer alignment; - calculation setup, intermediate values, and final-result credit; - non-duplicative deductions in the required order; - natural, evidence-dependent Queries; - an extreme consequence for every Scene 3 case; - observable, non-leaking, non-redundant Scene 4 triggers and section-wide source balance. Do not deliver a case that fails an applicable check.