mc-batch-contract-review

name	mc-batch-contract-review
description	Batch request the live MC/MeCheck contract-review agent for many local contract files. Use this when the user provides a folder, manifest, or list of .docx contracts and wants Codex to run online algorithm/agent reviews in bulk with fixed request parameters and a reusable user prompt whose dynamic part is the review side/stance, without Promptfoo scoring or prompt-version comparison.

MC Batch Contract Review

Use this skill to submit many local contracts to the live MC/MeCheck online review agent and save one result directory per contract.

This skill is execution-only. Do not run Promptfoo, do not change the online system prompt, and do not score against standard answers unless the user explicitly asks for evaluation afterwards.

Use the importable mc_eval.runners.mecheck_batch workflow. The bundled script is a compatibility wrapper around the same implementation, so CLI and harness runs cannot drift.

python packages/capabilities/mc-batch-contract-review/scripts/run_batch_review.py

This is the canonical live execution path for online MeCheck reviews in MC_Eval. Its output is normalized to the eval harness RunResult contract.

The default prompt profile is agent-default, which lets the online agent use its normal Skill routing. Use --profile legacy-no-skill only to reproduce the previous baseline that explicitly prohibited Skill use.

Before creating an online task, the script computes an exact content/configuration fingerprint and checks the shared data/cache/mecheck/ store plus completed historical Batch outputs. Reuse a completed local result immediately; if only a known online thread remains, recover its completed result. Use --force-rerun only when the user explicitly wants a fresh review. Filename and case ID are not cache identity fields.

Generate a batch manifest from the corpus registry:

python -m mc_eval.tools.manifest_to_batch_jsonl --case-ids C4 C5 -o src/mc_eval/tools/generated/batch-purchase-baseline.jsonl

The command defaults to the test environment https://mecheck3-test.alpha.milvzn.com. It requires MC_TOKEN in the environment, or in an env file passed with --env-file. If the user has not provided an account or token, try to reuse an existing local browser login for the same base URL before blocking. Pass --base-url only when the user explicitly wants another environment. Never print token values.

Default online endpoints:

• Upload file: /api/v3/file/upload
• Create contract: /api/v3/contract
• Create review task: /api/v3/review/create-task
• Start agent run: /copilot/api/v1/chat-runs
• Poll result: /copilot/api/v1/agent/conversations/{threadId}/review-result-view

Default base URL:

• https://mecheck3-test.alpha.milvzn.com

Default model:

• deepseek-v4-flash

If an older environment still uses the previous chat endpoint, pass --chat-endpoint /copilot/api/v1/chat-stream.

Inputs

Collect or infer:

• Contract files: a directory/glob of .docx files, or a manifest.
• Review side: usually 甲方 or 乙方; this is the only required dynamic part of the default user prompt.
• Model name: default to deepseek-v4-flash; if the user provides a model name, pass it with --model-name.
• Optional metadata: contract_name, contract_type, party_a_name, party_b_name, per-contract review_side.
• Output directory: default to ./mc-batch-contract-review-<timestamp>.
• Concurrency: default to --max-workers 5. Pass --max-workers 1 only when debugging a single contract or isolating an online issue.
• Result cache: default data/cache/mecheck; override with --cache-dir only when isolating a run.
• Prompt profile: default agent-default; use legacy-no-skill only for historical baseline comparison.

If the user supplies only files and one stance, generate the manifest or use --contract-glob. If different files need different stances, generate a JSONL manifest with one row per contract.

Token Setup

Before every real run, establish a valid MC_TOKEN.

Token source order:

1. Use MC_TOKEN already present in the shell environment.
2. Use MC_TOKEN from the provided --env-file, usually .env.local.
3. If the user has not provided an account or token, inspect the local browser for an existing authenticated session on the selected base URL, defaulting to https://mecheck3-test.alpha.milvzn.com.

When reusing a browser token:

• Search only the selected MC/MeCheck origin, such as https://mecheck3-test.alpha.milvzn.com.
• Look for an access token or bearer token in browser local storage, session storage, cookies, or network request authorization headers.
• Do not print the token, paste it into chat, include it in command output, or write it into normal run artifacts.
• Prefer passing the token to the script through process environment for the current command.
• If a temporary env file is unavoidable, create it outside the batch output directory with mode 600, use it only for the run, then delete it.

After obtaining a token, test it before uploading contracts. The bundled script automatically performs this validation before every non-dry-run batch by sending a lightweight authenticated request against the selected base URL. A 401 or 403 means the token is unusable. A response that proves the request reached an authenticated app endpoint, including an expected not-found response for a throwaway conversation id, is sufficient to proceed. Do not start uploading contracts until token validation passes.

Upload Smoke Test

When the user asks to test contract upload, use --upload-smoke-test.

Contract selection rules:

1. If the user provided a contract manifest or --contract-glob, use the first provided contract for the upload test.
2. If the user did not provide any contract, stop and ask the user to provide a .docx contract, manifest, or contract glob. Do not generate a synthetic contract.

Upload smoke testing only validates token and /api/v3/file/upload. It does not create a contract, does not create a review task, and does not start /copilot/api/v1/chat-runs.

python packages/capabilities/mc-batch-contract-review/scripts/run_batch_review.py \
  --manifest /path/to/contracts.jsonl \
  --output /path/to/output/upload-smoke-<timestamp> \
  --review-side 甲方 \
  --upload-smoke-test \
  --env-file /path/to/.env.local

Fixed User Prompt

The fixed user prompt lives in:

scripts/run_batch_review.py::USER_PROMPT_TEMPLATE

Do not recreate or paste the full prompt in normal use. Call the script with --review-side 甲方 or --review-side 乙方; it renders {review_side} into the prompt and writes the rendered text to each prepared manifest row as message.

For mixed stances, put review_side on each input manifest row. The script still ignores any existing message and regenerates it from the fixed template, so batch runs stay consistent.

Manifest Format

Prefer JSONL for batch jobs:

{"case_id":"contract-001","contract_file":"contracts/a.docx","contract_name":"A合同","contract_type":"采购合同","review_side":"甲方","model_name":"deepseek-v4-flash"}
{"case_id":"contract-002","contract_file":"contracts/b.docx","contract_name":"B合同","contract_type":"服务合同","review_side":"乙方","model_name":"deepseek-v4-flash"}

Accepted path fields are contract_file, contractFile, file, or path.

Accepted model fields are model_name or modelName. If omitted, the script uses --model-name, whose default is deepseek-v4-flash.

The script accepts .json, .jsonl, .csv, and --contract-glob.

Workflow

1. Build or receive a manifest/glob from the user's files.
2. Run the bundled script; it renders the fixed user prompt with the requested review_side and writes prepared-contracts.jsonl into the output directory.
3. Run a dry run first:

python packages/capabilities/mc-batch-contract-review/scripts/run_batch_review.py \
  --manifest /path/to/contracts.jsonl \
  --base-dir /path/to/base \
  --output /path/to/output/batch-contract-review-<timestamp> \
  --review-side 甲方 \
  --model-name deepseek-v4-flash \
  --base-url https://mecheck3-test.alpha.milvzn.com \
  --max-workers 5 \
  --dry-run

4. Inspect batch-summary.json; confirm the count matches the user's intended contract count.
5. Run a small real smoke batch, usually 1-3 contracts, unless the user explicitly says to run all immediately. Treat workspaceStatus: draft as incomplete even if items already contains risk cards.
6. Run the full batch:

python packages/capabilities/mc-batch-contract-review/scripts/run_batch_review.py \
  --manifest /path/to/contracts.jsonl \
  --base-dir /path/to/base \
  --output /path/to/output/batch-contract-review-<timestamp> \
  --review-side 甲方 \
  --model-name deepseek-v4-flash \
  --base-url https://mecheck3-test.alpha.milvzn.com \
  --stream-timeout 1800 \
  --poll-timeout 900 \
  --max-workers 5 \
  --env-file /path/to/.env.local \
  --resume

7. Rerun failed/pending contracts with --resume. Keep the default --max-workers 5 unless the online service is unstable; use --max-workers 1 to isolate failures.

--resume only resumes the current output directory. Cross-run deduplication is automatic through the shared content-addressed cache. --force-rerun bypasses both local reuse and online recovery without deleting prior artifacts.

Outputs

The batch output directory contains:

• prepared-contracts.jsonl: script-generated manifest with fixed rendered message.
• batch-summary.json: totals for success, failure, skipped, planned.
• batch-results.json: one row per contract with caseId, runDir, threadId, logs, and status.
• review-detail-links.json: one review detail link per successful completed contract, including contractName, reviewSide, and {baseUrl}/contract-review/detail?contractId={contractId}.
• review-detail-links.md: user-facing clickable Markdown table for completed contracts.
• items/<case_id>.json: per-contract status for resume.

Each successful contract has a run directory under runs/<case_id>/ inside the output directory with:

• request-body.json
• sse-events.jsonl
• review-result-view.json
• upload/create-task responses

A contract is successful only when review-result-view.json reports workspaceStatus: completed. If polling times out while the workspace is still draft, the script saves the latest intermediate review-result-view.json for debugging but marks the item as failed instead of success.

Final Response Requirements

After every real batch run, read review-detail-links.json or review-detail-links.md before replying.

The final response to the user must include:

• Output directory.
• Success/failure counts.
• Failed case ids, if any.
• A 审查详情链接 section containing a Markdown table with columns 合同文件名称, 立场, and 审查详情. The 审查详情 cell must be a directly clickable Markdown hyperlink, for example [查看详情](https://mecheck3-test.alpha.milvzn.com/contract-review/detail?contractId=5179).

Do not only provide the path to review-detail-links.md, and do not provide plain text URLs that the user has to copy. The final reply must include the table itself. For large batches, still include completed review links in the table; keep link text short, usually 查看详情. If there are no successful completed contracts, say that there are no completed review links yet.

Use review-detail-links.json as the machine-readable source of truth and review-detail-links.md as the generated user-facing link list. Do not paste full raw results unless the user asks.

Safety Rules

• Do not print MC_TOKEN, bearer tokens, cookies, or .env.local contents.
• Do not update online prompt. This script only submits review requests to the live APIs.
• Do not set with_checklist / checklist_id in the manifest unless the user explicitly provides checklist ids.
• Use --resume for reruns so successful contracts are not submitted again.
• Default concurrency is 5 contracts. Reduce with --max-workers 1 when debugging failures or when the online service is under load.
• Clean temporary manifests created outside the chosen output directory after the run.