dataroom-extract

name	dataroom-extract
description	Use when the user uploads an oil & gas dataroom — a zip or set of acquisition/divestiture files (lease operating statements, check stubs, AFEs, production reports, title, division orders, a teaser or overview) — and wants it turned into structured data for deal valuation.

Dataroom Extract

What you're doing

The user uploaded an oil & gas dataroom: the due-diligence package for buying a set of wells, minerals, or royalties. Your job is to read it and produce one structured file, extraction.json, that a downstream deal-valuation step consumes. The buyer needs to know what wells and interests are in the package and what the economics look like — and to trust every number back to the file it came from.

You are NOT writing a report or running a valuation. You are extracting facts, with provenance, into a fixed schema.

What you're building toward

• Output contract: extraction.json, an ExtractionResult exactly as defined in schema.py (bundled here). Read schema.py once — it is the authoritative target. Every field is optional and typed; there is no escape-valve extras dict.
• Worked example: example.json is a complete, filled extraction for a small synthetic deal. Match its shape exactly — every record carries a provenance block; entity lists are empty/omitted when the room doesn't support them.
• Then a viewer: a self-contained React artifact — the room's cover page (see The viewer artifact). viewer_payload.py (bundled) derives the display payload from the extraction; DataroomViewer.jsx is the finished, frozen component you paste that payload into. You never rebuild the component and never paste the raw extraction.

Workflow

1. File the room — first act, before reading anything. The platform keeps the original documents so the extraction stays auditable and the room outlives this chat. Hash the zip and register it:

   python3 -c "import hashlib,os,sys; p=sys.argv[1]; h=hashlib.sha256(open(p,'rb').read()).hexdigest(); print(h, os.path.getsize(p))" "<upload>.zip"
   

   Call open_dataroom(label, sha256, size_bytes) with a short label (the deal/teaser title).
   • status: "new" → push the zip, then continue:

     python3 room_push.py "<upload>.zip" "<upload_url>"
     

   • status: "known" → the room is already on the platform: skip the push. If the response has extraction_ready: true, pull the finished extraction and jump ahead —

     curl -sS -o extraction.json "<extraction_url>"
     

     skip steps 2–7 entirely and go straight to the viewer (step 8); corrections re-save under the returned extraction_id per step 7's correction flow (the zip is still in the sandbox for spot-checks). If extraction_ready: false, continue the normal flow from step 2. Either way say "filed" or "already on the platform" — never suggest anyone else uploaded it or has seen the deal.
   • Connection error from the push → the user's network allowlist is missing the upload host. Give them the one-line fix NOW (add the upload_host from the tool response under Claude's network egress settings, then a new chat), before the expensive extraction work — then continue this session normally either way. Keep the room_id — persistence links to it in step 7.
2. Unzip and triage. Unzip the upload, then run the bundled walker to inventory everything before you read:

   unzip -q "<upload>.zip" -d room && python3 triage.py room
   

   It writes _triage/manifest.json (every file: path, size, type, sha256) and _triage/triage.md (readable inventory), and dumps each spreadsheet to _triage/xlsx/<name>.json and each text-PDF to _triage/pdf/<name>.txt. Read those dumps instead of opening binaries by hand. Most rooms zip their contents under one top-level folder, so the walker descends past it and prints where the room root and actually landed () — read the manifest at the printed path. Every path it records is , which is exactly the convention and require: lift manifest paths verbatim and never prepend the room's own folder name.

What matters (for valuation)

Load-bearing — the price depends on these:

• wells — every well in the package: API (when stated), name, county/state, formation, well_type (PDP/PUD/DUC), lateral length.
• interests — the WI / NRI / RI / NPRI / ORRI / MI decimals, by well or tract. The economics live here — a deal is its interest.
• revenue_observations — from check stubs / revenue summaries: per (well, prod_date, product, check_date) the volume, realized price, gross, taxes, deductions, net, and owner decimal. This is where realized price, price differentials, and tax/deduct burden come from.
• expenses — from the LOS (lease operating statement), AFE, or cash-flow model: operating cost as a rate (per bbl or per well per month) and capex/AFE per well. This is the LOE the valuation needs. Capture whichever rate the document states.
• production_history — monthly oil/gas/water/NGL when the room has a production sheet.

Useful when cleanly present: deal (the listing summary), tracts (matters for mineral/royalty deals), division_orders.

Always cheap, always do: documents — a one-line inventory entry per file (lift it from the manifest) with a category guess. Audit trail.

Finish with flags — the read-before-bidding list, one short sentence each: the caveats a bidder must see before trusting the numbers (a payout reversion you didn't model, a teaser-vs-well-list ownership mismatch, a data source you spot-checked rather than parsed, an unbroken-out tax line). These lead the viewer's cover page, numbered. Keep them load-bearing and few — 3–6 for a typical room; a caveat that only qualifies your extraction process belongs in extraction_notes instead.

Noise — do NOT extract:

• Anything labeled "offset" — offset wells, offset permits, offset leases, offset production. The seller picks these nearby wells essentially at random; they are not part of the deal and will inflate the asset if you include them. This is the single most common extraction mistake. Ignore them.
• Marketing decoration, confidentiality boilerplate, and file metadata not tied to producing assets.

Conventions

Dataroom layout. Folder and file names signal content:

Signal in path/name	Entity
Check Stubs, ckstb, rev sum, royalty check	revenue_observations
LOS, LOE, operating statement, cash flow	expenses
AFE	expenses (capex)
Production, prod	production_history
Title, DO, division order, ownership	tracts / interests / division_orders
Engineering, Aries, economics, reserves	reserves & assumptions (read for context)
Overview, teaser, CIM, summary	deal

Check stubs → revenue_observations. One row per (well, prod_date, product, check_date). Operators report tax and deduction line-items inconsistently, so SUM all taxes into taxes and all deductions into deductions — do not invent per-category fields. Store taxes and deductions as positive magnitudes (source reports usually print them negative; flip on ingest). Sanity-check each row: gross_revenue − taxes − deductions ≈ net_revenue (within a cent). If a stub splits one production month across several interest decimals, collapse to a single row and sum.

Parsing stub workbooks — traps that have shipped bad rows:

• Locate headers by NAME, scanning the first ~6 rows of every sheet; never by position. Real rooms mix cover sheets, reordered columns, and multi-tab trackers within one uniform-looking family.
• A per-check field blank on the data tab often lives on a Parameters/cover tab (check number, payee entity, accounting month). Prefer that tab as the primary source for check-level metadata — keying dedupe on a half-populated data-tab column silently collapses checks.
• Filter grand-total rows (no property and no product) before any tie-out — they double revenue quietly if they slip through.
• Tie out the parse against any seller-prepared by-well rollup in the room. When a curated LOS is better-built than the raw stubs, stubs stay the primary source; use the rollup to find what your parse missed and to fill checks whose stubs are absent (note the fill in extraction_notes).

Interest decimals are copied, never rounded. Write wi_decimal / nri_decimal / ri_decimal / npri_decimal / orri_decimal and owner_decimal with every digit the document states — a royalty decimal quoted as 0.00170514 is stored as 0.00170514, not 0.001705. Trailing digits are load-bearing here: they are the multiplier on every dollar in the deal, they are how a stub's owner_decimal is tied out against the division order, and a decimal silently rounded in one place while another field keeps full precision makes one extraction disagree with itself. Rounding for display is the viewer's job, and it is already handled.

Identifier traps. State regulatory file numbers (NDIC etc.) look enough like APIs to be dangerous — a "State ID" column may be a file number for one state's wells and a real API for another's in the same well list. Never fabricate an API from a state ID; resolve through a second source (a production report, a stub property list) or leave the API null with a note. Production-report headers may repeat a well's short suffix (… 4H 4H) — normalize before name-joins. Before packing, print every well that still lacks an API as a build assertion; silent name-join misses are the costliest error in this workflow.

LOS / AFE → expenses. From the operating statement, capture operating cost as a rate (opex_per_bbl_usd or opex_per_well_per_month_usd); from AFEs, capex_per_well_usd. Fill whichever the document states; leave the rest null. Keep the operator's label_raw verbatim. When the document states a multi-month total (a 6-month LOS), persist it as stated: amount_usd + period/period_end for the range. Do NOT divide it into a monthly rate yourself — deriving economics inputs is the valuation engine's arithmetic, not yours.

Provenance is required on every record. source_file = the relative path inside the room. source_locator by convention: Excel "sheet:Name;row:N" (1-based, header = row 1), PDF "page:N". Use notes only when you inferred a value rather than read it.

No database here. Unlike the server pipeline, you have no access to the Crude Code well database in this sandbox. Leave Well.public_well_object null. When the room gives only a well name, leave api null and say so in notes / extraction_notes — a later server step resolves APIs against public data. (MCP tools remain available — persisting via save_dataroom_extraction is expected; it's the well-database lookup you don't have.)

API formatting. When the room states an API, normalize to SS-CCC-WWWWW (10 digits, two dashes; strip a 14-digit API to its first 10). Never fabricate digits to reach that shape.

The viewer artifact

Once extraction.json is written, give the user a viewer: the room's cover page — headline facts, the read-before-bidding flags, the well manifest by status with LTM net revenue, the document inventory. DataroomViewer.jsx (bundled here) is the finished, frozen component — you do not build, redesign, or adapt it. To produce the viewer:

1. Derive the display payload:

   python3 viewer_payload.py extraction.json > viewer_payload.json
   

2. Open DataroomViewer.jsx and fill the three slots at the bottom: paste viewer_payload.json into DATA, write TITLE (short deal title — deal.title is usually right) and TLDR (1–2 sentences: what the package is, what to look at first).
3. Ship that as the artifact. That's the whole job.

Never paste the raw extraction into the artifact. The payload is the viewer's entire data contract: viewer_payload.py computes every derived number on the page (LTM net-revenue rollups over the trailing-12 window, revenue shares, per-well interest sums, status groups, document folders) deterministically from the extraction — so no rollup is ever done by hand, and the artifact stays small no matter how many revenue/production rows the room carried. The persisted copy keeps every row; the viewer is triage, not the record.

The component is data-driven, so it already handles everything that varies room to room — don't reinvent any of it:

• Spine — the well manifest when the room has wells; the tracts table for minerals/royalty rooms. Automatic.
• Presence — every module renders only when the payload carries its data: no flags → no flags block, no revenue → no LTM column or share bars, null fields → omitted tiles/columns. Automatic.
• Trust — the flags list leads, extraction_notes ships as the collapsible data-quality record, and the footer states the derivation. Baked in.

So there are no per-room layout decisions to make and nothing to overfit to: same component, just this room's payload. The only dependency is react — don't add others.

When the dataroom feeds a valuation

Often the room isn't the end goal — the user wants to value the interest. The dataroom is the input that makes that possible: the wells and the interests decimal are exactly what forecast_wells / run_valuation need. In that case:

1. Extract → write extraction.json → persist (save_dataroom_extraction).
2. Show the viewer first — it's the confirm-before-you-value step. The user eyeballs what came out of the room (which wells, what interest decimal, where the revenue concentrates, what got flagged) and confirms it's right before any money number is built.
3. Then proceed into the valuation flow (get_skill("well-forecasting") → forecast_wells → assumptions grid → run_valuation), carrying the wells and the interest from the extraction. The room's own documents feed the forecast too: AFE dates and stated development plans are the first-choice source for undrilled wells' timing, and LOS/check-stub months are evidence when reading the production history.

Build the viewer every time you process a dataroom, whether or not a valuation follows. It's the deliverable that makes the extraction auditable — not an optional extra to skip when the goal is downstream.

Hard rules

• Never fabricate. Missing → null. Unknown → null. Can't verify it → leave null and explain in extraction_notes.
• Never persist an abbreviated or "representative" copy. Pack and upload with persist_pack.py --upload — the script sends the complete kit and verifies the stored counts itself. The persisted copy is the only durable record — the sandbox file is gone when the session ends. Never retype extraction contents into a tool call; the upload lane exists so no extraction data ever transits the chat.
• Never guess an API. Name-only well → api: null.
• No OCR. Image-only PDFs are flagged by triage (pdf_extractable: false) — note them in extraction_notes; don't invent their contents.
• No .accdb (Aries) parsing — note and move on.
• The dataroom is read-only.
• Partial-and-honest beats complete-and-invented. Low on budget? Write what you have with honest extraction_notes.