القائمة
Discover and build a complete bank integration by answering 9 required questions
التثبيت باستخدام Codex أو Claude انسخ هذا Prompt والصقه في Codex أو Claude أو مساعد آخر ليراجع صفحة Skill ويثبّتها لك.
استنادا إلى تصنيف SOC المهني
Daily financial summary — net worth, recent activity, payment due dates, alerts
Manage recurring scheduled tasks — add, list, pause, resume, remove
Build a custom dashboard view from a natural language request — query, API, React component, build, deploy
On-demand financial briefing — spending, balances, portfolio, transactions, reports, CSV export
Override transaction categories via natural language — update rules, reclassify
Guided first-bank setup — credentials, exploration, sync, extraction, import
| name | learn-institution |
| description | Discover and build a complete bank integration by answering 9 required questions |
| trigger | manual |
You are building a deterministic Playwright integration for a financial institution. You have 9 questions to answer. You do not stop until all 9 are answered and recorded in a working config file.
You are the builder, not the runner. Everything you discover gets written to a config file. Once you're done, daily execution uses that config with zero LLM cost.
You are self-correcting. When a tool fails, you diagnose the error, fix the tool or config, and try again. You do not stop and ask for help unless the user needs to provide something (like an MFA code or credentials).
You navigate the bank website visually, one step at a time — like a human would. Each step:
1. READ the annotated screenshot — elements are labeled [1], [2], [3]...
2. REASON about what you see — "I see a login form, element [3] is the username field"
3. ACT — send a command: click, type, scroll, etc.
4. READ the new screenshot — see what changed
5. RECORD — note the selector and behavior for the config
6. REPEAT until the question is answered
This is NOT batch exploration. You do NOT run a script end-to-end and parse the output. You navigate step by step, seeing the page at each step, adapting immediately when something unexpected happens.
NEVER use direct URL navigation (goto, navigate) inside an authenticated session. Once logged in, navigate ONLY by clicking elements on the page. Direct URL navigation breaks SPA session state on most bank sites, triggering a redirect to the login screen. This disorients the entire flow. If you need to get to a different section of the site, find the nav element, menu item, or link and click it. The only exceptions are: (1) the initial entry URL before login, and (2) the dashboardUrl recovery after login when the post-login page is unknown.
When you hit a wall, DO NOT improvise blindly. Follow this escalation:
1. STOP — don't try 5 variations of evaluate or DOM manipulation
2. READ existing code that solves this pattern:
- readers/tasks/download-transactions.js — selectTimePeriod(), selectDropdownOption(), fillDateInput()
- readers/tasks/download-statements.js — Chase saveMenu/directLink patterns, stale DOM purge
- readers/institutions/<bank>.js — the bank's existing config (if re-learning)
3. READ the template index:
- readers/institutions/templates/GUIDE.md — pattern lookup tables, component cheat sheet, obstacle solutions
- readers/institutions/templates/*.js — proven config structures with comments
4. APPLY the proven pattern to your current situation
5. THEN retry in the explorer with an informed approach
The codebase IS the answer key. Templates and working configs contain proven solutions for shadow DOM components, custom dropdowns, modal interactions, date pickers, and download flows. Check them BEFORE guessing.
If you find a bug in the exploration tools themselves (explore-interactive.js, browser-reader.js, run.js, download-transactions.js), you have permission to fix them. Fix the root cause, not just the symptom.
The user provides:
mybank)https://www.mybank.com/signin)<SLUG>_USERNAME / <SLUG>_PASSWORD where the slug is uppercased with hyphens removed (e.g., capital-one → CAPITALONE_USERNAME / CAPITALONE_PASSWORD). Already in .env.To verify credentials are set, use: node scripts/check-env.js CAPITALONE_USERNAME CAPITALONE_PASSWORD
Or check all vars for an institution: node scripts/check-env.js --prefix CAPITALONE
If re-learning an existing institution (the config file already exists at readers/institutions/<bank>.js):
config/institutions-status.md for notes on this bank's MFA, download patterns, and quirksIf building a new institution, check readers/institutions/templates/ for an existing pattern that matches. Read each template file and compare its login style (iframe vs direct, single vs multi-step) and download pattern to what you see on the live site.
If a template matches:
institution, entryUrl, security, credentials with the real valuesIf no template matches, proceed with full discovery below.
You must answer ALL of these. Each question has completion criteria — you are NOT done until every checkbox is checked. Do not move to the next question until the current one is fully complete.
IMPORTANT: Do not call done on the explorer until ALL questions are fully answered. Seeing a download dialog is not the same as discovering all the selectors inside it. You must interact with every control in the download flow.
IMPORTANT: Q10-Q15 are answered per account type. Do NOT assume one type's answer works for another. Cash accounts, credit cards, and loans almost always surface statement balances in different places. Each question requires its own navigation, download, and verification.
Start the interactive explorer:
node readers/explore-interactive.js <institution> <login-url> <usernameEnv> <passwordEnv>
Run this in the background. Then send commands and read screenshots step by step.
Step 1: Read the initial screenshot.
# Read the annotated screenshot
Read data/explore/<institution>-step-0.png
# Read the element list
Read data/explore/<institution>-state.json
The screenshot shows numbered labels [1], [2], [3] next to every interactive element. The state file maps each number to its CSS selector.
Step 2: Identify the login form.
Look at the screenshot. You'll see one of:
[3], password input [5], submit button [7][2], no form yet → click itdismiss command firstStep 3: Fill credentials and submit.
node readers/explore-cmd.js <institution> type 3 "{{USERNAME}}"
node readers/explore-cmd.js <institution> type 5 "{{PASSWORD}}"
node readers/explore-cmd.js <institution> click 7
{{USERNAME}} and {{PASSWORD}} are special tokens — the explorer replaces them with the actual env var values but records the tokens in history. Credentials never appear in logs.
Step 4: Read the new screenshot.
After each command, read the new screenshot. You'll see one of:
Step 5: Handle MFA (if present).
If you see an MFA page:
wait 3000 → screenshotStep 6: Inspect the iframe (if login is in one).
If the login form was in an iframe, you switched into it by index (frame 100). But the config needs a CSS selector. Inspect the iframe element:
node readers/explore-cmd.js <institution> frame main
node readers/explore-cmd.js <institution> evaluate "const iframes = document.querySelectorAll('iframe'); const info = Array.from(iframes).map(f => ({id: f.id, name: f.name, src: f.src?.substring(0,80)})); document.title = JSON.stringify(info)"
Record the iframe's id (e.g., iframe#logonbox) or name attribute for the config's iframeSelector.
Step 7: Extract MFA patterns from page text.
If MFA was triggered, look at the textPreview from the MFA page state. Extract the key phrases that indicate MFA:
These go into mfa.pushPatterns, mfa.smsPatterns, or mfa.emailPatterns. Extract at least 2-3 patterns from the actual text you saw.
Step 8: Record in config.
Q1 is complete when ALL of these are recorded:
entryUrl and security.expectedDomaincredentials.usernameEnv and passwordEnvlogin.usernameSelector, passwordSelector, submitSelectorlogin.iframePattern and iframeSelector (if applicable)login.landingPage and signInSelector (if applicable)login.multiStep and nextButtonSelector (if applicable)login.methodSelectionSelector (if applicable)mfa.push/sms/email — which types existmfa.pushPatterns/smsPatterns/emailPatterns — extracted from actual page textmfa.mfaInitiationSelector — e.g., button:has-text("Text me") (if applicable)mfa.codeInputSelectors and codeSubmitSelector (if SMS/email MFA)login.interstitials (if passkey enrollment or promos appeared)Write these into the config file now. Don't wait until the end.
After login, you should be on a dashboard. Read the screenshot — do you see balances?
textPreview in the state file should contain dollar amounts. Record the dashboard URL as dashboardUrl if it differs from entryUrl.wait 5000 then screenshot.Balance extraction uses the LLM approach — page text is captured and saved as pendingExtraction. The agent extracts structured balances from the text. No external API calls. You don't need selectors for individual balance elements.
Extract loggedInPatterns NOW. Read the dashboard textPreview. Pick at least 3 phrases that appear on the dashboard but would NOT appear on the login page. Good examples:
Write these into login.loggedInPatterns in the config. The browser-reader uses these to detect whether login succeeded.
Q2 is complete when:
dashboardUrl recorded (if different from entryUrl)loggedInPatterns extracted (at least 3 phrases from actual dashboard text)Scroll through the ENTIRE dashboard. Accounts are often grouped in sections (bank accounts, credit cards, investments) that extend below the fold. Scroll down repeatedly, taking screenshots, until you see the page footer or content starts repeating. Do NOT assume the first screenshot shows all accounts.
For each account found, record:
Q3 & Q4 are complete when:
config/accounts.jsonWrite the config file now (login + security + credentials + mfa + extraction sections — transactions will be added in Q7/Q8). Then verify the LLM extraction picks up all accounts:
node readers/run.js <institution> --balances
If any are missing, the dashboard text may not contain enough information. Check:
extract-balances.js task navigate away from the dashboard? Check if dashboardUrl is needed.loggedInPatterns too generic? They should match dashboard content but NOT the login page.Q5 & Q6 are complete when:
readers/institutions/<institution>.jsrun.js --balances succeeds and outputs correct balance countThis is the most complex question. Do NOT stop at finding the download button. You must interact with every control in the download flow, record every selector, and trigger an actual download.
Step 1: Find the download entry point.
You may need to navigate from the dashboard into an account detail page first. Look for:
If you don't see a download option on the dashboard:
Record the selector of whatever you click to open the download flow.
Step 2: Identify the download pattern.
Read the new screenshot after clicking. You'll see one of:
| What you see | Pattern | Next action |
|---|---|---|
| Dialog with account/file type/activity dropdowns | A: Central dialog | Interact with EACH dropdown |
| Account-specific page with download link | B: Per-account | Click download, find date range |
| Statements page with PDF links | C: PDF statements | Click a statement download |
| Modal with calendar date picker | D: Export modal | Navigate the date picker |
| Instant CSV download | E: Direct export | Verify the file |
Step 3: Discover EVERY selector in the download flow.
This is where you must be thorough. For each pattern:
Pattern A (Central dialog) — required selectors:
#select-account-selector).'Spreadsheet (Excel, CSV)').input[placeholder="mm/dd/yyyy"]).button[type="submit"]. Inspect its id, aria-labelledby, or data-testid.data/downloads/.The download task (download-transactions.js) iterates through all accounts automatically using these selectors — you don't need to manually download for each account. But you DO need to verify the dropdown contains all accounts and the cycle-back flow works.
Pattern B (Per-account) — required selectors:
accounts.json.Pattern C (PDF statements) — required selectors:
Pattern D (Export modal) — required selectors:
evaluate → note this in configPattern E (Direct export) — required selectors:
Step 4: Handle custom web components (shadow DOM).
Many banks use custom elements (<mds-select>, <c1-ease-select>, <ui-button>) with shadow DOM. The interactive explorer CANNOT see inside shadow roots — annotations won't label those elements and document.querySelector() in evaluate won't find them.
Use Playwright's page.locator() instead — it pierces shadow DOM automatically. When the explorer can't reach an element, switch to the browser-reader primitive:
// This WORKS — Playwright pierces shadow DOM
await page.locator('text=Statements & documents').click();
await page.locator('#select-account-selector').click();
// This DOES NOT WORK — querySelector can't pierce shadow DOM
await page.evaluate(() => document.querySelector('text=Statements').click());
When you hit shadow DOM during exploration:
BrowserReader that logs in and uses page.locator() with text matching or ID selectorsThe existing download-transactions.js already demonstrates this pattern — study it for the bank you're working on.
Step 5: Verify the download worked.
ls -la data/downloads/
Check that a CSV or PDF file was actually saved. If not, the download button may need a different interaction (evaluate click, longer wait, etc.).
Q7 & Q8 are complete when:
transactions sectiontransactions section is fully populated for the identified patternThe download task iterates through accounts using account-matcher.js, which maps bank display names (e.g., "CHECKING (...XXXX)") to accounts.json entries by last-4 digits. This means:
accounts.json must have a last4 fieldWhen exploring, verify that every account you found in Q3/Q4 is reachable in the download flow — either visible in the account dropdown (Pattern A) or navigable from the dashboard (Pattern B).
Five proven transaction download patterns:
| Pattern | How it works | Config key |
|---|---|---|
| A: Central dialog | One dialog serves all accounts. Account dropdown + date range + format + download button. | transactions: { downloadButtonSelector, accountDropdownButton, ... } |
| B: Per-account | Navigate to each account page individually, click download, set date range. | transactions: { perAccount: true, downloadLinkSelector, ... } |
| C: PDF statements | Download monthly PDF statements. Parse with LiteParse + LLM. | transactions: { pdfBased: true, statementsUrl, ... } |
| D: Export modal | Modal with calendar date picker. May need evaluate() for backdrop bypass. | transactions: { exportModal: true, prevMonthSelector, ... } |
| E: Direct export | Navigate to page, click one button, CSV downloads. Simplest pattern. | transactions: { directExport: true, navigationSelector, exportButtonSelector } |
Statements vs current-month transactions: Statements only cover closed/past months. Current month transactions need a live activity view or an export that includes today's date range. Look for both paths.
Before trying to solve any obstacle, check readers/institutions/templates/GUIDE.md. It has lookup tables for login patterns, MFA patterns, download patterns, custom web components, and common obstacle solutions. If the obstacle matches a known pattern, apply the proven solution — don't reinvent it.
When you encounter an obstacle, handle it AND record how you handled it:
| Obstacle | How to handle | Record in config |
|---|---|---|
| Cookie consent banner | Dismiss before login. Add selector to dismissPopups in browser-reader.js if not already there. | Part of dismissPopups |
| MFA (SMS/device code) | Click initiation button ("Text me", "Send code") → wait for code input → enter code → click submit | mfa.mfaInitiationSelector, mfa.codeSubmitSelector |
| MFA (email) | Auto-poll via scripts/gmail-mfa.js | mfa.emailPatterns, bankEmailSender |
| MFA (push) | Click push option, wait for approval | mfa.pushPatterns |
| Popup/modal | Dismiss it. Add new dismiss selectors to browser-reader.js if needed. | dismissPopups patterns |
| Passkey enrollment | Skip by navigating to dashboard URL. Do NOT click Continue. | login.interstitials |
| Landing page login | Click "Sign In" to reveal form. May load in iframe. | login.landingPage, login.signInSelector |
| Method selection | Click "Continue with Password" after email submit. | login.methodSelectionSelector |
| Multi-step login | Fill email → Next → fill password → Submit. Adaptive: detect if password already visible. | login.multiStep, login.nextButtonSelector |
| Individual digit MFA | 6 separate input[type="tel"] fields. Click first, type each digit. | mfa.individualDigitInputs |
| Backdrop overlay | locator.click() fails. Use page.evaluate(() => el.click()). | Note in config, use evaluate-based patterns |
| No IDs on inputs | Use input[name="..."] or input[type="..."] selectors. | Use name-based selectors |
| Dashboard URL differs | Post-login URL != login URL. Balance task navigates to wrong page. | dashboardUrl in config |
| Session doesn't persist | Some banks require full login+2FA every run despite persistent profiles. | Note in config comments |
| Custom web components | <mds-select>, <c1-ease-select>, <ui-button>. Click button to open, click option. | Note component type in config |
| Shadow DOM (no annotations) | Elements inside shadow roots are invisible to the explorer's annotations. Do NOT use evaluate with document.querySelector() — it cannot pierce shadow DOM. Instead, use Playwright's page.locator() which pierces automatically. Switch from the explorer to the browser-reader primitive for shadow DOM navigation. | Record the Playwright locator selector in config |
| Shadow DOM navigation | Write a Playwright script using BrowserReader to log in, then use page.locator('text=Target Text').click() or page.locator('#element-id').click(). Capture page text after navigation. Study download-transactions.js for the existing bank's shadow DOM patterns. | Record working locator selectors in config |
| Collapsed accordions | Expand before clicking download. Check aria-expanded. | Expand in download flow |
| SPA stale dropdown DOM | Some SPAs keep dropdown menu items visible in the DOM even after their parent accordion is collapsed. Multiple accounts with the same dropdown pattern (e.g., "Save as PDF") create duplicate element IDs. Fix: purge stale dropdown elements via el.remove() after each account's downloads complete. Always verify with the explorer by checking element counts via evaluate after accordion collapse. | Purge after download, note in config |
| ZIP downloads | "Download all" may produce ZIP. Unzip and process each file. | Handle in download task |
| PDF transactions | Use lit parse <file.pdf> --format text → LLM extraction. | LiteParse pipeline |
When a tool breaks on a new pattern, fix it. Common fixes you may need to make:
explore-full.js:
initMfaSelectors listcodeSubmitSelectors listbrowser-reader.js:
dismissSelectors array in dismissPopups()page.evaluate() click patterndownload-transactions.js:
run<Pattern>() function and route it in run()run.js:
This question is answered AFTER Q7/Q8, once you have actual transaction data. You must understand how this platform represents debits, credits, and balances before the data can be imported correctly.
Step 1: Inspect the raw transaction data.
After run.js --transactions --all succeeds, read the sync output:
node -e "const d = JSON.parse(require('fs').readFileSync('data/sync-output/<institution>.json','utf-8')); const txns = d.transactions || []; txns.slice(0,10).forEach(t => console.log(JSON.stringify(t.raw).substring(0,200)))"
Step 2: Determine the transaction format.
Look at the raw data. You'll see one of:
format: "signed"format: "typed"format: "split"Step 3: Determine what sign a debit has.
Find a transaction you can identify as a debit (purchase, withdrawal, payment to a vendor). Check: is the raw amount positive or negative?
Record: "debit": "negative" or "debit": "positive"
Step 4: Determine what sign a credit has.
Find a transaction you can identify as a credit (payment received, deposit, refund). Check: is the raw amount positive or negative?
Record: "credit": "positive" or "credit": "negative"
Step 5: Confirm with the user.
Present your findings:
"I inspected the raw transaction data for [institution]. Here's what I found:
- A purchase at TARGET shows as +$8.38 (positive) in the raw data
- An ACH payment shows as -$753.56 (negative) in the raw data
- This means debits are positive and credits are negative (issuer perspective)
- Import will flip signs so purchases become negative (money out) in the database Does this look correct?"
Step 6: Select validation anchors.
Pick 2-3 transactions that are unambiguously identifiable as debits or credits. These will be checked on every future import to verify the convention hasn't changed:
Step 7: Write to config/data-semantics.json.
Read the existing file and add the new institution entry:
"<institution>": {
"transactionConvention": {
"format": "signed",
"debit": "positive",
"credit": "negative"
},
"balanceConvention": {
"<accountType>": "<what positive balance means>"
},
"anchors": [
{ "descriptionPattern": "TARGET", "is": "debit", "rawSign": "positive" },
{ "descriptionPattern": "ACH DEPOSIT", "is": "credit", "rawSign": "negative" }
],
"learnedAt": "<today's date>",
"learnedFrom": "<what you inspected>"
}
For typed format (unsigned amounts with type column), also include:
"transactionConvention": {
"format": "typed",
"typeColumn": "Transaction Type",
"debitValue": "Debit",
"creditValue": "Credit",
"amountSign": "unsigned (always positive)"
}
Step 8: Re-import and validate.
Run the import with validation:
node sync-engine/import.js --bank <institution>
Check the output for "Data semantics validation passed" or warnings. If warnings appear, the convention you recorded may be wrong — go back to Step 3.
Then spot-check in the database:
node -e "const db = require('better-sqlite3')('data/foliome.db'); db.prepare(\"SELECT date, description, amount FROM transactions WHERE institution = '<institution>' ORDER BY date DESC LIMIT 5\").all().forEach(r => console.log(r.date, r.amount > 0 ? '+' : '', r.amount.toFixed(2), r.description.substring(0,50)))"
Verify: purchases should be negative, deposits/payments should be positive.
Q9 is complete when:
config/data-semantics.jsonSkip if this institution has no checking or savings accounts.
Step 1: Read the dashboard text from Q2. For EACH cash account, search for a statement balance label.
Look for text near the account like "Last statement balance", "Previous balance", "Statement balance", "Interest saving balance". Record the exact label and dollar amount if found.
Step 2: If not found on dashboard — navigate to the statements/documents section.
Use the browser-reader primitive (page.locator() — pierces shadow DOM):
const reader = new BrowserReader(config);
// ... login ...
await reader.page.locator('text=Statements').first().click({ timeout: 10000 });
Capture the page text after navigation. Record:
text=Statements & documents)Step 3: On the statements page, find the cash account's statements.
Read the page text. Is there an account selector (dropdown, tabs, list)? Record:
Step 4: Determine the source — is the closing balance visible in HTML, or do you need to download a PDF?
Balance column → check for a natural month-end anchor like "Monthly Interest Paid" on the last day of each month. The balance after this transaction IS the month-end closing balance. Source: csv-balance-column. Record the anchor description and date pattern in the config. No additional navigation needed — data comes from the existing CSV download pipeline (Pattern S-E).Step 5: If PDF required — download ONE statement, parse it, and verify it contains closing balance data.
ls -la data/downloads/ — record the filenameliteparse parse data/downloads/<file>.pdfStatement Period: Feb 13, 2026)through Mar 12, 2026)Beginning Balance on Feb 13 $ 100.00)Ending Balance on Mar 12, 2026 $ 100.00)Step 6: Record the actual extracted values.
Write down: "For [account name], the most recent statement period is [start] to [end], closing balance is $[amount]." This is the proof that the pipeline works.
Q10 is complete when:
statementBalances.checking / .savingsSkip if this institution has no credit cards.
Step 1: Read the dashboard text from Q2. For EACH credit card, search for a statement balance.
Credit cards commonly show this on the dashboard. Look for:
Check EVERY credit card individually. Different cards may use different labels or may not show it at all.
Step 2: For each credit card, record what you found.
| Card | Label found | Amount | Source |
|---|---|---|---|
| Freedom (...7890) | "Interest saving balance" | $340.47 | Dashboard |
| Sapphire (...1234) | "Last statement balance" | $514.99 | Dashboard |
| Prime Visa (...5678) | (not found) | — | Need statements page |
Step 3: For any card NOT showing a statement balance on the dashboard — navigate to the statements page.
Same navigation as Q10 Step 2. On the statements page:
Step 4: If PDF required — download ONE credit card statement, parse it, verify it has closing balance.
Same process as Q10 Step 5. Credit card PDFs typically use different field names:
Record the exact text lines from the parsed PDF.
Step 5: Record the actual extracted values for every credit card.
| Card | Statement period end | Closing balance | Source | Label/field name |
|---|---|---|---|---|
| Freedom (...7890) | ~Mar 5 | -$340.47 | Dashboard | "Interest saving balance" |
| Sapphire (...1234) | ~Mar 20 | -$514.99 | Dashboard | "Last statement balance" |
| Prime Visa (...5678) | Mar 20 | -$126.66 | "New Balance" |
Q11 is complete when:
statementBalances.creditSkip if this institution has no mortgage, loan, or line of credit accounts. Mark N/A and move to Q13.
Step 1: Check the dashboard text for loan/mortgage statement balances.
Mortgage dashboards usually show "current principal balance" — this is the LIVE balance, not necessarily the statement closing balance. Look for whether there's a separate "statement balance" or "previous principal balance" shown.
Step 2: Navigate to the statements page and select the loan account.
Step 3: Download ONE loan statement PDF, parse it, find the principal balance.
liteparse parsePrincipal Balance (Not a Payoff Amount) $634,362.94)Step 4: Confirm the sign convention.
The principal balance is a LIABILITY. Record it as NEGATIVE in the database. Verify: if the PDF says "$634,362.94", the stored value should be -634362.94.
Step 5: Record the actual extracted value.
"For [mortgage account], the statement dated [date] shows principal balance of $[amount] (stored as -$[amount])."
Q12 is complete when:
statementBalances.mortgageGoal: Download and parse multiple months of statements to build trend data. This proves the download flow is deterministic and repeatable.
Step 1: Navigate to the statements page using the selector from Q10.
Record how many months of statements are visible for cash accounts. Are they paginated? Is there a "Show more" or year selector?
Step 2: Download 3 consecutive statements for ONE cash account.
For each statement:
ls -la data/downloads/liteparse parse data/downloads/<file>.pdfRecord the results:
| # | Filename | Period start | Period end | Opening balance | Closing balance |
|---|---|---|---|---|---|
| 1 | (actual filename) | YYYY-MM-DD | YYYY-MM-DD | $X | $X |
| 2 | (actual filename) | YYYY-MM-DD | YYYY-MM-DD | $X | $X |
| 3 | (actual filename) | YYYY-MM-DD | YYYY-MM-DD | $X | $X |
Step 3: Verify consistency.
Step 4: Record the deterministic download pattern.
For the Playwright script to download all historical statements, it needs:
Q13 is complete when:
Skip if all credit card statement balances come from dashboard text (S-C) and you don't need historical data beyond the current period.
Step 1: Navigate to the statements page, select a credit card account.
Step 2: Download 3 consecutive credit card statements for ONE card.
For each statement:
liteparse parseRecord the results:
| # | Filename | Statement date | Previous balance | New balance |
|---|---|---|---|---|
| 1 | (actual filename) | YYYY-MM-DD | $X | $X |
| 2 | (actual filename) | YYYY-MM-DD | $X | $X |
| 3 | (actual filename) | YYYY-MM-DD | $X | $X |
Step 3: Note differences from cash account PDF format.
Record these — the extraction prompt must handle both formats.
Step 4: Verify consistency.
Q14 is complete when:
Skip if this institution has no loan accounts. Mark N/A.
Step 1: Navigate to loan statements, same approach as Q12.
Step 2: Download 3 consecutive loan statements.
For each:
| # | Filename | Statement date | Principal balance |
|---|---|---|---|
| 1 | (actual filename) | YYYY-MM-DD | $X |
| 2 | (actual filename) | YYYY-MM-DD | $X |
| 3 | (actual filename) | YYYY-MM-DD | $X |
Step 3: Verify principal decreases over time.
Q15 is complete when:
After Q10-Q15, write the complete statementBalances section to the config. Every selector must be one that was actually tested and verified.
Each account type with source: 'pdf' must provide three download hooks that download-statements.js calls:
beforeDownloads(page, accountId) — setup before row iteration (expand accordion, select account tab, etc.)download(page, accountId, rowIdx) — download one statement PDF, return the Playwright Download objectafterDownloads(page, accountId) — cleanup after row iteration (collapse accordion, purge stale DOM, etc.)The generic wrapper handles navigation, page readiness, iteration over accounts (filtered from config.accounts by type) and months, file saving, PDF parsing via LiteParse, and error handling. You only write the Playwright click sequences you discovered during exploration.
statementBalances: {
// Navigation to statements page (shared across account types, or per-type if different)
statementsNavSelector: 'text=Statements & documents', // verified in Q10 Step 2
statementsUrl: 'https://...#/dashboard/documents/...', // URL after navigation
pageReadyPatterns: ['ACCOUNT NAME 1', 'ACCOUNT NAME 2'], // text to wait for (optional)
errorRetry: { text: "isn't working right now", selector: 'text=Try again' }, // optional
checking: {
source: 'pdf', // or 'dashboard' or 'html' — what actually worked
// PDF field patterns (exact text from parsed output):
periodStartPattern: 'Statement Period:',
periodEndPattern: 'through',
openingBalancePattern: 'Beginning Balance on',
closingBalancePattern: 'Ending Balance on',
monthsAvailable: 12, // how many months visible on the page
// Download hooks — the Playwright click sequence you discovered
beforeDownloads: async (page, accountId) => {
// Setup: expand section, click tab, select account, etc.
await page.locator('#checking-section').click({ timeout: 5000 });
await page.waitForTimeout(2000);
},
download: async (page, accountId, rowIdx) => {
// Download one statement — return the Playwright Download object
const link = page.locator(`#download-row-${rowIdx}`);
const [dl] = await Promise.all([
page.waitForEvent('download', { timeout: 15000 }),
link.click({ timeout: 5000 }),
]);
return dl;
},
afterDownloads: async (page, accountId) => {
// Cleanup: collapse section, purge stale DOM elements, etc.
},
},
savings: { /* same structure — agent writes download hooks from Q13 exploration */ },
credit: {
currentSource: 'dashboard', // most recent from dashboard text
dashboardLabels: {
// per-card labels discovered in Q11 Step 2
'institution-card-1': 'Interest saving balance',
'institution-card-2': 'Last statement balance',
},
historicalSource: 'pdf', // older periods from PDF statements
closingBalancePattern: 'New Balance',
openingBalancePattern: 'Previous Balance',
// Download hooks — accountId distinguishes between multiple cards
beforeDownloads: async (page, accountId) => {
// Map accountId to the correct card section
},
download: async (page, accountId, rowIdx) => {
const [dl] = await Promise.all([
page.waitForEvent('download', { timeout: 15000 }),
page.locator(`#card-download-row-${rowIdx}`).click({ timeout: 5000 }),
]);
return dl;
},
},
mortgage: {
source: 'pdf',
closingBalancePattern: 'Principal Balance (Not a Payoff Amount)',
signConvention: 'negative', // store as negative (liability)
download: async (page, accountId, rowIdx) => {
// Agent writes the discovered click sequence
},
},
brokerage: { source: 'not-available' },
},
Import writes to the statement_balances table. The dashboard shows "vs. Last Period" using the most recent entry per account.
After all 15 questions are answered and the config is written, close the interactive explorer (done) and test the config through the actual primitive:
1. Test balances:
node readers/run.js <institution> --balances
Check: does it log in, extract balances, and write data/sync-output/<institution>.json? If MFA is triggered, provide the code.
2. Test transactions:
node readers/run.js <institution> --transactions --all
Check: does it download transactions and write them to the output file? Verify the transaction count makes sense.
3. Test full pipeline:
node readers/run.js <institution>
4. Verify statement balances:
Check that statementBalances data was written to the JSON output (for S-A/S-C patterns) and imported to the statement_balances table:
node -e "const db = require('better-sqlite3')('data/foliome.db'); db.prepare(\"SELECT * FROM statement_balances WHERE institution='<institution>'\").all().forEach(r => console.log(r.account_id, r.period_end, r.closing_balance))"
Verify: at least one statement balance per account type that has pattern S-A or S-C. Check: both balances and transactions in one run.
If any step fails, diagnose and fix the config — don't just report the error. Common issues:
The skill is NOT complete until run.js succeeds with both balances and transactions.
| Tool | Purpose | When to use |
|---|---|---|
node readers/explore-interactive.js <bank> <url> <userEnv> <passEnv> | Interactive visual explorer — background process, step-by-step | Primary exploration tool for all banks |
node readers/explore-cmd.js <bank> <action> [args] | Send commands to the interactive explorer | Every step of exploration |
node readers/explore-full.js <url> <userEnv> <passEnv> --profile <name> | Batch exploration (legacy) — runs all phases at once | Quick scan fallback |
node readers/explore.js <url> --profile <name> | Dump any page without login | Quick page inspection |
node readers/run.js <bank> --balances | Test balance extraction through the primitive | After config is written |
node readers/run.js <bank> --transactions --all | Test transaction download through the primitive | After transaction config is written |
node readers/run.js <bank> --explore | Login + MFA + dump dashboard via the primitive | Testing the config's login flow |
| Command | Example | What it does |
|---|---|---|
screenshot | explore-cmd.js <bank> screenshot | Retake annotated screenshot |
click <N> | explore-cmd.js <bank> click 5 | Click element [5] |
type <N> <text> | explore-cmd.js <bank> type 3 "{{USERNAME}}" | Type into element [3] |
key <key> | explore-cmd.js <bank> key Enter | Press a keyboard key |
scroll <dir> [px] | explore-cmd.js <bank> scroll down 500 | Scroll the page |
frame <N> | explore-cmd.js <bank> frame 100 | Switch into iframe [100] |
frame main | explore-cmd.js <bank> frame main | Switch back to main page |
navigate <url> | explore-cmd.js <bank> navigate https://... | Go to URL |
dismiss | explore-cmd.js <bank> dismiss | Dismiss popups/banners |
wait <ms> | explore-cmd.js <bank> wait 5000 | Wait for page to settle |
back | explore-cmd.js <bank> back | Browser back |
evaluate <js> | explore-cmd.js <bank> evaluate "el.click()" | Run JS in page context |
done | explore-cmd.js <bank> done | Close browser, save history |
Credential tokens: Use {{USERNAME}} and {{PASSWORD}} in type commands. The explorer replaces them with env var values but records the tokens (not actual values) in history.
When running in the background, MFA codes are exchanged via files:
data/mfa-pending/<institution>.request.jsondata/mfa-pending/<institution>.codeTo submit a code: node -e "require('./readers/mfa-bridge').submitCode('<institution>', '<code>')"
const accounts = require('../../config/accounts.json');
module.exports = {
institution: '<institution>',
entryUrl: '<login-url>',
dashboardUrl: '<post-login-url>', // if different from entryUrl
security: {
expectedDomain: '<domain>',
requireHttps: true,
},
credentials: {
usernameEnv: '<SLUG>_USERNAME', // Slug uppercased, hyphens removed (e.g., capital-one → CAPITALONE)
passwordEnv: '<SLUG>_PASSWORD',
},
login: {
landingPage: false, // true if login form is behind a "Sign In" click
signInSelector: null, // e.g., 'text=Sign In'
iframePattern: null,
iframeSelector: null,
usernameSelector: '<selector>',
passwordSelector: '<selector>',
submitSelector: '<selector>',
multiStep: false,
nextButtonSelector: null,
methodSelectionSelector: null, // e.g., 'button:has-text("Continue with Password")'
postLoginWaitMs: 5000,
loggedInPatterns: [], // text that appears on dashboard but NOT login page
interstitials: [], // [{ urlPattern, action: 'skip', targetUrl }]
},
mfa: {
sms: false, email: false, push: false,
bankEmailSender: null, bankEmailSubject: null,
smsPatterns: [], emailPatterns: [], pushPatterns: [],
mfaInitiationSelector: null, // e.g., 'button:has-text("Text me")'
codeInputSelectors: ['input[type="tel"]', 'input[type="text"]'],
codeSubmitSelector: null, // e.g., 'button:has-text("Next")' — null for auto-submit
trustDevicePatterns: [],
individualDigitInputs: false, // true for Apple-style 6 separate inputs
},
transactions: {
// === Choose ONE pattern ===
//
// Pattern A — Central dialog:
// downloadButtonSelector, accountDropdownButton, fileTypeDropdownButton,
// activityDropdownButton, downloadSubmitSelector
//
// Pattern B — Per-account:
// perAccount: true, downloadLinkSelector, timePeriodOptions,
// downloadSubmitSelector, postDownloadDismiss, backButtonSelector
//
// Pattern C — PDF statements:
// pdfBased: true, statementsUrl
//
// Pattern D — Export modal:
// exportModal: true, statementsLinkSelector, exportLinkSelector,
// prevMonthSelector, dayButtonSelector, exportButtonSelector
//
// Pattern E — Direct export:
// directExport: true, navigationSelector, exportButtonSelector
},
accounts: accounts['<institution>']?.accounts || [],
extraction: async (page, accountList) => {
const { extractBalances } = require('../llm-extract');
const pageText = await page.evaluate(() => document.body.innerText);
if (pageText.length < 100) throw new Error('Dashboard text too short');
const result = await extractBalances(pageText, '<institution>', accountList);
return { balances: result.balances || [], transactions: [], holdings: [] };
},
};
Before building a new config, study the existing working examples in readers/institutions/. Each demonstrates a different combination of login flow, MFA type, and download pattern:
ls readers/institutions/*.js
Read 2-3 existing configs to understand the structure. Pay attention to: