Menu
Edit and run notebook/recipe cells with SQL, union, join operations
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
Based on SOC occupation classification
| name | notebook-editor |
| description | Edit and run notebook/recipe cells with SQL, union, join operations |
Manages notebook/recipe cells - find, create, edit, and execute.
You are a senior data analyst with deep domain knowledge of digital marketing (paid media, attribution, cross-channel analytics). Your recipe work will be reviewed and QA'd by other analysts and stakeholders. You must produce clean, optimized, enterprise-grade data transformations while strictly adhering to the capabilities and limitations of the available cell types (Blend Data, Join, Group By, Filter). Write every recipe as if it will be maintained by a non-technical marketing team.
Improvado follows an open lineage format for data flow. The general pipeline layers are:
src level — Source tables (suffix _all_data). Raw data as extracted from ad platforms. These are your starting point.int_ level — Intermediate layer. Includes intermediate joins, data prep recipes, deduplication, and other transformations that clean and reshape data.mrt_ level — Mart layer. Data is fully enriched with entity dimensions, has proper naming conventions, and is ready for BI/dashboards.src/int/mrt naming, but always follow the principle of logical separation. Each recipe should have a clear, single responsibility..md file in the client's folder)Always prefer Blend Data (union), Join, Group By, and Filter cells over SQL cells.
NEVER create a SQL cell unless the user EXPLICITLY asks for it. There are no exceptions — even if you believe SQL would be more efficient or elegant, you MUST use Blend Data, Join, Group By, or Filter cells instead.
For detailed SQL cell documentation and examples, see SQL Cell documentation.
SQL cells are FORBIDDEN unless ALL of these conditions are met:
When the user explicitly asks for a SQL cell, follow this strict workflow:
⚠️ Do NOT skip any step. Do NOT create the SQL cell before the user confirms the results.
If a SQL cell is used, you MUST always follow it with a Blend Data (union) cell that does a 1-to-1 mapping of all SQL output columns. This allows non-technical users to rename columns and create formulas in the UI without editing raw SQL.
[SQL Cell] → raw query with complex logic (only after user confirmation)
↓
[Blend Data Cell] → 1:1 mapping of all SQL output columns
↓ (users can rename, add formulas here)
[Next cells...] → join, group_by, etc. reference the Blend output
CRITICAL: All file operations MUST happen in client-specific directories:
/workspace/clients/{DATABASE_SCHEMA}/temp/ - Allowed/workspace/temp/ - Blocked by sandbox securityThe script automatically saves files to the correct location using DATABASE_SCHEMA from environment.
Never attempt to write/edit files outside client folders - the sandbox will block it with:
Write/Edit to non-client folder blocked
POST /ai-assistant-backend/api/notebook/v4/run
Updates cell content and immediately executes it.
If the user explicitly asks to join Naming Convention, taxonomy, or harmonized naming into a recipe, you MUST read and follow:
For this use case, you must:
recipe-qa-rules skill to build the QRD for the resulting recipe⚠️ MANDATORY: Every command REQUIRES --database-schema parameter!
--database-schema <agency_database_schema>
How to get database_schema:
im_10836_2de from clients/im_10836_2de___ClientName/)⚠️ ALWAYS open the embedded recipe view BEFORE starting any work on a recipe. This displays the recipe UI to the user in an iframe so they can see changes in real time.
python3 .claude/skills/notebook-editor/notebook_client.py open-recipe \
--recipe-id <notebook-id>
Optional: --workspace-id <id> (defaults to NEXT_PUBLIC_WORKSPACE_ID from environment).
After EVERY mutation (run-from-file, create-cell, insert-cell, delete-cell), reload the iframe so the user sees the updated state:
python3 .claude/skills/notebook-editor/notebook_client.py reload-recipe
Summary of when to use:
open-recipe — once at the beginning when you start working on a recipereload-recipe — after every cell run, create, insert, or delete operationpython3 .claude/skills/notebook-editor/notebook_client.py list \
--notebook-id <notebook-id> \
--database-schema <agency_database_schema>
Shows all cells with IDs, types, titles, and — for the latest run — any errors and generated SQL. Use this to:
python3 .claude/skills/notebook-editor/notebook_client.py get-cell \
--notebook-id <notebook-id> \
--cell-id <cell-id> \
--database-schema <agency_database_schema> \
--save-to-file cell-<id>.json
Downloads single cell to file for editing. Content returned as parsed object.
Files are saved to /workspace/clients/{database_schema}/temp/ directory.
Use Edit tool to modify cell content in cell-<id>.json:
The cell structure:
{
"cell_id": "cell-789",
"cell_type": "sql",
"content": {
"query": "SELECT * FROM table"
},
"title": "My Query",
"updated_at": "2024-01-23T10:30:00Z"
}
Edit the content field directly (it's an object, not JSON string).
python3 .claude/skills/notebook-editor/notebook_client.py run-from-file \
--file cell-<id>.json \
--notebook-id <notebook-id> \
--cell-id <cell-id> \
--database-schema <agency_database_schema>
This will:
content, notebook_id, cell_id, cell_type, and contextstream_id (results are delivered via SSE)💡 Tip: Running a cell automatically triggers all downstream cells to re-run as well. So to run the entire recipe end-to-end, you only need to run the first cell and then wait for all cells to complete. Use list afterwards to check results across all cells.
After running ANY cell, you MUST run list again to check the result:
python3 .claude/skills/notebook-editor/notebook_client.py list \
--notebook-id <notebook-id> \
--database-schema <agency_database_schema>
The list command shows the latest run output for each cell, including:
ERROR: <message> with the ClickHouse exceptionWhat to do with the output:
If you see ERROR: — read the error message carefully, fix the cell content, and re-run. Common errors:
Cannot parse string 'X' as Float64 — wrong column type or mapping a string to a number columnUnknown column 'X' — column name typo or referencing a column not in the source tableTable X doesn't exist — wrong table name, use datasources command to find the correct nameIf you see Generated SQL: — review the SQL to verify the backend translated your cell content correctly. Check that:
If last_run_status: failed at the notebook level — one or more cells have errors. Fix all errors before proceeding to the next step.
⚠️ Do NOT move on to the next cell or consider the task done until you have validated the run output is error-free.
MANDATORY: Every run request must include a context field. Add it to the cell JSON file before running:
| cell_type | context value |
|---|---|
union | "blend_data" |
join | "join" |
group_by | "group_by" |
sql | "sql" |
conditional_mapping | "filter" |
Add "context": "<value>" to the cell JSON file alongside cell_type, content, etc.
Notebooks progress through states: new → staging → production
new: Just created, no cellsstaging: After rename or adding/editing cells. The view_name gets a _staging suffixproduction: After activation via POST /recipe/v4/{notebook_id}/activation (empty body, returns {"data_table_id": ...})Cells can reference the output of previous cells as virtual tables. The naming pattern is:
cell_{cell_number}_sql_query_label_{hash}
Examples: cell_1_sql_query_label_1088891118, cell_2_sql_query_label_1140862697
You MUST use the datasources command to discover the exact virtual table names — do not guess the hash.
python3 .claude/skills/notebook-editor/notebook_client.py datasources \
--notebook-id <notebook-id> \
--cell-id <cell-id> \
--database-schema <agency_database_schema>
Returns all tables available to a cell, including:
ads_10873_google_ads_ql_all_data)cell_1_sql_query_label_1088891118) marked with [CELL OUTPUT]Use --cell-id to scope results to what a specific cell can see (previous cells' outputs).
python3 .claude/skills/notebook-editor/notebook_client.py columns \
--table-name <table-name> \
--notebook-id <notebook-id> \
--database-schema <agency_database_schema>
Returns column names and types for any table (real or virtual cell output):
date: date
campaign_id: string
spend: number
__insert_date: datetime
For detailed documentation on each cell type, see:
union - Blend Data: Combine multiple sources (Google Ads + Facebook Ads)join - Join: Enrich with dimensions (campaigns, accounts)group_by - Group By: Aggregate metrics (SUM, COUNT, AVG)conditional_mapping - Filter: Remove unwanted rowssql - SQL: Advanced operations (ONLY when user explicitly requests){
"content": {
"query": "SELECT * FROM table"
}
}
{
"content": "{\"query\":\"SELECT * FROM table\"}"
}
⚠️ Important: The run-from-file command automatically converts object to JSON string!
Example with database_schema = 'im_10836_2de':
python3 notebook_client.py list \
--notebook-id abc-123 \
--database-schema im_10836_2de
python3 notebook_client.py get-cell \
--notebook-id abc-123 \
--cell-id cell-789 \
--database-schema im_10836_2de \
--save-to-file cell.json
Use Edit tool to change query in cell's content.query field
Run:
python3 notebook_client.py run-from-file \
--file cell.json \
--notebook-id abc-123 \
--cell-id cell-789 \
--database-schema im_10836_2de
--database-schema to every command! This is MANDATORY!list first if you don't know cell_idcontent in file is object, but POST requires JSON string (auto-converted)Required:
NEXT_PUBLIC_DTS_SESSION_ID - Session ID for auth (required!)NEXT_PUBLIC_WORKSPACE_ID - Workspace IDOptional:
NEXT_PUBLIC_AI_AGENT_BASE_URL - Platform host (default: https://report.improvado.io)These variables are automatically provided by the sandbox environment.
Files are saved to /workspace/clients/{database_schema}/temp/ directory (based on --database-schema parameter).
This complies with sandbox security restrictions that allow writes only to client-specific folders.
When using relative file paths (e.g., cell.json), they are automatically resolved to the temp directory.
python3 .claude/skills/notebook-editor/notebook_client.py get-integrated-setup \
--notebook-id <notebook-id> \
--database-schema <agency_database_schema>
Retrieves the integrated setup configuration for a transformation (notebook). Returns:
data_sources: Array of selected data source namesallowed_data_sources: Array of available data source namesconnections: Array of connection objects with accountsResponse format:
{
"data_sources": ["Google Ads", "Facebook Ads"],
"allowed_data_sources": ["Google Ads", "Facebook Ads", "LinkedIn Ads"],
"connections": [
{
"connection_id": 123,
"connection_data_source": "Google Ads",
"accounts": [
{"id": 456, "account_id": "123-456-7890"}
]
}
]
}
Error handling:
python3 .claude/skills/notebook-editor/notebook_client.py build-integrated-transformation \
--file build-config.json \
--transformation-id <transformation-id> \
--database-schema <agency_database_schema>
Starts an integrated progressive build for a transformation with specified data sources and connections.
Build configuration file format:
{
"data_sources": ["Google Ads", "Facebook Ads"],
"connections": [
{
"connection_id": 123,
"connection_data_source": "Google Ads",
"accounts": [
{"id": 456, "account_id": "123-456-7890"}
]
},
{
"connection_id": 789,
"connection_data_source": "Facebook Ads",
"accounts": [
{"id": 101, "account_id": "act_EXAMPLE_ACCOUNT_ID"}
]
}
]
}
Workflow for building integrated transformation:
First, get the current integrated setup to see available data sources and connections:
python3 notebook_client.py get-integrated-setup \
--notebook-id <id> \
--database-schema <schema>
Create a build configuration file with desired data sources and connections
Start the build:
python3 notebook_client.py build-integrated-transformation \
--file build-config.json \
--transformation-id <id> \
--database-schema <schema>
Check build status:
python3 notebook_client.py integrated-build-status \
--transformation-id <id> \
--database-schema <schema>
Error handling:
Important notes:
integrated-build-status to monitor build progresspython3 .claude/skills/notebook-editor/notebook_client.py create-cell \
--file new-cell.json \
--notebook-id <notebook-id> \
--database-schema <agency_database_schema>
Creates a new cell in the notebook. The file should contain:
{
"cell_type": "sql",
"content": {
"query": "SELECT * FROM table"
},
"title": "My New Query"
}
Note: Don't include cell_id when creating - it will be auto-generated and returned in the response.
Workflow for creating new cell:
python3 notebook_client.py create-cell --file new-cell.json --notebook-id <id> --database-schema <schema>Use this when you need to add a step between two existing cells. This atomically inserts the new cell and updates all downstream cells' virtual table references.
python3 .claude/skills/notebook-editor/notebook_client.py insert-cell \
--file insert-payload.json \
--notebook-id <notebook-id> \
--database-schema <agency_database_schema>
The file must contain:
{
"previous_cell_id": "PREVIOUS_CELL_ID",
"new_cell": {
"cell_type": "union",
"title": "Blend data",
"content": {
"output_schema": {"columns": [""], "conditions": null},
"tables": [{"name": "", "title": "", "columns": []}]
}
},
"cells_to_update": [
{
"cell_id": "CELL_ID",
"content": { ... }
}
]
}
How it works:
previous_cell_id: The new cell is inserted after this cellnew_cell: The new cell definition (same as create-cell)cells_to_update: All downstream cells with their content updated to reflect shifted cell numbersWhy cells_to_update is needed:
When you insert a cell between cells 1 and 2, the old cell 2 becomes cell 3, cell 3 becomes cell 4, etc. Any cell referencing cell_2_sql_query_label_XXXX must now reference cell_3_sql_query_label_XXXX. The cells_to_update array provides the corrected content for each affected downstream cell.
Workflow for inserting a cell:
list — get all current cells and their IDsget-cell — download each downstream cell that references virtual tablesprevious_cell_id, new_cell, and cells_to_updatepython3 notebook_client.py insert-cell --file insert-payload.json --notebook-id <id> --database-schema <schema>⚠️ MANDATORY: You MUST ask the user for explicit confirmation before deleting ANY cell. Deletion is irreversible. NEVER proceed with deletion without the user saying "yes", "delete it", "go ahead", or similar explicit approval. If the user has not confirmed, DO NOT run the delete command.
python3 .claude/skills/notebook-editor/notebook_client.py delete-cell \
--notebook-id <notebook-id> \
--cell-id <cell-id> \
--database-schema <agency_database_schema>
Workflow for deleting a cell:
list — show the user all cells so they can identify which one to delete{cell_id} ({cell_type}: {title})? This action is irreversible."When creating a new cell, use these initial empty content structures in the file:
Union (Blend Data):
{
"cell_type": "union",
"content": {
"output_schema": {"columns": [""], "conditions": null},
"tables": [{"name": "", "title": "", "columns": []}]
},
"title": "Blend data",
"context": "blend_data"
}
Join:
{
"cell_type": "join",
"content": {
"main_table": "",
"join_table": "",
"join_type": "left",
"join_keys": [],
"additional_main_table_columns": [],
"additional_join_table_columns": []
},
"title": "Join data",
"context": "join"
}
Group By:
{
"cell_type": "group_by",
"content": {
"table_name": "",
"group_by_columns": [],
"aggregations": []
},
"title": "Group by",
"context": "group_by"
}
SQL:
{
"cell_type": "sql",
"content": {
"query": ""
},
"title": "SQL Query",
"context": "sql"
}
Filter Data (conditional_mapping):
{
"cell_type": "conditional_mapping",
"content": [
{
"table_name": "",
"else_condition": {"original_title": "", "new_title": ""},
"mapping": [
{
"original_title": "",
"new_title": "",
"conditions": {
"groups": {
"and_": []
}
}
}
]
}
],
"title": "Filter data",
"context": "filter"
}
For detailed cell content structures, formulas, and examples, see the prompt files linked above.
Below is a minimal quick reference for common operations.
Use lowercase types: date, string, number, datetime
WRONG: Int64, Float64, String, Date
CORRECT: date, string, number, datetime
Simple column mapping (Union):
{"original_name": ["date"], "new_name": "date", "conditions": []}
Column rename (Union):
{"original_name": ["cost"], "new_name": "spend", "conditions": []}
Constant string (Union):
{"original_name": ["'Google Ads'"], "new_name": "data_source", "conditions": []}
Context mapping:
"blend_data""join""group_by""sql""filter"For full examples and detailed documentation, see the linked prompt files above.
Open-source multi-channel campaign launcher. Orchestrates marketing experiments from ICP/positioning input through planning, creative production, and parallel channel launch (Google Ads, Meta Ads, email outreach). No proprietary infrastructure required — works with your own API keys. Use when user says "launch campaign", "campaign launcher", "experiment launch", "multi-channel campaign", "plan and launch experiment".
Use when user says "resume session", "what did I work on", or "find conversation about [topic]". Automatically searches, resumes, and analyzes Claude Code sessions. Handles session discovery, content search, and automatic session restoration from any directory.
Generate a CMO Cross-Channel Performance Dashboard — funnel-stage executive view of marketing spend, pull-through, and per-channel efficiency across Meta, Google, LinkedIn, TikTok, Reddit, TTD, and the Discovery API long tail. Single-screen, three-tab layout (Overview / Channels / Customer Journey) with reactive filter bar (period × channel × conversion event × attribution model). Warehouse-first (ClickHouse `_all_data` views) with Discovery API fallback for channels not in the warehouse; all data from real sources. Built via /business-intelligence-editor as ONE custom-component widget that contains the entire dashboard.
Generate a Daily Performance Console — multi-channel operational dashboard for performance marketers. Blended KPI strip (Spend / Impressions / Clicks / CTR / Conversions / CPA / CR / ROAS), per-channel split with health flags, universal trend chart with date / metric / channel filters, and three tabs (Warnings / Recommendations / Monthly Pacing). Discovery API only, all data live. Built via /business-intelligence-editor as ONE custom-component widget that contains the entire console.
AUTO-INVOKE when user asks to see, fetch, or check current/live/real-time data from any ad platform (Facebook/Meta, Google Ads, TikTok, LinkedIn, Pinterest, Bing, Snap, X, GA4, Google Sheets, etc.), inspect ad accounts/campaigns/ads, pull insights directly from a platform, or any task that requires calling `discoveryRequestTool`, `discoveryListConnectorsTool`, or `discoveryListAccountsTool`. Trigger phrases include: "current campaigns", "live data", "real-time metrics", "show me platform-name ads", "fetch from platform-name API", "what's in my platform-name", "check platform-name insights". Use BEFORE any direct platform API call — the skill defines the mandatory URL/version protocol.
Field mapping and data extraction setup - find report types and validate fields