| name | dataset-question-generator |
| description | Generate fresh, visualisable analysis questions from a raw tabular dataset. Use when Codex is given a CSV/XLSX/Parquet/database extract and asked what to ask, what to explore, what charts to make, what visualisation workshop prompts to use, or what data stories might be interesting; especially for Karthik-style exploratory analysis where obvious/stale questions should be filtered out before charting. |
| metadata | {"short-description":"Find fresh questions from raw data","claude-description":"Generate fresh, visualisable data questions from raw datasets; reject stale prompts before charting."} |
Dataset Question Generator
Use this before analysis planning or charting. The job is to turn a raw dataset into a short ranked set of good questions.
Core sequence:
raw dataset → profile → signals → candidate questions → freshness filter → ranked prompts
This skill is deliberately upstream of karthik-analysis-planner, dataviz-selector, and karthik-data-visualization. Do not start with chart forms. Start with what the data makes worth asking.
Workflow
- Inspect the data first. Do not brainstorm from the filename alone.
- Profile the dataset: row count, row grain, date range, entities, measures, categories, missingness, and format breaks.
- Identify visible signals: slope change, crossing, plateau, rebound, spike, seasonality, outlier, cluster, concentration, substitution, divergence, or denominator trap.
- Generate 8-15 rough candidate questions.
- Reject questions that are stale, obvious, non-measurable, or only restate column names.
- Return the best 3-7 questions, ranked. Fewer is fine if only a few are good.
What to inspect
Minimum profile:
- Unit: what does one row represent?
- Coverage: time range, geography, entities, cohorts, currentness.
- Measures: counts, totals, shares, rates, rankings, amounts, durations, text fields.
- Dimensions: entity, segment, region, channel, cohort, status, category.
- Missingness: columns/periods/entities with suspicious gaps.
- Format breaks: old/new schemas, changed definitions, renamed categories, one-off shocks.
- Derived metrics: share, per-capita/per-unit, average size, rate, gap, index, concentration, volatility.
Karthik-style question patterns
Prefer questions with a visible comparison or mechanism:
- Has X actually changed, or has denominator Y changed?
- Is growth coming from more events, bigger events, or a change in mix?
- Which series has become boring/mature, and is that the interesting fact?
- Where is the gap widening or narrowing?
- Which entities are exceptions to the overall trend?
- Did an event or format break change the level, slope, or composition?
- Is the latest period unusual versus history, or only versus memory?
- Are totals driven by a few big entities or broad participation?
- Does the same story hold in value, volume, share, and average size?
- Where are things stuck, delayed, concentrated, or leaking?
Local analysis priors behind this:
- Bangalore weather: test lived disputes against historical normals and records.
- Bangalore wind/rain: abandon obvious but useless charts; ask a simpler mechanism question.
- Payments/demonetisation: separate volume, value, average size, and counterfactual trend.
- Elections: look for swings, regions, margins, corners, and seat-vote conversion - not just winners.
- Operations data: find bottlenecks, time-in-stage, delay cost, and peer benchmarks.
- Survey data: use ordered subgroup comparisons, concentration, and correlation structure.
- Urban morphology: compare measured features against terrain, history, or periphery - do not just map them.
Freshness filter
Before final output, test each candidate:
| Criterion | Reject if weak |
|---|
| Visual contrast | no likely crossing, gap, slope change, outlier, distribution, or cluster |
| Freshness | obvious, dated, or already answered by the public narrative |
| Dataset support | needs external causal story not in the data |
| Denominator clarity | no clear unit or denominator |
| Karthik fit | dashboard-y, generic, or corporate-slop phrasing |
| Teachability | no useful lesson in chart choice or metric design |
If a question sounds like "trend of X over time", rewrite it around the comparison: compared to what, split by whom, measured how, and why now?
Output modes
Questions only
Return only short questions, one per line. No chart notes.
Workshop seeds
### <question>
Use: <fields/entities>
Chart idea: <simple visual>
Why useful: <one sentence>
Watch out: <denominator/caveat>
Analysis-ready
- Question:
- Unit:
- Metric:
- Comparison:
- Visual:
- Caveat/falsifier:
Hard rules
- Do not answer questions before profiling.
- Do not overfit to column names; inspect values and ranges.
- Do not include stale defaults just because the domain suggests them.
- Do not recommend dashboards as the output. Recommend interpreted questions.
- Do not use causal words unless the comparison design supports them.
- Prefer fewer, sharper questions over a long generic list.
