analyst

name

analyst

description

Support academic research data analysis with technology-agnostic principles — research-data immutability, a versioned/tested/reproducible transformation layer, statistical methodology, and self-documenting research. Use this skill for any computational research project with an empirical data pipeline. The skill enforces academicOps best practices for reproducible, transparent research with a collaborative single-step workflow. Tech-specific how-to (dbt, Streamlit, Python plotting/stats) lives in the aops-tools package.

Analyst

Overview

Support academic research data analysis through technology-agnostic principles: reproducible data pipelines, automated testing, self-documenting code, and fail-fast validation. The principles here hold regardless of which transformation engine or dashboard tool you use. When you have settled on specific tooling, pair this skill with the relevant aops-tools skill (dbt, streamlit, python-viz) for the concrete commands.

Core principle: Take ONE action at a time (generate a chart, update database, create a test), then yield to the user for feedback before proceeding.

Academic research disposition (non-negotiable floor for all academic work):

• Data immutability — source datasets, ground-truth labels, and research configs are sacred; never modify, reformat, or "fix" them — HALT and report rather than reshaping data to fit infrastructure. Violations are scholarly misconduct.
• Research questions drive design — methods serve the question; restate the question, confirm the method fits it, and refuse convenience shortcuts that compromise validity. A result that doesn't answer the question is worthless however technically sound.
• Methodological justification — ensure all model, variable, and sample choices are justified by the research design, not by computational convenience. Do not drop variables, models, or conditions, or simplify experimental designs unless there is a clear methodological justification. Preserve all theoretically meaningful distinctions.
• Dry run / pilot verification — before full-scale execution, run a qualitative pilot audit. Evaluate representative samples of actual outputs for content substance, completeness across all conditions, edge-case behavior, and face validity. Do not declare a dry run successful based on error-free execution or aggregate statistics alone.
• Reproducibility & versioning — every transformation is version-controlled, testable by re-running, and separated from display (never compute in the display layer).
• Methodological transparency — name the assumptions and limitations a result rests on; flag uncertainty rather than smoothing it over.
• Fail-fast on data quality — stop and report quality problems rather than patching around them; the discovery IS the result.
• Report as argument — structure research reports as cohesive arguments where every chapter, section, and visualization directly supports a specific claim. Ground all reported metrics in their practical and theoretical implications. Collaborate section-by-section with the user to refine narrative framing.

The data-pipeline specifics below EXTEND this floor.

🚨 CRITICAL: Data directory separation

Local data files (data/) and build output directories (output/, _book/, etc.) MUST NOT overlap. Build tools clean their output directories — any data stored there will be destroyed. See [[instructions/research-documentation.md#data-directory-separation-critical]] for the full convention.

🚨 CRITICAL: Transformation Layer vs Presentation Layer

ALL data transformation happens in a versioned, tested, reproducible transformation layer. The presentation layer ONLY displays pre-computed data. Period.

This is non-negotiable for academic integrity, reproducibility, and auditability. It is a property of the architecture, not of any particular tool. (e.g. the transformation layer might be a dbt project, a SQL pipeline, or scripted notebooks under version control; the presentation layer might be a Streamlit dashboard, a static report, or a notebook viewer. See the aops-tools dbt and streamlit skills for those concrete implementations.)

Layer	Allowed	Prohibited
Transformation	ALL transformations, joins, aggregations, filtering, business logic	-
Presentation	Display, formatting, interactive filtering of PRE-COMPUTED data	Any operation that transforms, joins, aggregates, or applies logic

Why This Matters (Academic Integrity)

1. Reproducibility: Anyone can re-run the transformation layer and get identical results
2. Auditability: Transformation logic is version-controlled and testable
3. Transparency: Reviewers see exactly how data was processed
4. Testing: Tests in the transformation layer PROVE transformations work correctly

The Rule in Practice

Need a new metric? → Add it to the transformation layer with tests Need to filter data? → Pre-compute the filtered view in the transformation layer OR filter on EXISTING columns in the presentation layer (no new calculations) Need to join tables? → Do the join in the transformation layer Need aggregations? → Compute them in the transformation layer

Presentation Layer: Display ONLY

The presentation layer may:

• ✅ Read pre-computed outputs (SELECT * FROM precomputed_table)
• ✅ Filter on EXISTING columns (WHERE column = :user_selection)
• ✅ Format numbers, dates for display
• ✅ Create interactive widgets that filter existing data
• ✅ Render charts from pre-computed metrics

The presentation layer must NEVER:

• ❌ Aggregate (SUM(...) GROUP BY ... = transformation)
• ❌ Join (a.*, b.* FROM a JOIN b = transformation)
• ❌ Apply business logic (CASE WHEN ... END = transformation)
• ❌ Calculate derived metrics inline
• ❌ Apply any formula that changes the meaning of data

If You're Tempted to Transform in the Presentation Layer

STOP. Move the transformation into the transformation layer instead:

1. Add the transformation as a versioned model/script
2. Add tests proving it works
3. Build/run the transformation layer
4. THEN read the pre-computed output from the presentation layer

This takes more time. That's the point. Transformations deserve scrutiny.

Documentation Index

Instructions (_CHUNKS/)

• Investigation: [[instructions/exploratory-analysis.md]]
• Research docs: [[instructions/research-documentation.md]] (REQUIRED), [[instructions/methodology-files.md]], [[instructions/methods-vs-methodology.md]], [[instructions/experiment-logging.md]]

References

[[references/context-discovery.md]]

Technology-Specific Skills (aops-tools)

The concrete how-to for particular tools lives in the aops-tools package, so it can be swapped for official/community-consensus skills:

• dbt — transformation-layer implementation (models, tests, marts).
• streamlit — presentation-layer implementation (display-only dashboards).
• python-viz — Python plotting & statistical-modelling libraries (matplotlib, seaborn, statsmodels). Use the python-dev skill for code standards.

When to Use This Skill

Invoke this skill when:

1. Working in computational research projects - An empirical data pipeline, analytical database, or transformation/presentation layer is present
2. User requests data analysis - "Analyze X", "Create a chart showing Y", "Explore the relationship between Z"
3. Building or updating dashboards - Presentation-layer visualization work (see the aops-tools streamlit skill for that engine)
4. Creating or modifying transformations - Transformation-layer pipeline work (see the aops-tools dbt skill for that engine)
5. Validating data quality - Adding tests, checking consistency

Key indicators in project structure:

• A version-controlled transformation layer (e.g. a dbt/models/ directory — staging, intermediate, marts)
• A presentation layer (e.g. a streamlit/ directory or dashboard .py files)
• data/warehouse.db or similar analytical database
• Academic research focus (papers, empirical analysis)

Workflow Decision Tree

START
│
├─ Is this a new analysis task?
│  ├─ YES → Go to: Context Discovery
│  └─ NO → Is context already loaded?
│     ├─ YES → Go to: Task Execution
│     └─ NO → Go to: Context Discovery
│
Context Discovery (REQUIRED FIRST STEP)
│
├─ Read project context files:
│  ├─ README.md (current directory + all parents to project root)
│  ├─ data/README.md (if exists)
│  └─ data/projects/[project-name].md (if exists)
│
├─ Identify project conventions:
│  ├─ Research questions
│  ├─ Data sources and access patterns
│  ├─ Existing transformation-layer models (list them)
│  ├─ Testing strategy
│  └─ Project-specific rules
│
└─ Proceed to: Task Execution
│
Task Execution
│
├─ What type of task?
│  ├─ Data access → Go to: Data Access Workflow
│  ├─ Visualization → Go to: Visualization Workflow
│  ├─ Transformation model → Go to: Transformation Model Workflow
│  ├─ Testing → Go to: Testing Workflow
│  └─ Exploration → Go to: Exploratory Analysis
│
└─ After completing ONE step: see "Collaborative Workflow Principles" below.

Context Discovery

CRITICAL FIRST STEP: Before any analysis work, automatically discover and read project context.

Required Context Files

1. Project README files

   • Current working directory README.md
   • All parent directories up to project root (e.g., papers/automod/, projects/buttermilk/)
   • Purpose: Understand research questions, conventions, project structure

2. Data README

   • data/README.md in the project
   • Purpose: Understand data sources, schema, access patterns

3. Project overview

   • data/projects/[project-name].md corresponding to current project
   • Purpose: Strategic context, goals, status

Context Extraction

From these files, identify:

• Research questions - What is this project investigating?
• Data sources - Where does data come from? (BigQuery, APIs, files?)
• Existing transformation models - What models already exist in the transformation layer?
• Conventions - Naming patterns, coding standards, project-specific rules
• Testing strategy - What tests exist? What quality expectations?
• Tools and technologies - Which transformation engine and presentation tool? (e.g. dbt + Streamlit — see the aops-tools skills.) DuckDB? PostgreSQL? Specific Python packages?

Example context discovery:

# List existing transformation-layer models (engine-specific; e.g. dbt)
ls -1 dbt/models/staging/*.sql dbt/models/marts/*.sql

# Check for presentation-layer apps (engine-specific; e.g. Streamlit)
ls -1 streamlit/*.py

# Understand project structure
cat README.md
cat data/README.md

The example commands above assume a dbt + Streamlit stack. For the concrete per-engine discovery commands, see the aops-tools dbt and streamlit skills.

After context discovery, summarize findings to the user — research topic and questions, transformation-layer scope (staging/mart model counts), existing work areas — then ask what to help with.

Follow Data Access Workflow

🚨 CRITICAL RULE: ALL data access MUST go through the modelled transformation layer. NEVER query raw upstream sources directly.

Decision Tree

Need data for analysis?
│
├─ Does required data exist in the modelled (mart) layer?
│  ├─ YES → Read it (e.g. `SELECT * FROM mart_name`)
│  │         └─ Done! Use this data in analysis.
│  │
│  └─ NO → Does it exist in staging models?
│     ├─ YES → Should this become a new mart?
│     │  ├─ YES → Go to: Transformation Model Workflow (create mart)
│     │  └─ NO → Use staging model for exploratory work
│     │
│     └─ NO → Data doesn't exist in the transformation layer yet
│        └─ Ask user: "Should I create a model for [data source]?"
│           ├─ YES → Go to: Transformation Model Workflow (create staging model)
│           └─ NO → Stop. Cannot proceed without a modelled source.

Prohibited Actions

❌ NEVER do this:

# Direct BigQuery query against raw source - PROHIBITED
df = client.query("SELECT * FROM bigquery.raw.cases").to_dataframe()

# Direct database query against raw schema - PROHIBITED
df = pd.read_sql("SELECT * FROM raw_schema.table", engine)

# Direct API call for analysis data - PROHIBITED
response = requests.get("https://api.example.com/data")

✅ ALWAYS do this:

# Query through the modelled layer - CORRECT
import duckdb

conn = duckdb.connect("data/warehouse.db")
df = conn.execute("SELECT * FROM fct_case_decisions").df()  # fct_* = a tested mart

See: the aops-tools dbt skill for the dbt implementation of this policy.

Follow Transformation Model Workflow

Create or modify transformation-layer models following academicOps layered architecture. The layering below is engine-neutral; the aops-tools dbt skill gives the dbt-specific commands and file layout.

Quick Reference: Model Layers

1. Staging (stg_*) - Clean and standardize raw data (no business logic)
2. Intermediate (int_*) - Business logic transformations (can be ephemeral)
3. Marts (fct_*, dim_*) - Analysis-ready datasets (materialized)

ALWAYS check for duplicate models before creating new ones.

See: the aops-tools dbt skill for complete workflow details and comprehensive patterns.

Follow Visualization Workflow

Create presentation-layer visualizations following the single-step collaborative pattern. The presentation layer is display only — see the aops-tools streamlit skill for the engine-specific workflow.

Follow Testing Workflow

Add tests to validate data quality at every pipeline stage.

Testing Strategy

Use appropriate test type for the validation:

Test Type	Use For	Example
Schema tests	Column-level checks	not_null, unique, accepted_values
Singular tests	Multi-column logic	Date range validation, cross-table consistency
Package tests	Common patterns	Recency checks, multi-column uniqueness
Diagnostic models	Quality monitoring	Aggregated metrics for manual review

Follow Single-Step Testing Workflow

Work one step at a time, checkpointing with the user between each:

1. Identify what to test — which columns should never be null, must be unique, have accepted-value lists, or carry date/range logic. STOP; agree the test plan with the user.
2. Add declarative schema tests alongside the model. STOP; show to user.
3. Run the tests. STOP; report results. If failures, discuss before fixing.
4. Add singular/multi-column tests for logic a column-level test can't express. STOP; show, then run and report.

The engine-specific syntax (test declarations, severity: warn for aspirational/known issues, run commands) lives in the aops-tools dbt skill.

Pipeline/Template Validation Tests

When testing LLM pipelines or templated content, validate substantive content not just error patterns:

• ✅ Check content length minimums (e.g., criteria block > 100 chars)
• ✅ Verify required sections exist AND have content
• ✅ Use position-based length for multiline content (regex .*? doesn't cross newlines)
• ❌ Don't just check for specific error strings - upstream bugs are unpredictable

See: the aops-tools dbt skill for complete engine-specific testing patterns.

Follow Data Investigation Workflow

When investigating data quality issues (missing values, unexpected patterns, join coverage), create REUSABLE investigation scripts in analyses/ directory. Never use throwaway one-liners for data investigation — the finding has to be re-runnable by someone else, which a shell history is not.

Exploratory Analysis

When exploring data patterns and relationships, follow collaborative discovery process. Take one analytical step at a time, yielding to user after each finding.

For complete exploration workflow and anti-patterns, see [[instructions/exploratory-analysis.md]]

NOTE: For data quality issues (missing values, unexpected nulls), use Data Investigation Workflow instead.

Documentation Philosophy

Self-documenting work: Do NOT create separate analysis reports or random documentation files.

🚨 CRITICAL: Research projects must follow the STRICT documentation structure in [[instructions/research-documentation.md]] — that file is the complete requirement, including which files are mandatory and which are forbidden. Its per-file detail: [[instructions/methodology-files.md]], [[instructions/methods-vs-methodology.md]], [[instructions/experiment-logging.md]].

Where Analysis Documentation Lives

1. Presentation-layer dashboards - Interactive exploration and validation (e.g. Streamlit)
2. Jupyter notebooks - Detailed analysis with inline markdown (in experiments/ if exploratory)
3. GitHub issues - Track analysis tasks and decisions
4. Code comments - Explain analytical decisions in transformation-layer models
5. Commit messages - Document why changes were made
6. Transformation-layer schema docs - Document model purposes and column meanings (e.g. dbt/schema.yml)
7. methods/*.md - Technical method specifications

Documentation is updated in the SAME commit as the code it describes, and each fact has one home.

Statistical Methodology

The formulas, test-selection trees, and APA reporting shapes are public knowledge and are not restated here. What binds is the methodology, and it is the researcher's call before it is yours:

• The question picks the test, not the data. Choosing a test after seeing which one gives a significant result is p-hacking whatever else it is called. Where the analysis plan was not fixed in advance, say so in the write-up.
• State and check the assumptions the test rests on — independence, distributional form, homogeneity of variance, whatever the specific test requires — and report what you found, including when an assumption fails and you proceeded anyway with a justification.
• Effect sizes and intervals, always. A p-value alone is not a result. Report the magnitude and its uncertainty, and interpret both in the units the research question is asked in.
• Every exploratory pass is exploratory in the write-up. Multiple comparisons, subgroup hunts, and post-hoc contrasts are labelled as such and corrected or flagged; they never migrate into the confirmatory frame.
• Halt on a methodological choice nobody made. Which model, which covariates, which exclusions, how to handle missing data — these are the researcher's, not conveniences to settle so the pipeline runs. Ask.

Where you need a specific library's API, reach for the aops-tools python-viz skill rather than reconstructing it here.

Collaborative Workflow Principles

One step at a time: perform ONE action (create chart, write model, run test), show the result, explain what it means, then STOP and wait for the user's direction. Never run a complex workflow end to end without checkpoints, and never assume the next step — offer the options and ask.

Quick Reference

For engine-specific commands, see the aops-tools dbt and streamlit skills.