Exploratory data analysis (EDA) on a scientific data file — auto-detects the format, runs structure/quality/statistics checks, and writes a markdown EDA report with downstream recommendations. Use when asked to "explore", "analyze", "summarize", "profile", or "QC" a data file, or to understand its structure/content/quality before deciding what analysis to run. Covers tabular (.csv .tsv .xlsx .parquet), arrays (.npy .npz .hdf5 .h5 .mat .fits), sequence/genomics (.fasta .fastq .sam .bam .vcf .bed .gff .gtf .h5ad), microscopy (.tif .nd2 .czi .lif .ims .dcm .nii), spectroscopy/MS (.mzML .mzXML .mgf .fid .jdx), chemistry (.pdb .cif .mol .sdf .xyz .gro), and proteomics/metabolomics (.pepXML .mzid .mzTab). For zero-shot forecasting of a series use alterlab-timesfm; to create/configure a chunked cloud array store use alterlab-zarr. Part of the AlterLab Academic Skills suite.
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
A direct command skips the review prompt. Inspect the source before running it.
Exploratory data analysis (EDA) on a scientific data file — auto-detects the format, runs structure/quality/statistics checks, and writes a markdown EDA report with downstream recommendations. Use when asked to "explore", "analyze", "summarize", "profile", or "QC" a data file, or to understand its structure/content/quality before deciding what analysis to run. Covers tabular (.csv .tsv .xlsx .parquet), arrays (.npy .npz .hdf5 .h5 .mat .fits), sequence/genomics (.fasta .fastq .sam .bam .vcf .bed .gff .gtf .h5ad), microscopy (.tif .nd2 .czi .lif .ims .dcm .nii), spectroscopy/MS (.mzML .mzXML .mgf .fid .jdx), chemistry (.pdb .cif .mol .sdf .xyz .gro), and proteomics/metabolomics (.pepXML .mzid .mzTab). For zero-shot forecasting of a series use alterlab-timesfm; to create/configure a chunked cloud array store use alterlab-zarr. Part of the AlterLab Academic Skills suite.
No API key required. Run scripts via `uv run python` (this machine is uv-first); install format-specific parsers with `uv pip install <pkg>` on demand.
metadata
{"skill-author":"AlterLab","version":"1.0.0"}
Exploratory Data Analysis
Overview
Perform comprehensive exploratory data analysis (EDA) on scientific data files across multiple domains. This skill provides automated file type detection, format-specific analysis, data quality assessment, and generates detailed markdown reports suitable for documentation and downstream analysis planning.
Key Capabilities:
Automatic detection and analysis of 88 scientific file formats
Comprehensive format-specific metadata extraction
Data quality and integrity assessment
Statistical summaries and distributions
Visualization recommendations
Downstream analysis suggestions
Markdown report generation
When to Use This Skill
Use this skill when:
User provides a path to a scientific data file for analysis
User asks to "explore", "analyze", or "summarize" a data file
User wants to understand the structure and content of scientific data
User needs a comprehensive report of a dataset before analysis
User wants to assess data quality or completeness
User asks what type of analysis is appropriate for a file
Supported File Categories
The skill has comprehensive coverage of scientific file formats organized into six major categories:
1. Chemistry and Molecular Formats (60+ extensions)
Structure files, computational chemistry outputs, molecular dynamics trajectories, and chemical databases.
File types include:.pdb, .cif, .mol, .mol2, .sdf, .xyz, .smi, .gro, .log, .fchk, .cube, .dcd, .xtc, .trr, .prmtop, .psf, and more.
When a user provides a file path, first identify the file type:
Extract the file extension
Look up the extension in the appropriate reference file
Identify the file category and format description
Load format-specific information
Example:
User: "Analyze data.fastq"
→ Extension: .fastq
→ Category: bioinformatics_genomics
→ Format: FASTQ Format (sequence data with quality scores)
→ Reference: references/bioinformatics_genomics_formats.md
Step 2: Load Format-Specific Information
Based on the file type, read the corresponding reference file to understand:
Typical Data: What kind of data this format contains
Use Cases: Common applications for this format
Python Libraries: How to read the file in Python
EDA Approach: What analyses are appropriate for this data type
Search the reference file for the specific extension (e.g., search for "### .fastq" in bioinformatics_genomics_formats.md).
Step 3: Perform Data Analysis
Use the scripts/eda_analyzer.py script OR implement custom analysis:
Option A: Use the analyzer script (auto-detects type, loads the reference, runs format-specific analysis, writes the report)
uv run python scripts/eda_analyzer.py <filepath> [output.md]
The script has built-in analyzers for tabular (.csv/.tsv), arrays (.npy/.npz/.hdf5), JSON, sequence (.fasta/.fastq), and basic imaging (.tif). For every other format it still detects the type and embeds the reference info, but you perform the data analysis yourself (Option B).
Option B: Custom analysis in the conversation
Based on the format information from the reference file, perform appropriate analysis:
Attempt generic analysis based on file structure (text vs binary)
Provide general recommendations
Large Files
For very large files:
Use sampling strategies (first N records)
Use memory-mapped access (for HDF5, NPY)
Process in chunks (for CSV, FASTQ)
Provide estimates based on samples
Script Usage
uv run python scripts/eda_analyzer.py data.csv # report -> data_eda_report.md
uv run python scripts/eda_analyzer.py data.csv output_report.md
The script auto-detects the file type, loads the matching reference section, runs built-in analysis where available, and writes the markdown report. For formats without a built-in analyzer, prefer custom analysis in the conversation (Option B) for domain-specific insight. Note: for .csv/.tsv the script samples the first 10,000 rows, so report dimensions/missing counts as sampled unless you re-run on the full file.
Advanced Usage
Multi-file: EDA each file individually, then write a comparison report noting relationships, dependencies, and integration strategy.
Quality control: check format compliance, validate metadata consistency (stated vs actual dimensions), assess completeness, flag outliers/anomalies against expected ranges.
Preprocessing recommendations: tailor to the data — normalization, missing-value imputation, outlier handling, batch correction, format conversion.
Resources
scripts/
eda_analyzer.py: Comprehensive analysis script that can be run directly or imported
