Exploratory data analysis (EDA) on a scientific data file — auto-detects the format, runs structure/quality/statistics checks, and writes a markdown EDA report with downstream recommendations. Use when asked to "explore", "analyze", "summarize", "profile", or "QC" a data file, or to understand its structure/content/quality before deciding what analysis to run. Covers tabular (.csv .tsv .xlsx .parquet), arrays (.npy .npz .hdf5 .h5 .mat .fits), sequence/genomics (.fasta .fastq .sam .bam .vcf .bed .gff .gtf .h5ad), microscopy (.tif .nd2 .czi .lif .ims .dcm .nii), spectroscopy/MS (.mzML .mzXML .mgf .fid .jdx), chemistry (.pdb .cif .mol .sdf .xyz .gro), and proteomics/metabolomics (.pepXML .mzid .mzTab). For zero-shot forecasting of a series use alterlab-timesfm; to create/configure a chunked cloud array store use alterlab-zarr. Part of the AlterLab Academic Skills suite.
Exploratory data analysis (EDA) on a scientific data file — auto-detects the format, runs structure/quality/statistics checks, and writes a markdown EDA report with downstream recommendations. Use when asked to "explore", "analyze", "summarize", "profile", or "QC" a data file, or to understand its structure/content/quality before deciding what analysis to run. Covers tabular (.csv .tsv .xlsx .parquet), arrays (.npy .npz .hdf5 .h5 .mat .fits), sequence/genomics (.fasta .fastq .sam .bam .vcf .bed .gff .gtf .h5ad), microscopy (.tif .nd2 .czi .lif .ims .dcm .nii), spectroscopy/MS (.mzML .mzXML .mgf .fid .jdx), chemistry (.pdb .cif .mol .sdf .xyz .gro), and proteomics/metabolomics (.pepXML .mzid .mzTab). For zero-shot forecasting of a series use alterlab-timesfm; to create/configure a chunked cloud array store use alterlab-zarr. Part of the AlterLab Academic Skills suite.
No API key required. Run scripts via `uv run python` (this machine is uv-first); install format-specific parsers with `uv pip install <pkg>` on demand.
metadata
{"skill-author":"AlterLab","version":"1.0.0"}
Exploratory Data Analysis
Overview
Perform comprehensive exploratory data analysis (EDA) on scientific data files across multiple domains. This skill provides automated file type detection, format-specific analysis, data quality assessment, and generates detailed markdown reports suitable for documentation and downstream analysis planning.
Key Capabilities:
Automatic detection and analysis of 88 scientific file formats
Comprehensive format-specific metadata extraction
Data quality and integrity assessment
Statistical summaries and distributions
Visualization recommendations
Downstream analysis suggestions
Markdown report generation
When to Use This Skill
Use this skill when:
User provides a path to a scientific data file for analysis
User asks to "explore", "analyze", or "summarize" a data file
User wants to understand the structure and content of scientific data
User needs a comprehensive report of a dataset before analysis
User wants to assess data quality or completeness
User asks what type of analysis is appropriate for a file
Supported File Categories
The skill has comprehensive coverage of scientific file formats organized into six major categories:
1. Chemistry and Molecular Formats (60+ extensions)
Structure files, computational chemistry outputs, molecular dynamics trajectories, and chemical databases.
File types include:.pdb, .cif, .mol, .mol2, .sdf, .xyz, .smi, .gro, .log, .fchk, .cube, .dcd, .xtc, .trr, .prmtop, .psf, and more.
When a user provides a file path, first identify the file type:
Extract the file extension
Look up the extension in the appropriate reference file
Identify the file category and format description
Load format-specific information
Example:
User: "Analyze data.fastq"
→ Extension: .fastq
→ Category: bioinformatics_genomics
→ Format: FASTQ Format (sequence data with quality scores)
→ Reference: references/bioinformatics_genomics_formats.md
Step 2: Load Format-Specific Information
Based on the file type, read the corresponding reference file to understand:
Typical Data: What kind of data this format contains
Use Cases: Common applications for this format
Python Libraries: How to read the file in Python
EDA Approach: What analyses are appropriate for this data type
Search the reference file for the specific extension (e.g., search for "### .fastq" in bioinformatics_genomics_formats.md).
Step 3: Perform Data Analysis
Use the scripts/eda_analyzer.py script OR implement custom analysis:
Option A: Use the analyzer script (auto-detects type, loads the reference, runs format-specific analysis, writes the report)
uv run python scripts/eda_analyzer.py <filepath> [output.md]
The script has built-in analyzers for tabular (.csv/.tsv), arrays (.npy/.npz/.hdf5), JSON, sequence (.fasta/.fastq), and basic imaging (.tif). For every other format it still detects the type and embeds the reference info, but you perform the data analysis yourself (Option B).
Option B: Custom analysis in the conversation
Based on the format information from the reference file, perform appropriate analysis:
Attempt generic analysis based on file structure (text vs binary)
Provide general recommendations
Large Files
For very large files:
Use sampling strategies (first N records)
Use memory-mapped access (for HDF5, NPY)
Process in chunks (for CSV, FASTQ)
Provide estimates based on samples
Script Usage
uv run python scripts/eda_analyzer.py data.csv # report -> data_eda_report.md
uv run python scripts/eda_analyzer.py data.csv output_report.md
The script auto-detects the file type, loads the matching reference section, runs built-in analysis where available, and writes the markdown report. For formats without a built-in analyzer, prefer custom analysis in the conversation (Option B) for domain-specific insight. Note: for .csv/.tsv the script samples the first 10,000 rows, so report dimensions/missing counts as sampled unless you re-run on the full file.
Advanced Usage
Multi-file: EDA each file individually, then write a comparison report noting relationships, dependencies, and integration strategy.
Quality control: check format compliance, validate metadata consistency (stated vs actual dimensions), assess completeness, flag outliers/anomalies against expected ranges.
Preprocessing recommendations: tailor to the data — normalization, missing-value imputation, outlier handling, batch correction, format conversion.
Resources
scripts/
eda_analyzer.py: Comprehensive analysis script that can be run directly or imported
