bio-proteomics-dia-analysis

name	bio-proteomics-dia-analysis
description	Analyzes data-independent acquisition (DIA) proteomics by scoring reconstructed fragment-chromatogram peak groups against a decoy null with DIA-NN (library-free directDIA, library-based, or deep-learning predicted-library routes), Spectronaut, OpenSWATH, and EncyclopeDIA. Frames the deliverable around q-value LEVEL (precursor/peptide/protein-group) and CONTEXT (run vs experiment-wide/global) rather than a bare "1% FDR", and around the duty-cycle-vs-selectivity acquisition tradeoff (window design, staggered demultiplexing, diaPASEF, narrow-window Astral). Use when identifying and quantifying proteins from DIA mass spectrometry runs and filtering DIA-NN report.parquet/matrix output. Building the spectral library itself is spectral-libraries; normalization and protein roll-up is quantification; statistical testing of the matrix is differential-abundance.
origin	openai4s
category	bioskills/proteomics
metadata	{"tool_type":"cli","primary_tool":"diann","third_party":{"name":"GPTomics/bioSkills","repository":"https://github.com/GPTomics/bioSkills","commit":"d91ed3d563019e649dc854c56ccd62551359488a","license":"MIT"}}

Version Compatibility

Reference examples tested with: DIA-NN 1.9+, pandas 2.2+, pyarrow 15+

Before using code patterns, verify installed versions match. If versions differ:

• Python: pip show <package> then help(module.function) to check signatures
• CLI: <tool> --version then <tool> --help to confirm flags

If code throws ImportError, AttributeError, or TypeError, introspect the installed package and adapt the example to match the actual API rather than retrying.

DIA Analysis -- Scoring Reconstructed Peak Groups Against a Decoy Null, Filtered at the Right q-Value Context

"Identify and quantify proteins from my DIA runs" -> Reconstruct, per candidate peptide, a set of co-eluting fragment extracted-ion chromatograms (XICs) and score whether that peak group is real against a decoy null -- because every wide-isolation-window MS2 is chimeric, so the problem is deconvolution and peak-group scoring, not spectrum matching.

• CLI: diann --fasta-search for library-free (directDIA) discovery and quantification
• CLI: diann --lib predicted.speclib for the deep-learning predicted-library route (the modern default)
• CLI: diann --lib experimental.tsv for an experimental or chromatogram library
• CLI: OpenSwathWorkflow + pyprophet when explicit run/experiment/global FDR contexts must be auditable

Scope: this skill OWNS running the DIA search engine and FILTERING its output at the correct q-value level and context. Building the library (experimental, chromatogram, predicted) -> spectral-libraries. Normalization, MaxLFQ roll-up, and matrix summarization -> quantification. Statistical testing of the protein matrix -> differential-abundance. Loading raw vendor/mzML data -> data-import. OUT OF SCOPE: DDA spectrum-to-peptide matching (peptide-identification); pathway enrichment of the hit list; acquiring the data (the analyst inherits the window design from the core facility).

The Single Most Important Modern Insight -- The q-Value Context Is a Study-Design Choice, Not a Default

1. DIA quantification is peak-group SCORING against a decoy null, not spectrum matching. Gillet 2012 inverted DDA: instead of asking "what peptide is this spectrum", DIA asks, per library peptide, "does a co-eluting peak group of this peptide's expected fragments exist in the chimeric MS2 stream". Decoys are shuffled or reversed peptide queries scored identically; the q-value is the expected fraction of accepted IDs that are decoy-like false peak groups. The count of "proteins found" is therefore a function of the decoy-calibrated threshold, never a quality metric in itself.

2. "1% FDR" is meaningless without naming the LEVEL and the CONTEXT -- state both. LEVEL = precursor vs peptide vs protein-group (filtering precursors at 1% does NOT give proteins at 1%; control both). CONTEXT = run-specific vs experiment-wide vs global (Rosenberger 2017). Naively filtering N runs at per-run 1% inflates the experiment-wide error: 1% per run accumulates false positives across the union, severe at hundreds-to-thousands of runs. For a cross-run matrix, filter on the GLOBAL protein-group q-value, not the per-run one. The column chosen (Q.Value vs Global.PG.Q.Value) is the decision.

3. The predicted-library route is now the default recommendation. Library-based search is sensitive but capped by an ill-matched library (wrong organism/tissue/mods silently limits coverage with no error). Library-free directDIA finds sample-specific content but its larger implicit search space can INFLATE IDs if FDR is not controlled across the two-pass process -- worst on wide-window chimeric data. The compromise the field converged on: predict an in-silico library for the whole FASTA digest (DIA-NN built-in predictor, or Prosit/AlphaPeptDeep) and search against THAT, getting directDIA's "no wet-lab library" with library-based's bounded, better-calibrated search.

Chimerism, Deconvolution, and the Window Design the Analyst Inherits

DDA picks top-N precursors and fragments each in isolation, so every MS2 is nominally one peptide. DIA abandons selection: the quadrupole steps through wide isolation windows (4-25 Th classically, 2 Th on Astral, mobility-gated on timsTOF) and co-fragments EVERY precursor in each window. Consequence: every DIA MS2 is chimeric, a superposition of fragments from all co-isolated precursors. The engine must deconvolve -- reconstruct each candidate's fragment XICs and score the peak group.

Selectivity is set by isolation-window WIDTH. Narrower window = fewer co-isolated precursors = less chimerism = cleaner XICs = fewer false peak groups. But narrower windows mean MORE windows per cycle -> longer duty cycle -> fewer points across each LC peak -> worse quant precision. The central acquisition tradeoff is duty cycle (sampling speed) vs selectivity (window width). Rule of thumb: aim for >= 6 MS2 points across the FWHM of an LC peak for reliable quant. The analyst INHERITS this design and must not pretend all DIA is equivalent:

• Fixed windows (classic SWATH): 32 x 25 Th. Simple; wastes selectivity because precursor density is non-uniform across m/z.
• Variable windows: widths chosen so each holds roughly equal precursor density (narrow where the proteome is dense ~600-800 m/z). Orbitrap best practice.
• Staggered / overlapping windows + demultiplexing (Amodei 2019): two interleaved patterns offset by half a window; demultiplexing recovers effective windows of half the physical width without halving duty cycle. CRITICAL trap: staggered data MUST be demultiplexed at conversion (msconvert --filter "demultiplex optimization=overlap_only") or every tool sees the wide physical window and the selectivity benefit is silently lost. MSX (randomized window combinations) is largely historical.
• diaPASEF (Meier 2020): on timsTOF the isolation tile rides the m/z-vs-ion-mobility diagonal, so only precursors sharing BOTH m/z AND mobility co-isolate -- the mobility dimension is an orthogonal selectivity filter for free. The engine extracts a 4D peak group (RT x m/z x fragment x mobility).
• Narrow-window Astral (Guzman 2024): >200 Hz MS/MS makes 2-Th windows feasible across the whole range; at 2 Th the MS2 is nearly non-chimeric, which makes library-free directDIA far more trustworthy than it was on 25-Th SWATH.

Tool Taxonomy

Tool / method	Citation	Mechanism / role	When
DIA-NN	Demichev 2020	Deep-NN peak-group scoring + interference correction + QuantUMS quant; library-free, predicted, or library-based	Default for high-throughput, large cohorts, diaPASEF, Astral; free, scriptable
Spectronaut	Biognosys (commercial)	directDIA+ pipeline with in-app DL prediction; mature GUI/QC	Regulated/clinical work, polished QC, mixed vendors, when licensed
OpenSWATH + PyProphet	Rost 2014; Rosenberger 2017	Classic peptide-centric extraction + semi-supervised scoring with explicit run/experiment/global q-contexts	When auditable FDR-context control is required; library-based ONLY, needs iRT/RT alignment
EncyclopeDIA / Walnut	Searle 2018	Chromatogram-library search (.dlib/.elib) + GPF; Walnut = library-free mode	Building project-specific chromatogram-library depth on Orbitrap
FragPipe (MSFragger-DIA / DIA-Umpire)	--	Spectrum-centric via pseudo-spectra + Philosopher FDR; IonQuant explicit MBR-FDR	Unified DDA+DIA shop in the MSFragger ecosystem
Skyline	MacCoss lab	Targeted/visual peak inspection and demultiplexing; not a discovery engine	Manual peak curation, PRM, library curation -- the microscope, not the engine
AlphaDIA	Mann lab	Open transformer-based end-to-end Python; AlphaPeptDeep predictions	Cutting-edge open research, Astral, Python-native pipelines
Library build	(route OUT)	Experimental/chromatogram/predicted library construction	-> spectral-libraries
Stats on the matrix	(route OUT)	Normalization, roll-up, moderated testing	-> quantification, differential-abundance

Tool leadership moves fast (Astral, AlphaDIA, DIA-NN releases). Confirm the current recommended engine and version for the specific instrument before committing rather than hard-coding "DIA-NN is best".

Decision Tree by Scenario

Scenario	Recommended	Why
Discovery cohort, no wet-lab library	diann --fasta-search --gen-spec-lib (predicted route) then search against it	Bounded, better-calibrated search vs raw directDIA; the modern default
Quick single-run discovery, Astral 2-Th data	DIA-NN library-free (directDIA)	Near-non-chimeric MS2 makes directDIA trustworthy
Have a deep experimental/chromatogram library	diann --lib library.tsv (no --fasta-search)	Targeted extraction is most sensitive when the library matches
Need auditable run/experiment/global FDR for a regulated submission	OpenSWATH + PyProphet	Explicit q-value contexts per Rosenberger 2017
Building chromatogram-library depth for one project on Orbitrap	EncyclopeDIA (GPF) -> .elib -> DIA-NN	Empirical RT and real fragmentation in the project's own LC
Large cohort (hundreds-thousands of runs)	DIA-NN --reanalyse, filter on Global.PG.Q.Value	Two-pass global FDR controls cross-run error accumulation
Staggered/overlapping acquisition	Demultiplex at conversion FIRST, then any engine	Skipping demux silently keeps wide-window interference
PTM / peptidoform-resolved work	DIA-NN --peptidoforms + matched variable mods	Peptidoform-resolved target-decoy scoring

Default when uncertain: DIA-NN with the predicted-library route (--fasta-search --gen-spec-lib --reanalyse), letting --mass-acc 0 auto-optimize, then filter Q.Value <= 0.01 & PG.Q.Value <= 0.01 per run and Global.PG.Q.Value <= 0.01 for cross-run matrices.

DIA-NN -- Predicted-Library (directDIA) Route

The default route: digest the FASTA in silico, predict a library, and search the DIA data against it in one command. --mass-acc 0 lets DIA-NN auto-optimize tolerances per file (do not hard-code ppm from another instrument). --reanalyse enables the two-pass global FDR / MBR that controls directDIA double-dipping.

diann \
    --f sample1.mzML --f sample2.mzML \
    --lib "" --fasta uniprot_human.fasta --fasta-search \
    --gen-spec-lib --predictor \
    --out diann_out/report.parquet \
    --out-lib diann_out/report-lib.tsv \
    --qvalue 0.01 \
    --matrices \
    --mass-acc 0 \
    --reanalyse --smart-profiling \
    --cut K*,R* --missed-cleavages 1 \
    --min-pep-len 7 --max-pep-len 30 \
    --unimod4 --var-mods 1 --var-mod UniMod:35,15.994915,M \
    --threads 8

DIA-NN -- Library-Based Route

Supply an existing library (experimental, chromatogram-derived, or a previously predicted .speclib). Omit --fasta-search: extraction is targeted to the library content.

diann \
    --f sample1.mzML --f sample2.mzML \
    --lib spectral_library.tsv \
    --out diann_out/report.parquet \
    --qvalue 0.01 --matrices \
    --mass-acc 0 \
    --reanalyse --smart-profiling \
    --threads 8

DIA-NN Output and Correct Filtering

DIA-NN 1.9+ writes the main report as Apache Parquet (report.parquet) by default; 2.0 makes it the only default. Matrices stay TSV. Pipelines hard-coding report.tsv silently break or read a stale file -- read parquet. Filter on q-value columns BEFORE pivoting to a matrix.

report.parquet          # main report (1.9+ default; was report.tsv pre-1.9)
report.stats.tsv        # per-run statistics
report.pg_matrix.tsv    # protein-group wide matrix
report.pr_matrix.tsv    # precursor wide matrix (verify exact dotting vs installed version)
report.gg_matrix.tsv    # gene-group wide matrix
report-lib.tsv          # generated library (if --gen-spec-lib)

import pandas as pd, numpy as np
report = pd.read_parquet('diann_out/report.parquet')  # NOT report.tsv on 1.9+

# Per-run filter: both LEVELS. Add Global.PG.Q.Value for the cross-run matrix.
filt = report[(report['Q.Value'] <= 0.01) &
              (report['PG.Q.Value'] <= 0.01) &
              (report['Global.PG.Q.Value'] <= 0.01)]  # 0.01 = standard 1% FDR

# Pivot to a protein matrix from the filtered long report.
pg = filt.pivot_table(index='Protein.Group', columns='Run', values='PG.MaxLFQ', aggfunc='first')
pg = np.log2(pg.replace(0, np.nan))  # DIA-NN writes 0 for not-quantified; log2(0) = -Inf

The *_matrix.tsv files apply an EXTRA 5% run-specific protein FDR (DIA-NN default, --matrix-spec-q), so the matrix protein count can be lower than the report count. A report-vs-matrix mismatch is EXPECTED, not a bug -- do not panic and do not compare the two counts as if they should match.

Generating a Predicted Library Outside DIA-NN

DIA-NN's built-in predictor covers the common case. For an external predicted library from FragPipe search results, EasyPQP builds it; FragPipe can also emit a DIA-NN-format library directly.

# Real easypqp subcommands: library, convert, insilico-library (NOT a convert --format diann).
easypqp library \
    --psmtsv psm.tsv \
    --rt_reference irt.tsv \
    --peptide_fdr_threshold 0.01 \
    --protein_fdr_threshold 0.01 \
    --out library.tsv
# FragPipe's DIA workflow can emit a DIA-NN-format library directly -- prefer that when in FragPipe.

Per-Method Failure Modes

Library-free (directDIA)

Trigger: --fasta-search on wide-window (25-Th SWATH) data without two-pass global FDR. Mechanism: building the library from the same data then quantifying it reuses the data twice; the large implicit search space inflates IDs when decoys are not controlled across both passes. Symptom: implausibly high protein counts, poor reproducibility across replicates. Fix: keep --reanalyse (two-pass global FDR) ON and filter on Global.PG.Q.Value; prefer the predicted-library route on chimeric data.

Library-based

Trigger: library organism/tissue/modifications/gradient do not match the sample. Mechanism: targeted extraction can only find what is in the library; a mismatched library caps coverage with no error. Symptom: low ID counts, conserved-peptide bias (e.g. a human library on a mouse sample finds only conserved peptides). Fix: match the library to the biology (mods especially); regenerate via spectral-libraries or switch to the predicted route.

Cross-run cohort FDR

Trigger: filtering N runs at per-run Q.Value <= 0.01 and unioning. Mechanism: 1% per run accumulates across the union -> experiment-wide error far above 1%. Symptom: inflated total protein list; irreproducible "hits" in differential testing. Fix: filter on Global.PG.Q.Value <= 0.01 (and optionally Lib.PG.Q.Value <= 0.01) for the matrix.

Match-between-runs (MBR)

Trigger: treating MBR-transferred quant values as equally confident as directly identified ones. Mechanism: transferring an ID by RT/m/z/CCS matching can be a false transfer, especially for low-abundance precursors; MBR has its OWN FDR. Symptom: spurious low-abundance quant filling missing values that should stay missing. Fix: rely on DIA-NN's global/empirical-library q-values (IonQuant uses an explicit MBR-FDR model); do not disable --reanalyse then trust per-run counts.

Staggered acquisition

Trigger: running staggered/overlapping data through any engine without demultiplexing. Mechanism: the engine sees the wide physical window, keeping all the interference the staggering was meant to remove. Symptom: noisy directDIA, poor selectivity on data that should be clean. Fix: demultiplex at conversion (msconvert --filter "demultiplex optimization=overlap_only") before search.

Quantitative Thresholds

Threshold	Source	Rationale
Precursor q-value <= 0.01	DIA-NN default (--qvalue 0.01)	Standard 1% per-precursor FDR (run context).
Global.PG.Q.Value <= 0.01	Rosenberger 2017	Experiment-wide protein-group FDR for cross-run matrices; run-specific PG q-value is NOT enough for a cohort.
Matrix run-specific PG filter 0.05	DIA-NN default (--matrix-spec-q)	Extra 5% run-specific protein FDR applied only when building matrices -> report vs matrix count differs (expected).
Lib.(PG.)Q.Value <= 0.01	Demichev recommendation	For very large cohorts, also filter the global library-pass q-values to keep experiment-wide FDR honest.
Points per peak >= 6 across FWHM	community rule of thumb	Below this, quant precision and peak detection degrade; drives window/cycle design.
Mass accuracy auto (--mass-acc 0)	DIA-NN	Wrong tolerance silently kills IDs; let DIA-NN auto-calibrate rather than hard-coding ppm from another instrument.
Missed cleavages 1	--	Trypsin standard; raising it expands the search space and the multiple-testing burden.

Common Errors

Error / symptom	Cause	Solution
FileNotFoundError: report.tsv or stale data	DIA-NN 1.9+ default is report.parquet, not report.tsv	Read report.parquet (pd.read_parquet); request legacy TSV explicitly only if needed
Matrix protein count < report count, looks like data loss	Matrices apply an extra 5% run-specific PG filter	Expected; do not compare the two counts as if equal
-Inf after log2 of the matrix	DIA-NN writes 0 for not-quantified	Convert 0 -> NaN BEFORE log2/normalization
Cohort "hits" do not reproduce	Filtered per-run Q.Value only, not global	Filter Global.PG.Q.Value <= 0.01 for the matrix
easypqp convert --format diann errors	No such interface; convert/library/insilico-library are the real subcommands	Use easypqp library (with --psmtsv/--rt_reference) or let FragPipe emit a DIA-NN-format library
KeyError: 'report.pr.matrix.tsv'	Matrix filename dotting varies by version (pr_matrix vs pr.matrix)	ls the output dir after a run and match the installed version's exact names
directDIA looks noisy on overlapping-window data	Staggered data not demultiplexed	Demultiplex at conversion before search

References

• Gillet LC, Navarro P, Tate S, et al. Targeted data extraction of the MS/MS spectra generated by data-independent acquisition: a new concept for consistent and accurate proteome analysis. Mol Cell Proteomics 2012;11(6):O111.016717.
• Rost HL, Rosenberger G, Navarro P, et al. OpenSWATH enables automated, targeted analysis of data-independent acquisition MS data. Nat Biotechnol 2014;32(3):219-223.
• Rosenberger G, Bludau I, Schmitt U, et al. Statistical control of peptide and protein error rates in large-scale targeted data-independent acquisition analyses. Nat Methods 2017;14(9):921-927.
• Searle BC, Pino LK, Egertson JD, et al. Chromatogram libraries improve peptide detection and quantification by data independent acquisition mass spectrometry. Nat Commun 2018;9:5128.
• Demichev V, Messner CB, Vernardis SI, Lilley KS, Ralser M. DIA-NN: neural networks and interference correction enable deep proteome coverage in high throughput. Nat Methods 2020;17(1):41-44.
• Meier F, Brunner AD, Frank M, et al. diaPASEF: parallel accumulation-serial fragmentation combined with data-independent acquisition. Nat Methods 2020;17(12):1229-1236.
• Amodei D, Egertson J, MacLean BX, et al. Improving precursor selectivity in data-independent acquisition using overlapping windows. J Am Soc Mass Spectrom 2019;30(4):669-684.
• Savitski MM, Wilhelm M, Hahne H, Kuster B, Bantscheff M. A scalable approach for protein false discovery rate estimation in large proteomic data sets. Mol Cell Proteomics 2015;14(9):2394-2404.
• Guzman UH, Martinez-Val A, Olsen JV, et al. Ultra-fast label-free quantification and comprehensive proteome coverage with narrow-window data-independent acquisition. Nat Biotechnol 2024;42:1855-1866.

Related Skills

• spectral-libraries - Build experimental, chromatogram, or predicted libraries to search against
• quantification - Normalization, MaxLFQ roll-up, and matrix summarization after filtering
• differential-abundance - Moderated statistical testing of the protein matrix
• proteomics-qc - Per-run ID counts, missing-value rates, and acquisition QC
• data-import - Convert and load raw vendor/mzML data before the search
• peptide-identification - DDA spectrum-to-peptide matching (the non-DIA counterpart)
• workflows/proteomics-pipeline - End-to-end DIA-to-differential-abundance orchestration