VGP assembly pipeline - Galaxy workflow selection, execution patterns, QC checkpoints, and batch orchestration
VGP assembly pipeline - Galaxy workflow selection, execution patterns, QC checkpoints, and batch orchestration
npx skills add https://github.com/Delphine-L/claude_global --skill vgp-pipelineこのコマンドをClaude Codeにコピー＆ペーストしてスキルをインストール
Run shell commands bare — no decorative echo headers ("=== X ==="), no echo-then-cmd chains, no trailing "echo done". Use the Bash tool's description field for any narration. Triggers any time you're about to issue a Bash command.
Expert guide for managing Claude Code global skills and commands. Use when creating new skills, symlinking to projects, updating existing skills, or organizing the centralized skill repository.
Token optimization best practices for cost-effective Claude Code usage. Automatically applies efficient file reading, command execution, and output handling strategies. Includes model selection guidance (Opus for learning, Sonnet for development/debugging). Prefers bash commands over reading files.
BioBlend and Planemo expertise for Galaxy workflow automation. Galaxy API usage, workflow invocation, status checking, error handling, batch processing, and dataset management. Essential for any Galaxy automation project.
Expert in Galaxy tool wrapper development, XML schemas, Planemo testing, and best practices for creating Galaxy tools
Expert in Galaxy workflow development, testing, and IWC best practices. Create, validate, and optimize .ga workflows following Intergalactic Workflow Commission standards.
| name | vgp-pipeline |
| description | VGP assembly pipeline - Galaxy workflow selection, execution patterns, QC checkpoints, and batch orchestration |
The Vertebrate Genome Project (VGP) assembly pipeline consists of Galaxy workflows for producing high-quality, phased, chromosome-level genome assemblies. This skill covers workflow selection, execution patterns, and quality control checkpoints.
Supporting files (detailed reference material):
VGP user accounts have a 250 GB quota. If quota_percent is at 100%, all new upload jobs land in paused state immediately and planemo dies with a cryptic message:
Exception: Upload job [...] failed with state [paused]
The error gives no hint that quota is the cause. Check before launching any planemo test:
mcp__Galaxy__connect(url=$GXYVGP, api_key=$TESTKEY)
mcp__Galaxy__get_user() # Inspect quota_percent in response
If quota is full, free up space (delete old test histories) and re-check before retrying.
When documenting workflow selection in publications:
"For species with available parental data (trio datasets), we employed
VGP2 -> VGP5 workflows. For species without parental data (non-trio datasets),
we performed VGP1 -> VGP3 workflows."
| Workflow | Name | Description |
|---|---|---|
| WF0 | Mitochondrial Assembly | MitoHiFi assembly (runs in parallel, may fail if no mito reads) |
| WF1 | K-mer Profiling | Genome size, heterozygosity estimation (HiFi) |
| WF2 | Trio K-mer Profiling | K-mer profiling with parental data |
| WF3 | Hifiasm | HiFi-only assembly |
| WF4 | Hifiasm + HiC | HiC-phased assembly |
| WF5 | Hifiasm Trio | Trio-phased assembly |
| WF6 | Purge Duplicates | Remove haplotypic duplications |
| Deprecated - no longer used | ||
| WF8 | Hi-C Scaffolding | YAHS chromosome scaffolding |
| WF9 | Decontamination | Remove contaminants |
| PreCuration | Pretext Snapshot | Prepare files for manual curation |
| PostCuration | Post Curation | Apply curation, split haplotypes, rename/reorient, QC |
Takes curated AGP + combined haplotype FASTA from pre-curation, applies curation changes, splits by haplotype, assigns chromosome names, renames/reorients hap2 to match hap1, generates QC.
Key inputs:
Haplotype labels in AGP: Hap_1, Hap_2, Z, W, Unloc, Haplotig
Rename/Reorient logic: Chromosomes assigned by size independently per haplotype, then mashmap (seqLength=50000) aligns hap2 to hap1 to detect mismatches → renames + inversions.
Key parameter: mashmap seqLength: 50000 — sequences must be ≥50kb for rename/reorient to be exercised.
Compleasm parameter gotcha: The workflow input "Database for Compleasm Genes" maps to compleasm's lineage_dataset parameter — pass a lineage like vertebrata_odb10, NOT the busco database version (v5). The busco_database: v5 is hardcoded in the tool_state.
Chromosome naming: vgp_chromosome_assignment produces SUPER_N names (SUPER_1, SUPER_2, SUPER_Z, SUPER_W), not chr1/chr2.
Known issue: vgp_chromosome_assignment v1.1 crashes when AGP contains multi-component scaffolds with Unloc pieces (e.g., Scaffold_1 = main + gap + unloc). Workaround: make Unloc scaffolds standalone single-component entries in the AGP.
Tag inheritance pitfall: The curation subworkflow sets #hap1 (via name:hap1 in .ga) on the Hap1 AGP at the "convert to agp" step. This propagates through chromosome assignment → mashmap → sak generation → reoriented hap2, causing Curated Hap2 to inherit #hap1. Fix: add RemoveTagDatasetAction for name:hap1 at key junction points (sak_file output, Curated Hap2 output).
| Workflow | IWC Repo | Latest Version | Dockstore ID |
|---|---|---|---|
| WF1 | kmer-profiling-hifi-VGP1 | v0.6 | github.com/iwc-workflows/kmer-profiling-hifi-VGP1/main |
| WF4 | Assembly-Hifi-HiC-phasing-VGP4 | v0.5 | github.com/iwc-workflows/Assembly-Hifi-HiC-phasing-VGP4/main |
| WF8 | Scaffolding-HiC-VGP8 | v3.3 | github.com/iwc-workflows/Scaffolding-HiC-VGP8/main |
BUSCO -> Compleasm (WF4 v0.5, WF8 v3.3):
0.2.5+galaxy0) replaced BUSCO for gene completeness assessmentHi-C reads format change (WF4 v0.5, WF8 v3.3):
New required inputs across all workflows:
WF4 additional new inputs: Trim Hi-C reads? (boolean), Name for Haplotype 1/2 (defaults: Hap1/Hap2), Bits for bloom filter (default: 37) WF8 additional new inputs: Haplotype (restricted: Haplotype 1/2, Maternal/Paternal, Primary/Alternate), Trim Hi-C Data? (boolean), Minimum Mapping Quality (default: 10)
Check latest versions via Dockstore API:
https://dockstore.org/api/ga4gh/trs/v2/tools/%23workflow%2Fgithub.com%2Fiwc-workflows%2F{REPO}%2Fmain/versions
Check workflow inputs by fetching the .ga file from GitHub:
https://raw.githubusercontent.com/iwc-workflows/{REPO}/main/{WORKFLOW_NAME}.ga
if trajectory == "C" (HiFi only):
WF6 is REQUIRED
WF6 border: solid
else: # Trajectory A or B
WF6 is OPTIONAL
WF6 border: dashed
Can skip directly to WF8
When to skip WF6 (Trajectories A/B):
When to run WF6 (Trajectories A/B):
| Data Type | Minimum Coverage | Notes |
|---|---|---|
| HiFi | 30x | Diploid genome |
| Hi-C | 60x | Diploid genome |
Problem: VGP assemblies often place both sex chromosomes (X+Y or Z+W) in the main haplotype, requiring adjustment to expected chromosome counts.
Solution: When both sex chromosomes present, expected = n + 1 (not n)
Implementation:
# Adjust haploid expected when BOTH sex chromosomes in main haplotype
df['num_chromosomes_haploid_adjusted'] = df['num_chromosomes_haploid'].copy()
both_sex_chr_patterns = [
'Has X and Y',
'Has Z and W',
'has Z and W',
'Has X1, X2, and Y',
'Has Z1, Z2, and W',
'Has 5X and 5Y'
]
if 'Sex chromosomes main haploptype' in df.columns:
has_both_sex = df['Sex chromosomes main haploptype'].isin(both_sex_chr_patterns)
df.loc[has_both_sex & df['num_chromosomes_haploid'].notna(),
'num_chromosomes_haploid_adjusted'] = \
df.loc[has_both_sex & df['num_chromosomes_haploid'].notna(),
'num_chromosomes_haploid'] + 1
Biological Reasoning:
Impact: Improved perfect match rate from 0% to ~90% in validation analyses
Validation Metrics:
# Use adjusted counts for validation
achieved = df['total_number_of_chromosomes']
expected = df['num_chromosomes_haploid_adjusted']
perfect_matches = (achieved == expected).sum()
within_1 = ((achieved - expected).abs() <= 1).sum()
ratio = achieved / expected
Wrong: Compare diploid expected (2n) to haploid assembly
Wrong: Use haploid (n) when both sex chromosomes present
Correct: Use adjusted haploid (n or n+1 depending on sex chromosome configuration)
WF0 runs in parallel with the main pipeline and may fail if:
def check_mitohifi_failure(wf0_result):
"""Distinguish biological vs technical failure"""
if "no_mito_reads" in wf0_result.log:
return "biological" # Expected for some samples
else:
return "technical" # Investigate further
When creating workflow diagrams:
#fff3e0)#e8f5e9)#f3e5f5)#e3f2fd)#e8f5e9)#fce4ec)#e3f2fd): "x2 per haplotype" - runs separately#e8f5e9): "both haplotypes" - runs together#4285f4)#34a853)#ea4335)| Trajectory | Inputs | K-mer | Assembly | Purge | Scaffold | Finish | Output |
|---|---|---|---|---|---|---|---|
| A | HiFi+HiC | WF1 | WF4 | [WF6] | WF8 | WF9->Pre | hap1/hap2 |
| B | HiFi+Trio | WF2 | WF5 | [WF6] | WF8 | WF9->Pre | mat/pat |
| C | HiFi only | WF1 | WF3 | WF6 | - | WF9->Pre | pri/alt |
[WF6] = optional, WF6 = required, - = skipped
GCA_011100685.1 - Frequently used reference genome for RagTag scaffolding in canid genome assemblies.
When documenting scaffolding in methods sections:
For reproducibility: