القائمة
Build a Hail Table from the Interactome Insider genomic BED (protein-protein interface residues projected to GRCh38) for interval-based variant annotation.
التثبيت باستخدام Codex أو Claude انسخ هذا Prompt والصقه في Codex أو Claude أو مساعد آخر ليراجع صفحة Skill ويثبّتها لك.
استنادا إلى تصنيف SOC المهني
Onboard, build, or update the ClinGen Gene-Disease Validity resource for hvantk
Onboard, build, or update the ClinVar resource for hvantk
Build a Hail Table from GTEx cis-eQTL summary statistics (per-tissue significant variant-gene pairs) for qtlcascade and variant annotation.
Build a Hail Table from the EBI GWAS Catalog v1.0 full-associations TSV for variant-annotation joins.
Build a Hail Table from an MSigDB GMT gene-set file (e.g., C2 Canonical Pathways) for enrichment / burden / overlap analyses.
HGNC gene nomenclature lookup table — authoritative human gene symbols, IDs, and cross-references keyed by hgnc_id.
| name | hvantk:resource-insider |
| description | Build a Hail Table from the Interactome Insider genomic BED (protein-protein interface residues projected to GRCh38) for interval-based variant annotation. |
| status | provisional |
| backend | hail |
| domain | protein |
Read hvantk/skills/_conventions/SKILL.md first. This skill assumes every convention there.
Whole_Human_Interactome_Interface_hg38.bed — the genomic projection product. UCSC-style BED with browser/track metadata; 18.6M data rows across 208,448 named PPI tracks.interval-keyed Hail Table from the BED file for variant-interval intersection (e.g., "does this variant fall in any predicted interface residue?").H_sapiens_interfacesALL.txt product (protein-pair keyed; encodes per-protein interface residue arrays with Source = ECLAIR / PDB / I3D). This is a separately-onboardable skill — same resource, different shape and different builder. Not in this PR._conventions § 11 and the downloader strategy (CLAUDE.md), acquisition is manual.This skill is the first interval-keyed skill in hvantk. Conventions § 3 declares interval keying as valid; this skill anchors it.
INSIDER_v1.0 in hvantk/resources/registry/genomics/datasets.json. Filename and metadata corrected in the same PR that adds this skill — the prior entry listed insider_interaction_sites.tsv which is not a real INSIDER distribution product (see § 4 Gap 2).Stable note (not in catalog): INSIDER releases two complementary products from one source. This skill anchors the genomic BED only. The protein-residue TXT is documented in the catalog as a follow-up; see § 8.
backend: hail, domain: protein. Per _conventions § 3, protein-level annotations key on interval or protein_id. The genomic projection BED is naturally interval-keyed: each row is a genomic span (typically 1-3 bp) corresponding to a codon at a predicted interface residue. The builder produces an interval<locus<rg>>-keyed Table with a ppi_ids: array<str> field per interval, aggregating PPI identifiers across the track name=... directives.
Ingestion path: custom track-aware parser, not hl.import_bed directly. hl.import_bed silently skips track name=... directives, dropping PPI identity (see § 4 historical note on the prior implementation). This builder instead pre-processes the BED line-by-line in Python: it tracks the current PPI from each track name=<P1>_ppi_<P2> directive, writes a 4-column TSV (contig\tstart\tend\tppi_id) to a Hail temp file, then imports via hl.import_table, constructs intervals with hl.locus_interval(contig, start+1, end+1, ...) to match hl.import_bed's 0-based-BED → 1-based-Hail conversion, and aggregates group_by(interval).aggregate(ppi_ids=hl.agg.collect_as_set(ppi_id)). The ppi_ids array is then hl.sorted(hl.array(...)) for deterministic snapshots.
Snapshot utility upgrade. The _snapshot_utils._jsonable_to_hail_python helper did not previously handle hl.tinterval. PR #105 added an interval branch so interval-keyed snapshots round-trip cleanly. Format: {"start": "<contig>:<pos>", "end": "<contig>:<pos>"} — symmetric with locus serialization, half-open [start, end) per the BED convention. See hvantk/tests/_snapshot_utils.py (the interval branch).
UCSC-style BED with browser/track metadata. Three structural lines to be aware of:
browser hide all
track name=A0A0A0MS80_ppi_P56705 description="Interface Residues for A0A0A0MS80_P56705 (SOURCE: Predicted Interface)" visibility=dense itemRgb="On"
chr11 700235 700235 . 0 + 700235 700235 247,176,91
browser directive (line 1) is ignored.track name=<P1>_ppi_<P2> description="..." lines name each PPI. The builder parses these and assigns the parsed name as the row's ppi_id for the subsequent data block, preserving PPI identity (this is the fix to the historical Gap 1; see below).chr, start, end, name, score, strand, thickStart, thickEnd, itemRgb. Only the first three columns are read; column 4 (name) is always . in this file (PPI identity is in the track header instead).start == end (e.g., chr11 700235 700235). The custom parser skips these (matching hl.import_bed(skip_invalid_intervals=True)'s historical behavior). In the test fixture, 21 raw data rows produced 17 valid intervals (4 zero-length skipped); the aggregation then collapses overlapping intervals across PPIs, yielding 17 unique-interval rows here (each with a singleton ppi_ids since the 5-track fixture doesn't include cross-PPI overlaps).group_by(interval).aggregate(ppi_ids=collect_as_set(ppi_id)) produces a length-N array per such position. The fixture doesn't exercise this case (each interval has a singleton array), so the multi-PPI behavior is by inspection only — not covered by the round-trip test.Historical note (Gap 1, fixed in PR #105 via the track-aware parser): prior implementation used hl.import_bed(...).distinct(), which silently dropped track name=... headers and then collapsed overlapping intervals from different PPIs. The output table answered "does any PPI interface touch this position?" but not "which PPI(s)?". The current builder restores that identity via the custom parser described in § 3 (now _parse_insider_bed_to_temp_tsv in hvantk/skills/insider/builder.py).
Gap 2 (fixed in PR #105): wrong filename in the catalog entry. Prior INSIDER_v1.0 catalog entry listed insider_interaction_sites.tsv which does not exist in any INSIDER distribution. The two real products are Whole_Human_Interactome_Interface_hg38.bed (this skill) and H_sapiens_interfacesALL.txt (separate skill). The filename, format, and size_bytes were corrected in PR #105.
Gap 3 (documented, not a bug): >1 GB file → manual acquisition. Acknowledged by the downloader strategy. No skill-side downloader.
AnnotationTable artifact wrapping a lazy hl.Table (materialized to a .ht directory on artifact.save()).[interval] (interval<locus<GRCh38>>).ctx.provenance(schema_id="insider-variants-v1"); persisted as a sidecar .provenance.json.interval: interval<locus<GRCh38>> — half-open [start, end). Constructed via hl.locus_interval(contig, start+1, end+1, ...) to match hl.import_bed's 0-based-BED → 1-based-Hail conversion, so semantic compatibility with prior interval-based variant annotation downstream is preserved.ppi_ids: array<str> — sorted, deduplicated PPI identifiers (<P1>_ppi_<P2> format) from track name=... directives whose data rows cover this interval. Length ≥ 1; arrays of length > 1 indicate the position is shared between multiple PPI interfaces.reference_genome keyword param to build_insider_interactome (default "GRCh38").{"start": "<contig>:<pos>", "end": "<contig>:<pos>"} (the branch in _snapshot_utils added in PR #105).After aggregation, intervals are unique-in-table. Test inlines sample keys.
build_insider_interactome(parsed_input, ctx, *, reference_genome="GRCh38") -> AnnotationTable in hvantk/skills/insider/builder.py. Builds the table inline: calls _parse_insider_bed_to_temp_tsv(str(parsed_input)) (track-aware Python pre-processor), then hl.import_table + hl.locus_interval, then group_by(interval).aggregate(ppi_ids=collect_as_set(ppi_id)) + hl.sorted(hl.array(...)), and wraps the result via AnnotationTable.from_hail(grouped, provenance=ctx.provenance(schema_id="insider-variants-v1")). The only shared helper it imports is cleanup_temp_file from hvantk/core/utils/hail_helpers.py (used on the error path). It does not use create_table_base._parse_insider_bed_to_temp_tsv (private) lives in hvantk/skills/insider/builder.py (only consumer is this plugin). Reads the BED, tracks current_ppi_id from track name=... headers, skips zero-length and malformed rows, writes a 4-column TSV to hl.utils.new_temp_file(extension="tsv").hvantk/skills/insider/plugin.yaml declares dataset variants with builder.function: build_insider_interactome. The plugin loader (hvantk/core/plugin/loader.py) auto-resolves the dataset under compound key insider:variants via get_registry().get_dataset("insider:variants"); the top-level build runs through run_builder_for_spec (hvantk/core/plugin/run_builder.py). There is no TABLE_BUILDERS registry or registry.py.hvantk reprocess insider:variants --raw-dir <dir> --output <out> [--plugin-arg reference_genome=GRCh38]. reprocess passes the --raw-dir directory to the builder as parsed_input (insider has no lifecycle.parse); _parse_insider_bed_to_temp_tsv consumes it as the BED path.hvantk/tests/_snapshot_utils.py — hl.tinterval handlers added in PR #105.lifecycle.download entry in plugin.yaml).--raw-dir at the directory containing the BED):
hvantk reprocess insider:variants \
--raw-dir /path/to/insider_raw_dir \
--output /path/to/insider.ht \
--plugin-arg reference_genome=GRCh38
build_insider_interactome):
_parse_insider_bed_to_temp_tsv(input_path) — Python-side BED iterator: tracks current PPI from track name=..., skips browser lines, skips zero-length and malformed rows, writes contig\tstart\tend\tppi_id to a Hail temp TSV.hl.import_table(tmp_tsv, types={"start": tint32, "end": tint32}) → annotate with hl.locus_interval(contig, start+1, end+1, reference_genome=...). The +1 matches hl.import_bed's 0-based-BED → 1-based-Hail conversion so intervals are consistent with prior hl.import_bed-derived outputs.group_by(interval).aggregate(ppi_ids=hl.agg.collect_as_set(ppi_id)) → grouped.annotate(ppi_ids=hl.sorted(hl.array(ppi_ids))) — collapse overlapping intervals while preserving the union of PPI IDs, sorted for deterministic output.AnnotationTable.from_hail(grouped, provenance=ctx.provenance(schema_id="insider-variants-v1")) wraps the lazy Table; checkpointing happens on artifact.save().interval.contains(variant.locus) or equivalent interval-overlap operator (hl.is_defined(insider_ht[variant.locus]) after appropriate keying). Downstream consumers can filter / explode on ppi_ids to attribute hits to specific PPIs.INSIDER updates are irregular. To onboard a new release:
path / size_bytes / last_updated in the catalog entry; bump accession if the release version changes (INSIDER_v1.0 → INSIDER_v1.x).track name=... directives in the established <P1>_ppi_<P2> shape), no builder change. If a release changes the track naming pattern (e.g., adds a third underscore-separated field), the _TRACK_NAME_RE regex in hvantk/skills/insider/builder.py and the <P1>_ppi_<P2> convention in the catalog description need updating..txt product as a sibling skill. It carries Source provenance (ECLAIR / PDB / I3D) which the BED does not; downstream filtering on confidence level requires the TXT. Builder would need range-notation parsing for *_IRES arrays (e.g., [1-11,13-14,...]).Per _conventions § 9:
hvantk/skills/insider/tests/testdata/raw/insider/insider_sample.bed. 5 PPI tracks (~21 raw data rows; 17 valid after zero-length filtering). ~1.9 KB. Sliced from the full BED by a track-aware sub-sampler (keeps the browser directive plus the first N track blocks, each header paired with its data rows) — a plain head -N would split a track block and produce an invalid BED.hvantk/skills/insider/tests/snapshots/schema.json. Records the {interval, ppi_ids: array<str>} shape.hvantk/skills/insider/tests/snapshots/sample_rows.json. Intervals are unique-in-table after the aggregation; test inlines 3 sample keys (per _conventions § 9 post-#101 rule — unique-key skills inline).pytest hvantk/skills/insider/tests -m hail.Round-trip test (hvantk/skills/insider/tests/test_builder.py, via phase_b_snapshot_adapter(build_insider_interactome, "insider:variants")) asserts: checkpointed schema matches schema.json; deterministic sample-row slice matches sample_rows.json. The test exercises the hl.tinterval handling in _snapshot_utils (added in PR #105) — if that branch breaks, this test breaks.
Regenerate via --regenerate-snapshots when:
source: array<str> derived from track description="..." Source values).