Menu
DuckDB glob scan patterns, connection management, and performance for kreview
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
Based on SOC occupation classification
Testing conventions, patterns, and best practices for kreview. MANDATORY reading before adding or modifying tests.
nbdev cell directives, module export conventions, editing workflow, and testing patterns for notebook-first development. MANDATORY reading before editing any kreview Python module.
5-tier ctDNA labeling hierarchy with IMPACT tissue rescue, CH hotspot filtering, configurable VAF/variant thresholds, and continuous VAF regression stats.
How to parse cBioPortal MAF, SV, CNA, and clinical files for the labeling pipeline.
Structured debugging framework for kreview data pipeline issues
cfDNA fragmentomics biology — what each feature measures and how to interpret it for ctDNA detection.
| name | duckdb-patterns |
| description | DuckDB glob scan patterns, connection management, and performance for kreview |
| Feature | Suffix | Rows Per Sample |
|---|---|---|
| FSC.gene | .FSC.gene.parquet | ~128 |
| FSD | .FSD.chr_arm.parquet | ~39 |
| FSR | .FSR.parquet | 1 |
| OCF.ontarget | .OCF.ontarget.parquet | ~100 |
| WPS.background | .WPS_background.parquet | ~22 |
| metadata | .metadata.parquet | 1 |
# Single feature, all samples
read_parquet('results/*/*.FSC.gene.parquet', filename=true)
# Multiple features via UNION ALL
SELECT 'FSC' AS feature, * FROM read_parquet('results/*/*.FSC.gene.parquet', filename=true)
UNION ALL
SELECT 'FSD' AS feature, * FROM read_parquet('results/*/*.FSD.chr_arm.parquet', filename=true)
# ✅ Per-function connection (recommended)
def load_data():
conn = get_duckdb_conn() # from kreview.core
return conn.sql("...").df()
# ❌ Global connection (fragile in notebooks)
CONN = duckdb.connect() # dies on kernel restart
.df()SELECT col1, col2 not SELECT *SET threads TO 4 (default is all cores)SET memory_limit = '4GB' for large cohortsWhen DuckLake reaches v1.0, the migration is:
# Before (DuckDB glob):
conn.sql("SELECT * FROM read_parquet('results/*/*.FSC.gene.parquet')")
# After (DuckLake):
conn.execute("ATTACH 'ducklake:catalog.ducklake' AS eval")
conn.sql("SELECT * FROM eval.fsc_gene")
Downstream code is identical — only the data source changes.