nemo-curator

GPU-accelerated data curation for LLM training. Supports text/image/video/audio. Features fuzzy deduplication (16× faster), quality filtering (30+ heuristics), semantic deduplication, PII redaction, NSFW detection. Scales across GPUs with RAPIDS. Use for preparing high-quality training datasets, cleaning web data, or deduplicating large corpora.

Jump to install Review files

Repository: synthetic-sciences/openscience
Last source activity: September 16, 2026 at 06:26
Detected SKILL.md language: English
Stars: 4,370
Forks: 550

Install options

The review-first prompt is selected by default. You can switch to a direct command or download a local copy.

Install with an AI assistant

Copy this prompt into the AI assistant you're using.

Follow https://skillsmp.com/skill-install/prompt.md to install only the Agent Skill at https://github.com/synthetic-sciences/openscience/tree/main/backend/cli/skills/ml-training/nemo-curator.

A direct command skips the review prompt. Inspect the source before running it.

npx skills add https://github.com/synthetic-sciences/openscience --skill nemo-curator

The command stays on one line. Scroll horizontally to inspect it before copying.

Prefer a local copy? Download the files currently available to SkillsMP.

Review the source files

Read SKILL.md and any companion files shown by SkillsMP before deciding whether to install.

File Explorer

3 files

SKILL.md

Source instructions · Read-only preview

name	nemo-curator
description	GPU-accelerated data curation for LLM training. Supports text/image/video/audio. Features fuzzy deduplication (16× faster), quality filtering (30+ heuristics), semantic deduplication, PII redaction, NSFW detection. Scales across GPUs with RAPIDS. Use for preparing high-quality training datasets, cleaning web data, or deduplicating large corpora.

Back to search Back to installation

neuropixels-analysis

Neuropixels neural recording analysis. Load SpikeGLX/OpenEphys data, preprocess, motion correction, Kilosort4 spike sorting, quality metrics, Allen/IBL curation, AI-assisted visual analysis, for Neuropixels 1.0/2.0 extracellular electrophysiology. Use when working with neural recordings, spike sorting, extracellular electrophysiology, or when the user mentions Neuropixels, SpikeGLX, Open Ephys, Kilosort, quality metrics, or unit curation.

2026-10-08

plotly

Interactive visualization library. Use when you need hover info, zoom, pan, or web-embeddable charts. Best for dashboards, exploratory analysis, and presentations. For static publication figures use matplotlib or scientific-visualization.

2026-10-08

cancer-genomics-analysis

Computational cancer genomics workflows. Somatic mutation detection and annotation, structural variation characterization, copy number analysis, tumor purity/ploidy estimation, NMF metagene extraction, and DNA damage response network analysis. For cancer mutation databases use cosmic-database; for variant clinical significance use clinvar-database.

2026-10-05

View all from this repository

nemo-curator

Source facts

Install options

Install with an AI assistant

Review the source files

More from this repository

Install with an AI assistant

More from this repository