菜单
Dataset discovery and download toolkit covering HuggingFace, GitHub, Google, and academic paper search. Use when you need to find, inspect, or download ML datasets.
用 Codex 或 Claude 帮你安装 复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。
基于 SOC 职业分类
| name | dataset-search-tools |
| description | Dataset discovery and download toolkit covering HuggingFace, GitHub, Google, and academic paper search. Use when you need to find, inspect, or download ML datasets. |
| license | Complete terms in LICENSE.txt |
Search, inspect, and download datasets from HuggingFace Hub, GitHub, Google, and academic papers. All operations available as CLI scripts under scripts/. Source implementations copied from playground/search_dataset_tools/.
Download target: {workspace}/data_links/
Primary source for ML datasets. HuggingFace search is keyword-exact, not semantic.
Key concepts: dataset_id (org/name), config (subset/language), split (train/validation/test).
| Tool | CLI |
|---|---|
search_datasets | python scripts/hf_search.py search -q "sentiment" -l 10 |
search_datasets (author) | python scripts/hf_search.py search -q "translation" -a Helsinki-NLP |
inspect_dataset | python scripts/hf_search.py inspect --id stanfordnlp/sst2 |
inspect_dataset (config) | python scripts/hf_search.py inspect --id stanfordnlp/sst2 -c default |
get_dataset_configs | python scripts/hf_search.py configs --id glue |
get_dataset_splits | python scripts/hf_search.py splits --id stanfordnlp/sst2 |
get_dataset_sample | python scripts/hf_search.py sample --id stanfordnlp/sst2 -n 5 -s train |
get_dataset_readme | python scripts/hf_search.py readme --id stanfordnlp/sst2 |
download_dataset | python scripts/hf_search.py download --id stanfordnlp/sst2 -o ./data_links/sst2 |
Typical workflow:
python scripts/hf_search.py search -q "question answering" -l 10
python scripts/hf_search.py inspect --id openai/gsm8k
python scripts/hf_search.py sample --id openai/gsm8k -n 3
python scripts/hf_search.py download --id openai/gsm8k -o ./data_links/gsm8k
Good for research datasets, benchmarks, curated collections.
| Tool | CLI |
|---|---|
comprehensive_github_search | python scripts/github_search.py comprehensive -k "benchmark dataset" --search-type all |
search_repositories | python scripts/github_search.py repos -k "nlp dataset" --sort stars |
search_code | python scripts/github_search.py code -k "load_dataset" --language python |
search_issues | python scripts/github_search.py issues -k "dataset release" --state open |
search_pull_requests | python scripts/github_search.py prs -k "add dataset" |
search_users | python scripts/github_search.py users -k "researcher" |
get_repository_readme | python scripts/github_search.py readme --owner awesomedata --repo awesome-public-datasets |
Workflow: search first to get URLs, then parse specific URLs.
| Tool | CLI |
|---|---|
google_search | python scripts/web_search.py search -q "public NLP dataset download" |
web_parse | python scripts/web_search.py parse -u "https://..." |
After finding PDF URLs, use web_parse to read content.
| Tool | CLI |
|---|---|
arxiv_search_by_content | python scripts/scholar_search.py arxiv -q "benchmark dataset" -n 10 |
arxiv_search_by_author | python scripts/scholar_search.py arxiv-author -a "Yann LeCun" -n 5 |
google_scholar_search | python scripts/scholar_search.py scholar -q "sentiment dataset" |
search_dblp_papers | python scripts/scholar_search.py dblp-papers -q "Diffusion Model" -n 5 |
search_dblp_authors | python scripts/scholar_search.py dblp-authors -q "Geoffrey Hinton" -n 5 |
search_dblp_venues | python scripts/scholar_search.py dblp-venues -q "CVPR" -n 5 |
python scripts/validate_dataset.py -p ./data_links/my_dataset
Checks file structure, CSV/JSON/text/parquet quality, missing values, duplicates, basic statistics.
scripts/
├── github_search.py
├── hf_search.py
├── scholar_search.py
├── src
│ ├── search_github.py
│ ├── search_huggingface.py
│ ├── search_scholar.py
│ └── search_web.py
└── web_search.py
Tree-structured memory management for node-based exploration. Manages global memory, data link records, and per-node manifest (TL;DR + recordings). Use when agents need to persist, share, or retrieve exploration knowledge.
Code execution and submission tools for DataNode (Black/Red) agents. Manages DataLoader code, runs assembled scripts, validates and grades submissions.
Data loading, cleaning, external-data merge, and validation-diagnosis guidance for black-node style ML agents. Use when a coding agent must improve data quality or integrate external datasets without changing the core model logic, especially for image tasks but also for tabular and text tasks.
Retrieval-Augmented Generation (RAG) skill for generic semantic search and knowledge retrieval. Use when you need to build or call vector-based search over your own documents using vector indexes (optional FAISS) and embedding models (local transformers or OpenAI embeddings), or when integrating LLMs with external knowledge bases in a project-agnostic way.