ncbi-datasets-api

# NCBI Datasets v2 API ## Overview NCBI Datasets is the modern API for accessing NCBI's genomic, gene, and taxonomic data — replacing older E-utilities for sequence data retrieval. It provides clean REST endpoints for genome assemblies, gene records, taxonomy trees, and sequence downloads. Covers all organisms in NCBI's databases including RefSeq and GenBank. Free, no authentication required. ## API Endpoints ### Base URL ``` https://api.ncbi.nlm.nih.gov/datasets/v2 ``` ### Genome Data ```bash # Search genome assemblies by organism curl "https://api.ncbi.nlm.nih.gov/datasets/v2/genome/taxon/9606?page_size=5" # Get assembly by accession curl "https://api.ncbi.nlm.nih.gov/datasets/v2/genome/accession/GCF_000001405.40" # Download genome package curl -o genome.zip \ "https://api.ncbi.nlm.nih.gov/datasets/v2/genome/accession/GCF_000001405.40/download?\ include_annotation_type=GENOME_FASTA,GENOME_GFF" ``` ### Gene Data ```bash # Search genes by symbol curl "https://api.ncbi.nlm.nih.gov/datasets/v2/gene/symbol/TP53/taxon/human" # Get gene by NCBI Gene ID curl "https://api.ncbi.nlm.nih.gov/datasets/v2/gene/id/7157" # Search genes by keyword curl "https://api.ncbi.nlm.nih.gov/datasets/v2/gene/search?query=BRCA&taxon=9606&page_size=20" # Download gene data package curl -o gene.zip \ "https://api.ncbi.nlm.nih.gov/datasets/v2/gene/id/7157/download?include_annotation_type=FASTA_GENE" ``` ### Taxonomy ```bash # Get taxonomy info curl "https://api.ncbi.nlm.nih.gov/datasets/v2/taxonomy/taxon/9606" # Search taxonomy by name curl "https://api.ncbi.nlm.nih.gov/datasets/v2/taxonomy/name_report?taxon_query=Homo+sapiens" # Get taxonomy tree (subtree) curl "https://api.ncbi.nlm.nih.gov/datasets/v2/taxonomy/taxon/9443/subtree" ``` ### Query Parameters | Parameter | Description | Example | |-----------|-------------|---------| | `page_size` | Results per page | `page_size=20` | | `page_token` | Pagination token | From previous response | | `include_annotation_type` | Download content | `GENOME_FASTA`, `GENOME_GFF`, `PROT_FASTA` | | `filters.assembly_level` | Assembly quality | `complete_genome`, `chromosome` | | `filters.refseq_only` | RefSeq assemblies | `true` | ## Response Structure (Gene) ```json { "genes": [ { "gene": { "gene_id": 7157, "symbol": "TP53", "description": "tumor protein p53", "taxname": "Homo sapiens", "tax_id": 9606, "type": "PROTEIN_CODING", "chromosomes": ["17"], "genomic_ranges": [ { "accession_version": "NC_000017.11", "range": [{"begin": 7668402, "end": 7687550, "orientation": "minus"}] } ], "nomenclature": { "symbol": "TP53", "name": "tumor protein p53" }, "annotations": [ {"release_date": "2024-03-15", "release_name": "GRCh38.p14"} ] } } ] } ``` ## Python Usage ```python import requests import zipfile import io BASE_URL = "https://api.ncbi.nlm.nih.gov/datasets/v2" def search_genes(query: str, taxon: str = "human", page_size: int = 20) -> list: """Search NCBI genes by keyword.""" resp = requests.get( f"{BASE_URL}/gene/search", params={"query": query, "taxon": taxon, "page_size": page_size}, ) resp.raise_for_status() data = resp.json() results = [] for item in data.get("genes", []): gene = item.get("gene", {}) results.append({ "gene_id": gene.get("gene_id"), "symbol": gene.get("symbol"), "description": gene.get("description"), "type": gene.get("type"), "chromosomes": gene.get("chromosomes", []), "taxname": gene.get("taxname"), }) return results def get_gene(gene_id: int) -> dict: """Get detailed gene information.""" resp = requests.get(f"{BASE_URL}/gene/id/{gene_id}") resp.raise_for_status() genes = resp.json().get("genes", []) return genes[0].get("gene", {}) if genes else {} def search_genomes(taxon: str, refseq_only: bool = True, page_size: int = 10) -> list: """Search genome assemblies by organism.""" params = {"page_size": page_size} if refseq_only: params["filters.refseq_only"] = "true" resp = requests.get( f"{BASE_URL}/genome/taxon/{taxon}", params=params, ) resp.raise_for_status() data = resp.json() results = [] for report in data.get("reports", []): assembly = report.get("assembly_info", {}) stats = report.get("assembly_stats", {}) results.append({ "accession": report.get("accession"), "name": assembly.get("assembly_name"), "level": assembly.get("assembly_level"), "organism": report.get("organism", {}).get("organism_name"), "total_length": stats.get("total_sequence_length"), "contig_n50": stats.get("contig_n50"), }) return results # Example: search cancer-related genes genes = search_genes("tumor suppressor", taxon="human") for g in genes[:5]: print(f"{g['symbol']} (ID: {g['gene_id']}): {g['description']}") print(f" Type: {g['type']} | Chr: {', '.join(g['chromosomes'])}") # Example: find reference genomes genomes = search_genomes("Mus musculus", refseq_only=True) for g in genomes[:3]: print(f"{g['accession']}: {g['name']} ({g['level']})") print(f" Length: {g['total_length']:,} bp") ``` ## CLI Tool NCBI also provides a command-line tool: ```bash # Install curl -o datasets "https://ftp.ncbi.nlm.nih.gov/pub/datasets/command-line/v2/linux-amd64/datasets" chmod +x datasets # Download human genome ./datasets download genome taxon "Homo sapiens" --reference --include genome # Download gene data ./datasets download gene gene-id 7157 --include gene ``` ## References - [NCBI Datasets](https://www.ncbi.nlm.nih.gov/datasets/) - [Datasets API Reference](https://www.ncbi.nlm.nih.gov/datasets/docs/v2/reference-docs/rest-api/) - [Datasets CLI](https://www.ncbi.nlm.nih.gov/datasets/docs/v2/download-and-install/)