Extract, process, and deduplicate reads using Unique Molecular Identifiers (UMIs) with umi_tools. Use when library prep includes UMIs and accurate molecule counting is needed, such as in single-cell RNA-seq, low-input RNA-seq, or targeted sequencing to distinguish PCR from biological duplicates.
Extract, process, and deduplicate reads using Unique Molecular Identifiers (UMIs) with umi_tools. Use when library prep includes UMIs and accurate molecule counting is needed, such as in single-cell RNA-seq, low-input RNA-seq, or targeted sequencing to distinguish PCR from biological duplicates.
Before using code patterns, verify installed versions match. If versions differ:
Python: pip show <package> then help(module.function) to check signatures
CLI: <tool> --version then <tool> --help to confirm flags
If code throws ImportError, AttributeError, or TypeError, introspect the installed
package and adapt the example to match the actual API rather than retrying.
UMI Processing
"Deduplicate reads using UMIs" → Extract UMI barcodes, group reads by UMI+position, and collapse PCR duplicates to count unique molecules.
UMIs (Unique Molecular Identifiers) are short random sequences added during library preparation to tag individual molecules before PCR amplification. This enables accurate PCR duplicate removal and molecule counting.
UMI Workflow Overview
Raw FASTQ with UMIs
|
v
[umi_tools extract] --> Move UMI to read header
|
v
[Alignment] --> bwa/STAR/bowtie2
|
v
[umi_tools dedup] --> Remove PCR duplicates based on UMI + position
|
v
Deduplicated BAM
Extract UMIs from Reads
UMI in Read Sequence
# UMI at start of R1 (8bp UMI)
umi_tools extract \
--stdin=R1.fastq.gz \
--read2-in=R2.fastq.gz \
--stdout=R1_extracted.fastq.gz \
--read2-out=R2_extracted.fastq.gz \
--bc-pattern=NNNNNNNN
# UMI at start of R2
umi_tools extract \
--stdin=R1.fastq.gz \
--read2-in=R2.fastq.gz \
--stdout=R1_extracted.fastq.gz \
--read2-out=R2_extracted.fastq.gz \
--bc-pattern2=NNNNNNNN
# UMI in both reads
umi_tools extract \
--stdin=R1.fastq.gz \
--read2-in=R2.fastq.gz \
--stdout=R1_extracted.fastq.gz \
--read2-out=R2_extracted.fastq.gz \
--bc-pattern=NNNNNNNN \
--bc-pattern2=NNNNNNNN
# Must be sorted and indexed first
samtools sort -o aligned_sorted.bam aligned.bam
samtools index aligned_sorted.bam
# Deduplicate
umi_tools dedup \
--stdin=aligned_sorted.bam \
--stdout=deduplicated.bam \
--log=dedup.log
# Add UMI group tag to BAM (BX tag)
umi_tools group \
-I aligned_sorted.bam \
-S grouped.bam \
--group-out=groups.tsv \
--output-bam
Complete Workflows
Standard RNA-seq with UMIs
Goal: Process UMI-tagged RNA-seq reads from raw FASTQ through alignment to a deduplicated BAM file.
Approach: Extract UMIs from read headers with umi_tools, align with STAR, then deduplicate based on UMI + mapping position to produce a PCR-artifact-free BAM.
#!/bin/bashset -euo pipefail
SAMPLE=$1
REFERENCE=$2# 1. Extract UMIs (8bp at start of R1)
umi_tools extract \
--stdin=${SAMPLE}_R1.fastq.gz \
--read2-in=${SAMPLE}_R2.fastq.gz \
--stdout=${SAMPLE}_R1_umi.fastq.gz \
--read2-out=${SAMPLE}_R2_umi.fastq.gz \
--bc-pattern=NNNNNNNN
# 2. Align with STAR
STAR --runThreadN 8 \
--genomeDir $REFERENCE \
--readFilesIn ${SAMPLE}_R1_umi.fastq.gz ${SAMPLE}_R2_umi.fastq.gz \
--readFilesCommand zcat \
--outFileNamePrefix ${SAMPLE}_ \
--outSAMtype BAM SortedByCoordinate
# 3. Index
samtools index ${SAMPLE}_Aligned.sortedByCoord.out.bam
# 4. Deduplicate
umi_tools dedup \
-I ${SAMPLE}_Aligned.sortedByCoord.out.bam \
-S ${SAMPLE}_deduplicated.bam \
--output-stats=${SAMPLE}_dedup_stats \
--paired
# 5. Index deduplicated BAM
samtools index ${SAMPLE}_deduplicated.bam
echo"Done: ${SAMPLE}_deduplicated.bam"
Single-Cell Workflow (Post-CellRanger)
# CellRanger output has CB (cell barcode) and UB (UMI) tags# Deduplicate per cell per gene
umi_tools dedup \
-I possorted_genome_bam.bam \
-S deduplicated.bam \
--per-cell \
--cell-tag=CB \
--umi-tag=UB \
--extract-umi-method=tag \
--per-gene \
--gene-tag=GX
