foundations-git-version-control

name	foundations-git-version-control
description	Version-control bioinformatics scripts with git init/add/commit/branch/merge/stash/tag and .gitignore for FASTQ/BAM/VCF. Use when setting up a repo, undoing a commit, or resolving a merge conflict.
tool_type	bash
primary_tool	git

Git Version Control for Bioinformatics

When to Use

• Version-controlling analysis scripts, pipelines (Snakemake/Nextflow), and configs
• Collaborating on code across lab members via GitHub/GitLab
• Tracking which exact script version produced which result (for reproducibility and paper submissions)
• Safely experimenting with an alternative method (normalization, threshold, model) via a branch
• Recovering from a bad edit, a bad commit, or a merge conflict

Version Compatibility

• Git ≥ 2.23 (adds git switch / git restore as clearer alternatives to checkout/reset); examples below work on any Git ≥ 2.0 using the classic commands too.
• GitHub/GitLab web UI for remotes, pull requests, and issue linking (Fixes #42).

Prerequisites

• Git installed (git --version); a GitHub/GitLab account for remotes.
• Basic shell familiarity (cd, mkdir, cat).
• No prior Git knowledge required — this covers setup through branches and conflicts.

Setup (one-time per machine)

git config --global user.name "Your Name"
git config --global user.email "[e-mail]"
git config --global init.defaultBranch main
git config --global core.editor "nano"   # or "vim", "code --wait"
git config --list                        # verify

Core Workflow

Goal: track changes to a bioinformatics project (scripts, pipeline configs) without accidentally tracking raw data or results. Approach: init a repo, write .gitignore before adding anything, then use the stage → commit loop for every logical change.

# One-time: create repo + structure
mkdir -p my_project/{data,scripts,results,docs}
cd my_project
git init

# Daily loop
git status                    # what changed?
git diff                      # unstaged changes (working dir vs staging)
git diff --staged             # staged changes (staging vs last commit)
git add scripts/deseq2.R      # stage a specific file (prefer this over `git add .`)
git commit -m "Fix off-by-one in exon boundary parsing"
git log --oneline -10         # compact history
git log --graph --oneline     # visual branch history
git show HEAD                 # full diff of the latest commit

A small helper to bootstrap a new bioinformatics repo consistently:

# setup_bioinfo_repo.sh
# Create a Git repo with the standard bioinformatics layout and .gitignore.
setup_bioinfo_repo() {
    # $1: project name, $2: one-line project description
    local name="$1" desc="${2:-Bioinformatics analysis project}"

    mkdir -p "$name"/{data,scripts,results,docs}
    cd "$name" || return 1
    git init -q

    cat > README.md <<EOF
# ${name}

${desc}

## Layout
- data/    raw + processed data (not tracked)
- scripts/ analysis code and pipeline definitions (tracked)
- results/ generated outputs (not tracked)
- docs/    notes, methods
EOF

    write_bioinfo_gitignore   # see next section

    git add README.md .gitignore
    git commit -q -m "Initialize ${name} with README and .gitignore"
    echo "Repo ready at $(pwd)"
}

Bioinformatics .gitignore

Rule of thumb: track code and configuration; never track data, references, or generated outputs.

# Large data — never track
*.fastq *.fastq.gz *.fq.gz *.bam *.bam.bai *.sam *.cram *.bcf *.vcf *.vcf.gz *.sra
data/raw/

# Reference genomes
*.fa *.fasta *.fa.fai *.dict

# Generated outputs (regenerate from scripts + raw data)
results/ *.log *.tmp *.out

# Python
__pycache__/ *.pyc .ipynb_checkpoints/ *.egg-info/

# R
.Rhistory .RData

# OS / IDE
.DS_Store Thumbs.db .vscode/ .idea/

# write_bioinfo_gitignore
# Write the standard bioinformatics .gitignore to the current directory.
write_bioinfo_gitignore() {
    cat > .gitignore <<'EOF'
*.fastq *.fastq.gz *.fq.gz *.bam *.bam.bai *.sam *.cram *.bcf *.vcf *.vcf.gz *.sra
data/raw/
*.fa *.fasta *.fa.fai *.dict
results/ *.log *.tmp *.out
__pycache__/ *.pyc .ipynb_checkpoints/ *.egg-info/
.Rhistory .RData
.DS_Store Thumbs.db .vscode/ .idea/
EOF
}

Track: scripts, pipeline definitions, configs, README, environment.yml, small sample sheets. Never track: raw data, reference genomes, generated results, anything > 50 MB (GitHub hard-rejects files ≥ 100 MB).

Remotes and Collaboration (GitHub/GitLab)

git remote add origin https://github.com/user/repo.git
git remote -v                       # verify
git push -u origin main             # first push: sets upstream tracking
git push                            # subsequent pushes

git fetch                           # download changes, do not merge
git pull                            # fetch + merge
git pull --rebase                   # fetch + rebase (cleaner linear history)

Daily loop: git pull at start of day → edit → git status/git add/git commit (repeatedly) → git push at end of day.

Pull requests: git checkout -b feature-x → commit → git push -u origin feature-x → open a PR on GitHub from feature-x into main → after review/approval, merge on GitHub → locally git checkout main && git pull.

Undo Operations

Situation	Command	Destructive?
Discard file edits (unstaged)	git restore file.py	Loses edits
Unstage a file	git restore --staged file.py	No
Undo last commit, keep changes staged	git reset --soft HEAD~1	No
Undo last commit, keep changes unstaged	git reset HEAD~1	No
Undo last commit, discard changes	git reset --hard HEAD~1	Yes
Undo an old commit in a shared/pushed repo	git revert <hash>	No (adds a new commit)
Shelve uncommitted work to switch branches	git stash / git stash pop	No

Branches and Merge Conflicts

git checkout -b feature/normalize-rpkm   # create + switch
git switch -c feature/normalize-rpkm     # same, modern syntax

git merge feature/normalize-rpkm         # merge into current branch
git branch -d feature/normalize-rpkm     # delete after merge (only if merged)

Use branches when: testing a different normalization without breaking the working pipeline; multiple lab members work on different analyses simultaneously; fixing a bug while a new feature is half-done.

When a merge conflict occurs, Git marks the file:

<<<<<<< HEAD
alpha = 0.01    # your version
=======
alpha = 0.05    # their version
>>>>>>> feature-branch

Edit the file to keep the correct version, delete the <<<<<<</=======/>>>>>>> markers, then git add <file> and git commit to complete the merge.

Tags (mark a paper/release version)

git tag -a v1.0 -m "Pipeline version used for Smith et al. 2024 paper"
git push --tags
git checkout v1.0     # inspect the exact code that produced a result

Commit Message Style

# Format: <verb> <what> [context]
# Verbs: Add, Fix, Update, Remove, Refactor, Optimize

Fix off-by-one error in exon boundary parsing
Add DESeq2 analysis with batch correction (LRT test)
Update STAR alignment to use 2-pass mode
Remove deprecated RPKM normalization function

Bad: "fix", "update", "stuff", "final version". For significant changes, write a multi-line message (50-char summary, blank line, wrapped body, Fixes #42).

Pitfalls

• Staging vs. committing: git add marks files for the next commit — it does not save your work. Unstaged files are excluded from the commit.
• Never commit large data files: a single BAM committed to a repo permanently bloats it and makes git clone slow; GitHub rejects files ≥ 100 MB outright. Use .gitignore and data tools (DVC, Git LFS) instead.
• git reset --hard is irreversible: unlike most Git operations, it discards working-directory changes with no undo. Use --soft unless you are certain.
• git revert is safe for shared repos; reset --hard on already-pushed commits rewrites history and breaks collaborators' clones.
• .gitignore must be committed to take effect, and it only ignores untracked files — if a data file was already committed, add it to .gitignore then git rm --cached <file> to stop tracking it.
• Commit messages are forever: "Fix bug" is useless six months later. Explain why, not just what.
• git branch -d refuses to delete an unmerged branch (use -D to force) — that safety check exists for a reason; check git log first.

See Also

• foundations-bash-scripting — shell scripting patterns used alongside Git hooks and pipeline glue code
• bio-workflow-management-snakemake-workflows / bio-workflow-management-nextflow-pipelines — versioning pipeline definitions tracked by Git
• bio-reporting-jupyter-reports — pairing notebooks with Git for reproducible analysis records