git-workflow

# Git Workflow Standards Covers both **research project version control** (data analysis, empirical research) and general software development Git standards. --- ## Part 1: Research Project Version Control ### Research Project Directory Structure ``` research-project/ ├── .git/ ├── .gitignore ← critical: what to exclude ├── .dvc/ ← DVC metadata (commit this) ├── README.md ← project overview + run instructions │ ├── data/ │ ├── raw/ ← NEVER commit; track with DVC or git-lfs │ │ ├── census_2020.dta │ │ └── README.md ← document sources, access, download │ ├── processed/ ← derived data; git-ignore or DVC │ └── census_2020.dta.dvc ← DVC pointer file (commit this) │ ├── analysis/ │ ├── 00_setup.do/.py ← environment + package setup │ ├── 01_clean.do/.py ← data cleaning │ ├── 02_main_reg.do/.py ← main specifications │ ├── 03_robustness.do/.py │ ├── 04_figures.do/.py │ └── 05_tables.do/.py │ ├── output/ │ ├── figures/ ← generated; can git-ignore or track │ └── tables/ ← generated LaTeX tables │ ├── paper/ │ ├── main.tex │ ├── refs.bib │ └── appendix.tex │ └── config/ └── paths.do/.py ← project-wide path configuration ``` ### Research Commit Scopes | Scope | What it covers | Example | |-------|---------------|---------| | `data` | Data acquisition, cleaning scripts | `feat(data): add county-level census panel 2000-2020` | | `analysis` | Regression specs, estimation code | `feat(analysis): implement 2×2 DID with state FE` | | `results` | Output files, tables, figures | `update(results): regenerate Table 2 with alt clustering` | | `paper` | LaTeX writing | `feat(paper): draft identification section` | | `config` | Paths, globals, settings | `fix(config): correct output directory paths` | | `repkg` | Replication package preparation | `chore(repkg): add AER replication README` | ### Research Branch Strategy ``` master ← publication-ready / submission version develop ← working integration branch │ ├── analysis/main-spec ← main identification strategy ├── analysis/robustness ← robustness checks & extensions ├── analysis/hetero ← heterogeneity analysis ├── paper/draft ← active writing ├── paper/revision-r1 ← first revision └── data/acquisition ← data collection phase ``` ### Research .gitignore ```gitignore # ── Raw data (large files) ────────────────────────── data/raw/ data/processed/ *.dta # Stata datasets *.sav # SPSS *.xlsx # Excel data # ── Output (derived; regenerate from code) ────────── output/ results/ # ── Stata intermediates ────────────────────────────── *.smcl *.log *.ster *.gph # ── Python ──────────────────────────────────────────── __pycache__/ *.pyc .venv/ *.pkl # large pickles *.parquet # large datasets # ── OS / Editor ─────────────────────────────────────── .DS_Store .idea/ .vscode/ Thumbs.db # ── Sensitive ───────────────────────────────────────── .env credentials.json api_keys.txt # ── LaTeX intermediates ─────────────────────────────── *.aux *.bbl *.blg *.out *.synctex.gz ``` ### Data Version Control: DVC **Install and initialize:** ```bash uv pip install dvc dvc-s3 # or dvc-gdrive, dvc-ssh dvc init # creates .dvc/ in project root git add .dvc .dvcignore git commit -m "chore(config): initialize DVC" ``` **Track large data files:** ```bash # Add file to DVC (creates .dvc pointer file) dvc add data/raw/census_2020.dta # Commit the pointer (not the data itself) git add data/raw/census_2020.dta.dvc data/raw/.gitignore git commit -m "data(raw): add census 2020 panel via DVC" # Configure remote storage dvc remote add -d myremote ~/dvc-storage # local disk # or: dvc remote add -d s3remote s3://mybucket/dvc dvc push # upload data to remote ``` **Sync data across machines:** ```bash dvc pull # download tracked data files ``` **Alternative: git-lfs (simpler, for <2GB files):** ```bash git lfs install git lfs track "*.dta" "*.csv" "*.parquet" git add .gitattributes git commit -m "chore(config): enable git-lfs for data files" git add data/raw/census_2020.dta git commit -m "data(raw): add census 2020 panel" ``` ### Research Commit Workflow (Daily) ```bash # Morning: sync latest git pull origin develop # Work on analysis # Edit: analysis/02_main_reg.py # Stage and commit analysis changes git add analysis/02_main_reg.py git commit -m "feat(analysis): add state×year FE to main DID spec" # If data changed dvc add data/processed/panel_clean.dta git add data/processed/panel_clean.dta.dvc git commit -m "data(processed): update cleaned panel with income vars" dvc push # Update results git add output/tables/table2.tex output/figures/event_study.pdf git commit -m "update(results): regenerate Table 2 and event study plot" ``` ### Tagging Analysis Milestones ```bash # Tag at key milestones (submission, revision, etc.) git tag -a v1.0-submission -m "First submission to APSR (2026-03-14)" git tag -a v1.1-revision-r1 -m "Revision R1 response submitted" git push origin --tags ``` --- ## Commit Message Standards The project follows the **Conventional Commits** specification: ``` <type>(<scope>): <subject> <body> <footer> ``` ### Type Reference | Type | Description | Example | | :--- | :--- | :--- | | `feat` | New feature | `feat(user): add user export functionality` | | `fix` | Bug fix | `fix(login): fix captcha not refreshing` | | `docs` | Documentation update | `docs(api): update API documentation` | | `refactor` | Refactoring | `refactor(utils): refactor utility functions` | | `perf` | Performance improvement | `perf(list): optimize list performance` | | `test` | Test related | `test(user): add unit tests` | | `chore` | Other changes | `chore: update dependency versions` | ### Subject Rules - Start with a verb: add, fix, update, remove, optimize - No more than 50 characters - No period at the end For more detailed conventions and examples, see `references/commit-conventions.md`. ## Branch Management Strategy ### Branch Types | Branch Type | Naming Convention | Description | Lifecycle | | :--- | :--- | :--- | :--- | | master | `master` | Main branch, releasable state | Permanent | | develop | `develop` | Development branch, latest integrated code | Permanent | | feature | `feature/feature-name` | Feature branch | Delete after completion | | bugfix | `bugfix/issue-description` | Bug fix branch | Delete after fix | | hotfix | `hotfix/issue-description` | Emergency fix branch | Delete after fix | | release | `release/version-number` | Release branch | Delete after release | ### Branch Naming Examples ``` feature/user-management # User management feature feature/123-add-export # Issue-linked feature bugfix/login-error # Login error fix hotfix/security-vulnerability # Security vulnerability fix release/v1.0.0 # Version release ``` ### Branch Protection Rules **master branch:** - No direct pushes allowed - Must merge via Pull Request - Must pass CI checks - Requires at least one Code Review approval **develop branch:** - Direct pushes restricted - Pull Request merges recommended - Must pass CI checks For detailed branch strategies and workflows, see `references/branching-strategies.md`. ## Workflows ### Daily Development Workflow ```bash # 1. Sync latest code git checkout develop git pull origin develop # 2. Create feature branch git checkout -b feature/user-management # 3. Develop and commit git add . git commit -m "feat(user): add user list page" # 4. Push to remote git push -u origin feature/user-management # 5. Create Pull Request and request Code Review # 6. Merge to develop (via PR) # 7. Delete feature branch git branch -d feature/user-management git push origin -d feature/user-management ``` ### Hotfix Workflow ```bash # 1. Create fix branch from master git checkout master git pull origin master git checkout -b hotfix/critical-bug # 2. Fix and commit git add . git commit -m "fix(auth): fix authentication bypass vulnerability" # 3. Merge to master git checkout master git merge --no-ff hotfix/critical-bug git tag -a v1.0.1 -m "hotfix: fix authentication bypass vulnerability" git push origin master --tags # 4. Sync to develop git checkout develop git merge --no-ff hotfix/critical-bug git push origin develop ``` ### Release Workflow ```bash # 1. Create release branch git checkout develop git checkout -b release/v1.0.0 # 2. Update version numbers and documentation # 3. Commit version update git add . git commit -m "chore(release): prepare release v1.0.0" # 4. Merge to master git checkout master git merge --no-ff release/v1.0.0 git tag -a v1.0.0 -m "release: v1.0.0 official release" git push origin master --tags # 5. Sync to develop git checkout develop git merge --no-ff release/v1.0.0 git push origin develop ``` ## Merge Strategy ### Merge vs Rebase | Feature | Merge | Rebase | | :--- | :--- | :--- | | History | Preserves complete history | Linear history | | Use case | Public branches | Private branches | | Recommended for | Merging to main branch | Syncing upstream code | ### Recommendations - **Feature branch syncing develop**: Use `rebase` - **Feature branch merging to develop**: Use `merge --no-ff` - **develop merging to master**: Use `merge --no-ff` ```bash # ✅ Recommended: Feature branch syncing develop git checkout feature/user-management git rebase develop # ✅ Recommended: Merge feature branch to develop git checkout develop git merge --no-ff feature/user-management # ❌ Not recommended: Rebase on public branch git checkout develop git rebase feature/xxx # Dangerous operation ``` **Project convention**: Use `--no-ff` when merging feature branches to preserve branch history. For detailed merge strategies and techniques, see `references/merge-strategies.md`. ## Conflict Resolution ### Identifying Conflicts ``` <<<<<<< HEAD // Current branch code const name = 'Alice' ======= // Branch being merged const name = 'Bob' >>>>>>> feature/user-management ``` ### Resolving Conflicts ```bash # 1. View conflicting files git status # 2. Manually edit files to resolve conflicts # 3. Mark as resolved git add <file> # 4. Complete the merge git commit # merge conflict # or git rebase --continue # rebase conflict ``` ### Conflict Resolution Strategies ```bash # Keep current branch version git checkout --ours <file> # Keep incoming branch version git checkout --theirs <file> # Abort merge git merge --abort git rebase --abort ``` ### Preventing Conflicts 1. **Sync code regularly** - Pull latest code before starting work each day 2. **Small commits** - Commit small changes frequently 3. **Modular features** - Implement different features in different files 4. **Communication** - Avoid modifying the same file simultaneously For detailed conflict handling and advanced techniques, see `references/conflict-resolution.md`. ## .gitignore Standards ### Basic Rules ``` # Ignore all .log files *.log # Ignore directories node_modules/ # Ignore directory at root /temp/ # Ignore files in all directories **/.env # Don't ignore specific files !.gitkeep ``` ### Common .gitignore ``` node_modules/ dist/ build/ .idea/ .vscode/ .env .env.local logs/ *.log .DS_Store Thumbs.db ``` For detailed .gitignore patterns and project-specific configurations, see `references/gitignore-guide.md`. ## Tag Management Uses **Semantic Versioning**: ``` MAJOR.MINOR.PATCH[-PRERELEASE] ``` ### Version Change Rules - **MAJOR**: Incompatible API changes (v1.0.0 → v2.0.0) - **MINOR**: Backward-compatible new features (v1.0.0 → v1.1.0) - **PATCH**: Backward-compatible bug fixes (v1.0.0 → v1.0.1) ### Tag Operations ```bash # Create annotated tag (recommended) git tag -a v1.0.0 -m "release: v1.0.0 official release" # Push tags git push origin v1.0.0 git push origin --tags # View tags git tag git show v1.0.0 # Delete tag git tag -d v1.0.0 git push origin :refs/tags/v1.0.0 ``` ## Team Collaboration Standards ### Pull Request Standards PRs should include: ```markdown ## Change Description <!-- Describe the content and purpose of this change --> ## Change Type - [ ] New feature (feat) - [ ] Bug fix (fix) - [ ] Code refactoring (refactor) ## Testing Method <!-- Describe how to test --> ## Related Issue Closes #xxx ## Checklist - [ ] Code has been self-tested - [ ] Documentation has been updated ``` ### Code Review Standards Review focus areas: - **Code quality**: Clear and readable, proper naming, no duplicate code - **Logic correctness**: Business logic correct, edge cases handled - **Security**: No security vulnerabilities, sensitive information protected - **Performance**: No obvious performance issues, resources properly released