core-internals

name	core-internals
description	Provides guidance and best practices on concurrent synchronization, IPC, filesystem atomicity, git subprocess wrapping, XML manifests, and CLI argument parsing for git-repo.

Git Repo Core Internals Engineering Guide

Executive Summary

Welcome to the authoritative engineering guide for Git Repository Management within the git-repo codebase. This repository of folk knowledge exists to safeguard the intricate orchestration of our multi-repository synchronization toolchain. Historically, complex operations such as parallelized network fetches, layered manifest overrides, and interactive terminal rendering have been vulnerable to subtle concurrency races, deadlocks, and fragile system states. This guide captures these critical failure modes—ranging from IPC serialization bottlenecks and filesystem lock contentions to non-hermetic test pollution—and establishes rigid engineering constraints to prevent their regression.

To maintain system stability, this guide enforces strict architectural boundaries across the tooling ecosystem. It mandates stateless execution for multiprocessing pools, guaranteed atomicity for worktree layout modifications, and deterministic error translation for all standard Git subprocesses. Furthermore, it defines the standard operating procedures for canonical manifest object deduplication, the modernization of our hermetic testing frameworks, and the delivery of consistent, machine-readable CLI interfaces. By adhering to these paradigms, incoming engineers will ensure the reliability, performance, and extensibility of git-repo's core repository management infrastructure.

Summary

Chapter Theme / Title	Scope & Objective
Concurrent Synchronization & IPC	This domain governs the stable
: : orchestration of parallel network :
: : fetches, local checkouts, and :
: : interleaved subprocess routines. It :
: : strictly enforces safe :
: : multiprocessing IPC, deterministic :
: : Git locking via exponential backoff, :
: : and state synchronization across :
: : concurrent pool workers. :
**Filesystem Atomicity & Worktree	This domain governs the deterministic
: Layout** : creation, migration, and cleanup of :
: : internal repository structures and :
: : Git worktrees. It relies on ephemeral :
: : temporary directories, atomic rename :
: : operations, and robust error recovery :
: : to prevent corrupted states during :
: : unexpected interruptions. :
**Subprocess Git Integration & Error	This chapter defines the constraints
: Translation** : for wrapping, executing, and :
: : translating standard Git subprocesses :
: : within the Repo tooling ecosystem. It :
: : mandates the use of centralized :
: : command abstractions, strict version :
: : gating, and deterministic stream :
: : handling to guarantee reliable :
: : repository state management and :
: : actionable error reporting. :
**Manifest Object Model &	This chapter governs the parsing,
: Deduplication** : validation, and canonicalization of :
: : XML manifest components. It strictly :
: : enforces semantic immutability via :
: : NamedTuple implementations, defensive :
: : copying for hierarchical override :
: : scoping, and deterministic :
: : JSON-backed file tracking across the :
: : subsystem. :
**Hermetic Testing & Test	This domain governs the migration of
: Modernization** : legacy unittest suites to modern :
: : pytest functional paradigms and the :
: : establishment of hermetic session :
: : fixtures. It enforces strict :
: : environment isolation to prevent :
: : global state pollution (e.g., :
: : developer .gitconfig bleeding) while :
: : safely intercepting standard streams :
: : and filesystem paths. :
**CLI Argument Parsing & UX	This chapter governs the lifecycle,
: Consistency** : validation, and execution of :
: : command-line arguments, enforcing :
: : strict standardization for :
: : machine-readable serialization, :
: : unified logging, and deterministic, :
: : thread-safe terminal interactions. :
Repo Hooks Framework	The Repo Hooks Framework governs the
: : execution, parameter validation, and :
: : lifecycle management of user-defined :
: : scripts within the repository :
: : ecosystem. It ensures seamless :
: : integration of extensions like :
: : post-sync or pre-upload while :
: : strictly isolating their execution :
: : failures from core operational :
: : workflows. :

---

---

Chapter: Concurrent Synchronization & IPC

Context: This domain governs the stable orchestration of parallel network fetches, local checkouts, and interleaved subprocess routines. It strictly enforces safe multiprocessing IPC, deterministic Git locking via exponential backoff, and state synchronization across concurrent pool workers.

Summary

Rule ID	Principle / Constraint	Priority	Primary Symptom / Trap
T1-01	Stateless Class Methods	High	Passing bound instance
: : for Parallel Pool Workers : : methods to a parallel :
: : : : executor, dragging :
: : : : unnecessary state into :
: : : : the multiprocessing :
: : : : serialization pipeline. :
T1-02	Buffered Serialization of	High	Executing terminal print
: : Concurrent Standard : : calls directly from :
: : Output : : within a parallelized :
: : : : worker routine. :
T1-03	Interleaved Sync Path	High	Dispatching parallel
: : Validation : : checkout jobs across all :
: : : : project variants without :
: : : : verifying layout types. :
T1-04	Guaranteed	Critical	Placing save operations
: : Synchronization State : : at the very end of an :
: : Persistence : : execution path without :
: : : : exception guards. :
T1-05	Jittered Exponential	Critical	Firing `git submodule
: : Backoff for Git : : init` concurrently :
: : Configuration Locks : : without retry logic for :
: : : : config.lock failures. :
T1-06	Exponential Backoff for	High	Failing a git command
: : Concurrent Git Config : : immediately without :
: : Locks : : evaluating standard :
: : : : streams for transient :
: : : : lock errors. :
T1-07	Hard Termination on	Critical	Logging an error state
: : Interleaved Sync Stalls : : inside a while loop but :
: : : : allowing the next loop :
: : : : iteration to execute. :
T1-08	Resource Management via	High	Initializing proxy
: : Context Handlers for IPC : : connections or :
: : : : multiprocessing managers :
: : : : as raw variable :
: : : : assignments without a :
: : : : guaranteed teardown :
: : : : phase. :
T1-09	Dual-Channel Error	High	Iterating over aggregated
: : Handling in Parallel : : data lists to decide if a :
: : Processing : : parallel orchestration :
: : : : should abort. :
T1-10	Minimum Process Pool Job	Critical	Directly using
: : Count Safeguard : : min(target, len(items)) :
: : : : to determine pool size, :
: : : : which breaks if the item :
: : : : list is unexpectedly :
: : : : empty. :
T1-11	Safeguarding Object State	Critical	Truncating object
: : Across Parallel Execution : : payloads to scalar :
: : Boundaries : : indices during IPC :
: : : : context setup without :
: : : : updating the receiver :
: : : : logic to rehydrate the :
: : : : objects. :
T1-12	Explicit Context	High	Assuming child workers
: : Initialization for : : inherit updated class :
: : Multiprocessing Pools : : variables inherently :
: : : : without explicit :
: : : : initialization. :
T1-13	Dynamic Task Chunk Sizing	Medium	Passing a hardcoded batch
: : in Parallel Execution : : integer to the :
: : : : chunksize parameter. :
T1-14	Deferred Worktree	High	Returning early from the
: : Operations in Sync Local : : sync phase without :
: : Half : : applying required file :
: : : : operations. :

---

Rules

T1-01: Stateless Class Methods for Parallel Pool Workers

Rule: Always implement multiprocessing pool targets as stateless class methods or standalone functions to bypass process serialization constraints.

What: Worker execution targets within multiprocessing pools must be constructed as fully decoupled class methods or standalone functions to bypass process serialization constraints.

Applies To: Multiprocessing process pools, concurrent task scheduling, and ExecuteInParallel implementations.

Why: Command execution objects were heavily bound to system state. Attempting to pass instance methods (self.method) to a multiprocessing pool frequently caused PicklingError crashes, as the underlying Python serialization mechanism cannot cleanly isolate bound object graphs. Failing to adhere to this typically results in IPC Serialization Error.

Trap 1: Passing bound instance methods to a parallel executor, dragging unnecessary state into the multiprocessing serialization pipeline.

Don't:

class InfoCommand:
    def _worker_logic(self, project):
        pass

    def run(self):
        self.ExecuteInParallel(jobs, self._worker_logic, projects)

Do:

class InfoCommand:
    @classmethod
    def _worker_logic(cls, project_idx):
        project = cls.get_parallel_context()["projects"][project_idx]
        pass

    def run(self):
        self.ExecuteInParallel(jobs, self._worker_logic, range(len(projects)))

Exceptions: Threading-based execution models where memory is shared and pickling is not strictly enforced.

---

T1-02: Buffered Serialization of Concurrent Standard Output

Rule: Must capture standard output generated by concurrently executing tasks into isolated memory buffers for sequential display in the parent process.

What: Data emitted by concurrently executing tasks must be captured into isolated memory buffers and returned to the parent process for sequential display.

Applies To: Parallel process execution layers generating human-readable CLI output.

Why: When command execution was parallelized, worker processes wrote directly to standard output. This created severe race conditions resulting in garbled, interleaved text output on the user's terminal. Failing to adhere to this typically results in Interleaved Terminal Output.

Trap 1: Executing terminal print calls directly from within a parallelized worker routine.

Don't:

@classmethod
def _DiffHelper(cls, project):
    print(f"Project: {project.name}")
    print(f"Revision: {project.rev}")

Do:

@classmethod
def _DiffHelper(cls, project):
    buf = io.StringIO()
    buf.write(f"Project: {project.name}\n")
    buf.write(f"Revision: {project.rev}\n")
    return buf.getvalue()

# In main process:
for output in results:
    print(output, end="")

---

T1-03: Interleaved Sync Path Validation

Rule: Always validate the presence of a worktree when executing interleaved sync processes to prevent layout evaluation crashes.

What: When executing interleaved sync processes (parallelized network fetches and checkouts), the operation must explicitly validate the presence of a worktree, as not all Git repository types (e.g., mirrors) maintain local file checkouts.

Applies To: Concurrency logic within sync.py.

Why: Changing the default mode of repo sync to interleaved parallelized checkout tasks indiscriminately. This immediately broke AOSP mirror syncing because mirrors lack local checkout paths, resulting in a TypeError when evaluating NoneType paths. Failing to adhere to this typically results in TypeError / Sync Failure.

Trap 1: Dispatching parallel checkout jobs across all project variants without verifying layout types.

Don't:

• Assuming project.worktree always contains an os.PathLike object during interleaved syncing.

Do:

• Adding guard clauses to verify project.worktree is not None and handling --mirror modes explicitly before adding checkout tasks to the thread pool.

---

T1-04: Guaranteed Synchronization State Persistence

Rule: Must execute synchronization telemetry saving operations within try...finally blocks to guarantee data retention across execution interrupts.

What: Core sync operations must wrap network and filesystem operations in try...finally blocks, ensuring that synchronization metadata (_fetch_times and _local_sync_state) is persisted even if the sync operation fails or is interrupted.

Applies To: subcmds/sync.py, particularly the _Fetch logic and multiprocessing worker loops.

Why: If a synchronization process encountered an error (like a fetch failure), the operation exited early, dropping valuable telemetry and optimization data (fetch times) for projects that successfully synced prior to the crash. Failing to adhere to this typically results in Telemetry Loss / Sync State Inconsistency.

Trap 1: Placing save operations at the very end of an execution path without exception guards.

Don't:

result = self._Fetch(to_fetch, opt, err_event)
if not result.success:
    raise SyncError("failed")

# BAD: Never reached if an error is raised
self._fetch_times.Save()

Do:

try:
    result = self._Fetch(to_fetch, opt, err_event)
    if not result.success:
        raise SyncError("failed")
finally:
    # GOOD: Always saves telemetry state
    self._fetch_times.Save()

---

T1-05: Jittered Exponential Backoff for Git Configuration Locks

Rule: Always wrap concurrent Git mutations to shared configurations in an exponential backoff routine with jitter to mitigate transient filesystem locks.

What: When initializing submodules concurrently (e.g., git submodule init), the system must employ an exponential backoff mechanism with randomized jitter to handle transient filesystem lock errors on .git/config.

Applies To: Submodule initialization and any highly parallel Git subprocesses mutating shared configuration state.

Why: Running repo sync -j<N> caused multiple child processes to simultaneously attempt git submodule init, which led to lock contention on the parent project's config file and caused the entire sync operation to fail. Failing to adhere to this typically results in File Lock Contention / Sync Failure.

Trap 1: Firing git submodule init concurrently without retry logic for config.lock failures.

Don't:

# BAD: Fails immediately if .git/config.lock exists
subprocess.run(["git", "submodule", "init", "--", path], check=True)

Do:

# GOOD: Catch lock errors and retry with jitter
for attempt in range(MAX_RETRIES):
    p = subprocess.run(["git", "submodule", "init", "--", path], stderr=subprocess.PIPE)
    if p.returncode == 0:
        break
    if "could not lock config file" in p.stderr:
        time.sleep(base_delay * (2 ** attempt) + random.uniform(0, jitter))

Exceptions: Non-lock related Git errors should still fail immediately without retrying.

---

T1-06: Exponential Backoff for Concurrent Git Config Locks

Rule: Must analyze captured Git error streams to detect config lock contention and orchestrate retries via structured logging instead of immediate process failure.

What: Concurrent git operations modifying .git/config (such as submodule initialization) must utilize an exponential retry mechanism with jitter to handle transient filesystem lock contention. Standard output and error streams must be captured and parsed to detect these locks.

Applies To: project.py, specifically concurrent git submodule init operations or any parallel processes altering local repository configuration.

Why: When synchronizing with high job values (-j), multiple parallel processes modifying .git/config triggered race conditions, throwing 'could not lock config file' errors and aborting the sync. Failing to adhere to this typically results in Transient Sync Failure / Lock Contention.

Trap 1: Failing a git command immediately without evaluating standard streams for transient lock errors.

Don't:

# BAD: Fails immediately on lock contention
if GitCommand(self, cmd).Wait() != 0:
    raise GitError(f"{self.name} submodule init failed")

Do:

# GOOD: Captures stdout/stderr and retries on 'lock' matches
git_cmd = GitCommand(None, cmd, capture_stdout=True, capture_stderr=True)
if git_cmd.Wait() != 0:
    error = git_cmd.stderr or git_cmd.stdout
    if "lock" in error:
        # apply exponential backoff + jitter logic
    else:
        git_cmd.VerifyCommand()  # Propagate real error

Trap 2: Using raw print statements to log retry attempts, which breaks structured output formats.

Don't:

# BAD: Unstructured print to stdout
print(f"Attempt {attempt+1}: git {' '.join(cmd)} failed. Sleeping...")

Do:

# GOOD: Forwarded to centralized logger
logger.warning("Attempt %d/%d: git %s failed. Error: %s. Sleeping %.2fs before retrying.", attempt+1, max_retries, cmd, error, delay)

Exceptions: Non-lock related Git errors should break the retry loop and be propagated immediately via git_cmd.VerifyCommand().

---

T1-07: Hard Termination on Interleaved Sync Stalls

Rule: Must trigger a definitive execution break and flag error signals whenever an unresolvable stall is detected in an interleaved control loop.

What: When an unresolvable stall is detected within an interleaved or parallel execution loop, the system must trigger a definitive state break (e.g., setting global events and breaking the loop) to prevent infinite deadlocks.

Applies To: subcmds/sync.py, specifically _SyncInterleaved or any looping logic monitoring pending sets.

Why: A bug existed where a stall was logged but the loop terminator (break) and error trigger (err_event.set()) were accidentally omitted. This allowed the sync routine to lock up infinitely when interdependent projects failed to checkout. Failing to adhere to this typically results in Infinite Loop / Deadlock.

Trap 1: Logging an error state inside a while loop but allowing the next loop iteration to execute.

Don't:

# BAD: Omitted termination control flow
if prev_pending == pending_relpaths:
    logger.error("Stall detected")
prev_pending = pending_relpaths

Do:

# GOOD: Set event signals and immediately break
if prev_pending == pending_relpaths:
    logger.error("Stall detected")
    err_event.set()
    break

---

T1-08: Resource Management via Context Handlers for IPC

Rule: Always initialize multiprocessing components and background proxies within with statement context blocks to assure teardown semantics.

What: Multiprocessing managers and SSH proxies initialized for parallel workflows must be wrapped in with block context managers to guarantee cleanup of background processes and multiplexing sockets.

Applies To: Concurrent network operations and parallelized repository synchronization (subcmds/sync.py).

Why: When adding an interleaved fetch/checkout feature, parallel operations required shared synchronization dictionaries and SSH connection multiplexing. Without context managers, exceptions or interrupts (e.g., KeyboardInterrupt) would leave orphaned background processes and abandoned sockets. Failing to adhere to this typically results in Resource Leaks / Orphaned Sockets.

Trap 1: Initializing proxy connections or multiprocessing managers as raw variable assignments without a guaranteed teardown phase.

Don't:

# BAD: No guaranteed cleanup on exception
manager = multiprocessing.Manager()
ssh_proxy = ssh.ProxyManager(manager)
process_tasks(ssh_proxy)

Do:

# GOOD: Context managers assure RAII semantics
with multiprocessing.Manager() as manager:
    with ssh.ProxyManager(manager) as ssh_proxy:
        process_tasks(ssh_proxy)

---

T1-09: Dual-Channel Error Handling in Parallel Processing

Rule: Must decouple fast-fail signaling events from aggregated batch data processing logic to ensure immediate worker termination across processes.

What: In concurrent execution environments, inter-process signaling (for immediate control flow) must be strictly separated from data aggregation (for end-of-batch error reporting).

Applies To: Multi-process synchronization pools (_SyncInterleaved, _SyncPhased) and worker callback processors.

Why: Reviewers questioned why a separate event flag was used when an array of errors was already being populated. Using only an aggregated list prevented workers from rapidly terminating each other via fail-fast mechanisms. Failing to adhere to this typically results in Delayed Worker Termination.

Trap 1: Iterating over aggregated data lists to decide if a parallel orchestration should abort.

Don't:

# BAD: Waiting to process the whole batch before aborting
errors.append(result.error)
if errors and opt.fail_fast:
    pool.close()

Do:

# GOOD: Using an IPC Event for immediate cross-process signaling
err_event.set()
errors.append(result.error)
if not ret and opt.fail_fast:
    pool.close()

---

T1-10: Minimum Process Pool Job Count Safeguard

Rule: Must bind dynamically computed multiprocessing job allocations to a minimum floor of 1 to prevent initialization crashes.

What: When determining the number of worker processes for parallel execution based on a variable target length (e.g., an array of projects), the resulting worker count must be explicitly bound to a minimum of 1.

Applies To: Network fetch logic and multiprocessing pool initializations (e.g., _Fetch in sync.py).

Why: A regression occurred where passing an empty project list evaluated the worker count to 0. This crashed the multiprocessing pool initialization with a ValueError. Returning early was rejected as a fix because secondary side-effects (like state saves) within the function still needed to execute. Failing to adhere to this typically results in ValueError / Process Pool Crash.

Trap 1: Directly using min(target, len(items)) to determine pool size, which breaks if the item list is unexpectedly empty.

Don't:

jobs = min(opt.jobs_network, len(projects_list))

Do:

jobs = max(1, min(opt.jobs_network, len(projects_list)))

---

T1-11: Safeguarding Object State Across Parallel Execution Boundaries

Rule: Always transmit deeply serialized objects directly across parallel boundaries, or strictly reconstruct object relationships within the downstream worker.

What: When offloading command execution to parallel processes, workers must receive properly serialized complex object data, not dissociated identifiers, unless the downstream API explicitly reconstructs the object state.

Applies To: Parallel execution contexts (ParallelContext, ExecuteInParallel) within multi-project commands like repo forall and repo upload.

Why: A refactoring attempt to optimize IPC by passing integer project indices instead of full project objects to parallel workers caused critical breakage. Worker processes subsequently attempted to access attributes (like .manifest) on integer types, leading to unhandled exceptions. Failing to adhere to this typically results in AttributeError / Early Exit Regression.

Trap 1: Truncating object payloads to scalar indices during IPC context setup without updating the receiver logic to rehydrate the objects.

Don't:

# Worker receives an integer index instead of a Project
manifests = {
    project.manifest.topdir: project.manifest
}

Do:

# Worker receives fully serialized Project objects (or properly reconstructs them)
manifests = {
    project.manifest.topdir: project.manifest
}

---

T1-12: Explicit Context Initialization for Multiprocessing Pools

Rule: Must dictate shared states through an explicit initializer argument injected into the multiprocessing pool rather than implicitly relying on execution fork behavior.

What: Shared parallel context must be managed using an explicit context manager and passed via initializer and initargs in multiprocessing.Pool, rather than relying on unvalidated class attributes or fork memory semantics.

Applies To: Any command or module utilizing multiprocessing.Pool (e.g., ExecuteInParallel).

Why: Relying on fork memory semantics for sharing context works on some Linux systems but fails deterministically on environments (like macOS/Windows) where memory is not automatically shared or when the multiprocessing start method defaults to spawn. Failing to adhere to this typically results in State Leakage / Uninitialized Context.

Trap 1: Assuming child workers inherit updated class variables inherently without explicit initialization.

Don't:

# BAD: Implicit state sharing
cls.parallel_context = data
with multiprocessing.Pool(jobs) as pool:
    pool.imap_unordered(func, inputs)

Do:

# GOOD: Explicit initializer passed to workers
with multiprocessing.Pool(
    jobs,
    initializer=cls._SetParallelContext,
    initargs=(cls._parallel_context,)
) as pool:
    pool.imap_unordered(func, inputs)

Trap 2: A context manager directly yielding an internal dictionary, allowing unvalidated and lingering usage.

Don't:

# BAD: Direct yield of state without lifecycle validation
@contextlib.contextmanager
def ParallelContext(cls):
    yield cls._parallel_context

Do:

# GOOD: Lifecycle management and defensive assertions
@contextlib.contextmanager
def ParallelContext(cls):
    assert cls._parallel_context is None
    cls._parallel_context = {}
    try:
        yield
    finally:
        cls._parallel_context = None

---

T1-13: Dynamic Task Chunk Sizing in Parallel Execution

Rule: Always formulate the chunksize of mapping workers dynamically based on the dataset to job ratio to avoid workload serialization blocks.

What: When utilizing multiprocessing.Pool.imap_unordered, the chunksize must be dynamically calculated based on the ratio of inputs to worker jobs, rather than statically hardcoded.

Applies To: Parallel task dispatch mechanisms (ExecuteInParallel, synchronization loops).

Why: Using a statically hardcoded chunk size (e.g., WORKER_BATCH_SIZE = 32) forced all tasks onto a single worker thread when the total number of projects in the manifest was fewer than 32, completely negating the benefit of parallelism for smaller workloads. Failing to adhere to this typically results in Thread Serialization / Under-utilization.

Trap 1: Passing a hardcoded batch integer to the chunksize parameter.

Don't:

# BAD: Hardcoded chunksize leads to serialization for small input lists
submit(func, inputs, chunksize=WORKER_BATCH_SIZE)

Do:

# GOOD: Dynamic chunking balances load across workers
calc_chunk = min(max(1, len(inputs) // jobs), WORKER_BATCH_SIZE)
submit(func, inputs, chunksize=calc_chunk)

Exceptions: When evaluating specifically within tests that forcefully isolate to a single job execution (jobs=1).

---

T1-14: Deferred Worktree Operations in Sync Local Half

Rule: Must enqueue manifest copy and link actions into the synchronization buffer explicitly prior to issuing early return signals for up-to-date repositories.

What: File link (linkfile) and copy (copyfile) directives must be scheduled using the synchronization buffer (syncbuf.later1) within the Sync_LocalHalf execution phase. They must not be bypassed if the repository exits its sync early due to being up-to-date.

Applies To: Local synchronization logic of projects (project.py -> Sync_LocalHalf).

Why: A bug existed where if a project had published commits in Gerrit (meaning its local state was merged), the sync process would return early and entirely skip applying the manifest's linkfile and copyfile operations, leading to missing or stale files in the developer's worktree. Failing to adhere to this typically results in Worktree Desync / Missing Files.

Trap 1: Returning early from the sync phase without applying required file operations.

Don't:

# BAD: Skipped file modifications when commits are published
if pub == head:
    return

Do:

# GOOD: Schedule file operations via syncbuf before returning
if pub == head:
    syncbuf.later1(self, _doff, not verbose)
    return

---

Cross-Domain Dependencies

• Upstream: T3 | Subprocess Git Integration & Error Translation - Standardizes Git command stream capturing and exception typing, enabling reliable lock contention detection.
• Downstream: T2 | Filesystem Atomicity & Worktree Layout - Relies on parallel execution guard clauses to ensure worktrees exist before checkout operations are dispatched.
• Downstream: T6 | CLI Argument Parsing & UX Consistency - Requires buffered standard outputs and structured logger directives from parallel tasks to maintain coherent terminal formatting.

Chapter: Filesystem Atomicity & Worktree Layout

Context: This domain governs the deterministic creation, migration, and cleanup of internal repository structures and Git worktrees. It relies on ephemeral temporary directories, atomic rename operations, and robust error recovery to prevent corrupted states during unexpected interruptions.

Summary

| Rule ID | Principle / Constraint | Priority | Primary Symptom / | : : : : Trap : | :-------- | :---------------------------- | :------- | :-------------------- | | T2-01 | Unique Subproject Keys in | High | Using the git | : : Worktree Environments : : directory or : : : : : repository name as a : : : : : unique hash key for : : : : : deduplicating project : : : : : objects. : | T2-02 | Non-Destructive Bottom-Up | High | Using aggressive | : : Directory Pruning : : recursive delete : : : : : calls to wipe an old : : : : : link destination : : : : : without verifying its : : : : : internal contents. : | T2-03 | Guaranteed Cleanup of | Critical | Running destructive | : : Temporary Git Artifacts : : or state-modifying : : : : : operations without : : : : : wrapping the cleanup : : : : : routine in a : : : : : finally block. : | T2-04 | Atomic Directory | High | Initializing a | : : Initialization via Temporary : : complex directory : : : Worktrees : : structure directly at : : : : : its final target path : : : : : without atomicity : : : : : guarantees. : | T2-05 | Pathlib Adoption over os.path | Medium | Relying on heavily | : : : : nested string-based : : : : : os.path : : : : : constructions. : | T2-06 | Atomic Directory | Critical | Creating and mutating | : : Initialization via Temporary : : a system directory at : : : Renames : : its final, visible : : : : : destination. : | T2-07 | Encapsulation of Transient | High | Swapping out instance | : : Configuration State : : configuration : : : : : variables back and : : : : : forth during setup. : | T2-08 | Explicit Temporary Resource | Critical | Keeping a stale | : : Ownership and Release : : variable reference to : : : : : a temporary directory : : : : : after it has been : : : : : renamed, allowing a : : : : : deferred cleanup : : : : : routine to operate on : : : : : an unowned path. : | T2-09 | Configuration Object | High | Using a temporary | : : Invalidation Post-Rename : : config object to set : : : : : flags after the : : : : : underlying directory : : : : : has been renamed. : | T2-10 | Granular Safety Checks for | Critical | Deleting a shared | : : Destructive Worktree : : object directory : : : Operations : : immediately upon : : : : : deleting a single : : : : : project that uses it. : | T2-11 | Explicit Warnings for Legacy | Critical | Silently executing a | : : Garbage Collection Fallbacks : : legacy fallback for : : : : : repository safety : : : : : configurations : : : : : without informing the : : : : : user of the potential : : : : : unreliability. : | T2-12 | Atomic Commits for Filesystem | High | Bundling filesystem | : : Restructuring : : path migrations, : : : : : structural : : : : : re-designs, and : : : : : initialization logic : : : : : changes into a single : : : : : Pull : : : : : Request/Patchset. : | T2-13 | Precise Symlink Target | High | Using a generic | : : Replacement During Submodule : : string .replace() : : : Migration : : to update a portion : : : : : of a file path. : | T2-14 | Isolating Filesystem | High | Executing internal | : : Migrations from Network-Only : : directory migrations : : : Operations : : globally prior to : : : : : downloading network : : : : : changes. : | T2-15 | Idempotent Cross-Platform | High | Relying on standard | : : File Cleanup : : library file removal : : : : : and failing to handle : : : : : the scenario where : : : : : the file no longer : : : : : exists. : | T2-16 | Self-Healing Binary JSON | High | Opening JSON state | : : Deserialization : : files in text mode : : : : : without validating : : : : : structure, allowing : : : : : exceptions to bubble : : : : : up and break the CLI. : | T2-17 | Atomic File Writes for | Critical | Using the standard | : : Repository Configurations : : python open() : : : : : context manager and : : : : : print() to mutate : : : : : core git directories. : | T2-18 | File Descriptor Lock Scope | Medium | Processing strings | : : Minimization : : and building : : : : : directory paths : : : : : inside the file : : : : : reading scope. : | T2-19 | Two-Phase Atomic Deletion for | High | Directly triggering | : : Worktrees : : rmtree on a live : : : : : repository path. : | T2-20 | Conditional Absolute Path | High | Blindly deleting and | : : Resolution for Git Worktrees : : rewriting the : : : : : gitdir file using : : : : : os.path.relpath : : : : : without checking if : : : : : the source path is : : : : : absolute first. : | T2-21 | Independent Manifest | High | Defaulting to a | : : Repository Shallow Cloning : : shallow clone for : : : : : configuration : : : : : repositories, causing : : : : : subsequent : : : : : initialization/branch : : : : : switching commands to : : : : : fail due to missing : : : : : objects. :

---

Rules

T2-01: Unique Subproject Keys in Worktree Environments

Rule: Always use repository-relative paths instead of git directory names as unique deduplication keys to prevent collisions in worktree environments.

What: Repository-relative paths must be used instead of git directory names as unique deduplication keys to prevent collisions in environments utilizing git worktrees.

Applies To: Subproject iteration and deduplication logic, primarily within command.py or project discovery utilities.

Why: When tracking derived subprojects, the system originally used gitdir as a unique identifier. In git worktree setups, multiple distinct project instances can share the same underlying gitdir, leading to silent data collisions and missed projects during sync operations. Failing to adhere to this typically results in Data Collision / Omission.

Trap 1: Using the git directory or repository name as a unique hash key for deduplicating project objects.

Don't:

derived_projects.update(
    (p.gitdir, p) for p in project.GetDerivedSubprojects()
)

Do:

derived_projects.update(
    (p.RelPath(local=False), p) for p in project.GetDerivedSubprojects()
)

T2-02: Non-Destructive Bottom-Up Directory Pruning

Rule: Always utilize non-destructive, bottom-up directory iteration that exclusively targets empty directories and symlinks during legacy path cleanup.

What: Legacy path cleanup must utilize non-destructive, bottom-up directory iteration that specifically targets empty directories and symlinks, avoiding recursive tree destruction.

Applies To: Workspace migration, obsolete symlink cleanup, and platform_utils file deletion wrappers.

Why: When a subproject's destination directory changed, the synchronization tool utilized aggressive recursive deletion (rmtree) to remove the previous location. This routinely endangered untracked developer code or extraneous files housed inside the obsolete tree. Failing to adhere to this typically results in Accidental Data Deletion.

Trap 1: Using aggressive recursive delete calls to wipe an old link destination without verifying its internal contents.

Don't:

if platform_utils.isdir(absDest):
    platform_utils.rmtree(absDest)

Do:

# Use a safe helper that only removes empty dirs/symlinks bottom-up
platform_utils.removedirs(absDest)

Trap 2: Attempting directory deletion without explicitly pruning stale manifest files first.

Don't:

platform_utils.removedirs(need_remove_path)
# Misses files entirely as removedirs stops at regular files

Do:

if os.path.isfile(need_remove_path) or os.path.islink(need_remove_path):
    platform_utils.remove(need_remove_path)
platform_utils.removedirs(os.path.dirname(need_remove_path))

Exceptions: Explicitly defined cases where a directory is formally tracked and deleted from the manifest metadata, explicitly commanding removal.

T2-03: Guaranteed Cleanup of Temporary Git Artifacts

Rule: Must wrap any operation mutating the local .git repository with temporary states inside a try/finally block to guarantee restoration.

What: Operations mutating the local .git repository with temporary states (e.g., tracking alt refs during fetch) must wrap the operation in a try/finally block to guarantee state restoration.

Applies To: Subprocess network operations and filesystem interactions affecting the .git database.

Why: If a network operation threw an exception (like a GitAuthError prompting for credentials), the cleanup logic was bypassed. This left orphaned references in the object database and corrupted future repository syncs. Failing to adhere to this typically results in Stale Repository State.

Trap 1: Running destructive or state-modifying operations without wrapping the cleanup routine in a finally block.

Don't:

setup_temporary_refs()
run_git_fetch() # Might throw GitAuthError
cleanup_temporary_refs()

Do:

setup_temporary_refs()
try:
    run_git_fetch()
finally:
    cleanup_temporary_refs()

T2-04: Atomic Directory Initialization via Temporary Worktrees

Rule: Always perform directory initialization via temporary directories and an atomic rename to prevent fragmented repository states.

What: Directory initialization processes must use temporary directories and atomic renames to prevent partial or corrupted states upon unexpected interruption.

Applies To: Filesystem operations involving git repository setup, specifically superproject initialization and dot-git dir creation.

Why: Historically, initializing git directories directly in their final path left behind broken, partially-initialized structures if the user or system interrupted the process. This caused subsequent workflow executions to fail permanently. Failing to adhere to this typically results in Corrupt State / Initialization Failure.

Trap 1: Initializing a complex directory structure directly at its final target path without atomicity guarantees.

Don't:

os.makedirs(final_git_dir, exist_ok=True)
subprocess.run(['git', 'init', final_git_dir])

Do:

temp_dir = tempfile.mkdtemp(dir=target_base)
try:
    subprocess.run(['git', 'init', temp_dir])
    platform_utils.rename(temp_dir, final_git_dir)
finally:
    platform_utils.rmtree(temp_dir, ignore_errors=True)

T2-05: Pathlib Adoption over os.path

Rule: Avoid legacy os.path APIs; prefer modern pathlib.Path structures for traversing, joining, and creating files.

What: Use modern Python pathlib.Path paradigms for traversing, joining, and creating files instead of the legacy os.path APIs.

Applies To: Filesystem and worktree directory manipulation.

Why: Chaining os.path.join and calling os.mkdir repeatedly bloated the codebase, was less readable, and invited platform separator path bugs. Failing to adhere to this typically results in Pathing Errors / High Complexity.

Trap 1: Relying on heavily nested string-based os.path constructions.

Don't:

manifest_dir = os.path.join(self.repodir, 'manifests')
os.mkdir(manifest_dir)
with open(os.path.join(manifest_dir, 'config'), 'w') as fp:
    fp.write(data)

Do:

manifest_dir = self.repodir / 'manifests'
manifest_dir.mkdir(parents=True, exist_ok=True)
(manifest_dir / 'config').write_text(data)

Exceptions: Legacy modules pending modernization where intermingling os.path strings and Path objects would break typing contracts.

T2-06: Atomic Directory Initialization via Temporary Renames

Rule: Never construct complex internal .git directories in-situ; execute initialization inside an ephemeral directory and atomically rename it upon completion.

What: Complex directory structures (like internal .git directories) must be constructed inside an ephemeral temporary directory on the same filesystem volume, and only moved to their final path via an atomic rename once fully validated.

Applies To: Filesystem creation (os.makedirs) and repository initialization (_InitGitDir).

Why: Interrupted or failed operations left partially initialized .git directories in-situ. Follow-up operations encountered corrupt directory trees, which the legacy recovery logic could not reliably clean up. Failing to adhere to this typically results in Corrupt Repository State.

Trap 1: Creating and mutating a system directory at its final, visible destination.

Don't:

os.makedirs(self.gitdir)
self._ReferenceGitDir(self.objdir, self.gitdir)

Do:

tmp_gitdir = create_tmp_dir(os.path.dirname(self.gitdir))
os.makedirs(tmp_gitdir)
self._ReferenceGitDir(self.objdir, tmp_gitdir)
platform_utils.rename(tmp_gitdir, self.gitdir)

T2-07: Encapsulation of Transient Configuration State

Rule: Never reassign class-level instance attributes to intermediate paths during object setup; use local variables for temporary state tracking.

What: Never temporarily reassign class-level instance attributes (e.g., self.config) to point to intermediate or ephemeral paths. Use local variables for temporary objects during setup, assigning the final result to the class instance only upon success.

Applies To: Object initialization workflows within project.py.

Why: During atomic .git directory initialization, self.config was temporarily overwritten to point to a temporary working directory. If the operation crashed, the object was left in an invalid state, pointing to a deleted transient path. Failing to adhere to this typically results in Corrupt Instance State.

Trap 1: Swapping out instance configuration variables back and forth during setup.

Don't:

self.config = GitConfig.ForRepository(gitdir=tmp_gitdir)
# initialization steps
platform_utils.rename(tmp_gitdir, self.gitdir)
self.config = GitConfig.ForRepository(gitdir=self.gitdir)

Do:

tmp_config = GitConfig.ForRepository(gitdir=tmp_gitdir)
# apply setup using tmp_config
platform_utils.rename(tmp_gitdir, self.gitdir)
self.config = GitConfig.ForRepository(gitdir=self.gitdir)

T2-08: Explicit Temporary Resource Ownership and Release

Rule: Always nullify references to a temporary directory immediately after successfully renaming it to its permanent destination.

What: When executing atomic directory initializations via temporary paths, the reference to the temporary directory must be explicitly nullified immediately after successfully renaming it to its final destination.

Applies To: Filesystem operations, specifically in project.py atomic Git directory initializations (_InitGitDir).

Why: If a temporary directory reference was maintained after being atomic-renamed, the subsequent error-handling or finally cleanup block could mistakenly delete the directory. In parallel environments, another concurrent job could validly create a new temporary directory at that exact freed path, causing the current process to inadvertently nuke an active resource belonging to another job. Failing to adhere to this typically results in Data Loss / Race Condition.

Trap 1: Keeping a stale variable reference to a temporary directory after it has been renamed, allowing a deferred cleanup routine to operate on an unowned path.

Don't:

try:
    tmp_gitdir = tempfile.mkdtemp()
    # ... initialize ...
    platform_utils.rename(tmp_gitdir, self.gitdir)
    # BAD: tmp_gitdir still holds the path string
finally:
    if tmp_gitdir and os.path.exists(tmp_gitdir):
        platform_utils.rmtree(tmp_gitdir) # Risk of race condition

Do:

try:
    tmp_gitdir = tempfile.mkdtemp()
    # ... initialize ...
    platform_utils.rename(tmp_gitdir, self.gitdir)
    tmp_gitdir = None  # GOOD: explicitly release ownership of the path
finally:
    if tmp_gitdir and os.path.exists(tmp_gitdir):
        platform_utils.rmtree(tmp_gitdir)

T2-09: Configuration Object Invalidation Post-Rename

Rule: Must discard intermediate configuration instances bound to temporary paths once the underlying directory is atomic-renamed.

What: Configuration instances (e.g., GitConfig) bound to a temporary filesystem path must be abandoned after the path is atomic-renamed. A new instance tracking the permanent directory path must be used for subsequent settings.

Applies To: Git config lifecycle management across atomic filesystem boundary events.

Why: Applying configuration changes like gc.pruneExpire using an object instantiated against a temporary path resulted in IO errors or lost state if the operations occurred after the directory was moved to its final, permanent location. Failing to adhere to this typically results in Lost Configuration / IO Error.

Trap 1: Using a temporary config object to set flags after the underlying directory has been renamed.

Don't:

curr_config = GitConfig.ForRepository(gitdir=tmp_gitdir)
platform_utils.rename(tmp_gitdir, self.gitdir)
curr_config.SetString("gc.pruneExpire", "never")  # BAD: Path no longer valid

Do:

curr_config = GitConfig.ForRepository(gitdir=tmp_gitdir)
platform_utils.rename(tmp_gitdir, self.gitdir)
self.config.SetString("gc.pruneExpire", "never")  # GOOD: Using permanent config tracking self.gitdir

T2-10: Granular Safety Checks for Destructive Worktree Operations

Rule: Must evaluate explicit dependency subsets and prompt overrides before wiping shared object directories.

What: Destructive operations (like wiping directories) must separately evaluate and prompt overrides for distinct risk categories (uncommitted local changes vs. shared object directories), and only delete a shared object directory if all dependent projects are successfully wiped.

Applies To: Commands modifying or deleting project directories and .repo/project-objects.

Why: Deleting shared .repo state indiscriminately could leave other referencing projects broken if they relied on the same object directory, requiring strict subset checks before rmtree execution. Failing to adhere to this typically results in Repository Corruption / Data Loss.

Trap 1: Deleting a shared object directory immediately upon deleting a single project that uses it.

Don't:

# BAD: Deletes shared objdir, breaking other projects
project.DeleteWorktree(force=True)
if os.path.exists(project.objdir):
    platform_utils.rmtree(project.objdir)

Do:

# GOOD: Verify all users of the objdir are deleted first
project.DeleteWorktree(force=True)
successful_wipes.add(project.relpath)

# Later...
if users.issubset(successful_wipes):
    platform_utils.rmtree(objdir)

T2-11: Explicit Warnings for Legacy Garbage Collection Fallbacks

Rule: Must emit a visible warning to sys.stderr when disabling Git garbage collection via legacy fallback configurations.

What: When disabling Git garbage collection for shared network repositories using legacy mechanisms (gc.pruneExpire=never), the tooling must emit a visible sys.stderr warning due to the unreliability of older Git clients.

Applies To: Git configuration manipulation and local filesystem management (_GCProjects).

Why: Older Git clients (pre 2.7.0) do not natively support the extensions.preciousObjects flag. To prevent garbage collection from corrupting shared objects, repo forces gc.pruneExpire to never. Because this is imperfect, it can lead to unreliable repository states, requiring explicit user notification. Failing to adhere to this typically results in Data Loss / Corruption.

Trap 1: Silently executing a legacy fallback for repository safety configurations without informing the user of the potential unreliability.

Don't:

# BAD: Silent fallback to legacy GC directive
if git_require((2, 7, 0)):
    project.config.SetString('extensions.preciousObjects', 'true')
else:
    project.config.SetString('gc.pruneExpire', 'never')

Do:

# GOOD: Alerting the user to the legacy system risk
if git_require((2, 7, 0)):
    project.config.SetString('extensions.preciousObjects', 'true')
else:
    print('WARNING: shared projects are unreliable when using old versions of git...', file=sys.stderr)
    project.config.SetString('gc.pruneExpire', 'never')

T2-12: Atomic Commits for Filesystem Restructuring

Rule: Always strictly isolate structural filesystem modifications and migrations into single atomic commits to maintain bisectability.

What: Refactoring critical filesystem paths (like migrating internal .git submodule structures) must be strictly isolated into atomic commits to preserve bisectability.

Applies To: Worktree layout updates, .git metadata migrations, and submodule structure changes.

Why: A monolithic commit attempted to restructure how submodules were stored (from subprojects/ to modules/), update initialization logic, and alter fetching. This density made code review difficult and created a high risk of regressions that would be impossible to bisect cleanly. Failing to adhere to this typically results in Unbisectable Regressions.

Trap 1: Bundling filesystem path migrations, structural re-designs, and initialization logic changes into a single Pull Request/Patchset.

Don't:

• Submit 1 patchset: [Refactor] Move submodules to modules/, update git config flags, and modify sync --init behaviors.

Do:

• Submit 3 sequential patchsets: 1) Isolate init flag changes. 2) Migrate filesystem directories from subprojects to modules. 3) Enable Git recurse submodules integration.

T2-13: Precise Symlink Target Replacement During Submodule Migration

Rule: Always use explicit path prefixes instead of generic substring replacements to update symlink targets during legacy layout migrations.

What: Symlink updates during filesystem layout migrations must use exact path prefixes rather than global substring replacements to update relative targets safely.

Applies To: Repository migration logic (project.py); specifically _MigrateSubprojectLinks or similar symlink target modifications.

Why: During the migration of legacy submodule directories, using generic string replacement (target.replace) on symlink paths unintentionally corrupted paths if a user's directory structure legitimately contained the target string (e.g., subproject-objects) in a higher-level parent directory name. Failing to adhere to this typically results in Broken Symlinks / Corruption.

Trap 1: Using a generic string .replace() to update a portion of a file path.

Don't:

if "subproject-objects" in target:
    new_target = target.replace(
        "subproject-objects", "module-objects"
    )

Do:

if target.startswith("../../subproject-objects/"):
    new_target = target.replace(
        "../../subproject-objects/", "../../module-objects/"
    )

T2-14: Isolating Filesystem Migrations from Network-Only Operations

Rule: Never execute internal repository layout migrations as part of network fetch iterations.

What: Internal repository layout migrations must only execute during local worktree initialization phases, never during network fetch iterations.

Applies To: Repository sync operations (subcmds/sync.py) and worktree checkout logic (project.py).

Why: Binding irreversible filesystem layout migrations to the sync command's network-fetch loop risked partially migrating the repository if the process was interrupted, or if a user requested a network-only sync, leaving the local workspace out of sync with internal references. Failing to adhere to this typically results in Repository Corruption.

Trap 1: Executing internal directory migrations globally prior to downloading network changes.

Don't:

# In subcmds/sync.py during network fetching
for p in all_projects:
    p._MigrateOldSubprojectDirs()
    # ... fetch logic

Do:

# Inside project.py during _InitWorkTree() or checkout
def _InitWorkTree(self):
    self._MigrateOldSubmoduleDirs()
    # ... symlink creation and checkout logic

T2-15: Idempotent Cross-Platform File Cleanup

Rule: Must employ platform_utils.remove() for file cleanup to safely absorb OS access violations and gracefully swallow ENOENT.

What: File cleanup operations must use the internal platform_utils.remove() instead of the standard os.remove() to handle Windows-specific OS limitations. Furthermore, cleanup must be idempotent by explicitly catching and swallowing errno.ENOENT errors.

Applies To: Local worktree operations, internal state file cleanup (e.g., deleting obsolete copyfiles or linkfiles).

Why: Sync operations would abruptly crash on Windows due to unhandled EACCES issues on read-only files/symlinks with native os.remove(). Furthermore, if a tracking file was already deleted manually by the user or a prior interrupted run, unhandled ENOENT exceptions would needlessly fail the operation. Failing to adhere to this typically results in Crash on Missing/Locked File.

Trap 1: Relying on standard library file removal and failing to handle the scenario where the file no longer exists.

Don't:

if path:
  try:
    os.remove(path)
  except OSError as error:
    print(f'error: remove {path} failed.')
    return 1

Do:

try:
  platform_utils.remove(path)
except OSError as e:
  if e.errno == errno.ENOENT:
    # File does not exist, safe to ignore
    pass

T2-16: Self-Healing Binary JSON Deserialization

Rule: Always read internal JSON state files in binary mode (rb) and implement explicit error catching to seamlessly discard corrupted states.

What: Internal JSON tracking files must be read in binary mode (rb) to allow the JSON parser to detect text encoding safely. Additionally, parsing must be wrapped in a try/except block to automatically delete the file and recover if the JSON is corrupted.

Applies To: Manifest metadata tracking files, internal cache parsing (e.g., copy-link-files.json).

Why: When the operating system's default text encoding did not match the file's encoding, a UnicodeDecodeError could occur. Additionally, if the JSON state file was corrupted (e.g., due to an abrupt power loss or killed process), repo sync would permanently fail until the user manually discovered and deleted the corrupted state file. Failing to adhere to this typically results in Corrupt State Deadlock.

Trap 1: Opening JSON state files in text mode without validating structure, allowing exceptions to bubble up and break the CLI.

Don't:

with open(copylinkfiles_path, 'r') as fd:
  old_copylinkfiles_path = json.load(fd)

Do:

with open(copylinkfiles_path, 'rb') as fp:
  try:
    old_copylinkfiles_path = json.load(fp)
  except:
    print('error: %s is not a json formatted file.' % copylinkfiles_path, file=sys.stderr)
    platform_utils.remove(copylinkfiles_path)
    return False

T2-17: Atomic File Writes for Repository Configurations

Rule: Must utilize atomic file writes rather than standard contexts when generating core .git configuration files.

What: System state modifications, specifically writing the .git pointer files for submodules, must be performed using an atomic write pattern (e.g., writing to a lockfile and renaming) rather than standard file descriptors.

Applies To: All file I/O operations directly modifying .git contents, worktrees, or manifest definitions.

Why: Using standard open() and writing to .git files directly leaves the repository vulnerable to corruption if the process is interrupted (e.g., via Ctrl+C or SIGTERM) during the write loop, resulting in a partially written pointer. Failing to adhere to this typically results in Corrupted Git Worktree.

Trap 1: Using the standard python open() context manager and print() to mutate core git directories.

Don't:

with open(dotgit, "w") as fp:
    print(f"gitdir: {rel_path}", file=fp)

Do:

_lwrite(dotgit, f"gitdir: {rel_path}\n")

T2-18: File Descriptor Lock Scope Minimization

Rule: Minimize file descriptor lock retention by executing string and state logic outside of standard reading contexts.

What: Data processing and variable manipulation must be extracted out of file descriptor lock contexts (with open(...)) to minimize resource retention.

Applies To: Any block parsing local text or configuration files.

Why: Holding file descriptors open while executing CPU-bound string manipulations increases lock contention probability and expands the attack surface for race conditions when multiple concurrent processes read/write repository states. Failing to adhere to this typically results in Resource Lock Contention.

Trap 1: Processing strings and building directory paths inside the file reading scope.

Don't:

with open(dotgit) as fp:
    setting = fp.read()
    gitdir = setting.split(":")[1].strip()
    dotgit_path = os.path.normpath(os.path.join(self.worktree, gitdir))

Do:

with open(dotgit) as fp:
    setting = fp.read()

# Lock released, safe to process
gitdir = setting.split(":")[1].strip()
dotgit_path = os.path.normpath(os.path.join(self.worktree, gitdir))

T2-19: Two-Phase Atomic Deletion for Worktrees

Rule: Must enact permanent .git and worktree removal via a two-phase destruction pattern utilizing a temporary trash path.

What: When permanently removing a .git directory or worktree, the system must first perform an atomic rename to a temporary trash path before initiating the recursive deletion.

Applies To: Garbage collection (gc.py), worktree cleanup, and submodule removal logic.

Why: Recursive deletion is not instantaneous. If a user forcefully interrupted (Ctrl+C) the rmtree operation, the directory was left in a 'wedged' (partially deleted but still registered) state, permanently breaking future syncs for that repository. Failing to adhere to this typically results in Wedged / Orphaned Worktree.

Trap 1: Directly triggering rmtree on a live repository path.

Don't:

for path in to_delete:
    platform_utils.rmtree(path)

Do:

for path in to_delete:
    temp_path = rename_to_trash(path)
    platform_utils.rmtree(temp_path)

T2-20: Conditional Absolute Path Resolution for Git Worktrees

Rule: Always independently verify if a tracked worktree path is absolute before initiating internal path recreation procedures.

What: Updates to .git directory reference files (e.g., gitdir:) must verify whether the stored worktree path is absolute before attempting to recreate it as a relative path to avoid path corruption and permissions errors.

Applies To: Internal Git directory layout management, specifically cross-platform Git worktree initialization (project.py).

Why: On certain platforms like Windows, modifying the internal dotgit file in situ fails due to file permissions. Code was added to delete and recreate the file using relative paths. However, depending on the Git version, the initial path could already be relative, making conversion unsafe without an isabs() check. Failing to adhere to this typically results in Permission Denied / Path Corruption.

Trap 1: Blindly deleting and rewriting the gitdir file using os.path.relpath without checking if the source path is absolute first.

Don't:

platform_utils.remove(dotgit)
with open(dotgit, "w", newline="\n") as fp:
    print("gitdir:", os.path.relpath(git_worktree_path, self.worktree), file=fp)

Do:

if os.path.isabs(git_worktree_path):
    platform_utils.remove(dotgit)
    with open(dotgit, "w", newline="\n") as fp:
        print("gitdir:", os.path.relpath(git_worktree_path, self.worktree), file=fp)

T2-21: Independent Manifest Repository Shallow Cloning

Rule: Never propagate the primary manifest --depth flag to govern the core manifest repository's clone boundaries.

What: The depth configuration for cloning the central manifest repository must be explicitly decoupled from the depth configuration applied to the child projects it governs, and its default must not regress existing multi-stage init workflows.

Applies To: Repo initialization commands (repo init, subcmds/init.py).

Why: Using the global --depth flag inadvertently forced shallow clones on all child projects. Introducing a dedicated --manifest-depth option fixed this, but defaulting it to 1 caused a regression during 'double repo init' workflows (where subsequent checkouts failed against truncated manifest histories). Failing to adhere to this typically results in Workflow Regression / History Truncation.

Trap 1: Defaulting to a shallow clone for configuration repositories, causing subsequent initialization/branch switching commands to fail due to missing objects.

Don't:

group.add_option('--manifest-depth', type='int', default=1, metavar='DEPTH',
                 help='create a shallow clone of the manifest repo')

Do:

group.add_option('--manifest-depth', type='int', default=0, metavar='DEPTH',
                 help='create a shallow clone of the manifest repo')

---

Cross-Domain Dependencies

• Upstream: T3 | Subprocess Git Integration & Error Translation - Network operations and git process invocations require safe staging paths and catchable exceptions to trigger filesystem cleanups.
• Downstream: T1 | Concurrent Synchronization & IPC - Parallel checkout operations depend heavily on atomic path modifications and lock-free directory states to prevent race conditions.
• Downstream: T4 | Manifest Object Model & Deduplication - Filesystem migrations and subproject tree building directly parse and reflect canonicalized XML manifest declarations.

Chapter: Subprocess Git Integration & Error Translation

Context: This chapter defines the constraints for wrapping, executing, and translating standard Git subprocesses within the Repo tooling ecosystem. It mandates the use of centralized command abstractions, strict version gating, and deterministic stream handling to guarantee reliable repository state management and actionable error reporting.

Summary

Rule ID	Principle / Constraint	Priority	Primary Symptom / Trap
T3-01	Unborn Branch Resolution	High	Catching a rev-parse
: : via Symbolic-Ref : : GitError and immediately :
: : : : falling back to manual :
: : : : file system reads or :
: : : : emitting warnings. :
T3-02	Centralized GitCommand	Medium	Wrapping subprocess.run
: : Abstraction Usage : : manually inside a class :
: : : : to invoke Git. :
T3-03	NUL-Byte Delimiters for	High	Formatting git fields
: : Git Output Parsing : : with tabs and defending :
: : : : against length :
: : : : variations. :
T3-04	Subprocess Execution	Medium	Stringifying Path
: : Context and Native Path : : objects, using :
: : Passing : : Git-specific directory :
: : : : shifts, and suppressing :
: : : : piped output. :
T3-05	Runtime Git Version	High	Invoking a modern git CLI
: : Constraints for Promisor : : flag without ensuring :
: : Packs : : compatibility first. :
T3-06	Centralized Dependency	Medium	Catching missing binary
: : Availability Caching : : exceptions to terminate :
: : : : the process instead of :
: : : : returning control to the :
: : : : caller. :
T3-07	Subprocess Return Code	High	Calling a non-existent
: : Validation : : subprocess execution :
: : : : method on an already :
: : : : completed process object. :
T3-08	Strict Git Version	High	Injecting convenient,
: : Compatibility : : modern CLI flags into :
: : : : subprocess wrappers. :
T3-09	Rebase Override for	High	Throwing a
: : Published Local Branches : : LocalSyncFail when an :
: : : : upstream gain exists, :
: : : : completely ignoring the :
: : : : force_rebase argument. :
T3-10	SSO Authentication	High	Treating all exit code
: : Failure Translation : : 128 errors equally :
: : : : without sniffing stdout :