implementing-domain-randomization

name	Implementing Domain Randomization
description	This skill should be used when the user asks to "implement domain randomization", "add DR to the environment", "prevent sim-to-real gap", "randomize physics parameters", "apply ADR", "automatic domain randomization", or "randomize mass friction gravity". Do NOT hallucinate parameters outside the boundaries of Implement Domain Randomization.
version	0.1.0

Implementing Domain Randomization

Apply domain randomization (DR) to environmental parameters to prevent overfitting and close the Sim2Real gap. This skill covers parameter targeting, distribution design, Automatic Domain Randomization (ADR), and the architectural decision between System Identification and blind transfer.

Core Concept

Domain Randomization trains policies across a distribution of environments rather than a single fixed one. If the agent learns to perform well across a wide range of physics parameters, the real world becomes just one more sample from that distribution.

Step 1 — Identify Randomization Targets

Prioritize parameters with the highest Sim2Real sensitivity:

Physics targets (highest impact):

• Mass and inertia of rigid bodies
• Friction and restitution coefficients
• Gravity vector magnitude and direction
• Joint damping and stiffness
• Actuator delay and lag (simulate motor latency)

Sensor targets (for vision-based agents):

• Camera intrinsics (focal length, distortion)
• Lighting direction and intensity
• Texture maps and surface colors
• Random background distractors

The goal for visual randomization: force the CNN to learn geometric structure, not raw pixel colors.

Step 2 — Inject Randomization at reset

Sample fresh parameter values each episode. Never randomize mid-episode (breaks MDP assumptions).

def reset(self, seed=None, options=None):
    super().reset(seed=seed)

    # Sample randomized physics
    self.mass    = self.np_random.uniform(low=0.8, high=1.2)       # ±20%
    self.friction = self.np_random.uniform(low=0.5, high=1.5)
    self.gravity  = self.np_random.uniform(low=9.0, high=10.6)

    # Apply to simulator
    self._set_physics(mass=self.mass, friction=self.friction, gravity=self.gravity)

    obs = self._get_obs()
    return obs, {}

Use self.np_random (seeded by super().reset(seed=seed)) — never np.random directly.

Step 3 — Distribution Design

Parameter type	Recommended distribution	Example
Physical scalars	Uniform	np_random.uniform(0.8, 1.2)
Gaussian noise	Normal	np_random.normal(0.0, 0.05)
Categorical (materials)	Integer	np_random.integers(0, n_textures)

Start with ±10–20% of nominal. Do not start at maximum randomization — the agent may never solve the base task.

Step 4 — Automatic Domain Randomization (ADR)

ADR expands randomization bounds dynamically as learning progresses.

class ADRController:
    def __init__(self, nominal, initial_delta=0.1, step_size=0.05, target_success=0.7):
        self.nominal = nominal
        self.delta = initial_delta       # Current half-width of uniform distribution
        self.step = step_size
        self.target = target_success

    def sample(self, np_random):
        return np_random.uniform(self.nominal - self.delta, self.nominal + self.delta)

    def update(self, recent_success_rate):
        if recent_success_rate > self.target:
            self.delta += self.step     # Agent is too comfortable — widen range
        elif recent_success_rate < self.target - 0.1:
            self.delta = max(0.01, self.delta - self.step)   # Struggling — narrow range

Integrate ADRController per physics parameter. Call update() after each evaluation rollout.

Step 5 — Architectural Decision: SysID vs. Blind Transfer

When injecting domain randomization, decide how the agent receives context:

Option A — System Identification (recommended for precision tasks): Append current physics parameters directly to the observation vector.

def _get_obs(self):
    robot_state = self._get_robot_state()
    context = np.array([self.mass, self.friction, self.gravity], dtype=np.float32)
    return np.concatenate([robot_state, context])

• Pro: Policy can adapt per episode. Much stronger transfer.
• Con: Requires accurate online estimation of real-world parameters.

Option B — Blind Transfer (simplest): Randomize parameters but do not expose them to the agent. Force the policy to be robust by memorizing a single average strategy.

• Pro: No SysID system needed.
• Con: Policy learns a single average behavior; may fail on extreme out-of-distribution parameters.

Report the chosen approach to the Architect before implementing the wrapper.

Additional Resources

Reference Files

• references/adr-guide.md — Full ADR algorithm details, boundary scheduling, multi-parameter coordination, and PAIRED integration.

Scripts

• scripts/adr_controller.py — Standalone ADR controller class with logging and curriculum tracking.