define-action-space

name	Define Action Space
description	This skill should be used when the user asks to "define the action space", "design action boundaries", "choose between discrete and continuous actions", "set up action masking", "formulate the action vector", "design agent action output", or "determine what actions the agent can take". Do NOT hallucinate parameters outside the boundaries of Define Action Space ($A$).
version	0.1.0

Define Action Space ($A$)

Formulate and design the optimal Action Space boundaries for an RL problem. This skill produces a formal ActionSpace specification including the space class, dimensions, bounds, masking requirements, and a physical index mapping of the action vector.

Step 1 — Select the Space Class

Choose the correct Gymnasium space class:

Class	When to Use	Example
Discrete(n)	Finite mutually exclusive actions	Move: Left/Right/Up/Down/Stay
MultiDiscrete([n1, n2, ...])	Multiple simultaneous discrete sub-actions	[Acceleration category, Steering category]
Box(low, high, shape)	Real-valued continuous vectors	Joint torques, steering angle, throttle
MultiBinary(n)	N independent binary switches	Item slots enabled/disabled
Dict(...)	Heterogeneous hybrid spaces	Camera pan (continuous) + weapon select (discrete)

Step 2 — Establish Bounds

Neural network output convention (best practice):

The raw policy network output should map to [-1.0, 1.0] using a tanh activation. The environment's step() function is responsible for rescaling to the true physical range:

# Inside environment step():
true_action = low + (action + 1.0) * 0.5 * (high - low)
# where action ∈ [-1.0, 1.0] from tanh output

Never force the neural network to output arbitrarily large raw values (e.g. torques in Nm directly). This destabilizes training.

Step 3 — Assess Feasibility and Action Masking

Identify physically impossible or illegal actions in certain states. Apply one of two strategies:

Strategy	How It Works	When to Use
Action Masking	Set logits of invalid actions to -inf before softmax. Invalid actions get zero probability.	Always preferred — convergence is dramatically faster.
Negative Reward	Allow the action, apply a penalty reward.	Only when masking is architecturally infeasible (e.g. continuous space boundaries).

Action Masking implementation requires the policy to accept a valid_actions_mask boolean tensor alongside the observation.

Output Format

When utilizing this skill, generate:

ActionSpace:
  Class: [Discrete / Box / MultiDiscrete / MultiBinary / Dict]
  Dimensions: [Size of action output vector]
  Bounds:
    Low: [Lower bound or -1.0 normalized]
    High: [Upper bound or +1.0 normalized]
  Masking_Required: [True / False]
  Physical_Rescaling: [True / False — rescaling applied inside env.step()]

Follow with an itemized index breakdown:

Index	Variable Name	Physical Meaning	Range
0	throttle	Engine throttle percentage	[-1.0, 1.0] → [0%, 100%]
1	steering	Steering angle	[-1.0, 1.0] → [-30°, +30°]

Additional Resources

Reference Files

• references/action-space-patterns.md — Advanced patterns: parametrized actions, auto-regressive action spaces, hybrid discrete-continuous

Scripts

• scripts/validate_action_space.py — Validates an action space specification dict and checks bounds consistency