define-state-space

name	Define State Space
description	This skill should be used when the user asks to "define the state space", "design the observation space", "specify what the agent observes", "determine the agent's inputs", "assess the Markov property", "design observation normalization", or "formulate the observation vector". Do NOT hallucinate parameters outside the boundaries of Define State Space ($S$).
version	0.1.0

Define State Space ($S$)

Formulate and design the optimal State Space representation for an RL problem. This skill produces a formal StateSpace specification including space type, dimensions, Markov assessment, normalization requirements, and a physical observation index mapping.

Step 1 — Identify the Core Variables

Enumerate the absolute minimum set of physical/mathematical values required to uniquely describe the current state. Group by variable type:

Type	Examples
Positions	x, y, z coordinates, joint angles
Velocities	Linear velocity, angular velocity, joint velocities
Relative metrics	distance_to_target, angle_to_goal (preferred over absolute coordinates)
Categorical / binary	Inventory flags, weapon loaded, door open
Temporal	Steps remaining, cooldown timer
External	Opponent positions, wind speed

Prefer relative over absolute coordinates. Instead of (x_agent, y_agent) and (x_target, y_target), pass (rel_dx, rel_dy). This removes global position dependence and forces the agent to learn a translation-invariant, generalizable policy.

Step 2 — Assess the Markov Property

The Markov Property requires: $P(s_{t+1} \mid s_t, a_t) = P(s_{t+1} \mid s_1, ..., s_t, a_1, ..., a_t)$

The current observation must be sufficient to predict the next state. Test each variable:

Scenario	Problem	Solution
Only position available, but velocity needed	Agent cannot infer trajectory direction	Add velocity variables OR use frame-stacking
Partially occluded environment	Agent cannot observe all relevant entities	POMDP treatment: use RNN/LSTM policy
Hidden opponent intentions	True state partially unobservable	POMDP treatment: use memory-augmented policy

If the observation is not Markovian, formally classify the problem as a POMDP (Partially Observable MDP). This mandates using an LSTM/GRU policy or frame-stacking in the downstream algorithm design.

Step 3 — Dimensionality Reduction and Feature Engineering

Before finalizing the observation vector, apply these reduction checks:

1. Remove absolute global frames → replace with relative frames.
2. Remove redundant derivable quantities → if distance = sqrt(dx² + dy²) is computable from dx, dy already in obs, omit it.
3. Collapse symmetric representations → if two joints are always symmetric, represent their difference rather than both values.
4. Cap or log-scale unbounded quantities → wealth, cumulative distance, time-alive grow unboundedly and should be log-transformed or normalized with running statistics.

Step 4 — Normalization and Bounds Strategy

Every observation element must have a defined normalization strategy before passing to a neural network:

Bound Type	Normalization Strategy	Tool
Hard bounded [a, b]	Scale to [-1.0, 1.0] statically: (x - a) / (b - a) * 2 - 1	Manual rescaling in env
Soft bounded (approx range known)	Scale by known typical range	Manual rescaling
Unbounded (wealth, distance)	Running mean/std normalization	VecNormalize wrapper or RunningMeanStd
Image pixels [0, 255]	Divide by 255.0 → [0, 1]	Observation wrapper

Flag all unbounded elements explicitly — these require a VecNormalize or RunningMeanStd wrapper in the training pipeline.

Output Format

When utilizing this skill, generate:

StateSpace:
  Type: [Box / Discrete / MultiBinary / Dict]
  Dimensions: [Numeric size of flat observation vector]
  Markovian: [True / False]
  POMDP_Treatment: [None / FrameStack(N) / LSTM / GRU]
  Normalizations_Required: [List of wrappers needed]

Follow with a detailed observation index table:

Index	Variable Name	Physical Meaning	Raw Range	Normalized Range
0	rel_dx	Relative X distance to target	[-50m, +50m]	[-1.0, 1.0]
1	rel_dy	Relative Y distance to target	[-50m, +50m]	[-1.0, 1.0]
2	vel_x	Agent X velocity	[-10, +10 m/s]	[-1.0, 1.0]

Additional Resources

Reference Files

• references/observation-design-patterns.md — Frame stacking, POMDP treatment, Dict observation spaces, image observation preprocessing

Scripts

• scripts/validate_state_space.py — Validates a state space specification dict and checks normalization coverage for all dimensions