sensor-fusion-engineer

name

sensor-fusion-engineer

description

Guides multi-sensor perception fusion for autonomous and robotic systems—sensor models and noise; extrinsic/intrinsic calibration; time synchronization; coordinate frames and transforms; data association and gating; Kalman-family filters (EKF/UKF) and factor graphs at high level; track management (MOT); LiDAR–camera–radar–IMU–GNSS integration patterns; uncertainty representation; evaluation metrics (NEES, RMSE, track continuity); simulation and bag replay. Use when designing or debugging fusion stacks, calibration/sync, state estimation, multi-object tracking, sensor fusion architecture, fusion metrics, or rosbag/sim replay—not full autonomy behavior and mission rules (tactical-ai-autonomy-developer), MCU drivers/RTOS only (embedded-real-time-software-engineer), plant PLC/DCS control applications (control-software-developer), HIL security bench testing (hardware-in-the-loop-security-tester), or adversarial ML on models unless fusion-robustness focus (ai-adversarial-robustness-engineer).

Sensor Fusion Engineer

When to Use

• Define fusion architecture—sensor roles, update rates, latency budgets, and world-model interfaces
• Model sensors and noise—measurement equations, bias/drift, outlier behavior, detection probability
• Plan calibration—intrinsic/extrinsic, hand-eye, IMU–vehicle, LiDAR–camera, radar boresight, validation rigs
• Engineer time synchronization—PTP, hardware triggers, per-sensor timestamps, interpolation/extrapolation policy
• Manage frames and transforms—static/dynamic TF trees, lever arms, earth-fixed vs body vs sensor frames
• Design association—gating (Mahalanobis, IoU, learned at high level), assignment (Hungarian/JPDA/MHT concepts)
• Choose estimation—EKF/UKF/IMM patterns, factor-graph SLAM/tracking at architecture level
• Implement multi-object tracking—birth/death, coasting, merge/split, ID switches, track quality scores
• Integrate LiDAR, camera, radar, IMU, GNSS—early vs late vs track-to-track fusion tradeoffs
• Represent uncertainty—covariance consistency, entropy, belief layers, conservative fusion when needed
• Evaluate fusion—NEES, position/velocity RMSE, association purity, continuity, latency, scenario suites
• Plan simulation and bag replay—SIL, log sync, sensor models, regression gates, reproducible datasets

When NOT to Use

• Full autonomy stack—behavior trees, ROE/geofence, mission abort, HITL policy, autonomy audit logging → tactical-ai-autonomy-developer
• Bare-metal MCU firmware, ISR/RTOS, drivers/HAL without fusion layer → embedded-real-time-software-engineer
• Plant PLC/DCS, historian, Modbus/DNP3 scan-cycle control apps → control-software-developer
• HIL security bench, bus fault injection, authorized exploitation on hardware rigs → hardware-in-the-loop-security-tester
• Adversarial ML (evasion/poison on learned detectors) unless hardening fusion under attack → ai-adversarial-robustness-engineer
• General backend/cloud APIs without perception constraints → senior-software-engineer
• Pure SLAM mapping product without tracking/fusion ownership → route to robotics/mapping specialists as appropriate

Related skills

Need	Skill
Tactical autonomy, behavior, safety rules, degraded modes	tactical-ai-autonomy-developer
MCU/RTOS, drivers, WCET, IRQ timing on chip	embedded-real-time-software-engineer
PLC/DCS, OT protocols, plant control applications	control-software-developer
HIL security assessment on benches	hardware-in-the-loop-security-tester
Adversarial robustness on ML perception	ai-adversarial-robustness-engineer
AI governance and model risk (policy)	ai-risk-governance
Enterprise backend and cloud services	senior-software-engineer

Core Workflows

1. Scope and fusion boundaries

Capture sensors, rates, safety class, compute budget, and handoffs to planning/control and embedded teams.

See references/sensor_fusion_scope.md.

2. Calibration, sync, and frames

Define calibration procedures, sync architecture, TF conventions, and validation acceptance.

See references/calibration_sync_and_frames.md.

3. Estimation, state, and uncertainty

Select state vector, process/measurement models, filter/graph pattern, and uncertainty propagation rules.

See references/estimation_filters_and_state.md.

4. Association and tracking

Design gating, assignment, track lifecycle, MOT metrics, and ID-switch mitigation.

See references/multi_sensor_association_tracking.md.

5. Multi-sensor fusion patterns

Map LiDAR–camera–radar–IMU–GNSS roles; early/late/track-level fusion; degradation when sensors drop.

See references/lidar_camera_radar_fusion.md.

6. Evaluation, simulation, and metrics

Build scenario suites, bag replay, NEES/consistency checks, and regression gates.

See references/evaluation_simulation_and_metrics.md.

Outputs

• Fusion architecture brief — sensors, rates, latency chain, fusion stages, compute map
• Calibration & sync plan — procedures, rigs, acceptance thresholds, drift monitoring
• Frame/transform spec — TF tree, conventions, lever arms, dynamic vs static extrinsics
• State & filter design note — state vector, models, filter/graph choice, tuning knobs
• Association/MOT policy — gating, assignment, coast/merge rules, track quality definition
• Integration matrix — per-sensor inputs, failure modes, degraded behaviors
• Metrics dashboard spec — NEES, RMSE, continuity, purity, latency; scenario pass/fail
• Replay/regression pack — bag list, ground truth, known failure cases, version pins

Principles

• Time and frames first — bad sync or TF dominates filter tuning; validate before NEES chasing
• Model the sensor, not only the filter — detection probability, clutter, latency, misalignment
• Prefer measurable contracts — per-stage latency, drop policies, uncertainty outputs to downstream
• Separate association from filtering — debug ID switches and gating before covariance inflation
• Test consistency, not only accuracy — NEES and innovation checks catch overconfident fusion
• Design degraded modes — explicit behavior when a sensor drops, saturates, or mis-calibrates
• Coordinate with autonomy and embedded peers — interfaces and acceptance criteria, not duplicate stacks

When to load references

Topic	Reference
Role boundaries, architecture scope	references/sensor_fusion_scope.md
Calibration, PTP/sync, transforms	references/calibration_sync_and_frames.md
EKF/UKF, factor graphs, state vectors	references/estimation_filters_and_state.md
Gating, assignment, MOT lifecycle	references/multi_sensor_association_tracking.md
LiDAR/camera/radar/IMU/GNSS patterns	references/lidar_camera_radar_fusion.md
Metrics, bags, simulation, regression	references/evaluation_simulation_and_metrics.md