| name | tower-mind-agent-benchmark |
| title | TowerMind: A Tower Defence Game Learning Environment and Benchmark for LLM as Agents |
| version | 0.0.2 |
| engine | skillxiv-v0.0.2-claude-opus-4.6 |
| license | MIT |
| url | https://arxiv.org/abs/2601.05899 |
| keywords | ["agent-evaluation","real-time-strategy","planning-benchmark","hallucination-measurement","game-environment"] |
| description | Evaluate LLM agent capabilities using tower defense game environment with multimodal observations (pixel, text, structured state). Benchmark reveals critical agent limitations: inadequate planning validation, inflexible decision-making, and inefficient action use. Demonstrates significant performance gap between current LLMs and human experts, providing structured framework for measuring agent planning, adaptation, and hallucination tendencies. |
Problem
Current LLM agent evaluation relies on narrow benchmarks:
- Limited Domains: Most benchmarks test single skills (QA, tool use, reasoning)
- Unrealistic Complexity: Real-world agents face simultaneous constraints and dynamic environments
- Missing Capability Assessment: No systematic measurement of planning, adaptation, and strategic reasoning
- Hallucination Blindness: Benchmarks don't explicitly measure when agents fabricate information
- Human-Agent Gap Unknown: Unclear how far agents lag behind human performance on complex tasks
Agents need evaluation frameworks that measure real-world reasoning capabilities beyond text understanding.
Solution
TowerMind introduces a Tower Defense Game Benchmark for LLM agents:
- Multimodal Observations: Agents receive three observation types:
- Pixel-Based: Raw game visuals (tower positions, enemy locations)
- Textual: Natural-language game state descriptions
- Structured: JSON game state with numeric values
- Strategic Complexity: Tower defense requires simultaneous optimization:
- Planning: Predict enemy paths and plan tower placement
- Resource Management: Budget constraints on tower construction
- Adaptation: Adjust strategies as enemy waves change
- Explicit Hallucination Measurement: Benchmark includes scenarios designed to expose fabricated knowledge
- Out-of-place game elements
- Contradictions between observation modes
- Impossible win conditions
When to Use
- Agent Capability Assessment: Measure planning, adaptation, and strategic reasoning
- Multimodal Evaluation: Test agents on multiple observation formats
- Hallucination Testing: Diagnose tendency to fabricate information
- Comparative Benchmarking: Track agent improvements across versions
- Planning Validation: Verify that agents think ahead vs. reacting locally
- Academic Research: Study how LLMs approach real-time strategy
When NOT to Use
