dynaact-dynamic-action-spaces-reasoning

name	dynaact-dynamic-action-spaces-reasoning
title	DynaAct: LLM Reasoning with Dynamic Action Spaces
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2511.08043
keywords	["Action Space Design","Submodular Optimization","Sequential Reasoning","LLM Planning","Greedy Selection"]
description	Automatically construct compact, diverse action spaces for LLM reasoning through corpus-based estimation and submodular optimization—enabling efficient decision-making without manual specification or expensive exhaustive search.

Construct Dynamic Action Spaces for Efficient LLM Reasoning

Language model agents typically use manually-defined action spaces (fixed operations available at each step) that lack scalability, or generate all possible actions exhaustively—an expensive approach. DynaAct solves this through a two-stage method: automatically estimate the space of viable actions from a corpus of reasoning tasks, then greedily select a compact subset optimized for relevance and diversity to the current problem.

The result is efficient, adaptive action spaces that improve performance on six reasoning benchmarks while maintaining fast inference without significant latency overhead.

Core Concept

DynaAct treats action space construction as a submodular optimization problem. The system learns general action patterns from diverse reasoning examples, then selects task-specific actions balancing two criteria:

1. Relevance: How applicable is each action to the current problem state?
2. Diversity: Do selected actions cover different solution strategies?

This two-stage approach avoids both manual specification burdens and exhaustive generation costs, enabling scalable, efficient reasoning agents.

Architecture Overview

• Corpus Analysis Module: Scans diverse reasoning tasks to extract general action patterns
• Action Sketch Extraction: Uses LLM to identify high-level solution strategies from examples
• Space Estimation: Builds compact representation of feasible action space
• Submodular Objective: Defines relevance + diversity tradeoff as optimization function
• Greedy Selection: Efficiently selects k actions maximizing submodular objective
• Inference Integration: Routes selected actions into LLM prompts during reasoning

Implementation Steps

Step 1: Extract Action Sketches from Corpus

Analyze diverse reasoning problems to identify common solution patterns.

def extract_action_sketches(problem_corpus: List[str], llm_api) -> List[str]:
    """
    Extract high-level action patterns from diverse reasoning problems.

    Args:
        problem_corpus: List of reasoning problem descriptions
        llm_api: Language model API for analysis

    Returns:
        action_sketches: List of general solution strategy descriptions
    """
    prompt_template = 

    action_sketches = ()

    
    sampled_problems = random.sample(
        problem_corpus,
        (, (problem_corpus))
    )

     problem  sampled_problems:
        prompt = prompt_template.(problem=problem)
        sketch = llm_api.generate(prompt, max_tokens=)
        action_sketches.add(sketch.strip())

     (action_sketches)

 () -> [, []]:
    
    action_categories = {
        : [],
        : [],
        : [],
        : [],
        : []
    }

     sketch  action_sketches:
        sketch_lower = sketch.lower()

        
         (word  sketch_lower  word  [, , ]):
            action_categories[].append(sketch)
         (word  sketch_lower  word  [, , ]):
            action_categories[].append(sketch)
         (word  sketch_lower  word  [, , ]):
            action_categories[].append(sketch)
         (word  sketch_lower  word  [, , ]):
            action_categories[].append(sketch)
         (word  sketch_lower  word  [, , ]):
            action_categories[].append(sketch)

     action_categories