search-hardness-aware-llm-problem-formulation

name	search-hardness-aware-llm-problem-formulation
description	Search Hardness-Aware LLM-Based Problem Formulation (SHA-PF) framework for expensive simulation-driven design. Prioritizes formulations that guide efficient search by focusing on rare samples with greater progress potential, reducing evaluation requirements significantly.
arxiv_id	2607.2122
date	2026-07-23T00:00:00.000Z
authors	["Yuchen Li","Handing Wang","Bing Xue","Mengjie Zhang"]
categories	["LLM-based optimization","expensive simulation-driven design","problem formulation","search hardness awareness","evolutionary refinement"]

Search Hardness-Aware LLM-Based Problem Formulation for Expensive Simulation-Driven Design

Overview

This methodology introduces SHA-PF (Search Hardness-Aware Problem Formulation), an LLM-based framework that addresses the limitations of existing automatic problem formulation methods by considering not just design-intent alignment but also whether the formulation induces an efficient search process. The approach recognizes that problem formulation itself shapes the search landscape by defining objectives and constraints, and prioritizes formulations that guide efficient search through rare samples with greater progress potential.

Core Contributions

Search Hardness Awareness

• Beyond Design-Intent Alignment: Moves beyond traditional LLM-based formulation methods that focus solely on matching natural-language requirements
• Search Process Efficiency: Considers whether formulations induce efficient search processes in expensive simulation-driven design
• Rare Sample Prioritization: Identifies that formulations are more likely to guide efficient search when they prioritize rare samples with greater progress potential

SHA-PF Framework

• Formulation Search Objective: Defines objective guided by search hardness, scoring candidate formulations according to their priority for efficient search
• LLM-Based Generation: Uses large language models for initial formulation generation from natural-language requirements
• Repair and Evolutionary Refinement: Implements repair mechanisms and evolutionary refinement to optimize formulations under the search hardness objective
• Progress Potential Scoring: Scores formulations based on their ability to identify solutions with greater progress potential toward design requirements

Experimental Validation

• Real-World Benchmark: Tested on real-world multi-objective benchmark problems
• Antenna Design Applications: Validated on five expensive antenna design benchmarks
• Significant Evaluation Reduction: Demonstrates that SHA-PF discovered formulations require significantly fewer evaluations to reach design requirements compared to baselines

Key Applications

Engineering Design Optimization

• Reduces computational cost in expensive simulation-driven design processes
• Improves efficiency of engineering design optimization under limited evaluation budgets
• Enables more practical application of simulation-driven design in resource-constrained scenarios

LLM-Based Problem Solving

• Enhances LLM capabilities for complex problem formulation beyond simple requirement translation
• Provides framework for LLMs to consider search efficiency in addition to semantic correctness
• Bridges gap between natural language understanding and efficient optimization

Automated Design Systems

• Supports development of fully automated design systems that can both understand requirements and formulate efficient search strategies
• Enables adaptive problem formulation that evolves based on search performance feedback
• Facilitates integration of domain knowledge with general-purpose LLM reasoning

Implementation Guidelines

When to Use

Use this framework when:

• Working with expensive simulation-driven design problems with limited evaluation budgets
• Need to automatically formulate optimization problems from natural-language requirements
• Traditional LLM-based formulation methods produce inefficient search landscapes
• Seeking to reduce the number of expensive simulations required to reach design requirements
• Developing automated design systems that need to balance semantic correctness with search efficiency

Technical Requirements

• Large Language Model access for formulation generation and repair
• Optimization solver integration for evaluating formulation effectiveness
• Search hardness estimation capabilities for scoring candidate formulations
• Evolutionary algorithm framework for formulation refinement
• Performance monitoring system for tracking progress potential

Implementation Steps

1. Requirement Processing: Convert natural-language design requirements into initial problem formulations using LLM
2. Search Hardness Estimation: Estimate search hardness for each candidate formulation based on rare sample identification
3. Progress Potential Scoring: Score formulations according to their ability to prioritize samples with greater progress potential
4. Formulation Generation: Generate diverse candidate formulations using LLM-based methods
5. Repair Mechanism: Implement repair operations to fix invalid or inefficient formulations
6. Evolutionary Refinement: Apply evolutionary algorithms to refine formulations under the search hardness objective
7. Validation and Selection: Validate top formulations on representative problems and select the most efficient

Validation and Results

The framework has been validated through:

• Real-world multi-objective benchmark demonstrating significant evaluation reduction
• Five expensive antenna design benchmarks showing consistent performance improvements
• Comparison against baseline methods proving superior efficiency in reaching design requirements
• Empirical evidence supporting the rare sample prioritization hypothesis

References

• Original paper: arXiv:2607.21220 [cs.NE]
• Related work on LLM-based problem formulation and expensive optimization
• Foundational papers on search hardness and progress potential in optimization

Activation Keywords

search hardness awareness, LLM problem formulation, expensive simulation design, SHA-PF framework, rare sample prioritization, progress potential, evolutionary refinement, antenna design optimization, multi-objective benchmark, evaluation efficiency