execution-feedback-driven-iterative-refinement

# Skill: execution-feedback-driven-iterative-refinement ## 1. Capability Definition & Real Case * **Professional Definition**: The ability to plan, generate, and execute an action sequence within a closed-loop environment, subsequently performing a dynamic 'Error -> Reflection -> Correction' cycle based on execution feedback (such as compiler errors, API failure logs, or numeric optimization scores). This capability encompasses meta-verification of reasoning trajectories to ensure step-wise accuracy and the adaptive discovery of hidden environment parameters through interactive tool probing. * **Dimension Hierarchy**: Open-World Real-World Planning -> Technical Research and Optimization -> execution-feedback-driven-iterative-refinement ### Real Case **[Case 1]** * **Initial Environment**: A user order-retrieval system with two APIs: one for listing order history (get_user_orders) and another for fetching details (get_order_details). The documentation is incomplete, missing the specific user-identification parameter for the history list and utilizing an outdated parameter name for the details tool. * **Real Question**: Can you fetch the details of my recent orders? I need to know the products, status, and delivery date. * **Real Trajectory**: 1. [Tool Planning] Attempt to fetch history using `get_user_orders()`. 2. [Observation] Receive error: 'Input parameters missing; provide user ID'. 3. [Reflection] Identify the missing constraint and decide to provide `user_id`. 4. [Tool Invocation] Execute `get_user_orders(user_id='123')`. 5. [Observation] Success, found Order #456. 6. [Tool Invoation] Execute `get_order_details(is_id='456')` based on documentation. 7. [Observation] Error: 'Invalid parameter is_id'. 8. [Reflection] Hypothesize the parameter name evolved and try `order_id` as an integer. 9. [Tool Invocation] Execute `get_order_details(order_id=456)`. 10. [Observation] Success, retrieved status 'Shipped'. * **Real Answer**: Your recent order (ID: 456) for the Apple iPhone 12 is currently 'Shipped'. * **Why this demonstrates the capability**: This case demonstrates iterative refinement because the agent encountered multiple execution failures (missing parameters and outdated naming) and used each specific error message as a 'learning signal' to adjust its tool invocation strategy. It proves the agent can navigate production environments with imperfect documentation by applying a deliberate reflection loop rather than simply giving up. --- **[Case 2]** * **Initial Environment**: A weather forecasting tool `get_weather_forecast` that requires a specific date format. The user provided the date in a natural language format ('Christmas Day') which is not recognized by the backend server. * **Real Question**: What is the weather forecast for London on Christmas Day 2024? * **Real Trajectory**: 1. [Thought] I will query the weather for London on December 25, 2024. 2. [Action] `get_weather_forecast(city='London', date='December 25, 2024')`. 3. [Observation] 'Invalid date format. Use YYYY-MM-DD format'. 4. [Reflection] The tool feedback explicitly specifies the YYYY-MM-DD pattern. I must convert 'Christmas Day' to '2024-12-25'. 5. [Action] `get_weather_forecast(city='London', date='2024-12-25')`. 6. [Observation] Success: 'Light Snow, 2°C'. * **Real Answer**: On Christmas Day 2024, London will experience light snow with a temperature of 2°C. * **Why this demonstrates the capability**: This demonstrates execution-feedback-driven refinement by showing an 'Error -> Reflection -> Correction' paradigm. The agent recognizes its initial invocation failed due to a format constraint provided in the feedback and re-invokes the tool with corrected parameters to achieve the goal. ## Pipeline Execution Instructions To synthesize data for this capability, you must strictly follow a 3-phase pipeline. **Do not hallucinate steps.** Read the corresponding reference file for each phase sequentially: 1. **Phase 1: Environment Exploration** Read the exploration guidelines to discover raw knowledge seeds: `references/EXPLORATION.md` 2. **Phase 2: Trajectory Selection** Once Phase 1 is complete, read the selection criteria to evaluate the trajectory: `references/SELECTION.md` 3. **Phase 3: Data Synthesis** Once a trajectory passes Phase 2, read the synthesis instructions to generate the final data: `references/SYNTHESIS.md`