Evaluate retrieval and generation quality in RAG pipelines. Separate scoring for retrieval (recall, precision, MRR) and generation (faithfulness, relevance, completeness). End-to-end pipeline assessment with bottleneck identification. Triggers on: "eval rag",…
Design binary pass/fail LLM-as-Judge evaluators. Structured prompt engineering for evaluation: criteria definition, rubric construction, few-shot calibration, and bias mitigation. Produces a ready-to-deploy judge prompt with scoring instructions. Triggers on:…
Language Agent Tree Search - Monte Carlo planning - 92.7% on HumanEval
Self-learning system based on SICA, VIGIL, and Mem0 patterns. Auto-triggers after task completion. Captures patterns, consolidates memory, generates skills, recovers from errors.
Self-improving prompts through meta-level optimization
Optimize token usage through prompt caching and compression
Thought-Action-Observation loop for transparent reasoning
Self-correction via critique loop - 18.5 percentage point improvement