Implement comprehensive evaluation strategies for LLM applications using automated metrics, human feedback, and benchmarking. Use when testing LLM performance, measuring AI application quality, or establishing evaluation frameworks.
Skills in this repository
majiayu000/claude-skill-registry - Page 22
SkillsMP has collected 5,417 skills from majiayu000/claude-skill-registry. Open a skill to review its source and details.
majiayu000/claude-skill-registryShowing 40 of 5,417 collected skills.
Implement comprehensive evaluation strategies for LLM applications using automated metrics, human feedback, and benchmarking. Use when testing LLM performance, measuring AI application quality, or establishing evaluation frameworks.
Implementing function calling (tool use) with LLMs for structured outputs and external integrations.
LLM gateway and routing configuration using OpenRouter and LiteLLM. Invoke when: - Setting up multi-model access (OpenRouter, LiteLLM) - Configuring model fallbacks and reliability - Implementing cost-based or latency-based routing - A/B testing different…
LLM integration patterns for Claude, GPT, Gemini, and Ollama. Activate for AI API integration, prompt engineering, token management, extended thinking, and multi-model orchestration.
Guide for using LLM utilities in speedy_utils, including memoized OpenAI clients and chat format transformations.
Comprehensive guide to using LLMs as judges for automated evaluation including prompt patterns, calibration, bias reduction, and multi-judge ensembles
Comprehensive guide for deploying LLMs locally using Ollama, vLLM, and llama.cpp. Local deployment offers privacy, cost control, and reduced latency compared to cloud APIs. This skill covers everythin
Local LLM health checks and cache management. Probe Ollama/vLLM/SGLang endpoints, clean model caches.
Pydantic-AI agents, RAG, embeddings for Pulse Radar knowledge extraction.
Build production LLM streaming UIs with Server-Sent Events, real-time token display, cancellation, error recovery. Handles OpenAI/Anthropic/Claude streaming APIs. Use for chatbots, AI assistants, real-time text generation. Activate on "LLM streaming", "SSE",…
LLM streaming response patterns. Use when implementing real-time token streaming, Server-Sent Events for AI responses, or streaming with tool calls.
See the main LLM Cost Optimization skill for comprehensive coverage of token economics and optimization strategies.
LLM Text Protocol provides standardized patterns for text-based interactions with language models. It encompasses prompt engineering, response formatting, and text processing patterns that ensure cons
Detect and use llms.txt files for LLM-optimized documentation. Use when checking if a site has LLM-ready docs before scraping.
Guides users through the process of preparing datasets and fine-tuning local Large Language Models (LLMs) using techniques like LoRA and QLoRA.
Local Retrieval-Augmented Generation with vector search.
Use for Pydantic Logfire observability, tracing, and debugging. Query exceptions, spans, logs with SQL. View traces in Logfire UI. 4 tools for application monitoring and error analysis.
Use when processing documents over 50K characters through LLM APIs with section splitting and batch cost reduction.
Detect behavioral drift and alignment degradation over time.
Library documentation via Context7. Use for API references, code examples, framework docs.
Parameter-efficient fine-tuning with Low-Rank Adaptation (LoRA). Use when fine-tuning large language models with limited GPU memory, creating task-specific adapters, or when you need to train multiple specialized models from a single base.
Imported skill lora_ofl from anthropic
Master AI-powered natural language data exploration with Lumen AI. Use this skill when building conversational data analysis interfaces, enabling natural language queries to databases, creating custom AI agents for domain-specific analytics, implementing RAG…
Transform vague inputs into precision-optimized AI prompts for Claude, ChatGPT, Gemini, or other LLMs. Use when user mentions "optimize prompt", "improve prompt", "lyra", "prompt engineering", or needs help crafting effective AI prompts.
Provides score interpretation (< 0.3 good, > 0.5 skip), query optimization, and source naming for query_documents, ingest_file, ingest_data tools. Use this skill when working with RAG, searching documents, ingesting files, saving web content, or handling PDF,…
AI self-improvement analyst. Tracks AI agent mistakes, analyzes failure patterns, and proposes system improvements. Implements continuous learning loop for trading system enhancement.
Source text: Korean
Analyze and improve the improvement process. Use for detecting regressions and meta-optimization.
Optimal patterns for MiniMax MCP tools (web_search + understand_image)
Nano-Banana AI service integration for content generation, image creation, and AI-powered workflows. Use when integrating AI services for content creation.
Inspect or change Babysitter model-routing policy by phase.
Use when validating model downgrades for skills or agents. Runs A/B comparison between current and proposed model, scores outputs against test criteria, produces verdict.
模型自动降级与故障切换。当主模型请求失败、超时、达到速率限制或配额耗尽时,自动切换到备用模型,确保服务连续性。支持多供应商、多优先级的智能模型选择,提供健康监控、自动重试和错误恢复机制。
Source text: Chinese
CLI commands to add, update, list, and deprecate AI models in the database. Triggers on "add model", "new model", "update model", "deprecate model", "model pricing", "model cli".
Test, validate, and add new AI models to the eval suite. Use when user asks to add new models, test model access, check pricing, or update models.yml.
Route work to the cheapest model tier that can reliably complete it.
Model selection strategy for subagents — haiku for exploration, sonnet for implementation, opus for architecture/security
Switch the agent's model at runtime and restart. Use when the user asks to change models (e.g., "switch to kimi", "use opus", "go cheaper"), or when the agent decides a different model is better for the current workload.
Step-by-step guidance for model usage.
Use CodexBar CLI local cost usage to summarize per-model usage for Codex or Claude, including the current (most recent) model or a full model breakdown. Trigger when asked for model-level usage/cost data from codexbar, or when you need a scriptable per-model…