dspy-ruby

# DSPy.rb > Build LLM apps like you build software. Type-safe, modular, testable. DSPy.rb brings software engineering best practices to LLM development. Instead of tweaking prompts, define what you want with Ruby types and let DSPy handle the rest. ## Overview DSPy.rb is a Ruby framework for building language model applications with programmatic prompts. It provides: - **Type-safe signatures** — Define inputs/outputs with Sorbet types - **Modular components** — Compose and reuse LLM logic - **Automatic optimization** — Use data to improve prompts, not guesswork - **Production-ready** — Built-in observability, testing, and error handling ## Core Concepts ### 1. Signatures Define interfaces between your app and LLMs using Ruby types: ```ruby class EmailClassifier < DSPy::Signature description "Classify customer support emails by category and priority" class Priority < T::Enum enums do Low = new('low') Medium = new('medium') High = new('high') Urgent = new('urgent') end end input do const :email_content, String const :sender, String end output do const :category, String const :priority, Priority # Type-safe enum with defined values const :confidence, Float end end ``` ### 2. Modules Build complex workflows from simple building blocks: - **Predict** — Basic LLM calls with signatures - **ChainOfThought** — Step-by-step reasoning - **ReAct** — Tool-using agents - **CodeAct** — Dynamic code generation agents (install the `dspy-code_act` gem) ### 3. Tools & Toolsets Create type-safe tools for agents with comprehensive Sorbet support: ```ruby # Enum-based tool with automatic type conversion class CalculatorTool < DSPy::Tools::Base tool_name 'calculator' tool_description 'Performs arithmetic operations with type-safe enum inputs' class Operation < T::Enum enums do Add = new('add') Subtract = new('subtract') Multiply = new('multiply') Divide = new('divide') end end sig { params(operation: Operation, num1: Float, num2: Float).returns(T.any(Float, String)) } def call(operation:, num1:, num2:) case operation when Operation::Add then num1 + num2 when Operation::Subtract then num1 - num2 when Operation::Multiply then num1 * num2 when Operation::Divide return "Error: Division by zero" if num2 == 0 num1 / num2 end end end # Multi-tool toolset with rich types class DataToolset < DSPy::Tools::Toolset toolset_name "data_processing" class Format < T::Enum enums do JSON = new('json') CSV = new('csv') XML = new('xml') end end tool :convert, description: "Convert data between formats" tool :validate, description: "Validate data structure" sig { params(data: String, from: Format, to: Format).returns(String) } def convert(data:, from:, to:) "Converted from #{from.serialize} to #{to.serialize}" end sig { params(data: String, format: Format).returns(T::Hash[String, T.any(String, Integer, T::Boolean)]) } def validate(data:, format:) { valid: true, format: format.serialize, row_count: 42, message: "Data validation passed" } end end ``` ### 4. Type System & Discriminators DSPy.rb uses sophisticated type discrimination for complex data structures: - **Automatic `_type` field injection** — DSPy adds discriminator fields to structs for type safety - **Union type support** — `T.any()` types automatically disambiguated by `_type` - **Reserved field name** — Avoid defining your own `_type` fields in structs - **Recursive filtering** — `_type` fields filtered during deserialization at all nesting levels ### 5. Optimization Improve accuracy with real data: - **MIPROv2** — Advanced multi-prompt optimization with bootstrap sampling and Bayesian optimization - **GEPA** — Genetic-Pareto Reflective Prompt Evolution with feedback maps, experiment tracking, and telemetry - **Evaluation** — Comprehensive framework with built-in and custom metrics, error handling, and batch processing ## Quick Start ```ruby # Install gem 'dspy' # Configure DSPy.configure do |c| c.lm = DSPy::LM.new('openai/gpt-4o-mini', api_key: ENV['OPENAI_API_KEY']) end # Define a task class SentimentAnalysis < DSPy::Signature description "Analyze sentiment of text" input do const :text, String end output do const :sentiment, String # positive, negative, neutral const :score, Float # 0.0 to 1.0 end end # Use it analyzer = DSPy::Predict.new(SentimentAnalysis) result = analyzer.call(text: "This product is amazing!") puts result.sentiment # => "positive" puts result.score # => 0.92 ``` ## Provider Adapter Gems Two strategies for connecting to LLM providers: ### Per-provider adapters (direct SDK access) ```ruby # Gemfile gem 'dspy' gem 'dspy-openai' # OpenAI, OpenRouter, Ollama gem 'dspy-anthropic' # Claude gem 'dspy-gemini' # Gemini ``` Each adapter gem pulls in the official SDK (`openai`, `anthropic`, `gemini-ai`). ### Unified adapter via RubyLLM (recommended for multi-provider) ```ruby # Gemfile gem 'dspy' gem 'dspy-ruby_llm' # Routes to any provider via ruby_llm gem 'ruby_llm' ``` RubyLLM handles provider routing based on the model name. Use the `ruby_llm/` prefix: ```ruby DSPy.configure do |c| c.lm = DSPy::LM.new('ruby_llm/gemini-2.5-flash', structured_outputs: true) # c.lm = DSPy::LM.new('ruby_llm/claude-sonnet-4-20250514', structured_outputs: true) # c.lm = DSPy::LM.new('ruby_llm/gpt-4o-mini', structured_outputs: true) end ``` ## Events System DSPy.rb ships with a structured event bus for observing runtime behavior. ### Module-Scoped Subscriptions (preferred for agents) ```ruby class MyAgent < DSPy::Module subscribe 'lm.tokens', :track_tokens, scope: :descendants def track_tokens(_event, attrs) @total_tokens += attrs.fetch(:total_tokens, 0) end end ``` ### Global Subscriptions (for observability/integrations) ```ruby subscription_id = DSPy.events.subscribe('score.create') do |event, attrs| Langfuse.export_score(attrs) end # Wildcards supported DSPy.events.subscribe('llm.*') { |name, attrs| puts "[#{name}] tokens=#{attrs[:total_tokens]}" } ``` Event names use dot-separated namespaces (`llm.generate`, `react.iteration_complete`). Every event includes module metadata (`module_path`, `module_leaf`, `module_scope.ancestry_token`) for filtering. ## Lifecycle Callbacks Rails-style lifecycle hooks ship with every `DSPy::Module`: - **`before`** — Runs ahead of `forward` for setup (metrics, context loading) - **`around`** — Wraps `forward`, calls `yield`, and lets you pair setup/teardown logic - **`after`** — Fires after `forward` returns for cleanup or persistence ```ruby class InstrumentedModule < DSPy::Module before :setup_metrics around :manage_context after :log_metrics def forward(question:) @predictor.call(question: question) end private def setup_metrics @start_time = Time.now end def manage_context load_context result = yield save_context result end def log_metrics duration = Time.now - @start_time Rails.logger.info "Prediction completed in #{duration}s" end end ``` Execution order: before → around (before yield) → forward → around (after yield) → after. Callbacks are inherited from parent classes and execute in registration order. ## Fiber-Local LM Context Override the language model temporarily using fiber-local storage: ```ruby fast_model = DSPy::LM.new("openai/gpt-4o-mini", api_key: ENV['OPENAI_API_KEY']) DSPy.with_lm(fast_model) do result = classifier.call(text: "test") # Uses fast_model inside this block end # Back to global LM outside the block ``` **LM resolution hierarchy**: Instance-level LM → Fiber-local LM (`DSPy.with_lm`) → Global LM (`DSPy.configure`). Use `configure_predictor` for fine-grained control over agent internals: ```ruby agent = DSPy::ReAct.new(MySignature, tools: tools) agent.configure { |c| c.lm = default_model } agent.configure_predictor('thought_generator') { |c| c.lm = powerful_model } ``` ## Evaluation Framework Systematically test LLM application performance with `DSPy::Evals`: ```ruby metric = DSPy::Metrics.exact_match(field: :answer, case_sensitive: false) evaluator = DSPy::Evals.new(predictor, metric: metric) result = evaluator.evaluate(test_examples, display_table: true) puts "Pass Rate: #{(result.pass_rate * 100).round(1)}%" ``` Built-in metrics: `exact_match`, `contains`, `numeric_difference`, `composite_and`. Custom metrics return `true`/`false` or a `DSPy::Prediction` with `score:` and `feedback:` fields. Use `DSPy::Example` for typed test data and `export_scores: true` to push results to Langfuse. ## GEPA Optimization GEPA (Genetic-Pareto Reflective Prompt Evolution) uses reflection-driven instruction rewrites: ```ruby gem 'dspy-gepa' teleprompter = DSPy::Teleprompt::GEPA.new( metric: metric, reflection_lm: DSPy::ReflectionLM.new('openai/gpt-4o-mini', api_key: ENV['OPENAI_API_KEY']), feedback_map: feedback_map, config: { max_metric_calls: 600, minibatch_size: 6 } ) result = teleprompter.compile(program, trainset: train, valset: val) optimized_program = result.optimized_program ``` The metric must return `DSPy::Prediction.new(score:, feedback:)` so the reflection model can reason about failures. Use `feedback_map` to target individual predictors in composite modules. ## Typed Context Pattern Replace opaque string context blobs with `T::Struct` inputs. Each field gets its own `description:` annotation in the JSON schema the LLM sees: ```ruby class NavigationContext < T::Struct const :workflow_hint, T.nilable(String), description: "Current workflow phase guidance for the agent" const :action_log, T::Array[String], default: [], description: "Compact one-line-per-action history of research steps taken" const :iterations_remaining, Integer, description: "Budget remaining. Each tool call costs 1 iteration." end class ToolSelectionSignature < DSPy::Signature input do const :query, String const :context, NavigationContext # Structured, not an opaque string end output do const :tool_name, String const :tool_args, String, description: "JSON-encoded arguments" end end ``` Benefits: type safety at compile time, per-field descriptions in the LLM schema, easy to test as value objects, extensible by adding `const` declarations. ## Schema Formats (BAML / TOON) Control how DSPy describes signature structure to the LLM: - **JSON Schema** (default) — Standard format, works with `structured_outputs: true` - **BAML** (`schema_format: :baml`) — 84% token reduction for Enhanced Prompting mode. Requires `sorbet-baml` gem. - **TOON** (`schema_format: :toon, data_format: :toon`) — Table-oriented format for both schemas and data. Enhanced Prompting mode only. BAML and TOON apply only when `structured_outputs: false`. With `structured_outputs: true`, the provider receives JSON Schema directly. ## Storage System Persist and reload optimized programs with `DSPy::Storage::ProgramStorage`: ```ruby storage = DSPy::Storage::ProgramStorage.new(storage_path: "./dspy_storage") storage.save_program(result.optimized_program, result, metadata: { optimizer: 'MIPROv2' }) ``` Supports checkpoint management, optimization history tracking, and import/export between environments. ## Rails Integration ### Directory Structure Organize DSPy components using Rails conventions: ``` app/ entities/ # T::Struct types shared across signatures signatures/ # DSPy::Signature definitions tools/ # DSPy::Tools::Base implementations concerns/ # Shared tool behaviors (error handling, etc.) modules/ # DSPy::Module orchestrators services/ # Plain Ruby services that compose DSPy modules config/ initializers/ dspy.rb # DSPy + provider configuration feature_flags.rb # Model selection per role spec/ signatures/ # Schema validation tests tools/ # Tool unit tests modules/ # Integration tests with VCR vcr_cassettes/ # Recorded HTTP interactions ``` ### Initializer ```ruby # config/initializers/dspy.rb Rails.application.config.after_initialize do next if Rails.env.test? && ENV["DSPY_ENABLE_IN_TEST"].blank? RubyLLM.configure do |config| config.gemini_api_key = ENV["GEMINI_API_KEY"] if ENV["GEMINI_API_KEY"].present? config.anthropic_api_key = ENV["ANTHROPIC_API_KEY"] if ENV["ANTHROPIC_API_KEY"].present? config.openai_api_key = ENV["OPENAI_API_KEY"] if ENV["OPENAI_API_KEY"].present? end model = ENV.fetch("DSPY_MODEL", "ruby_llm/gemini-2.5-flash") DSPy.configure do |config| config.lm = DSPy::LM.new(model, structured_outputs: true) config.logger = Rails.logger end # Langfuse observability (optional) if ENV["LANGFUSE_PUBLIC_KEY"].present? && ENV["LANGFUSE_SECRET_KEY"].present? DSPy::Observability.configure! end end ``` ### Feature-Flagged Model Selection Use different models for different roles (fast/cheap for classification, powerful for synthesis): ```ruby # config/initializers/feature_flags.rb module FeatureFlags