n8n-automation

name	n8n-automation
description	n8n workflow automation for building analytics including SkySpark multi-agent systems, FastAPI tool servers, workflow orchestration, and automated building system alert triage

n8n Automation Skill

This skill provides expertise in building automation workflows using n8n, specifically focused on SkySpark integration and multi-agent systems for building analytics.

Core Capabilities

SkySpark Multi-Agent Architecture

• Gatekeeper agent with specialist routing
• Automated spark triage and prioritization
• HVAC specialist for mechanical system analysis
• Energy calculation agent for savings estimates
• Automated report generation workflows

n8n Workflow Development

• REST API integrations with building systems
• Multi-agent coordination patterns
• Error handling and logging strategies
• Scheduled automation and notifications

SkySpark Integration

• Haystack REST API connectivity
• ZINC and JSON data format handling
• Spark and trend data extraction
• Authentication and security patterns

Available Scripts

• scripts/fastapi_tool_server.py: Template for FastAPI tool servers
• scripts/n8n_workflow_templates/: Common workflow patterns
• scripts/skyspark_mock_data.py: Generate test data for development

Reference Materials

• references/multi_agent_patterns.md: Proven multi-agent architectures
• references/skyspark_api.md: SkySpark Haystack API documentation
• references/n8n_best_practices.md: Workflow development guidelines
• references/roadmap.md: 5-phase project roadmap (Foundation → Production)

Asset Templates

• assets/workflow_exports/: n8n workflow export files
• assets/api_schemas/: OpenAPI schemas for tool servers
• assets/mock_data/: Sample SkySpark data for testing

Current Project: SkySpark Multi-Agent System

5-Phase Roadmap

1. Foundation: FastAPI tool server + basic agent interaction
2. Mock Integration: Agent logic with simulated SkySpark data
3. Real SkySpark: Live Haystack API connection
4. Multi-Agent: Full specialist coordination
5. Production: Error handling, scheduling, notifications

Multi-Agent Pattern

User Request → Gatekeeper Agent (routes by intent)
                ↓
    ┌───────────┼───────────┬──────────────┐
    ↓           ↓           ↓              ↓
Triage      HVAC        Energy         Report
Agent       Specialist  Calc Agent     Generator

Agent Architecture for Future RL Orchestration

Stateless Agent Design Principles

All SkySpark agents should be designed with future dynamic orchestration in mind:

Statelessness Requirements:

• Agents receive complete diagnostic context as input (no persistent memory)
• All necessary context passed in, new state returned out
• Enables any agent to follow any agent (maximum routing flexibility)
• Simplifies debugging and testing

Standardized I/O Schema:

Input: DiagnosticState object containing:

• alert_context: Current alert/issue details (Haystack tags, spark data)
• prior_actions: Array of prior agent outputs/reasoning
• available_tools: List of tool endpoints and capabilities
• token_budget_remaining: Tokens available for continued diagnosis
• confidence_score: Current diagnostic confidence (0.0-1.0)
• metadata: Alert severity, facility info, timestamps

Output: AgentResponse object containing:

• updated_state: Refreshed DiagnosticState with new findings
• agent_findings: This agent's analysis, calculations, or actions
• recommended_next_agents: Suggested routing (even if not used in Phase 0-1)
• token_cost: Tokens consumed by this agent call
• confidence_delta: Change in diagnostic confidence (+/- 0.0-1.0)
• actions_taken: List of tools called or queries executed

Role-Based Architecture: Separate agent identity (role prompts) from action logic:

Tool-Use Agents:

• SkySpark_Query_Agent: Execute Axon queries against SkySpark database
• Python_Analysis_Agent: Statistical analysis, calculations, trending
• File_Reader_Agent: Parse equipment specs, CSV logs, JSON configs
• Weather_Data_Agent: Fetch external weather data via API
• Documentation_Agent: Search equipment manuals, ASHRAE standards

Reasoning Agents:

• Diagnostic_Planner_Agent: Decompose complex issues into sub-diagnostics
• Root_Cause_Analyzer_Agent: Synthesize data to identify probable causes
• Validation_Agent: Verify diagnostic logic and calculations
• Reflection_Agent: Assess diagnostic trajectory, propose refinements
• Summary_Agent: Generate concise diagnostic reports
• Resolution_Agent: Recommend corrective actions
• Modification_Agent: Correct errors in prior reasoning

Orchestration Evolution Roadmap

Phase 0-1: Static n8n Workflows (Current)

• Hardcoded agent routing via n8n nodes is acceptable
• BUT design workflows to call centralized routing functions
• Log every routing decision with full context
• Focus: Get agents working with standardized I/O

Phase 2-3: Rule-Based Orchestrator

• Implement agent_router function (Python/FastAPI endpoint)
• Route based on: alert type/severity, confidence score, prior sequence
• Manually tune rules based on logged diagnostic patterns
• A/B test against static workflows
• Focus: Data-driven routing without ML complexity

Phase 4-5: RL-Based Orchestrator

• Replace rule-based router with learned policy
• Train on logged diagnostic trajectories (minimum 200 episodes)
• Optimize for: R = accuracy - λ × token_cost - β × time_penalty
• Enable online learning from production outcomes
• Focus: Continuous optimization and adaptation

Instrumentation & Logging Requirements

Critical: Start logging from Day 1, even with static workflows

Log every diagnostic episode with schema:

{
  "episode_id": "alert_123456_2025-12-02",
  "facility_name": "Building XYZ",
  "alert_context": {
    "alert_type": "High AHU static pressure",
    "severity": "warning",
    "haystack_refs": [...],
    "triggered_at": "2025-12-02T10:15:00Z"
  },
  "agent_sequence": [
    {
      "timestamp": "2025-12-02T10:30:00Z",
      "agent_type": "SkySpark_Query_Agent",
      "input_state": {...},
      "output_state": {...},
      "tokens_used": 1500,
       
        
    
    ...
  
   
     
     
     
     
     
     
     
  

Storage Strategy:

• Store in n8n database OR dedicated TimescaleDB
• Retain minimum 200 episodes per alert type for RL training
• Enable easy export for offline analysis and model training
• Version control state schemas alongside agents

What to Log:

• Complete agent call sequence per diagnostic episode
• Full input/output states at each step
• Token consumption per agent (measure via API or estimate)
• Human validation/corrections (critical for reward labels)
• Final diagnostic outcome and resolution effectiveness
• Time to resolution relative to SLA thresholds

Reward Function Design

Preliminary specification for future RL training:

R = accuracy_score - λ × token_cost - β × time_penalty

Where:

• accuracy_score ∈ [0, 1] - Validated against human review OR resolution outcome
• token_cost = tokens_used / max_token_budget - Normalized efficiency metric
• time_penalty = max(0, (time_to_resolve / sla_threshold) - 1) - Timeliness factor
• λ = 0.1 - Efficiency weight (tunable based on production costs)
• β = 0.05 - Timeliness weight (varies by alert severity)

Accuracy Validation:

• Binary: Did diagnostic identify correct root cause? (0 or 1)
• Graduated: Partial credit for narrowing to subsystem (0.0-1.0 scale)
• Ground truth: FM confirmation or system behavior post-resolution

Efficiency Considerations:

• Critical alerts: Prioritize accuracy over token cost (λ → 0)
• Informational alerts: Prioritize efficiency (λ → 0.2)
• Balance prevents both over-analysis and premature conclusions

Timeliness Thresholds:

• Critical: 15 min SLA → high β penalty if exceeded
• Warning: 60 min SLA → moderate β penalty
• Info: 4 hour SLA → minimal β penalty

SkySpark Agent Taxonomy

Agents aligned with Puppeteer research categories:

Tool-Use Agents (External data access):

1. SkySpark_Query_Agent - Axon query execution
2. Python_Analysis_Agent - Statistical calculations, trending
3. File_Reader_Agent - Equipment specs, logs, configs
4. Weather_Data_Agent - External weather APIs
5. Documentation_Agent - Manuals, standards, knowledge base

Reasoning Agents (Internal cognition):

1. Diagnostic_Planner_Agent - Task decomposition
2. Root_Cause_Analyzer_Agent - Data synthesis
3. Validation_Agent - Logic verification
4. Reflection_Agent - Meta-reasoning
5. Summary_Agent - Report generation
6. Resolution_Agent - Action recommendation
7. Modification_Agent - Error correction

Critical Don'ts

Avoid these pitfalls:

• ❌ Don't jump to RL before having logged data (minimum 200 episodes)
• ❌ Don't hardcode agent sequences into prompts (keep routing external)
• ❌ Don't skip instrumentation "for now" (you'll never add it later)
• ❌ Don't design agents that assume specific predecessors
• ❌ Don't sacrifice debuggability for sophistication
• ❌ Don't optimize prematurely (get working system first)
• ❌ Don't force-fit RL where rules work fine

Do these instead:

• ✅ Keep current n8n approach for Phase 0-1 (it's pragmatic)
• ✅ Design agents with standardized I/O from day 1
• ✅ Start logging immediately, even with static workflows
• ✅ Validate every agent output initially (build ground truth dataset)
• ✅ Build tooling to visualize agent sequences (aid debugging)
• ✅ Document reward function assumptions (evolve with data)
• ✅ Plan human override mechanisms (safety net)

Success Metrics by Phase

Phase 1 (Static n8n + Logging):

• 5+ agents implemented with standardized I/O
• 100% of diagnostic episodes logged with schema
• Static workflow resolves 70%+ of test alerts correctly
• Average 8-12k tokens per diagnostic episode

Phase 2 (Rule-Based Router):

• Rule-based router handles 80%+ of alert types
• 10% reduction in average tokens vs. static workflow
• Diagnostic accuracy maintained or improved (≥70%)
• Routing decisions explainable to FM stakeholders

Phase 3 (RL Orchestrator):

• RL policy trained on 200+ episodes
• Matches or exceeds rule-based accuracy
• 20%+ token reduction vs. static baseline
• Cyclic reasoning patterns emerge for complex diagnostics

Phase 4 (Production RL):

• Online learning maintains 90%+ diagnostic accuracy
• 30%+ token reduction vs. baseline
• Handles novel alert types through adaptation
• Human intervention required <10% of cases

Integration with Existing Architecture

Compatible with current project structure:

• Project instructions: Project-Specific-Instructions/SkySpark-n8n-Workflow.md
• n8n workflows: n8n/
• Agent design aligns with FastAPI tool server architecture
• State schema accommodates SkySpark Haystack data model
• Reward function maps to ComEd program requirements

Backward Compatibility:

• Phase 0-1 static workflows remain fully functional
• Standardized I/O is additive, not breaking
• Logging infrastructure runs alongside existing logic
• Can defer RL indefinitely if rule-based routing suffices

Usage Examples

• "Build n8n workflow to triage SkySpark alerts"
• "Create FastAPI server for energy calculations"
• "Set up multi-agent routing for HVAC diagnostics"
• "Design workflow for automated building system reports"
• "Integrate n8n with SkySpark Haystack API"
• "Design stateless agent with standardized I/O for future RL"
• "Set up diagnostic episode logging infrastructure"
• "Implement rule-based agent router for Phase 2"

---

Model Selection Constraints (Added 2025-01-01)

Hard Requirement: US/EU Origin Only

Company works with US government - no Chinese-origin models permitted.

This excludes: Qwen, DeepSeek, Yi, Baichuan, ChatGLM, and derivatives.

Approved Local Models

Agent Role	Model	Origin	Size	Rationale
Classifier	Microsoft Phi-4	🇺🇸 US	14B	Fast, accurate structured output, MIT license
Test Designer	Mistral Codestral 22B	🇫🇷 EU	22B	Purpose-built for code
Physics Explainer	Meta Llama 3.1 70B	🇺🇸 US	70B	Best open reasoning model
Reporter	Mistral Small 24B	🇫🇷 EU	24B	High quality summaries
Triage (local option)	Meta Llama 3.1 8B	🇺🇸 US	8B	Good balance for routine alerts
Coder Agent	Claude API	🇺🇸 US	-	Best for code generation (keep on API)

Alternative Smaller Models (CPU/Light GPU)

• Microsoft Phi-4-mini-reasoning (3.8B) - Classification tasks
• Mistral 7B Instruct v0.3 - General purpose
• Meta CodeLlama-7B-Instruct - Code understanding

---

Physics Checker Design (Added 2025-01-01)

Two-Layer Architecture

Layer 1: Rule Engine (Python, deterministic)

• ASHRAE bounds checking
• Equipment capacity limits
• Thermodynamic constraints (energy balance)
• Returns: PASS/FAIL + which rule violated
• No LLM involved - pure Python logic

Layer 2: Explanation Agent (local LLM, only if FAIL)

• Takes: rule violation details + equipment context
• Returns: human-readable explanation of why it's impossible
• Model: Meta Llama 3.1 8B Instruct (US origin)
• Only invoked when Layer 1 fails

Initial Physics Bounds

PHYSICS_BOUNDS = {
    "zone_temp": {"min": 55, "max": 85, "unit": "°F"},
    "supply_air_temp": {"min": 50, "max": 65, "unit": "°F"},
    "chilled_water_delta_t": {"min": 8, "max": 16, "unit": "°F"},
    "hot_water_delta_t": {"min": 15, "max": 40, "unit": "°F"},
    "airflow_per_sqft": {"min": 0.5, "max": 3.0, "unit": "CFM/sqft"},
    "cooling_efficiency": {"min": 0.3, "max": 1.0, "unit": "kW/ton"},
    "boiler_efficiency": {"min": 0.75, "max": 0.98, "unit": "fraction"},
    "fan_static_pressure": {"min": 0.5, "max": 6.0, "unit": "inWG"},
}

Integration with Agent Taxonomy

Physics Checker slots into the Reasoning Agents category:

• Input: Recommendation from Triage Agent + equipment context
• Output: PASS (proceed) or FAIL (block + explain)
• Invoked before any recommendation reaches the Reporter

---

Calculation Tool Verification Pipeline (Added 2025-01-01)

Core Principle: Agents Don't Do Math

• All calculations performed by human-approved Python scripts
• Scripts wrapped as callable tools for agents
• Results are auditable and reproducible
• Minimizes token count and eliminates hallucinated math

Verification Workflow

Coder Agent (Claude) → Test Designer (Codestral) → Static Analyzer (AST)
       ↓                        ↓                         ↓
  Initial script         Edge cases, bounds         Security check
                         Reference values           Purity check
                                                    Type hints
       ↓                        ↓                         ↓
       └────────────────────────┴─────────────────────────┘
                                ↓
                    Test Executor (pytest)
                                ↓
                    Documentation Generator
                                ↓
                    Human Review Package
                    [Approve / Reject / Edit]

Verification Types (Diversity = Trust)

Type	What It Catches	Implementation
Generated tests	Logic errors, edge cases	Local LLM (Codestral 22B)
Reference tests	Wrong formulas	Human-provided ASHRAE examples
Static analysis	Security, side effects	Python AST parsing
Type checking	Interface mismatches	mypy
Unit conversion	Dimensional errors	pint library
Bounds checking	Impossible outputs	Physics Checker rules

CalcTool Registration Schema

@dataclass
class CalcTool:
    id: str
    name: str                    # e.g., "chw_energy"
    version: str                 # Semantic versioning
    description: str
    script_path: str             # Path to .py file
    function_name: str           # Entry point
    
    # Interface
    input_schema: dict           # JSON schema for inputs
    output_schema: dict          # JSON schema for outputs
    
    # Approval chain
    approved_by: str             # Human who reviewed
    approved_at: datetime
    test_cases: List[dict]       # Input/output pairs that must pass
    
    # Usage tracking
    invocation_count: int
    last_invoked: datetime
    error_count: int

---

Value Economy System (Added 2025-01-01)

Core Concept

Treat diagnostic strategies as an economy:

• Strategies have value scores that change based on outcomes
• Usefulness pays rent; wrongness gets evicted
• High-value strategies get promoted to Skills
• Unused strategies decay over time

Strategy Artifact Schema

@dataclass
class StrategyArtifact:
    id: str
    created_at: datetime
    updated_at: datetime
    
    # What it matches
    alert_pattern: dict               # Conditions this strategy applies to
    building_types: List[str]         # Hospital, university, office, etc.
    system_types: List[str]           # AHU, VAV, chiller, etc.
    
    # What it recommends
    diagnosis_template: str
    action_template: str
    calc_tools: List[str]             # Which calculation scripts to invoke
    
    # Value economy
    status: Literal["candidate", "validated", "skill"]
    value_score: float
    confidence: float
    
    # Evidence counters
    times_matched: int
    times_confirmed: int              # Human said "yes, correct"
    times_rejected: int               # Human said "no, wrong"
    energy_saved_kbtu: float          # Cumulative attributed savings
    last_used: datetime
    
    # Lineage
    source: Literal["human", "synthesized", "promoted"]
    parent_ids: List[str]

Promotion Gates

Candidate → Validated:

VALIDATION_THRESHOLDS = {
    "min_matches": 3,
    "min_confirmations": 2,
    "max_rejection_rate": 0.3,
    "min_distinct_buildings": 2,
    "physics_check_pass": True,
}

Validated → Skill:

SKILL_PROMOTION_THRESHOLDS = {
    "min_matches": 10,
    "min_confirmations": 7,
    "max_rejection_rate": 0.15,
    "min_distinct_buildings": 3,
    "min_age_days": 14,
    "attributed_savings_kbtu": 1000,
}

Decay Function

def calculate_decay(artifact: StrategyArtifact, days_since_use: int) -> float:
    """Value decays 2% per day of non-use, floor at 0.1"""
    decay_factor = 0.98 ** days_since_use
    return max(artifact.value_score * decay_factor, 0.1)

Value Update Logic

REWARDS = {
    "confirmed_correct": 1.0,
    "energy_saved_per_1000_kbtu": 0.5,
    "reused_different_building": 0.3,
}

PENALTIES = {
    "rejected_wrong": -2.0,
    "physics_violation": -3.0,
    "caused_comfort_complaint": -1.5,
}

---

Governance Layer (Added 2026-01-05)

Core Principle

Derive "goodness" from observable deltas, not model judgments.

The governance layer monitors agent and strategy performance through measurable signals, uses a small local LLM for classification only, and modifies routing weights and configuration—never prompts or training data.

Architecture

┌─────────────────────────────────────────────────────────────┐
│                    TELEMETRY LAYER                          │
│  Captures: latency, retries, overrides, tool failures,     │
│           user edits, session completion                    │
└────────────────────────┬────────────────────────────────────┘
                         ↓
┌─────────────────────────────────────────────────────────────┐
│              CLASSIFICATION ENGINE (Ollama)                 │
│  Model: Llama 3.1 8B Instruct (Q8)                         │
│  Task: Categorize failure types, edit substantiveness      │
│  Output: Strict JSON enums only (no explanatory text)      │
└────────────────────────┬────────────────────────────────────┘
                         ↓
┌─────────────────────────────────────────────────────────────┐
│           SCORE CALCULATOR (Deterministic)                  │
│  Computes health scores from weighted signals              │
│  NO LLM INVOLVED - pure Python logic                       │
└────────────────────────┬────────────────────────────────────┘
                         ↓
┌─────────────────────────────────────────────────────────────┐
│                  CONFIG MUTATOR                             │
│  Writes to: routing weights, canary patterns, deprecation  │
│  NEVER modifies: agent prompts, training data              │
└─────────────────────────────────────────────────────────────┘

Measurable Signals (Observable Deltas)

What we measure:

• User acceptance: Did user accept recommendation without edits?
• Edit size: How much did user modify agent output? (character delta)
• Override frequency: How often did user reject and provide alternative?
• Session completion: Did diagnostic episode complete successfully?
• Latency: Time to produce recommendation
• Retry count: How many attempts before acceptable output?
• Tool failures: Number of failed tool calls during episode

What we DON'T measure:

• ❌ LLM judgment of "quality"
• ❌ Subjective ratings
• ❌ Vibes-based assessment

Classification Engine Specs

Model: Llama 3.1 8B Instruct (Q8 quantization) Deployment: Ollama (local) Temperature: 0.0 (deterministic) Max Tokens: 50 Output Format: Strict JSON schema with enums only

Example Classification Task:

{
  "edit_type": "substantive" | "formatting" | "none",
  "failure_mode": "incorrect_diagnosis" | "missing_context" | "tool_failure" | "physics_violation" | "null",
  "confidence": 0.0-1.0
}

Acceptable Failures:

• Misclassification (wrong enum selected)
• Refusal to classify (null output)

Unacceptable Failures:

• Hallucinated categories not in schema
• Explanatory text instead of enum
• Non-JSON output

Health Score Calculation (Deterministic)

health_score = (
    success_rate * 0.40 +
    non_replacement_rate * 0.25 +
    non_override_rate * 0.20 +
    completion_rate * 0.15
)

Where:

• success_rate = accepted recommendations / total recommendations
• non_replacement_rate = 1 - (substantive edits / total recommendations)
• non_override_rate = 1 - (explicit rejections / total recommendations)
• completion_rate = completed episodes / started episodes

Thresholds:

• Reinforce (≥0.80): Increase routing weight, consider promotion
• Stable (0.50-0.80): Maintain current routing
• Decay Warning (<0.50): Flag for review, reduce routing weight
• Deprecation Candidate (<0.35 for 14+ days): Human review required

Strategy Lifecycle Management

Canary Pattern (Gradual Rollout):

New Strategy → 10% of matching alerts
  ↓ (5 successful uses)
50% of matching alerts
  ↓ (20 successful uses)
100% of matching alerts (promoted)

Deprecation Workflow:

1. Automated flag when health <0.35 for 14+ days
2. Generate deprecation report with metrics
3. Human review REQUIRED before archiving
4. Archive strategy with audit log

Skill Cap: Maximum 50 active strategies

• Prevents strategy sprawl
• Forces prioritization and consolidation
• Must deprecate before adding when at cap

Phased Implementation

Phase 1: Telemetry Foundation (2-3 weeks)

• Add telemetry emission endpoints to tool_server.py
• Define event schema (JSON)
• Create append-only event store (SQLite or JSONL)
• Instrument agent call sites with telemetry
• Verify data collection end-to-end

Phase 2: Deterministic Scoring (1 week)

• Implement health score calculation (pure Python)
• Create scoring dashboard/report script
• Test with historical data (if available)
• Define threshold policies

Phase 3: Classification Engine (1-2 weeks)

• Deploy Ollama locally
• Install Llama 3.1 8B Instruct
• Create classification service (FastAPI endpoint)
• Define JSON schemas for classification tasks
• Test enum-only output enforcement
• Integrate with telemetry pipeline

Phase 4: Config Mutation (1 week)

• Implement routing weight adjustment logic
• Create canary pattern manager
• Build deprecation workflow with human approval
• Add audit logging for all config changes
• Test end-to-end governance loop

Integration Points

SkySpark n8n Workflow:

• Monitor agent tool calls via telemetry
• Capture user acceptance/rejection signals
• Log session completion status

Tool Server (FastAPI):

• Add /telemetry/emit endpoint
• Add /governance/health-score endpoint
• Add /governance/classify endpoint (calls Ollama)

Event Store Schema:

{
  "event_id": "uuid",
  "timestamp": "2026-01-05T10:30:00Z",
  "event_type": "agent_recommendation" | "user_feedback" | "session_complete",
  "agent_id": "triage_agent_v1",
  "strategy_id": "ahu_vfd_savings_v2",
  "signals": {
    "latency_ms": 3400,
    "retry_count": 0,
    "tool_failures": 0,
    "user_accepted": true,
    "edit_delta_chars": 0,
    "override": false,
    "session_completed": true
  },
   
     
     
  

Observability Stack (LangSmith + Telemetry)

Two-Layer Observability:

1. LangSmith - Agent execution tracing and evaluation

   • Traces all LangGraph agent calls within n8n workflows
   • Captures input/output for each agent step
   • Evaluation metrics for agent performance
   • Token usage tracking per agent call

2. Governance Telemetry - Outcome measurement

   • User acceptance signals
   • Edit deltas and overrides
   • Session completion status
   • Feeds into health score calculation

Integration Architecture:

n8n Workflow
  ↓
LangGraph Agent Node
  ├─→ LangSmith (trace execution, token costs)
  └─→ Governance Telemetry (emit outcome signals)
        ↓
  Event Store → Health Score → Config Mutation

LangSmith Setup:

# Install LangSmith
pip install langsmith langchain-anthropic

# Set API key
export LANGCHAIN_TRACING_V2=true
export LANGCHAIN_API_KEY="your-key-here"
export LANGCHAIN_PROJECT="skyspark-alert-triage"

LangGraph Integration in n8n:

# In n8n Code node or custom LangGraph service
from langchain_anthropic import ChatAnthropic
from langgraph.graph import StateGraph
from langsmith import traceable

@traceable(name="alert_router")
def route_alert(state: DiagnosticState) -> str:
    """Route alert to appropriate specialist agent."""
    # LangSmith automatically traces this function
    # Governance telemetry emits outcome after completion
    router = ChatAnthropic(model="claude-sonnet-4")
    routing_decision = router.invoke(state)

    # Emit telemetry event
    emit_telemetry({
        "event_type": "routing_decision",
        "decision": routing_decision,
        "confidence": state.confidence_score
    })

    return routing_decision

What LangSmith Captures:

• Agent call sequences and routing decisions
• Token usage per agent (for cost optimization)
• Latency per agent step
• Input/output schemas validation
• Error traces and retry patterns

What Governance Telemetry Captures:

• User accepted/rejected recommendation
• Edit delta size (character count)
• Override frequency
• Session completion status
• Final diagnostic outcome

Why Both?

• LangSmith: Observability during execution (what agents did)
• Telemetry: Observability of outcomes (did it work?)
• Together: Complete feedback loop for continuous improvement

Ollama Deployment

Installation:

# Install Ollama
curl -fsSL https://ollama.com/install.sh | sh

# Pull Llama 3.1 8B Instruct
ollama pull llama3.1:8b-instruct-q8_0

# Verify
ollama run llama3.1:8b-instruct-q8_0 "Classify this edit: {'type': 'formatting'}"

FastAPI Integration:

import requests

def classify_edit(edit_context: dict) -> dict:
    """Call Ollama for classification."""
    prompt = f"""Classify this edit strictly using the schema:
{json.dumps(edit_context)}

Output ONLY valid JSON with these exact enums:
{{"edit_type": "substantive" | "formatting" | "none"}}
"""

    response = requests.post(
        "http://localhost:11434/api/generate",
        json={
            "model": "llama3.1:8b-instruct-q8_0",
            "prompt": prompt,
            "temperature": 0.0,
            "max_tokens": 50,
            "format": "json"  # Enforce JSON output
        }
    )

    return response.json()

Success Metrics

Phase 1 (Telemetry) Complete When:

• 100% of agent calls emit telemetry events
• Event store captures all signals defined in schema
• No data loss or missing events
• Dashboard shows real-time event stream

Phase 2 (Scoring) Complete When:

• Health scores calculated for all active strategies
• Thresholds trigger correct workflow actions
• Scores correlate with manual assessment
• Report generation automated

Phase 3 (Classification) Complete When:

• Ollama classification endpoint working
• 95%+ enum-only outputs (no hallucinated categories)
• Classification latency <500ms per event
• Integrates with telemetry pipeline

Phase 4 (Mutation) Complete When:

• Routing weights auto-adjust based on health scores
• Canary pattern deployed for new strategies
• Deprecation workflow tested end-to-end with human approval
• Audit log captures all config changes

Critical Don'ts

❌ DON'T use governance layer to modify agent prompts ❌ DON'T let classification LLM make "goodness" judgments ❌ DON'T skip human review for deprecation decisions ❌ DON'T use cloud-based LLMs for governance (privacy/cost) ❌ DON'T allow governance to modify training data directly

✅ DO use observable signals only ✅ DO keep classification deterministic (temp=0.0) ✅ DO require human approval for all deprecations ✅ DO use local Ollama for privacy and zero cost ✅ DO modify routing config, not agent logic

---

Related Documentation

• NEXT-STEPS.md - Implementation roadmap and action items
• PHASE_2B_SUMMARY.md - Current SkySpark endpoint status
• references/integration-architecture.md - n8n + pyHVAC design
• ../../engineering_calcs/README.md - Calculation tools project

Saving Next Steps

When n8n-automation work is complete or paused:

node .claude/skills/work-command-center/tools/add-skill-next-steps.js \
  --skill "n8n-automation" \
  --content "## Priority Tasks
1. Build n8n workflow for SkySpark alert triage
2. Test multi-agent system integration
3. Deploy FastAPI tool server"

See: .claude/skills/work-command-center/skill-next-steps-convention.md